[Imap-protocol] SELECT/EXAMINE clarification of UNSEEN

Brandon Long blong at google.com
Wed Nov 16 10:58:49 PST 2011

On Wed, Nov 16, 2011 at 12:33 AM, Dave Cridland <dave at cridland.net> wrote:

> On Wed Nov 16 02:24:17 2011, Mark Crispin wrote:


>> My understanding is that Google perceived keywords as not commonly

>> implemented, and thus needing to do view/mailbox mapping. It's been a

>> major problem; a classic case of kludge tower building because it was

>> perceived that they couldn't do it right to begin with.


> I thought this at one point, but I recall seeing somewhere that the

> limitation was that keywords are pure ASCII, compared to mailbox names (and

> Google tags or whatever they're called) which are full unicode, contain

> spaces, etc.

It was never really considered a viable option, because it wasn't
technically possible with the standard, and because almost no client
we looked at had any support for keywords (the most we saw was
Thunderbird which uses like 6 keywords that mean various things, but
doesn't even store them in IMAP as anything more than
keyword1...keyword6, others only used it for things like spam).

I'm sure there are some clients with better support for keywords, but
they weren't common enough to make a difference in our decision.

> Personally I think Google are big enough that fixing this problem "properly"

> (ie, a more free-text naming of FLAGS via the standards process) would have

> been viable, but the desire of many companies (Google included) to assume

> that marketing splash always trumps technical merit means that a kludge they

> can release without consultation trumps revealing some portion of the plan,

> or releasing incrementally, but with a better design.


> I do understand the pressure, and sympathize with it, but I still find it

> annoying how little the technical argument counts for, especially when it's

> a standardization issue, and these companies are often too happy to talk

> about how much they support open standards.

There are two separate things here: supporting the current standard
and working to extend it.

Our goal was supporting the current standard. Our primary motivation
was access from mobile devices, where the IMAP clients are probably
never going to receive an update. Designing our system to require
extensive client changes to support us, even if those changes were
"standard", was not a viable option.

We would have essentially had to have a client which:
1) Only had a single folder (INBOX) which corresponds to ... every
message in the system, spam included.
2) Export all of our labels as IMAP keywords hacked into each being an
atom, or alternatively, each being a UTF8 astring perhaps with some
ENABLE called
3) Reject any attempt to create/rename/delete folders

In that scenario, IMAP would have looked worse than POP to most
clients, because all of the users spam would be visible with no
indication that it was spam. Eventually, some clients would have
started to support this, and they probably would have shoe-horned
keywords into working similar to folders, giving users an experience
that's nearly identical to what they get now anyways.

We could have made spam special and put it in a separate folder, but
then you start going down the path to what we actually did.

Also, there were specific popular clients which would fail outright or
eventually badly if they were unable to create folders like their
draft folder.

We could have also always returned labels as keywords on top of what
we did, but that would have increased the amount of data to sync a
folder in many clients by up to 2x, and the amount of data to do a
folder sync on a 100k message folder is already on the order of 6MB of
data using the UID FLAGS method. Doubling that to 12MB for data for
99% of clients which don't care about it, not pretty.

Our goal was interoperability, which meant making Gmail look like IMAP
as used by current clients. At the end of the day, the point of our
service is to let people read their mail using the tools available to

 Brandon Long <blong at google.com>
 Staff Engineer
 Gmail Delivery TLM

More information about the Imap-protocol mailing list