I presume all the CDNs support expiry headers when they are correctly set at the source and you are talking about setting expiry information on files actually hosted by the CDN or adding headers that were missing on the source.

I would expect a caching solution like cacheFly to use headers set at the source location not be adding it's own headers or modifying headers – strictly speaking that breaks the HTTP rules be ignoring the headers from the source server and adding it's own headers.

S3 has support for setting headers, because it's not a cache – it is the actual source of the files.

Where the CDNs support hosting files as well as caching, the issue get's more blurred – one would want to set headers where the CDN is acting as the source, but not if it is just caching.

I've been asking CacheFly to add this “simple” feature (being able to upload gzipped content and have the Content-Encoding header set correctly when it gets downloaded) for years, but they're just uninterested I guess.

That's a very good approach (to rewrite the URLs based on the user agent) to workaround the S3 limitations, unfortunately this won't make S3 run any faster. I just can't believe that I can't get more than 10-12Mbit/sec of sustained download speed. Will wait few more months before to consider this seriously — hopefully they will resolve these issues by then.

Regards,

Lenkov
SiteKreator.com

Here's a quick blog post I wrote detailing how to serve gzipped content from S3 only to users whose browser supports it. http://clickontyler.com/blog/2008/05/using-amaz...

Another issue with S3 you should be aware of is that you can't really host gzipped content on it. You can upload a gzipped files, but the S3 will not serve the non-gzipped version to browsers who does not support gzipped content properly (like IE 5.5,some mobile browsers, etc).

I was trying to use S3 as edge node for our inhouse CDN, but that was the dealbreaker for me (we just have too much JS that has to be gzipped for most of the users, but not for all). The second dealbreakers was that I couldn't get any decent download speed out of it (max download speed was ~12Mbit/sec). And it was very unstable (which usually means that their pipes are very much saturated).

For me right now the cheapest and most reliable option remains to push the “hot” content to a network of dedicated servers, located at different ISPs. I pay ~$100 per such server, but I get a real 100Mbit pipe and a few Tb of transfer (the whole thing is around 3-4 times cheaper than S3 and much most faster).

Cheers,
Lenkov
SiteKreator.com

As a side note, I wrote up an FTP-style method of setting far futures expires headers in s3 last year : Setting Far Future Expires Headers For Images In Amazon S3