At Lookery our Javascript analytics tracker is now pushing more than 250GB of bandwidth per month. This javascript file has grown a bit but is still about 6kb.

We’ve been serving that file from a Nginx webserver running on Amazon EC2 instances. Currently Amazon has 3 EC2 data centers but strangely they are all located on the East Coast of the US (Virginia). Since a lot of our users are international we needed to move that file to a CDN in order to reduce latency.

Less than a month ago we moved that single file off to CacheFly a low-priced CDN. I thought CacheFly would be an interim solution but from our performance testing they seem to be a good longer term option.

For this round of testing I tested the following serving options:

1. CacheFly
2. EdgeCast
3. Amazon S3
4. Nginx running on an Amazon EC2 Instance (DIY option)

For the next round of tests I’ll add Panther Express to the mix.

Performance testing Amazon S3 is a bit unfair but since so many people are using this as a cheap solution I thought I would test it out myself. The performance tests show that you’re much better serving your static content from a lightweight server like NGINX or using an inexpensive option like CacheFly.

For the performance tests I used Pingdom, a 3rd party performance monitoring service that we’ve been quite happy with.

The monitoring servers were geographically distributed as follows:

Summary

CacheFly performed the best but only slightly better than EdgeCast. The S3 option was the worst with the Nginx/DIY option performing just over 100 ms faster.

Details

Below are the details for a single day. I ran these tests for over 2 weeks, the results were identical to this single day.

CacheFly CDN	Nginx on an Amazon EC2 Instance (DIY)
EdgeCast CDN	Amazon S3 used as a CDN

Notes

• I also tested a second DIY option, running a Varnish cache on an Amazon EC2 instance, but for static content Nginx performed much better so I omitted the results.
• EdgeCast has an option that allows frequently used content to be served directly from RAM. My trial account did not allow me to test this option. This option would allow for even better performance, possibly matching/beating CacheFly’s performance.

So far we’re sticking with CacheFly and testing a few other options. I’ll post the Panther Express performance after our tests are complete.

Let me know if you’ve found similar results or if I should be testing any other solutions.

I’m going to be moderating the Web 2.0: Viable Business Model or Bursting Bubble? panel at TiECON East 2008 later this week (May 30th).

The panelists I’ll be interviewing are:

• Fred Wilson of Union Square Ventures
• Don Dodge, Director of Business Development for Microsoft’s Emerging Business Team
• Nabeel Hyatt, Founder of Conduit Labs.
• Brian Balfour, Founder of Viximo

Early this year Fred blogged about Twitter’s Business Model. Given Twitter’s downtime these days seems like a great time to follow-up on that post.

If you have any suggestions for what I should ask any of the panelists please post a comment.

Andy Payne has organized a Virtual/Hosting/EC2 Birds-of-a-feather meet-up for tomorrow morning (May 21st) in Waltham, MA.

We’ll be comparing notes and experiences on virtual hosting options (e.g. EC2). Thanks to Matrix Partners for hosting us at their offices.

I’ll be giving a short presentation on how we’ve been using EC2 at Lookery. I’ll post my presentation here after the meet-up tomorrow.

If you’re interested in attending let me know.