Distributing content world wide using Amazon CloudFront

A few days ago, I announced that many of the samples in Tour de Flex are now hosted in 16 distributed locations.  As promised, here are the details:

Prior to last Friday, most of the Tour de Flex samples were hosted from a dedicated server in California.  For those of us in the US, the load time for these samples was good, but for users in Europe and Asia, there were noticeably slower download times.   Lately, I’ve been playing around with Amazon EC2 and other Amazon web services and discovered a fairly new service called Amazon CloudFront, a web service for distributed content delivery.   In less than an hour, I had the Tour de Flex samples hosted from 16 edge servers across the globe:

All of the samples are now hosted on samples.tourdeflex.com, which is dynamically redirected by Amazon to the nearest edge server to the user.  For example, when I access samples.tourdeflex.com from my home in Tampa, I get directed to the Miami edge server.  Here’s my traceroute:

/Users/admin $ traceroute samples.tourdeflex.com
traceroute: Warning: samples.tourdeflex.com has multiple addresses; using 216.137.47.148
traceroute to d1icesobeldsjy.cloudfront.net (216.137.47.148), 64 hops max, 52 byte packets
 1  192.168.1.1 (192.168.1.1)  0.875 ms  0.623 ms  0.573 ms
 2  l5003.tampfl-vfttp-02.verizon-gni.net (71.251.98.1)  3.846 ms  4.139 ms  5.080 ms
 3  p15-1.tampfl-lcr-04.verizon-gni.net (130.81.59.42)  5.036 ms  3.919 ms  5.892 ms
 4  so-6-0-0-0.tpa01-bb-rtr2.verizon-gni.net (130.81.29.53)  7.424 ms  6.787 ms  8.721 ms
 5  ge-1-0-0-0.atl01-bb-rtr2.verizon-gni.net (130.81.17.48)  28.346 ms  29.175 ms  27.396 ms
 6  0.xe-8-1-0.br3.atl4.alter.net (152.63.80.209)  24.960 ms  24.046 ms  24.922 ms
 7  xe-11-3-0.edge4.atlanta2.level3.net (4.68.62.21)  27.255 ms  26.675 ms  28.086 ms
 8  * ae-62-51.ebr2.atlanta2.level3.net (4.68.103.29)  34.805 ms
    ae-72-52.ebr2.atlanta2.level3.net (4.68.103.61)  26.122 ms
 9  ae-2-2.ebr2.miami1.level3.net (4.69.140.141)  44.494 ms  47.026 ms  37.952 ms
10  ae-2-52.edge1.miami2.level3.net (4.69.138.107)  50.364 ms  41.447 ms  42.235 ms
11  amazoncom.edge1.miami2.level3.net (4.59.80.38)  42.755 ms  42.136 ms  42.491 ms
12  server-216-137-47-148.mia3.cloudfront.net (216.137.47.148)  42.543 ms  42.823 ms  41.503 ms

I’m hoping that Amazon adds a few edge servers in South America, South Africa and Australia soon.

Getting this working was amazingly simple:

• I created an Amazon S3 bucket.  Amazon S3 is a web service for storing and retrieving files that has been around for awhile.
• I copied all of the Tour de Flex samples from our dedicated server to the S3 bucket and set global read permissions.
• I created an Amazon CloudFront distribution on top of the S3 bucket, which provided me with a public URL
• I created a CNAME record for samples.tourdeflex.com that points to the public CloudFront URL
• I then did a batch change to the Tour de Flex XML that drives the samples and converted the old host to the new one.  For example, http://tourdeflex.adobe.com/flex4.0/Sample-Adobe-AdvancedDataGrid/sample1.html is now http://samples.tourdeflex.com/flex4.0/Sample-Adobe-AdvancedDataGrid/sample1.html

The cost:

The pricing is simple.  Depending on the edge server location, Amazon charges between 15 and 20 cents per GB of data transferred and about a penny for every 10,000 file transfers.  Below is the bill for nearly 5 days of Tour de Flex sample serving to give you an idea of how things work:

As you can see, it’s not bad!  It’s costing us less than $1/day!  For more details on pricing, go to http://aws.amazon.com/cloudfront/pricing/

Tooling:

For my project, I needed a way to easily transfer thousands of files to my S3 bucket.  I also needed a way to easily flag a file for redistribution to the edge servers, which is required when we update a sample.  Amazon calls this process “file invalidation”.  It basically forces the file to be redistributed to the edge servers.  It’s like clearing the cache for a specific file or group of files.  The Amazon web interface does not provide the ability to invalidate a file yet, so I had to find a tool.  For Mac OS X, Windows and Linux, there is Bucket Explorer (http://www.bucketexplorer.com/) that provides everything needed.  Another good Windows option is CloudBerry Explorer (http://cloudberrylab.com/).  CyberDuck (http://cyberduck.ch/ – open source) for Mac OS X comes close but is missing file invalidation.

There is a great Linux command line tool for syncing folders with S3 called S3 Tools (http://s3tools.org/s3cmd) but it doesn’t yet support CloudFront file invalidation.

Flex/Flash, AIR and Amazon CloudFront

This is a great service for Flex/AIR developers because it provides a way to distribute our application and assets more efficiently.  I have also used CloudFront  for my side project, ChessJam.   I use it for the 7MB AIR file downloads and for the web version of ChessJam.  For the web version, I moved the SWF and all needed assets (images, audio files, etc.) to my CloudFront server.  Several users in Asia and Europe noticed an immediate improvement in load times.  For about 90 cents per day, I get happier users!  Nobody likes waiting on preloaders and lengthy downloads.

~ by Greg on October 5, 2010.