Once your app becomes popular enough, you may want to consider moving your assets from internal servers to the cloud. Amazon Web Services (AWS) is one of the most popular options, but a lot of owners fail to utilize its full capacity. Here's how to optimize your Amazon servers and get the most of it.
Storing assets (such as stylesheets, scripts and images) locally is fine for developing web applications that are just starting to get off the ground. But once your app becomes popular enough, you may want to consider moving your assets to the cloud. Amazon Web Services (AWS) is one of the most popular options, but a lot of web app owners don’t use AWS properly. Or rather - they use AWS’s services sub-optimally. Here’s what you need to know.
Serving assets from S3 is a no-no
In most cases, S3 buckets - originally designed and intended for storing assets - are used for both storing and serving everything. Even by big players. It’s neither the most efficient nor the correct way of doing things. It might seem more convenient - storing and serving assets from the same location means fewer configuration headaches - but let’s consider how it works.
You set up an S3 bucket and choose a location for it (the default option is somewhere in the United States). From now on, if you upload your assets there, when someone from Europe visits your site, they will have to wait for the files to reach them from the US. Every single time.
If you set up your bucket in Europe, people outside Europe will have the same problem and will end up waiting longer for images and other assets to load. What’s more, when you look at S3’s pricing, you’ll see that uploading TO buckets is very cheap (or even free) but the traffic OUT of the S3 buckets to the Internet has a heftier price tag. In terms of performance and cost-effectiveness, you can do a lot better.
CloudFront is your friend
This is where CloudFront comes in. CloudFront is a content distribution network (CDN) intended for the sole purpose of serving assets. It was designed to work optimally with S3 buckets and you should definitely use it if you’re already storing your assets on S3. Data transfer from S3 to CloudFront is free. In addition, the cost of serving assets from CloudFront to the Internet is much lower compared to S3. And the best part is that CloudFront is a lot faster for the end users.
The big picture
Here’s how it works: you set up the CloudFront service to serve assets on your website and connect it to your S3 bucket. Let’s say that your S3 bucket is located in the US. When the first user from Europe visits your website, CloudFront checks if the assets are already available from Amazon’s server located in Europe. These are called edge servers or edge locations and there are many such servers distributed around the globe. If the assets are not yet cached in Europe, CloudFront will fetch the assets from the original S3 bucket it’s connected to.
The waiting time in this single instance will be the same as it would be with assets served through S3. But when this same user (or some other user from Europe) visits your website again, the assets are served with lightning speed from a cache in the nearby edge server. The difference can be significant. To reiterate, by using S3 in conjunction with CloudFront:
- you’re using Amazon Web Services the way they were intended,
- you pay less for data transfers,
- you get much better performance.
Here’s a real-life example with benchmarks and numbers that should convince you about the “better performance” part.
S3 vs CloudFront - testing app
We’ve created a very simple app that has two functions - it uploads pictures to the S3 bucket and displays them all on the main page. We also created two configurations for it. In the first, all pictures are served through the original S3 bucket they were uploaded to. In the second, we serve pictures through CloudFront.
We used the following benchmarking script to compare both configurations by downloading the same image from the S3 bucket and CloudFront the same number of times to get a reliable mean value:
Below are the results from 100 requests:
and 1000 requests:
What does this benchmark tell us? In short, it tells us that CloudFront transfers are at least twice as fast as S3 bucket transfers when serving assets such as images. This is to be expected - our S3 bucket used in this benchmark was the default bucket located in the US and our nearest CloudFront edge location was located in Warsaw, Poland. Amazon has quite an extensive network of edge servers.
Now let’s look at the costs. We’ll use 50TB as the baseline for comparing transfer costs, using current AWS pricing.
Pure S3 option
- Transfer TO S3 bucket: $0.00
- First 10TB transfer TO the Internet: $900.00
- Next 40TB TO the Internet: $3400.00
- Total: $4300.00 per month
S3 + CloudFront option
- Transfer TO S3 bucket: $0.00
- Transfer FROM S3 TO Cloudfront: $0.00
- First 10TB TO the Internet: $850.00
- Next 40TB TO the Internet: $3200.00
- Total: $4050.00 per month
The difference is not huge, but not insignificant. The S3 + CloudFront option is calculated for US and Europe only (we assumed that most of your traffic will come from these regions, which may not necessarily be true). Traffic from other, more costly, regions may increase the final total cost, so these slight differences in costs between the two options might not matter in the long run. Still, by using S3 + CloudFront you can double your transfer speed without impacting your wallet. If most of your traffic comes from more costly regions, you may have to consider whether transfer speed or cost effectiveness are more important to you.
What are your experiences so far? Have you found out about any other tips worth mentioning? Feel free to share your thoughts in the comments, we'll be happy to hear from you!