Home
Blog
Boosting File Uploads With Amazon S3 Transfer Acceleration

Boosting File Uploads With Amazon S3 Transfer Acceleration

・4 min read
Boosting File Uploads With Amazon S3 Transfer Acceleration

This is a case study of how we managed to boost file uploads with Amazon S3 Transfer Acceleration on one of our projects.

In response to the challenges faced by users in India who experienced slow transfer speeds to the centralized S3 bucket, we implemented a solution: Amazon S3 Transfer Acceleration (S3TA). This technology accelerates data uploads by routing data to the nearest AWS edge location, utilising AWS's global backbone and network protocol optimizations. S3 Transfer Acceleration significantly reduces the variability in internet speeds, routing, and congestion, often speeding up file transfers by 50-500%. To integrate this solution, the S3 was configured for Transfer Acceleration using Terraform, and upload links were switched from region-based URLs to accelerated ones. This resulted in a notable improvement in upload speeds for users across different geographical locations, enhancing the overall user experience.

What you will learn from this article:

  • How does Amazon S3 Transfer Acceleration (S3TA) improve upload data transfer speed for global users?
  • In what ways does S3 Transfer Acceleration manage to reduce the variability in internet routing, transfer speeds, and congestion?
  • How does S3 Transfer Acceleration benefit applications with a high volume of user-generated content?
  • What steps does we take to configure S3 to work with Transfer Acceleration?
  • How do we utilize Terraform to enable transfer acceleration status on S3 resources?
  • What changes were required to switch from a regional S3 bucket to using s3-accelerate.amazonaws.com for uploads?

    Back story

First of all, let me start by introducing the problem we had. There’s an application where we deploy PWAs for our clients from around the world. Each PWA has the ability to take a picture and sent it to Amazon S3 for further processing. Of course, while the image processes user has to wait for the response.

However, because we use centralized S3 that is located in Europe (Ireland) our India-based users encountered some problems. They had to wait much longer for the transfer data to finish and this was our bottleneck. After researching potential solutions we decided to use Amazon S3 Transfer Acceleration.

What is Amazon S3 Transfer Acceleration?

S3TA allows its users to accelerate data or content uploads from all over the world to a centralized S3 bucket.

It accelerates transfers by automatically routing data to the closest AWS edge location by passing that data over AWS global backbone and by using Network Protocol Optimizations.

S3 Transfer Acceleration enables fast, easy and secure transfers of files over long distances between your client and S3 bucket. It can speed up content transfers by as much as 50-500%. The solution reduces the variability in Internet routing, speeds and congestion that affect the transfers.

Common use cases

S3 Transfer Acceleration can be used in various cases, but the most common use cases are:

  • customers with web and mobile applications that have an international user base to speed up content upload or downloads
  • applications that have a lot of user-generated content that create PUTs to the S3 bucket
  • customers with globally distributed offices and IT infrastructure that work with files, content and images locally to have an S3 integrated application push that data to a centralized S3 bucket

Implementing the solution

Let’s move on to the process of implementing the solution. First of all, in order to introduce the changes to the application, we have to configure S3 to work with Transfer Acceleration. We can then replace all generated upload links from a region-based URL to an accelerated one, and it’s extremely easy to do so.

  • we need to enable the acceleration status on S3; we use Terraform to manage our infrastructure so all we had to do was enable acceleration_status for our S3 resource: https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#acceleration_status
  • before we upload the file we ask our server for a presigned upload URL so the PWA can upload the file. The generated URL contains the S3 region we operate in, by default: s3.eu-west-1.amazonaws.com/....

    To actually use Transfer Acceleration we have to switch from a regional bucket to s3-accelerate.amazonaws.com

For a more detailed process look into the documentation: https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html

Conclusions

The entire process of migration to the production environment took place the day of implementation. It was fast, easy and generally pain-free. All the changes were transparent to the end-users.

Thanks to S3 Transfer Acceleration we were able to decrease the full upload speed and time significantly, so with little to no effort, we achieved great success in transferring faster data transfer well.

If you too have a problem with slow uploads or need help with AWS, feel free to contact us!


Rate this article:

5,0

based on 0 votes
Our services
See what we can create for You
Our services

Awards & Certificates

reviewed on
30 reviews
  • Top 1000 Companies Global 2021
  • Top Development Company Poland 2021
HR dream team
  • 2020 HR Dream Team Award
  • 2016 Employer Branding Featured
  • 2015 HR Dream Team Award
ISO CertificateISO Certificate
  • Information Security Management System compliant with PN-EN ISO/IEC 27001
  • Business Continuity Management compliant with ISO 22301

© 2024 Selleo. Privacy Policy & Cookies