How to directly upload files to Amazon S3 from your client side web app

Why you need this?

You don’t want your heavy-weight data to travel 2 legs from Client to Server to S3, incurring the cost of IO and clogging the pipe 2 times.

Instead, you want to ask your server to give your client one-time permission to upload your data directly to S3. The process is still 2 legged, but heawy-weight data travels only on 1 leg.

On Amazon S3 this is implemented with CORS (Cross Origin Resource Sharing)

We use this at Dubjoy, where customers upload their huge video files to S3 for translation and voice-over, and we don’t want our Heroku server to have anything to do with heavy-weight video files.

Steps to implement this

Set up Amazon S3 bucket CORS configuration
Implement client-side JavaScript (CoffeScript, JavaScript)
Implement server-side upload request signing (Ruby/Sinatra, trivial to do in any other language)

1. Amazon S3 bucket CORS configuration

Set this in AWS S3 management console. Right-click on the desired bucket and select Properties. Below, on the permissions tab, click Edit CORS configuration, paste the XML below and click Save.

<?xml version="1.0" encoding="UTF-8"?><corsconfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><corsrule><allowedorigin>*</allowedorigin><allowedmethod>GET</allowedmethod><maxageseconds>3000</maxageseconds><allowedheader>Authorization</allowedheader></corsrule><corsrule><allowedorigin>*</allowedorigin><allowedmethod>PUT</allowedmethod><maxageseconds>3000</maxageseconds><allowedheader>Content-Type</allowedheader><allowedheader>x-amz-acl</allowedheader><allowedheader>origin</allowedheader></corsrule></corsconfiguration>

2. Your web app

Head to GitHub repo with CoffeScript and JavaScript Class files to include. In your app, do the following:

HAML

%input#file{ :type =&gt; 'file', :name =&gt; 'files[]'}

or HTML for the chevron-lovers

<input type="file" name="files[]">

CoffeScript

s3upload = s3upload ? new S3Upload
    file_dom_selector: '#files'
  s3_sign_put_url: '/signS3put'
    onProgress: (percent, message) -&gt;
        console.log 'Upload progress: ', percent, message # Use this for live upload progress bars
    onFinishS3Put: (public_url) -&gt;
        console.log 'Upload finished: ', public_url # Get the URL of the uploaded file
  onError: (status) -&gt;
    console.log 'Upload error: ', status

or JavaScript for the brace-lovers

var s3upload = s3upload != null ? s3upload : new S3Upload({
  file_dom_selector: '#files',
  s3_sign_put_url: '/signS3put',
  onProgress: function(percent, message) { // Use this for live upload progress bars
    console.log('Upload progress: ', percent, message);
  },
  onFinishS3Put: function(public_url) { // Get the URL of the uploaded file
    console.log('Upload finished: ', public_url);
  },
  onError: function(status) {
    console.log('Upload error: ', status);
  }
});

Be sure to set the right DOM selector name file_dom_selector for file input tag, #files in our case. s3_sign_put_url is an end-point on your server where you will be signing S3 PUT requests.

3. Server-side request signing

Be sure to set S3_BUCKET_NAME,S3_SECRET_KEY,S3_ACCESS_KEY. Create a bucket and get the keys under Security Credentials menu in AWS management console.

S3_BUCKET_NAME = 'CREATE_A_BUCKET_AND_SET_THE_NAME_HERE'
S3_SECRET_KEY = 'GET_THIS_IN_AWS_CONSOLE'
S3_ACCESS_KEY = 'GET_THIS_IN_AWS_CONSOLE'

get '/signS3put' do
  objectName = params[:s3_object_name]
  mimeType = params['s3_object_type']
  expires = Time.now.to_i + 100 # PUT request to S3 must start within 100 seconds

  amzHeaders = "x-amz-acl:public-read" # set the public read permission on the uploaded file
  stringToSign = "PUT\n\n#{mimeType}\n#{expires}\n#{amzHeaders}\n/#{S3_BUCKET_NAME}/#{objectName}";
  sig = CGI::escape(Base64.strict_encode64(OpenSSL::HMAC.digest('sha1', S3_SECRET_KEY, stringToSign)))

  {
    signed_request: CGI::escape("#{S3_URL}#{S3_BUCKET_NAME}/#{objectName}?AWSAccessKeyId=#{S3_ACCESS_KEY}&amp;Expires=#{expires}&amp;Signature=#{sig}"),
    url: "http://s3.amazonaws.com/#{S3_BUCKET_NAME}/#{objectName}"
  }.to_json
end

Resources

The code is a based on these resources, but has been put in to an easy to use CoffeeScript/JavaScript Class

You can learn more about CORS here

Rok Krulec / @tantadruj

Tags: cors amazon s3 coffeescript javascript dubjoy uplaod