Amazon S3 direct file upload from client browser - private key disclosure

Question

I m implementing a direct file upload from client machine to Amazon S3 via REST API using only JavaScript  without any server-side code  All works fine but one thing is worrying me     When I send a request to Amazon S3 REST API  I need to sign the request and put a signature into Authentication header  To create a signature  I must use my secret key  But all things happens on a client side  so  the secret key can be easily revealed from page source  even if I obfuscate encrypt my sources    How can I handle this  And is it a problem at all  Maybe I can limit specific private key usage only to REST API calls from a specific CORS Origin and to only PUT and POST methods or  maybe link key to only S3 and specific bucket  May be there are another authentication methods    Serverless  solution is ideal  but I can consider involving some serverside processing  excluding uploading a file to my server and then send in to S3

User · Answer

You can do this by AWS S3 Cognito try this link here :

http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/browser-examples.html#Amazon_S3

Also try this code

Just change Region, IdentityPoolId and Your bucket name

_x000D_

<!DOCTYPE html>_x000D_
<html>_x000D_
_x000D_
<head>_x000D_
    <title>AWS S3 File Upload</title>_x000D_
    <script src="https://sdk.amazonaws.com/js/aws-sdk-2.1.12.min.js"></script>_x000D_
</head>_x000D_
_x000D_
<body>_x000D_
    <input type="file" id="file-chooser" />_x000D_
    <button id="upload-button">Upload to S3</button>_x000D_
    <div id="results"></div>_x000D_
    <script type="text/javascript">_x000D_
    AWS.config.region = 'your-region'; // 1. Enter your region_x000D_
_x000D_
    AWS.config.credentials = new AWS.CognitoIdentityCredentials({_x000D_
        IdentityPoolId: 'your-IdentityPoolId' // 2. Enter your identity pool_x000D_
    });_x000D_
_x000D_
    AWS.config.credentials.get(function(err) {_x000D_
        if (err) alert(err);_x000D_
        console.log(AWS.config.credentials);_x000D_
    });_x000D_
_x000D_
    var bucketName = 'your-bucket'; // Enter your bucket name_x000D_
    var bucket = new AWS.S3({_x000D_
        params: {_x000D_
            Bucket: bucketName_x000D_
        }_x000D_
    });_x000D_
_x000D_
    var fileChooser = document.getElementById('file-chooser');_x000D_
    var button = document.getElementById('upload-button');_x000D_
    var results = document.getElementById('results');_x000D_
    button.addEventListener('click', function() {_x000D_
_x000D_
        var file = fileChooser.files[0];_x000D_
_x000D_
        if (file) {_x000D_
_x000D_
            results.innerHTML = '';_x000D_
            var objKey = 'testing/' + file.name;_x000D_
            var params = {_x000D_
                Key: objKey,_x000D_
                ContentType: file.type,_x000D_
                Body: file,_x000D_
                ACL: 'public-read'_x000D_
            };_x000D_
_x000D_
            bucket.putObject(params, function(err, data) {_x000D_
                if (err) {_x000D_
                    results.innerHTML = 'ERROR: ' + err;_x000D_
                } else {_x000D_
                    listObjs();_x000D_
                }_x000D_
            });_x000D_
        } else {_x000D_
            results.innerHTML = 'Nothing to upload.';_x000D_
        }_x000D_
    }, false);_x000D_
    function listObjs() {_x000D_
        var prefix = 'testing';_x000D_
        bucket.listObjects({_x000D_
            Prefix: prefix_x000D_
        }, function(err, data) {_x000D_
            if (err) {_x000D_
                results.innerHTML = 'ERROR: ' + err;_x000D_
            } else {_x000D_
                var objKeys = "";_x000D_
                data.Contents.forEach(function(obj) {_x000D_
                    objKeys += obj.Key + "<br>";_x000D_
                });_x000D_
                results.innerHTML = objKeys;_x000D_
            }_x000D_
        });_x000D_
    }_x000D_
    </script>_x000D_
</body>_x000D_
_x000D_
</html>

_x000D_

For more details, Please check - Github

User · Answer

If you are willing to use a 3rd party service  auth0 com supports this integration  The auth0 service exchanges a 3rd party SSO service authentication for an AWS temporary session token will limited permissions   See  https   github com auth0-samples auth0-s3-sample  and the auth0 documentation

User · Answer

Adding more info to the accepted answer  you can refer to my blog to see a running version of the code  using AWS Signature version 4   Will summarize here   As soon as the user selects a file to be uploaded  do the followings  1  Make a call to the web server to initiate a service to generate required params   In this service  make a call to AWS IAM service to get temporary cred Once you have the cred  create a bucket policy  base 64 encoded string   Then sign the bucket policy with the temporary secret access key to generate final signature send the necessary parameters back to the UI Once this is received  create a html form object  set the required params and POST it    For detailed info  please refer  https   wordpress1763 wordpress com 2016 10 03 browser-based-upload-aws-signature-version-4

User · Answer

I have given a simple code to upload files from Javascript browser to AWS S3 and list the all files in S3 bucket   Steps    To know how to create Create IdentityPoolId http   docs aws amazon com cognito latest developerguide identity-pools html   Goto S3 s console page and open cors configuration from bucket properties  and write following XML code into that    lt  xml version  1 0  encoding  UTF-8   gt   lt CORSConfiguration xmlns  http   s3 amazonaws com doc 2006-03-01   gt    lt CORSRule gt         lt AllowedMethod gt GET lt  AllowedMethod gt     lt AllowedMethod gt PUT lt  AllowedMethod gt     lt AllowedMethod gt DELETE lt  AllowedMethod gt     lt AllowedMethod gt HEAD lt  AllowedMethod gt     lt AllowedHeader gt   lt  AllowedHeader gt    lt  CORSRule gt   lt  CORSConfiguration gt   Create HTML file containing following code change the credentials  open file in browser and enjoy    lt script type  text javascript  gt   AWS config region    ap-north-1      Region  AWS config credentials   new AWS CognitoIdentityCredentials    IdentityPoolId   ap-north-1      -              var bucket   new AWS S3    params     Bucket   MyBucket            var fileChooser   document getElementById  file-chooser     var button   document getElementById  upload-button     var results   document getElementById  results      function upload      var file   fileChooser files 0    console log file name     if  file     results innerHTML        var params      Key  n     pdf    ContentType  file type   Body  file      bucket upload params  function err  data     results innerHTML   err    ERROR      UPLOADED            else    results innerHTML    Nothing to upload             lt  script gt   lt body gt    lt input type  file  id  file-chooser    gt    lt input type  button  onclick  upload    value  Upload to S3  gt    lt div id  results  gt  lt  div gt   lt  body gt

User · Answer

I think what you want is Browser-Based Uploads Using POST   Basically  you do need server-side code  but all it does is generate signed policies  Once the client-side code has the signed policy  it can upload using POST directly to S3 without the data going through your server   Here s the official doc links    Diagram  http   docs aws amazon com AmazonS3 latest dev UsingHTTPPOST html  Example code  http   docs aws amazon com AmazonS3 latest dev HTTPPOSTExamples html  The signed policy would go in your html in a form like this    lt html gt     lt head gt               lt meta http-equiv  Content-Type  content  text html  charset UTF-8    gt             lt  head gt     lt body gt           lt form action  http   johnsmith s3 amazonaws com   method  post  enctype  multipart form-data  gt      Key to upload   lt input type  input  name  key  value  user eric     gt  lt br   gt       lt input type  hidden  name  acl  value  public-read    gt       lt input type  hidden  name  success action redirect  value  http   johnsmith s3 amazonaws com successful upload html    gt      Content-Type   lt input type  input  name  Content-Type  value  image jpeg    gt  lt br   gt       lt input type  hidden  name  x-amz-meta-uuid  value  14365123651274    gt      Tags for File   lt input type  input  name  x-amz-meta-tag  value      gt  lt br   gt       lt input type  hidden  name  AWSAccessKeyId  value  AKIAIOSFODNN7EXAMPLE    gt       lt input type  hidden  name  Policy  value  POLICY    gt       lt input type  hidden  name  Signature  value  SIGNATURE    gt      File   lt input type  file  name  file    gt   lt br   gt       lt  -- The elements after this will be ignored -- gt       lt input type  submit  name  submit  value  Upload to Amazon S3    gt     lt  form gt         lt  html gt    Notice the FORM action is sending the file directly to S3 - not via your server   Every time one of your users wants to upload a file  you would create the POLICY and SIGNATURE on your server  You return the page to the user s browser  The user can then upload a file directly to S3 without going through your server   When you sign the policy  you typically make the policy expire after a few minutes  This forces your users to talk to your server before uploading  This lets you monitor and limit uploads if you desire   The only data going to or from your server is the signed URLs  Your secret keys stay secret on the server

User · Answer

Here is how you generate a policy document using node and serverless   use strict    const uniqid   require  uniqid    const crypto   require  crypto     class Token                    param  Object  config SSM Parameter store JSON config             constructor config                Ensure some required properties are set in the SSM configuration object         this constructor  validateConfig config            this region   config region     AWS region e g  us-west-2         this bucket   config bucket     Bucket name only         this bucketAcl   config bucketAcl     Bucket access policy  private  public-read          this accessKey   config accessKey     Access key         this secretKey   config secretKey     Access key secret             Create a really unique videoKey  with folder prefix         this key   uniqid     uniqid process                The policy requires the date to be this format e g  20181109         const date   new Date   toISOString            this dateString   date substr 0  4    date substr 5  2    date substr 8  2               The number of minutes the policy will need to be used by before it expires         this policyExpireMinutes   15              HMAC encryption algorithm used to encrypt everything in the request         this encryptionAlgorithm    sha256               Client uses encryption algorithm key while making request to S3         this clientEncryptionAlgorithm    AWS4-HMAC-SHA256                         Returns the parameters that FE will use to directly upload to s3                returns  Object              getS3FormParameters             const credentialPath   this  amazonCredentialPath            const policy   this  s3UploadPolicy credentialPath           const policyBase64   new Buffer JSON stringify policy   toString  base64            const signature   this  s3UploadSignature policyBase64            return                key   this key               acl   this bucketAcl               success action status    201                policy   policyBase64               endpoint    https       this bucket     s3-accelerate amazonaws com                x-amz-algorithm   this clientEncryptionAlgorithm               x-amz-credential   credentialPath               x-amz-date   this dateString    T000000Z                x-amz-signature   signature                                 Ensure all required properties are set in SSM Parameter Store Config                param  Object  config         private             static  validateConfig config            if   config hasOwnProperty  bucket                  throw   bucket  is required in SSM Parameter Store Config                     if   config hasOwnProperty  region                  throw   region  is required in SSM Parameter Store Config                     if   config hasOwnProperty  accessKey                  throw   accessKey  is required in SSM Parameter Store Config                     if   config hasOwnProperty  secretKey                  throw   secretKey  is required in SSM Parameter Store Config                                   Create a special string called a credentials path used in constructing an upload policy                returns  String          private              amazonCredentialPath             return this accessKey         this dateString         this region     s3 aws4 request                         Create an upload policy                param  String  credentialPath                returns   expiration  string  conditions                private              s3UploadPolicy credentialPath            return               expiration  this  getPolicyExpirationISODate                conditions                     bucket  this bucket                    key  this key                    acl  this bucketAcl                    success action status   201                      x-amz-algorithm    AWS4-HMAC-SHA256                      x-amz-credential   credentialPath                     x-amz-date   this dateString    T000000Z                                                  ISO formatted date string of when the policy will expire                returns  String          private              getPolicyExpirationISODate             return new Date  new Date  getTime      this policyExpireMinutes   60   1000   toISOString                          HMAC encode a string by a given key                param  String  key         param  String  string                returns  String          private              encryptHmac key  string            const hmac   crypto createHmac              this encryptionAlgorithm  key                    hmac end string            return hmac read                          Create an upload signature from provided params        https   docs aws amazon com AmazonS3 latest API sig-v4-authenticating-requests html signing-request-intro                param policyBase64                returns  String          private              s3UploadSignature policyBase64            const dateKey   this  encryptHmac  AWS4    this secretKey  this dateString           const dateRegionKey   this  encryptHmac dateKey  this region           const dateRegionServiceKey   this  encryptHmac dateRegionKey   s3            const signingKey   this  encryptHmac dateRegionServiceKey   aws4 request             return this  encryptHmac signingKey  policyBase64  toString  hex             module exports   Token    The configuration object used is stored in SSM Parameter Store and looks like this         bucket    my-bucket-name        region    us-west-2        bucketAcl    private        accessKey    MY ACCESS KEY        secretKey    MY SECRET ACCESS KEY

User · Answer

If you don t have any server side code  you security depends on the security of the access to your JavaScript code on the client side  ie everybody who has the code could upload something    So I would recommend  to simply create a special S3 bucket which is public writeable  but not readable   so you don t need any signed components on the client side   The bucket name  a GUID eg  will be your only defense against malicious uploads  but a potential attacker could not use your bucket to transfer data  because it is write only to him

User · Answer

To create a signature  I must use my secret key  But all things   happens on a client side  so  the secret key can be easily revealed   from page source  even if I obfuscate encrypt my sources     This is where you have misunderstood  The very reason digital signatures are used is so that you can verify something as correct without revealing your secret key  In this case the digital signature is used to prevent the user from modifying the policy you set for the form post   Digital signatures such as the one here are used for security all around the web  If someone  NSA   really were able to break them  they would have much bigger targets than your S3 bucket

User · Answer

You re saying you want a  serverless  solution  But that means you have no ability to put any  of  your  code in the loop   NOTE  Once you give your code to a client  it s  their  code now    Locking down CORS is not going to help  People can easily write a non-web-based tool  or a web-based proxy  that adds the correct CORS header to abuse your system   The big problem is that you can t differentiate between the different users  You can t allow one user to list access his files  but prevent others from doing so   If you detect abuse  there is nothing you can do about it except change the key   Which the attacker can presumably just get again    Your best bet is to create an  IAM user  with a key for your javascript client  Only give it write access to just one bucket   but ideally  do not enable the ListBucket operation  that will make it more attractive to attackers    If you had a server  even a simple micro instance at  20 month   you could sign the keys on your server while monitoring preventing abuse in realtime   Without a server  the best you can do is periodically monitor for abuse after-the-fact   Here s what I would do   1  periodically rotate the keys for that IAM user  Every night  generate a new key for that IAM user  and replace the oldest key  Since there are 2 keys  each key will be valid for 2 days   2  enable S3 logging  and download the logs every hour  Set alerts on  too many uploads  and  too many downloads   You will want to check both total file size and number of files uploaded  And you will want to monitor both the global totals  and also the per-IP address totals  with a lower threshold    These checks can be done  serverless  because you can run them on your desktop   i e  S3 does all the work  these processes just there to alert you to abuse of your S3 bucket so you don t get a giant AWS bill at the end of the month

[javascript] Amazon S3 direct file upload from client browser - private key disclosure

Examples related to javascript

Examples related to amazon-web-services

Examples related to authentication

Examples related to amazon-s3