I'm implementing a direct file upload from client machine to Amazon S3 via REST API using only JavaScript, without any server-side code. All works fine but one thing is worrying me...
When I send a request to Amazon S3 REST API, I need to sign the request and put a signature into Authentication header. To create a signature, I must use my secret key. But all things happens on a client side, so, the secret key can be easily revealed from page source (even if I obfuscate/encrypt my sources).
How can I handle this? And is it a problem at all? Maybe I can limit specific private key usage only to REST API calls from a specific CORS Origin and to only PUT and POST methods or maybe link key to only S3 and specific bucket? May be there are another authentication methods?
"Serverless" solution is ideal, but I can consider involving some serverside processing, excluding uploading a file to my server and then send in to S3.
This question is related to
javascript
amazon-web-services
authentication
amazon-s3
You're saying you want a "serverless" solution. But that means you have no ability to put any of "your" code in the loop. (NOTE: Once you give your code to a client, it's "their" code now.) Locking down CORS is not going to help: People can easily write a non-web-based tool (or a web-based proxy) that adds the correct CORS header to abuse your system.
The big problem is that you can't differentiate between the different users. You can't allow one user to list/access his files, but prevent others from doing so. If you detect abuse, there is nothing you can do about it except change the key. (Which the attacker can presumably just get again.)
Your best bet is to create an "IAM user" with a key for your javascript client. Only give it write access to just one bucket. (but ideally, do not enable the ListBucket operation, that will make it more attractive to attackers.)
If you had a server (even a simple micro instance at $20/month), you could sign the keys on your server while monitoring/preventing abuse in realtime. Without a server, the best you can do is periodically monitor for abuse after-the-fact. Here's what I would do:
1) periodically rotate the keys for that IAM user: Every night, generate a new key for that IAM user, and replace the oldest key. Since there are 2 keys, each key will be valid for 2 days.
2) enable S3 logging, and download the logs every hour. Set alerts on "too many uploads" and "too many downloads". You will want to check both total file size and number of files uploaded. And you will want to monitor both the global totals, and also the per-IP address totals (with a lower threshold).
These checks can be done "serverless" because you can run them on your desktop. (i.e. S3 does all the work, these processes just there to alert you to abuse of your S3 bucket so you don't get a giant AWS bill at the end of the month.)
You can do this by AWS S3 Cognito try this link here :
http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/browser-examples.html#Amazon_S3
Also try this code
Just change Region, IdentityPoolId and Your bucket name
<!DOCTYPE html>_x000D_
<html>_x000D_
_x000D_
<head>_x000D_
<title>AWS S3 File Upload</title>_x000D_
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.1.12.min.js"></script>_x000D_
</head>_x000D_
_x000D_
<body>_x000D_
<input type="file" id="file-chooser" />_x000D_
<button id="upload-button">Upload to S3</button>_x000D_
<div id="results"></div>_x000D_
<script type="text/javascript">_x000D_
AWS.config.region = 'your-region'; // 1. Enter your region_x000D_
_x000D_
AWS.config.credentials = new AWS.CognitoIdentityCredentials({_x000D_
IdentityPoolId: 'your-IdentityPoolId' // 2. Enter your identity pool_x000D_
});_x000D_
_x000D_
AWS.config.credentials.get(function(err) {_x000D_
if (err) alert(err);_x000D_
console.log(AWS.config.credentials);_x000D_
});_x000D_
_x000D_
var bucketName = 'your-bucket'; // Enter your bucket name_x000D_
var bucket = new AWS.S3({_x000D_
params: {_x000D_
Bucket: bucketName_x000D_
}_x000D_
});_x000D_
_x000D_
var fileChooser = document.getElementById('file-chooser');_x000D_
var button = document.getElementById('upload-button');_x000D_
var results = document.getElementById('results');_x000D_
button.addEventListener('click', function() {_x000D_
_x000D_
var file = fileChooser.files[0];_x000D_
_x000D_
if (file) {_x000D_
_x000D_
results.innerHTML = '';_x000D_
var objKey = 'testing/' + file.name;_x000D_
var params = {_x000D_
Key: objKey,_x000D_
ContentType: file.type,_x000D_
Body: file,_x000D_
ACL: 'public-read'_x000D_
};_x000D_
_x000D_
bucket.putObject(params, function(err, data) {_x000D_
if (err) {_x000D_
results.innerHTML = 'ERROR: ' + err;_x000D_
} else {_x000D_
listObjs();_x000D_
}_x000D_
});_x000D_
} else {_x000D_
results.innerHTML = 'Nothing to upload.';_x000D_
}_x000D_
}, false);_x000D_
function listObjs() {_x000D_
var prefix = 'testing';_x000D_
bucket.listObjects({_x000D_
Prefix: prefix_x000D_
}, function(err, data) {_x000D_
if (err) {_x000D_
results.innerHTML = 'ERROR: ' + err;_x000D_
} else {_x000D_
var objKeys = "";_x000D_
data.Contents.forEach(function(obj) {_x000D_
objKeys += obj.Key + "<br>";_x000D_
});_x000D_
results.innerHTML = objKeys;_x000D_
}_x000D_
});_x000D_
}_x000D_
</script>_x000D_
</body>_x000D_
_x000D_
</html>Here is how you generate a policy document using node and serverless
"use strict";
const uniqid = require('uniqid');
const crypto = require('crypto');
class Token {
/**
* @param {Object} config SSM Parameter store JSON config
*/
constructor(config) {
// Ensure some required properties are set in the SSM configuration object
this.constructor._validateConfig(config);
this.region = config.region; // AWS region e.g. us-west-2
this.bucket = config.bucket; // Bucket name only
this.bucketAcl = config.bucketAcl; // Bucket access policy [private, public-read]
this.accessKey = config.accessKey; // Access key
this.secretKey = config.secretKey; // Access key secret
// Create a really unique videoKey, with folder prefix
this.key = uniqid() + uniqid.process();
// The policy requires the date to be this format e.g. 20181109
const date = new Date().toISOString();
this.dateString = date.substr(0, 4) + date.substr(5, 2) + date.substr(8, 2);
// The number of minutes the policy will need to be used by before it expires
this.policyExpireMinutes = 15;
// HMAC encryption algorithm used to encrypt everything in the request
this.encryptionAlgorithm = 'sha256';
// Client uses encryption algorithm key while making request to S3
this.clientEncryptionAlgorithm = 'AWS4-HMAC-SHA256';
}
/**
* Returns the parameters that FE will use to directly upload to s3
*
* @returns {Object}
*/
getS3FormParameters() {
const credentialPath = this._amazonCredentialPath();
const policy = this._s3UploadPolicy(credentialPath);
const policyBase64 = new Buffer(JSON.stringify(policy)).toString('base64');
const signature = this._s3UploadSignature(policyBase64);
return {
'key': this.key,
'acl': this.bucketAcl,
'success_action_status': '201',
'policy': policyBase64,
'endpoint': "https://" + this.bucket + ".s3-accelerate.amazonaws.com",
'x-amz-algorithm': this.clientEncryptionAlgorithm,
'x-amz-credential': credentialPath,
'x-amz-date': this.dateString + 'T000000Z',
'x-amz-signature': signature
}
}
/**
* Ensure all required properties are set in SSM Parameter Store Config
*
* @param {Object} config
* @private
*/
static _validateConfig(config) {
if (!config.hasOwnProperty('bucket')) {
throw "'bucket' is required in SSM Parameter Store Config";
}
if (!config.hasOwnProperty('region')) {
throw "'region' is required in SSM Parameter Store Config";
}
if (!config.hasOwnProperty('accessKey')) {
throw "'accessKey' is required in SSM Parameter Store Config";
}
if (!config.hasOwnProperty('secretKey')) {
throw "'secretKey' is required in SSM Parameter Store Config";
}
}
/**
* Create a special string called a credentials path used in constructing an upload policy
*
* @returns {String}
* @private
*/
_amazonCredentialPath() {
return this.accessKey + '/' + this.dateString + '/' + this.region + '/s3/aws4_request';
}
/**
* Create an upload policy
*
* @param {String} credentialPath
*
* @returns {{expiration: string, conditions: *[]}}
* @private
*/
_s3UploadPolicy(credentialPath) {
return {
expiration: this._getPolicyExpirationISODate(),
conditions: [
{bucket: this.bucket},
{key: this.key},
{acl: this.bucketAcl},
{success_action_status: "201"},
{'x-amz-algorithm': 'AWS4-HMAC-SHA256'},
{'x-amz-credential': credentialPath},
{'x-amz-date': this.dateString + 'T000000Z'}
],
}
}
/**
* ISO formatted date string of when the policy will expire
*
* @returns {String}
* @private
*/
_getPolicyExpirationISODate() {
return new Date((new Date).getTime() + (this.policyExpireMinutes * 60 * 1000)).toISOString();
}
/**
* HMAC encode a string by a given key
*
* @param {String} key
* @param {String} string
*
* @returns {String}
* @private
*/
_encryptHmac(key, string) {
const hmac = crypto.createHmac(
this.encryptionAlgorithm, key
);
hmac.end(string);
return hmac.read();
}
/**
* Create an upload signature from provided params
* https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html#signing-request-intro
*
* @param policyBase64
*
* @returns {String}
* @private
*/
_s3UploadSignature(policyBase64) {
const dateKey = this._encryptHmac('AWS4' + this.secretKey, this.dateString);
const dateRegionKey = this._encryptHmac(dateKey, this.region);
const dateRegionServiceKey = this._encryptHmac(dateRegionKey, 's3');
const signingKey = this._encryptHmac(dateRegionServiceKey, 'aws4_request');
return this._encryptHmac(signingKey, policyBase64).toString('hex');
}
}
module.exports = Token;
The configuration object used is stored in SSM Parameter Store and looks like this
{
"bucket": "my-bucket-name",
"region": "us-west-2",
"bucketAcl": "private",
"accessKey": "MY_ACCESS_KEY",
"secretKey": "MY_SECRET_ACCESS_KEY",
}
Adding more info to the accepted answer, you can refer to my blog to see a running version of the code, using AWS Signature version 4.
Will summarize here:
As soon as the user selects a file to be uploaded, do the followings: 1. Make a call to the web server to initiate a service to generate required params
In this service, make a call to AWS IAM service to get temporary cred
Once you have the cred, create a bucket policy (base 64 encoded string). Then sign the bucket policy with the temporary secret access key to generate final signature
send the necessary parameters back to the UI
Once this is received, create a html form object, set the required params and POST it.
For detailed info, please refer https://wordpress1763.wordpress.com/2016/10/03/browser-based-upload-aws-signature-version-4/
To create a signature, I must use my secret key. But all things happens on a client side, so, the secret key can be easily revealed from page source (even if I obfuscate/encrypt my sources).
This is where you have misunderstood. The very reason digital signatures are used is so that you can verify something as correct without revealing your secret key. In this case the digital signature is used to prevent the user from modifying the policy you set for the form post.
Digital signatures such as the one here are used for security all around the web. If someone (NSA?) really were able to break them, they would have much bigger targets than your S3 bucket :)
I have given a simple code to upload files from Javascript browser to AWS S3 and list the all files in S3 bucket.
Steps:
To know how to create Create IdentityPoolId http://docs.aws.amazon.com/cognito/latest/developerguide/identity-pools.html
Goto S3's console page and open cors configuration from bucket properties and write following XML code into that.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Create HTML file containing following code change the credentials, open file in browser and enjoy.
<script type="text/javascript">
AWS.config.region = 'ap-north-1'; // Region
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: 'ap-north-1:*****-*****',
});
var bucket = new AWS.S3({
params: {
Bucket: 'MyBucket'
}
});
var fileChooser = document.getElementById('file-chooser');
var button = document.getElementById('upload-button');
var results = document.getElementById('results');
function upload() {
var file = fileChooser.files[0];
console.log(file.name);
if (file) {
results.innerHTML = '';
var params = {
Key: n + '.pdf',
ContentType: file.type,
Body: file
};
bucket.upload(params, function(err, data) {
results.innerHTML = err ? 'ERROR!' : 'UPLOADED.';
});
} else {
results.innerHTML = 'Nothing to upload.';
} }
</script>
<body>
<input type="file" id="file-chooser" />
<input type="button" onclick="upload()" value="Upload to S3">
<div id="results"></div>
</body>
If you are willing to use a 3rd party service, auth0.com supports this integration. The auth0 service exchanges a 3rd party SSO service authentication for an AWS temporary session token will limited permissions.
See:
https://github.com/auth0-samples/auth0-s3-sample/
and the auth0 documentation.
If you don't have any server side code, you security depends on the security of the access to your JavaScript code on the client side (ie everybody who has the code could upload something).
So I would recommend, to simply create a special S3 bucket which is public writeable (but not readable), so you don't need any signed components on the client side.
The bucket name (a GUID eg) will be your only defense against malicious uploads (but a potential attacker could not use your bucket to transfer data, because it is write only to him)
Source: Stackoverflow.com