After really struggling to get AWS S3 working with Uppy using NodeJS and Express, I thought to post a complete working example here.
This example uses pre-signed URLs generated by your NodeJS server (i.e. no Companion needed), allows you to upload metadata to your AWS S3 bucket, and also allows you to add tags (adding tags to a pre-signed upload isn’t well documented anywhere - not by Uppy or by AWS - and involves a bit of voodoo).
We use a CDN rather than Webpack (I’ll separately post a tutorial on Webpack once I figure it out…the Uppy documentation seems limited to “Use Webpack”…), and in our examples we use the Uppy Dashboard plugin, and allow the user to add a filename and a caption before uploading. We’ll use the filename and caption as metadata, and as tags, for demonstration purposes. And we use a button to trigger the Uppy modal (I haven’t shown any styling here to keep things simple, but in my use case I use Bootstrap to style the button).
What this example doesn’t include: bucket permissions and CORS settings. You can learn more about AWS CORS settings here, while there’s also some guidance from the Uppy folks here. You’ll need to figure out appropriate settings for your particular needs. I’m developing in AWS Cloud 9, which allows me to test uploads despite my bucket being set to “Block all public access”. Your mileage will almost certainly vary, but this topic is beyond the scope of this example.
I’ve tried to make this as clear and simple as possible, and included comments on virtually every line.
This assumes you have an AWS account and credentials, and have set up an S3 bucket. Visit AWS and get set up before proceeding, if you haven’t already.
Spot errors or have suggestions? I strongly welcome both.
Let’s get started!
The logic of this process, as I understand it, is as follows:
- The user adds file(s) to Uppy and clicks the “Upload” button on the client/front end
- Uppy sends a call to the server/back end for upload instructions
- The server/back end prepares a pre-signed URL and sends it to the client/front end
- The client/front end uploads the file(s) directly to the AWS S3 bucket using the URL provided by the server/back end - the file does NOT get uploaded to, or pass through, the server/back end
- Create your upload template as an Express .ejs file (i.e. ‘upload.ejs’):
UPPY CONTAINER in the ‘upload.ejs’ file - this is the client/front end file (note: if you hover near the bottom of the code box, a scroll bar will appear, allowing you to scroll to the right and see the hidden code):
<div>
<label for="uppyDemo">Upload Documents:</label>
<button type="button" class="UppyModalOpenerBtn">Attach documents:</button> <!-- you'll need to add a class - 'UppyModalOpenerBtn' in this case - to use as the trigger in the Dashboard below... -->
<div class="DashboardContainer" id="drag-drop-area"></div>
<script src="https://transloadit.edgly.net/releases/uppy/v1.6.0/uppy.min.js"></script> <!-- the Uppy CDN -->
<script> // our Uppy configuration starts here
const AwsS3 = Uppy.AwsS3 // that's an uppercase "A" and lowercase "ws"...
const uppy = Uppy.Core({
id: 'uppyUpload', // use an id if you plan to use multiple Uppys (on different pages etc.)
autoProceed: false, // if true the file uploads as soon as you drag it into the upload area, or select it - your user cannot edit file name or add metadata or tags - for this reason, we use 'false'
restrictions: { // we can add restrictions here:
maxFileSize: 31457280, //i.e. 30MB
maxNumberOfFiles: 20,
minNumberOfFiles: null, // if null, no minimum requirement
allowedFileTypes: null // can use an array of extensions, i.e. ['.doc', '.docx', '.xls', '.xlsx']
},
logger: Uppy.debugLogger, // use this for debugging to see what's gone wrong
})
.use(Uppy.Dashboard, { // configure the Uppy Dashboard plugin...
trigger: ".UppyModalOpenerBtn", // what we click to make Uppy dashboard modal appear - this is the class of our button, above
inline: false, // if true, dashboard becomes part of layout rather than a modal - you can eliminate the button in that case
closeModalOnClickOutside: true, // if true, we can click anywhere outside the modal to close it
showLinkToFileUploadResult: false, // if true, a link to the uploaded file is generated and copies to the user's clipboard when clicked. Note the link won't contain credentials, and you'll need to configure your bucket's permissions and CORS settings to get this to work
target: '#drag-drop-area', // this is the id of the <div> above - use a class instead of an id if multiple drag drop areas are required
replaceTargetContent: false, // if true, removes all children of the target element - use this to put an upload form in place which disappears if Uppy appears - a fallback option
showProgressDetails: true, // false shows % only while true shows % + MB of MB plus time left
proudlyDisplayPoweredByUppy: true, // true shows 'Powered by Uppy' branding; false removes this. Attribution is nice...
note: "Images, Word, Excel, PDF, and similar files only, max 20 files of 30 MB each", // instructions to your users
height: 470, // height of the Dashboard in pixels - only applies if "inline: true" above - doesn't apply here since we are using the modal - have included here for reference only
metaFields: [ // here we can include user editable fields, and we can use these elsewhere (i.e. upload as metadata, use as tags, use in our DB, etc.) Note that these don't seem to work out of the box - we need to add our own logic to make them work (notwithstanding the Uppy documentation...)
{id: 'name', name: 'Name', placeholder: 'You can rename the file here'}, // id is what we'll use to refer to this; name is what the user sees; placeholder is placeholder text
{id: 'caption', name: "Caption", placeholder: "Briefly describe what the file contains"}
],
browserBackButtonClose: true // true allows the user to click the browser's back button to close the modal, rather than go back a page - this is a good idea!
})
.use(AwsS3, { // use the AwsS3 plugin
fields: [ ], // empty array
getUploadParameters(file){ // here we prepare our request to the server for the upload URL
return fetch('/uploader', { // we'll send the info asynchronously via fetch to our nodejs server endpoint, '/uploader' in this case
method: 'POST', // all the examples I found via the Uppy site used 'PUT' and did not work
headers: {
'content-type': 'application/x-www-form-urlencoded', // examples I found via the Uppy site used 'content-type': 'application/json' and did not work
},
body: JSON.stringify({
filename: file.name, // here we are passing data to the server/back end
contentType: file.type,
metadata: {
'name': file.meta['name'], // here we pass the 'name' variable to the back end, with 'file.meta['name']' referring to the 'name' from our metaFields id above
'caption': file.meta['caption'] // here we pass the 'caption' variable to the back end, with 'file.meta['caption']' referring to the 'caption' from our metaFields id above
},
})
}).then((response) => {
return response.json(); // return the server's response as a JSON promise
}).then((data) => {
console.log('>>>', data); // here you can have a look at the data the server sent back - get rid of this for production!
return {
method: data.method, // here we send method, url, fields and headers to the AWS S3 bucket
url: data.url,
fields: data.fields,
headers: data.headers,
};
});
},
})
uppy.on('complete', (result) => {
if(result.successful){
console.log('Upload complete! We’ve uploaded these files:', result.successful); // if upload succeeds, let's see what we uploaded
} else {
console.log('Upload error: ', result.failed); // if upload failed, let's see what went wrong
}
})
</script>
</div>
- Create your NodeJS backend code:
require('dotenv').config(); const express = require('express'), var aws = require('aws-sdk'); // require the aws-sdk aws.config.update({ // configure your AWS access accessKeyId: process.env.AWS_ACESS_KEY_ID, // remember to use environment variables - store you credentials in a .env file and use .gitignore! secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, signatureVersion: 'v4', // important to include this region: process.env.AWS_REGION }); var s3 = new aws.S3(); const myBucket = process.env.AWS_S3_BUCKET; // the bucket you'll be uploading to // UPPY UPLOAD ROUTE router.post("/uploader", (req, res) => { // you can put middleware here to check if user is logged in var metaName = ""; // we're going to use the user input file name as metadata, and if the user hasn't input anything, we'll use the original file name. Here we prepare the variable if(JSON.parse(Object.keys(req.body)[0]).metadata.name === ""){ // here we test if the user has input anything in the 'name' field - this is the convoluted way we need to extract the name from the passed stringified JSON object - if there's a better way to do this, I'm all ears... metaName = JSON.parse(Object.keys(req.body)[0]).filename; // if the user hasn't made any changes, just use the original filename } else { metaName = JSON.parse(Object.keys(req.body)[0]).metadata.name; // otherwise, use whatever the user has input (this should probably be sanitized...) } var metaCaption = ""; // set up a variable to capture whatever the user input in the 'caption' field if(!JSON.parse(Object.keys(req.body)[0]).metadata.caption){ // if there is no caption... metaCaption = ""; // set it blank } else { metaCaption = JSON.parse(Object.keys(req.body)[0]).metadata.caption; // otherwise, use what the user input } let tag1 = 'fileName=' + metaName; // build our first tag - AWS tags look like 'key=value' so we'll use 'fileName' as a key and then take our metaName variable from above let tag2 = ""; // set up our second tag for the user input caption - we'll only add this tag if the user has input something if(metaCaption){ // if the caption exists... tag2 = 'fileDescription=' + metaCaption; // add it to the 'fileDescription' key } else { tag2 = ''; // otherwise, it's blank } const tags = tag1+"&"+tag2; // we need to concatenate the tags, joining them with an ampersand so they'll look the way our AWS S3 bucket expects: 'key1=value1&key2=value2' etc. const combinedTags = String(tags); // now let's turn our combined tags into a string const params = { // now let's set up our parameters for the pre-signed key... Metadata: { // here we're adding metadata. The key difference between metadata and tags is that tags can be changed - metadata cannot! 'fileName': metaName, // add the user-input filename as the value for the 'fileName' metadata key 'caption': metaCaption, // add the user-input caption as the value for the 'caption' metadata key 'user': req.user.username, // let's grab the user who uploaded this and use the username as the value with the 'user' key 'uploadDateUTC': Date(), // and let's grab the UTC date the file was uploaded }, Bucket: myBucket, // our AWS S3 upload bucket, from way above... Key: `Account_Uploads/${Date.now().toString()}-${JSON.parse(Object.keys(req.body)[0]).filename}`, // what we'll call our file - here I'm using a folder called "Account_Uploads" to put all my uploads into, then prepending a date string to the filename to avoid collisions - S3 will overwrite a file if another with the same key (i.e. name) is uploaded! We have to again extract the filename in a tedious fashion... ContentType: JSON.parse(Object.keys(req.body)[0]).contentType, // get the content type Tagging: "random=random", //inexplicably, if we don't put this in, tagging fails. Any tags will do here - I literally use 'random=random' - something just needs to be here...this is the voodoo section of our program... }; s3.getSignedUrl('putObject', params, (err, url) => { // get the pre-signed URL from AWS - if you alter this URL, it will fail res.status(200).json({ // send info back to the client/front end method: 'put', // our upload method url, // variable to hold the URL fields: {}, // leave this an empty object headers: {'x-amz-tagging': combinedTags} // here we add the tags we created above }); }); }); module.exports = router;
This works consistently for me: uploads files, adds metadata, and adds tags.
Edit: added screenshots:
Uppy Dashboard:
Adding metaFields - these will become tags and metadata in our example:
Ready to upload:
Upload complete - our files are now in our AWS S3 bucket:
Here in the AWS S3 console we see the metadata and tags successfully applied to our uploaded file:
I’m happy to help troubleshoot your issues, as I’m able - I spent weeks trying to get this right - if I can save you some frustration, I will.
Edit: some excellent information about the differences between metadata and tags in AWS here.
It really helped me understand how uppy works 
