Unable to get to work the AwsS3 Plugin

ameft · August 29, 2017, 2:35pm

Hello to everyone. First of all thanks for the amazing uploading library you’re building. I was trying to use the recent AwsS3 plugin but despite many tries, having read the documentation and having run the example (which doesn’t create bundle.js on my system) I still didn’t succeeded.

To test just the upload without having to relay on the backend to generate a signed URL I manually return an object in getUploadParameters() with method, url and fields manually set taking the values from another working upload form we have in the app which uses Jquery File Upload, but I don’t know if the format I use for the values is correct, that’s why I tried to run the example to see what are some expected values (but like I said the server starts but bundle.js is not created).

This is how I configured the plugin:

.use(Uppy.AwsS3, {
     getUploadParameters (file) {
       return {
           method: 'post',
           url: 'https://MYBUCKET.s3.eu-central-1.amazonaws.com',
           fields:  {"key":"uploads/development/5/d4310c35-bc5f-4a9b-acda-bdcf01124669/${filename}","success_action_status":"201","acl":"public-read","policy":"blablaBASE64stuff","x-amz-credential":"AKIAI2OUIBUCMIL6GFZA/20170829/eu-central-1/s3/aws4_request","x-amz-algorithm":"AWS4-HMAC-SHA256","x-amz-date":"20170829T110829Z","x-amz-signature":"f82582a0ec7658b7409a67190f84c3b9f2aaf151a1227e9cd2e187acd27d553c"}
         }
     }
})

These are the messages from the browser console with debug set to true for Uppy:

LOG: Dashboard width: 765 upload_widget.js:239:11045
Object { source: "DashboardUI", name: "DSC07459.jpg", type: "image/jpeg", data: File }  upload_widget.js:247:640
LOG: Added file: DSC07459.jpg, dsc07459jpg1497792680000, mime type: image,jpeg  upload_widget.js:239:11045
LOG: XHRUpload is uploading... upload_widget.js:239:11045
LOG: uploading 1 of 1 upload_widget.js:239:11045
uncaught exception: Upload error  (unknown)

I also don’t understand what could be the expected values in case of method PUT without fields set and just the URL.

Thank you so much for the help.

Greeting from Italy

ameft · August 30, 2017, 11:04am

I just realized that the bundle at https://unpkg.com/uppy@0.18.0/dist/uppy.min.js include all the plugins except the new AwsS3 one! Uppy should give better messages, it doesn’t complain at all that I call Uppy.AwsS3 which is undefined.
So my problem now is how to include that plugin using that bundle and not npm…

goto-bus-stop · August 30, 2017, 11:10am

Hi, thanks for reaching out! The upload parameter format looks fine.

Note that the signature generated by uppy-server does have an expiration time of 5 minutes, so it has to used pretty soon after it’s generated. If you’ve copied the parameters manually that may be the issue.
If not, there should be more information about the error in the Network devtools. In devtools, the POST request to BUCKET.s3.REGION.amazonaws.com should contain an XML document with an explanation for the error. We don’t display error messages from the upload endpoint (S3 in this case) in the console at the moment, I’ll open an issue for that.

goto-bus-stop · August 30, 2017, 11:15am

Oh, we were writing replies at the same time .use() should be throwing an error when it receives an undefined plugin, but it’s not a very clear one:

Is it not doing anything like that for you either?

We’ll add AwsS3 to the unpkg bundle in the next release, which should be out by the end of the week.

ameft · August 30, 2017, 12:58pm

Hi @goto-bus-stop, thanks for your reply! Actually in my browser console (FF) I’ve never saw that error message, but I get this if I try:

Nice to hear that the AwsS3 plugin will be added in the bundle by the end of the week, but there is now a way for me to include it manually or I have to wait?

Also, in the plugin if I specify PUT as method, what is the expected format for the URL? How can it contain everything it need to upload to S3? The documentation describes it but I can’t figure it out. This is mostly out of curiosity as I think I don’t need to use it.

BTW I don’t use uppy-server (yet). I’ve a Rails backend and I’m still trying to figure out how to integrate the two in our architecture.

Thanks for the help!

goto-bus-stop · August 30, 2017, 1:12pm

Right now it’s only possible to include the AwsS3 plugin if you’re using npm and a bundler like Browserify. Then you can require('uppy/lib/plugins/AwsS3') to use it. If you’d rather not introduce all that tooling to your project, you’ll have to wait a few days unfortunately, sorry

With PUT, the URL should be a presigned URL for the putObject operation, best generated with the AWS SDK. I think in Ruby that would be:

s3 = Aws::S3::Resource.new(region: 'REGION')
object = s3.bucket('BUCKET').object('FILENAME')

url = object.presigned_url(:put)

(I’m not very familiar with Ruby myself so I hope that’s right)

The AWS SDK adds all the metadata, content-type, signatures etc. to the URL in query parameters for those URLs. Everything that would be a form field in a POST upload is a query parameter with a presigned PUT URL.

ameft · September 4, 2017, 4:30pm

Hi @goto-bus-stop, do you have any news about the new bundled version?

Thanks!

goto-bus-stop · September 4, 2017, 4:48pm

Hi, yes. Uppy v0.19 is going to be delayed for a bit longer unfortunately, but I’ll release a 0.19-beta in the next few hours. I’ll get back to you with usage instructions after that. Sorry for the trouble!

ameft · September 4, 2017, 5:02pm

Well, thank you very much for your help!

goto-bus-stop · September 4, 2017, 6:15pm

Just released the version with the fixed bundle as uppy@next. Use this URL instead of unpkg.com/uppy/dist/uppy.min.js: https://unpkg.com/uppy@next/dist/uppy.min.js
It’s the same change (0.18.0→next) for the CSS URL. It also includes some better warnings for undefined plugins etc. I hope that fixes it!

ameft · September 5, 2017, 10:34am

Hi @goto-bus-stop, I tried the new 0.18.1 version with everything else set as I showed in the first post, unfortunately I get this error:

And in browser console:

LOG: Installing DashboardUI to a DOM element  upload_widget.js:238:12218
LOG: Installing StatusBarUI to DashboardUI  upload_widget.js:238:12218
LOG: Installing Informer to DashboardUI  upload_widget.js:238:12218
LOG: Installing Webcam to DashboardUI  upload_widget.js:238:12218
LOG: Core is run, initializing actions...  upload_widget.js:238:12218
LOG: Dashboard width: 2543
LOG: [99]</module.exports</t.prototype.prepareUpload/<@//XXX/upload_widget.js:270:2421
[99]</module.exports</t.prototype.prepareUpload@//XXX/upload_widget.js:270:2372
[84]</</Uppy</e.prototype.runUpload/</n<@//XXX/upload_widget.js:238:13445

Is it normal that there are not messages regarding the installing of XHRUpload and AwsS3 plugins?

goto-bus-stop · September 5, 2017, 11:03am

Ah! Yeah, it looks like there’s a bug in how getUploadParameters is handled. Right now it must return a Promise. So, instead of

getUploadParameters () {
  return { host: 'xyz' /* et cetera */ }
}

Use

getUploadParameters () {
  return Promise.resolve({ host: 'xyz', /* et cetera */ })
}

That should fix that issue. We’ll also address this in Uppy so you can return a plain object in the next release.

Yes, that’s normal The “Installing” log messages refer to installing UI plugins into other UI plugins.

ameft · September 5, 2017, 3:49pm

It finally works with the Promise! There are only some problems with the messages, strange encoding, never encountered before (I translated some strings but not that the ones with the issue):

After the upload to S3 has succeded I would like to post all the S3 URLs to our server. I’m thinking to listen to on core:upload-success for every file and create some kind of array or hash with all the url.location, then on core:success I do some fetch(); or there is a better way all inside core:success?
Do you think it’s the right approach?
Thank you!

goto-bus-stop · September 5, 2017, 5:13pm

Those are supposed to be Unicode characters like · and ↑. Adding <meta charset="utf-8"> at the top of your <head> element may fix it, if it’s not there already. If that’s not it, I’m not sure

Inside core:success, you can retrieve the files and use their uploadURL property:

uppy.on('core:success', () => {
  const urls = Object.keys(uppy.state.files).map((fileID) => {
    const file = uppy.getFile(fileID)
    const url = file.uploadURL
    return url
  })

  postToServer(urls)
})

That will include files that were already uploaded in case the user has used the widget to upload multiple times. If that’s a concern, you would indeed need to listen to core:upload-success and clear the list of files inside core:success once you’ve handled them.

(core:success currently receives a list of file IDs that were uploaded as the first parameter, but I think that’ll be changed in a future release, so it’s best not to rely on that for now.)

Thanks for using Uppy at this stage! You’re uncovering a lot of things we hadn’t thought of yet

ameft · September 7, 2017, 12:25pm

That characters worked perfectly with the previous bundle, but not with the new one, meta charset utf-8 is set.

The solution you suggested worked beautifully.
Thanks to you for the precious help and the excellent work you’ve done with Uppy, both code-wise and UX-wise!
Looking forward to test it deeply and to try Uppy Server.

ameft · September 7, 2017, 4:22pm

@goto-bus-stop I’ve encountered a strange bug (I suppose).

uppy.on('core:success', () => {
  const urls = Object.keys(uppy.state.files).map((fileID) => {
    const file = uppy.getFile(fileID)
    debugger
    const url = file.uploadURL
    return url
  })
})

In the piece of code you suggested, sometimes file.uploadURL is undefined (I noticed it when I upload more the one file, and debugging the issue is presented on the first file). Most of the times works smoothly, but 20-30% of the times this issue happens blocking the upload process:

Any idea?

Lastly, I’ve another question: I usually don’t neet getUploadParameters(file) to make a request to our server for each file, it’s enough the first time to obtain the reqired fields to upload over S3 all the files. It’s possible somehow?

Thank you!

goto-bus-stop · September 9, 2017, 12:06pm

Gotcha, that’s surprising I’ll see if I can reproduce that!

Ooh, you’re right–Uppy fires the core:success event after all uploads progress to 100%, but the browser can take some time before actually calling the XHR request’s onload handler that Uppy uses to configure the uploadURL. I’ve opened a PR at core: Remove `core:upload-complete` event. by goto-bus-stop · Pull Request #323 · transloadit/uppy · GitHub to deal with that. Will try to get it in ASAP.

It’s possible, although not very pretty…You can create a Promise for the request the first time, and return the same Promise for every next call.

let cachedParameters = null

uppy.use(AwsS3, {
  getUploadParameters () {
    if (!cachedParameters) {
      cachedParameters = fetch('/s3-fields-endpoint')
        .then((response) => response.json())
    }
    return cachedParameters
  }
})

That’d reuse the same parameters the entire time until the user leaves the page. Maybe we also need some way to tell the S3 plugin that parameters are valid for all files that are being uploaded

Vikas · February 16, 2018, 12:18pm

Hi Everyone !! I am facing an issue to plugin with Aws S3. I put all the keys in Server.js . I followed all the step of Uppy + AWS S3 example but it giving me blank screen even console don’t show any error.

Here is my output screen :-

Vikas · February 16, 2018, 12:19pm

Below is my code screenshot.

Index.html

Vikas · February 16, 2018, 12:20pm

Main.js

Topic		Replies	Views
Aws S3 getUploadParameters help Uppy	2	2292	September 18, 2019
Uploading S3 via API Uppy	1	1462	April 6, 2020
Unable to work the AwsS3 Plugin Uppy	2	1648	February 20, 2018
Uppy Aws S3 Unable to setup Uppy	3	2079	May 30, 2020
Vue 3, AwsS3 issue, 412 status Precondition failed Uppy	0	817	October 21, 2022

Unable to get to work the AwsS3 Plugin

Related Topics