Resumable AWS S3 multipart integration

Uppy
1

Hi,

What is the way correct of setting up resumable multipart uploads for aws?
When I configure companion and uppy and start uploading files, abort around 50%, reload my browser, then reupload the same file, it starts at 0% again instead of continuing at 50%.

What is the benefit of using multipart uploads if this feature is not working? Or is golden retriever needed for this? I’ve tried that but that gave errors for files being to big for local storage.

If you look at the multipart sdk doc for PHP for instance (https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/s3-multipart-upload.html#recovering-from-errors) you see it saved the state of the uploaded parts so it can continue where it left off.

Hope someone can help me out, thanks!
Rick

2

Hi! Yes, Golden Retriever is intended to handle this, and unfortunately it doesn’t support large files yet :confused: We’ve been meaning to do it since landing Golden Retriever almost two years ago: https://github.com/transloadit/uppy/issues/257 but priorities have been hard.

I’m setting a post-1.0 release Slack reminder for us to actually examine what we neeed to do to implement this, because it’s been a very vague todo thus far which has made it hard to start on. We’ll know more in 3 weeks :slight_smile:

3

Hi,

I’m trying to test AWS S3 multipart upload at the moment via some prototyping.

This thread is very relevant to what I’m doing, as the use case is multiple files each up to a maximum of 2GB in size.

Questions:

  • What is Golden Retriever? I’ve come across it a couple of times but no explanation in the docs.
    Edit: Never mind, found it: https://uppy.io/docs/golden-retriever/
    What’s the file size limits on that though? You say large file support but how large is large?
    (seeing as your maximum file size limit is 20gb)

  • Can https://uppy.io/docs/aws-s3-multipart/ integrate with Dashboard? The use case needs a seamless flexible UX and Dashboard is great. If so, is there a simple example illustrating this as I can’t get it working on the info in the current docs.

  • Has the issue raised in the original post re: large files and Resumable AWS S3 multipart integration been solved with Uppy++?

ty.

4

I’m not sure how you mean, do you want to fetch files from S3 in the Dashboard? If it’s for uploads then yes any Destination plugin can be used with it.

For large files, we show a ghost icon, which is an indicator to users that they need to drop the large file on top of the icon again. For smaller files, we can restore them from the browser’s memory.

The tipping point is currently 10MB per file and 300MB total, but we may tweak that in the future without doing a major version release, so beware to not build any business (logic) on top of knowing these values.

5

Thanks for the clarifications.

I’m not sure how you mean, do you want to fetch files from S3 in the Dashboard? If it’s for uploads then yes any Destination plugin can be used with it.

For uploads.

6

Right, so the Dashboard can be combined with any Destination plugin :ok_hand:

7

Thanks, I figured it out.

1 Like