HTML5 uploads - blobs vs files

Archived from the Transloadit support forum. Originally posted on July 8, 2011; details may be outdated, see our docs for current behavior. Original thread

Hi there,

I’m trying something relatively strange- using the new browser slice options (mozSlice) to cut out a smaller slice of a large video file, in order to get the metadata and a thumbnail of the opening frame (or thereabouts, still working on the idea) without uploading the whole file.

So far so good- I have sliced the file, and I’m ready to upload to you. However, I’m seeing different responses based on whether I send the whole file or my sliced blob, and I’m not sure why. When I upload a whole file, the POST data looks like this:

-----------------------------3195204853093
Content-Disposition: form-data; name="params" {"auth":{"key":"REDACTED"},"template_id":"[template]"}

-----------------------------3195204853093
Content-Disposition: form-data; name="file"; filename="demo (1).flv" Content-Type: application/octet-stream

[file data]

And my response looks like:

{
"ok": "ASSEMBLY_EXECUTING",
"message": "The assembly is currently being executed.",
[...]  
"uploads": [
    {
        "id": "REDACTED",
        "name": "demo (1).flv",
        "basename": "demo (1)",
        "ext": "flv",
        "size": 129444,
        [...]
    }
}

However, when I send my blob:

-----------------------------268691786114688
Content-Disposition: form-data; name="params" {"auth":{"key":"REDACTED"},"template_id":"[template]"}

-----------------------------268691786114688
Content-Disposition: form-data; name="file"; filename="" Content-Type: application/octet-stream

[file data]

I get a response that suggests the binary data wasn’t recognised as such:

{
"ok": "ASSEMBLY_COMPLETED",
"message": "The assembly was successfully completed.",
[...]
"fields": {
    "file": "FLV\u0000\u0000\u0000\t\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000[...]"
}
}

It seems like it ought to work, as the only actual POST difference I can see is in the filename=“” assignment- the file has one, the blob does not. I can’t see any easy way for me to assign a filename to a blob- is there any chance you guys are using filename as a check to see if the multipart data is, indeed, a file? If so, is there any way I can work around that?

I should add that this isn’t a cynical ploy to pay you less money- the full file will be uploaded eventually! This is just to provide a preview before the full upload…

I’m trying something relatively strange- using the new browser slice options (mozSlice) to cut out a smaller slice of a large video file, in order to get the metadata and a thumbnail of the opening frame (or thereabouts, still working on the idea) without uploading the whole file.

This is really cool. We are actually planning to build a new HTML5 version of our plugin that uses these new FileReader / Blob / etc. APIs. So far our main goal was to implement resumable file uploads, but what you are doing is very exciting as well!

is there any chance you guys are using filename as a check to see if the multipart data is, indeed, a file? If so, is there any way I can work around that?

Indeed. We are using our own multipart parser, and it does in fact check for the filename to be non-empty:

https://github.com/felixge/node-formidable/blob/master/lib/incoming_form.js#L150

Right now there is no workaround you could use, but if you would show me the code you are using to make the request, maybe I can see if there is a way to make your code send a filename.

Otherwise we could patch formidable to automatically assign the name of the field as the filename if an empty filename="" header is detected.

I should add that this isn’t a cynical ploy to pay you less money- the full file will be uploaded eventually! This is just to provide a preview before the full upload…

Who cares? This is a really exciting idea, so if you need some free credit at some point, let me know : )

Hi! Great to hear the enthusiasm for my hacking around :slight_smile: I’ve made an isolated example of the upload code- it’s FF5.0 only right now (haven’t got the cross-browser stuff wired in):

<input type="file" id="fileupload" />

<script type="text/javascript">

document.getElementById("fileupload").addEventListener("change", function(e) {

var file = this.files[0];

 var thumbRequest = {
    auth: {
        key: "[key]"
    },

    "template_id": "[template]",
};


var formPost = new FormData();
formPost.append("params", JSON.stringify(thumbRequest));
var f = file;

formPost.append("file", f.mozSlice(0,10000));

var xhr = new XMLHttpRequest();
xhr.open('POST', "http://api2.transloadit.com/assemblies", true);
xhr.send(formPost);  // multipart/form-data

},false);
</script>

As far as I can see, there’s no simple way to add a file name. mozSlice-ing a File object returns a Blob, which has no filename attribute. And I can’t find any way of converting a Blob back to a File. Any attempt at doing something like:

var b = f.mozSlice(0,10000);
b.filename = "test.avi";

Doesn’t solve the problem either. If it requires a workaround at your end, perhaps we could send some sort of “X-Multipart-Filenames” header that’ll allow us to pass through the original filenames?

Thanks!

perhaps we could send some sort of “X-Multipart-Filenames” header that’ll allow us to pass through the original filenames?

Nah, that’s nasty. Let me fix this in our upload stack for you, I’ll update this ticket hopefully later today with a solution.

just chiming in that I would LOVE to see HTML5 resumable uploads… this is very interesting.

Just to let you know- I’ve got this working using our server as a sort of pass-through- it takes the uploaded content, adds a file name, then forwards to you guys. Still a little short of the HTML5 client → Transloadit dream, but it works great.

Be sure to update us when the filename dependency issue is sorted!

Hi,

sorry for not getting this done earlier, but our upload parser now supports filename="" in multipart parts.

If your curious about the details of the fix: https://github.com/felixge/node-formidable/commit/REDACTED

I just deployed the new version of the parser to transloadit, so let me know if you can now remove the proxy on your end.

--fg