Unexpected result re-uploading file

Tus
1

I’m getting unexpected results attempting to use different TUS clients to upload the same file to the same server.

I’m expecting that if, after uploading a file, I try to upload the file again with a different TUS client, that it would resume where it left off, or skip uploading altogether if the last upload was completed.

However, I’m seeing inconsistent behavior. I’m testing with uppy (with multiple browsers to simulate multiple clients) and flutter tus_client. I see the similar behavior below with either client.

  1. Using tusd back end, it creates a new set of upload files in the datadir which are identical. Inside each .info file is metadata in JSON format. Name, filetype, upload status, etc. Essentially, data duplication. EG:
    8f807bac103bb8ab1a91a58cd2ce9399
    8f807bac103bb8ab1a91a58cd2ce9399.info
    a7df9010d4d20e5dc98100ef823eaf54
    a7df9010d4d20e5dc98100ef823eaf54.info

  2. Using ankitpokhrel/tus-php backend, I find that it appends data to the end of the target file in the data directory so that the file contains the same data twice - essentially file corruption.

I just want to confirm: what is the expected behavior? Is tus expected to resolve the case where a different client (or one with amnesia) resumes or re-uploads the same file again?

A careful read of the protocol spec doesn’t mention this actual scenario. My thoughts are that when using the Creation extension (as both clients are) that any of these seems “right”:

A) On the initial Creation POST, the pre-existing file is found and returns 409 CONFLICT.
B) On the initial Creation POST, the pre-existing file is zeroed out, allowing the new upload to continue normally.
B) On the initial Creation POST, the pre-existing file is found, and the new upload resource ID is set to resume at the offset of the existing file found on the server. In this case, Patch calls would also have to be checked to ensure that the offset matches the file on disk.

“The Upload-Offset header’s value MUST be equal to the current offset of the resource. In order to achieve parallel upload the Concatenation extension MAY be used. If the offsets do not match, the Server MUST respond with the 409 Conflict status without modifying the upload resource.”

I’d love to hear y’all’s $0.02 on this matter.

2

To be clear, I was expecting behavior somewhat akin to

curl -C - $URL to the destination leading me to option C (mistakenly written as B) in previous post.

3

Resumption of uploads is the responsibility of the client. When a client is tasked to upload a file, it may check if it started uploading this file before by consulting a local upload URL cache. If so, it can decide to resume the upload or start a new one. This decision does not involve the server and is fully handled by the client.

Therefore, if you want to resume an upload from another client, they need a shared and compatible local upload URL cache. Starting an upload on iOS and finishing it on Android, won’t work without additional work.

What you could do is setup an additional server that client can send their file’s checksums to and the server will point the client to an existing upload for this file, if one exists. But tus servers do not include this feature by default.

This sounds like a bug to me. You might want to report this to the authors of tus-php.

I hope this helps!