Metadata decoding implementation

Tus
1

Hi everyone,

Because I can only put string values in metadata, I’d like to use "" to represent empty list and use "foo, bar" to represent ["foo", "bar"]. Then I got null after tus server decoded them.

I checked the implementation.

const decodedValue = value ? Buffer.from(value, 'base64').toString('utf8') : null

So it can’t handle "". Every false value will become null.

I can handle null and comma separated values in my own application. But I am also wondering if there is a better way to encode and decode metadata and keep original values at the same time?

2

Is this a general question about the tus protocol or related to one specific implementation, such as tus-node-server?

3

I’m not sure. After checking the protocol, it only says how to encode metadata.

The key SHOULD be ASCII encoded and the value MUST be Base64 encoded. All keys MUST be unique. The value MAY be empty. In these cases, the space, which would normally separate the key and the value, MAY be left out.

I’m not familiar with Go. So I don’t know if tusd decodes metadata the same way.

4

I recommend picking any other value than "" to represent an empty list.

5

Yes, tusd handles it as dictated by the specification. Have a look at this test case: tusd/pkg/handler/post_test.go at fb61cc478712f03f8b4cc5aa00cd7a8e10f8f54a · tus/tusd · GitHub

It seems like this line comes from tus-node-server: tus-node-server/packages/utils/src/models/Metadata.ts at b4029ab913cd3ff53665ae6aa38199553f610f81 · tus/tus-node-server · GitHub

@Merlijn Should this default to "" instead of null, so that it can properly represent empty strings? In tus, metadata values are always strings and cannot be other data types.

1 Like
6

@marius Thanks for the clarification. @Merlijn Sorry for creating extra works.

Another related question is about how to modify metadata in the current tus-node-server implementation.

If I want to update it after the file has been uploaded, I can only access the configstore directly. But a KVStore allows me to set any serializable JSON object. So I have to guard metadata values by myself.

I know I can extend the FileStore for my use cases. Could it be possible to take in consideration to provide a new interface and let users modify metadata without breaking metadata abstraction in the future implementation.

7

Hi, what is your use case? You can set metadata in onUploadCreate, can’t you do it at this stage?

8

Hi. My use case is about implementing a file load proxy API for a non-resumable upload endpoint. Sorry about the late reply. My MR finally got merged few days ago in my workplace.

Because I have to monitor the upload result and processing result after file has been uploaded, I need to update metadata multiple times. I extend FileStore, add a new method to update metadata by using the inner configstore. I think I should not use configstore directly, that why I asked for a better interface for updating metadata.

It would be nice if I can be notified after metadata has been changed. Or maybe show me a proper way to extend stores.