Running multiple Docker Instances in the Cloud

Hi
We like to use TUS in a Docker Image and run it in Azure Cloud or Kubernetes Cluster. As per definition of the Clould, we have to have multiple Instances of the Image running in parallel. In a Cloud the Request are not Sticky, and Services are stopped and started “randomly”.

As is understand the Docu: FAQ | tus " How do I scale tus?", this setup will not work with TUS. Did i get that right?

Best regards
S

Hello S, you are not entirely correct. Sticky sessions is one way to avoid issues. As the FAQ mentions, data consistency can also be ensure if the server implements a locking mechanism to guarantee exclusive access to the upload resources. The details on this depend heavily on the tus server that is used. What server do you intend to use?

Hi Marius
Thanks for your response :-). We will have an Angular App and a Spring Boot App in the Cloud. We have to upload large Files.

As the FAQ mentions, data consistency can also be ensure if the server implements a locking mechanism
to guarantee exclusive access to the upload resources.

I understand the need that TUS need that exclusive Lock.
In my understanding of the Cloud very thing is stateless. The Nodes don’t know each other, and there is no native way to handle a lock.

As the FAQ mentions, data consistency can also be ensure if the server implements a locking mechanism
to guarantee exclusive access to the upload resources.

In the FAQ you mentioned etcd or Redis. But I don’t understand how i should handle the Lock mechanism of an Upload.
For me it feels more like a thing that TUS should handle. Eg when an Request is comming, checking the ID of the Client and than ask Redis if someone else is handle that ID.
Sorry I am a bit confused.

=> Do you have an example or a BlogPost or something similar where i get the idea how to Implement that.

@marius Thanks for your answer :-).

What server do you intend to use?

We like to use the standard Docker Imange from TUS. Or we can build our own Docker Imange with TUS.

I don’t understand how I should handle the Synchronisation? I would expect TUS to have an Adapter to etcd, there it stores information about the current uploads and ?ClientId?. It would sync the information about the different Instance’s.

Do you have a Article with an example where it show how a Locking Mechanism can be build?

Exactly, that is how most tus server implementations handle it. You, as the developer, do not have to implement that manually. You just need to configure your tus server properly and it will do that for you. For example, tusd has a package for using etcd as a locker: GitHub - tus/tusd-etcd3-locker

As I am right in assuming you want to use tusd? tus is the name of the protocol, but tusd is the name of our server implementation written in Go.

There is no direct article about this. You best friend is likely GitHub - tus/tusd: Reference server implementation in Go of tus: the open protocol for resumable file uploads and handler package - github.com/tus/tusd/pkg/handler - pkg.go.dev

Hope this helps a bit!