S3Store: Apply tags to temporary S3 objects

[acj]

tusd's temporary metadata objects (.info, .part, etc) can become a maintenance burden for certain applications. For example, applications may place uploaded objects directly into their final bucket in order to avoid slow copy operations. However, this means that the metadata objects are also placed in the same bucket. This can confuse automated tools and creates a nontrivial amount of cruft in the bucket over time.

This PR introduces an optional TransientObjectTags field on S3Store that takes a comma-separated list of tags to be applied to tusd's metadata objects. By tagging these objects, tusd operators can define an S3 object lifecycle policy that automatically removes the objects after a period of time.

I still need to test a handful of edge cases, but this is ready for general feedback.

The initial discussion for this feature is in #219.

[Acconut]

Thanks for the PR! If I recall correctly, you proposed this idea because S3 lifecycle rules can only filter objects based on prefixes but not suffixes, didn't you? If so, I wonder if it would be simpler to allow a custom object name prefix for the .part and .info files, so you could target those based on a simple prefix rule. What do you think?

[acj]

Sorry for the delay on my end. Yes, lifecycle rules can apply to prefixes. That approach would be simpler than using tags, but I'm concerned that existing tusd deployments wouldn't be able to safely transition to it. If there are pending uploads that are partially completed, they could fail when the prefix change is deployed because any existing .info and .part objects would still have the old name.

[Acconut]

That's a good point you bring up. However, personally I prefer the more simple approach (object prefixes) as in the long term more users will be using this feature rather than transitioning to it. So from that perspective it makes sense to choose the solution which is easier to use, understand and maintain.

I do agree that tusd has not the best support for changing storage options and would like to make this process easier but our resources are also limited so this point has received not much love in the past.

[acj]

Okay, sounds good. I'll rework this to use object prefixes and then open a followup PR.