Cloud_PubSub_to_GCS_Text replaces `mm` in path

[ericandrewmeadows]

recommendation becomes reco00endation, reco05endation, ... when using the following template:

gs://dataflow-templates/2020-07-13-00_RC00/Cloud_PubSub_to_GCS_Text

[sabhyankar]

Thanks for bringing this up @ericandrewmeadows

Yes, that is because of the "mm" in recommendation in the outputDirectory is being replaced by the 'minute' mark for the Window. (See WindowedFilenamePolicy).

I think we have the following options:

1. Use a different outputDirectory that doesnt have the shard names (For example, reco).
2. Pin to an older release of the template. (Use, gs://dataflow-templates/2020-05-13-00_RC00/Cloud_PubSub_to_GCS_Text or earlier)
3. Create your own custom version of the template to disable this.

[ericandrewmeadows]

I already do things on my own. Is this slated to get fixed?

[sabhyankar]

Currently, there is no plan to change the existing behavior with the formats being used for the template.

Having said that, I guess we could modify the WindowedFilenamePolicy class to accept an override for the formats. Would you be open to putting in a PR for that change?