Extended GCS Functionality

[Joshua-Lister98]

Summary

• Added model registry functions
• Added batch operations to google storage class utilizing those in the base class.
• Added Model specific operations

The aim here was to easily be able to manage model & weights

RFC

Does it make sense to have all these extra functions in this GCStorageHandler class or should it only contain primitives and we make other classes for model registry + model specific operation (i.e. upload/download weights/configs).

TO DOS

Add tests when all functionality is agreed upon

[canelbirlik]

I don't think extra domain specific functionality should be part of it and domains(like registry) should write services that use this adapter. My suggestion on how to break it:
layer1: GCS: Stays minimal defining infra ops,i.e. Infra adapter
layer2: TransferManager(Optional): integrity checks, deciding whether to download/upload something. Generic service, infra agnostic.
layer3: ModelRegistryService or RegistryService: implements versioning, task registration and rest of the tasks that require knowledge of what this domain is about.
If you think there won't be another service that needs integrity checks and deciding whether to download or skip some file, you could make transfer manager part of the registry service and skip that layer.

[vik-rant]

I agree that GCS should only be concerned with and offer primitives for handling storage ops.