Skip to content

Support model directories on S3 #323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Aug 19, 2019
Merged

Support model directories on S3 #323

merged 31 commits into from
Aug 19, 2019

Conversation

1vn
Copy link
Contributor

@1vn 1vn commented Aug 14, 2019

Closes #215


Checklist:

  • Run make test and make lint
  • Test end to end manually (e.g. build/push all images, restart operator, and run cx refresh)
  • Update examples
  • Update documentation (add any new files to summary.md)
  • Merge to master
  • Cherry-pick into release branches if it's a bugfix
  • Check gitbook that docs look good and links function properly
  • Alert team if dev environment changed
  • Delete the branch once it's merged

@1vn 1vn requested a review from ospillinger August 14, 2019 19:33
for root, dirs, files in os.walk(path):
for name in files:
objects.append(os.path.join(root, name))
for name in dirs:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the S3 version of list_objects() returns directories, just objects. Can you confirm, and if so, only return files here?

@deliahu deliahu merged commit 4f9fd93 into master Aug 19, 2019
@deliahu deliahu deleted the s3-model-dir branch August 19, 2019 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support specifying an s3 directory for TensorFlow model paths
3 participants