Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opendir raises a ResourceNotFound exception for a subdirectory which does exist. #8

Closed
nackjicholson opened this issue Oct 13, 2017 · 3 comments

Comments

@nackjicholson
Copy link

nackjicholson commented Oct 13, 2017

with open_fs('s3://my-bucket') as root_fs:
     with root_fs.opendir('a_real_directory') as dir_fs:
          print dir_fs.listdir(u'/')

Raises Not Found on the line which calls opendir

I find things to work entirely when using file:// or osfs:// protocol urls, but things tend to explode when using s3://. Because I've hit two show stopping issues in two attempts to use s3fs and pyfilesystem, I'm gonna conclude that the premise of this being a compliant FS implementation is false, untested, and broken.

It's a grand idea to write interchangeable filesystem code, best of luck!

@willmcgugan
Copy link
Member

I'm gonna conclude that the premise of this being a compliant FS implementation is false, untested, and broken.

It passes all the tests. Since you've already reached a conclusion, I guess you don't require assistance diagnosing the issue.

@nackjicholson
Copy link
Author

Apologies, was frustrated because my hopes were high that this library would be able to replace my kluge home-rolled code that abstracts between pythons os and boto3/s3. Felt like I'd wasted quite a bit of time implementing things with pyfilesystem, just to have to throw all that work away and go back. I didn't catch that the s3fs tests inherit from pyfilesystem2. Doesn't change the fact that they're apparently not catching cases with s3 that seem to work with osfs.

You're correct, I don't need assistance. If you're interested in why opendir works for osfs but not s3 it's there for you.

Cheers and thanks for the effort, I learned quite a lot about the challenge of os/s3 parity by working with and looking through your implementation.

@willmcgugan
Copy link
Member

I'm certain whatever issue it is can be solved easily, which would benefit both of us. No need to vent. If you do want to figure it out, feel free to open another ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants