Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Separate Env into Thread and FileSystem #4746

Closed
rockeet opened this issue Dec 4, 2018 · 3 comments
Closed

Proposal: Separate Env into Thread and FileSystem #4746

rockeet opened this issue Dec 4, 2018 · 3 comments

Comments

@rockeet
Copy link
Contributor

rockeet commented Dec 4, 2018

Expected behavior

Thread and FileSystem are independent components, when we implement a new FileSystem, we need not to implement Thread.

Actual behavior

Thread and FileSystem functions are wrapped by Env. If we just want to implement a new FileSystem, we also need to implement Thread.

I know we can use EnvWrapper or delegate to default env to mitigate this issue, but this may causing other issues:

  1. Need access to default env
  2. Increase unnecessary work efforts...

Steps to reproduce the behavior

Try to implement a new FileSystem.

@mrambacher
Copy link
Contributor

I wholeheartedly agree with this suggestion.

There are many classes that attempt to implement parts of the functionality of an Env and attempt to implement/override one portion or another (like HdfsEnv and MockEnv do for FileSystem and others do for overriding timers. It would be much simpler if these interfaces would be split into separate classes.

@rockeet
Copy link
Contributor Author

rockeet commented Apr 16, 2020

Now #5761 and #6552 have aimed on this proposal.

@mrambacher
Copy link
Contributor

I think most of this work has been accomplished through the FileSystem and SystemClock classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants