-
Notifications
You must be signed in to change notification settings - Fork 6.9k
[release test] move in join_cloud_storage_paths
#57679
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request refactors the codebase by moving the join_cloud_storage_paths function from the general util module to the anyscale_job_runner module, where it is exclusively used. This is a good change that improves modularity. I've added a review comment to improve the implementation of the moved function, making it more robust against edge cases and more Pythonic.
| def _join_cloud_storage_paths(*paths: str): | ||
| paths = list(paths) | ||
| if len(paths) > 1: | ||
| for i in range(1, len(paths)): | ||
| while paths[i][0] == "/": | ||
| paths[i] = paths[i][1:] | ||
| joined_path = os.path.join(*paths) | ||
| while joined_path[-1] == "/": | ||
| joined_path = joined_path[:-1] | ||
| return joined_path |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current implementation of _join_cloud_storage_paths has a couple of potential issues and can be simplified.
- Potential
IndexError: The expressionpaths[i][0]will raise anIndexErrorifpaths[i]is an empty string. Similarly,joined_path[-1]will raise anIndexErrorifos.path.joinreturns an empty string (e.g., if no paths are provided). - Inefficient string manipulation: The
whileloops for stripping leading and trailing slashes can be replaced with the more idiomatic and efficientlstrip('/')andrstrip('/')string methods.
Here is a suggested improvement that addresses these points, making the function more robust and Pythonic.
| def _join_cloud_storage_paths(*paths: str): | |
| paths = list(paths) | |
| if len(paths) > 1: | |
| for i in range(1, len(paths)): | |
| while paths[i][0] == "/": | |
| paths[i] = paths[i][1:] | |
| joined_path = os.path.join(*paths) | |
| while joined_path[-1] == "/": | |
| joined_path = joined_path[:-1] | |
| return joined_path | |
| def _join_cloud_storage_paths(*paths: str): | |
| path_list = list(paths) | |
| if len(path_list) > 1: | |
| for i in range(1, len(path_list)): | |
| path_list[i] = path_list[i].lstrip("/") | |
| joined_path = os.path.join(*path_list) | |
| return joined_path.rstrip("/") |
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: xgui <xgui@anyscale.com>
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
into `anyscale_job_runner`. it is only used in `anyscale_job_runner` now Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>
into
anyscale_job_runner. it is only used inanyscale_job_runnernow