-
Notifications
You must be signed in to change notification settings - Fork 184
feat: replace the workspace field in the project object with a workspace object #1429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
* feat: get all project of a group * update unit test using magicMock * update unit test & fix bug * opt import and rename some file - move api from folder core_python to api * fix wrong comment * remove pending status when getting entity projects * opt Project class - add EN comment for Project properties - add __str__ method for project label - adapt snake_case naming for project properties - change some project property name: projectLabels -> label - change api.projects() param name: entity -> workspace - delete some project properties (group, cuid) - prase url inside Project class * dynamically request project data according to the traversal of the project, rather than requesting all project information at once * opt import * iteratively request project data inside the Projects class - move api form api package to core_python package * fix bug & add type * move api's type.py to core_python
* feat: get runs metadata - add api in core_python to get all exps in a project - add new type and class to parse exps * update unit test & fix bugs * opt code style & add comment * raise value error if user's path is invaded * fix bug of Incorrectly raised ValueError * get single exp through OpenApi * resolve conflicts: filtering exp (basic implement) - get full single exp info through filter func # Conflicts: # swanlab/core_python/api/experiment.py * opt __dict__ for project and experiment * resolve conflict: update run.history() * feat: update run.scan_history() - add example codes * resolve conflict: fix bugs when importing pandas * import pandas only when used * fix: pandas warning * feat: run and user features of OpenApi - get metric data in batch - recover old OpenApi version for smooth transition - create & delete api_key through Api.user * opt Client importing in core_python's api * opt logic of run.history() - get latest api_keys when user delete api_key * rename the deprecated openApi folder * refactored run.history() & add return type - add return type for the backend interfaces - simplified get_experiment_metrics() and handle the csv inside HistoryPool * removed redundant types * update unit test using mock * feat: get user's team * feat: create user if user is the root user in self-hosted swanlab - refactor the models in OpenApi * feat: update unit test for create user and get user teams * delete test code of OpenApi * use ThreadPoolExecutor in HistoryPool * Refactor Api class and cleanup api module structure Moved the Api class implementation from swanlab/api/api.py to swanlab/api/__init__.py and deleted the redundant api.py file. Updated imports and references accordingly. Minor docstring and comment improvements, and fixed a message in thread.py to reference 'Api' instead of 'OpenApi'. * Fix circular import by updating Api export Combined the import of OpenApi and Api from .api to prevent circular import issues and simplify the export process. --------- Co-authored-by: ZeYi Lin <944270057@qq.com> Co-authored-by: Kang Li <cunykang@gmail.com>
* Refactor the OpenAPI codes, managing the code with a modular approach * combine 3 kinds of user into one - use @cached_property to cache some property of user * Place the API's custom type into a separate module - place class ApiBase into a single model * get the user identity at the init of the OpenApi - only the root user get by api.user() can create new account * refactor module name & structure in api and core_python * Modified parameter passing convention & add client property to ApiBase - pass param through kwargs except client in OpenApi and core_python * update code commit - opt str() of Label class
* add unit test - add unit test and fix bugs of getting all projects through OpenApi - add unit tests for OpenApi runs and run.history() * add unit test for api.user() * accept suggestions from gemini
* delete class ApiBase * accept suggestions - place yield inside the loop of the projects - check if creating & deleting user or api_keys inside the api func - add property is_self in user * opt user api func * fix bugs in projects * accept gemini suggestions - fix type error in self_hosted.py and user's api func - add a constant for project page size * opt over-encapsulated code - discard constant PAGE_SIZE - let adapter handle the exception when creating & deleting user info - opt unit test for projects * delete unused utils
…1425) * use a wrapper to check self_hosted info for some functions - update unit tests for user (add self_hosted context) * fix bug in OpenApi.list_workspaces() * accept gemini suggestions * raise value error when not being self_hosted * raise value error when self_hosted is not available - raise error when accessing unsupported user functions
Summary of ChangesHello @Bainianzzz, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the SwanLab Python API, transitioning from a procedural Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request significantly refactors the SwanLab OpenAPI by introducing a new Api class and restructuring the API client into a more modular, object-oriented design. The changes involve creating dedicated classes for Experiment, Projects, User, Workspace, and Workspaces, which greatly improves code organization and maintainability. The deprecated OpenApi class is moved to a deprecated module with a warning for future removal. Additionally, new API endpoints and utility functions are added, and the core Client class now includes a delete method. Test files have been updated and new tests introduced to cover the new API structure. Overall, this is a positive change towards a more robust and user-friendly API.
| path=proj_path, | ||
| web_host=self._web_host, | ||
| login_user=self._login_user, | ||
| line_count=1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The line_count parameter in the Experiment constructor is hardcoded to 1. This seems incorrect, as line_count is intended to represent the total number of historical experiments in the project. Hardcoding it to 1 will provide misleading information to the user when accessing experiment.history_line_count.
| line_count=1, | |
| line_count=len(data), |
| if self._x_axis is not None: | ||
| self._history = self._history.reset_index().iloc[:, 1:] | ||
| self._history = self._history.set_index(self._history.columns[0]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In HistoryPool.execute, when self._x_axis is not None, the code self._history = self._history.reset_index().iloc[:, 1:] drops the first column after resetting the index. This assumes that the first column is always the original index. If the original index was not named 'step' or if _history was already modified, this could lead to unintended data loss or incorrect data alignment. It's safer to explicitly specify the column to drop or rename, or ensure the index is always 'step' before this operation.
| self.metrics_data = metrics_data | ||
|
|
||
| def __enter__(self): | ||
| self.mock_get_metrics = patch('swanlab.api.experiment.thread.get_experiment_metrics').start() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The patch for get_experiment_metrics is targeting swanlab.api.experiment.thread.get_experiment_metrics. However, get_experiment_metrics is imported into swanlab.api.experiment.thread from swanlab.core_python.api.experiment. To correctly mock the function where it is originally defined and used, the patch should target swanlab.core_python.api.experiment.get_experiment_metrics.
| self.mock_get_metrics = patch('swanlab.api.experiment.thread.get_experiment_metrics').start() | |
| self.mock_get_metrics = patch('swanlab.core_python.api.experiment.get_experiment_metrics').start() |
| self.seats = kwargs.get('seats', 99) | ||
|
|
||
| def __enter__(self): | ||
| self.mock_get_metrics = patch('swanlab.api.utils.get_self_hosted_init').start() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The patch for get_self_hosted_init is targeting swanlab.api.utils.get_self_hosted_init. However, get_self_hosted_init is imported into swanlab.api.utils from swanlab.core_python.api.user. To correctly mock the function where it is originally defined and used, the patch should target swanlab.core_python.api.user.get_self_hosted_init.
| self.mock_get_metrics = patch('swanlab.api.utils.get_self_hosted_init').start() | |
| self.mock_get_metrics = patch('swanlab.core_python.api.user.get_self_hosted_init').start() |
| :param path: 实验路径,格式为 'username/project/run_id' | ||
| :return: Experiment 实例,包含实验信息 | ||
| """ | ||
| # TODO: 待后端完善后替换成专用的接口 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TODO comment indicates that the backend needs to be improved for the run method. This suggests a potential future refactoring or an incomplete implementation that might affect the reliability or functionality of this method. It's important to address such TODOs to ensure the API is fully robust.
| self._filters = filters | ||
|
|
||
| def __iter__(self) -> Iterator[Experiment]: | ||
| # TODO: 完善filter的功能(正则、条件判断) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| resp = get_user_groups(self._client, username=self._cur_username) | ||
| return [r['name'] for r in resp] | ||
|
|
||
| # TODO: 管理员可以对指定用户的api_key进行操作 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| """ | ||
| Refresh the list of api keys. | ||
| """ | ||
| del self.api_keys |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The _refresh_api_keys method attempts to delete a cached property (self.api_keys) using del self.api_keys. While cached_property typically recomputes the property when accessed after deletion, directly deleting it might not be the most robust or idiomatic way to invalidate a cached property. A more explicit way to invalidate a cached_property is often to delete the attribute from self.__dict__.
| del self.api_keys | |
| if 'api_keys' in self.__dict__: | |
| del self.__dict__['api_keys'] |
| if name.startswith("_"): | ||
| continue | ||
| if isinstance(getattr(type(obj), name, None), property): | ||
| value = getattr(obj, name, None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In get_properties, the recursive call get_properties(value) is made without a comprehensive check to ensure value is an object that defines properties. If value is a complex data structure (like a list of custom objects) or a non-property-containing object, this could lead to unexpected behavior or errors. The current check type(value).__module__ == 'builtins' only handles built-in types, but not other custom classes that don't have properties.
| parsed_filters = ( | ||
| [ | ||
| { | ||
| "key": to_camel_case(key) if parse_column_type(key) == 'STABLE' else key.split('.', 1)[-1], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In get_project_experiments, the filters parsing logic key.split('.', 1)[-1] for non-'STABLE' types might be problematic if the key does not contain a dot (e.g., just "config" or "summary"). In such cases, split('.', 1)[-1] would return the original key, which might not be the intended sub-key for the backend. Consider adding a check for the presence of a dot or handling such cases explicitly.
Description
将project对象中的workspace替换为workspace对象
example code: