Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[train] Fix wandb/comet integration API calls #38978

Merged
merged 6 commits into from
Aug 31, 2023
Merged

[train] Fix wandb/comet integration API calls #38978

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
222c653
[train] Fix integration API calls
Aug 28, 2023
04c9267
storage context flag
Aug 29, 2023
9dc8ea5
Merge remote-tracking branch 'upstream/master' into dir-or-data
Aug 29, 2023
4811717
review
Aug 29, 2023
1e2aedf
Merge remote-tracking branch 'upstream/master' into dir-or-data
Aug 31, 2023
5ec7132
review
Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
review 
Signed-off-by: Kai Fricke <kai@anyscale.com>
  • Loading branch information
Kai Fricke committed Aug 31, 2023
commit 5ec71329ab9e11bdfe3b6156391c64dc55cebd94
4 changes: 3 additions & 1 deletion python/ray/air/integrations/wandb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -674,13 +674,15 @@ def log_trial_result(self, iteration: int, trial: "Trial", result: Dict):

def log_trial_save(self, trial: "Trial"):
if self.upload_checkpoints and trial.checkpoint:
checkpoint_root = None
if _use_storage_context():
if isinstance(trial.checkpoint.filesystem, pyarrow.fs.LocalFileSystem):
checkpoint_root = trial.checkpoint.path
else:
checkpoint_root = trial.checkpoint.dir_or_data

self._trial_queues[trial].put((_QueueItem.CHECKPOINT, checkpoint_root))
if checkpoint_root:
self._trial_queues[trial].put((_QueueItem.CHECKPOINT, checkpoint_root))

def log_trial_end(self, trial: "Trial", failed: bool = False):
self._signal_logging_actor_stop(trial=trial)
Expand Down