Skip to content

[py312] Fix or skip tests+examples for Tune dependencies that do not support py312#46761

Merged
justinvyu merged 31 commits into
ray-project:masterfrom
justinvyu:py312/ml
Jul 25, 2024
Merged

[py312] Fix or skip tests+examples for Tune dependencies that do not support py312#46761
justinvyu merged 31 commits into
ray-project:masterfrom
justinvyu:py312/ml

Conversation

@justinvyu

@justinvyu justinvyu commented Jul 23, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why are these changes needed?

Policy:

  • Tests for library integrations will be skipped on py312 if that library doesn't support py312.
    • Add this library to a py312 ML issue tracker. (TODO)
  • Tests for library integrations will be skipped altogether if it imposes a version cap on a more important library.
    • For example, numpy needs to be upgraded 1.26+ for py312. However, hebo requries numpy<1.25. Since our CI chooses a single numpy version for across the board, we'd rather update numpy than keep the hebo test dependency.
    • Same issue for horovod.
    • Add this library to a py312 ML issue tracker. Note that these are no longer tested in CI and need to be manually tested. (TODO)
  • Important library integrations that do not support py312 yet will be added to the issue tracker and treated as a medium high priority that needs to be solved as soon as the library adds the py312 support.
    • For example, tensorflow DDP doesn't work well in py312. Whenever this gets fixed, a new tf version should be pinned in test deps and the tests should be re-enabled.

Related issue number

Closes #46220

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

justinvyu added 22 commits July 23, 2024 15:11
@justinvyu
upgrade comet-ml
3c5ae23 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove tune-sklearn
abf3fc2 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
skip hebo tests in >=py312
ef60ffe 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove aim usage in tests
f5cd5bc 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update bohb requirements
7105dfa 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
skip bohb tests
fe80ba1 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
reenable bohb example for py<312
6c0f0c9 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix lint
a632997 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
add failif fixture
a35e78a 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update rich to satisfy comet-ml dep
ec27ff3 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
[TEMP] force jupytext version bump
ba1a0b2 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update requirements compiled
1a76370 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
skipif -> failif on py312 w/ reason
80aa4dd 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
failif back to skipif
54dfae2 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove hebo (p2 dep) since it enforces a np (p0 dep) version cap
5402dec 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
py312 has new stack trace formatting with more newlines
5fce55d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix keras callback test
bb1a201 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
bump jupytext version
6cda64f 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fix req
6595b7d 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove failif fixture
bd1f652 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
remove failif fixture 2
487ec15 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
fail -> skip for bohb
2a6314f 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu justinvyu mentioned this pull request Jul 23, 2024
Closed
8 tasks
justinvyu added 3 commits July 23, 2024 17:15
@justinvyu
remove pip compile pipeline dep
844aa80 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update
48c500e 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update compiled requirements
fa3f408 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
can-anyscale
can-anyscale reviewed Jul 24, 2024
View reviewed changes

@can-anyscale can-anyscale left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for working on this @justinvyu, i'll leave to @matthewdeng to see if he has any comments for a few days otherwise i'll unblock, thankks

justinvyu added 4 commits July 24, 2024 10:52
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into py31…
00e4479 
…2/ml
@justinvyu
add back ray unpin
f591825 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
update req compiled
5da6cf6 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into py31…
c06df76 
…2/ml
@justinvyu justinvyu changed the title [py312] Skip tests+examples for Tune dependencies that do not support py312 [py312] Fix or skip tests+examples for Tune dependencies that do not support py312 Jul 24, 2024
@justinvyu justinvyu enabled auto-merge (squash) July 24, 2024 21:20
@github-actions github-actions Bot added the go add ONLY when ready to merge, run all tests label Jul 24, 2024
justinvyu added 2 commits July 24, 2024 14:38
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into py31…
24cf931 
…2/ml
@justinvyu
add back dep
ac31d46 
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
@github-actions github-actions Bot disabled auto-merge July 24, 2024 21:39
matthewdeng
matthewdeng approved these changes Jul 25, 2024
View reviewed changes
Comment thread python/ray/air/tests/test_tracebacks.py
tuner = Tuner(failing)
results = tuner.fit()
assert len(str(results[0].error).split("\n")) <= 12
assert len(str(results[0].error).split("\n")) <= 20

@matthewdeng matthewdeng Jul 25, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happened here?

@justinvyu justinvyu Jul 25, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

py312 adds a bunch of new lines + formatting to stack traces 😆

@justinvyu justinvyu merged commit eac96b5 into ray-project:master Jul 25, 2024
@justinvyu justinvyu deleted the py312/ml branch July 25, 2024 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@can-anyscale can-anyscale can-anyscale left review comments

@matthewdeng matthewdeng matthewdeng approved these changes

@hongpeng-guo hongpeng-guo Awaiting requested review from hongpeng-guo

@raulchen raulchen Awaiting requested review from raulchen

@woshiyyya woshiyyya Awaiting requested review from woshiyyya

Assignees

@matthewdeng matthewdeng

@can-anyscale can-anyscale

Labels

go add ONLY when ready to merge, run all tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[python 3.12][ml] fix tests for python 3.12

3 participants

@justinvyu @matthewdeng @can-anyscale