Add the Ray integration #15441

FlorentClarret · 2023-08-01T08:43:34Z

What does this PR do?

Add the Ray integration

Motivation

2023Q3 OKR

Additional Notes

I will add the documentation and the assets in follow-up PRs to keep this PR as small as possible to focus on the implementation.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
PR title must be written as a CHANGELOG entry (see why)
Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
PR must have changelog/ and integration/ labels attached
If the PR doesn't need to be tested during QA, please add a qa/skip-qa label.

github-actions · 2023-08-01T08:48:00Z

Test Results

2 files 2 suites 4m 47s ⏱️
7 tests 7 ✔️ 0 💤 0 ❌
9 runs 7 ✔️ 2 💤 0 ❌

Results for commit 4cda273.

♻️ This comment has been updated with latest results.

codecov · 2023-08-01T08:58:08Z

Codecov Report

Merging #15441 (4cda273) into master (713d6b4) will increase coverage by 0.12%.
Report is 20 commits behind head on master.
The diff coverage is 96.45%.

Flag	Coverage Δ
activemq	`?`
cassandra	`?`
hive	`?`
hivemq	`?`
hudi	`?`
ignite	`?`
jboss_wildfly	`?`
kafka	`?`
presto	`?`
ray	`96.45% <96.45%> (?)`
solr	`?`
tomcat	`?`
weblogic	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

github-actions · 2023-08-03T11:11:52Z

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

github-actions · 2023-08-04T12:32:23Z

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

github-actions · 2023-09-05T09:49:56Z

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

estherk15

@FlorentClarret This does not need Documentation review yet correct?

ray/README.md

ray/CHANGELOG.md

yzhan289 · 2023-09-05T19:40:38Z

ray/datadog_checks/ray/metrics.py

+    'ray_internal_num_infeasible_scheduling_classes': 'internal_num.infeasible_scheduling_classes',
+    'ray_internal_num_processes_skipped_job_mismatch': 'internal_num.processes.skipped.job_mismatch',
+    # There is a typo in their metric name, adding another entry to be ready as soon as they fix it
+    'ray_internal_num_processes_skipped_runtime_enviornment_mismatch': 'internal_num.processes.skipped.runtime_environment_mismatch',  # noqa: E501


Question: is there a doc for these metrics? I know some are system metrics but not all. I was trying to find where the misspelled ray_internal_num_processes_skipped_runtime_enviornment_mismatch is documented so we can check that doc in the future to know it has been fixed.

https://docs.ray.io/en/latest/ray-observability/reference/system-metrics.html#system-metrics

These metrics are all system metrics. "Application metrics" are manually added by the customer in their own code, so they would have to use the extras_metrics option

This particular metric is not documented and this is why I added both versions so we are already ready when a patch will be out

They aren't all documented on that page though? I do not see ray_internal_num_infeasible_scheduling_classes or ray_grpc_server_req_process_time_ms for example, so I wasn't sure if there's a separate page for some of these metrics.

Most of the metrics are not documented. I had to take a look at the code to get them 😭

Ah sad to hear 😢

ray/manifest.json

ray/assets/service_checks.json

FlorentClarret · 2023-09-06T07:41:48Z

@FlorentClarret This does not need Documentation review yet correct?

@estherk15 Yes, I will open a PR dedicated to the documentation update to make it easier on our side if this sounds good to you.

ofek

ray/tests/docker/serve_tasks/__init__.py needs a license header

🥇

ofek · 2023-09-06T12:02:05Z

ray/pyproject.toml

+[build-system]
+requires = [
+    "hatchling>=0.13.0",
+    "setuptools>=66; python_version > '3.0'",


Only required for editable installations on Python 2

Suggested change

"setuptools>=66; python_version > '3.0'",

Does it mean we can drop it from the templates? #15766

ray/tests/common.py

ray/tests/conftest.py

ofek · 2023-09-06T12:13:03Z

ray/tests/docker/docker-compose.yaml

+    command:
+      - /bin/bash
+      - -c
+      - |


Ha nice I never thought to use multi-line strings to send to bash before

ray/manifest.json

ray/tests/test_e2e.py

ray/tests/test_unit.py

ray/tests/test_e2e.py

ray/tests/common.py

yzhan289

LGTM!

yzhan289

LGTM again!

bgoldberg122 · 2023-09-21T16:38:38Z

@FlorentClarret Not sure if you're aware, but it looks like the metrics metadata sync task is failing from this PR: see logs.
I think you need to change percentage to percent in the unit_name for ray.node.gpus_utilization

ghost added documentation dev/testing dev/tooling integration/ray labels Aug 1, 2023

FlorentClarret added the changelog/Added label Aug 1, 2023

FlorentClarret force-pushed the florentclarret/ray/new_integration branch from ec7b399 to 45d0ef9 Compare August 1, 2023 08:53

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 2 times, most recently from 721223f to 3cfefce Compare August 14, 2023 14:05

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 5 times, most recently from ce5119f to fb1f75c Compare August 18, 2023 12:59

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 2 times, most recently from 71fd9df to 59decb6 Compare September 4, 2023 12:11

FlorentClarret marked this pull request as ready for review September 4, 2023 12:12

FlorentClarret requested review from a team as code owners September 4, 2023 12:12

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 2 times, most recently from 5397704 to c94210f Compare September 5, 2023 09:48

estherk15 reviewed Sep 5, 2023

View reviewed changes

ray/README.md Show resolved Hide resolved

yzhan289 reviewed Sep 5, 2023

View reviewed changes

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 2 times, most recently from bd712ab to 7fc36d6 Compare September 6, 2023 07:59

ofek requested changes Sep 6, 2023

View reviewed changes

FlorentClarret force-pushed the florentclarret/ray/new_integration branch from f11756b to 7add4e1 Compare September 6, 2023 12:29

FlorentClarret mentioned this pull request Sep 6, 2023

Remove setuptools from the build-system for new integrations #15766

Merged

3 tasks

iliakur reviewed Sep 7, 2023

View reviewed changes

ray/manifest.json Show resolved Hide resolved

ray/tests/test_e2e.py Outdated Show resolved Hide resolved

ray/tests/test_unit.py Outdated Show resolved Hide resolved

ray/tests/test_e2e.py Outdated Show resolved Hide resolved

ray/tests/common.py Show resolved Hide resolved

FlorentClarret force-pushed the florentclarret/ray/new_integration branch 3 times, most recently from 76a2cb8 to bdf9b76 Compare September 7, 2023 11:06

Add the Ray integration

4cda273

FlorentClarret force-pushed the florentclarret/ray/new_integration branch from bdf9b76 to 4cda273 Compare September 8, 2023 09:36

yzhan289 approved these changes Sep 11, 2023

View reviewed changes

FlorentClarret requested review from yzhan289, ofek and iliakur September 11, 2023 13:15

yzhan289 approved these changes Sep 12, 2023

View reviewed changes

FlorentClarret merged commit 8749d34 into master Sep 12, 2023

FlorentClarret deleted the florentclarret/ray/new_integration branch September 12, 2023 11:13

github-actions bot pushed a commit that referenced this pull request Sep 12, 2023

Add the Ray integration (#15441) 8749d34

8ecd244

FlorentClarret mentioned this pull request Oct 9, 2023

Add the log section in the configuration file #15978

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the Ray integration #15441

Add the Ray integration #15441

FlorentClarret commented Aug 1, 2023 •

edited

Loading

github-actions bot commented Aug 1, 2023 •

edited

Loading

codecov bot commented Aug 1, 2023 •

edited

Loading

github-actions bot commented Aug 3, 2023

github-actions bot commented Aug 4, 2023

github-actions bot commented Sep 5, 2023

estherk15 left a comment

yzhan289 Sep 5, 2023

FlorentClarret Sep 6, 2023 •

edited

Loading

yzhan289 Sep 11, 2023

FlorentClarret Sep 11, 2023

yzhan289 Sep 11, 2023

FlorentClarret commented Sep 6, 2023

ofek left a comment

ofek Sep 6, 2023

FlorentClarret Sep 6, 2023

ofek Sep 6, 2023

yzhan289 left a comment

yzhan289 left a comment

bgoldberg122 commented Sep 21, 2023 •

edited

Loading

Add the Ray integration #15441

Add the Ray integration #15441

Conversation

FlorentClarret commented Aug 1, 2023 • edited Loading

What does this PR do?

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

github-actions bot commented Aug 1, 2023 • edited Loading

Test Results

codecov bot commented Aug 1, 2023 • edited Loading

Codecov Report

github-actions bot commented Aug 3, 2023

github-actions bot commented Aug 4, 2023

github-actions bot commented Sep 5, 2023

estherk15 left a comment

Choose a reason for hiding this comment

yzhan289 Sep 5, 2023

Choose a reason for hiding this comment

FlorentClarret Sep 6, 2023 • edited Loading

Choose a reason for hiding this comment

yzhan289 Sep 11, 2023

Choose a reason for hiding this comment

FlorentClarret Sep 11, 2023

Choose a reason for hiding this comment

yzhan289 Sep 11, 2023

Choose a reason for hiding this comment

FlorentClarret commented Sep 6, 2023

ofek left a comment

Choose a reason for hiding this comment

ofek Sep 6, 2023

Choose a reason for hiding this comment

FlorentClarret Sep 6, 2023

Choose a reason for hiding this comment

ofek Sep 6, 2023

Choose a reason for hiding this comment

yzhan289 left a comment

Choose a reason for hiding this comment

yzhan289 left a comment

Choose a reason for hiding this comment

bgoldberg122 commented Sep 21, 2023 • edited Loading

FlorentClarret commented Aug 1, 2023 •

edited

Loading

github-actions bot commented Aug 1, 2023 •

edited

Loading

codecov bot commented Aug 1, 2023 •

edited

Loading

FlorentClarret Sep 6, 2023 •

edited

Loading

bgoldberg122 commented Sep 21, 2023 •

edited

Loading