Adds neptune plugin for experiment tracking #2686
Conversation
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
thomasjpfan
requested review from
wild-endeavor,
kumare3,
eapolinario,
pingsutw,
cosmicBboy,
samhita-alla and
Future-Outlier
as code owners
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2686 +/- ##
===========================================
- Coverage 78.69% 50.73% -27.97%
===========================================
Files 187 192 +5
Lines 19257 19425 +168
Branches 4029 3805 -224
===========================================
- Hits 15155 9855 -5300
- Misses 3404 9065 +5661
+ Partials 698 505 -193 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
| def neptune_init_run( | ||
| project: str, | ||
| secret: Union[Secret, Callable], | ||
| host: str = "https://app.neptune.ai", |
There was a problem hiding this comment.
Is there any particular reason host has to be passed explicitly?
This can be extracted from the NEPTUNE_API_TOKEN
json.loads(base64.b64decode(os.getenv("NEPTUNE_API_TOKEN")).decode("utf-8"))["api_url"]
There was a problem hiding this comment.
In a IRL conversation with @SiddhantSadangi, we discussed how this is a limitation of Flyte and dynamic log links.
Context: The Flyte UI needs to know the host URL during registration time. There are two options:
- Have the host passed in explicitly (this PR)
- Force users to have
NEPTUNE_API_TOKENdefined during registration time.
…ne_pr Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
| assert run_mock["Flyte Execution ID"] == host_name | ||
| assert run_mock["Flyte Execution URL"] == execution_url |
There was a problem hiding this comment.
need to update these assertions.
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
…ne_pr Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Why are the changes needed?
This PR adds a Neptune.ai plugin for experiment tracking. This is similar to the Comet and Weights & Biases plugin.
What changes were proposed in this pull request?
This PR adds a
neptune_init_rundecorator that starts a Neptune Run. Given the Neptune integration with third party libraries passes therunobject around, this PR injects theneptune_runobject intocurrent_context().How was this patch tested?
This PR includes unit tests. I also ran the example in flyteorg/flytesnacks#1723:
Docs link
[
](flyteorg/flytesnacks#1723)