Skip to content

Add TMPDIR to reducer and mapper enviroment #2248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Feb 27, 2024
Merged

Add TMPDIR to reducer and mapper enviroment #2248

merged 2 commits into from
Feb 27, 2024

Conversation

MrLolthe1st
Copy link
Collaborator

No description provided.

@MrLolthe1st MrLolthe1st requested a review from a team as a code owner February 26, 2024 14:52
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 14:54:14 UTC Pre-commit check for 3d851f3 has started.
⚪ 2024-02-26 14:54:16 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-26 14:56:51 UTC Build successful.
⚪ 2024-02-26 14:57:03 UTC Tests are running...
🔴 2024-02-26 16:23:17 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14847 14697 0 23 101 26

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 14:55:45 UTC Pre-commit check for 3d851f3 has started.
⚪ 2024-02-26 14:55:47 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-02-26 14:59:05 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 14:56:01 UTC Pre-commit check for 3d851f3 has started.
⚪ 2024-02-26 14:56:04 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-26 14:59:17 UTC Build successful.
⚪ 2024-02-26 14:59:30 UTC Tests are running...
🔴 2024-02-26 16:21:13 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
67948 57032 0 3 10891 22

vitstn
vitstn reviewed Feb 26, 2024
View reviewed changes
ydb/library/yql/providers/yt/gateway/native/yql_yt_spec.cpp Outdated
@@ -472,6 +472,8 @@ void FillSpec(NYT::TNode& spec,
spec["force_job_size_adjuster"] = *val;
}
}
spec["reducer"]["environment"]["TMPDIR"] = ".";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please avoid filling spec if reducer or mapper are absent

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MrLolthe1st see https://github.com/ydb-platform/ydb/blob/main/ydb/library/yql/providers/yt/gateway/native/yql_yt_spec.cpp#L367-L373, how to properly check mapper/reducer presence

rvu1024
rvu1024 requested changes Feb 26, 2024
View reviewed changes
ydb/library/yql/providers/yt/gateway/native/yql_yt_spec.cpp Outdated
@@ -472,6 +472,8 @@ void FillSpec(NYT::TNode& spec,
spec["force_job_size_adjuster"] = *val;
}
}
spec["reducer"]["environment"]["TMPDIR"] = ".";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MrLolthe1st see https://github.com/ydb-platform/ydb/blob/main/ydb/library/yql/providers/yt/gateway/native/yql_yt_spec.cpp#L367-L373, how to properly check mapper/reducer presence

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 16:48:12 UTC Pre-commit check for 302d3c1 has started.
⚪ 2024-02-26 16:48:13 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-26 16:50:35 UTC Build successful.
⚪ 2024-02-26 16:50:46 UTC Tests are running...
🔴 2024-02-26 18:19:10 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14847 14694 0 22 105 26

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 16:49:37 UTC Pre-commit check for 302d3c1 has started.
⚪ 2024-02-26 16:49:39 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-02-26 17:06:47 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 26, 2024 •
edited
Loading

⚪ 2024-02-26 16:49:42 UTC Pre-commit check for 302d3c1 has started.
⚪ 2024-02-26 16:49:44 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-26 16:52:52 UTC Build successful.
⚪ 2024-02-26 16:53:05 UTC Tests are running...
🔴 2024-02-26 18:06:36 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
67950 57035 0 5 10887 23

@MrLolthe1st MrLolthe1st merged commit 156670e into ydb-platform:main Feb 27, 2024
@shnikd shnikd mentioned this pull request Feb 27, 2024
Merged
@serbel324 serbel324 mentioned this pull request Mar 1, 2024
Merged
This was referenced Mar 4, 2024
Closed
Merged
This was referenced Mar 6, 2024
Merged
Open
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Mar 7, 2024
Merged
This was referenced Mar 8, 2024
Merged
Closed
This was referenced Mar 11, 2024
Merged
Merged
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Mar 13, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Mar 19, 2024
Merged
This was referenced Mar 20, 2024
Merged
Closed
Merged
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Mar 26, 2024
Merged
@shnikd shnikd mentioned this pull request Mar 26, 2024
Merged
@shnikd shnikd mentioned this pull request Apr 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vitstn vitstn vitstn approved these changes

@rvu1024 rvu1024 rvu1024 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@MrLolthe1st @vitstn @rvu1024