[CI]Adds pytest timeout to CI #792
Conversation
Adds a timeout to pytest runs in CI to prevent jobs from hanging indefinitely. This also adds `pytest-timeout` to the test requirements.
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds the pytest-timeout dependency and updates two CI workflows to pass a per-test timeout flag ( Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant DevOps as CI Runner
participant Pytest as pytest
note right of DevOps #DDEBF7: workflow job invokes test command
DevOps->>Pytest: Run `pytest ... --durations=0 --timeout=3600`
alt test completes within timeout
Pytest-->>DevOps: Exit 0 (success)
else timeout exceeded
Pytest-->>DevOps: Timeout (non-zero exit)
end
note left of Pytest #F7F7F7: pytest-timeout enforces per-test cap
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @Cunxiao2002, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request enhances the robustness of the CI pipeline by introducing a timeout mechanism for pytest test runs. This change ensures that CI jobs complete within a reasonable timeframe, preventing indefinite hangs and providing more reliable feedback on test execution status.
Highlights
- Pytest Timeout in CI: Implemented a timeout for pytest runs within the Continuous Integration (CI) pipeline to prevent jobs from hanging indefinitely.
- New Dependency Added: Added the
pytest-timeoutpackage to therequirements-test.txtfile, making it available for test environments.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
👋 Hi! Thank you for contributing to the TileLang project. Please remember to run We appreciate you taking this step! Our team will review your contribution, and we look forward to your awesome work! 🚀 |
There was a problem hiding this comment.
Code Review
This pull request correctly adds pytest-timeout to the test dependencies. However, it's missing the crucial step of configuring the timeout value. As a result, the plugin won't be active, and the goal of preventing hanging CI jobs is not met. I've left a specific comment with suggestions on how to configure the timeout.
| pytest>=6.2.4 | ||
| pytest_xdist>=2.2.1 | ||
| pytest-durations | ||
| pytest-timeout |
There was a problem hiding this comment.
Adding pytest-timeout is the first step, but the timeout itself is not configured anywhere in this PR. Without configuration, the plugin will not be active, and tests can still hang indefinitely. To fix this, you need to set a timeout value. You can do this, for example, by adding timeout = 600 (for a 10-minute timeout) under the [pytest] section of a pytest.ini file, or by adding the --timeout=600 flag to your pytest command in your CI scripts.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
.github/workflows/ci.yml (1)
121-121: Good addition; also cover the examples test run.--timeout=200 is fine. To ensure consistency, apply the same flag to the earlier “Run examples” pytest invocation so hanging example tests don’t slip through.
You can update that step’s command to:
python -m pytest -n 4 **/test*.py -v -r fE --durations=0 --timeout=200Optionally centralize via pytest.ini/pyproject.toml addopts (e.g., addopts = "-n 4 -v -r fE --durations=0 --timeout=200") to avoid divergence across workflows.
.github/workflows/amd_ci.yml (1)
118-118: Add a trailing newline.YAML linter flags “no new line character at the end of file”.
requirements-test.txt (1)
16-16: Dependency addition looks good; consider version pinning.Adding pytest-timeout unblocks the new flag. Consider pinning (exact or minimum) aligned with your pytest/xdist versions to reduce resolver drift in CI.
If helpful, I can propose a pin after checking what versions you currently resolve in CI.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
.github/workflows/amd_ci.yml(1 hunks).github/workflows/ci.yml(1 hunks)requirements-test.txt(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
.github/workflows/amd_ci.yml
[error] 118-118: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: bot-task
.github/workflows/amd_ci.yml
Outdated
| cd testing/python/amd | ||
| unset PYTHONPATH | ||
| python -m pytest -v test_tilelang_test_amd.py --durations=0 | ||
| python -m pytest -v test_tilelang_test_amd.py --durations=0 --timeout=200 |
There was a problem hiding this comment.
AMD job will fail without pytest-timeout installed.
This adds --timeout=200, but the AMD venv installs from requirements-rocm.txt (not requirements-test.txt), which doesn’t include pytest-timeout. Pytest will error with “unrecognized arguments: --timeout=200”.
Please ensure the plugin is present in the AMD environment. For example, in the “Ensure venv (local & persistent)” step after installing requirements-rocm.txt:
PIP_NO_BUILD_ISOLATION=1 pip install -r requirements-rocm.txt
pip install pytest-timeout
Or add pytest-timeout to requirements-rocm.txt.
🧰 Tools
🪛 YAMLlint (1.37.1)
[error] 118-118: no new line character at the end of file
(new-line-at-end-of-file)
🤖 Prompt for AI Agents
.github/workflows/amd_ci.yml around line 118: the AMD CI runs pytest with
--timeout=200 but the AMD virtualenv installs requirements-rocm.txt which
doesn’t include the pytest-timeout plugin, causing pytest to error with
"unrecognized arguments: --timeout=200"; fix by ensuring pytest-timeout is
installed into the AMD venv—either add a pip install pytest-timeout step
immediately after installing requirements-rocm.txt in the "Ensure venv (local &
persistent)" step, or add pytest-timeout to requirements-rocm.txt so the plugin
is present when pytest runs.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
.github/workflows/amd_ci.yml (1)
118-118: AMD pytest will error unless pytest-timeout is installed in the ROCm venvAMD job installs only requirements-rocm.txt; without adding pytest-timeout, pytest will fail with “unrecognized arguments: --timeout=3600”. Either add pytest-timeout to requirements-rocm.txt (preferred) or explicitly install it in the AMD venv (cover both reuse and recreate paths).
Option A (preferred): add
pytest-timeouttorequirements-rocm.txt.Option B: explicitly install in workflow. Minimal patch:
@@ if [[ -f "$MARKER" ]] && [[ -f "${{ runner.tool_cache }}/${{ env.VENV_DIR }}/bin/activate" ]]; then echo "venv exists and hash matches – reuse it" else @@ if [[ -f requirements-rocm.txt ]]; then pip install --pre torch torchvision torchaudio --index-url ${{ env.PYTORCH_INDEX_URL }} PIP_NO_BUILD_ISOLATION=1 pip install -r requirements-rocm.txt + pip install pytest-timeout fi @@ fi + + - name: Ensure pytest-timeout plugin (idempotent) + run: | + source "${{ runner.tool_cache }}/${{ env.VENV_DIR }}/bin/activate" + python - <<'PY' +import sys +try: + import pytest_timeout # noqa: F401 + print("pytest-timeout present") +except Exception: + sys.exit("pytest-timeout missing") +PY
🧹 Nitpick comments (1)
.github/workflows/amd_ci.yml (1)
118-118: Add newline at EOF to satisfy yamllintFix the “no new line character at the end of file” warning.
- python -m pytest -v test_tilelang_test_amd.py --durations=0 --timeout=3600 + python -m pytest -v test_tilelang_test_amd.py --durations=0 --timeout=3600 +
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/amd_ci.yml(1 hunks).github/workflows/ci.yml(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/ci.yml
🧰 Additional context used
🪛 YAMLlint (1.37.1)
.github/workflows/amd_ci.yml
[error] 118-118: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: format-check
- GitHub Check: bot-task
🔇 Additional comments (1)
.github/workflows/amd_ci.yml (1)
118-118: Timeout values consistent across workflows
Bothci.ymlandamd_ci.ymlset--timeout=3600per test; no discrepancies found. Proceed if 3600 s is the intended timeout.
2 participants
Adds a timeout to pytest runs in CI to prevent jobs from hanging indefinitely. This also adds
pytest-timeoutto the test requirements.Summary by CodeRabbit
Tests
Chores
Notes