Skip to content

Bump timeout in integration tests to 10 minutes - #3146

Open
eapolinario wants to merge 1 commit into
masterfrom
bump-timeout-to-10-min-in-integration-test
Open

eapolinario wants to merge 1 commit into
masterfrom
bump-timeout-to-10-min-in-integration-test

Conversation

@eapolinario

@eapolinario eapolinario commented Feb 19, 2025 •

Copy link
Copy Markdown
Collaborator

Tracking issue

NA

Why are the changes needed?

We're seeing some of the tests in the integration test suite fail semi-frequently. After investigating with https://github.com/tmate-io/tmate here I confirmed that the tasks end executing successfully, but the flyteremote call only waits for 5 minutes.

What changes were proposed in this pull request?

Set a unified timeout of 10 min per execution in flyteremote integration tests.

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

PR extends integration test timeout from 5 to 10 minutes by introducing TIMEOUT_IN_MIN constant and updating all remote.wait() calls. This change addresses semi-frequent test failures and ensures consistent timeout behavior across the test suite.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 1

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
@flyte-bot

flyte-bot commented Feb 19, 2025 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #a4599c

Actionable Suggestions - 1
  • tests/flytekit/integration/remote/test_remote.py - 1
Review Details
  • Files reviewed - 1 · Commit Range: c0dcee2..c0dcee2
    • tests/flytekit/integration/remote/test_remote.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@flyte-bot

Copy link
Copy Markdown
Contributor

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Testing - Increased Test Execution Timeout

- test_remote.py - Increased timeout from 5 to 10 minutes for remote execution tests to prevent premature timeouts

Comment thread tests/flytekit/integration/remote/test_remote.py
@codecov

codecov Bot commented Feb 19, 2025 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.96%. Comparing base (80ddcda) to head (c0dcee2).
⚠️ Report is 194 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (80ddcda) and HEAD (c0dcee2). Click for more details.

HEAD has 24 uploads less than BASE
Flag BASE (80ddcda) HEAD (c0dcee2)
25 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #3146       +/-   ##
===========================================
- Coverage   92.24%   51.96%   -40.29%     
===========================================
  Files         118      206       +88     
  Lines        4991    21851    +16860     
  Branches        0     2837     +2837     
===========================================
+ Hits         4604    11355     +6751     
- Misses        387     9854     +9467     
- Partials        0      642      +642     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pingsutw
pingsutw enabled auto-merge (squash) February 19, 2025 23:24
@eapolinario

eapolinario commented Feb 19, 2025 •

Copy link
Copy Markdown
Collaborator Author

This is weird. All task executions succeed, but the workflow execution still fails sometimes. It's weird that this only happens to the last 3 executions, kicked off by either test_attr_access_sd, test_sd_attr, and test_signal_approve_reject. I'm suspecting that it has to do with load, but still looking for the smoking gun.

@bochinski bochinski mentioned this pull request Sep 5, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants