Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: disk-stalled/wal-failover/among-stores failed #131553

Open
cockroach-teamcity opened this issue Sep 28, 2024 · 1 comment
Open

roachtest: disk-stalled/wal-failover/among-stores failed #131553

cockroach-teamcity opened this issue Sep 28, 2024 · 1 comment
Assignees
@sumeerbhola
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. branch-release-24.2.3-rc C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. P-3 Issues/test failures with no fix SLA T-storage Storage Team

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Sep 28, 2024 •
edited by cockroach-jira-scripts
Loading

roachtest.disk-stalled/wal-failover/among-stores failed with artifacts on release-24.2.3-rc @ 217b43e69da61e9712d73e3e19fd5076fbdb26df:

(disk_stall.go:172).runDiskStalledWALFailover: unexpectedly high p99.99 latency 1.198619286s at 2024-09-28T08:30:00Z
(cluster.go:2418).Run: context canceled
test artifacts and logs in: /artifacts/disk-stalled/wal-failover/among-stores/run_1

Parameters:

  • ROACHTEST_arch=amd64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=16
  • ROACHTEST_encrypted=true
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=true
  • ROACHTEST_metamorphicBuild=false
  • ROACHTEST_ssd=2
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

/cc @cockroachdb/storage

This test on roachdash | Improve this report!

Jira issue: CRDB-42596
@cockroach-teamcity cockroach-teamcity added branch-release-24.2.3-rc C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-storage Storage Team labels Sep 28, 2024
@blathers-crl blathers-crl bot added the A-storage Relating to our storage engine (Pebble) on-disk storage. label Sep 28, 2024
@github-project-automation github-project-automation bot moved this to Incoming in Storage Sep 28, 2024
@sumeerbhola sumeerbhola removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Oct 1, 2024
@sumeerbhola
Copy link
Collaborator

v24.2 dup of #129922

@sumeerbhola sumeerbhola self-assigned this Oct 1, 2024
@sumeerbhola sumeerbhola added P-3 Issues/test failures with no fix SLA and removed O-roachtest labels Oct 1, 2024
@nicktrav nicktrav moved this from Incoming to Tests (failures, skipped, flakes) in Storage Oct 1, 2024
This was referenced Oct 18, 2024
Open
Closed
Closed
Closed
@cockroach-teamcity cockroach-teamcity mentioned this issue Oct 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sumeerbhola sumeerbhola

Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. branch-release-24.2.3-rc C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. P-3 Issues/test failures with no fix SLA T-storage Storage Team
Projects
Storage
Status: Tests (failures, skipped, flakes)
Milestone
No milestone
Development

No branches or pull requests
3 participants
@itsbilal @cockroach-teamcity @sumeerbhola