[TT-3972] Mark/skip problematic tests, resolve some root issues #4035
Conversation
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
jeffy-mathew
force-pushed
the
fix/TT-3972/flaky-tests-hotfixes
branch
from
e46c668 to
9abaaf8
Compare
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
jeffy-mathew
force-pushed
the
fix/TT-3972/flaky-tests-hotfixes
branch
from
042e554 to
f403234
Compare
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
titpetric
force-pushed
the
fix/TT-3972/flaky-tests-hotfixes
branch
from
58b304a to
ea212d9
Compare
|
API tests result: success ✅ |
titpetric
force-pushed
the
fix/TT-3972/flaky-tests-hotfixes
branch
from
ea212d9 to
85bdb50
Compare
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
API tests result: success ✅ |
| echo "$@" >&2 | ||
| exit 1 | ||
| } | ||
|
|
There was a problem hiding this comment.
Redundant redirect of stdout output into stderr
| if [[ -z "$OPTS" ]]; then | ||
| OPTS="-race -count=1 -failfast -v" | ||
| fi | ||
|
|
There was a problem hiding this comment.
Ability to override test flags with ci-tests.sh arguments
- disable the DNS mock (unsafe, TT-5112) - disable emptying redis from tests (race conditions as redis is a singleton with reuse) - some cleanup for readibility, s.Gw to gw (scope) - remove useless s.gwMu mock, cleaner test cancellation
titpetric
force-pushed
the
fix/TT-3972/flaky-tests-hotfixes
branch
from
eba26e0 to
99bd4cf
Compare
|
API tests result: success ✅ |
|
API tests result: success ✅ |
|
SonarCloud Quality Gate failed. |
|
API tests result: success ✅ |
|
API tests result: success ✅
Since we test packages individually, we can avoid DNS Mock data races by not resetting the DefaultResolver to it's original value.
Co-authored-by: Jeff jeffy.mathew100@gmail.com |
|
@titpetric should this be merged to |
| m.NumGC = rtm.NumGC | ||
|
|
||
| // Just encode to json and print | ||
| b, _ := json.Marshal(m) |
There was a problem hiding this comment.
since its for stderr can we indent the json b, _ := json.MarshalIndent(&m, "", " ")
| // Number of goroutines | ||
| m.NumGoroutine = runtime.NumGoroutine() | ||
| if m.NumGoroutine > 4000 { | ||
| go func() { |
There was a problem hiding this comment.
this goroutine that handles SIGQUIT, might be dispatched multiples times(every timer + G > 4000) and I think we need it once, and it should return as long as the function returns
| var rtm runtime.MemStats | ||
| var interval = time.Duration(duration) * time.Second | ||
| for { | ||
| <-time.After(interval) |
There was a problem hiding this comment.
we should add a context arguments, that will receive a signal to stop this function(stop monitoring), just in case. so we could basically select between timer and context deadline
| // Number of goroutines | ||
| m.NumGoroutine = runtime.NumGoroutine() | ||
| if m.NumGoroutine > 4000 { | ||
| go func() { |
There was a problem hiding this comment.
let's remove this nested function
|
API tests result: skipped 🚫
Since we test packages individually, we can avoid DNS Mock data races by not resetting the DefaultResolver to it's original value.
Co-authored-by: Jeff jeffy.mathew100@gmail.com |
* Improve CI tests - Run go vet over whole repo, once - Avoid bash no-ops in favor of 'set -e' - Remove no-op stdout redirects - Fail fast with -failfast flag - No caching with -count=1 flag * Add testing utilities to mark Flaky and Racy tests with * Allow configuring python version for tests * Add debug output to print which python version we seek * Remove dead code, verified with @sredny * Mark TT-5220 flaky test * Mark TT-5222 flaky test * Mark TT-5223 flaky test * Resolve TT-5000, add cancellation ctx to redis to fix some racy tests * Mark TT-5224 as racy * Mark TT-5225 as racy * Add http client/transport utilities to avoid net.DefaultResolver * Fix: TT-5112 only replace defaultResolver once Since we test packages individually, we can avoid DNS Mock data races by not resetting the DefaultResolver to it's original value. While the ultimate goal is removal, we need to add replacements, which mostly add up to a Dialer function. * Use NewClientLocal where we know we have flaky tests caused by the DNS Mock * Tentative improvement for TT-5226, marking flaky tests * Fix TT-5227 by having a dirty dns mock shutdown * Mark TT-5228 as flaky * add test util function GetPythonVersion get python version from env in TestValueExtractorHeaderSource [changelog] added: test util function GetPythonVersion. * Mark TT-3973 as flaky * mark TT-4069 as flaky * fix flaky TT-4788 * Mark TT-5233 as racy * mark TestHMACAuthSessionSHA512Pass as flaky TT-3973 * Allow passing options to bin/ci-tests.sh for go test * Print notice test can be skipped with CI env * Remove unnecessary go background key deletion, race over b.store * Upgrade redis/v8 to get patches for all the data races * Mark TT-5236 as flaky * Fix goroutine leak in rpcReloadLoop * Fix: panic in tests if we can't connect to redis * Testing: add a console monitor for runtime stats, goroutines * Add flaky test to verify how many goroutines we leak in a test case * Remove running go-vet, it's run twice as part of golangci-lint * Mark TT-5249 as flaky * Mark flaky tests TT-5250, TT-5251 * Remove unused *gateway.cancelFn * Mark TT-5254 flaky test * Mark TT-5257 as flaky, note frequency * Mark TT-5258 as flaky * Mark TT-5259 as flaky * Mark TT-5260 flaky, improve tests fragility * Fix: Improve flakyness in Gateway tests * Mark TT-5112, 5261, 5261 as flaky * Remove unnecessary time.Sleep values from tests, tests still pass * Add enabling of pprof server with TEST_PPROF_ENABLE and TEST_PPROF_ADDR (tests only) * Improve TestAPISpec_StripListenPath flakyness, Mark TT-5255 as flaky * Mark TT-5263 as flaky * Mark flaky tests, remove SkipEmptyRedis * Mark TT-5264 as flaky * Test utilities improvements (signficant): - disable the DNS mock (unsafe, TT-5112) - disable emptying redis from tests (race conditions as redis is a singleton with reuse) - some cleanup for readibility, s.Gw to gw (scope) - remove useless s.gwMu mock, cleaner test cancellation * Fix leaking pubsub channels, decrease deprecated redis api surface to use channels * Fix golangci-lint reported issues in modified code * Fix resource leak with time.After in tests, analytics records flusher * Removed TestAnalytics, fixing incorrect rebase * Add comment to explain why func is empty, sonarcloud Co-authored-by: Jeff <jeffy.mathew100@gmail.com>
No description provided.