Open
Description
CI improvements (reduce time it takes for Github actions to run):
- DWDS tests should be sharded into groups (currently they take much longer than the other CI actions)
- Completed in Shard DWDS tests and fix CI test flakiness #1851
- Move tests with a large timeout limit (eg, the
devtools _test
at 5 minutes) to a cron job (tracked in Tests with long timeouts should be moved to a cron job #1846) - All tests should have a timeout set
Refactor TestContext
setup:
- Make
TestServer
more re-usable (described in Make DWDS test server more re-usable #1779) - Make it clearer what parameters should be passed to
TestContext
(eg, it is unclear the difference betweenpath
vspathToServe
). Also the default Dart entry file is inappend_body
, but the default HTML entry file is inhello_world
. - Currently a lot of the fields in
TestContext
arelate
, because they are only initialized afterTestContext.setUp()
is called. I think it would make more sense to have two separate classes for test setup:TestPackage
which is equivalent toTestContext
, andTestContext
which is equivalent toTestContext.setUp()
.
Other improvements:
- Callstack tests break if test
fixtures
are formatted (described in Call stack tests break ifmain.dart
infixtures/_testPackageSound
/fixtures/_testPackage
is formatted #1831)- Fixed in fix format breaking tests #2124
- All packages should have a
mono_pkg.yml
file, so that theanalyze
andformat
steps can be run against them during CI - Support Dart 3.0 breaking changes in tests (tracked in Support tests with breaking changes in Dart 3.0.0 #1818)
- Switch to using
puppeteer
for all integration tests (tracked in Switch to usingpuppeteer
instead ofWebDriver
for all integration tests #1847) -
dwds/test/fixtures
directory contains dill files (used for testing) along withutilities.dart
(test utilities) andcontext.dart
(test setup). This should be reorganized.