Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Web test reorganization #39984

Merged
merged 56 commits into from
Mar 28, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
c13cd89
Wrote felt_config file and parsing.
eyebrowsoffire Feb 10, 2023
b60b1b4
Some suite filtering and copy artifacts step refactor.
eyebrowsoffire Feb 14, 2023
8760d77
CompileBundleStep implemented.
eyebrowsoffire Feb 16, 2023
752547d
Suites running, but assets alongside tests are not being served.
eyebrowsoffire Feb 17, 2023
1bdd442
Reduce verbosity and fix serving assets from test directory.
eyebrowsoffire Feb 17, 2023
b6e520e
All filters except file filters are implemented.
eyebrowsoffire Feb 17, 2023
e26e6a3
File filter implemented.
eyebrowsoffire Feb 17, 2023
c2d464d
Move around tests themselves.
eyebrowsoffire Feb 17, 2023
3c9c37d
Test against canvaskit_chromium.
eyebrowsoffire Feb 17, 2023
a7906e8
Added and updated documentation.
eyebrowsoffire Feb 17, 2023
b0cfad9
Started some CI stuff.
eyebrowsoffire Mar 1, 2023
78859af
Add bringup:true to the new web engine step.
eyebrowsoffire Mar 1, 2023
2dd558d
Change felt parameters.
eyebrowsoffire Mar 1, 2023
35163cf
Remove old web engine builders.
eyebrowsoffire Mar 1, 2023
d721f10
Always hard code canvaskit variants for chrome and edge.
eyebrowsoffire Mar 2, 2023
ad81ea6
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 2, 2023
c48e279
improve some error messages
mdebbar Mar 2, 2023
83f67b8
support filtering by canvaskit variant
mdebbar Mar 2, 2023
c5f60d1
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 7, 2023
1687708
Change linux_web_engine to new felt commands.
eyebrowsoffire Mar 7, 2023
d6ec4e7
Skip the legacy build path for license checks.
eyebrowsoffire Mar 8, 2023
77e0119
Update lib/web_ui/dev/steps/compile_bundle_step.dart
eyebrowsoffire Mar 8, 2023
7fcc94b
Revert weird change to README.md
eyebrowsoffire Mar 8, 2023
8294f13
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 8, 2023
581023a
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 10, 2023
28808e7
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 10, 2023
55195f0
Separate web_tests build out.
eyebrowsoffire Mar 10, 2023
dbc98f9
Copy artifacts in a generator step.
eyebrowsoffire Mar 10, 2023
0c6bf80
Do copy artifacts as part of the test run.
eyebrowsoffire Mar 10, 2023
dd3c7b9
Allow testing against wasm_profile artifacts.
eyebrowsoffire Mar 11, 2023
38dd751
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 13, 2023
f390a96
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 16, 2023
db9c9e0
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 23, 2023
1071c87
Remove duplicated build step.
eyebrowsoffire Mar 23, 2023
5f5ff75
Add new fields for the v2 tests.
eyebrowsoffire Mar 23, 2023
0cc002e
Put the test invocation into `tasks`
eyebrowsoffire Mar 23, 2023
d5a9e7a
Name the test.
eyebrowsoffire Mar 23, 2023
d84c677
Try sharding compilation and tests.
eyebrowsoffire Mar 23, 2023
bb45d11
Fix bundle filtering.
eyebrowsoffire Mar 23, 2023
ee89022
Try making copy artifacts a subbuild.
eyebrowsoffire Mar 23, 2023
164483f
The main build should be `web_tests/artifacts`
eyebrowsoffire Mar 23, 2023
c25b4da
test_bundles, not test_bundle
eyebrowsoffire Mar 24, 2023
97f9459
Generate all builder json.
eyebrowsoffire Mar 24, 2023
a8bc5c4
Specify an empty tests list.
eyebrowsoffire Mar 24, 2023
442034d
Generate the builder json again.
eyebrowsoffire Mar 24, 2023
eb1e3ec
script, not scripts.
eyebrowsoffire Mar 24, 2023
d25ca9e
Get rid of extra array layer.
eyebrowsoffire Mar 24, 2023
aed3f1b
No need for empty tests array in build steps
eyebrowsoffire Mar 24, 2023
f47bfd3
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 24, 2023
2271295
It's `script` for tests, but `scripts` for generators.
eyebrowsoffire Mar 24, 2023
d4a7f5a
Mac, not MacOS
eyebrowsoffire Mar 24, 2023
6d5fa74
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 24, 2023
4ed8d43
Yegor's comments.
eyebrowsoffire Mar 27, 2023
d66c188
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 27, 2023
ba83669
Add back screenshot logging under verbose flag.
eyebrowsoffire Mar 27, 2023
9b61f3a
Merge branch 'main' into web_test_reorganization
eyebrowsoffire Mar 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Reduce verbosity and fix serving assets from test directory.
  • Loading branch information
eyebrowsoffire committed Mar 2, 2023
commit 1bdd4424fd16e81f5cd940adc4e34d00138ce4d4
22 changes: 18 additions & 4 deletions lib/web_ui/dev/steps/compile_bundle_step.dart
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ import '../utils.dart' show AnsiColors, FilePath, ProcessManager, cleanup, getBu
class CompileBundleStep implements PipelineStep {
CompileBundleStep({
required this.bundle,
required this.isVerbose,
this.testFiles,
});

final TestBundle bundle;
final bool isVerbose;
final Set<FilePath>? testFiles;

// Maximum number of concurrent compile processes to use.
Expand Down Expand Up @@ -66,12 +68,14 @@ class CompileBundleStep implements PipelineStep {
testSetDirectory,
outputBundleDirectory,
renderer: bundle.compileConfig.renderer,
isVerbose: isVerbose,
);
case Compiler.dart2wasm:
return Dart2WasmCompiler(
testSetDirectory,
outputBundleDirectory,
renderer: bundle.compileConfig.renderer,
isVerbose: isVerbose,
);
}
}
Expand Down Expand Up @@ -141,12 +145,16 @@ abstract class TestCompiler {
TestCompiler(
this.inputTestSetDirectory,
this.outputTestBundleDirectory,
{required this.renderer}
{
required this.renderer,
required this.isVerbose,
}
);

final io.Directory inputTestSetDirectory;
final io.Directory outputTestBundleDirectory;
final Renderer renderer;
final bool isVerbose;

Future<bool> compileTest(FilePath input);
}
Expand All @@ -155,7 +163,10 @@ class Dart2JSCompiler extends TestCompiler {
Dart2JSCompiler(
super.inputTestSetDirectory,
super.outputTestBundleDirectory,
{required super.renderer}
{
required super.renderer,
required super.isVerbose,
}
);

@override
Expand Down Expand Up @@ -199,7 +210,7 @@ class Dart2JSCompiler extends TestCompiler {
environment.dartExecutable,
arguments,
workingDirectory: inputTestSetDirectory.path,
evalOutput: true,
evalOutput: !isVerbose,
);
final int exitCode = await process.wait();
if (exitCode != 0) {
Expand All @@ -216,7 +227,10 @@ class Dart2WasmCompiler extends TestCompiler {
Dart2WasmCompiler(
super.inputTestSetDirectory,
super.outputTestBundleDirectory,
{required super.renderer}
{
required super.renderer,
required super.isVerbose,
}
);

@override
Expand Down
3 changes: 3 additions & 0 deletions lib/web_ui/dev/steps/run_suite_step.dart
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import '../utils.dart';
class RunSuiteStep implements PipelineStep {
RunSuiteStep(this.suite, {
required this.isDebug,
required this.isVerbose,
required this.doUpdateScreenshotGoldens,
required this.requireSkiaGold,
this.testFiles,
Expand All @@ -39,6 +40,7 @@ class RunSuiteStep implements PipelineStep {
final TestSuite suite;
final Set<FilePath>? testFiles;
final bool isDebug;
final bool isVerbose;
final bool doUpdateScreenshotGoldens;
final String? overridePathToCanvasKit;

Expand Down Expand Up @@ -92,6 +94,7 @@ class RunSuiteStep implements PipelineStep {
skiaClient: skiaClient,
overridePathToCanvasKit: overridePathToCanvasKit,
isWasm: isWasm,
isVerbose: isVerbose,
);
});

Expand Down
19 changes: 14 additions & 5 deletions lib/web_ui/dev/test_platform.dart
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ class BrowserPlatform extends PlatformPlugin {
required this.browserEnvironment,
required this.server,
required this.isDebug,
required this.isVerbose,
required this.doUpdateScreenshotGoldens,
required this.packageConfig,
required this.skiaClient,
Expand Down Expand Up @@ -123,6 +124,7 @@ class BrowserPlatform extends PlatformPlugin {
required SkiaGoldClient? skiaClient,
required String? overridePathToCanvasKit,
required bool isWasm,
required bool isVerbose,
}) async {
final shelf_io.IOServer server =
shelf_io.IOServer(await HttpMultiServer.loopback(0));
Expand All @@ -131,6 +133,7 @@ class BrowserPlatform extends PlatformPlugin {
browserEnvironment: browserEnvironment,
server: server,
isDebug: Configuration.current.pauseAfterLoad,
isVerbose: isVerbose,
doUpdateScreenshotGoldens: doUpdateScreenshotGoldens,
packageConfig: await loadPackageConfigUri((await Isolate.packageConfig)!),
skiaClient: skiaClient,
Expand All @@ -145,6 +148,8 @@ class BrowserPlatform extends PlatformPlugin {
/// breakpoints in the code.
final bool isDebug;

final bool isVerbose;

/// The underlying server.
final shelf.Server server;

Expand Down Expand Up @@ -255,7 +260,9 @@ class BrowserPlatform extends PlatformPlugin {
}

Future<shelf.Response> _fileNotFoundCatcher(shelf.Request request) async {
print('HTTP 404: ${request.url}');
if (isVerbose) {
print('HTTP 404: ${request.url}');
}
return shelf.Response.notFound('File not found');
}

Expand Down Expand Up @@ -386,10 +393,12 @@ class BrowserPlatform extends PlatformPlugin {
final String filename = requestData['filename'] as String;

if (!(await browserManager).supportsScreenshots) {
print(
'Skipping screenshot check for $filename. Current browser/OS '
'combination does not support screenshots.',
);
if (isVerbose) {
print(
'Skipping screenshot check for $filename. Current browser/OS '
'combination does not support screenshots.',
);
}
return shelf.Response.ok(json.encode('OK'));
}

Expand Down
6 changes: 5 additions & 1 deletion lib/web_ui/dev/test_runner.dart
Original file line number Diff line number Diff line change
Expand Up @@ -237,12 +237,16 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
CopyArtifactsStep(artifacts),
if (shouldCompile)
for (final TestBundle bundle in bundles)
CompileBundleStep(bundle: bundle),
CompileBundleStep(
bundle: bundle,
isVerbose: isVerbose
),
if (shouldRun)
for (final TestSuite suite in filteredSuites)
RunSuiteStep(
suite,
isDebug: isDebug,
isVerbose: isVerbose,
doUpdateScreenshotGoldens: doUpdateScreenshotGoldens,
requireSkiaGold: requireSkiaGold,
overridePathToCanvasKit: overridePathToCanvasKit,
Expand Down
2 changes: 1 addition & 1 deletion lib/web_ui/dev/utils.dart
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ Future<void> cleanup() async {
io.Directory getTestSetDirectory(TestSet testSet) {
return io.Directory(
path.join(
environment.webUiBuildDir.path,
environment.webUiTestDir.path,
testSet.directory,
)
);
Expand Down