diff --git a/docs/testing/web_platform_tests.md b/docs/testing/web_platform_tests.md index 6457a661fcd2d9..83e4df9eb3d486 100644 --- a/docs/testing/web_platform_tests.md +++ b/docs/testing/web_platform_tests.md @@ -249,6 +249,9 @@ Same as Blink web tests, you can use One thing to note is that glob patterns for WPT tests are not yet supported. +See [Running WPT tests in Content Shell](web_tests_in_content_shell.md#Running-WPT-Tests-in-Content-Shell) +for debugging etc. + ## Reviewing tests Anyone who can review code and tests in Chromium can also review changes in diff --git a/docs/testing/web_tests.md b/docs/testing/web_tests.md index e09fdd2a4526c4..850a26c2bfca90 100644 --- a/docs/testing/web_tests.md +++ b/docs/testing/web_tests.md @@ -104,31 +104,39 @@ Example: To run the web tests with a debug build of `content_shell`, but only test the SVG tests and run pixel tests, you would run: ```bash -python third_party/blink/tools/run_web_tests.py -t Default svg +[python] third_party/blink/tools/run_web_tests.py -t Default svg ``` *** As a final quick-but-less-robust alternative, you can also just use the -content_shell executable to run specific tests by using (for Windows): +content_shell executable to run specific tests by using (example on Windows): ```bash -out/Default/content_shell.exe --run-web-tests --no-sandbox full_test_source_path +out/Default/content_shell.exe --run-web-tests || ``` as in: ```bash -out/Default/content_shell.exe --run-web-tests --no-sandbox \ +out/Default/content_shell.exe --run-web-tests \ c:/chrome/src/third_party/blink/web_tests/fast/forms/001.html ``` +or + +```bash +out/Default/content_shell.exe --run-web-tests fast/forms/001.html +``` but this requires a manual diff against expected results, because the shell -doesn't do it for you. +doesn't do it for you. It also just dumps the text result only (as the dump of +pixels and audio binary data is not human readable). +See [Running Web Tests Using the Content Shell](web_tests_in_content_shell.md] +for more details of running `content_shell`. To see a complete list of arguments supported, run: ```bash -python run_web_tests.py --help +python third_party/blink/tools/run_web_tests.py --help ``` *** note diff --git a/docs/testing/web_tests_in_content_shell.md b/docs/testing/web_tests_in_content_shell.md index c8c0cd041b5e95..aa99754609af0c 100644 --- a/docs/testing/web_tests_in_content_shell.md +++ b/docs/testing/web_tests_in_content_shell.md @@ -1,55 +1,151 @@ # Running web tests using the content shell -## Basic usage - -Web tests can be run with `content_shell`. To just dump the render tree, use -the `--run-web-tests` flag: - -```bash -out/Default/content_shell --run-web-tests foo.html -``` - -### Compiling +## Compiling If you want to run web tests, [build the target `blink_tests`](web_tests.md); this includes all the other binaries required to run the tests. -### Running +## Running + +### Using `run_web_tests.py` You can run web tests using `run_web_tests.py` (in `src/third_party/blink/tools`). ```bash -third_party/blink/tools/run_web_tests.py storage/indexeddb +python third_party/blink/tools/run_web_tests.py storage/indexeddb ``` +To see a complete list of arguments supported, run with `--help`. + +***promo +You can add `/third_party/blink/tools` into `PATH` so that you can +run it from anywhere without the full path. +*** + +### Run Web Tests Directly with Content Shell -or execute the shell directly: +In some cases (e.g. for debugging), you can run web tests directly with +Content Shell executable, with the `--run-web-tests` flag: ```bash -out/Default/content_shell --remote-debugging-port=9222 +out/Default/content_shell --run-web-tests || +``` + +`` is relative to the [web_tests](../../third_party/blink/web_tests) +directory, regardless of the current directory. + +For example: + +```bash +out/Default/content_shell --run-web-tests fast/forms/001.html +``` +or + +```bash +out/Default/content_shell --run-web-tests \ + /home/user/chrome/src/third_party/blink/web_tests/fast/forms/001.html +``` +or + +```bash +out/Default/content_shell --run-web-tests ~/test/temp-test.html ``` -This allows you see how your changes look in Chromium, and even connect with -devtools (by going to http://127.0.0.1:9222 from another window) to inspect your -freshly compiled Blink. +By default, it dumps the text result only (as the dump of pixels and audio +binary data is not human readable). This can meet the requirement of +most debugging requirements. + +In rare cases, to run `content_shell` in the exact same way as +`run_web_tests.py` runs it, you need to run it in the +[protocol mode](../../content_shell/browser/web_test/test_info_extractor.h). *** note On the Mac, use `Content Shell.app`, not `content_shell`. ```bash -out/Default/Content\ Shell.app/Contents/MacOS/Content\ Shell --remote-debugging-port=9222 +out/Default/Content\ Shell.app/Contents/MacOS/Content\ Shell ... ``` +On Windows, use `content_shell.exe`. *** -### Debugging Renderer Crashes +#### Running HTTP Tests in Content Shell + +HTTP tests reside under [web_tests/http/tests](../../third_party/blink/web_tests/http/tests). +You need to start a web server first: + +```bash +python third_party/blink/tools/run_blink_httpd.py +``` +Then run the test with a localhost URL: + +```bash +out/Default/content_shell --run-web-tests http://localhost:8000/ +``` + +#### Running WPT Tests in Content Shell + +Similar to HTTP tests, many WPT (a.k.a. web-platform-tests under +[web_tests/external/wpt](../../third_party/blink/web_tests/external/wpt) directory) +tests require some setup before running in Content Shell: + +```bash +python third_party/blink/tools/run_blink_wptserve.py +``` +Then run the test: + +```bash +out/Default/content_shell --run-web-tests http://localhost:8001/ +``` +See [the blink-dev discussion](https://groups.google.com/a/chromium.org/forum/?utm_medium=email&utm_source=footer#!msg/blink-dev/iP_9ok1K1UM/F_qCmk5kDAAJ) for more details. + +### As a simple browser + +You can run the shell directly as a simple browser: + +```bash +out/Default/content_shell +``` + +This allows you see how your changes look in Chromium. You can inspect the page +by right clicking and selecting 'Inspect Element'. + +You can also use `--remote-debugging-port` + +```bash +out/Default/content_shell --remote-debugging-port=9222 +``` +and open `http://localhost:9222` from another browser to inspect the page. +This is useful when you don't want DevTools to run in the same Content Shell, +e.g. when you are logging a lot and don't want the log from DevTools +or when DevTools is unstable in the current revision due to some bugs. + +## Debugging + +### `--single-process` + +The command line switch `--single-process` is useful for starting +content_shell in gdb. In most cases, `--single-process` is good for debugging +except when you want to debug the multiple process behavior or when we have +some bug breaking `--single-process` in particular cases. + +### Web tests + +See [Run Web Tests Directly with Content Shell](#Run-Web-Tests-Directly-with-Content-Shell). +In most cases you don't need `--single-process` because `content_shell` is +in single process mode when running most web tests. + +See [DevTools frontend](../../third_party/blink/renderer/devtools/readme.md#basics) +for the commands that are useful for debugging devtools web tests. + +### In The Default Multiple Process Mode -To debug a renderer crash, ask Content Shell to wait for you to attach a -debugger once it spawns a renderer process by adding the -`--renderer-startup-dialog` flag: +In rare cases, you need to debug Content Shell in multiple process mode. +You can ask Content Shell to wait for you to attach a debugger once it spawns a +renderer process by adding the `--renderer-startup-dialog` flag: ```bash -out/Default/content_shell --renderer-startup-dialog +out/Default/content_shell --renderer-startup-dialog --no-sandbox ``` Debugging workers and other subprocesses is simpler with