Document more topics about running/debugging Content Shell

- The protocol mode - Running HTTP web tests in Content Shell - Running WPT tests in Content Shell - Debugging with --single-process - Debugging web tests - Some corrections Change-Id: I1d23d0f8a75b9d597edab153e4ec0ea9cdb7cb67 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1541863 Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org> Reviewed-by: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#645002}
marcosholgado · Mar 27, 2019 · 0a37e9d · 0a37e9d
1 parent dd9e27c
commit 0a37e9d
Show file tree

Hide file tree

Showing 3 changed files with 136 additions and 29 deletions.
diff --git a/docs/testing/web_platform_tests.md b/docs/testing/web_platform_tests.md
@@ -249,6 +249,9 @@ Same as Blink web tests, you can use
 
 One thing to note is that glob patterns for WPT tests are not yet supported.
 
+See [Running WPT tests in Content Shell](web_tests_in_content_shell.md#Running-WPT-Tests-in-Content-Shell)
+for debugging etc.
+
 ## Reviewing tests
 
 Anyone who can review code and tests in Chromium can also review changes in

diff --git a/docs/testing/web_tests.md b/docs/testing/web_tests.md
@@ -104,31 +104,39 @@ Example: To run the web tests with a debug build of `content_shell`, but only
 test the SVG tests and run pixel tests, you would run:
 
 ```bash
-python third_party/blink/tools/run_web_tests.py -t Default svg
+[python] third_party/blink/tools/run_web_tests.py -t Default svg
 ```
 ***
 
 As a final quick-but-less-robust alternative, you can also just use the
-content_shell executable to run specific tests by using (for Windows):
+content_shell executable to run specific tests by using (example on Windows):
 
 ```bash
-out/Default/content_shell.exe --run-web-tests --no-sandbox full_test_source_path
+out/Default/content_shell.exe --run-web-tests <url>|<full_test_source_path>|<relative_test_path>
 ```
 
 as in:
 
 ```bash
-out/Default/content_shell.exe --run-web-tests --no-sandbox \
+out/Default/content_shell.exe --run-web-tests \
     c:/chrome/src/third_party/blink/web_tests/fast/forms/001.html
 ```
+or
+
+```bash
+out/Default/content_shell.exe --run-web-tests fast/forms/001.html
+```
 
 but this requires a manual diff against expected results, because the shell
-doesn't do it for you.
+doesn't do it for you. It also just dumps the text result only (as the dump of
+pixels and audio binary data is not human readable).
+See [Running Web Tests Using the Content Shell](web_tests_in_content_shell.md]
+for more details of running `content_shell`.
 
 To see a complete list of arguments supported, run:
 
 ```bash
-python run_web_tests.py --help
+python third_party/blink/tools/run_web_tests.py --help
 ```
 
 *** note

diff --git a/docs/testing/web_tests_in_content_shell.md b/docs/testing/web_tests_in_content_shell.md
@@ -1,55 +1,151 @@
 # Running web tests using the content shell
 
-## Basic usage
-
-Web tests can be run with `content_shell`. To just dump the render tree, use
-the `--run-web-tests` flag:
-
-```bash
-out/Default/content_shell --run-web-tests foo.html
-```
-
-### Compiling
+## Compiling
 
 If you want to run web tests,
 [build the target `blink_tests`](web_tests.md); this includes all the other
 binaries required to run the tests.
 
-### Running
+## Running
+
+### Using `run_web_tests.py`
 
 You can run web tests using `run_web_tests.py` (in
 `src/third_party/blink/tools`).
 
 ```bash
-third_party/blink/tools/run_web_tests.py storage/indexeddb
+python third_party/blink/tools/run_web_tests.py storage/indexeddb
 ```
+To see a complete list of arguments supported, run with `--help`.
+
+***promo
+You can add `<path>/third_party/blink/tools` into `PATH` so that you can
+run it from anywhere without the full path.
+***
+
+### Run Web Tests Directly with Content Shell
 
-or execute the shell directly:
+In some cases (e.g. for debugging), you can run web tests directly with
+Content Shell executable, with the `--run-web-tests` flag:
 
 ```bash
-out/Default/content_shell --remote-debugging-port=9222
+out/Default/content_shell --run-web-tests <url>|<full_test_source_path>|<relative_test_path>
+```
+
+`<relative_test_path>` is relative to the [web_tests](../../third_party/blink/web_tests)
+directory, regardless of the current directory.
+
+For example:
+
+```bash
+out/Default/content_shell --run-web-tests fast/forms/001.html
+```
+or
+
+```bash
+out/Default/content_shell --run-web-tests \
+    /home/user/chrome/src/third_party/blink/web_tests/fast/forms/001.html
+```
+or
+
+```bash
+out/Default/content_shell --run-web-tests ~/test/temp-test.html
 ```
 
-This allows you see how your changes look in Chromium, and even connect with
-devtools (by going to http://127.0.0.1:9222 from another window) to inspect your
-freshly compiled Blink.
+By default, it dumps the text result only (as the dump of pixels and audio
+binary data is not human readable). This can meet the requirement of
+most debugging requirements.
+
+In rare cases, to run `content_shell` in the exact same way as
+`run_web_tests.py` runs it, you need to run it in the
+[protocol mode](../../content_shell/browser/web_test/test_info_extractor.h).
 
 *** note
 On the Mac, use `Content Shell.app`, not `content_shell`.
 
 ```bash
-out/Default/Content\ Shell.app/Contents/MacOS/Content\ Shell --remote-debugging-port=9222
+out/Default/Content\ Shell.app/Contents/MacOS/Content\ Shell ...
 ```
+On Windows, use `content_shell.exe`.
 ***
 
-### Debugging Renderer Crashes
+#### Running HTTP Tests in Content Shell
+
+HTTP tests reside under [web_tests/http/tests](../../third_party/blink/web_tests/http/tests).
+You need to start a web server first:
+
+```bash
+python third_party/blink/tools/run_blink_httpd.py
+```
+Then run the test with a localhost URL:
+
+```bash
+out/Default/content_shell --run-web-tests http://localhost:8000/<test>
+```
+
+#### Running WPT Tests in Content Shell
+
+Similar to HTTP tests, many WPT (a.k.a. web-platform-tests under
+[web_tests/external/wpt](../../third_party/blink/web_tests/external/wpt) directory)
+tests require some setup before running in Content Shell:
+
+```bash
+python third_party/blink/tools/run_blink_wptserve.py
+```
+Then run the test:
+
+```bash
+out/Default/content_shell --run-web-tests http://localhost:8001/<test>
+```
+See [the blink-dev discussion](https://groups.google.com/a/chromium.org/forum/?utm_medium=email&utm_source=footer#!msg/blink-dev/iP_9ok1K1UM/F_qCmk5kDAAJ) for more details.
+
+### As a simple browser
+
+You can run the shell directly as a simple browser:
+
+```bash
+out/Default/content_shell
+```
+
+This allows you see how your changes look in Chromium. You can inspect the page
+by right clicking and selecting 'Inspect Element'.
+
+You can also use `--remote-debugging-port`
+
+```bash
+out/Default/content_shell --remote-debugging-port=9222
+```
+and open `http://localhost:9222` from another browser to inspect the page.
+This is useful when you don't want DevTools to run in the same Content Shell,
+e.g. when you are logging a lot and don't want the log from DevTools
+or when DevTools is unstable in the current revision due to some bugs.
+
+## Debugging
+
+### `--single-process`
+
+The command line switch `--single-process` is useful for starting
+content_shell in gdb. In most cases, `--single-process` is good for debugging
+except when you want to debug the multiple process behavior or when we have
+some bug breaking `--single-process` in particular cases.
+
+### Web tests
+
+See [Run Web Tests Directly with Content Shell](#Run-Web-Tests-Directly-with-Content-Shell).
+In most cases you don't need `--single-process` because `content_shell` is
+in single process mode when running most web tests.
+
+See [DevTools frontend](../../third_party/blink/renderer/devtools/readme.md#basics)
+for the commands that are useful for debugging devtools web tests.
+
+### In The Default Multiple Process Mode
 
-To debug a renderer crash, ask Content Shell to wait for you to attach a
-debugger once it spawns a renderer process by adding the
-`--renderer-startup-dialog` flag:
+In rare cases, you need to debug Content Shell in multiple process mode.
+You can ask Content Shell to wait for you to attach a debugger once it spawns a
+renderer process by adding the `--renderer-startup-dialog` flag:
 
 ```bash
-out/Default/content_shell --renderer-startup-dialog
+out/Default/content_shell --renderer-startup-dialog --no-sandbox
 ```
 
 Debugging workers and other subprocesses is simpler with