diff --git a/docs/documentation_best_practices.md b/docs/documentation_best_practices.md index 328a98f7f41ff..4e4df45b945a2 100644 --- a/docs/documentation_best_practices.md +++ b/docs/documentation_best_practices.md @@ -97,7 +97,7 @@ to detailed prose: design. However, once the code is implemented, design docs should serve as archives of these decisions, not as half-correct docs (they are often misused). See - [Implementation state](#implementation_state_determines_document_location) + [Implementation state](#Implementation-state-determines-document-repository) below. ## Implementation state determines document repository diff --git a/docs/linux_sandboxing.md b/docs/linux_sandboxing.md index dfbdf3ac210c9..2bd09a2459aa2 100644 --- a/docs/linux_sandboxing.md +++ b/docs/linux_sandboxing.md @@ -41,11 +41,11 @@ network, something that is difficult to perform at the layer-2. | **Name** | **Layer and process** | **Linux flavors where available** | **State** | |:---------|:----------------------|:----------------------------------|:----------| -| [Setuid sandbox](#The_setuid_sandbox.md) | Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS | Enabled by default (old kernels) and maintained | -| [User namespaces sandbox](#User_namespaces_sandbox.md) | Modern alternative to the setuid sandbox. Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS (kernel >= 3.8) | Enabled by default (modern kernels) and actively developed | -| [Seccomp-BPF](#The_seccomp-bpf_sandbox.md) | Layer-2 in some Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient)), Layer-1 + Layer-2 in GPU process | Linux kernel >= 3.5, Chrome OS and Ubuntu | Enabled by default and actively developed | -| [Seccomp-legacy](#The_seccomp_sandbox.md) | Layer-2 in renderers | All | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=197301&view=revision) | -| [SELinux](#SELinux.md) | Layer-1 in Zygote processes (renderers, PPAPI) | SELinux distributions | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=200838&view=revision) | +| [Setuid sandbox](#The-setuid-sandbox) | Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS | Enabled by default (old kernels) and maintained | +| [User namespaces sandbox](#User-namespaces-sandbox) | Modern alternative to the setuid sandbox. Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS (kernel >= 3.8) | Enabled by default (modern kernels) and actively developed | +| [Seccomp-BPF](#The-sandbox-1) | Layer-2 in some Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient)), Layer-1 + Layer-2 in GPU process | Linux kernel >= 3.5, Chrome OS and Ubuntu | Enabled by default and actively developed | +| [Seccomp-legacy](#The-sandbox-2) | Layer-2 in renderers | All | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=197301&view=revision) | +| [SELinux](#SELinux) | Layer-1 in Zygote processes (renderers, PPAPI) | SELinux distributions | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=200838&view=revision) | | AppArmor | Outer layer-1 in Zygote processes (renderers, PPAPI) | Not used | Deprecated | ## The setuid sandbox diff --git a/docs/testing/test_browser_dialog.md b/docs/testing/test_browser_dialog.md index c96045556aeaa..44cf92ad1471c 100644 --- a/docs/testing/test_browser_dialog.md +++ b/docs/testing/test_browser_dialog.md @@ -49,7 +49,7 @@ Notes: * The body of the test is always just "`RunDialog();`". * "`default`" is the `std::string` passed to `ShowDialog()` and can be customized. See - [Testing additional dialog "styles"](#Testing-additional-dialog-styles_). + [Testing additional dialog "styles"](#Testing-additional-dialog-styles). * The text before `default` (in this case) must always be "`InvokeDialog_`". ### Concrete examples diff --git a/docs/testing/web_platform_tests.md b/docs/testing/web_platform_tests.md index 878ef50fcdd4a..e20e83af9f328 100644 --- a/docs/testing/web_platform_tests.md +++ b/docs/testing/web_platform_tests.md @@ -11,9 +11,14 @@ test suite where all browser engines are collaborating. There's also a that will [soon be merged into web-platform-tests](https://github.com/w3c/csswg-test/issues/1102). +Chromium has 2-way import/export process with the upstream web-platform-tests +repository, where tests are imported into +[LayoutTests/external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +and any changes to the imported tests are also exported to web-platform-tests. + [TOC] -## Import +## Importing tests Chromium has mirrors ([web-platform-tests](https://chromium.googlesource.com/external/w3c/web-platform-tests/), @@ -21,7 +26,7 @@ Chromium has mirrors GitHub repos, and periodically imports a subset of the tests so that they are run as part of the regular Blink layout test testing process. -The goal of this process are to be able to run the Web Platform Tests unmodified +The goal of this process are to be able to run Web Platform Tests unmodified locally just as easily as we can run the Blink tests, and ensure that we are tracking tip-of-tree in the Web Platform Tests repository as closely as possible, and running as many of the tests as possible. @@ -39,6 +44,8 @@ The easiest way to check the status of recent imports is to look at: - Recent CLs created by [blink-w3c-test-autoroller@chromium.org](https://codereview.chromium.org/search?owner=blink-w3c-test-autoroller%40chromium.org). +Automatic imports are intended to run at least once every 24 hours. + ### Skipped tests We control which tests are imported via a file called @@ -70,17 +77,17 @@ it, and on the next auto-import, the new tests should be imported. If you want to import immediately (in order to try the tests out locally, etc) you can also run `wpt-import --allow-local-commits`, but this is not required. -## Contributing tests back to the Web Platform Tests project. +## Writing tests -If you need to make changes to Web Platform Tests, just commit your changes -directly to -[LayoutTests/external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +To contribute changes to Web Platform Tests, just commit your changes directly +to [LayoutTests/external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) and the changes will be automatically upstreamed within 24 hours. -Note that tests in Web Platform Tests are expected to match behavior defined by -the relevant WHATWG or W3C specification, not simply Blink's behavior. If in -doubt, please request code review from someone with expertise in the relevant -specification text. +Changes involving adding, removing or modifying tests can all be upstreamed. +Any changes outside of +[external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) will not be +upstreamed, and any changes `*-expected.txt`, `OWNERS`, and `MANIFEST.json`, +will also not be upstreamed. Note: if you're adding a new test in `external/wpt`, you'll need to re-generate MANIFEST.json manually until [CL 2644783003](https://crrev.com/2644783003) is @@ -91,18 +98,47 @@ Tools/Scripts/webkitpy/thirdparty/wpt/wpt/manifest --work \ --tests-root=LayoutTests/external/wpt ``` -### What kinds of changes can be upstreamed? +Most tests are written using testharness.js, see +[Writing Layout Tests](./writing_layout_tests.md) and +[Layout Tests Tips](./layout_tests_tips.md) for general guidelines. -In general, changes involving adding, removing or modifying tests can all be -upstreamed. From a Chromium commit, any changes outside of -[external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) will not be -upstreamed, and any changes `*-expected.txt`, `OWNERS`, and `MANIFEST.json`, -will also not be upstreamed. +### Write tests against specifications + +Tests in Web Platform Tests are expected to match behavior defined by the +relevant specification. In other words, all assertions that a test makes +should be derived from a specification's normative requirements, and not go +beyond them. It is often necessary to change the specification to clarify what +is and isn't required. + +When the standards discussion is still ongoing or blocked on some implementation +successfully shipping the hoped-for behavior, write the tests outside of +web-platform-tests and upstream them when the specification is finally updated. +Optionally, it may be possible to write deliberately failing tests against the +current specification and later update them. + +### Tests that require testing APIs + +Tests that depend on `internals.*`, `eventSender.*` or other internal testing +APIs cannot yet be written as part of web-platform-tests. + +An alternative is to write manual tests that are automated with scripts from +[wpt_automation](../../third_party/WebKit/LayoutTests/external/wpt_automation). +Such tests still require case-by-case automation to run for other browser +engines, but are more valuable than purely manual tests. + +*** note +TODO(foolip): Figure out and document a more scalable test automation solution. +*** + +### Adding new top-level directories Entirely new top-level directories should generally be added upstream, since that's the only way to add an OWNERS file upstream. After adding a new top-level directory upstream, you should add a line for it in `W3CImportExpectations`. +Adding the new directory (and `W3CImportExpectations` entry) in Chromium and +later adding an OWNERS file upstream also works. + ### Will the exported commits be linked to my GitHub profile? The email you commit with in Chromium will be the author of the commit on @@ -123,3 +159,18 @@ It's still possible to make direct pull requests to web-platform-tests. The processes for getting new tests committed the W3C repos are at http://testthewebforward.org/docs/. Some specifics are at http://testthewebforward.org/docs/github-101.html. + +## Reviewing tests + +Anyone who can review code and tests in Chromium can also review changes in +[external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +that will be automatically upstreamed. There will be no additional review in +web-platform-tests as part of the export process. + +If upstream reviewers have feedback on the changes, discuss on the pull request +created during export, and if necessary work on a new pull request to iterate +until everyone is satisfied. + +When reviewing tests, check that they match the relevant specification, which +may not fully match the implementation. See also +[Write tests against specifications](#Write-tests-against-specifications). diff --git a/docs/testing/writing_layout_tests.md b/docs/testing/writing_layout_tests.md index ce83becb77624..13ad74eb252ab 100644 --- a/docs/testing/writing_layout_tests.md +++ b/docs/testing/writing_layout_tests.md @@ -212,10 +212,6 @@ event handling tests can use [synthetic events](https://developer.mozilla.org/docs/Web/Guide/Events/Creating_and_triggering_events). *** -*** note -TODO: document wpt_automation. Manual tests might end up moving here. -*** - ### Relying on Blink-Specific Testing APIs Tests that cannot be expressed using the Web Platform APIs or WPT's testing APIs diff --git a/docs/using_a_linux_chroot.md b/docs/using_a_linux_chroot.md index 96fadfd36543a..da37cfe279be3 100644 --- a/docs/using_a_linux_chroot.md +++ b/docs/using_a_linux_chroot.md @@ -80,7 +80,7 @@ file:///path/to/results.html, check the following conditions. 1. Make sure `DISPLAY` is set. See the - [Running X apps](#Running_X_apps) section above. + [Running X apps](#Running-X-apps) section above. 1. Install `xdg-utils`, which includes `xdg-open`, a utility for finding the right application to open a file or URL with. 1. Install [Chrome](https://www.google.com/intl/en/chrome/browser/).