Skip to content

Commit

Permalink
blinkpy: Update "layout test" in various places to "web test"
Browse files Browse the repository at this point in the history
Bug: 843412
Change-Id: I1bd7de018b74d32c17ed36c3cf4b0e56a4b7b467
Reviewed-on: https://chromium-review.googlesource.com/c/1371404
Commit-Queue: Kent Tamura <tkent@chromium.org>
Reviewed-by: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#615751}
  • Loading branch information
tkent-google authored and Commit Bot committed Dec 12, 2018
1 parent a0d2a9b commit 81f891f
Show file tree
Hide file tree
Showing 26 changed files with 56 additions and 56 deletions.
6 changes: 3 additions & 3 deletions docs/how_to_extend_web_test_framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ and MacPort (and maybe the android port class) should be extended as they
provide platform specific overrides/extensions that implement most of the
important functionality. While there are many functions in Port, overriding one
function will affect most of the other ones to get the desired behavior. For
example, if `layout_tests_dir()` is overridden, not only will the code look for
example, if `web_tests_dir()` is overridden, not only will the code look for
tests in that directory, but it will find the correct TestExpectations file, the
platform specific expected files, etc.

Expand All @@ -125,7 +125,7 @@ Here are some of the functions that most likely need to be overridden.
* This should return the name of the program test p. By default it returns
‘content_shell’, but you want to have it return the program you want to
run, such as `chrome` or `browser_tests`.
* `layout_tests_dir`
* `web_tests_dir`
* This tells the port where to look for all the and everything associated
with them such as resources files.
* By default it returns the absolute path to the web tests directory.
Expand Down Expand Up @@ -172,7 +172,7 @@ and has stdout, stdin, stderr.
#### Goals

Your goal for this part of the project is to create a program (or extend a
program) to interface with the web test framework. The layout test framework
program) to interface with the web test framework. The web test framework
will communicate with this program to tell it what to do and it will accept data
from this program to perform the regression testing or create new base line
files.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ def test_diff_parser(self, parser=None):
self.assertEqual((87, 90), diff.lines[22][0:2])

# Check if a newly added file is correctly handled.
diff = parser.files['LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum']
diff = parser.files['web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum']
self.assertEqual(1, len(diff.lines))
self.assertEqual((0, 1), diff.lines[0][0:2])

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
}
bool StyleRareInheritedData::shadowDataEquivalent(const StyleRareInheritedData& o) const
diff --git a/LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum b/LayoutTests/platform/mac/fast/flexbox/box-orient-button-expected.checksum
diff --git a/web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum b/web_tests/platform/mac/fast/flexbox/box-orient-button-expected.checksum
new file mode 100644
index 0000000..6db26bd
--- /dev/null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ class WebTestResultsTest(unittest.TestCase):
},
"skipped": 450,
"num_regressions": 15,
"layout_tests_dir": "/b/build/slave/Webkit_Mac10_5/build/src/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/build/slave/Webkit_Mac10_5/build/src/third_party/blink/web_tests",
"version": 3,
"num_passes": 77,
"fixable": 1220,
Expand Down
2 changes: 1 addition & 1 deletion third_party/blink/tools/blinkpy/common/unified_diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

"""A utility function to do text diffs of expected and actual layout test results."""
"""A utility function to do text diffs of expected and actual web test results."""

import difflib

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
W3C Web Platform Tests in Blink Layout Tests
W3C Web Platform Tests in Blink Web Tests

Design Doc: https://goo.gl/iXUaZd

Expand Down Expand Up @@ -28,7 +28,7 @@ changing the ports (HTTP/S, WS/S), make sure to also:

- update `WPT_HOST_AND_PORTS` in
`//third_party/blink/tools/blinkpy/web_tests/port/driver.py`
- update LayoutTestContentBrowserClient::GetOriginsRequiringDedicatedProcess()
- update WebTestContentBrowserClient::GetOriginsRequiringDedicatedProcess()

checkout.sh
===========
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ class _FakeOptimizer(BaselineOptimizer):

def read_results_by_directory(self, baseline_name):
if baseline_name.endswith('txt'):
return {'LayoutTests/passes/text.html': '123456'}
return {'web_tests/passes/text.html': '123456'}
return {}


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def setUp(self):
self._write(path, 'contents')

self.mac_port.host.filesystem.write_text_file(
'/test.checkout/LayoutTests/external/wpt/MANIFEST.json', '{}')
'/test.checkout/web_tests/external/wpt/MANIFEST.json', '{}')

def tearDown(self):
BaseTestCase.tearDown(self)
Expand Down
4 changes: 2 additions & 2 deletions third_party/blink/tools/blinkpy/w3c/chromium_commit_mock.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ def short_sha(self):

def filtered_changed_files(self):
return [
'third_party/WebKit/LayoutTests/external/wpt/one.html',
'third_party/WebKit/LayoutTests/external/wpt/two.html',
'third_party/blink/web_tests/external/wpt/one.html',
'third_party/blink/web_tests/external/wpt/two.html',
]

def url(self):
Expand Down
4 changes: 2 additions & 2 deletions third_party/blink/tools/blinkpy/web_tests/builder_list.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

"""Represents a set of builder bots running layout tests.
"""Represents a set of builder bots running web tests.
This class is used to hold a list of builder bots running layout tests and their
This class is used to hold a list of builder bots running web tests and their
corresponding port names and TestExpectations specifiers.
"""

Expand Down
16 changes: 8 additions & 8 deletions third_party/blink/tools/blinkpy/web_tests/merge_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

"""Classes for merging layout tests results directories together.
"""Classes for merging web tests results directories together.
This is split into three parts:
* Generic code to merge JSON data together.
* Generic code to merge directories together.
* Code to specifically merge the layout tests result data together.
* Code to specifically merge the web tests result data together.
The JSON data merger will recursively merge dictionaries by default.
Expand Down Expand Up @@ -518,7 +518,7 @@ def merge(self, output_dir, to_merge_dirs):
break


# Classes specific to merging LayoutTest results directory.
# Classes specific to merging web test results directory.
# ------------------------------------------------------------------------


Expand Down Expand Up @@ -577,7 +577,7 @@ def __init__(self, allow_unknown_if_matching=False):
NameRegexMatch(':interrupted$'),
lambda o, name=None: bool(sum(o)))

# Layout test directory value is randomly created on each shard, so
# Web test directory value is randomly created on each shard, so
# clear it.
self.add_helper(
NameRegexMatch(':layout_tests_dir$'),
Expand All @@ -596,8 +596,8 @@ def fallback_matcher(self, objs, name=None):
return JSONMerger.fallback_matcher(self, objs, name)


class LayoutTestDirMerger(DirMerger):
"""Merge layout test result directory."""
class WebTestDirMerger(DirMerger):
"""Merge web test result directory."""

def __init__(self, filesystem=None,
results_json_value_overrides=None,
Expand Down Expand Up @@ -674,7 +674,7 @@ def main(argv):

parser = argparse.ArgumentParser()
parser.description = """\
Merges sharded layout test results into a single output directory.
Merges sharded web test results into a single output directory.
"""
parser.epilog = """\
Expand Down Expand Up @@ -812,7 +812,7 @@ def main(argv):
results_json_value_overrides[result_key] = build_properties[build_prop_key]
logging.debug('results_json_value_overrides: %r', results_json_value_overrides)

merger = LayoutTestDirMerger(
merger = WebTestDirMerger(
results_json_value_overrides=results_json_value_overrides,
results_json_allow_unknown_if_matching=args.results_json_allow_unknown_if_matching)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -541,7 +541,7 @@ def test_seconds_since_epoch(self):
merger.merge([{'seconds_since_epoch': 12}, {}]))


class LayoutTestDirMergerTests(unittest.TestCase):
class WebTestDirMergerTests(unittest.TestCase):

# JSON files for shard 1
# Shard1 has the following tests;
Expand All @@ -555,7 +555,7 @@ class LayoutTestDirMergerTests(unittest.TestCase):
"chromium_revision": "123",
"fixable": 1,
"interrupted": false,
"layout_tests_dir": "/b/s/w/irJ1McdS/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/s/w/irJ1McdS/third_party/blink/web_tests",
"num_failures_by_type": {
"AUDIO": 2,
"CRASH": 3
Expand Down Expand Up @@ -677,7 +677,7 @@ class LayoutTestDirMergerTests(unittest.TestCase):
"chromium_revision": "123",
"fixable": 9,
"interrupted": false,
"layout_tests_dir": "/b/s/w/sadfa124/third_party/WebKit/LayoutTests",
"layout_tests_dir": "/b/s/w/sadfa124/third_party/blink/web_tests",
"num_failures_by_type": {
"AUDIO": 10,
"CRASH": 11
Expand Down Expand Up @@ -772,7 +772,7 @@ class LayoutTestDirMergerTests(unittest.TestCase):
[Wed Mar 01 22:20:07.400802 2017] [ssl:warn] [pid 15010] AH01909: RSA certificate configured for 127.0.0.1:443 does NOT include an ID which matches the server name
"""

layout_test_filesystem = {
web_test_filesystem = {
# Files for shard0
'/shards/0/layout-test-results/access_log.txt': shard0_access_log,
'/shards/0/layout-test-results/archived_results.json': shard0_archived_results_json,
Expand Down Expand Up @@ -1008,7 +1008,7 @@ class LayoutTestDirMergerTests(unittest.TestCase):
[Wed Mar 01 22:20:07.400802 2017] [ssl:warn] [pid 15010] AH01909: RSA certificate configured for 127.0.0.1:443 does NOT include an ID which matches the server name
"""

layout_test_output_filesystem = {
web_test_output_filesystem = {
'/out/layout-test-results/access_log.txt': output_access_log,
'/out/layout-test-results/archived_results.json': output_archived_results_json,
'/out/layout-test-results/error_log.txt': output_error_log,
Expand Down Expand Up @@ -1040,11 +1040,11 @@ class LayoutTestDirMergerTests(unittest.TestCase):
}

def test(self):
fs = MockFileSystem(self.layout_test_filesystem)
fs = MockFileSystem(self.web_test_filesystem)

merger = merge_results.LayoutTestDirMerger(fs, results_json_value_overrides={'layout_tests_dir': 'src'})
merger = merge_results.WebTestDirMerger(fs, results_json_value_overrides={'layout_tests_dir': 'src'})
merger.merge('/out', ['/shards/0', '/shards/1'])

for fname, contents in self.layout_test_output_filesystem.items():
for fname, contents in self.web_test_output_filesystem.items():
self.assertIn(fname, fs.files)
self.assertMultiLineEqual(contents, fs.files[fname])
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

"""A helper class for reading in and dealing with tests expectations for layout tests."""
"""A helper class for reading in and dealing with tests expectations for web tests."""

from collections import defaultdict

Expand Down Expand Up @@ -880,7 +880,7 @@ def _already_seen_better_match(self, test, expectation_line):

class TestExpectations(object):
"""Test expectations consist of lines with specifications of what
to expect from layout test cases. The test cases can be directories
to expect from web test cases. The test cases can be directories
in which case the expectations apply to all test cases in that
directory and any subdirectory. The format is along the lines of:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def __init__(self, expectations, num_tests):
self.unexpected_failures = 0
self.unexpected_timeouts = 0

# The wall clock time spent running the tests (layout_test_runner.run()).
# The wall clock time spent running the tests (web_test_runner.run()).
self.run_time = 0

# Map of test name to the *last* result for the test.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def start(self, per_test_args, deadline):
assign it to the stdin of the process that is owned by this driver's
server process.
"""
# FIXME(ivandavid): Need to handle case where the layout test doesn't
# FIXME(ivandavid): Need to handle case where the web test doesn't
# get a file name.
new_cmd_line = self.cmd_line(per_test_args)
if not self._server_process or new_cmd_line != self._current_cmd_line:
Expand All @@ -52,7 +52,7 @@ def start(self, per_test_args, deadline):
self._open_stdin_path(deadline)

# Gets the path of the directory that the file for stdin communication is
# in. Since the browser test cannot clean it up, the layout test framework
# in. Since the browser test cannot clean it up, the web test framework
# will. Everything the browser test uses is stored in the same directory as
# the stdin file, so deleting that directory recursively will remove all the
# other temp data, like the printed pdf. This function assumes the correct
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@
import unittest

from blinkpy.common.host_mock import MockHost
from blinkpy.web_tests.print_layout_test_times import main
from blinkpy.web_tests.print_web_test_times import main


class PrintLayoutTestTimesTest(unittest.TestCase):
class PrintWebTestTimesTest(unittest.TestCase):

def check(self, args, expected_output, files=None):
host = MockHost()
Expand Down
8 changes: 4 additions & 4 deletions third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ def parse_args(args):
action='append',
default=[],
dest='adb_devices',
help='Run Android layout tests on these devices'),
help='Run Android web tests on these devices'),
# FIXME: Flip this to be off by default once we can log the
# device setup more cleanly.
optparse.make_option(
Expand Down Expand Up @@ -300,7 +300,7 @@ def parse_args(args):
help='Number of times to run the set of tests (e.g. ABCABCABC)'),
optparse.make_option(
'--layout-tests-directory',
help=('Path to a custom layout tests directory')),
help=('Path to a custom web tests directory')),
optparse.make_option(
'--max-locked-shards',
type='int',
Expand Down Expand Up @@ -464,7 +464,7 @@ def parse_args(args):
'--zero-tests-executed-ok',
action='store_true',
help='If set, exit with a success code when no tests are run.'
' Used on trybots when layout tests are retried without patch.')
' Used on trybots when web tests are retried without patch.')
]))

option_group_definitions.append(
Expand Down Expand Up @@ -509,7 +509,7 @@ def parse_args(args):
option_parser = optparse.OptionParser(
prog='run_web_tests.py',
usage='%prog [options] [tests]',
description='Runs Blink layout tests as described in docs/testing/web_tests.md')
description='Runs Blink web tests as described in docs/testing/web_tests.md')

for group_name, group_options in option_group_definitions:
option_group = optparse.OptionGroup(option_parser, group_name)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"""A utility module for making standalone scripts to start servers.
Scripts in tools/ can use this module to start servers that are normally used
for layout tests, outside of the layout test runner.
for web tests, outside of the web test runner.
"""

import logging
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

"""Base class used to start servers used by the layout tests."""
"""Base class used to start servers used by the web tests."""

import errno
import logging
Expand All @@ -42,7 +42,7 @@ class ServerError(Exception):


class ServerBase(object):
"""A skeleton class for starting and stopping servers used by the layout tests."""
"""A skeleton class for starting and stopping servers used by the web tests."""

def __init__(self, port_obj, output_dir):
self._port_obj = port_obj
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ def print_summary(self, total_time, run_results):
if self._options.timing:
parallel_time = sum(result.total_run_time for result in run_results.results_by_name.values())

# There is serial overhead in layout_test_runner.run() that we can't easily account for when
# There is serial overhead in web_test_runner.run() that we can't easily account for when
# really running in parallel, but taking the min() ensures that in the worst case
# (if parallel time is less than run_time) we do account for it.
serial_time = total_time - min(run_results.run_time, parallel_time)
Expand Down
4 changes: 2 additions & 2 deletions third_party/blink/tools/check_testharness_expected_pass.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

"""Check if a LayoutTest expected file is an all-PASS testharness result.
"""Check if a web test expected file is an all-PASS testharness result.
LayoutTests/PRESUBMIT.py uses this script to identify generic all-PASS
web_tests/PRESUBMIT.py uses this script to identify generic all-PASS
testharness baselines, which are redundant because run_web_tests.py assumes
all-PASS results for testharness tests when baselines are not found.
"""
Expand Down
4 changes: 2 additions & 2 deletions third_party/blink/tools/print_web_test_times.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,6 @@
import sys

from blinkpy.common import host
from blinkpy.web_tests import print_layout_test_times
from blinkpy.web_tests import print_web_test_times

print_layout_test_times.main(host.Host(), sys.argv[1:])
print_web_test_times.main(host.Host(), sys.argv[1:])
Loading

0 comments on commit 81f891f

Please sign in to comment.