Survey of vendored dependencies (tools/third_party/)

[foolip]

#27377 and #28570 made me curious about our dependency management, and whether it can be made both more deterministic and easy for users with something like #20019. While playing with #28796 I went through all of tools/third_party/ and thought it would be useful to write down what I found.

A few observations before the list:

• It's a lot of work to figure out which version each dependency is, if it's up-to-date and if it should be updated.
• Direct and indirect dependencies aren't distinguished. If some dependencies become orphaned, it wouldn't be easy to notice.
• There is overlap between vendored dependencies and those installed via requirements.txt files. At least html5lib and pytest + their many dependencies.

See also #28809 which surveys the installed dependencies.

The list...

aioquic

Used in tools/quic, a custom Python 3 QUIC server

Version: 0.8.8 (latest is 0.9.11)

Last touched in #23804.

atomicwrites

Used to write MANIFEST.json atomically.

Version: 1.1.5 (latest is 1.4.0)

Last touched in #11951.

attrs

Dependency of pytest, no direct dependencies found.

Version: 18.1.0 (latest is 20.3.0)

Last touched in #11951.

certifi

Dependency of aioquic.

Version: 2018.4.16 (latest is 2020.12.5)

Last touched in #11806.

enum

This is enum34, dependency of h2, but ought not be needed now that we're Python 3-only.

Version: 1.1.10 (up-to-date)

Last touched in #26726 (version in title seems wrong)

funcsigs

Former dependency of pytest, seemingly no longer needed?

Version: 1.0.2 (up-to-date)

Last touched in #26704.

h2

Used by wptserve for HTTP/2 support.

Version: 3.2.0 (latest is 4.0.0)

Last touched in #26331.

hpack

Dependency of h2 and used directly in tools/wptserve/wptserve/response.py.

Version: 3.0.0 (latest is 4.0.0)

Last touched in #12193.

html5lib

Used to parse tests to generate MANIFEST.json.

Version: 1.1 (up-to-date)

Last touched in #24307.

hyper

Used only (?) in wptserve/tests/functional/base.py, and is no longer maintained.

Version: 0.7.0

Last touched in #14355.

hyperframe

Dependency of h2 and used directly in tools/wptserve/wptserve/response.py.

Version: 5.2.0 (latest is 6.0.1)

Last touched in #26331.

importlib_metadata

This is importlib-metadata

Dependency of pytest.

Version: 2.1.0 (latest is 4.0.1)

Last touched in #26704.

iniconfig

Dependency of pytest.

Version: 1.1.1 (up-to-date)

Last touched in 4f540d7.

more-itertools

Former dependency, now unused?

Version: 4.2.0 (latest is 8.7.0)

Last touched in #11951.

packaging

Dependency of pytest, via importlib-metadata?

Version: 20.7 (latest is 20.9)

Last touched in #26704.

pathlib2

Dependency of pytest, via importlib-metadata?

Version: 2.3.5 (up-to-date)

Last touched in #26704.

pdf_js

Unlike most directories this is not Python code, it's a JavaScript library used for print reftests. web-platform-tests/rfcs#46 is relevant.

Last touched in #21901.

pluggy

Dependency of pytest.

Version: 0.13.1 (up-to-date)

Last touched in #26704.

py

This is https://github.com/pytest-dev/py, a dependency of pytest.

Version: pytest-dev/py@35b742df9e, unclear which release this is.

Last touched in #8535.

pytest

Used for webdriver/tests/. The pytest used for wpt unit tests is separate, see #28737.

Version: 6.1.1 (latest is 6.2.3)

Last touched in #27195.

pytest-asyncio

Will be used for WebDriver BiDi tests, and probably more stuff over time. See #28381.

Version: 0.14.0 (latest is 0.15.1)

Last touched in #27195.

pywebsocket3

A copy of https://github.com/GoogleChromeLabs/pywebsocket3.

Version: GoogleChromeLabs/pywebsocket3@9db3d29db4, which is 3.0.1

Last touched in b4cb174.

six

Used in dependencies and directly, see #28776.

Version: 1.15.0 (up-to-date)

Last touched in #23733.

tooltool

Just a single tooltool.py file.

Version: N/A/

Last touched in #18406.

webencodings

Dependency of html5lib.

Version: 0.5.1 (up-to-date)

Last touched in #11380.

websockets

Used for WebDriver BiDi tests.

Version: python-websockets/websockets@139085fe26 is 8.2 (latest is 9.0.1)

Last touched in 5c81bd0.

zipp

Dependency of importlib-metadata.

Version: 1.2.0 (latest is 3.4.1)

Last touched in #26972.