Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a remix of tox.ini used for visual/explorative regression testing #1388

Conversation

benjaoming
Copy link
Contributor

@benjaoming benjaoming commented Dec 15, 2022

I want to save this for another time. I didn't want to play around with a combination of the tox environment matrix, so I've put it in a separate file for now.

In a future version, we could be using Firefox/Selenium to grab screenshots and save them to an artifacts directory on Circle CI.

image

@benjaoming benjaoming requested a review from a team as a code owner December 15, 2022 12:15
@humitos
Copy link
Member

humitos commented Aug 22, 2023

This is a good idea. I'd propose some changes here:

  1. update the PR with the latest changes after releasing 2.0
  2. use {-firefox} and call it with firefox: sh -c 'firefox -P sphinx-testing "{tox_root}/html/{env_name}/demo/demo.html" &'

This way, the developer can call:

tox -e py310-sphinx{53,62,72,dev}-firefox

humitos added a commit that referenced this pull request Aug 23, 2023
Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from #1388
Closes #1388
humitos added a commit that referenced this pull request Aug 23, 2023
* Visual test with Firefox and tox

Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from #1388
Closes #1388

* Default value for `DISPLAY` and `--new-tab`

* Simplify the idea :)
@benjaoming benjaoming deleted the visual-regression-test-shortcut branch August 23, 2023 18:50
humitos added a commit that referenced this pull request Nov 30, 2023
Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from #1388
Closes #1388
humitos added a commit that referenced this pull request Jan 3, 2024
* Remove Sphinx<5 compatibility

Since 2.0 we do not support Sphinx<5.

* Always use `writer-html5`

Since 2.0 HTML4 writer is not supported.

* Use `css_tag` helper to inject CSS files

* Revert "Remove Sphinx<5 compatibility"

Keep `sphinx_version_info`. It will be required.

This reverts commit 3afbed2.

* Set `data-content_root` for Sphinx >= 7.2

Copied from #1507

* Default value for `DISPLAY` and `--new-tab`

* Visual test with Firefox and tox

Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from #1388
Closes #1388

* Update sphinx_rtd_theme/layout.html

Co-authored-by: Dmitry Shachnev <mitya57@users.noreply.github.com>

* Remove code from merge conflict

* Revert a mistake done on merging

* Update CSS logic again

* Minor refactor to undocumented `extra_css_files` option

---------

Co-authored-by: Dmitry Shachnev <mitya57@users.noreply.github.com>
khancyr pushed a commit to ArduPilot/sphinx_rtd_theme that referenced this pull request Jul 15, 2024
* 2.0 milestone: drop supported versions

The new policy is:

- Python >=3.6
- Sphinx >= 5
- HTML4 and HTML5 writer
- docutils >= 0.14, < 0.19

* Update roadmap after discussion

By the beginning of 2024 we should support newer Sphinx versions and HTML5 only.

* Remove Python 2.7 from CircleCI

* Run only supported versions on Python 3.11

* Update with feedback from review

* Update documentation to match our support

* Add Sphinx 6.2 that was missing

* Update tox/circleci to work

* Revert tox command

* Drop support for HTML4 and keep `-qa` testing

- drop support for HTML4 on 2.0
- keep `-qa` TOX environment to be able to compare feature branches
- emit `logger.errors` on Python 2 and html_writer4=True
- require Sphinx >= 5.0
- remove logic for old Sphinx versions

* Update Tox and CircleCI

* Update contributing page to match our plans

---------

Co-authored-by: Anthony Johnson <aj@ohess.org>

Release 2.0.0rc1 (readthedocs#1510)

* Release 2.0.0rc1

* Update npm lock file

Bump to next release 2.0.0rc2 (readthedocs#1511)

Docs: Remove "git install" method from instructions (readthedocs#1375)

* Remove instructions "encouraging" to install directly from git

* Update changelog: Add reminder that people should not install directly from git

* Update docs/changelog.rst

---------

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>

Visual test with Firefox and tox (readthedocs#1513)

* Visual test with Firefox and tox

Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from readthedocs#1388
Closes readthedocs#1388

* Default value for `DISPLAY` and `--new-tab`

* Simplify the idea :)

Remove Python2 leftovers (readthedocs#1514)

Docs: update Read the Docs config (readthedocs#1518)

Use the latest config file style.

Set data-content_root for Sphinx ≥ 7.2 (readthedocs#1507)

This attribute is now required by searchtools.js:
https://github.com/sphinx-doc/sphinx/blob/v7.2.2/sphinx/themes/basic/static/searchtools.js#L65

When it's not set, loading search results via AJAX will be broken
because the JS code will attempt to load paths like /undefinedfoo.html.

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>

Use `css_tag` helper to inject CSS files (readthedocs#1519)

* Use `css_tag` helper to inject CSS files

* Typo

Add support for `docutils==0.20.x` (readthedocs#1517)

Closes readthedocs#1516

Release 2.0rc2 (readthedocs#1520)

Version bump for 2.0rc3 development (readthedocs#1521)

Next development version is 2.0rc3 for now, but more likely 2.1rc1

Fix readthedocs#1522: fix `'str' object has no attribute 'attributes'` (readthedocs#1528)

* Fix readthedocs#1522: fix attribute error if css is str

* Revert "Fix readthedocs#1522: fix attribute error if css is str"

This reverts commit b22a77f.

* Use `css_tag` only for standard CSS files from Sphinx

... and keep using the custom `link` HTML tag for CSS files included via the
HTML theme option `extra_css_files`.

* Normalize template variables

---------

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>
Co-authored-by: Anthony Johnson <aj@ohess.org>

Release 2.0.0rc3 (readthedocs#1535)

Increment for next potential release (readthedocs#1536)

Fix AttributeError when one of `css_files` is a string (readthedocs#1537)

Fix AttributeError when one of css_files is a string

Release 2.0.0rc4 (readthedocs#1538)

Bump for next potential release, 2.0.0rc5 (readthedocs#1539)

Release 2.0 final (readthedocs#1544)
khancyr pushed a commit to ArduPilot/sphinx_rtd_theme that referenced this pull request Sep 1, 2024
* 2.0 milestone: drop supported versions

The new policy is:

- Python >=3.6
- Sphinx >= 5
- HTML4 and HTML5 writer
- docutils >= 0.14, < 0.19

* Update roadmap after discussion

By the beginning of 2024 we should support newer Sphinx versions and HTML5 only.

* Remove Python 2.7 from CircleCI

* Run only supported versions on Python 3.11

* Update with feedback from review

* Update documentation to match our support

* Add Sphinx 6.2 that was missing

* Update tox/circleci to work

* Revert tox command

* Drop support for HTML4 and keep `-qa` testing

- drop support for HTML4 on 2.0
- keep `-qa` TOX environment to be able to compare feature branches
- emit `logger.errors` on Python 2 and html_writer4=True
- require Sphinx >= 5.0
- remove logic for old Sphinx versions

* Update Tox and CircleCI

* Update contributing page to match our plans

---------

Co-authored-by: Anthony Johnson <aj@ohess.org>

Release 2.0.0rc1 (readthedocs#1510)

* Release 2.0.0rc1

* Update npm lock file

Bump to next release 2.0.0rc2 (readthedocs#1511)

Docs: Remove "git install" method from instructions (readthedocs#1375)

* Remove instructions "encouraging" to install directly from git

* Update changelog: Add reminder that people should not install directly from git

* Update docs/changelog.rst

---------

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>

Visual test with Firefox and tox (readthedocs#1513)

* Visual test with Firefox and tox

Allows developer to run

```
tox -e py310-sphinx72-firefox
```

This will run the tests and open a Firefox after building the demo site
showing the `demo/demo.html` page on it.

Then, you can also run:

```
tox -e py310-sphinx61-firefox
```

and compare the visual differences.

Idea copied from readthedocs#1388
Closes readthedocs#1388

* Default value for `DISPLAY` and `--new-tab`

* Simplify the idea :)

Remove Python2 leftovers (readthedocs#1514)

Docs: update Read the Docs config (readthedocs#1518)

Use the latest config file style.

Set data-content_root for Sphinx ≥ 7.2 (readthedocs#1507)

This attribute is now required by searchtools.js:
https://github.com/sphinx-doc/sphinx/blob/v7.2.2/sphinx/themes/basic/static/searchtools.js#L65

When it's not set, loading search results via AJAX will be broken
because the JS code will attempt to load paths like /undefinedfoo.html.

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>

Use `css_tag` helper to inject CSS files (readthedocs#1519)

* Use `css_tag` helper to inject CSS files

* Typo

Add support for `docutils==0.20.x` (readthedocs#1517)

Closes readthedocs#1516

Release 2.0rc2 (readthedocs#1520)

Version bump for 2.0rc3 development (readthedocs#1521)

Next development version is 2.0rc3 for now, but more likely 2.1rc1

Fix readthedocs#1522: fix `'str' object has no attribute 'attributes'` (readthedocs#1528)

* Fix readthedocs#1522: fix attribute error if css is str

* Revert "Fix readthedocs#1522: fix attribute error if css is str"

This reverts commit b22a77f.

* Use `css_tag` only for standard CSS files from Sphinx

... and keep using the custom `link` HTML tag for CSS files included via the
HTML theme option `extra_css_files`.

* Normalize template variables

---------

Co-authored-by: Manuel Kaufmann <humitos@gmail.com>
Co-authored-by: Anthony Johnson <aj@ohess.org>

Release 2.0.0rc3 (readthedocs#1535)

Increment for next potential release (readthedocs#1536)

Fix AttributeError when one of `css_files` is a string (readthedocs#1537)

Fix AttributeError when one of css_files is a string

Release 2.0.0rc4 (readthedocs#1538)

Bump for next potential release, 2.0.0rc5 (readthedocs#1539)

Release 2.0 final (readthedocs#1544)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants