-
Notifications
You must be signed in to change notification settings - Fork 136
ci: add automated doc checks (and related starter pack updates) #2099
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
92ccdc5
add starter pack files for checks and updates
dwilding 9fc3373
update venv target to install vale packages (for spellcheck)
dwilding 34afdf3
add spelling exceptions
dwilding 28bcd04
mention the spellchecker in HACKING.md
dwilding f1b15c0
add workflow for spelling check
dwilding b0591c9
fix spelling in spelling CI -_-
dwilding a6c60b1
add linkcheck exceptions
dwilding 2565081
add manual workflow for linkcheck
dwilding 722a27c
fix linkcheck workflow and add comment
dwilding cada4d9
add a comment to explain excluded URLs
dwilding f981ce9
move Vale deps to requirements.txt
dwilding c3ebecd
improve comment about linkcheck_ignore
dwilding File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| name: Docs links | ||
|
|
||
| on: | ||
| # The docs have a lot of external links, so the link checker takes a long time to run. | ||
| # As of Oct 2025, the link checker fails on many Read the Docs links because RTD's | ||
| # servers respond with 429 (Too Many Requests) without allowing retry after delay. | ||
| # Given this, it's best to run the workflow manually and inspect the results. | ||
| workflow_call: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: {} | ||
|
|
||
| jobs: | ||
| docs-linkcheck: | ||
| name: Links | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Set up uv | ||
| uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0 | ||
| - name: Check links | ||
| run: make -C docs linkcheck | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| name: Docs | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| pull_request: | ||
| workflow_call: | ||
|
|
||
| permissions: {} | ||
|
|
||
| jobs: | ||
| docs-spelling: | ||
| name: Spelling | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Set up uv | ||
| uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0 | ||
| - name: Check spelling | ||
| run: make -C docs spelling |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Leave a blank line at the end of this file to support concatenation | ||
| backoff | ||
| databag | ||
| databags | ||
| dict's | ||
| failover | ||
| manpages | ||
| misconfiguration | ||
| pydantic | ||
| pytest | ||
| repo | ||
| storages | ||
| traceback | ||
| Wordpress | ||
| zizmor |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| #! /usr/bin/env python | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. File added from the starter pack, unchanged except for cleaning up whitespace (which I'm fixing in the starter pack: #448) |
||
|
|
||
| import os | ||
| import shutil | ||
| import subprocess | ||
| import tempfile | ||
| import sys | ||
| import logging | ||
| import argparse | ||
|
|
||
| # Configure logging | ||
| logging.basicConfig( | ||
| level=logging.INFO, | ||
| format='%(asctime)s - %(levelname)s - %(message)s', | ||
| datefmt='%Y-%m-%d %H:%M:%S' | ||
| ) | ||
|
|
||
| SPHINX_DIR = os.path.join(os.getcwd(), ".sphinx") | ||
|
|
||
| GITHUB_REPO = "canonical/documentation-style-guide" | ||
| GITHUB_CLONE_URL = f"https://github.com/{GITHUB_REPO}.git" | ||
|
|
||
| # Source paths to copy from repo | ||
| VALE_FILE_LIST = [ | ||
| "styles/Canonical", | ||
| "styles/config/vocabularies/Canonical", | ||
| "styles/config/dictionaries", | ||
| "vale.ini" | ||
| ] | ||
|
|
||
| def clone_repo_and_copy_paths(file_source_dest, overwrite=False): | ||
| """ | ||
| Clone the repository to a temporary directory and copy required files | ||
|
|
||
| Args: | ||
| file_source_dest: dictionary of file paths to copy from the repository, | ||
| and their destination paths | ||
| overwrite: boolean flag to overwrite existing files in the destination | ||
|
|
||
| Returns: | ||
| bool: True if all files were copied successfully, False otherwise | ||
| """ | ||
|
|
||
| if not file_source_dest: | ||
| logging.error("No files to copy") | ||
| return False | ||
|
|
||
| # Create temporary directory on disk for cloning | ||
| temp_dir = tempfile.mkdtemp() | ||
| logging.info("Cloning repository <%s> to temporary directory: %s", GITHUB_REPO, temp_dir) | ||
| clone_cmd = ["git", "clone", "--depth", "1", GITHUB_CLONE_URL, temp_dir] | ||
|
|
||
| try: | ||
| result = subprocess.run( | ||
| clone_cmd, | ||
| capture_output=True, | ||
| text=True, | ||
| check=True | ||
| ) | ||
| logging.debug("Git clone output: %s", result.stdout) | ||
| except subprocess.CalledProcessError as e: | ||
| logging.error("Git clone failed: %s", e.stderr) | ||
| return False | ||
|
|
||
| # Copy files from the cloned repository to the destination paths | ||
| is_copy_success = True | ||
| for source, dest in file_source_dest.items(): | ||
| source_path = os.path.join(temp_dir, source) | ||
|
|
||
| if not os.path.exists(source_path): | ||
| is_copy_success = False | ||
| logging.error("Source path not found: %s", source_path) | ||
| continue | ||
|
|
||
| if not copy_files_to_path(source_path, dest, overwrite): | ||
| is_copy_success = False | ||
| logging.error("Failed to copy %s to %s", source_path, dest) | ||
|
|
||
| # Clean up temporary directory | ||
| logging.info("Cleaning up temporary directory: %s", temp_dir) | ||
| shutil.rmtree(temp_dir) | ||
|
|
||
| return is_copy_success | ||
|
|
||
| def copy_files_to_path(source_path, dest_path, overwrite=False): | ||
| """ | ||
| Copy a file or directory from source to destination | ||
|
|
||
| Args: | ||
| source_path: Path to the source file or directory | ||
| dest_path: Path to the destination | ||
| overwrite: Boolean flag to overwrite existing files in the destination | ||
|
|
||
| Returns: | ||
| bool: True if copy was successful, False otherwise | ||
| """ | ||
| # Skip if source file doesn't exist | ||
| if not os.path.exists(source_path): | ||
| logging.warning("Source path not found: %s", source_path) | ||
| return False | ||
|
|
||
| logging.info("Copying %s to %s", source_path, dest_path) | ||
| # Handle existing files | ||
| if os.path.exists(dest_path): | ||
| if overwrite: | ||
| logging.info(" Destination exists, overwriting: %s", dest_path) | ||
| if os.path.isdir(dest_path): | ||
| shutil.rmtree(dest_path) | ||
| else: | ||
| os.remove(dest_path) | ||
| else: | ||
| logging.info(" Destination exists, skip copying (use overwrite=True to replace): %s", | ||
| dest_path) | ||
| return True # Skip copying | ||
|
|
||
| # Copy the source to destination | ||
| try: | ||
| if os.path.isdir(source_path): | ||
| # entire directory | ||
| shutil.copytree(source_path, dest_path) | ||
| else: | ||
| # individual files | ||
| shutil.copy2(source_path, dest_path) | ||
| return True | ||
| except (shutil.Error, OSError) as e: | ||
| logging.error("Copy failed: %s", e) | ||
| return False | ||
|
|
||
| def parse_arguments(): | ||
| parser = argparse.ArgumentParser(description="Download Vale configuration files") | ||
| parser.add_argument("--no-overwrite", action="store_true", help="Don't overwrite existing files") | ||
| return parser.parse_args() | ||
|
|
||
| def main(): | ||
| # Define local directory paths | ||
| vale_files_dict = {file: os.path.join(SPHINX_DIR, file) for file in VALE_FILE_LIST} | ||
|
|
||
| # Parse command line arguments, default to overwrite_enabled = True | ||
| overwrite_enabled = not parse_arguments().no_overwrite | ||
|
|
||
| # Download into /tmp through git clone | ||
| if not clone_repo_and_copy_paths(vale_files_dict, overwrite=overwrite_enabled): | ||
| logging.error("Failed to download files from repository") | ||
| return 1 | ||
|
|
||
| logging.info("Download complete") | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) # Keep return code | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we're talking to RTD about this