Edited release script & notebook

[validbeck]

Internal Notes for Reviewers

sc-7068

generate_release_objects.py

TEST OUTPUT: #611

• Cleaned this script up
• The main function now performs all the same features as the notebook (for example, auto_summary was previously missing)
• Instead of repeating the code in main we simply call the granular functions defined earlier in the script — much cleaner, and also if we need to edit any of the functions now we don't have to do it 2-3 times

Old	New

generate-release-notes.ipynb

TEST OUTPUT: #611

• Edited this ENTIRE notebook for coherency & adherence to our style guide
• Redid the ToC for updated sections
• Instead of repeating the code in the notebook and importing functions from generate_release_objects.py in piecemeal, we simply import the whole script with an alias — "single-sourcing" at its finest

Old	New

Set up OpenAI API

• Updated the "Set up OpenAI API" to ask for the location of your .env file (default is set to ../.env)
• Output confirms the location of the file and whether or not the key is found

YOU CAN TEST THIS BY INPUTTING A FILEPATH WITH NO VALID .ENV OR REMOVE THE EXPECTED KEY FROM YOUR .ENV FILE

Old	New

Prompt	Output

No .env file at location	No valid OpenAI API key

Set labels > Create categories from labels

• Here I modified the default hierarchy to what we actually expect when we first work with the release notes
• It also spits out a numbered list to confirm the hierarchy in the output now

Old	New

Write release notes to file

I updated this so that it automatically appends {{< include /releases/_how-to-upgrade.qmd >}} to the file we created 🎉

Old	New

def upgrade_info(output_file):
    """
    Appends the upgrade information single-source to the end of the new release notes.

    Args:
        output_file (str): Path to the file to append.

    Returns:
        None
    """
    include_directive = "\n\n{{< include /releases/_how-to-upgrade.qmd >}}\n"

    try:
        with open(output_file, "a") as file:
            file.write(include_directive)
            print(f"Include _how-to-upgrade.qmd added to {file.name}.")
    except Exception as e:
        print(f"Failed to include _how-to-upgrade.qmd to {output_file}: {e}")

Update index

• Added this section so that it automatically inserts the latest release file just created into index.qmd in the "Latest Releases" section
• It also removes the oldest entry automatically!
• This relies on two markers on the index.qmd page:

index.qmd markers	New Update index section

The new function in the script:

def update_index_qmd(release_date):
    """Updates the index.qmd file to include the new releases in `Latest Releases` and removes the oldest release from the list.

    Params:
        release_date - release notes use the release date as the file name.
    
    Modifies:
        index.qmd file
    """

    index_filename = "../site/index.qmd"
    temp_index_filename = "../site/index_temp.qmd"

    # Copy the original QMD file to a temporary file
    shutil.copyfile(index_filename, temp_index_filename)

    with open(temp_index_filename, 'r') as file:
        lines = file.readlines()

    # Format the release date for insertion into the QMD file
    formatted_release_date = release_date.strftime("%Y-%b-%d").lower()

    with open(index_filename, 'w') as file:
        add_release_content = False
        insert_index = -1

        for i, line in enumerate(lines):
            file.write(line)
            if line.strip() == "# MAKE-RELEASE-NOTES-LATEST-MARKER":
                add_release_content = True
                insert_index = i

            if add_release_content and i == insert_index:
                file.write(f'      - /releases/{formatted_release_date}/release-notes.qmd\n')
                add_release_content = False

    # Remove the temporary file
    os.remove(temp_index_filename)
    
    print(f"Added new release notes to index.qmd, line {insert_index + 2}")

    removed_line = None  # To store the line that gets removed

    with open(index_filename, 'r') as file:
        updated_lines = file.readlines()

    with open(index_filename, 'w') as file:
        for i, line in enumerate(updated_lines):
            # Identify the marker line
            if line.strip() == "# MAKE-RELEASE-NOTES-OLDEST-MARKER":
                # Check if the line above exists and starts with a list indicator "-"
                if i > 0 and updated_lines[i - 1].strip().startswith("-"):
                    # Store the line being removed
                    removed_line = updated_lines[i - 1].strip()
                    # Write all lines up to the one before the line to remove
                    file.writelines(updated_lines[:i - 1])
                    # Write the marker and subsequent lines
                    file.writelines(updated_lines[i:])
                    break
        else:
            # If no marker is found, rewrite the file as is
            file.writelines(updated_lines)

    print(f"Removed the oldest release note entry: '{removed_line}'")

Show files to commit

Here I modified the script so that it filters out the files changed in release-scripts/:

def show_files():
    """Print files to commit by running 'git status --short'."""
    try:
        # Get the absolute path of the current directory
        current_dir = os.getcwd()

        # Run 'git status --short'
        result = subprocess.run(
            ["git", "status", "--short"], check=True, text=True, capture_output=True
        )
        
        # Process and display the output
        lines = result.stdout.strip().split('\n')

        if not lines:
            print("No changes detected.")
            return

        print("\nFiles to commit (excluding 'release-scripts'):")
        for line in lines:
            # Get the full path of the file
            parts = line.split()
            if len(parts) > 1:
                file_path = os.path.abspath(parts[-1])
            else:
                continue

            # Exclude files in the 'release-scripts' folder
            if 'release-scripts' not in file_path and line.startswith((' M', '??', 'A ')):
                print(line)

    except subprocess.CalledProcessError as e:
        print("Failed to run 'git status':", e)

Makefile

Calling make release-notes now calls generate_release_objects.py, checks the git status, and provides a preview using quarto preview (meaning it has the exact same functionality as the notebook, just through the command line):

Old	New

Create Release Notes

Since adding the upgrade information and slotting in the new release to the index is done automatically, I updated our internal Notion documentation:

Old	New

[github-actions]

PR Summary

This pull request introduces several enhancements and refactoring to the release notes generation process. The key changes include:

1. Automation of Release Notes Drafting: The Jupyter notebook generate-release-notes.ipynb has been updated to automate the creation of release notes drafts using a custom script and the OpenAI API. This includes:

   • Simplified instructions for setting up the OpenAI API and collecting GitHub URLs.
   • Streamlined process for extracting and editing PR information, including titles and body content.
   • Automated fetching of GitHub PR summaries using the github-actions bot.

2. Refactoring of Python Script: The generate_release_objects.py script has been refactored to improve code organization and readability. Key changes include:

   • Introduction of new functions for specific tasks, such as get_env_location, setup_openai_api, create_release_folder, and more.
   • Use of dotenv_values for loading environment variables without affecting os.environ.
   • Use of defaultdict for managing release components.

3. Improved File Management: The script now automatically updates the _quarto.yml and index.qmd files to include new release notes and manage the list of latest releases.

4. Makefile Update: The Makefile has been updated to include a release-notes target that runs the release notes generation script and previews the site.

5. .gitignore Update: Added release-scripts/__pycache__ to the .gitignore file to prevent caching files from being tracked.

These changes aim to streamline the release notes generation process, reduce manual effort, and improve the maintainability of the codebase.

Test Suggestions

• Test the release notes generation process with a valid OpenAI API key and multiple GitHub release URLs.
• Verify that the generated release notes are correctly categorized and formatted.
• Check that the _quarto.yml and index.qmd files are updated correctly with new release notes.
• Ensure that the script handles missing or incorrect .env file paths gracefully.
• Test the exclusion of files in release-scripts/ from the git status output.

[github-actions]

A PR preview is available: Preview URL

[github-actions]

A PR preview is available: Preview URL

[github-actions]

A PR preview is available: Preview URL

[github-actions]

A PR preview is available: Preview URL

[nrichers]

As discussed, I will test the functionality of this PR when I work on the release notes up to Sprint 66 which we now have a story for.

I did take a look at the MANY changes you made here and I especially like that

• make release-notes and the notebook are functionally equivalent and you can run either
• index.qmd get updated with the latest release while the oldest release is removed

Bravo all around! 🥇🥇🥇