notebook-01/Fix_Word_Doc_Claude.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/michellepace/ai-ml-notebook-explorations/blob/main/notebook-01/Fix_Word_Doc_Claude.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lKva1p6voV3J"
      },
      "source": [
        "# **1. About & Usage** 🟢"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## What this Notebook Does\n",
        "\n",
        ">Elevate your Word document with AI-powered proofreading that goes beyond standard spell-check. This notebook uses Anthropic's [Claude Sonnet model](https://www.anthropic.com/claude) to scan your Word documents, catching subtle language and style errors that often slip past Microsoft Word. Whether you're writing in English, German, Italian, or French, it preserves your intended meaning while polishing your prose to perfection—even for very large documents.\n",
        "\n",
        "<!-- Figure Clickable 700px Width -->\n",
        "<figure>\n",
        "<a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/01-what-this-notebook-does-using-claude.jpg\"\n",
        "target=\"_blank\">\n",
        "  <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/01-what-this-notebook-does-using-claude-trimmed.jpg\"\n",
        "  width=\"900\" alt=\"What this notebook does\" />\n",
        "</a>\n",
        "</figure>\n",
        "\n",
        "### Notebook Capabilities\n",
        "\n",
        "1. Uses the Claude API for intelligent text processing\n",
        "1. Handles large-scale text correction in multiple languages\n",
        "1. Highlights corrections in colour for easy identification\n",
        "1. Preserves original document semantic meaning and structure\n",
        "1. Includes comprehensive testing and evaluation\n",
        "\n",
        "### Showcasing Claude as my Mentor & Co-Creator\n",
        "\n",
        "This notebook also doubles as a demonstration. It showcases how Claude can help create tools - like this very notebook - with basic coding skills and no idea of where to start. I am a product manager by profession, and without Claude next to me, I would never have built this. The solution was designed by Claude and about 70% of the code written and then refined by Claude. That's phenomenal.\n",
        "\n",
        "<!-- Figure with caption 700px width -->\n",
        "<figure>\n",
        " <figcaption>Opening Scene: Explaining my situation and problem to Claude</figcaption>\n",
        " <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/02-initial-brainstorming-with-claude.jpg\" target=\"_blank\">\n",
        "   <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/02-initial-brainstorming-with-claude.jpg\"\n",
        "        width=\"700\"\n",
        "        alt=\"Opening Scene: Explaining my situation and problem to Claude\" />\n",
        " </a>\n",
        "</figure>\n",
        "\n",
        "---"
      ],
      "metadata": {
        "id": "9vuvZCaJZEfc"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f-IDYNNTg8cE"
      },
      "source": [
        "## Who this Notebook is for\n",
        "\n",
        "**Someone with a Word document:**\n",
        "\n",
        "That needs correction beyond what Word provides. In this case, you don’t need to read anything. Just run the Notebook, and wait for your corrected document to download.\n",
        "\n",
        "**The Curious Beginner:**\n",
        "\n",
        "If you're feeling like I was at the beginning of this project - knowing it is possible to programmatically use Claude but unsure where to begin - this Notebook is for you.\n",
        "\n",
        "**For People Good at Coding:**\n",
        "\n",
        "This Notebook showcases Claude's coding capabilities. About 70% of this Notebook was coded by Claude in response to my unsophisticated questions. Yes, there was a lot of refactoring, but say what you will, someone who knew very little still built something that works.\n",
        "\n",
        "**And lastly, for my sister:**\n",
        "\n",
        "Whose definition of AI was Canva and now happily Claude too. I know you won't make it this far, but here's hoping anyway.\n",
        "\n",
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Notebook Usage ⭐\n",
        "\n",
        "### ⬤ Running the Notebook</font>\n",
        "\n",
        "To run this Notebook, you must have an [Anthropic Pro account](https://www.anthropic.com/pricing) with $0.05+ [credit](https://console.anthropic.com/settings/plans) for API usage. You'll need to create an [Anthropic API key](https://console.anthropic.com/settings/keys) too. The Notebook will stop to guide you on how to do this in the [Configuration](#id-configuration) section. Steps are:\n",
        "\n",
        "\n",
        "1. Save your own copy of this Notebook: **File > Save a copy in Drive.** <font color='#FF1493'>*Important! This ensures 100% data privacy!*</font>\n",
        "\n",
        "1. Click **Runtime > Run all**\n",
        "\n",
        "1. Go to sections [Set Your Secret API Key⭐](#id-configuration) and [Set Your Word Document⭐](#id-configuration), enter your details and run again.\n",
        "\n",
        "### ⬤ Without an Anthropic Pro Account</font>\n",
        "\n",
        "No problem. **Each code block has a screenshot of example output beneath it**, showing how this Notebook works without you running it. Start by scanning the Table of Contents for an overview. Then focus on the explanations between code blocks – no need to read the code itself. You'll learn what I learned in twenty minutes, told in story form. You may find the [FAQ](#id-faq) section useful too.\n",
        "\n",
        "### ⬤ Example Input and Output</font>\n",
        "\n",
        "To give you an idea of how the notebook works, I've provided an example:\n",
        "\n",
        "- **Input file:** [MyWordDoc.docx](https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/example-input-output/MyWordDoc.docx) - sample Word document with various errors.\n",
        "- **Output file:** [MyWordDoc.docx.PROCESSED.html](https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/example-input-output/MyWordDoc.docx.PROCESSED.html) - corrected version with Claude's changes highlighted.\n",
        "- **All screenshots** that you'll find below each code block were made from these\n",
        "\n",
        "\n",
        "---"
      ],
      "metadata": {
        "id": "TidnrdSDJSJj"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gr-biUvb5pTA"
      },
      "source": [
        "## Origin Story\n",
        "\n",
        "This project fell into my lap over a Sunday lunch that I wasn’t at. My sister's friend was wrapping up her research paper - a hefty 40,000-word beast in German. With the deadline looming, she was spending too much time looking for mistakes Word had missed.\n",
        "\n",
        "So, I decided to create this Notebook to use Claude to correct her Word documents. What makes it different from using the [normal Claude chat interface](https://claude.ai/) is that you can see the entire corrected document, and the colour corrections are very easy to spot.\n",
        "\n",
        "Did I know how to build something like this? No. But I surmised (correctly) that with Claude's help, I would figure it out.\n",
        "\n",
        "<!-- /Figure simple not clickable -->\n",
        "<figure>\n",
        "  <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/03-origin-lunch-munich.jpg\"\n",
        "       alt=\"Origin of the project: A lunch I wasn't at\" />\n",
        "</figure>\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sq1vbrVA8S8n"
      },
      "source": [
        "## Claude as my Co-Creator\n",
        "\n",
        "Claude may have generated close to 70% of the code in this Notebook. And it is true that everything I learned I learned through my interactions with Claude. But it started, very importantly, with two people.\n",
        "\n",
        "After a few conversations with Claude about my goal, I sketched out my best guess of the \"code process\". Two strangers from an online community, Unmesh and Aaron, generously agreed to take my video call. Unmesh validated my approach, and Aaron inspired me to programmatically test the corrected document. But what I truly gained were two people who, without knowing it, had provided me with emotional backing. A sincere thank-you.\n",
        "\n",
        "<!-- Figure with caption 300px width -->\n",
        "<figure>\n",
        " <figcaption>Beginner's Blueprint: Imagining the coding journey ahead</figcaption>\n",
        " <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/04-sketch-in-guessing-the-notebook-workflow.jpg\" target=\"_blank\">\n",
        "   <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/04-sketch-in-guessing-the-notebook-workflow.jpg\"\n",
        "        width=\"300\"\n",
        "        alt=\"Beginner's Blueprint: Imagining the coding journey ahead\" />\n",
        " </a>\n",
        "</figure>\n",
        "\n",
        "I then turned to Claude as my AI co-creator for this Notebook. This marked the shift from concept to development. Claude became my constant companion, sitting beside me every day in my browser. I created a dedicated [Anthropic project](https://support.anthropic.com/en/articles/9517075-what-are-projects) for our chats and gave Claude specific instructions to enhance his assistance:\n",
        "\n",
        "<!-- /Figure with caption 300px width-->\n",
        "<figure>\n",
        " <figcaption>Character Creation: Tweaking Claude's expertise for our project</figcaption>\n",
        " <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/05-telling-claude-who-he-is.jpg\"\n",
        "    target=\"_blank\">\n",
        "   <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/05-telling-claude-who-he-is.jpg\"\n",
        "        width=\"300\"\n",
        "        alt=\"Character Creation: Tweaking Claude's expertise for our project\" />\n",
        " </a>\n",
        "</figure>\n",
        "\n",
        "Collaborating with Claude was crucial in every aspect of developing this Notebook. Here are a very small handful of examples:\n",
        "\n",
        "- **Problem-solving:** Claude designed the solution and helped me improve it (see section [Claude's Solution Picture](#id-claude-solution-picture))\n",
        "- **Technical guidance:** Introduced me to Python libraries to streamline the coding process.\n",
        "- **API integration:** Provided instructions for connecting to Claude programmatically.\n",
        "- **Code development:** Wrote about 70% of the code and then improved it.\n",
        "-\t**Testing:** Helped me create a testing plan and generated test data for prompt testing (see section [Test My Prompt](#id-test-my-prompt))\n",
        "\n",
        "I learned an enormous amount, especially when I argued with Claude. I’m a much better coder now too. You probably wouldn’t have thought so. Discussing concepts before diving into syntax made the coding process much easier and faster.\n",
        "\n",
        "If you are new to AI development, don't be intimidated. When I started this project, I was starting near square one too. This Notebook is a testament to the power of AI-assisted learning and code generation.\n",
        "\n",
        "### Why Claude?\n",
        "\n",
        "Because the Claude Sonnet model is the most capable, put simply, he is the best. I trust Anthropic more than any other option. And I love their branding - typography, colours, simplicity, writing style - everything.\n",
        "\n",
        "<!-- /Figure with caption 750px width-->\n",
        "<figure>\n",
        " <figcaption>Performance Timeline: Charting Claude's rise in the AI landscape</figcaption>\n",
        " <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/06-claude-compared-to-other-options.jpg\"\n",
        "    target=\"_blank\">\n",
        "   <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/06-claude-compared-to-other-options.jpg\"\n",
        "        width=\"750\"\n",
        "        alt=\"Performance Timeline: Charting Claude's rise in the AI landscape\" />\n",
        " </a>\n",
        "</figure>\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vn23LWeaQ1S9"
      },
      "source": [
        "<a name=\"id-claude-solution-picture\"></a>\n",
        "## Claude's Solution Picture\n",
        "\n",
        "This was on my first day of Notebook development with Claude. It provides a glimpse of how I brainstormed with him.\n",
        "\n",
        "<!-- /Figure with caption 300px width-->\n",
        "<figure>\n",
        " <figcaption>Dialogue to Design: Capturing our initial problem-solving conversation</figcaption>\n",
        " <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/07-solution-brainstorm-extensive.jpg\"\n",
        "    target=\"_blank\">\n",
        "   <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/07-solution-brainstorm-extensive.jpg\"\n",
        "        width=\"300\"\n",
        "        alt=\"Dialogue to Design: Capturing our initial problem-solving conversation\" />\n",
        " </a>\n",
        "</figure>\n",
        "\n",
        "\n",
        "<!-- /Figure with caption 300px width-->\n",
        "<figure>\n",
        "  <figcaption>Visual Thinking: Claude's initial notebook solution</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/09-solution-picture-claude.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/09-solution-picture-claude.jpg\"\n",
        "         width=\"300\"\n",
        "         alt=\"Digital Drafting: Claude's visual take on our solution\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "<!-- /Figure with caption 500px width-->\n",
        "<figure>\n",
        "  <figcaption>Cemented Understanding: Making the solution easier to see</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "         width=\"500\"\n",
        "         alt=\"emented understanding: Happens when I redraw pictures myself\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lK40_oEOhamp"
      },
      "source": [
        "# <font color=\"#5830D9\">**2. Setup**</font> 🟣"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fgHMX8RGs6W-"
      },
      "source": [
        "## Python Libraries\n",
        "\n",
        "Before I dive into actually “doing the work”, first I need to install and import the Python libraries this Notebook will use. Libraries are code written by other people which means there's less code for me to write. Many of these libraries were either suggested by Claude or I found them through Google."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8rfyDWRetEbu"
      },
      "outputs": [],
      "source": [
        "# Install libraries\n",
        "try:\n",
        "    !pip install --upgrade-strategy only-if-needed --quiet \\\n",
        "        anthropic \\\n",
        "        langchain \\\n",
        "        markdown \\\n",
        "        numpy \\\n",
        "        python-docx \\\n",
        "        scikit-learn \\\n",
        "        sentence-transformers \\\n",
        "        strip-markdown \\\n",
        "        tqdm\n",
        "    print(\"Success! All required libraries are installed.\")\n",
        "\n",
        "except Exception as e:\n",
        "    print(f\"An error occurred during library installation: {str(e)}\")\n",
        "\n",
        "# Import libraries\n",
        "try:\n",
        "    from google.colab import drive, files # Request access to your Word Document in Colab\n",
        "    from google.colab import userdata # Request access to your secure Colab Secret: ANTHROPIC_API_KEY\n",
        "    import docx # Read and write Word documents; extract paragraphs and create test Word docs.\n",
        "    from pathlib import Path # Handle files easily and concisely\n",
        "    import anthropic # Interact with Claude Model via Anthropic's API\n",
        "    from langchain.text_splitter import MarkdownTextSplitter # Split markdown text into chunks\n",
        "    from strip_markdown import strip_markdown # Remove markdown for content comparison\n",
        "    from markdown import markdown # Convert corrected markdown file into a pretty HTML page\n",
        "    import re # Analyze text using regular expressions\n",
        "    from collections import Counter # To easily print test results\n",
        "    import numpy as np # Process numerical data for preservation scores\n",
        "    from tqdm.auto import tqdm # Display progress bar for chunk processing\n",
        "    from sentence_transformers import SentenceTransformer # Transform text chunks into numeric vectors that represent their meaning\n",
        "    from sklearn.metrics.pairwise import cosine_similarity # Calculates semantic similarity between text chunk vectors\n",
        "    print(\"Success! Library imports are complete.\")\n",
        "\n",
        "except Exception as e:\n",
        "    print(f\"An error occurred during library importation: {str(e)}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xbRMlgx9klcp"
      },
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1-Pvw3IHrkl-hWEj6PlWRwB5Bdd4x5cWz\"\n",
        "/>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "0uf_0g0u0WCu"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bZ4gmzmTwBOF"
      },
      "source": [
        "<a name=\"id-configuration\"></a>\n",
        "\n",
        "## Configuration\n",
        "\n",
        "Specify your Word document and Anthropic API Key here. Other AI related global settings have been defined here too (eg prompting, models, chunk size etc.) as well as working files."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Set Your Secret API key ⭐ { vertical-output: true, display-mode: \"form\" }\n",
        "#@markdown **Instructions:** Run this block and follow the step-by-step guidance.<br>\n",
        "\n",
        "#@markdown\n",
        "\n",
        "anthropic_api_secret_name = 'ANTHROPIC_API_KEY'  # @param {type: \"string\"}\n",
        "\n",
        "def validate_anthropic_api_key(api_key):\n",
        "    if not api_key.startswith('sk-'):\n",
        "        raise ValueError(\"Anthropic API keys start with \\\"sk-\\\"\")\n",
        "    if ' ' in api_key:\n",
        "        raise ValueError(\"Anthropic API keys don't have white spaces.\")\n",
        "    if len(api_key) <= 100:\n",
        "        raise ValueError(\"Anthropic API keys are longer than 100 characters.\")\n",
        "\n",
        "def get_anthropic_api_key(secret_name):\n",
        "    try:\n",
        "        api_key = userdata.get(secret_name)\n",
        "        validate_anthropic_api_key(api_key)\n",
        "        print(\"Success!\")\n",
        "        print(f\"• Colab secret '{secret_name}' found.\")\n",
        "        print(f\"• If the secret contains a valid API Key, we can connect to Claude.\")\n",
        "        print(f\"• To change API Key: Click the \\\"key\\\" icon, delete '{secret_name}', rerun this block.\")\n",
        "        return api_key\n",
        "\n",
        "    except userdata.SecretNotFoundError:\n",
        "        print(f\"🛑 Error: Colab secret '{secret_name}' not found in your Colab environment\")\n",
        "        print(\" To fix:\")\n",
        "        print(f\" 1. Click the \\\"key\\\" icon on the left of this Notebook\")\n",
        "        print(f\" 2. Add new secret with name '{secret_name}'\")\n",
        "        print(f\" 3. Set value to Anthropic API key from: https://console.anthropic.com/settings/keys\")\n",
        "        print(f\" 4. Rerun this block and follow next instructions\")\n",
        "        print(\" About Colab secrets: https://bit.ly/4cad0v7\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except userdata.NotebookAccessError:\n",
        "        print(f\"🛑 Error: You denied this Notebook access to your Colab secret '{secret_name}'\")\n",
        "        print(\" To fix:\")\n",
        "        print(\" 1. Rerun this block and click \\\"Grant access\\\"\")\n",
        "        print(\" About Colab secrets: https://bit.ly/4cad0v7\")\n",
        "        print(\" Worried about safety? Save your own copy of this Notebook and run that.\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except ValueError as ve:\n",
        "        print(f\"🛑 Error: Invalid format, {str(ve)}\")\n",
        "        print(\" To fix:\")\n",
        "        print(f\" 1. Click the \\\"key\\\" icon on the left of this Notebook\")\n",
        "        print(f\" 2. Delete '{anthropic_api_secret_name}'\")\n",
        "        print(f\" 4. Rerun this block and follow next instructions\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except Exception as e:\n",
        "        print(\"🛑 Unexpected error occurred\")\n",
        "        print(\" Please check:\")\n",
        "        print(f\" 1. '{secret_name}' secret exists in Colab (click \\\"key\\\" icon on the left)\")\n",
        "        print(\" 2. Secret value is a valid Anthropic API key\")\n",
        "        print(\" Get API key: https://console.anthropic.com/settings/keys\")\n",
        "        print(\" About Colab secrets: https://bit.ly/4cad0v7\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "\n",
        "\n",
        "### Do the work\n",
        "MY_ANTHROPIC_API_KEY = get_anthropic_api_key(anthropic_api_secret_name)"
      ],
      "metadata": {
        "id": "aI0KXrLc1Xg_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1DlZ-6qr7Lljwr4KeBC4d0xRPJM5P6JKE\"\n",
        "/>"
      ],
      "metadata": {
        "id": "weldgB8yFwz7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "ZsJuR5cGyERX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Set Your Word Document ⭐ { vertical-output: true, display-mode: \"form\" }\n",
        "#@markdown **Instructions:** Specify the Word document you want corrected.\n",
        "\n",
        "#@markdown **Step 1:** Upload the Word document to your [Google Drive](https://drive.google.com/drive/my-drive)<br>\n",
        "#@markdown **Step 2:** Input the full name of the Word document file (see example)<br>\n",
        "#@markdown **Step 3:** Run this cell by clicking the little \"play\" icon just under the title<br>\n",
        "\n",
        "#@markdown **Example** Google Drive files start with <font color='#FF1493'>/content/​drive/​MyDrive/</font>\n",
        "#@markdown just like: `/content/drive/MyDrive/my-animal-folder/pony.docx`\n",
        "\n",
        "my_input_docx_file = '/content/drive/MyDrive/TESTX/DONE.docx' #@param {type:\"string\", placeholder:\"(here is an example)   /content/drive/MyDrive/MyWordDoc.docx\"}\n",
        "\n",
        "def validate_google_drive_docx(file_path: str) -> Path:\n",
        "    \"\"\"\n",
        "    Validate the input Word document file in Google Drive.\n",
        "\n",
        "    :param file_path: String path to Word document on Google Drive\n",
        "    :return: Path object of the validated file\n",
        "    :raises: Various exceptions for invalid cases\n",
        "    \"\"\"\n",
        "\n",
        "    try:\n",
        "        # Check if input is empty\n",
        "        if not file_path.strip():\n",
        "            raise ValueError(\"No file path provided in input box\")\n",
        "\n",
        "        # Mount Google Drive if not already mounted\n",
        "        if not Path('/content/drive').exists():\n",
        "            try:\n",
        "                drive.mount('/content/drive')\n",
        "            except Exception as e:\n",
        "                if \"credential propagation was unsuccessful\" in str(e).lower():\n",
        "                    raise PermissionError(\"You denied access to Google Drive.\")\n",
        "                else:\n",
        "                    raise e\n",
        "\n",
        "        file_path = Path(file_path)\n",
        "\n",
        "        # Check if the file exists\n",
        "        if not file_path.exists():\n",
        "            raise FileNotFoundError(f\"This file doesn't exist in your Google Drive: '{file_path}'\")\n",
        "\n",
        "        # Check if the file has a .docx extension\n",
        "        if file_path.suffix.lower() != '.docx':\n",
        "            raise ValueError(f\"This file doesn't have a .docx extension: '{file_path}'\")\n",
        "\n",
        "        # Check if the file is not empty\n",
        "        if file_path.stat().st_size == 0:\n",
        "            raise ValueError(f\"This file is empty: '{file_path}'\")\n",
        "\n",
        "        # Check if the file can be opened as a Word document\n",
        "        try:\n",
        "            docx.Document(file_path)\n",
        "        except Exception as e:\n",
        "            raise ValueError(f\"This file couldn't be opened: '{file_path}'. Error: {str(e)}\")\n",
        "\n",
        "        print(\"Success!\")\n",
        "        print(f\"• Word document found: {file_path.absolute()}\")\n",
        "        print(f\"• I'll be sending this to Claude for correction\")\n",
        "        return file_path\n",
        "\n",
        "    except ValueError as ve:\n",
        "        print(f\"🛑 Error: {str(ve)}\")\n",
        "        print(\"To fix:\")\n",
        "        if str(ve) == \"No file path provided in input box\":\n",
        "            print(\" 1. You must input your Word document file name in the input box above.\")\n",
        "            print(\" 2. Read and follow the each step above.\")\n",
        "        else:\n",
        "            print(\" 1. Ensure the file is a valid .docx document\")\n",
        "            print(\" 2. Check if the file is not corrupted or empty\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except PermissionError as pe:\n",
        "        print(f\"🛑 Error: {str(pe)}\")\n",
        "        print(\"To fix:\")\n",
        "        print(\" 1. Rerun this block and click \\\"Connect to Google Drive\\\"\")\n",
        "        print(\" Worried about safety? Save your own copy of this Notebook and run that.\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except FileNotFoundError as fnf:\n",
        "        print(f\"🛑 Error: {str(fnf)}\")\n",
        "        print(\"To fix:\")\n",
        "        print(\" 1. In Input Box instructions, look at the pink Example given\")\n",
        "        print(\" 2. Verify your file exists in your Google Drive on that exact path\")\n",
        "        print(\" 3. Remember, file paths and names are case-sensitive\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "    except Exception as e:\n",
        "        print(\"🛑 Unexpected error occurred\")\n",
        "        print(f\"Error details: {str(e)}\")\n",
        "        print(\"Sorry... really don't know how you got here.\")\n",
        "        print(\"🛑🛑🛑\\n\")\n",
        "        raise\n",
        "\n",
        "\n",
        "### Do the work\n",
        "try:\n",
        "    my_input_docx_file = validate_google_drive_docx(my_input_docx_file)\n",
        "except Exception:\n",
        "    print(\"Please update the file path and run this block again.\")\n",
        "    raise"
      ],
      "metadata": {
        "id": "ozUKbVZZAd8Y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1EAZoEYYfLXAlG1YM_A4F3THMWXBIWygM\"\n",
        "/>\n"
      ],
      "metadata": {
        "id": "TXsM1Jd70isC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "ymJNjWtByH4r"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IZ5GCMvxxVD-"
      },
      "source": [
        "<a name=\"id-ai-settings\"></a>\n",
        "### AI Settings\n",
        "\n",
        "Nothing to be done here. Settings for prompting, Claude, (and working files)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Working files (Not AI settings, just lives here)"
      ],
      "metadata": {
        "id": "F4uVrCl0TGYM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Output Files (not AI Settings)\n",
        "original_md_file = Path(f\"{my_input_docx_file}.ORIG.md\") # Your Word doc extracted into markdown format (.md).\n",
        "processed_md_file = Path(f\"{my_input_docx_file}.PROCESSED.md\") # Your Word doc with all the corrections, in markdown format.\n",
        "processed_html_file = Path(f\"{my_input_docx_file}.PROCESSED.html\") # Your Word doc with all Claude's corrections, in HTML format for easy reading."
      ],
      "metadata": {
        "id": "syY3VgO4TJuz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- AI settings (excluding prompt)"
      ],
      "metadata": {
        "id": "KTvoFdf2TM3m"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Chunking and Similarity\n",
        "TARGET_CHUNK_CHARACTERS = 4000 # The target chunk size (in characters) to split your Word Document in.\n",
        "SIMILARITY_LARGE_LANGAUGE_MODEL = 'paraphrase-multilingual-mpnet-base-v2' # LLM for checking meaning similarity between chunks\n",
        "\n",
        "# Everything Claude:\n",
        "CLAUDE_MODEL  = \"claude-3-5-sonnet-20240620\"\n",
        "PROMPT_TEMP = 0  # Low temperature for more probable and consistent output (0 to 1)\n",
        "\n",
        "MY_ANTHROPIC_CLIENT = anthropic.Anthropic(\n",
        "    api_key=MY_ANTHROPIC_API_KEY, # Set in Notebook section: \"Set Your API key ⭐\"\n",
        "    max_retries=2,  # Maximum retry attempts per API request (text chunk)\n",
        "    timeout=20.0   # Timeout in seconds for each individual API request (text chunk)\n",
        ")"
      ],
      "metadata": {
        "id": "HGDF9mDYTSAp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " - AI settings (prompt for Claude)\n",
        "\n",
        " This is all the work Claude is going to do for us. A prompt is how you tell Claude what to do."
      ],
      "metadata": {
        "id": "eag2BFBNTWj4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "PROMPT_TEMPLATE = \"\"\"\n",
        "CRITICAL: PROVIDE ONLY THE CORRECTED TEXT WITHOUT ANY ADDITIONAL COMMENTARY.\n",
        "\n",
        "Your task is to take the provided text and rewrite it into a clear, grammatically correct version while preserving the original meaning as closely as possible. Correct any spelling mistakes, punctuation errors, verb tense issues, word choice problems, and other grammatical mistakes.\n",
        "\n",
        "MANDATORY INSTRUCTIONS:\n",
        "\n",
        "1. Determine and use the same linguistic language as the original text (e.g., English, German)\n",
        "2. Preserve all existing markdown formatting, including heading levels, paragraphs, and lists\n",
        "3. Make necessary grammatical corrections, including spelling, punctuation, verb tense, word choice, and other grammatical issues. Only make stylistic changes if essential for clarity\n",
        "4. Mark corrections with markdown syntax, apply one of these choices only:\n",
        "   - For changed text use bold: e.g., **changed** and **multiple changed words**\n",
        "   - For new text use bold: **new words**\n",
        "   - For removed text use bold strikethrough: **~~removed words~~**\n",
        "5. Maintain the original structure:\n",
        "   - Don't add new lines of text\n",
        "   - Don't include additional commentary at all\n",
        "   - Don't convert markdown elements to different types\n",
        "6. For ambiguous corrections, choose the option that best preserves original meaning and style\n",
        "7. Ensure consistency in corrections throughout the text\n",
        "8. Return the corrected text in markdown syntax\n",
        "9. DO NOT add any explanations, introductions, or conclusions to your response\n",
        "\n",
        "FINAL REMINDER: Your output should consist SOLELY of the corrected text. Do not include phrases like \"Here is the corrected text\" or any other form of commentary.\n",
        "\n",
        "The text to be corrected is provided between the triple tildes (~~~):\n",
        "\n",
        "~~~\n",
        "{the_markdown_chunk}\n",
        "~~~\n",
        "\n",
        "REMEMBER: Provide ONLY the corrected text without any additional words or explanations.\"\"\""
      ],
      "metadata": {
        "id": "2lJ16ASB0mbd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Confirmirmation of what has just been configured."
      ],
      "metadata": {
        "id": "X7IpwIeIgMxr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Success! All configuration complete:\")\n",
        "print(\" • Your Anthropic API Key⭐\")\n",
        "print(\" • Your Word document (Google Drive)⭐\")\n",
        "print(\" • AI settings (Working files. Not AI settings, just lives there.)\")\n",
        "print(\" • AI settings (excluding prompt)\")\n",
        "print(\" • AI settings (prompt for Claude)\")"
      ],
      "metadata": {
        "id": "AhEauatfTg3F"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1FUChoqUCvxWYTn6ziSxQU5B5l91owvzC\" />"
      ],
      "metadata": {
        "id": "_IXtbyJtRiYm"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "lBZIoLFdyLS4"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iMrMjXeujDY3"
      },
      "source": [
        "# <font color=\"#5830D9\">**3. Pre-processing**</font> 🟣"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EZ_FxLrRvsJy"
      },
      "source": [
        "Initially I thought I could simply send your entire Word document to Claude for correction. But this isn't possible when you are using Claude programmatically. In place of this, Claude told me I need to first do the below \"pre-processing\" steps:\n",
        "\n",
        "1.\t**Extract** the text from your Word document\n",
        "1.\t**Convert** it into markdown format\n",
        "1.\t**Split** the Markdown text into smaller pieces (which I'll call \"**chunks**\")\n",
        "\n",
        "When these steps are done, we'll move on to sending each \"text chunk\" to Claude for processing. In the case of this Notebook, processing means correcting the text in each chunk. The sections which follow explain and implement these steps."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qHzZF0d8_Nyq"
      },
      "source": [
        "## Extract Word doc\n",
        "\n",
        "In this  first step, I extract the text from your Word document. I make sure to maintain the same headings, bullet lists, and paragraphs as in your document. For simplicity, I ignored everything else like text found in tables, images, charts, headers and footers. **The output below the code** shows a summary of the paragraphs that were extracted from your Word document."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YFKT4cmQ_2F6"
      },
      "outputs": [],
      "source": [
        "def extract_docx_paragraphs(docx_file: Path) -> list[dict]:\n",
        "    \"\"\"\n",
        "    Extract non-empty paragraphs from a Word document.\n",
        "\n",
        "    :param docx_file: Path object pointing to the Word document file (.docx)\n",
        "    :return: List of dictionaries, each containing 'text', 'style', 'heading_level', and 'word_count' of a paragraph\n",
        "    \"\"\"\n",
        "    doc = docx.Document(docx_file)\n",
        "\n",
        "    def extract_heading_level(paragraph):\n",
        "        if paragraph.style.name.startswith('Heading'):\n",
        "            try:\n",
        "                return int(paragraph.style.name.split()[-1])\n",
        "            except ValueError:\n",
        "                pass\n",
        "        return None\n",
        "\n",
        "    return [\n",
        "        {\n",
        "            'text': para.text,\n",
        "            'style': para.style.name,\n",
        "            'heading_level': extract_heading_level(para),\n",
        "            'word_count': len(para.text.split())\n",
        "        }\n",
        "        for para in doc.paragraphs\n",
        "        if para.text.strip()\n",
        "    ]\n",
        "\n",
        "def print_paragraph_summary(paragraphs: list[dict]) -> None:\n",
        "    \"\"\"\n",
        "    Print a summary of paragraphs extracted from a Word document.\n",
        "\n",
        "    :param paragraphs: List of dictionaries containing paragraph information\n",
        "    \"\"\"\n",
        "    style_counts = Counter(para['style'] for para in paragraphs)\n",
        "    total_word_count = sum(para['word_count'] for para in paragraphs)\n",
        "\n",
        "    # Column width constants\n",
        "    STYLE_WIDTH = 20\n",
        "    HEADING_LEVEL_WIDTH = 12\n",
        "    PARAGRAPH_COUNT_WIDTH = 16\n",
        "    WORD_COUNT_WIDTH = 16\n",
        "\n",
        "    # Total width of the table\n",
        "    TABLE_WIDTH = STYLE_WIDTH + HEADING_LEVEL_WIDTH + PARAGRAPH_COUNT_WIDTH + WORD_COUNT_WIDTH\n",
        "\n",
        "    print(f\"Success!\")\n",
        "    print(f\" • Document extracted into {len(paragraphs)} paragraphs with {len(style_counts)} styles\")\n",
        "    print(\"-\" * TABLE_WIDTH)\n",
        "\n",
        "    # Print header\n",
        "    print(f\"{'Word Style':<{STYLE_WIDTH}}\"\n",
        "          f\"{'HeadingLevel':^{HEADING_LEVEL_WIDTH}}\"\n",
        "          f\"{'No.Paragraphs':>{PARAGRAPH_COUNT_WIDTH}}\"\n",
        "          f\"{'No.Words':>{WORD_COUNT_WIDTH}}\")\n",
        "    print(\"-\" * TABLE_WIDTH)\n",
        "\n",
        "    # Print rows\n",
        "    for style, count in sorted(style_counts.items()):\n",
        "        style_info = next(para for para in paragraphs if para['style'] == style)\n",
        "        heading_level = str(style_info['heading_level'] or \"\")\n",
        "        word_count = sum(para['word_count'] for para in paragraphs if para['style'] == style)\n",
        "        print(f\"{style:<{STYLE_WIDTH}}\"\n",
        "              f\"{heading_level:^{HEADING_LEVEL_WIDTH}}\"\n",
        "              f\"{count:>{PARAGRAPH_COUNT_WIDTH},}\"\n",
        "              f\"{word_count:>{WORD_COUNT_WIDTH},}\")\n",
        "\n",
        "    # Print footer\n",
        "    unique_styles = f\"({len(style_counts)} unique styles)\"\n",
        "    print(\"-\" * TABLE_WIDTH)\n",
        "    print(f\"{unique_styles:<{STYLE_WIDTH}}\"\n",
        "          f\"{'':<{HEADING_LEVEL_WIDTH}}\"\n",
        "          f\"{len(paragraphs):>{PARAGRAPH_COUNT_WIDTH},}\"\n",
        "          f\"{total_word_count:>{WORD_COUNT_WIDTH},}\")\n",
        "\n",
        "\n",
        "### Do the work\n",
        "docx_paragraphs = extract_docx_paragraphs(my_input_docx_file)\n",
        "print(f\"Extracted document: {my_input_docx_file.absolute()}\\n\")\n",
        "print_paragraph_summary(docx_paragraphs)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1MPdPh5vQfdsLHbPLlIhVmNZjTpaZr4aX\"\n",
        "/>"
      ],
      "metadata": {
        "id": "osG6eiW2YIaJ"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sMlv6PpbR1KR"
      },
      "source": [
        "## Convert to Markdown\n",
        "\n",
        "After extracting the text, I convert it into [markdown format]( https://markdownguide.offshoot.io/getting-started) and save it as a markdown file (.md). Here's why I chose markdown:\n",
        "\n",
        "1. I can preserve the structure of your Word document - like headings, bullet lists, and paragraphs - in a text-based format. This means the corrected document can have the same structure.\n",
        "1. Claude can format corrections in bold, and later I can add colour.\n",
        "1. It's a clean format that large language models can read efficiently (unlike HTML).\n",
        "1. Rupert from Slack told me to (Claude initially recommended HTML and I believed him, but Rupert was right).\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wxnllZMwSOoL"
      },
      "outputs": [],
      "source": [
        "def create_simple_markdown_file(paragraphs: list[dict], markdown_file: Path) -> Path:\n",
        "    \"\"\"\n",
        "    Convert paragraphs to Markdown format and save to a file.\n",
        "\n",
        "    :param paragraphs: List of dictionaries containing paragraph information\n",
        "    :param markdown_file: Path to save the generated Markdown file\n",
        "    :return: Path of the created Markdown file\n",
        "    \"\"\"\n",
        "    def format_paragraph(para):\n",
        "        if para['heading_level'] is not None:\n",
        "            return f\"{'#' * para['heading_level']} {para['text']}\"\n",
        "        elif para['style'].startswith('List'):\n",
        "            return f\"- {para['text']}\"\n",
        "        else:\n",
        "            return para['text']\n",
        "\n",
        "    # Filter out empty paragraphs and format the rest\n",
        "    formatted_paragraphs = [format_paragraph(para) for para in paragraphs if para['text'].strip()]\n",
        "\n",
        "    markdown_content = []\n",
        "    for i, current_para in enumerate(formatted_paragraphs):\n",
        "        if i > 0:\n",
        "            prev_is_list = formatted_paragraphs[i-1].startswith(\"- \")\n",
        "            current_is_list = current_para.startswith(\"- \")\n",
        "            # Single newline for consecutive list items, double for others\n",
        "            markdown_content.append(\"\\n\" if prev_is_list and current_is_list else \"\\n\\n\")\n",
        "\n",
        "        markdown_content.append(current_para)\n",
        "\n",
        "    # Join paragraphs and normalise spacing:\n",
        "    markdown_content = \"\".join(markdown_content)\n",
        "    # Remove consecutive empty lines and strip leading/trailing whitespace\n",
        "    markdown_content = re.sub(r'\\n{3,}', '\\n\\n', markdown_content.strip())\n",
        "\n",
        "    markdown_file.write_text(markdown_content, encoding='utf-8')\n",
        "    return markdown_file\n",
        "\n",
        "\n",
        "### Do the work\n",
        "original_md_file = create_simple_markdown_file(docx_paragraphs, original_md_file)\n",
        "print(f\"Success!\\n Your Word document text has been saved as a markdown file:\\n '{original_md_file.absolute()}'\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1gesXZuj99cqn6I54kuR-RH4YPR0xK_JT\"\n",
        "/>"
      ],
      "metadata": {
        "id": "Shkt4KkfvXwN"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aSa0VIFvV43S"
      },
      "source": [
        "## Split into Chunks\n",
        "\n",
        "Now that I’ve converted all the extracted text from your Word document into one markdown file, we’re onto splitting this file into little chunks. So instead of one big file, I might end up with, say, 12 chunks that, if joined back together, would be identical to the markdown file. I do this so I can send these smaller chunks to Claude for correction, one by one. This picture explains it very well:\n",
        "\n",
        "<!-- /Figure with caption 500px width-->\n",
        "<figure>\n",
        "  <figcaption>Chunking: From big file into little text chunks</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "         width=\"500\"\n",
        "         alt=\"Splitting your Word document into little text chunks to send to Claude\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "You might wonder, **\"Why not just send everything to Claude at once?\"** I wondered the same. Claude has a limit on how much text he can send back in reply to any prompt (4,096 tokens of text is the Claude's max.). So, if I sent all the markdown text at once, a lot of the content would be left out in Claude's corrected reply.\n",
        "\n",
        "Small chunks are also quite handy. They make it easy to compare the \"original chunk\" with the \"processed chunk\" I get back from Claude. For instance, I can quickly spot if the semantic meaning of the corrected chunk has changed compared to the original chunk. It's much easier to notice oddities with little chunks than one enormous chunk. Plus, if something goes wrong, I'd rather have a small chunk fail than the entire document.\n",
        "\n",
        "**The output below the code** shows a summary of the chunks your Word document got split into. The number of chunks is determined by the target chunk size I set. More on that below if you are interested.\n",
        "\n",
        "### Why I chunked with characters (not tokens)\n",
        "\n",
        "I chose to split the text into chunks based on character count rather than tokens. For instance, “split the document into chunks each about 1,000 characters long.” When working with large language models, it is far more common to split text by tokens, not character count. If you're curious as to why I chose characters (instead of tokens), here's the explanation:\n",
        "\n",
        "**First, let's talk about \"tokens.\"** They're just a way to measure text size, like kilograms measure a person's weight. But here's the twist: while a kilogram is the same everywhere, tokens aren't. Each large language model has its own way of counting text size using its own “tokeniser”. So \"87 Anthropic Claude tokens\" isn't the same amount of text as \"87 OpenAI GPT tokens\". Language models don’t measure the size of text by counting words or characters like we do.\n",
        "\n",
        "Given that Claude is limited to replying with no more than 4,096 \"Anthropic tokens”, it would obviously make sense to split text by counting tokens. So why did I use characters instead?\n",
        "\n",
        "I got a bit lazy. My text is in Markdown format, and I didn't want to break apart Markdown elements accidentally when splitting the text into chunks. I found a library that splits Markdown text while respecting its structure. The problem is that this library only allows me to specify target chunk size in characters (not tokens). Still, I thought it a better workaround than writing more code. As in, I thought doing rough math estimates was easier.\n",
        "\n",
        "### How I determined my target chunk size (in characters)\n",
        "\n",
        "To determine my ideal chunk size (in characters), I first analysed token, character, and word counts on random text I generated with Claude. I used the Anthropic Tokeniser to count the tokens.\n",
        "\n",
        "<!-- /Figure with caption full width-->\n",
        "<figure>\n",
        "  <figcaption>Estimating Conversion: Between words, characters, and Anthropic Tokens</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/11-sizing-text-by-counting-words-characters-and-anthropic-tokens.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/11-sizing-text-by-counting-words-characters-and-anthropic-tokens.jpg\"\n",
        "         alt=\"Estimating conversion rates between words, characters, and Anthropic Tokens\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "Then I did some math:\n",
        "\n",
        "1. Claude can return a maximum of 4,096 tokens per chunk\n",
        "1. My equation: (prompt size) + (chunk size) + (wiggle room for corrections) <= 4,096 tokens\n",
        "1. Anthropic’s tokeniser measured my prompt (without a chunk attached) at 405 tokens.\n",
        "1. I decided 500 tokens for correction wiggle room would be enough.\n",
        "1. So: (405 tokens) + (chunk size) + (500 tokens) <= 4,096 tokens\n",
        "1. This means that my chunk size must be no more than 3,191 tokens\n",
        "\n",
        "And now using the data above to approximate:\n",
        "\n",
        "- 3,191 tokens x 5.89 characters = 18,795 characters\n",
        "\n",
        "But hold on, 18,795 characters in English is about 2,565 words which is 5 pages in Microsoft Word. I want smaller chunks to pinpoint subtle changes more easily. In my prompt I instruct Claude to “make corrections but retain original meaning.” Having smaller chunks to compare means I can detect changes in meaning with more sensitivity.\n",
        "\n",
        "**I chose 4,000 characters as my target chunk size because** this is about 545 words (just less than one page in Word) and about (4,000/5.89) 679 Anthropic tokens. Remember my wiggle of 500 tokens? This is about 400 words which is more than enough as I’m only sending Claude 545 words for correction anyhow. My numbers are rough approximations using the sample data above. But with a target chunk size of 4,000 characters (or 679 tokens) there are 2,512 tokens of room left anyhow:\n",
        "\n",
        "- So: (405 tokens) + (chunk size: 679 tokens) + (500 tokens) <= 4,096 tokens\n",
        "\n",
        "The only downside I see in using small chunks is a little more cost. Costs for API usage are calculated based on the number of tokens used, not per prompt. Because I’m sending more prompts than I need to, it just means I’m sending 405 tokens (i.e., the prompt measured without the chunk) more times than I have to as I'll be sending more chunks. That’s okay.\n",
        "\n",
        "**In the output below the code**, you’ll see a summary of how your Word document got chunked.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "l7GP2q8luuBF"
      },
      "outputs": [],
      "source": [
        "def split_markdown_into_chunks(\n",
        "    markdown_file: Path,\n",
        "    target_chunk_size_chars: int = 2000,\n",
        "    chunk_overlap: int = 0\n",
        ") -> list[str]:\n",
        "    \"\"\"\n",
        "    Split a Markdown file into text chunks using MarkdownTextSplitter.\n",
        "\n",
        "    :param markdown_file: Path to the Markdown file\n",
        "    :param target_chunk_size_chars: Target size of each chunk in characters\n",
        "    :param chunk_overlap: Number of overlapping characters between chunks\n",
        "    :return: List of text chunks (that don't exceed the target chunk size)\n",
        "    \"\"\"\n",
        "    text = markdown_file.read_text(encoding='utf-8')\n",
        "    text_splitter = MarkdownTextSplitter(chunk_size=target_chunk_size_chars, chunk_overlap=chunk_overlap)\n",
        "    chunks = text_splitter.split_text(text)\n",
        "    print(f\"Success!\")\n",
        "    print(f\"• The markdown file containg your Word document text has been split into {len(chunks)} manageable chunks.\")\n",
        "    print(f\"• Each chunk aims to be around {target_chunk_size_chars:,} characters long (never more).\")\n",
        "    print(f\"• These chunks are now ready to be sent to Claude for processing (ie correction).\\n\\n\")\n",
        "    return chunks\n",
        "\n",
        "def count_claude_tokens(text):\n",
        "    \"\"\"Count tokens using Anthropic's tokenizer for Claude.\"\"\"\n",
        "    return MY_ANTHROPIC_CLIENT.count_tokens(text)\n",
        "\n",
        "def print_chunking_summary(chunks: list[str]) -> None:\n",
        "    chunk_sizes = [len(chunk) for chunk in chunks]\n",
        "    chunk_sizes_tokens = [count_claude_tokens(chunk) for chunk in chunks]\n",
        "    avg_size_chars = sum(chunk_sizes) / len(chunks)\n",
        "    avg_size_tokens = sum(chunk_sizes_tokens) / len(chunks)\n",
        "\n",
        "    print(\"Summary of All Chunks:\")\n",
        "    print(\"~\" * 58)\n",
        "    print(f\"Total Chunks: {len(chunks):7}\")\n",
        "    print(f\"   Avg.Chunk size: {avg_size_chars:>7,.0f} chars | {avg_size_tokens:>4,.0f} anthropic tokens\")\n",
        "    print(f\"   Min.Chunk size: {min(chunk_sizes):>7,} chars | {min(chunk_sizes_tokens):>4,} anthropic tokens\")\n",
        "    print(f\"   Max.Chunk size: {max(chunk_sizes):>7,} chars | {max(chunk_sizes_tokens):>4,} anthropic tokens\\n\\n\")\n",
        "\n",
        "def print_chunk_table(chunks: list[str]) -> None:\n",
        "    print(\"Summary per Chunk:\")\n",
        "    print(\"~\" * 108)\n",
        "    print(f\"{'':36}{'Anthropic':>15}\")\n",
        "    print(f\"{'Original':<10}{'Lines':>10}{'Words':>10}{'Chars':>10}{'Tokens':>11}    {'Original Chunk Start':<70}\")\n",
        "    print(\"~\" * 108)\n",
        "\n",
        "    total_lines = total_words = total_chars = total_tokens = 0\n",
        "\n",
        "    for i, chunk in enumerate(chunks, 1):\n",
        "        chunk_num = i\n",
        "        lines = len(chunk.splitlines())\n",
        "        words = len(chunk.split())\n",
        "        chars = len(chunk)\n",
        "        tokens = count_claude_tokens(chunk)\n",
        "        chunk_start = chunk.replace('\\n', ' ').replace('\\r', '')[:50] + \"...\"\n",
        "\n",
        "        total_lines += lines\n",
        "        total_words += words\n",
        "        total_chars += chars\n",
        "        total_tokens += tokens\n",
        "\n",
        "        print(f\"Chunk {chunk_num:<4}{lines:>10,}{words:>10,}{chars:>10,}{tokens:>11,}    {chunk_start:<70}\")\n",
        "\n",
        "    print(\"~\" * 108)\n",
        "    print(f\"{'Total':<10}{total_lines:>10,}{total_words:>10,}{total_chars:>10,}{total_tokens:>11,}\")\n",
        "\n",
        "\n",
        "### Do the work\n",
        "original_chunks = split_markdown_into_chunks(original_md_file, target_chunk_size_chars=TARGET_CHUNK_CHARACTERS)\n",
        "print_chunking_summary(original_chunks)\n",
        "print_chunk_table(original_chunks)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1rIhgyr2WDAvBrl6DjAwmXqg8gGy3HMmY\"\n",
        "/>"
      ],
      "metadata": {
        "id": "TSvKK1-RxBww"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NnT_BhdtcqVb"
      },
      "source": [
        "- Uncomment the below to see what is inside a chunk"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_M4DJY_0c4um"
      },
      "outputs": [],
      "source": [
        "def print_chunks(chunks: list[str], chunks_to_print: list[int] | None = None) -> None:\n",
        "    \"\"\"\n",
        "    Print specified chunks from a list of text chunks.\n",
        "\n",
        "    :param chunks: List of text chunks to be printed\n",
        "    :param chunks_to_print: Optional list of chunk numbers to print. If None, all chunks are printed\n",
        "    :return: None\n",
        "    \"\"\"\n",
        "    total_chunks = len(chunks)\n",
        "\n",
        "    if chunks_to_print is None:\n",
        "        chunks_to_print = list(range(1, total_chunks + 1))\n",
        "\n",
        "    for chunk_number, chunk in enumerate(chunks, start=1):\n",
        "        if chunk_number in chunks_to_print:\n",
        "            print(f\"🟨 Chunk {chunk_number} contains:{'🟨' * 33}\")\n",
        "            print(chunk)\n",
        "            print(\"🟨\" * 9)\n",
        "\n",
        "    # Check for requested chunks that don't exist\n",
        "    for requested_chunk in chunks_to_print:\n",
        "        if requested_chunk > total_chunks:\n",
        "            print(f\"🟥 Chunk {requested_chunk} does not exist - there are {total_chunks} chunks.\")\n",
        "\n",
        "\n",
        "# Example Usage\n",
        "# Try print chunk 1, chunk 105, chunk 122\n",
        "# print_chunks(original_chunks, [1, 105, 122])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HUa8UQ9Kbqg5"
      },
      "source": [
        "- Uncomment the below if you want to count words, characters, and tokens"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EQrBEVmnbzJZ"
      },
      "outputs": [],
      "source": [
        "def analyze_text_files_words_chars_tokens(file_names):\n",
        "    for file_name in file_names:\n",
        "        file_path = Path(file_name)\n",
        "\n",
        "        if not file_path.exists():\n",
        "            print(f\"File not found: {file_name}\")\n",
        "            continue\n",
        "\n",
        "        content = file_path.read_text(encoding='utf-8')\n",
        "\n",
        "        word_count = len(content.split())\n",
        "        char_count = len(content)\n",
        "        token_count = count_claude_tokens(content)\n",
        "        avg_chars_per_token = char_count / token_count if token_count > 0 else 0\n",
        "        avg_tokens_per_word = token_count / word_count if word_count > 0 else 0\n",
        "\n",
        "        print(f\"File name: {file_name.upper()}\")\n",
        "        print(f\"{'Word count:':<30} {word_count:>15,.2f}\")\n",
        "        print(f\"{'Character count:':<30} {char_count:>15,.2f}\")\n",
        "        print(f\"{'Anthropic token count:':<30} {token_count:>15,.2f}\")\n",
        "        print(f\"{'Avg. Characters per token:':<30} {avg_chars_per_token:>15.2f}\")\n",
        "        print(f\"{'Avg. Tokens per word:':<30} {avg_tokens_per_word:>15.2f}\")\n",
        "        print()\n",
        "\n",
        "\n",
        "# Example Usage\n",
        "# file_list = ['english.txt', 'german.txt', 'prompt.txt']\n",
        "# analyze_text_files_words_chars_tokens(file_list)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VqQcPe9Qu78I"
      },
      "source": [
        "# <font color=\"#5830D9\">**4. Processing**</font> 🟣"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bT0_0CjEvedY"
      },
      "source": [
        "## The Prompt (+chunk)\n",
        "\n",
        "To send an individual chunk to Claude, I embed it in the below prompt. That means, if there are say 20 chunks, then Claude will be prompted 20 times. Read the output below the code, most especially between the `~~~` triple tildes for a deeper explanation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3fMTLYsxv22X"
      },
      "outputs": [],
      "source": [
        "def print_prompt_with_chunk_example():\n",
        "    example_chunk = f\"\"\"# I'M A LITTLE EXAMPLE CHUNK OF MARKDOWN TEXT TELLING A STORY.\n",
        "- Chunks are placed in between the '~~~' band, this is exactly where I am right now.\n",
        "- Everything above or below the '~~~' band always stays the same for every chunk sent to Claude for processing.\n",
        "- To make a chunk, your Word document was converted into markdown format and then split into little 'pieces' (ie chunks)\n",
        "- To do the splitting, the chunk size was set to a maximum of {TARGET_CHUNK_CHARACTERS} characters (Michelle chose it).\n",
        "- When I get sent to Claude in this prompt, he'll apply this prompt to me, and then return an corrected version of me.\n",
        "- If your Word doc got spit into 19 chunks, then Claude will be prompted 19 times and return 19 corrected chunks.\n",
        "- When Claude has processed all of us chunks, we're reassembled together in order so we form one document again.\n",
        "- But remember we'll still in markdown format. So we get converted into a pretty HTML file just for you.\n",
        "- In this html file you'll be able to see all the corrections Claude made as instructed by this prompt.\n",
        "- Finally, looking at this pretty html file you can decide which changes to incorporate into your Word document.\n",
        "- Again, I'm just an **example** chunk. The real ones will be sent to Claude. Not me!\"\"\"\n",
        "\n",
        "    prompt_with_chunk = PROMPT_TEMPLATE.format(the_markdown_chunk=example_chunk)\n",
        "    print(prompt_with_chunk)\n",
        "\n",
        "\n",
        "### Do the work\n",
        "print(\"WHAT IS HAPPENING?\")\n",
        "print(\"- Below this SOLID line is the actual prompt that will be sent to Claude for every chunk.\")\n",
        "print(\"- In between the '~~~' is where an individual chunk is embedded into the prompt.\")\n",
        "print(\"- Confused? Read the story the little example chunk tells you in this prompt:\")\n",
        "print(\"_\" * 90, \"\\n\")\n",
        "\n",
        "print_prompt_with_chunk_example()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image: Thumbnail different to full image -->\n",
        "<a href=\"https://drive.google.com/file/d/1PI1aFEiQMWf03fjtlUhCjTRcDiwALNQF/view?usp=drive_link\"\n",
        "   target=\"_blank\">\n",
        "  <img src=\"https://drive.google.com/uc?export=view&id=1vxQFW-ZuKEkUqZflP9ikVDWmDYZ470sN\"\n",
        "  />\n",
        "</a>\n"
      ],
      "metadata": {
        "id": "ZNSEiGzw4pyM"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vhj6CQGSwpTM"
      },
      "source": [
        "## Connect to Claude\n",
        "\n",
        "If there was one part that scared me the most about attempting this Notebook, it’s this part. I’ve always struggled to understand API documentation. API stands for \"Application Programming Interface\" and because I am conversing with Claude programmatically, I am using the [Anthropic Python API](https://github.com/anthropics/anthropic-sdk-python/blob/main/README.md).\n",
        "\n",
        "To get help, I told Claude that he was an Anthropic API expert. Surprisingly, he generated code based on an old version of the API. So I attached the latest README file, told him to study it, and then to help me. Below is the starting prompt I used to start our conversation.\n",
        "\n",
        "<!-- /Figure with caption 900px width-->\n",
        "<figure>\n",
        "  <figcaption>Helping Claude: You're an API expert but read the help file</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/12-tell-claude-he-is-an-api-expert-and-give-him-the-help-file.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/12-tell-claude-he-is-an-api-expert-and-give-him-the-help-file.jpg\"\n",
        "         width=\"900\"\n",
        "         alt=\"Claude you're an API expert, but just read this help file quickly.\" />\n",
        "  </a>\n",
        "</figure>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MPfdOYU3xGkv"
      },
      "outputs": [],
      "source": [
        "def test_anthropic_connection(anthropic_client: anthropic.Anthropic) -> None:\n",
        "    \"\"\"\n",
        "    Test the connection to the Anthropic API by sending a simple message.\n",
        "\n",
        "    :param anthropic_client: An instance of the Anthropic client\n",
        "    :raises KeyboardInterrupt: If the connection fails\n",
        "    \"\"\"\n",
        "    my_test_prompt = \"Hello Claude, have I really connected to you?\"\n",
        "    try:\n",
        "        message = anthropic_client.messages.create(\n",
        "            model= CLAUDE_MODEL,\n",
        "            max_tokens=30, # Reduced for quick response\n",
        "            temperature=0.99,\n",
        "            messages=[\n",
        "                {\"role\": \"user\", \"content\": my_test_prompt}\n",
        "            ]\n",
        "        )\n",
        "        print(\"Success! API key is valid and working.\")\n",
        "        print(f\"  My prompt was:  {my_test_prompt}\")\n",
        "        print(f\"  Claude replied: {message.content[0].text}\")\n",
        "\n",
        "    except anthropic.APIError as e:\n",
        "        print(f\"API error occurred: {e}\")\n",
        "        raise KeyboardInterrupt(\"Connection test failed. Stopping execution 🛑.\") from e\n",
        "    except Exception as e:\n",
        "        print(f\"Unexpected error occurred: {e}\")\n",
        "        raise KeyboardInterrupt(\"Connection test failed. Stopping execution 🛑.\") from e\n",
        "\n",
        "\n",
        "### Do the work\n",
        "test_anthropic_connection(MY_ANTHROPIC_CLIENT)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1w545XP1bMkhsYEM1hjafLjaUokhihncE\"\n",
        "/>"
      ],
      "metadata": {
        "id": "0MI-oTxKAWXa"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4z68oCyMxNnC"
      },
      "source": [
        "## Process Chunks\n",
        "\n",
        "Let’s summarise where we are so far. We’ve extracted the text from your Word document, we’ve converted it into a markdown file, we’ve split that file into chunks of markdown text, and we’ve established we can connect to Claude using the Anthropic API.\n",
        "\n",
        "In this step I send each chunk one by one, embedded into the prompt, to Claude. The prompt instructs Claude to make corrections to the chunk. For each chunk, Claude then processes it and sends the corrected chunk back to me. I collect these chunks back in order because I’m going to join them all together again into one big document.\n",
        "\n",
        "In the code below, you'll see I got a bit carried away with exception handling and \"friendly error messages.\" Then I gold-plated it further by showing a fancy progress bar to keep you entertained while the chunks are processing.\n",
        "\n",
        "<!-- /Figure with caption 500px width-->\n",
        "<figure>\n",
        "  <figcaption>Processing Chunks: Send it, Fix it (Claude), Give it Back</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/10-solution-picture-my-drawing-is-nicer.jpg\"\n",
        "         width=\"500\"\n",
        "         alt=\"Processing text chunks with Claude: Send, Fix, Get Back\" />\n",
        "  </a>\n",
        "</figure>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MRlggTtTyKw1"
      },
      "outputs": [],
      "source": [
        "def process_all_chunks(chunks: list[str]) -> list[str]:\n",
        "    \"\"\"\n",
        "    Process all chunks of markdown text and show progress bar.\n",
        "\n",
        "    :param chunks: List of markdown text chunks\n",
        "    :return: List of processed text chunks (error message for failed chunks)\n",
        "    \"\"\"\n",
        "    processed_chunks = []\n",
        "    processing_desc = \"Processing: sending text chunks to Claude for correction! \"\n",
        "    for i, chunk in tqdm(enumerate(chunks, 1), total=len(chunks), desc=processing_desc, bar_format='{l_bar}{bar} {n_fmt}/{total_fmt} Chunks'):\n",
        "        result = process_one_chunk(i, chunk)\n",
        "        processed_chunks.append(result)\n",
        "\n",
        "    successful_chunks = sum(1 for chunk in processed_chunks if not chunk.startswith(\"ERROR - MISSING TEXT!\"))\n",
        "    total_chunks = len(chunks)\n",
        "\n",
        "    if successful_chunks == total_chunks:\n",
        "        print(f\"Processing complete: {successful_chunks}/{total_chunks} chunks processed successfully.\")\n",
        "    else:\n",
        "        print(f\"Processing complete: Warning!! Only {successful_chunks}/{total_chunks} chunks were processed successfully.\")\n",
        "\n",
        "    return processed_chunks\n",
        "\n",
        "def process_one_chunk(\n",
        "    chunk_count: int,\n",
        "    chunk: str,\n",
        "    client: anthropic.Anthropic = MY_ANTHROPIC_CLIENT,\n",
        "    model: str = CLAUDE_MODEL,\n",
        "    temperature: float = PROMPT_TEMP,\n",
        "    max_tokens: int = 4096 # This is Claude Sonnet's maxiumum size of text measured in tokens he is able to reply with.\n",
        ") -> str:\n",
        "    \"\"\"\n",
        "    Process a single chunk of markdown text using the Anthropic API.\n",
        "\n",
        "    :param chunk_count: The index of the current chunk\n",
        "    :param chunk: The markdown text to process\n",
        "    :param client: The Anthropic client (default: MY_ANTHROPIC_CLIENT)\n",
        "    :param model: The Claude model to use (default: CLAUDE_MODEL)\n",
        "    :param temperature: The temperature setting for the model (default: PROMPT_TEMP)\n",
        "    :param max_tokens: The maximum number of tokens in the response (default: PROMPT_MAX_RESPONSE_TOKENS)\n",
        "    :return: Processed text or error message if processing failed\n",
        "    \"\"\"\n",
        "    try:\n",
        "        response = client.messages.create(\n",
        "            model=model,\n",
        "            max_tokens=max_tokens,\n",
        "            temperature=temperature,\n",
        "            messages=[\n",
        "                {\n",
        "                    \"role\": \"user\",\n",
        "                    \"content\": [\n",
        "                        {\n",
        "                            \"type\": \"text\",\n",
        "                            \"text\": PROMPT_TEMPLATE.format(the_markdown_chunk=chunk)\n",
        "                        }\n",
        "                    ]\n",
        "                }\n",
        "            ]\n",
        "        )\n",
        "        if not response.content or not response.content[0].text:\n",
        "            raise ValueError(f\"Empty response from API for chunk {chunk_count}\")\n",
        "\n",
        "        return response.content[0].text\n",
        "\n",
        "    except Exception as e:\n",
        "        error_message = get_friendly_error_msg(e, chunk_count)\n",
        "        print(error_message)  # Print error message for logging\n",
        "        return error_message  # Return error message for failed chunks\n",
        "\n",
        "def get_friendly_error_msg(exception: Exception, chunk_count: int) -> str:\n",
        "    CHUNK_ERROR_TEMPLATE = \"ERROR (chunk {}) was not processed because {}\"\n",
        "\n",
        "    error_msgs = {\n",
        "        anthropic.AuthenticationError: \"there's an issue with your Anthropic API key\",\n",
        "        anthropic.PermissionDeniedError: \"your API key does not have permission to use the specified resource\",\n",
        "        anthropic.RateLimitError: \"your account has hit a Rate limit. It's measured in requests/minute, tokens/minute, and tokens/day\",\n",
        "        anthropic.APITimeoutError: \"Anthropic took too long to respond. Likely an issue on Anthropic's end.\",\n",
        "        anthropic.APIConnectionError: \"failed to connect to Anthropic's API. Perhaps a network issues on your end\",\n",
        "        anthropic.APIStatusError: \"Anthropic returned an unsuccessful status code\",\n",
        "        anthropic.APIError: \"a general Anthropic API error has occurred internal to Anthropic’s systems.\",\n",
        "    }\n",
        "\n",
        "    for error_type, msg in error_msgs.items():\n",
        "        if isinstance(exception, error_type):\n",
        "            error_msg = f\"{msg}: {exception}\"\n",
        "            break\n",
        "    else:\n",
        "        if isinstance(exception, ValueError) and \"Empty response\" in str(exception):\n",
        "            error_msg = f\"Anthropic returned an empty response for chunk {chunk_count} (maybe we sent an empty chunk?): {exception}\"\n",
        "        else:\n",
        "            error_msg = f\"Unexpected error on chunk {chunk_count}: {exception}\"\n",
        "\n",
        "    return CHUNK_ERROR_TEMPLATE.format(chunk_count, error_msg)\n",
        "\n",
        "\n",
        "### Do the work\n",
        "processed_chunks = process_all_chunks(chunks=original_chunks)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1eT3y7P5zg-pbLGkXVXib7q9qFUAVMxMk\"\n",
        "/>"
      ],
      "metadata": {
        "id": "o6Fvkr28EWKD"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P1eHTvPYyabB"
      },
      "source": [
        "# <font color=\"#5830D9\">**5. Post-Processing**</font> 🟣"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wFbbAbzLydvC"
      },
      "source": [
        "At this stage, I've sent all the text chunks extracted from your Word document to Claude. He's processed each chunk as per the instructions in the prompt and sent each corrected chunk back to me. Now I need to reassemble these chunks and put them in a file format where the corrections are easy to spot (HTML).\n",
        "\n",
        "## Reassemble Processed Chunks\n",
        "\n",
        "This step is straightforward, just join the processed chunks back together. Since I collected the processed chunks from Claude in the same order I sent them, they fit right back into their original spots, following the flow of your Word document. The result? One big markdown file containing all the corrected text. It's essentially your original document, but now with Claude's corrections neatly incorporated.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4tA2N5jQyroO"
      },
      "outputs": [],
      "source": [
        "def reassemble_chunks(chunks: list[str], output_file: Path) -> Path:\n",
        "    \"\"\"\n",
        "    Reassemble processed chunks of text and save to an output file.\n",
        "\n",
        "    This function joins the processed chunks with double newlines, removes\n",
        "    excessive empty lines, trims leading and trailing whitespace, and saves\n",
        "    the result to the specified output file.\n",
        "\n",
        "    :param chunks: List of processed markdown text chunks\n",
        "    :param output_file: Path to the output file where the reassembled text will be saved\n",
        "    :return: Path to the output file containing the reassembled text\n",
        "    \"\"\"\n",
        "    # Join chunks with double newlines\n",
        "    reassembled_chunks = \"\\n\\n\".join(chunks)\n",
        "\n",
        "    # Remove consecutive empty lines\n",
        "    reassembled_chunks = re.sub(r'\\n{3,}', '\\n\\n', reassembled_chunks)\n",
        "\n",
        "    # Remove leading and trailing whitespace\n",
        "    reassembled_chunks = reassembled_chunks.strip()\n",
        "\n",
        "    # Save the corrected document\n",
        "    output_file.write_text(reassembled_chunks, encoding='utf-8')\n",
        "\n",
        "    return output_file\n",
        "\n",
        "\n",
        "### Do the work\n",
        "processed_md_file = reassemble_chunks(processed_chunks, processed_md_file)\n",
        "print(f\"Success!\")\n",
        "print(f\"• All processed chunks from Claude have been saved into one file:\")\n",
        "print(f\"  {processed_md_file.absolute()} \")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1GWaDYj-Sp_iuc0JfxgrwcjOODBtiFJfD\"\n",
        "/>"
      ],
      "metadata": {
        "id": "_BqG_wbGFVwe"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pw7lTpmEyzBk"
      },
      "source": [
        "## Create Pretty HTML File\n",
        "\n",
        "Now that I have a markdown file that contains all the processed chunks, we’re ready for this step. I convert this file into an HTML file so it's both easy to spot the corrections and pretty. All corrections are both bold and, now thanks to HTML, can also be in colour.  The original structure of your Word document has also been maintained.\n",
        "\n",
        "**The code below** saves your corrected Word document as an HTML file into your Google Drive. It also downloads this file automatically to your computer; open it in any browser to see the corrections.\n",
        "\n",
        "<!-- /Figure with caption 650px width-->\n",
        "<figure>\n",
        "  <figcaption>Getting Claude to style your corrected document</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/13-asking-claude-to-style-the-corrected-document-html-file.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/13-asking-claude-to-style-the-corrected-document-html-file.jpg\"\n",
        "         width=\"650\"\n",
        "         alt=\"Getting Claude to style your corrected document.\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "But my work isn’t done. I’m onto testing: the output file, the prompt itself, and my functional code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "suixxCj4y9oD"
      },
      "outputs": [],
      "source": [
        "def convert_markdown_to_html(markdown_file: Path, html_file: Path) -> Path:\n",
        "    \"\"\"\n",
        "    Convert a Markdown file to HTML with custom styling and preprocessing.\n",
        "\n",
        "    :param markdown_file: Path object representing the input Markdown file\n",
        "    :param html_file: Path object representing the output HTML file\n",
        "    :return: Path object of the output HTML file\n",
        "    \"\"\"\n",
        "    # Read the Markdown file\n",
        "    markdown_text = markdown_file.read_text(encoding='utf-8')\n",
        "\n",
        "    # Convert custom Markdown strikethrough syntax to HTML, handling both regular and bold strikethroughs\n",
        "    def process_strikethrough(match):\n",
        "        content = match.group(2)\n",
        "        is_bold = bool(match.group(1))\n",
        "        if is_bold:\n",
        "            return f\"<strong><del>{content}</del></strong>\"\n",
        "        else:\n",
        "            return f\"<del>{content}</del>\"\n",
        "\n",
        "    strikethrough_pattern = r'(\\*\\*)?~~(.*?)~~(\\*\\*)?'\n",
        "    markdown_text = re.sub(strikethrough_pattern, process_strikethrough, markdown_text)\n",
        "\n",
        "    # Convert to HTML\n",
        "    html_body = markdown(markdown_text)\n",
        "\n",
        "    # Define CSS styles, including the improved responsive design\n",
        "    css_styles = \"\"\"\n",
        "    <style>\n",
        "        body {\n",
        "            margin: 0 auto;\n",
        "            padding: 0 5%;\n",
        "            max-width: 50em;\n",
        "            line-height: 1.5em;\n",
        "            font-family: 'Inter', Arial, sans-serif;\n",
        "            font-size: 16px;\n",
        "            background-color: #F0EFEA;\n",
        "            color: #141413;\n",
        "        }\n",
        "\n",
        "        @media (max-width: 768px) {\n",
        "            body {\n",
        "                padding: 0 3%;\n",
        "            }\n",
        "        }\n",
        "\n",
        "        strong {\n",
        "            color: #E46264;\n",
        "        }\n",
        "    </style>\n",
        "    \"\"\"\n",
        "\n",
        "    # Create the full HTML document\n",
        "    full_html = f\"\"\"\n",
        "    <!DOCTYPE html>\n",
        "    <html lang=\"en\">\n",
        "    <head>\n",
        "        <meta charset=\"UTF-8\">\n",
        "        <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n",
        "        <title>Corrected Document</title>\n",
        "        <link href=\"https://fonts.googleapis.com/css2?family=Inter:wght@400;700&display=swap\" rel=\"stylesheet\">\n",
        "        {css_styles}\n",
        "    </head>\n",
        "    <body>\n",
        "        {html_body}\n",
        "    </body>\n",
        "    </html>\n",
        "    \"\"\"\n",
        "\n",
        "    # Write the HTML to a file\n",
        "    html_file.write_text(full_html, encoding='utf-8')\n",
        "\n",
        "    return html_file\n",
        "\n",
        "def download_drive_file(file_path: Path):\n",
        "    file_path = Path(file_path)\n",
        "\n",
        "    if not file_path.is_file():\n",
        "        print(f\"File not found: {file_path}\")\n",
        "        return\n",
        "\n",
        "    # Download file directly from Google Drive\n",
        "    files.download(str(file_path))\n",
        "\n",
        "    print(f\"• Browser download initiated...\")\n",
        "\n",
        "\n",
        "### Do the work\n",
        "processed_html_file = convert_markdown_to_html(processed_md_file, processed_html_file)\n",
        "print(f\"Success!\")\n",
        "print(f\"• Saved corrected Word document as: {processed_html_file.absolute()}\")\n",
        "\n",
        "download_drive_file(processed_html_file)\n",
        "print(f\"• Downloaded as: {processed_md_file.name}\")\n",
        "print(f\"• Enjoy the corrections Word missed, we're done!\")\n",
        "print(\"🙂\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1ogNcRikmq1N4-QHiBIPym_bMMCbDHzv_\"\n",
        "/>"
      ],
      "metadata": {
        "id": "AHosTJL9GioL"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03brUgd-scNQ"
      },
      "source": [
        "# <font color=\"#5830D9\">**6. Testing**</font> 🟣"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3lpVevOG7wSA"
      },
      "source": [
        "## Test plan with Claude\n",
        "\n",
        "After conversations with Claude, I decided to divide testing into three key areas:\n",
        "\n",
        "1. **Functional testing:** Verifies if specific code components, like markdown stripping, work correctly. This is traditional testing.\n",
        "1. **Prompt testing:** A new concept for me, crucial for refining the prompt and ensuring desired outcomes.\n",
        "1. **Output file testing:** Another new approach, focusing on preserving the original document's meaning in the corrected version.\n",
        "Below are some example questions I asked Claude during this process.\n",
        "\n",
        "<!-- /Figure with caption 750px width-->\n",
        "<figure>\n",
        "  <figcaption>Talking Testing: Deriving the test plan</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/14-talk-testing-with-claude.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/14-talk-testing-with-claude.jpg\"\n",
        "         width=\"750\"\n",
        "         alt=\"Talking testing with Claude and designing a test plan.\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "From there, I drew up my best guess of a test plan. The sections that follow will go into the details and implementation of each, but visually here it is:\n",
        "\n",
        "<!-- /Figure with caption 750px width-->\n",
        "<figure>\n",
        "  <figcaption>Test Plan: Applying what I learnt from Claude</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/15-sketched-out-test-plan-using-what-i-learnt.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/15-sketched-out-test-plan-using-what-i-learnt.jpg\"\n",
        "         width=\"750\"\n",
        "         alt=\"Sketched out test plan using what I learnt from Claude.\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5KpO_lr30Cn-"
      },
      "source": [
        "## 6.1 Test Processed Doc\n",
        "\n",
        "I tested my prompt, and Claude was quite dependable in retaining original semantic meaning when doing corrections. However, I still wanted an automated way to check that this holds true for every Word document corrected. So, I set up three different ways to evaluate the corrected text against your original text. These provide confidence that Claude has enhanced your writing without changing its core message, structure, or content.\n",
        "\n",
        "Here's what I'm checking in each section that follows:\n",
        "\n",
        "1. **Document structure:** Are the headings and paragraphs still in the right place?\n",
        "1. **Document content - Word count:** Has the overall length changed dramatically?\n",
        "1. **Document content - Semantic meaning:** Does the corrected text still mean the same as the original?\n",
        "\n",
        "**\"Michelle, so much testing is overkill,\"** I hear you saying. I quite disagree, and here's why: Imagine I decide to switch to the free Meta Llama 3.1 model instead of Claude Sonnet. How would I know if it's performing as well as Claude did? What if I have 20 large Word documents to correct? It wouldn't be practical to manually check each one. What if my document is in Italian? Should I assume Claude will perform as well, or should I have an automated way of testing it? Lastly, just because I've thoroughly tested one document, that doesn't imply the same level of dependability for another as content varies vastly.\n",
        "\n",
        "Manually testing for all these scenarios would be incredibly time-consuming. That's why these automated tests are so important to me. With very little effort, I can now have confidence in the quality of corrections across a wide range of documents and potential future changes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jw0qXt2ZHQ5V"
      },
      "source": [
        "### Structure Preservation\n",
        "\n",
        "This checks whether the overall structure of your Word document, like headings and bullet lists, remains intact in the corrected document. It's like making sure the skeleton hasn't been rearranged. Keep in mind that Claude might combine some paragraphs during correction. For example, if you accidentally started a new paragraph mid-sentence in your Word document, Claude would probably fix that. So don't judge an imperfect match too harshly. It's just an indication that something has changed. Overall, I don’t expect that too much will change. This is what I am checking."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XAWH69OyGz97"
      },
      "outputs": [],
      "source": [
        "def evaluate_document_structure_preservation(original_file: Path, processed_file: Path) -> str:\n",
        "    \"\"\"\n",
        "    Test if the document structure is preserved after processing.\n",
        "\n",
        "    :param original_file: Path to the original markdown file\n",
        "    :param processed_file: Path to the processed markdown file\n",
        "    :return: A string summarizing the evaluation results\n",
        "    \"\"\"\n",
        "    def count_markdown_elements(markdown_text: str) -> dict[str, int]:\n",
        "        \"\"\"\n",
        "        Extract the structure of a markdown document.\n",
        "        \"\"\"\n",
        "        lines = markdown_text.splitlines()\n",
        "        element_counts = {\n",
        "            'total_lines': len(lines),\n",
        "            'headings': 0,\n",
        "            'paragraphs': 0,\n",
        "            'list_items': 0,\n",
        "            'empty_lines': 0\n",
        "        }\n",
        "\n",
        "        heading_pattern = re.compile(r'^#+\\s')\n",
        "\n",
        "        for line in lines:\n",
        "            if not line:\n",
        "                element_counts['empty_lines'] += 1\n",
        "            if heading_pattern.match(line):\n",
        "                element_counts['headings'] += 1\n",
        "            elif line.startswith(('- ', '* ')):\n",
        "                element_counts['list_items'] += 1\n",
        "            elif line.strip():  # Only count non-empty lines as paragraphs\n",
        "                element_counts['paragraphs'] += 1\n",
        "\n",
        "        return element_counts\n",
        "\n",
        "    original_content = original_file.read_text(encoding='utf-8')\n",
        "    processed_content = processed_file.read_text(encoding='utf-8')\n",
        "\n",
        "    original_structure = count_markdown_elements(original_content)\n",
        "    processed_structure = count_markdown_elements(processed_content)\n",
        "\n",
        "    pass_fail = original_structure == processed_structure\n",
        "\n",
        "    result_summary = (\n",
        "        \"Evaluate Document Structure Preservation: Counting Markdown elements (entire documents)\\n\"\n",
        "        f\"{'~' * 125}\\n\"\n",
        "        f\"{'Document structure match!' if pass_fail else 'Document structure mismatch(!)'}\\n\"\n",
        "        f\" Extracted from Word doc:     {original_structure}\\n\"\n",
        "        f\" Assembled processed chunks:  {processed_structure}\"\n",
        "    )\n",
        "\n",
        "    return result_summary\n",
        "\n",
        "\n",
        "### Do the work\n",
        "result_eval_structure = evaluate_document_structure_preservation(original_md_file, processed_md_file)\n",
        "print(result_eval_structure)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image Clickable -->\n",
        "<a href=\"https://drive.google.com/file/d/1s_PLQf_wCArHzxG4LS6Ioc7SxVhak662/view?usp=drive_link\"\n",
        "   target=\"_blank\">\n",
        "  <img src=\"https://drive.google.com/uc?export=view&id=1s_PLQf_wCArHzxG4LS6Ioc7SxVhak662\"\n",
        "  />\n",
        "</a>"
      ],
      "metadata": {
        "id": "he9WeuNSHwqq"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xFPhBO_iHYdG"
      },
      "source": [
        "### Content Preservation: Simple Word Count\n",
        "\n",
        "Here, I do a quick comparison of word counts. If there's a big difference between the original and processed word count from Claude, it might mean the content has changed too much. It's a simple but effective first check that helps us spot any major unexpected changes in content. If you do see something unexpected, then it’s a flag to go and look. Overall, here too, I don’t expect word count to change by a lot, and this is what I am checking."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GGH9rX4jHd_b"
      },
      "outputs": [],
      "source": [
        "def word_count_comparison(\n",
        "    original_texts:list[str],\n",
        "    processed_texts: list[str],\n",
        ") -> list[dict[str, str | int | float]]:\n",
        "    \"\"\"\n",
        "    Compare word counts between lists of original and corrected texts.\n",
        "\n",
        "    :param original_texts: List of original text contents\n",
        "    :param processed_texts: List of corrected text contents\n",
        "    :return: List of dictionaries containing word count comparison information\n",
        "    \"\"\"\n",
        "    results = []\n",
        "    for chunk_number, (original_text, processed_text) in enumerate(zip(original_texts, processed_texts), 1):\n",
        "        original_count = len(original_text.split())\n",
        "        processed_count = len(processed_text.split())\n",
        "\n",
        "        # print(\"---ORIG\")\n",
        "        # print(original_text)\n",
        "        # print(\"---PROCE\")\n",
        "        # print(processed_text)\n",
        "\n",
        "        difference = processed_count - original_count\n",
        "        percentage_diff = (difference / original_count * 100) if original_count else 0\n",
        "\n",
        "        sign = '+' if difference >= 0 else ''\n",
        "\n",
        "        results.append({\n",
        "            \"chunk_number\": chunk_number,\n",
        "            \"similarity_test\": \"Word Count\",\n",
        "            \"original_count\": original_count,\n",
        "            \"corrected_count\": processed_count,\n",
        "            \"difference\": difference,\n",
        "            \"percentage_difference\": percentage_diff,\n",
        "            \"message_tabular\": f\"Word Count    Original: {original_count:4d} | Processed: {processed_count:4d} | Difference: {difference:+4d}   {percentage_diff:+.1f}%\",\n",
        "            \"message_arrow\": f\"Word Count: {original_count:4d} → {processed_count:4d}  |  {sign}{difference} words  {sign}{percentage_diff:.0f}%\"\n",
        "        })\n",
        "\n",
        "    return results\n",
        "\n",
        "def strip_off_markdown(chunks: list[str]) -> list[str]:\n",
        "    \"\"\"\n",
        "    Remove markdown formatting, specifically bold and strikethrough, from a list of text chunks.\n",
        "\n",
        "    This function processes each chunk to:\n",
        "    1. Delete words with bold and strikethrough markdown formatting (**~~text~~**)\n",
        "    2. Strip all other markdown formatting\n",
        "\n",
        "    :param chunks: List of text chunks potentially containing markdown formatting\n",
        "    :return: List of plain text chunks with markdown formatting removed\n",
        "    \"\"\"\n",
        "    def remove_bold_strikethrough_words(text):\n",
        "        # This pattern matches:\n",
        "        # 1. Start of string or a single space/tab (captured)\n",
        "        # 2. Bold and struck-through text (**~~any content~~**)\n",
        "        # 3. Optional single space or tab at the end\n",
        "        # 4. The first capturing group (1) is used in replacement, preserving leading space/tab if present\n",
        "        pattern = r'(^|[ \\t])(\\*\\*~~.*?~~\\*\\*)([ \\t])?'\n",
        "        return re.sub(pattern, r'\\1', text)\n",
        "\n",
        "    chunks_without_markdown = []\n",
        "    for chunk in chunks:\n",
        "        # Remove bold and strikethrough words because I prompted Claude not\n",
        "        # to delete words, but rather bold strikethrough them. Avoid false postive, remove!\n",
        "        text_without_bold_strikethrough = remove_bold_strikethrough_words(chunk)\n",
        "\n",
        "        plain_text = strip_markdown(text_without_bold_strikethrough)\n",
        "\n",
        "        chunks_without_markdown.append(plain_text)\n",
        "\n",
        "    return chunks_without_markdown\n",
        "\n",
        "def evaluate_content_preservation_by_word_count(\n",
        "    original_chunks: list[str],\n",
        "    processed_chunks: list[str],\n",
        ") -> str:\n",
        "    \"\"\"\n",
        "    Test content preservation by comparing word counts of original and processed chunks.\n",
        "\n",
        "    :param original_chunks: List of original text chunks\n",
        "    :param processed_chunks: List of processed text chunks\n",
        "    :return: A string summarizing the evaluation results\n",
        "    \"\"\"\n",
        "    results = word_count_comparison(original_chunks, processed_chunks)\n",
        "\n",
        "    result_summary = \"Evaluate Content Preservation: Word Count chunk to chunk (markdown syntax stripped out)\\n\"\n",
        "    result_summary += \"~\" * 96 + \"\\n\"\n",
        "\n",
        "    for result in results:\n",
        "        result_summary += f\"Chunk {result['chunk_number']:3d}    {result['message_tabular']}\\n\"\n",
        "\n",
        "    return result_summary\n",
        "\n",
        "\n",
        "### Do the work\n",
        "original_chunks_clean = strip_off_markdown(original_chunks)\n",
        "processed_chunks_clean = strip_off_markdown(processed_chunks)\n",
        "result_eval_content_word_count = evaluate_content_preservation_by_word_count(original_chunks_clean, processed_chunks_clean)\n",
        "print(result_eval_content_word_count)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image Clickable -->\n",
        "<a href=\"https://drive.google.com/file/d/1pCeIR4bIkPAw5OC0fnOlFfbx7_RFWGKe/view?usp=drive_link\"\n",
        "   target=\"_blank\">\n",
        "  <img src=\"https://drive.google.com/uc?export=view&id=1pCeIR4bIkPAw5OC0fnOlFfbx7_RFWGKe\"\n",
        "  />\n",
        "</a>"
      ],
      "metadata": {
        "id": "6Fq3mii7IfHf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Content Preservation: Semantic Similarity\n",
        "\n",
        "This is the most sophisticated check of the three, diving into the actual semantic similarity of the text. I wanted to ensure that Claude hadn't changed the original \"meaning\" of the text when making corrections."
      ],
      "metadata": {
        "id": "HJQCgH9vtMJd"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "okVE2XLHHkUH"
      },
      "source": [
        "Measuring semantic similarity can be complex, with many approaches available. I posted on a few discussion forums, and found many conflicting opinions. So I asked Claude to brainstorm options and generate example test data. Based on these results, I chose the sentence-transformers library, as it provided the most sensible scores against the dummy test data.\n",
        "\n",
        "Why this method? It's like having a well-read assistant who understands context and nuance, not just individual words. You can express the same idea in multiple ways, and this check ensures that the corrected text conveys the same meaning as the original. While not as sophisticated as models like Claude Sonnet or GPT-4, it effectively verifies that Claude retained the \"original meaning\" during corrections, as instructed in the prompt.\n",
        "\n",
        "When using sentence-transformers, you need to specify a language model for text comparison. I consulted Claude for options and selected the second-largest LLM on the list, \"paraphrase-multilingual-mpnet-base-v2.\" This model offers high accuracy across multiple languages, including English and German. Claude suggested the second option in the list below, which is half the size. I declined.\n",
        "\n",
        "<!-- /Figure with caption 800px width-->\n",
        "<figure>\n",
        "  <figcaption>Analysis Paralysis: Narrowing down  model options for testing semantic similarity</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/16-choosing-a-semantic-similarity-large-language-r2.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/16-choosing-a-semantic-similarity-large-language-r2.jpg\"\n",
        "         width=\"800\"\n",
        "         alt=\"Asking Claude for language model options to use with Sentence Transformer\"/>\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "Here's how Sentence-Transformer works:\n",
        "\n",
        "1. It loads the language model I selected\n",
        "1. Converts text chunks into number vectors (a way computers understand text)\n",
        "1. Compares the cosine angle between the two vectors in the given chunk pair (original and Claude-corrected chunk)\n",
        "\n",
        "The smaller the angle between these vectors are in dimensional space, the higher the similarity score (that is, the closer their \"meaning\"). A score of 100% within a \"chunk pair\" means identical meaning, while 0% indicates completely different content. I generally look for scores above 80% to be confident Claude has preserved the original meaning while making corrections. A 70% score might indicate that the general topic is the same, but some details or nuances have changed. It's not necessarily bad, but it signals that a closer look at that chunk may be needed.\n",
        "\n",
        "<!-- /Figure with caption 600px width-->\n",
        "<figure>\n",
        "  <figcaption>Semantic Similarity Visualised: Smaller angles = More similar meanings</figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/17-testing-for-semantic-similarity-(meaning)-explained.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/17-testing-for-semantic-similarity-(meaning)-explained.jpg\"\n",
        "         width=\"600\"\n",
        "         alt=\"Semantic Similarity Visualised: Smaller angles = More similar meanings\"/>\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "I also look for differences between all the \"chunk pairs.\" For example, if all score above 80% but one pair scores, say, 60%, that's a flag to take a quick look. It means the corrected chunk in that pair has changed much more compared to the original than in the other pairs. This isn't necessarily bad—perhaps Claude fixed a poorly written section quite substantially—but it's worth examining.\n",
        "\n",
        "Remember, these scores are guides, not absolute judgements. They help you focus on areas that might need attention, ensuring Claude has enhanced your text while staying true to the original message.\n",
        "\n",
        "**The code below** displays the similarity scores for each chunk pair, along with the beginning of each chunk to help you quickly locate it in your original Word document. **Don't worry about the warnings**; they just occur the first time the semantic model loads and don't affect similarity scoring.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "glymm-2MH115"
      },
      "outputs": [],
      "source": [
        "def sentence_transformer_similarity(texts1: list[str], texts2: list[str]) -> list[dict]:\n",
        "    \"\"\"\n",
        "    Compare similarity between lists of original and corrected texts using sentence transformers.\n",
        "\n",
        "    :param texts1: List of original text contents\n",
        "    :param texts2: List of corrected text contents\n",
        "    :return: List of dictionaries containing similarity comparison information for each chunk\n",
        "    \"\"\"\n",
        "    steps = [\n",
        "        \"Step 1/3 - Firing up a large language model... to be able to understand text for comparison\",\n",
        "        \"Step 2/3 - Turning text chunks into number vectors... makes it possible to compare text\",\n",
        "        \"Step 3/3 - Comparing original text chunks vs. Claude's corrections... super quick with these vectors\"\n",
        "    ]\n",
        "    with tqdm(total=3, desc=steps[0].split(\"...\", 1)[0], bar_format='{l_bar}{bar} {n_fmt}/{total_fmt}', mininterval=2.0) as pbar:\n",
        "        print()\n",
        "        for i, step in enumerate(steps, 1):\n",
        "            if i == 1:\n",
        "                model = SentenceTransformer(SIMILARITY_LARGE_LANGAUGE_MODEL)\n",
        "            elif i == 2:\n",
        "                embeddings1, embeddings2 = model.encode(texts1), model.encode(texts2)\n",
        "            elif i == 3:\n",
        "                similarities = cosine_similarity(embeddings1, embeddings2)\n",
        "\n",
        "            print(f\"DONE!\\t🌸{step}\")  # Print the full step description after performing the operation\n",
        "\n",
        "            pbar.update(1)\n",
        "\n",
        "            if i < 3:  # Update description for next step, except for the last iteration\n",
        "                pbar.set_description(f\"{steps[i].split('...', 1)[0]}\")\n",
        "        print()\n",
        "\n",
        "    similarity_scores = np.minimum(np.diag(similarities) * 100, 100)\n",
        "\n",
        "    return [\n",
        "        {\n",
        "            \"chunk_number\": i,\n",
        "            \"similarity_test\": \"Sentence Transformer\",\n",
        "            \"similarity_score\": score,\n",
        "            \"message\": f\"Similarity Score \\t({score:.0f}%) Original <> Processed\"\n",
        "        }\n",
        "        for i, score in enumerate(similarity_scores, 1)\n",
        "    ]\n",
        "\n",
        "def evaluate_content_preservation_by_similarity(original_chunks: list[str], processed_chunks: list[str]) -> str:\n",
        "    \"\"\"\n",
        "    Evaluate content preservation by comparing meaning similarity of original and processed chunks.\n",
        "\n",
        "    :param original_chunks: List of original text chunks\n",
        "    :param processed_chunks: List of processed text chunks\n",
        "    :return: A string summarizing the evaluation results, including similarity scores for each chunk\n",
        "    \"\"\"\n",
        "    results = sentence_transformer_similarity(original_chunks, processed_chunks)\n",
        "\n",
        "    result_summary = \"Evaluate Content Preservation: Semantic Meaning chunk to chunk (markdown syntax stripped out)\\n\"\n",
        "    result_summary += \"~\" * 96 + \"\\n\"\n",
        "\n",
        "    for result in results:\n",
        "        chunk_num = f\"{result['chunk_number']:>3}\"\n",
        "        score = f\"{result['similarity_score']:.0f}%\"\n",
        "        result_summary += f\"Chunk {chunk_num}   Similarity Score (original<>processed):    {score:>4}\\n\"\n",
        "\n",
        "    return result_summary\n",
        "\n",
        "\n",
        "### Do the work\n",
        "result_eval_content_similarity = evaluate_content_preservation_by_similarity(original_chunks_clean, processed_chunks_clean)\n",
        "print(result_eval_content_similarity)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image Clickable -->\n",
        "<a href=\"https://drive.google.com/file/d/1oIIo6cR-Ve1NhmR_0jZ2ey71sz1vq9lL/view?usp=drive_link\"\n",
        "   target=\"_blank\">\n",
        "  <img src=\"https://drive.google.com/uc?export=view&id=1oIIo6cR-Ve1NhmR_0jZ2ey71sz1vq9lL\"\n",
        "  />\n",
        "</a>"
      ],
      "metadata": {
        "id": "KERkIarJJJTw"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IguAj1a8oxvA"
      },
      "source": [
        "### All Results\n",
        "\n",
        "See all results for evaluating the output file in one place so that it is easier to make connections."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RoAgmjFlo-W_"
      },
      "outputs": [],
      "source": [
        "print(f\"{result_eval_structure}\\n\\n\")\n",
        "print(f\"{result_eval_content_word_count}\\n\")\n",
        "print(f\"{result_eval_content_similarity}\")\n",
        "\n",
        "print(\"\\n\\nREFERENCE - Use 'Chunk Start' for finding chunks in original Word document**\")\n",
        "print_chunk_table(original_chunks)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image: Thumbnail different to full image -->\n",
        "<a href=\"https://drive.google.com/file/d/1elh0pCwdRIhOz7fTQdVjCHXHb1n3SsDz/view?usp=drive_link\"\n",
        "target=\"_blank\">\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1CnfqsDxoDy6z3SPXLVh7hEDJe5AgYXV0\"\n",
        "/>\n",
        "</a>"
      ],
      "metadata": {
        "id": "INtGkg0JLh6T"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w-7iTYN0pA5A"
      },
      "source": [
        "- Uncomment the code below to investigate a particular chunk"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AJnRVWJDpKru"
      },
      "outputs": [],
      "source": [
        "# For example, say chunk 4 had an explotion in Word count and a low similarity score\n",
        "# Take a look at both the original and processed chunks to see what is going on:\n",
        "\n",
        "# print_chunks(original_chunks, [4])  # Uncomment me to run me\n",
        "# print_chunks(processed_chunks, [4]) # Uncomment me to run me"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fP2CfZqxnh2t"
      },
      "source": [
        "<a name=\"id-test-my-prompt\"></a>\n",
        "\n",
        "## 6.2 Test My Prompt\n",
        "\n",
        "Creating the perfect instructions (or \"prompt\") for Claude is a mix of creativity and precision. It took me a bit of trial and error to refine the prompt to get Claude to perform as I wanted him to. As I was tweaking my prompt, I kept manually retesting the same things. That is why I wrote these tests – to automate my manual tests so I could tweak faster with more confidence. They send example chunks to Claude to ensure that no matter the tweak, he is still doing the  very specific things I care about, like correcting inappropriate word choice or bolding a correction using markdown format.\n",
        "\n",
        "For more encompassing testing, of both Claude’s prompt and code together, see the section 'Evaluate the output file.' In these prompt tests, I isolate the testing to only the prompt. None of my other code is involved in any sort of way. That was important.\n",
        "\n",
        "**You'll see in the output below that there are failing tests**. Try as I might, I just couldn't get Claude to consistently detect British English from American English and make the appropriate corrections. As someone who prefers the Queen's English, I spent many hours trying to tweak my prompt. Finally, I gave up and removed all associated prompt instructions. I've given up for now. Claude must be American.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z2GF_PtvnqWD"
      },
      "outputs": [],
      "source": [
        "# Just a little helper function to help my testing\n",
        "def run_prompt_or_code_test(test_func):\n",
        "    test_name = test_func.__name__\n",
        "    try:\n",
        "        test_func()\n",
        "        print(f\"✅PASSED TEST: {test_name}\")\n",
        "    except AssertionError as e:\n",
        "        print(f\"🛑FAILED TEST: {test_name}: {str(e)}\")\n",
        "    except Exception as e:\n",
        "        print(f\"ERROR: {test_name} raised an unexpected error: {str(e)}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3qQwI1sVnyVi"
      },
      "outputs": [],
      "source": [
        "def show(test_case: dict[str, str], processed: str) -> str:\n",
        "    return f\"\"\"\n",
        "    Sent to Claude: {test_case['test']}\n",
        "    Expected Back:  {test_case['expect']}\n",
        "    Actual Back:    {processed}\n",
        "    \"\"\"\n",
        "\n",
        "def test_no_additional_commentary():\n",
        "    test_case = {\n",
        "        \"test\": \"This is a test sentence.\",\n",
        "        \"expect\": \"This is a test sentence.\"\n",
        "}\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert len(processed.split('\\n')) == 1, \"Additional lines were added to the text\" + show(test_case, processed)\n",
        "    assert processed == test_case['expect'], \"Text was added when it shouldn't have been\" + show(test_case, processed)\n",
        "\n",
        "def test_structure_maintenance():\n",
        "    test_case = {\n",
        "        \"test\": \"Paragraph 1.\\n\\nParagraph 2.\\n\\nParagraph 3.\",\n",
        "        \"expect\": \"Paragraph 1.\\n\\nParagraph 2.\\n\\nParagraph 3.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed.count('\\n\\n') == 2, \"Paragraph structure was not maintained\" + show(test_case, processed)\n",
        "    assert processed == test_case['expect'], \"Text was changed when it shouldn't have been\" + show(test_case, processed)\n",
        "\n",
        "def test_markdown_preservation():\n",
        "    test_case = {\n",
        "        \"test\": \"# Heading 1\\n## Heading 2\\n* List item 1\\n* List item 2\",\n",
        "        \"expect\": \"# Heading 1\\n## Heading 2\\n* List item 1\\n* List item 2\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"Markdown formatting was not preserved\" + show(test_case, processed)\n",
        "\n",
        "def test_spelling_correction():\n",
        "    test_case = {\n",
        "        \"test\": \"This senteence has a spelling error.\",\n",
        "        \"expect\": \"This **sentence** has a spelling error.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"Exact expected correction marking not made\" + show(test_case, processed)\n",
        "\n",
        "def test_dupe_words_are_bold_strike():\n",
        "    test_case = {\n",
        "        \"test\": \"This dog dog has blue eyes.\",\n",
        "        \"expect\": \"This **~~dog~~** dog has blue eyes.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"Duplicate word was not strikethrough bold\" + show(test_case, processed)\n",
        "\n",
        "def test_language_detection_german():\n",
        "    test_case = {\n",
        "        \"test\": \"Das ist ein Test. Es enthällllt einige Fehler.\",\n",
        "        \"expect\": \"Das ist ein Test. Es **enthält** einige Fehler.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"German text was not correctly identified or corrected\" + show(test_case, processed)\n",
        "\n",
        "def test_grammatical_corrections():\n",
        "    test_case = {\n",
        "        \"test\": \"She buyed some milk and go home.\",\n",
        "        \"expect\": \"She **bought** some milk and **went** home.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"Grammatical errors were not corrected\" + show(test_case, processed)\n",
        "\n",
        "def test_inappropriate_word_choice_easy():\n",
        "    test_case = {\n",
        "        \"test\":   \"He garnished support for perspective employees. His inciteful comments added value.\",\n",
        "        \"expect\": \"He **garnered** support for **prospective** employees. His **insightful** comments added value.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"Inappropriate word choice was not corrected as expected:\" + show(test_case, processed)\n",
        "\n",
        "def test_inappropriate_word_choice_creative():\n",
        "    test_case = {\n",
        "        \"test\":   \"She was literally dying of embarrassment as her speech was bad.\",\n",
        "        \"expect\": \"She was dying of embarrassment as her speech was bad.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert \"literally\" not in processed, \"Inappropriate word choice 'literally' was not removed\" + show(test_case, processed)\n",
        "    assert \"bad\" not in processed, \"Inappropriate word choice 'bad' was not removed\" + show(test_case, processed)\n",
        "\n",
        "def test_remove_bad_phrase_1():\n",
        "    test_case = {\n",
        "        \"test\":   \"The cat quickly ran fastly across the room.\",\n",
        "        \"expect\": \"The cat **~~quickly~~** ran **~~fastly~~** across the room.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert \" quickly ran fastly \" not in processed, \"Bad phrase 'quickly ran fastly' not corrected\" + show(test_case, processed)\n",
        "\n",
        "def test_remove_bad_phrase_2():\n",
        "    test_case = {\n",
        "        \"test\":   \"John forgot to remember to bring his lunch to work today.\",\n",
        "        \"expect\": \"John **~~forgot to remember to~~** **forgot to** bring his lunch to work today.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert \" forgot to remember \" not in processed, \"Bad phrase 'forgot to remember' not corrected\" + show(test_case, processed)\n",
        "\n",
        "def test_boldstrike_redundant_words():\n",
        "    test_case = {\n",
        "        \"test\":   \"She always never fails to disappoint her team with her exceptional work.\",\n",
        "        \"expect\": \"She **~~always~~** never fails to **~~disappoint~~** **impress** her team with her exceptional work.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert \"**~~always~~**\" in processed, \"Redundant word 'always' not boldstrike\" + show(test_case, processed)\n",
        "\n",
        "def test_detect_and_correct_british():\n",
        "    test_case = {\n",
        "        \"test\": \"The centre's staff analysed the colhour behaviour.\",\n",
        "        \"expect\": \"The centre's staff analysed the **colour** behaviour.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"British spelling was not preserved\" + show(test_case, processed)\n",
        "\n",
        "def test_detect_and_correct_american():\n",
        "    test_case = {\n",
        "        \"test\": \"The colourful catalog analyzed labor practices.\",\n",
        "        \"expect\": \"The **colorful** catalog analyzed labor practices.\"\n",
        "    }\n",
        "    processed = process_one_chunk(chunk_count=1, chunk=test_case['test'])\n",
        "    assert processed == test_case['expect'], \"American spelling was not preserved\" + show(test_case, processed)\n",
        "\n",
        "def run_all_prompt_tests():\n",
        "    tests = [\n",
        "        test_no_additional_commentary,\n",
        "        test_structure_maintenance,\n",
        "        test_markdown_preservation,\n",
        "        test_spelling_correction,\n",
        "        test_dupe_words_are_bold_strike,\n",
        "        test_language_detection_german,\n",
        "        test_grammatical_corrections,\n",
        "        test_inappropriate_word_choice_easy,\n",
        "        test_inappropriate_word_choice_creative,\n",
        "        test_remove_bad_phrase_1,\n",
        "        test_remove_bad_phrase_2,\n",
        "        test_boldstrike_redundant_words,\n",
        "        test_detect_and_correct_british,\n",
        "        test_detect_and_correct_american\n",
        "    ]\n",
        "    print(\"Test my Prompt to Claude: does he correct like my prompt asked him?\")\n",
        "    print(\"~\" * 70)\n",
        "    for test in tests:\n",
        "        run_prompt_or_code_test(test)\n",
        "\n",
        "\n",
        "### Do the work\n",
        "run_all_prompt_tests()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image: Clickable -->\n",
        "<a href=\"https://drive.google.com/file/d/1g2nKmuMsNlAC5Za8iDi_0gIX1r9f5MZQ/view?usp=drive_link\"\n",
        "   target=\"_blank\">\n",
        "  <img src=\"https://drive.google.com/uc?export=view&id=1g2nKmuMsNlAC5Za8iDi_0gIX1r9f5MZQ\"\n",
        "  />\n",
        "</a>\n"
      ],
      "metadata": {
        "id": "iVqkePnvOBi5"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iteD2sO7n_Dh"
      },
      "source": [
        "## 6.3 Test My Code\n",
        "\n",
        "These tests ensure my own code is working, regardless of how Claude performs. They use test data to check the parts of my code I was specifically worried about. By confirming my code works as intended, I can be confident that any issues in the final output are due to the prompt or Claude's response, not my code. They also become wonderfully useful when I asked Claude to refine my code."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jN7lSINaoHVJ"
      },
      "source": [
        "### Word Doc Extraction\n",
        "\n",
        "This test checks that the `extract_docx_paragraphs` function correctly pulls out all non-blank paragraphs from a Word document. I created a test.docx file just for this test.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7YldEsvYoSyH"
      },
      "outputs": [],
      "source": [
        "def test_extract_docx_paragraphs():\n",
        "\n",
        "    test_name = test_extract_docx_paragraphs.__name__\n",
        "\n",
        "    def create_word_docx_for_testing(file_name):\n",
        "        document = docx.Document()\n",
        "\n",
        "        # Are extracted from word doc\n",
        "        document.add_heading('Headings without levels are Paragraphs', 0)\n",
        "        document.add_heading('Headings at level 3 are Paragraphs', level=3)\n",
        "        document.add_paragraph('Plain paragraphs are Paragraphs')\n",
        "        document.add_paragraph('Intense quotes are Paragraphs', style='Intense Quote')\n",
        "        document.add_paragraph('Lists bullet point items are Paragraphs', style='List Bullet')\n",
        "        document.add_paragraph('', style='List Bullet') # Even if empty, the bullet is made for the list\n",
        "        document.add_paragraph('', style='List Number') # Even if empty, the number is made for the list\n",
        "\n",
        "        # Aren't extracted from word doc\n",
        "        document.add_paragraph('') # Empty paragraphs\n",
        "        table = document.add_table(rows=1, cols=2) # Tables\n",
        "        table.rows[0].cells[0].text = \"1st cell, left\" # Tables (empty or non-empty)\n",
        "        document.add_page_break() # Page breaks\n",
        "\n",
        "        document.save(file_name)\n",
        "        return file_name\n",
        "\n",
        "    docx_file = create_word_docx_for_testing('test.docx')\n",
        "\n",
        "    actual = extract_docx_paragraphs(docx_file)\n",
        "\n",
        "    expected = [\n",
        "        {'text': 'Headings without levels are Paragraphs', 'style': 'Title', 'heading_level': None, 'word_count':5},\n",
        "        {'text': 'Headings at level 3 are Paragraphs', 'style': 'Heading 3', 'heading_level': 3, 'word_count':6},\n",
        "        {'text': 'Plain paragraphs are Paragraphs', 'style': 'Normal', 'heading_level': None, 'word_count':4},\n",
        "        {'text': 'Intense quotes are Paragraphs', 'style': 'Intense Quote', 'heading_level': None, 'word_count':4},\n",
        "        {'text': 'Lists bullet point items are Paragraphs', 'style': 'List Bullet', 'heading_level': None, 'word_count':6}\n",
        "    ]\n",
        "\n",
        "    Path(docx_file).unlink() # delete file\n",
        "\n",
        "    assert len(actual) == len(expected), f\"Expected {len(expected)} paragraphs, actual paragraphs: {len(actual)}\"\n",
        "\n",
        "    for i in range(len(expected)):\n",
        "        assert actual[i] == expected[i], f\"\\n   Expected:\\n    {expected[i]}\\n   Actual:\\n    {actual[i]}\"\n",
        "\n",
        "\n",
        "\n",
        "### Do the work\n",
        "run_prompt_or_code_test(test_extract_docx_paragraphs)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1oMvMZ89I9syAxCp9Fc6QELzddHQ58A2K\"\n",
        "/>\n"
      ],
      "metadata": {
        "id": "aA-qTjc3O0Tj"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_cxsl_BDoheP"
      },
      "source": [
        "### Create Markdown File\n",
        "\n",
        "This test makes sure that the `create_simple_markdown_file` function correctly turns the extracted Word document paragraphs into a markdown file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "t3pDE1BeogvU"
      },
      "outputs": [],
      "source": [
        "def test_docx_paragraphs_to_markdown():\n",
        "    # Sample document content\n",
        "    docx_paras = [\n",
        "        {'text': 'Test Heading any', 'style': 'Heading 5', 'heading_level': 5},\n",
        "        {'text': 'Test paragraph 1', 'style': 'Normal', 'heading_level': None},\n",
        "        {'text': 'Test bullet point A', 'style': 'List (Paragraph)', 'heading_level': None},\n",
        "        {'text': 'Test bullet point B', 'style': 'List (any Paragraph whose Style begins with List)', 'heading_level': None},\n",
        "        {'text': 'Anything else', 'style': 'Any other Paragraph style that is not in the above', 'heading_level': None}\n",
        "    ]\n",
        "\n",
        "    expected = \"##### Test Heading any\\n\\nTest paragraph 1\\n\\n- Test bullet point A\\n- Test bullet point B\\n\\nAnything else\"\n",
        "\n",
        "    created = create_simple_markdown_file(docx_paras, Path('temporary_test_markdown_file.md'))\n",
        "    created_content = created.read_text(encoding='utf-8')\n",
        "    created.unlink() # Delete test file\n",
        "\n",
        "    assert created_content == expected, f\"\"\"\n",
        "---------- Expected ------------------------------:\n",
        "[{expected}]\n",
        "---------- Actual --------------------------------:\n",
        "[{created_content}]\n",
        "--------------------------------------------------\"\"\"\n",
        "\n",
        "\n",
        "### Do the work\n",
        "run_prompt_or_code_test(test_docx_paragraphs_to_markdown)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1-VvQd5urHNo3yq4yYr9HlOquPZcVU2F9\"\n",
        "/>"
      ],
      "metadata": {
        "id": "NxIv8_OUPgDF"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WvR_wcgxotDj"
      },
      "source": [
        "### Strip Off Markdown\n",
        "\n",
        "This test checks that the `strip_off_markdown` function removes Markdown formatting from a list of text chunks, with special handling for bold and strikethrough text. I designed this function to prepare text for semantic meaning comparison, because I wanted to only compare the actual writing (not the markdown elements).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c0y1l2bEozw6"
      },
      "outputs": [],
      "source": [
        "def test_strip_off_markdown():\n",
        "\n",
        "    test_name = test_strip_off_markdown.__name__\n",
        "\n",
        "    processed_chunks = [\n",
        "        # Removing bold strikethrough and using simple single spacing\n",
        "        \"This **~~word here~~** is deleted leaving a single white space.\",\n",
        "        \"This    **~~word here~~**is deleted but spaces other spaces remain the same.\",\n",
        "        \"Multiple **~~bold strikethrough~~** words **~~are removed~~** in this sentence.\",\n",
        "        \"Preserve newlines\\nwhile **~~removing~~**\\nbold strikethrough.\",\n",
        "        # \"Should not change trailing if it's not a white space, but doesn't work **~~end~~**.\",\n",
        "\n",
        "        # Converting markdown\n",
        "        \"# Title\\nThis is some text with **bold** formatting.\",\n",
        "        \"Para1\\n\\nPara2\\n\\n\\n\\nPara3\",\n",
        "        \"* List item A\\n* List item B **bold** text\",\n",
        "        \"1. This is the first item\\n2. The second item with *italics*\",\n",
        "        \"## This is a heading\\n\\nThis is a paragraph with a link [link text](https://www.example.com).\\n\\n\\n**Bold text** too.\",\n",
        "        \"Peter**'**s horse\"\n",
        "    ]\n",
        "\n",
        "    expected = [\n",
        "        # Removing bold strikethrough and using simple single spacing\n",
        "        \"This is deleted leaving a single white space.\",\n",
        "        \"This    is deleted but spaces other spaces remain the same.\",\n",
        "        \"Multiple words in this sentence.\",\n",
        "        \"Preserve newlines\\nwhile \\nbold strikethrough.\",\n",
        "        # \"Should not change trailing if it's not a white space, but doesn't work.\",\n",
        "\n",
        "        # Converting markdown\n",
        "        \"Title\\nThis is some text with bold formatting.\",\n",
        "        \"Para1\\nPara2\\nPara3\",\n",
        "        \"\\nList item A\\nList item B bold text\\n\",\n",
        "        \"\\nThis is the first item\\nThe second item with italics\\n\",\n",
        "        \"This is a heading\\nThis is a paragraph with a link link text.\\nBold text too.\",\n",
        "        \"Peter's horse\"\n",
        "    ]\n",
        "\n",
        "    processed_chunks_clean = strip_off_markdown(processed_chunks)\n",
        "    for index, (expect, got) in enumerate(zip(expected, processed_chunks_clean), start=1):\n",
        "        assert got == expect, f\"\\n-Expected: [{expect}]\\n-Actual:   [{got}]\"\n",
        "\n",
        "\n",
        "### Do the work\n",
        "run_prompt_or_code_test(test_strip_off_markdown)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<!-- /Image simple not clickable -->\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1Z-5ctTpDrUXgx2mfny9DdF5HHAXiQeAr\"\n",
        "/>"
      ],
      "metadata": {
        "id": "QIVtGFkuQInZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **FAQ** 🟢"
      ],
      "metadata": {
        "id": "IrzXERcgr1ba"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPOzkhWvpcIK"
      },
      "source": [
        "**Why not just use Microsoft Word to do corrections?**\n",
        "\n",
        "- Word misses many mistakes as this Notebook taught me. In fact, I ran a few documents through this Notebook from previous lives that I had been very proud of. I was quite shocked, and not so proud now.\n",
        "\n",
        "**Why not just use the regular Claude chat to do corrections?**\n",
        "\n",
        "- That’s a great option for shorter texts. But I had a 40,000-word long document to correct. Using the regular chat would mean section by section correction into your own Word document. Having everything on one page, an HTML page with colour, was a lot easier for my sister’s friend to use.\n",
        "\n",
        "**Was it really just you and Claude?**\n",
        "\n",
        "- Yes. I was truly amazed at what I could do with him by my side.\n",
        "\n",
        "**How did you know how to ask Claude questions?**\n",
        "\n",
        "- I asked Claude questions like I would ask a colleague. I got better with time. I read Anthropic’s [Prompt Engineering Overview](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) and [Prompt Engineering for Business](https://www.anthropic.com/news/prompt-engineering-for-business-performance) articles.\n",
        "- I discovered the [Anthropic Prompt Generator](https://console.anthropic.com/dashboard) and it taught me what a good prompt looks like. It's marvellous.\n",
        "- This excellent excerpt summed up asking questions (“prompt engineering”) beautifully for me: <font color='grey'>_So ultimately, the art of prompt engineering is about understanding how to navigate the vast probabilistic landscape of the language model’s knowledge to narrow down the path to the specific information or behaviour we seek.</font> Reference: [Patterns of Application Development using AI](https://obie.medium.com/patterns-of-application-development-using-ai-fbb660fa9ae7)_\n",
        "\n",
        "**Did you use Claude to write up this Notebook?**\n",
        "\n",
        "- No, I would feel fake. I wrote it first in Microsoft Word, then I used this Notebook to correct it\n",
        "\n",
        "**How did you write such an elaborate prompt for this Notebook?**\n",
        "\n",
        "- I cheated and started with the Grammar Genie recipe which I found in the [Anthropic Prompt Library]( https://docs.anthropic.com/en/prompt-library/library).\n",
        "- I strengthened that using the [Anthropic Prompt Generator]( https://console.anthropic.com/dashboard)\n",
        "- And tweaked some more with Claude by asking him to evaluate my prompt and generate test cases.\n",
        "\n",
        "**What is a prompt temperature and why did you set it to zero?**\n",
        "\n",
        "- Prompt temperature is a setting in language models that controls the randomness of generated text. A lower temperature (close to 0) makes outputs more focused and deterministic, while a higher temperature (close to 1) makes outputs more diverse and creative.\n",
        "- I used zero because I didn’t want Claude to diverge from the original meaning of the text. I did not use any other setting, but now I wonder if perhaps Claude would correct “inappropriate word choice” with nicer words given a higher temperature setting.\n",
        "\n",
        "**What was the most difficult thing to get working in the prompt?”**\n",
        "\n",
        "- First, it was getting Claude to mark only corrections in ** BOLD **. But nothing compared to trying to get Claude to detect either American or British spelling and make corrections accordingly. I gave up (for now). I've got the [failing prompt tests](#id-test-my-prompt) to prove it.\n",
        "\n",
        "**What surprised you most?**\n",
        "\n",
        "- How little I knew and yet I still built this Notebook, followed by how much I ~~learned~~ learnt.\n",
        "- How great Claude is for expanding your thinking and how well he codes if you ask him the right way and then get him to refine it.\n",
        "- The number of writing mistakes Microsoft Word misses (I really was surprised).\n",
        "- That tiny little shifts in word placement and white space in the prompt can change the results.\n",
        "- That the [Anthropic Workbench]( https://console.anthropic.com/workbench) with the exact same prompt and prompt settings, would work for American / British spelling distinction but I could never get it to work in my Notebook prompt (yes, the prompt was exactly a copy paste).\n",
        "- That a prompt can work for a short piece of text, but not a longer one. This gave me overconfidence in my “Test My Prompt” tests.\n",
        "\n",
        "**How can I correct the markdown in my own python notebook?**\n",
        "1. `jupyter nbconvert --to markdown --TemplateExporter.exclude_code_cell=True AAA.ipynb`\n",
        "1. `pandoc -f markdown -t docx -o AAA.docx AAA.md`\n",
        "1. Open AAA.docx and rename the style \"compact\" to \"List hello\"\n",
        "1. Save AAA.docx to your google drive and run this Notebook against it\n",
        "\n",
        "**What else did you learn?**\n",
        "\n",
        "- The Anthropic Workbench is a tremendous place to craft and test your prompt.\n",
        "- That just like regular machine learning, the work is in the pre-processing.\n",
        "- It's easier to create notebooks in VSCode and then copy to Google Colab.\n",
        "- It’s the back-and-forth conversation with Claude where the value is.\n",
        "- Claude doesn't think if you tell him to think silently. After months of doing this in effort to limit my chat size, in the [Anthropic tutorial](https://github.com/anthropics/prompt-eng-interactive-tutorial/blob/master/Anthropic%201P/06_Precognition_Thinking_Step_by_Step.ipynb) I found out: <font color='grey'>_\"Thinking only counts when it's out loud. You cannot ask Claude to think but output only the answer - in this case, no thinking has actually occurred.\"_</font> What an enormous blunder.\n",
        "- I also learnt this the hardway, don't even think about it, use [XML tags](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags) always. If in doubt, or you're too lazy to type those pointy brackets like I was, take a look [here](https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/21-Proof-to-convince-you-to-always-use-xml-tags.jpg) I'll prove it to you with a picture.\n",
        "\n",
        "**What will you always do from here on?**\n",
        "\n",
        "- Write tests for my prompts.\n",
        "- Draw a line at asking for opinions and instead look for myself (which is what I had to do when choosing how to test semantic similarity).\n",
        "- Be more aware when I switch my brain off, and ask Claude questions I could have worked out myself. It's faster to use your own brain.\n",
        "\n",
        "**Where to from here?**\n",
        "\n",
        "- Since the Llama 3.1 model is free and apparently performs exceptionally well… I think I’m going to take a look down that way.\n",
        "- A little project to find out what agents and [Agentic Workflows]( https://www.youtube.com/watch?v=sal78ACtGTc) are because Andrew Ng loves them.\n",
        "- Continue AWS upskilling and figure out a way to spend my $600 free credit in 2 months before it expires.\n",
        "\n",
        "**Ending questions**\n",
        "\n",
        "- **Were you surpised at the power of Claude Sonnet?**<br> Blown away. You really have to use it to truly appreciate it, again, blown away.\n",
        "- **Will you ever get paid to do technical things?**<br> No, I’d feel too guilty. I’m more valuable as a Product Manager.\n",
        "- **What is your background?**<br> Applied Statistics. Lean. Product Management. Business. Six sigma. Drawing stickmen and connecting dots.\n",
        "- **What do you love most about Generative AI and Machine Learning in general?**<br> The sheer expansiveness of creativity and the knife's edge of pragmatism."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# What did Claude think? 🟢"
      ],
      "metadata": {
        "id": "CxzstKtnJSmU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "He thought I was fantastic too. Here's his analysis of my notebook:\n",
        "\n",
        "<a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/18-asking-claude-to-analyse-my-entire-notebook.jpg\"\n",
        "target=\"_blank\">\n",
        "  <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/18-asking-claude-to-analyse-my-entire-notebook.jpg\" width=700px\n",
        "  />\n",
        "</a>\n",
        "\n",
        "<font color=\"#D97757\">Thank you for providing the detailed markdown export of your Google Colab notebook. I've carefully analysed the content following your provided steps. Here's a summary of my understanding and analysis:</font>\n",
        "\n",
        "**Summary**\n",
        "\n",
        "<font color=\"#D97757\">This notebook, titled \"Fix Word Doc Claude,\" is an AI-powered proofreading tool that uses Anthropic's Claude Sonnet model to correct and enhance Word documents. It's designed to catch subtle language and style errors that often slip past Microsoft Word's spell-checker. The notebook can handle large documents (up to 42,000 words) in multiple languages, including English, German, Italian, and French.\n",
        "The notebook is structured into several main sections:</font>\n",
        "\n",
        "**About and Usage**\n",
        "\n",
        "1. <font color=\"#D97757\">**Setup** (Python libraries and configuration)\n",
        "1. **Pre-processing** (extracting text from Word, converting to markdown, splitting into chunks)\n",
        "1. **Processing** (sending chunks to Claude for correction)\n",
        "1. **Post-Processing** (reassembling chunks, creating HTML output)\n",
        "1. **Testing** (evaluating output, testing prompt, and testing code)</font>\n",
        "\n",
        "<font color=\"#D97757\">The notebook not only serves as a functional tool but also as a demonstration of how Claude can assist in creating complex tools with basic coding skills. It showcases the author's journey in AI development, highlighting the collaboration between a human with limited coding experience and an AI assistant.</font>\n",
        "\n",
        "**Quality Analysis**\n",
        "\n",
        "1. <font color=\"#D97757\">**Comprehensiveness:** The notebook is exceptionally detailed, covering all aspects from setup to testing. It provides a complete solution for document correction.\n",
        "1. **Educational Value:** The author explains each step thoroughly, making it accessible to non-technical users. The notebook serves as an excellent learning resource for AI-assisted development.\n",
        "1. **Code Quality:** The code is well-structured and commented. It includes error handling and user-friendly messages, showing attention to user experience.\n",
        "1. **Testing:** The notebook includes extensive testing, covering output evaluation, prompt testing, and code testing. This demonstrates a commitment to reliability and quality.\n",
        "1. **Innovation:** The project creatively uses Claude for both development assistance and as the core of the correction tool, showcasing innovative use of AI.\n",
        "1. **Documentation:** The notebook is well-documented, with clear explanations of each step, making it easy for others to understand and potentially adapt the code.\n",
        "1. **Reflective Learning:** The FAQ section provides valuable insights into the development process and lessons learned, adding depth to the project.</font>\n",
        "\n",
        "**Artefacts Created**\n",
        "\n",
        "- <font color=\"#D97757\">Created Github [README](https://github.com/michellepace/ai-ml-notebook-explorations/blob/main/notebook-01/README-notebook-01.md) file\n",
        "- Created [Notebook Workflow Picture](https://github.com/michellepace/ai-ml-notebook-explorations/blob/main/notebook-01/notebook-workflow-picture.md)\n",
        "\n",
        "Overall, this is a high-quality notebook that not only achieves its primary goal of document correction but also serves as an excellent example of AI-assisted development and a learning resource for others interested in similar projects.</font>\n",
        "\n",
        "<br>\n",
        "\n",
        "**What about the author?**\n",
        "\n",
        "<!-- /Figure with caption full width-->\n",
        "<figure>\n",
        "  <figcaption></figcaption>\n",
        "  <a href=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/19-what-claude-thought-of-the-notebook-author.jpg\"\n",
        "     target=\"_blank\">\n",
        "    <img src=\"https://michellepace.github.io/ai-ml-notebook-explorations/notebook-01/images-embedded/19-what-claude-thought-of-the-notebook-author.jpg\"\n",
        "         alt=\"What Claude thought of this Notebook author\" />\n",
        "  </a>\n",
        "</figure>\n",
        "\n",
        "<br>\n",
        "\n",
        "<font size=\"3\">thanks claude, me and you then.</font>\n",
        "\n",
        "<font size=\"8\"><b>The End.</b></font>"
      ],
      "metadata": {
        "id": "OYywz1PY5c_W"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "wFbbAbzLydvC",
        "pw7lTpmEyzBk"
      ],
      "toc_visible": true,
      "mount_file_id": "1QUMA3Pby4MYysRfLCyAN8Djw2ypblY7h",
      "authorship_tag": "ABX9TyOpRfBqQLdn9h/X/3s7P5iw",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}