01f - Receipts with Form Recognizer.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 2,
  "metadata": {
    "language_info": {
      "name": "python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "version": "3.5.3-final"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Analyzing Receipts with Form Recognizer\n",
        "\n",
        "<p style='text-align:center'><img src='./images/receipt_analysis.jpg' alt='A robot holding a receipt'/></p>\n",
        "\n",
        "In the artificial intelligence (AI) field of computer vision, optical character recognition (OCR) is commonly used to read printed or handwritten documents. Often, the text is simply extracted from the documents into a format that can be used for further processing or analysis.\n",
        "\n",
        "A more advanced OCR scenario is the extraction of information from forms, such as purchase orders or invoices, with a semantic understanding of what the fields in the form represent. The **Form Recognizer** service is specifically designed for this kind of AI problem.\n",
        "\n",
        "## View a receipt\n",
        "\n",
        "In this example, you'll use the Form Recognizer's built-in model for analyzing receipts.\n",
        "\n",
        "Click the green <span style=\"color:green\">&#9655</span> button at the top left of the cell below to run it and see an example of a receipt that you'll use Form Recognizer to analyze."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "from PIL import Image\n",
        "import os\n",
        "%matplotlib inline\n",
        "\n",
        "\n",
        "# Load and display a receipt image\n",
        "fig = plt.figure(figsize=(6, 6))\n",
        "image_path = os.path.join('data', 'form-receipt', 'receipt.jpg')\n",
        "img = Image.open(image_path)\n",
        "plt.axis('off')\n",
        "plt.imshow(img)"
      ]
    },
    {
      "cell_type": "markdown",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "## Create a Form Recognizer resource\n",
        "\n",
        "Start by creating a Form Recognizer resource in your Azure subscription:\n",
        "\n",
        "1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n",
        "2. Select **+ Create a resource**, and search for *Form Recognizer*.\n",
        "3. In the list of services, select **Form Recognizer**.\n",
        "4. In the **Form Recognizer** blade, select **Create**.\n",
        "5. In the **Create** blade, enter the following details and select **Create**\n",
        "  * **Name**: A unique name for your service\n",
        "  * **Subscription**: Your Azure subscription\n",
        "  * **Location**: Any available location\n",
        "  * **Pricing tier**: F0\n",
        "  * **Resource Group**: The existing resource group you used previously\n",
        "  * **I confirm I have read and understood the notice below**: Selected.\n",
        "6. Wait for the service to be created.\n",
        "7. View your newly created Form Recognizer service in the Azure portal and on the **Quick Start** page, copy the **Key1** and **Endpoint** values and paste them in the code cell below, replacing **YOUR_FORM_KEY** and **YOUR_FORM_ENDPOINT**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "form_key = 'YOUR_FORM_KEY'\n",
        "form_endpoint = 'YOUR_FORM_ENDPOINT'\n",
        "\n",
        "print('Ready to use form recognizer at {} using key {}'.format(form_endpoint, form_key))"
      ]
    },
    {
      "cell_type": "markdown",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "## Analyze a receipt\n",
        "\n",
        "Now you're ready to use Form Recognizer to analyze a receipt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from python_code import form_recognizer\n",
        "import os\n",
        "\n",
        "try:\n",
        "    # Read the image data\n",
        "    image_path = os.path.join('data', 'form-receipt', 'receipt.jpg')\n",
        "    with open(image_path, \"rb\") as f:\n",
        "        data = f.read()\n",
        "    \n",
        "    # Send the image data to Form Recognizer\n",
        "    receipt_data = form_recognizer.get_form_data(form_key, form_endpoint, data)\n",
        "\n",
        "    # Print the results\n",
        "    print('\\n')\n",
        "    for field in receipt_data:\n",
        "        print(field, receipt_data[field])\n",
        "\n",
        "except Exception as ex:\n",
        "    print('Error:', ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "Note that Form Recognizer is able to interpret the data in the form, correctly identifying the merchant address and phone number, and the transaction date and time, as well as the line items, subtotal, tax, and total amounts.\n",
        "\n",
        "> **Note**: If you're curious about the code used to submit the request to Form Recognizer and process the results, look at the **form_recognizer.py** file in the **python_code** folder.\n",
        "\n",
        "## More Information\n",
        "\n",
        "For moer information about the Form Recognizer service, see [the Form Recognizer documentation](https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/index)"
      ]
    }
  ]
}