tutorials/2_Document_Intelligence_Markdown.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Document Intelligence with Markdown\n",
    "\n",
    "In this notebook we will experiment with Document Intelligence and its Markdown output. We will try reading documents and experiment whether Document Intelligence can manage advanced actions, such as reading multi-page tables. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Imports\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "\n",
    "import sys\n",
    "sys.path.append('..\\\\code')\n",
    "\n",
    "\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()\n",
    "\n",
    "from IPython.display import display, Markdown, HTML\n",
    "from PIL import Image\n",
    "from doc_utils import *\n",
    "\n",
    "\n",
    "def show_img(img_path, width = None):\n",
    "    if width is not None:\n",
    "        display(HTML(f'<img src=\"{img_path}\" width={width}>'))\n",
    "    else:\n",
    "        display(Image.open(img_path))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make sure we have the OpenAI Models information\n",
    "\n",
    "We will need the GPT-4-Turbo and GPT-4-Vision models for this notebook.\n",
    "\n",
    "When running the below cell, the values should reflect the OpenAI reource you have created in the `.env` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_info = {\n",
    "        'AZURE_OPENAI_RESOURCE': os.environ.get('AZURE_OPENAI_RESOURCE'),\n",
    "        'AZURE_OPENAI_KEY': os.environ.get('AZURE_OPENAI_KEY'),\n",
    "        'AZURE_OPENAI_MODEL_VISION': os.environ.get('AZURE_OPENAI_MODEL_VISION'),\n",
    "        'AZURE_OPENAI_MODEL': os.environ.get('AZURE_OPENAI_MODEL'),\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experimenting with Document Intelligence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First make sure to install the right version of Document Intelligence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## This version corresponds to API Version 2024-02-29-preview\n",
    "## Visit: https://learn.microsoft.com/en-us/python/api/overview/azure/ai-documentintelligence-readme?view=azure-python-preview\n",
    "\n",
    "%pip install azure-ai-documentintelligence>=1.0.0b2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Defining a helper function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.core.credentials import AzureKeyCredential\n",
    "from azure.ai.documentintelligence import DocumentIntelligenceClient\n",
    "from azure.ai.documentintelligence.models import AnalyzeResult\n",
    "from azure.ai.documentintelligence.models import AnalyzeDocumentRequest, ContentFormat\n",
    "\n",
    "endpoint = os.environ[\"DI_ENDPOINT\"]\n",
    "key = os.environ[\"DI_KEY\"]\n",
    "\n",
    "def analyze_document(path):\n",
    "    document_intelligence_client = DocumentIntelligenceClient(endpoint=endpoint, credential=AzureKeyCredential(key))\n",
    "\n",
    "    with open(path, \"rb\") as f:\n",
    "        poller = document_intelligence_client.begin_analyze_document(\n",
    "            \"prebuilt-layout\", analyze_request=f, output_content_format=ContentFormat.MARKDOWN, content_type=\"application/octet-stream\"\n",
    "        )\n",
    "    result: AnalyzeResult = poller.result()\n",
    "    \n",
    "    return result\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading in Sample Documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Margie’s Travel Presents…\n",
       "\n",
       "London\n",
       "===\n",
       "\n",
       "London is the capital and most populous city of England and the United Kingdom. Standing on the River Thames in the south east of the island of Great Britain, London has been a major settlement for two millennia. It was founded by the Romans, who named it Londinium.\n",
       "\n",
       "London's ancient core, the City of London, largely retains its 1.12-square- mile medieval boundaries.\n",
       "\n",
       "Since at least the 19th century, London has also referred to the metropolis around this core, historically split between Middlesex, Essex, Surrey, Kent, and Hertfordshire, which today largely makes up Greater London, governed by the Mayor of London and the London Assembly.\n",
       "\n",
       "\n",
       "## Mostly popular for:\n",
       "\n",
       "Leisure, Outdoors, Historical, Arts & Culture\n",
       "\n",
       "\n",
       "## Best time to visit:\n",
       "\n",
       "Jun-Aug\n",
       "\n",
       "Averag Precipitation: 1.9 in Average Temperature: 56-67°F\n",
       "\n",
       "London Hotels\n",
       "\n",
       "Margie’s Travel offers the following accommodation options in London:\n",
       "\n",
       "\n",
       "## The Buckingham Hotel\n",
       "\n",
       "Comfortable hotel close to major sights like Buckingham Palace, Regent’s Park, and Trafalgar Square.\n",
       "\n",
       "\n",
       "## The City Hotel\n",
       "\n",
       "Luxury rooms in the city, within walking distance of Tower Bridge and the Tower of London..\n",
       "\n",
       "\n",
       "## The Kensington Hotel\n",
       "\n",
       "Budget accommodation near Earl’s Court.\n",
       "\n",
       "To book your trip to London, visit www.margiestravel.com\n",
       "\n",
       "| Category | Information |\n",
       "| - | - |\n",
       "| Country | United Kingdom |\n",
       "| Capital Of | England |\n",
       "| Currency | Pound Sterling (GBP) |\n",
       "| Population (2021 census) | Approximately 8.8 million |\n",
       "| Famous For | Historical landmarks, museums, cultural diversity |\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r\"sample_data/1_London_Brochure.docx\"\n",
    "london_docx_result = analyze_document(path)\n",
    "Markdown(london_docx_result['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<!-- PageHeader=\"Margie's Travel Presents ...\" -->\n",
       "\n",
       "London\n",
       "===\n",
       "\n",
       "London is the capital and most populous city of England and the United Kingdom. Standing on the River Thames in the south east of the island of Great Britain, London has been a major settlement for two millennia. It was founded by the Romans, who named it Londinium. London's ancient core, the City of London, largely retains its 1.12-square-\n",
       "<figure>\n",
       "\n",
       "![](figures/0)\n",
       "\n",
       "<!-- FigureContent=\"Tower Bridge\" -->\n",
       "\n",
       "</figure>\n",
       "\n",
       " mile medieval boundaries. Since at least the 19th century, London has also referred to the metropolis around this core, historically split between Middlesex, Essex, Surrey, Kent, and Hertfordshire, which today largely makes up Greater London, governed by the Mayor of London and the London Assembly.\n",
       "\n",
       "<figure>\n",
       "\n",
       "![](figures/1)\n",
       "\n",
       "<!-- FigureContent=\"Fountains in Trafalgar Square\" -->\n",
       "\n",
       "</figure>\n",
       "\n",
       "\n",
       "Mostly popular for: Leisure, Outdoors, Historical, Arts & Culture\n",
       "\n",
       "Best time to visit: Jun-Aug Averag Precipitation: 1.9 in Average Temperature: 56-67°F\n",
       "\n",
       "\n",
       "# London Hotels\n",
       "\n",
       "Margie's Travel offers the following accommodation options in London:\n",
       "\n",
       "The Buckingham Hotel Comfortable hotel close to major sights like Buckingham Palace, Regent's Park, and Trafalgar Square.\n",
       "\n",
       "The City Hotel Luxury rooms in the city, within walking distance of Tower Bridge and the Tower of London ..\n",
       "\n",
       "The Kensington Hotel Budget accommodation near Earl's Court.\n",
       "\n",
       "To book your trip to London, visit www.margiestravel.com\n",
       "\n",
       "| Category | Information |\n",
       "| - | - |\n",
       "| Country | United Kingdom |\n",
       "| Capital Of | England |\n",
       "| Currency | Pound Sterling (GBP) |\n",
       "| Population (2021 census) | Approximately 8.8 million |\n",
       "| Famous For | Historical landmarks, museums, cultural diversity |\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r\"sample_data/1_London_Brochure.pdf\"\n",
    "london_pdf_result = analyze_document(path)\n",
    "Markdown(london_pdf_result['content'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Figures\n",
    "\n",
    "For the same document, Figures are detected for the PDF version, and not for the Docx version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['apiVersion', 'modelId', 'stringIndexType', 'content', 'pages', 'tables', 'paragraphs', 'contentFormat', 'sections', 'figures'])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "london_pdf_result.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['apiVersion', 'modelId', 'stringIndexType', 'content', 'pages', 'tables', 'paragraphs', 'contentFormat', 'sections'])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "london_docx_result.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'boundingRegions': [{'pageNumber': 1, 'polygon': [2.7496, 2.5964, 7.1241, 2.597, 7.1249, 5.0656, 2.7505, 5.0645]}], 'spans': [{'offset': 406, 'length': 77}], 'elements': ['/paragraphs/2']},\n",
       " {'boundingRegions': [{'pageNumber': 1, 'polygon': [0.9681, 5.8269, 5.0532, 5.8263, 5.0538, 8.1167, 0.9687, 8.1169]}], 'spans': [{'offset': 784, 'length': 94}], 'elements': ['/paragraphs/3']}]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "london_pdf_result['figures']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cropped image will be saved under name: sample_data/pdf_outputs\\page_0_1a66dee8-61a3-4e30-48dd-929b673b632b.png\n",
      "DPI (299.9994, 299.9994)\n",
      "bbox_in_pixels [824, 778, 2137, 1519]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"sample_data/pdf_outputs\\page_0_1a66dee8-61a3-4e30-48dd-929b673b632b.png\" width=400>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cropped image will be saved under name: sample_data/pdf_outputs\\page_0_5e83e46a-210b-d944-e977-c920acff5d84.png\n",
      "DPI (299.9994, 299.9994)\n",
      "bbox_in_pixels [290, 1747, 1516, 2435]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"sample_data/pdf_outputs\\page_0_5e83e46a-210b-d944-e977-c920acff5d84.png\" width=400>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from PIL import Image\n",
    "\n",
    "def get_dpi(image):\n",
    "    try:\n",
    "        dpi = image.info['dpi']\n",
    "        print(\"DPI\", dpi)\n",
    "    except KeyError:\n",
    "        dpi = (300, 300)\n",
    "    return dpi\n",
    "\n",
    "def polygon_to_bbox(polygon):\n",
    "    xs = polygon[::2]  # Extract all x coordinates\n",
    "    ys = polygon[1::2]  # Extract all y coordinates\n",
    "    left = min(xs)\n",
    "    top = min(ys)\n",
    "    right = max(xs)\n",
    "    bottom = max(ys)\n",
    "    return (left, top, right, bottom)\n",
    "\n",
    "\n",
    "def inches_to_pixels(inches, dpi):\n",
    "    dpi_x, dpi_y = dpi\n",
    "    return [int(inches[i] * dpi_x if i % 2 == 0 else inches[i] * dpi_y) for i in range(len(inches))]\n",
    "\n",
    "polygon = [2.7496, 2.5964, 7.1241, 2.597, 7.1249, 5.0656, 2.7505, 5.0645]\n",
    "\n",
    "\n",
    "def extract_figure(image_path, polygon):\n",
    "    bbox_in_inches = polygon_to_bbox(polygon)\n",
    "\n",
    "    # Load the image\n",
    "    image = Image.open(image_path)\n",
    "\n",
    "    filename = os.path.splitext(os.path.basename(image_path))[0].strip()\n",
    "    extension = os.path.splitext(os.path.basename(image_path))[1].strip()\n",
    "    crop_name = os.path.join(os.path.dirname(image_path), f\"{filename}_{generate_uuid_from_string(str(polygon))}{extension}\")\n",
    "    print(f\"Cropped image will be saved under name: {crop_name}\")\n",
    "\n",
    "    # Get DPI from the image\n",
    "    dpi = get_dpi(image)\n",
    "\n",
    "    # Convert the bounding box to pixels using the image's DPI\n",
    "    bbox_in_pixels = inches_to_pixels(bbox_in_inches, dpi)\n",
    "    print(\"bbox_in_pixels\", bbox_in_pixels)\n",
    "\n",
    "    # Crop the image\n",
    "    cropped_image = image.crop(bbox_in_pixels)\n",
    "    cropped_image.save(crop_name)  \n",
    "    show_img(crop_name, 400)\n",
    "\n",
    "for bounding in london_pdf_result['figures']:\n",
    "    polygon = bounding['boundingRegions'][0]['polygon']\n",
    "    extract_figure(png_files[0], polygon)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Document Intelligence with MS Word documents (docx)\n",
    "\n",
    "Another sample document .."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "AI Facts\n",
       "===\n",
       "\n",
       "\n",
       "# Fun Facts\n",
       "\n",
       "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
       "\n",
       "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision-making processes in business and science.\n",
       "\n",
       "\n",
       "# AI Facts\n",
       "\n",
       "| Fact Category | Detail |\n",
       "| - | - |\n",
       "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
       "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
       "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
       "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
       "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
       "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
       "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
       "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
       "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
       "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
       "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
       "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
       "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
       "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
       "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
       "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
       "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
       "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
       "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
       "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
       "\n",
       "\n",
       "# Some More Fun Facts\n",
       "\n",
       "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
       "\n",
       "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r\"sample_data/2_ai_facts.docx\"\n",
    "docx_result = analyze_document(path)\n",
    "Markdown(docx_result['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AI Facts\n",
      "===\n",
      "\n",
      "\n",
      "# Fun Facts\n",
      "\n",
      "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
      "\n",
      "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision-making processes in business and science.\n",
      "\n",
      "\n",
      "# AI Facts\n",
      "\n",
      "| Fact Category | Detail |\n",
      "| - | - |\n",
      "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
      "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
      "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
      "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
      "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
      "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
      "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
      "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
      "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
      "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
      "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
      "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
      "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
      "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
      "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
      "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
      "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
      "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
      "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
      "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
      "\n",
      "\n",
      "# Some More Fun Facts\n",
      "\n",
      "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
      "\n",
      "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(docx_result['content'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Document Intelligence with PDF Documents\n",
    "\n",
    "Notice the broken table over 3 pages, which gives 3 tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "AI Facts\n",
       "===\n",
       "\n",
       "\n",
       "# Fun Facts\n",
       "\n",
       "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
       "\n",
       "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision- making processes in business and science.\n",
       "\n",
       "\n",
       "# AI Facts\n",
       "\n",
       "|||\n",
       "| - | - |\n",
       "| Fact Category | Detail |\n",
       "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
       "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
       "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
       "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
       "\n",
       "|||\n",
       "| - | - |\n",
       "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
       "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
       "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
       "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
       "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
       "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
       "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
       "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
       "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
       "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
       "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
       "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
       "\n",
       "|||\n",
       "| - | - |\n",
       "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
       "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
       "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
       "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
       "\n",
       "\n",
       "# Some More Fun Facts\n",
       "\n",
       "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
       "\n",
       "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = r\"sample_data/2_ai_facts.pdf\"\n",
    "pdf_result = analyze_document(path)\n",
    "Markdown(pdf_result['content'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems that for the PDF, Document Intelligence did not correctly identify this as a single table, but rather as three separate tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AI Facts\n",
      "===\n",
      "\n",
      "\n",
      "# Fun Facts\n",
      "\n",
      "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
      "\n",
      "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision- making processes in business and science.\n",
      "\n",
      "\n",
      "# AI Facts\n",
      "\n",
      "|||\n",
      "| - | - |\n",
      "| Fact Category | Detail |\n",
      "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
      "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
      "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
      "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
      "\n",
      "|||\n",
      "| - | - |\n",
      "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
      "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
      "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
      "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
      "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
      "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
      "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
      "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
      "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
      "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
      "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
      "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
      "\n",
      "|||\n",
      "| - | - |\n",
      "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
      "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
      "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
      "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
      "\n",
      "\n",
      "# Some More Fun Facts\n",
      "\n",
      "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
      "\n",
      "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(pdf_result['content'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table Merging\n",
    "\n",
    "Let's try to merge the tables using OpenAI GPT-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yes\n"
     ]
    }
   ],
   "source": [
    "#### Let's try something \"hacky\"\n",
    "\n",
    "if len(pdf_result['content'].split('|\\n')) > 0: ## This is a hacky way to check if there is a Markdown table in the text\n",
    "    print('yes')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First Step\n",
    "\n",
    "First try to establish the table boundaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"total_number_of_markdown_tables_located\": \"3\",\n",
      "    \"table_first_and_last_rows_array\":\n",
      "    [\n",
      "        {\n",
      "            \"first_row\": \"| Fact Category | Detail |\",\n",
      "            \"last_row\": \"| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\"\n",
      "        },\n",
      "        {\n",
      "            \"first_row\": \"| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\",\n",
      "            \"last_row\": \"| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\"\n",
      "        },\n",
      "        {\n",
      "            \"first_row\": \"| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\",\n",
      "            \"last_row\": \"| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\"\n",
      "        }\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "## Experiment with prompting, to see if broken tables can be \"merged\" based on LLM's output\n",
    "## This is work in progress .. NOT YET COMPLETE\n",
    "\n",
    "\n",
    "prompt = \"\"\"\n",
    "You are a Markdown expert whose objective is to check whether Markdown tables have been split into two or more tables because of page breaks or because of the OCR extraction process. You are designed to output JSON.\n",
    "\n",
    "You are given the following Markdown text:\n",
    "## START OF MARKDOWN TEXT\n",
    "{text}\n",
    "## END OF MARKDOWN TEXT\n",
    "\n",
    "In the markdown text above, detect whether there are Markdown tables. If there are, you **MUST** then output the first row and the last row of each table. We define a Markdown table by having consecutive rows with no space between, with the each row separated from the next by '|\\n'. Do not separate Markdown table semantically. \n",
    "\n",
    "\n",
    "JSON OUTPUT FORMAT:\n",
    "You **MUST** output a JSON object with the following key-value pairs and following format:\n",
    "\n",
    "{{\n",
    "    \"total_number_of_markdown_tables_located:\": \"the total number of Markdown tables located\",,\n",
    "    \"table_first_and_last_rows_array\":\n",
    "    [\n",
    "        {{\n",
    "            \"first_row\": \"The first row in Markdown table format. Make sure that the last row of the first table is outputted verbatim word-for-word, including any leading or trailing spaces, and any leading or trailing pipes.\",\n",
    "            \"last_row\": \"The last row of the table in Markdown table format. Make sure that the first row of the second table is outputted verbatim word-for-word, including any leading or trailing spaces, and any leading or trailing pipes.\",\n",
    "        }}\n",
    "    ]\n",
    "}}\n",
    "\"\"\"\n",
    "\n",
    "p = prompt.format(text = pdf_result['content'])\n",
    "table_edges = ask_LLM_with_JSON(p, model_info=model_info)\n",
    "print(table_edges)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Second Step\n",
    "\n",
    "Second, try to use the tables boundaries to determine whether the tables are split because of the pages, or OCR extraction, and if they are, try to merge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"total_number_of_markdown_tables_located:\": \"3\",\n",
      "    \"num_of_tables_to_be_merged\": \"2\",\n",
      "    \"tables_to_be_merged\":\n",
      "    [\n",
      "        {\n",
      "            \"first_table_last_row\": \"| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\",\n",
      "            \"second_table_first_row\": \"| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\",\n",
      "            \"dividing_string\": \"\\n\\n|||\\n| - | - |\\n\"\n",
      "        },\n",
      "        {\n",
      "            \"first_table_last_row\": \"| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\",\n",
      "            \"second_table_first_row\": \"| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\",\n",
      "            \"dividing_string\": \"\\n\\n|||\\n| - | - |\\n\"\n",
      "        }\n",
      "    ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "\n",
    "prompt = \"\"\"\n",
    "You are a Markdown expert whose objective is to check whether Markdown tables have been split into two or more tables because of page breaks or because of the OCR extraction process. You are designed to output JSON.\n",
    "\n",
    "You are given the following Markdown text:\n",
    "## START OF MARKDOWN TEXT\n",
    "{text}\n",
    "## END OF MARKDOWN TEXT\n",
    "\n",
    "The below are the edges that define the start and end of each Markdown table. Use the first and last row mentioned in the \"table_first_and_last_rows_array\" as the definition of a Markdown table:\n",
    "## START OF TABLE EDGES\n",
    "{table_edges}\n",
    "## END OF TABLE EDGES\n",
    "\n",
    "In the markdown text above, detect whether there are Markdown tables. If there are, you **MUST** follow the Chain of Thought below:\n",
    "    1. Locate every Markdown table in the text using the provided Table Edges, and write down the number of those tables in your scratchpad.\n",
    "    2. Work on every two consecutive Markdown tables in a pairwise manner. For every pair of consecutive Markdown tables, you **MUST** check if they have any text between them. If the two tables are immediately consecutive, but there is text between them and this text is a footnote or a page number, you can safely ignore it, and mark the two tables as consecutive in your scratchpad. If, however, there is valid text between the two tables, then you **MUST** mark the two tables as not consecutive in your scratchpad.\n",
    "    3. Write down the number of checks you have to perform. You will have to perform checks for consecutive tables pair-wise.\n",
    "\n",
    " After completing the above steps, you **MUST** then do the following for every two immediately consecutive Markdown table:\n",
    "    A. If the separated Markdown tables are immediately consecutive, then check the headers and the data type of the columns of each table to estimate if the two tables originated from the same table but got split into two because of the page break, or because of the OCR extraction process.\n",
    "    B. If you find that these two Markdown tables likely belong to the same original table, then output **EXACTLY** the full last row of the first table, and the first full row of the first table in Markdown table format. Also output the exact string of characters that exist between the two tables in the JSON output.\n",
    "    C. Repeat the above steps for **ALL** immediately consecutive Markdown tables located in your scratchpad, to match the number of checks in your scratchpad.\n",
    "\n",
    "**SUPER IMPORTANT**: We define a Markdown table by having consecutive rows with no space between, with the each row separated from the next by '|\\n'. Do not separate Markdown table semantically. \n",
    "\n",
    "\n",
    "JSON OUTPUT FORMAT:\n",
    "Remember that you **MUST** process every two immediately consecutive or consecutive Markdown tables. You **MUST** output a JSON object with the following key-value pairs and following format:\n",
    "\n",
    "{{\n",
    "    \"total_number_of_markdown_tables_located:\": \"the total number of Markdown tables located\",\n",
    "    \"num_of_tables_to_be_merged\": \"the number of tables to be merged\",\n",
    "    \"tables_to_be_merged\":\n",
    "    [\n",
    "        {{\n",
    "            \"first_table_last_row\": \"The last row of the first table in Markdown table format. Make sure that the last row of the first table is outputted verbatim word-for-word, including any leading or trailing spaces, and any leading or trailing pipes.\",\n",
    "            \"second_table_first_row\": \"The first row of the second table in Markdown table format. Make sure that the first row of the second table is outputted verbatim word-for-word, including any leading or trailing spaces, and any leading or trailing pipes.\",\n",
    "            \"dividing_string\": \"The exact string of characters that exist between the two tables. Make sure that the string is outputted verbatim word-for-word, including any leading or trailing spaces, and any leading or trailing pipes.\",\n",
    "        }}\n",
    "    ]\n",
    "}}\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "p = prompt.format(text = pdf_result['content'], table_edges=table_edges)\n",
    "output = ask_LLM_with_JSON(p, model_info=model_info)\n",
    "print(output)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row in first table found:  True\n",
      "Row in last table found:  True\n",
      "Row in first table found:  True\n",
      "Row in last table found:  True\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "AI Facts\n",
       "===\n",
       "\n",
       "\n",
       "# Fun Facts\n",
       "\n",
       "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
       "\n",
       "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision- making processes in business and science.\n",
       "\n",
       "\n",
       "# AI Facts\n",
       "\n",
       "|||\n",
       "| - | - |\n",
       "| Fact Category | Detail |\n",
       "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
       "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
       "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
       "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
       "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
       "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
       "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
       "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
       "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
       "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
       "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
       "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
       "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
       "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
       "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
       "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
       "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
       "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
       "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
       "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
       "\n",
       "\n",
       "# Some More Fun Facts\n",
       "\n",
       "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
       "\n",
       "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "import copy\n",
    "\n",
    "\n",
    "## Let's try to merge the tables\n",
    "findings = json.loads(output)\n",
    "pdf_content = copy.deepcopy(pdf_result['content'])\n",
    "\n",
    "\n",
    "for table in findings['tables_to_be_merged']:\n",
    "    first = table['first_table_last_row']\n",
    "    second = table['second_table_first_row']\n",
    "\n",
    "    print(\"Row in first table found: \", first in pdf_result['content'])\n",
    "    print(\"Row in last table found: \", second in pdf_result['content'])\n",
    "\n",
    "    for table in findings['tables_to_be_merged']:\n",
    "        first_table_last_row = table['first_table_last_row']\n",
    "        second_table_first_row = table['second_table_first_row']\n",
    "\n",
    "        first_part = pdf_content.split(first_table_last_row)[0]\n",
    "        second_part = pdf_content.split(second_table_first_row)[1]\n",
    "\n",
    "        pdf_content = first_part + first_table_last_row + '\\n' + second_table_first_row + second_part\n",
    "\n",
    "\n",
    "Markdown(pdf_content)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AI Facts\n",
      "===\n",
      "\n",
      "\n",
      "# Fun Facts\n",
      "\n",
      "Artificial Intelligence (AI) has evolved dramatically from its inception, leading to fascinating developments and applications that stretch the boundaries of what we once thought possible. One of the most engaging aspects of AI is its application in creative fields. For instance, AI has been used to compose music, generate unique pieces of art, and even write poetry, demonstrating its ability to learn and mimic human creativity. These applications have not only provided new tools for artists and creators but have also sparked debates about the nature of creativity and whether it is a uniquely human trait.\n",
      "\n",
      "Another fun fact about AI is its ability to play and excel in complex games, a domain once believed to be the exclusive realm of human intelligence. AI systems like DeepMind's AlphaGo have defeated world champions in games like Go, which is known for its intricate strategy and depth. These achievements are not just about winning games; they represent significant advancements in AI's problem-solving and strategic planning capabilities. The algorithms developed for these purposes have broader applications, such as solving complex logistical problems and improving decision- making processes in business and science.\n",
      "\n",
      "\n",
      "# AI Facts\n",
      "\n",
      "|||\n",
      "| - | - |\n",
      "| Fact Category | Detail |\n",
      "| Definition | AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think and learn like humans. |\n",
      "| First AI Concept | The concept of artificial intelligence was first theorized by Alan Turing, a British mathematician, in 1950 through the Turing Test. |\n",
      "| First AI Program | The first AI program was developed by Allen Newell and Herbert A. Simon in 1955, which was called the Logic Theorist. |\n",
      "| Machine Learning | Machine learning is a subset of AI that involves the development of algorithms allowing computers to learn from and make decisions based on data. |\n",
      "| Deep Learning | Deep learning is a subset of machine learning involving neural networks with many layers, enabling advanced pattern recognition. |\n",
      "| AI in Healthcare | AI applications in healthcare include diagnostics, personalized medicine, and robot-assisted surgery. |\n",
      "| AI in Autonomous Vehicles | AI enables autonomous vehicles to perceive their environment and make decisions, reducing the need for human intervention. |\n",
      "| Natural Language Processing (NLP) | NLP is a field of AI focused on the interaction between computers and humans through natural language. |\n",
      "| Ethics in AI | Ethical concerns in AI include privacy, bias, accountability, and the potential for job displacement. |\n",
      "| AI in Finance | AI is used in finance for algorithmic trading, fraud detection, and customer service through chatbots. |\n",
      "| Robotics | Robotics is a field related to AI, focusing on the design and manufacturing of robots that can perform tasks autonomously. |\n",
      "| AI in Gaming | AI enhances gaming experiences by enabling more complex and realistic NPC behaviors and game environments. |\n",
      "| AI in Education | AI can personalize learning experiences through adaptive learning systems and automate administrative tasks. |\n",
      "| AI in Entertainment | AI is used in entertainment for content recommendation algorithms, music composition, and visual effects. |\n",
      "| Quantum Computing and AI | Quantum computing has the potential to significantly increase the processing power available for AI tasks, leading to breakthroughs in complex problem-solving. |\n",
      "| Bias in AI | AI systems can exhibit bias if they are trained on biased data sets, leading to unfair outcomes. |\n",
      "| AI and Job Creation | While AI can automate tasks, it also creates new job opportunities in AI development, supervision, and maintenance. |\n",
      "| AI in Agriculture | AI applications in agriculture include crop monitoring, predictive analysis for yield optimization, and automated machinery. |\n",
      "| AI and Climate Change | AI can analyze large datasets to model climate change impacts and optimize energy consumption patterns. |\n",
      "| AI Art and Creativity | AI is used to generate art, music, and literature, challenging traditional notions of creativity. |\n",
      "\n",
      "\n",
      "# Some More Fun Facts\n",
      "\n",
      "AI has also made unexpected discoveries and contributions to various scientific fields. For instance, AI algorithms have identified new patterns in data that were previously overlooked by human researchers, leading to new scientific discoveries. In the realm of astronomy, AI has helped identify new exoplanets and galaxies by analyzing vast amounts of data from telescopes and space missions. Similarly, in biology, AI has accelerated the discovery of new molecules and drugs by predicting their structures and interactions more efficiently than traditional methods.\n",
      "\n",
      "The relationship between AI and animals presents another intriguing aspect. Researchers have used AI to understand animal behavior and communication better. For example, AI has been employed to decode the complex language of dolphins and to track and predict the migration patterns of birds and other wildlife. These studies not only deepen our understanding of the natural world but also demonstrate AI's potential to bridge the communication gap between humans and other species. As AI continues to evolve, its applications expand into new and unexpected territories, promising to revolutionize our understanding of the world around us and beyond.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(pdf_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mmdoc",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}