Artificial General Intelligence (AGI) Demystified

This repository contains code for live session and video for the O'Reilly Course on Artificial General Intelligence (AGI) Demystified

This course offers an exploration of the current approaches toward Artificial General Intelligence (AGI), focusing on state-of-the-art reasoning models and agent architectures. Participants will learn about the evolution of AGI research, understand key benchmarks used to measure progress, and gain practical knowledge in working with advanced models like Claude 3.7, DeepSeek-R1, and OpenAI's o3. Through hands-on exercises and case studies, attendees will develop the skills needed to evaluate these models' capabilities, understand their limitations, and apply them effectively to complex tasks.

Notebooks

• Using Reasoning Models

  • An Introduction to Reasoning Models - Using 6 reasoning models across 5 providers

  • Short Term Memory with OpenAI Agents - OpenAI Agents doesn't implement short term memory by default, this code will add the concept to the agent

  • Computer Use with reasoning models - Letting a reasoning model control our laptop (caution advised when running this code)

• Benchmarking

  • Benchmarking Reasoning Models - Running questions from Humanity's Last Exam on reasoning models

  • Benchmarking Llama 3.2 Instruct on MMLU and Embedders on MTEB

    • Follow-up Evaluating Llama 3.2 non-instruct on MMLU

    • Evaluating Llama 3.1 vs Mistral on Truthful Q/A

• Case Studies

  • GRPO + Qwen: Using Reinforcement Learning to train a reasoning model.

  • Prompt Engineering Deepseek R1 - Using fewshot learning, vector databases, and even chain of thought prompting to make reasoning models even better

  Probing

  • Probing Chess Playing LLMs

  • There are over a dozen notebooks for the birth year/death year probing example so I will only share a few key ones here:

    • Llama-3 8B Instruct with prompt "Who is {NAME}"
    • BERT-large-cased no prompt
    • Mistral-7B-Instruct-v0.3 with prompt "Who is {NAME}"

Instructor

Sinan Ozdemir is a former lecturer of Data Science at Johns Hopkins University and the author of multiple textbooks on data science and machine learning. Additionally, he is the founder of the recently acquired Kylie.ai, an enterprise-grade conversational AI platform with RPA capabilities. He holds a master’s degree in Pure Mathematics from Johns Hopkins University and is based in San Francisco, CA.