Rusty Common Sense Guide to AI Engineering

Rust implementations of the book A Common Sense Guide to AI Engineering published by Pragmatic Programmers.

Most ports expect the environment variable OPENAI_API_KEY set and support the follow CLI arguments:

• --model: use the specified model for the demo
• --base-url: connect to model at the specified url; defaults to https://api.openai.com/v1
• --secondary-model: use the specified model as the secondary one for demo; currently only expand-query demo from chapter 11 uses this
• --temperature: set the model's temperature; defaults to 0.0
• --help: show the arguments available

Ports that differ from the above are:

• token-count from chapter 6

As this repo uses Cargo workspaces to organize the demonstrations, each demo's source directory has the book's chapter as its prefix, e.g., 01-hello-world is the demo from chapter 1. However, when running the demo, drop the chapter prefix from the package -p argument (as shown below):

OPENAI_API_KEY=<api-key> cargo run -p hello-world -- ...

Ports by Chapter

• Chapter 1: HeLLMo, World!
  • Creating Our First App
• Chapter 5: Building a Chatbot
  • Augmenting the Prompt
    • Pirate tone
    • Spanish translation
    • Grammar checker
  • Adding Multi-Turn Dialogue
  • Managing State With Memory Systems
  • Adding a System Prompt
  • Building the Messages Array
    • This demo uses the helpers::History struct to simplify the message collection; as of openai-oxide version 0.14, when openai_oxide::resources::responses::create receives the ResponseCreateRequest with input type Vec<ResponseInputItem>, openai_oxide::resources::responses::Response::output_text method will always return an empty string. The helper function helper::extract_texts extracts all texts and return a string.
• Chapter 6: Augmenting a Prompt with Knowledge
  • token-count: does not require an OpenAI client connection
  • prepare-data: does not require an OpenAI client connection
  • knowledge-chatbot
• Chapter 7: Efficient Adding Knowledge with RAG
  • rag-chatbot
• Chapter 8: Measuring Quality with Evals
  • eval-chatbot: main.rs houses the interaction loop and lib.rs contains the functions extracted for reuse in trace-generator, rewrite-history, and reduce-hallucinations demos.
  • trace-generator
• Chapter 9: Prompt Engineering
  • rewrite-history
• Chapter 10: Reducing Hallucinations
  • reduce-hallucinations; exports SystemPrompt and remove_bracket_tags for reuse by expand-query demo
• Chapter 11: Evaluating and Optimizing RAG
  • expand-query
• Chapter 12: Equipping an LLM with Tools
  • multiplication; exports some functions for reuse by feed-results-back and website-reader demos
  • feed-results-back
  • website-reader
  • tools-api: for simplicity, this assumes there's only one function call per user input; this demo only reuses the helpers package and exports some functionalities for reuse in agent-loop
• Chapter 13: Running the Agent Loop
  • agent-loop
  • podcast-agent: instead of using an OpenAI audio model, this port uses any-tts for generating the audio locally using the open-weight TTS model kokoro-tts; its lib.rs exports tool functions for reuse in agentic-workflow and agentic-rag
• Chapter 14: Architecting Agentic Workflow
  • agentic-workflow: main.rs contains the main (interactive) loop; its lib.rs houses the workflow
• Chapter 15: Implementing a RAG Agent
  • agentic-rag: this port doesn't assume the presence of the vector database, thus it contains the populating of the vector database.
    • One important note: classify prompt text adds the explicit instruction to output the response that matches ConversationData; without that explicit instruction, the demo doesn't work with local models, even OpenAI OSS ones.

Notable differences

Other than the port in chapter 1, all other ports use commonly used functionalities in helpers crate. Chapter 1's port is deliberately left as-is to show that the full implementation in Rust isn't that complicated, as compared to Python's. Using helpers crate in the rest of the ports minimize distractions from implementation details.

Instead of Pinecone, this repo uses sqlite-vec and fastembed-rs as its vector database. This repo originally used embedded LanceDB as its vector database but replaces it with sqlite-vec because the former bloats the binaries rather significantly and requires protoc toolchain for compilation; sqlite-vec is lighter comparatively and meets the need of this repo.

Resources

For convenience, all the files in resources directory are copied from the book's source code; the copyright of those files belong to the book's author and the publisher.