Practical LLMs

[manisnesan]

Landing Section

https://github.com/Aggregate-Intellect/practical-llms/blob/main/README.md

• The Emergence of KnowledgeOps
• LLMOps: Expanding the Capabilities of Language Models with External Tools - See the respective comment
• Leveraging Language Models for Training Data Generation and Tool Learning

Update: Twitter thread, Slides and Recording available as of Mar 14, 2023

LLM Interfaces Workshop and Hackathon

https://lu.ma/llm-interfaces - Apr 28, 2023

Excellent talks

• State of GPT talk by Andrej Karpathy | Youtube | Summary | Wayde Gilliam tips on Retrieval LLM | Notes
• Augmented Language Model from LLM Bootcamp | Youtube | Summary

Considerations

• Where and when to use LLMs? Determine the task complexity and Data Drift
• Prompt Engineering
• Cost Analysis
• Hosting vs API considerations based on Cost Analysis
• Model Distillation

When the task at hand has low complexity and low drift, it may be possible to generate outputs with LLMs to train an in-house model. This can help reduce the cost of using LLMs, which is especially relevant for companies using LLMs at scale for chatbots or summarization.

• Evaluation Methodologies)
• Others ( Access Patterns such as natural lang interfaces, Longer Contexts, Hallucination)

Source : Pratik Pakodas - Substack

Courses

• W&B - Building LLM Powered Apps
• LangChain Chat with your data
• Generative AI Courses

• Building Systems with the ChatGPT API, with OpenAI’s
• LangChain for LLM Application Development, with LangChain’s
• How Diffusion Models Work, by

Check them out: deeplearning.ai/short-courses/

[manisnesan]

https://gist.github.com/joeddav/a11e5cc0850f0e540324177a53b547ae

Python wrapper around ChatGPT API

[manisnesan]

https://til.simonwillison.net/gpt3/chatgpt-api

[manisnesan]

Prompt Engineering Guide

Instruction: Tell the model what you want

Extract the name of the author from the quotation below.

“Some humans theorize that intelligent species go extinct before they can expand into outer space. If they're correct, then the hush of the night sky is the silence of the graveyard.”
― Ted Chiang, Exhalation
Output:

Ted Chiang

Completion: Induce the model to complete the beginning of what you want

“Some humans theorize that intelligent species go extinct before they can expand into outer space. If they're correct, then the hush of the night sky is the silence of the graveyard.”
― Ted Chiang, Exhalation

The author of this quote is
Output:

Ted Chiang

Demonstration: Show the model what you want, with either:
A few examples in the prompt
Many hundreds or thousands of examples in a fine-tuning training dataset - Few Shot prompts

Quote:
“When the reasoning mind is forced to confront the impossible again and again, it has no choice but to adapt.”
― N.K. Jemisin, The Fifth Season
Author: N.K. Jemisin

Quote:
“Some humans theorize that intelligent species go extinct before they can expand into outer space. If they're correct, then the hush of the night sky is the silence of the graveyard.”
― Ted Chiang, Exhalation
Author:
Output:

Ted Chiang

Source : Openai cookbook

Tricks

Source: cohere.ai - prompt Engineering
• Give clearer instructions – the more explicit you articulate the desired task, it's inputs, and outputs, the better the results will be.

• Ask the model to answer as if it was an expert.

• Supply better examples. If you're demonstrating examples in your prompt, make sure that your examples are diverse and high quality.

• Prompt the model to explain its reasoning using a prefix like "Let's think step by step". This is known as chain-of-thought reasoning

• Generate many outputs, and then use the model to pick the best one (an example of iterative refinement)

• If you're still having trouble, try splitting complex tasks into simpler subtasks

Takeaways

• be effective at writing instructions, selecting samples for few-shot, batching, breaking the task into steps & balancing quality/cost tradeoffs.

Blogs

the two that immediately came to mind: https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/ and https://www.oneusefulthing.org/p/a-guide-to-prompting-ai-for-what and this repo also gave me lots of ideas: https://github.com/f/awesome-chatgpt-prompts

Additional Resources for Prompt Engineering
1️⃣ Microsoft - Introduction to prompt engineering https://lnkd.in/dAfVftGG

2️⃣ Chip Huyen - Building LLM applications for production https://lnkd.in/dEiZqzpT

3️⃣ DAIR.AI - Prompt Engineering Guide
https://lnkd.in/dXZdgM7e

4️⃣ PromptsLab - Awesome Prompt Engineering https://lnkd.in/dF9naw2a

5️⃣ Lilian Weng (OpenAI) - Prompt Engineering https://lnkd.in/dBTJRpZd

6️⃣ Microsoft - Prompt engineering techniques https://lnkd.in/dDzsJXhb

7️⃣ Cobus Greyling - Generative AI Prompt Pipelines https://lnkd.in/d5U5XkFd

8️⃣ Xavier (Xavi) Amatriain (LinkedIn) - Prompt Engineering 101 - Introduction and resources https://lnkd.in/drT8Z_Rq

9️⃣ Xavier (Xavi) Amatriain - Prompt Engineering: How to Talk to the AIs (course) https://lnkd.in/dGKqcWFj

[manisnesan]

Paper - Language models and cognitive automation for economic research

[manisnesan]

Retrieval

Problem: ChatGPT - it doesn't know about YOUR data

Solution:

• The main way of doing this is through a process commonly referred to as "Retrieval Augmented Generation".

• In this process, rather than just passing a user question directly to a language model, the system "retrieves" any documents that could be relevant in answering the question, and then passes those documents (along with the original question) to the language model for a "generation" step.

• The main way most people - including us at LangChain - have been doing retrieval is by using semantic search

[manisnesan]

[manisnesan]

Quoting Simon Willison analogy for language models

Think of language models like ChatGPT as a “calculator for words”

Source

[manisnesan]

https://arxiv.org/pdf/2304.03153v1.pdf
Twitter

[manisnesan]

cookbook

[manisnesan]

Twitter thread - how do we overcome 4096 token limit in OpenAI GPT requests?

[manisnesan]

Prompt Injection example to leak system prompts

[manisnesan]

Idea

To enhance an existing Large Language Model with custom knowledge, there are 2 main methods:

• Fine-tuning: further training the model using the custom dataset.
• In context learning: supplying the necessary information from the custom dataset related to user query while querying.

Pros & Cons

• Fine-tuning provides a high degree of accuracy and completeness but it requires significant time and resources to train and host the custom model.
• In-context learning, on the other hand, offers greater flexibility and costs much less, but it’s limited by the model’s token limit.

Prompt Engineering and Retrieval Augmented Generation

instead of simply it’d be:
answer following question given ,

Source : Integrating ChatGPT with internal KB and Q&A

[manisnesan]

Demonstrate-Search-Predict

dsp - GitHub

[manisnesan]

Dolly - Open source Instruction tuned LLM

[manisnesan]

Related to #33