Andy Burgin will explain how to work with LLMs in containerised environments where CPU, memory, GPUs and let's face it cash are in short supply. He'll talk about LLMs, prompting, quantisation, LangChain, Fine-tuning, RAG and K8sGPT. There will be demos and wisdom, Andy will share what works and mostly what doesn't so you don't have to go down the AI rabbit hole unprepared - warning may contain some badly written Python code
- llamacpp - insructions for the manual creation of llamacpp base image (will be replaced by Dockerfile at a later date).
- langchain - sample files to run basic llm querying and RAG.
- k8sgpt - files and instructions for the creation of a local kubernetes cluster to deploy k8sgpt and k8sgpt operator - use of in cluster local-ai to run LLMs for enhances support.
- slides - presetation deck for talk.