Skip to content

Resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.

License

Notifications You must be signed in to change notification settings

rh-aiservices-bu/llm-on-openshift

Repository files navigation

LLM on OpenShift

In this repo you will find resources, demos, recipes... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.

Content

Inference Servers

The following Inference Servers for LLMs can be deployed standalone on OpenShift:

Serving Runtimes deployment

The following Runtimes can be imported in the Single-Model Serving stack of Open Data Hub or OpenShift AI.

Vector Databases

The following Databases can be used as a Vector Store for Retrieval Augmented Generation (RAG) applications:

  • Milvus: Full recipe to deploy the Milvus vector store, in standalone or cluster mode.
  • PostgreSQL+pgvector: Full recipe to create an instance of PostgreSQL with the pgvector extension, making it usable as a vector store.
  • Redis: Full recipe to deploy Redis, create a Cluster and a suitable Database for a Vector Store.

Inference and application examples

  • Caikit: Basic example demonstrating how to work with Caikit+TGIS for LLM serving.
  • Langchain examples: Various notebooks demonstrating how to work with Langchain. Examples are provided for different types of LLM servers (standalone or using the Single-Model Serving stack of Open Data Hub or OpenShift AI) and different vector databases.
  • Langflow examples: Various examples demonstrating how to work with Langflow.
  • UI examples: Various examples on how to create and deploy a UI to interact with your LLM.

LLM clients

  • AnythingLLM: Anything LLM is the "All-in-one AI application. Any LLM, any document, any agent, fully private". It is implemented here as a RHOAI custom workbench that can be easily connected to your LLM as well as other resources.

About

Resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published