📦 v0.0.2-rc.2 (#155)

## Features - ✨ #142: [Lang Chat Router] Ollama Support (@mkrueger12) - ✨ #131: [Lang Chat Router] AWS Bedrock Support (@mkrueger12) ## Miscellaneous - 👷 #155 Fixing the dockerhub authorization step in the release workflow (@roma-glushko) - ♻️ #151: Moved specific provider schemas closer to provider's packages (@roma-glushko)
EinStack · Feb 22, 2024 · ec5600c · ec5600c
1 parent 62a0c59
commit ec5600c
Show file tree

Hide file tree

Showing 45 changed files with 1,666 additions and 340 deletions.
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
@@ -5,9 +5,6 @@ on:
     tags:
       - "*"
 
-    branches:
-      - main
-
 permissions:
   contents: write
   packages: write
@@ -70,7 +67,10 @@ jobs:
           fetch-depth: 0
 
       - name: login into Github Container Registry
-        run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u $ --password-stdin
+        run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u --password-stdin
+
+      - name: login into Github Container Registry
+        run: echo "${{ secrets.DOCKER_HUB_TOKEN }}" | docker login -u einstack --password-stdin
 
       - name: login into Github Container Registry
         run: echo "${{ secrets.DOCKER_HUB_TOKEN }}" | docker login -u einstack $ --password-stdin
@@ -85,6 +85,6 @@ jobs:
         working-directory: ./images
         run: VERSION=${{ github.ref_name }} make publish-ghcr-${{ matrix.image }}
 
-      - name: publish ${{ matrix.image }} image to Github Container Registry
+      - name: publish ${{ matrix.image }} image to DockerHub
         working-directory: ./images
         run: VERSION=${{ github.ref_name }} make publish-docherhub-${{ matrix.image }}
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,18 @@ The changelog consists of three categories:
 - **Improvements** - bugfixes, performance and other types of improvements to existing functionality
 - **Miscellaneous** - all other updates like build, release, CLI, etc.
 
+## 0.0.2-rc.2 (Feb 22nd, 2024)
+
+### Features
+
+- ✨ #142: [Lang Chat Router] Ollama Support (@mkrueger12)
+- ✨ #131: [Lang Chat Router] AWS Bedrock Support (@mkrueger12)
+
+### Miscellaneous
+
+- 👷 #155 Fixing the dockerhub authorization step in the release workflow (@roma-glushko)
+- ♻️  #151: Moved specific provider schemas closer to provider's packages (@roma-glushko)
+
 ## 0.0.2-rc.1 (Feb 12th, 2024)
 
 ### Features

diff --git a/README.md b/README.md
@@ -1,22 +1,24 @@
-# Glide: Cloud-Native LLM Gateway for Seamless LLMOps
 <div align="center">
     <img src="docs/logo/glide_no_bgd.png" width="300px" alt="Glide GH Header" />
+    <h1>Glide: Cloud-Native LLM Gateway for Seamless LLMOps</h1>
+    <a href="https://codecov.io/github/EinStack/glide"><img src="https://codecov.io/github/EinStack/glide/graph/badge.svg?token=F7JT39RHX9" alt="CodeCov" /></a>
+    <a href="https://discord.gg/pt53Ej7rrc"><img src="https://img.shields.io/discord/1181281407813828710" alt="Discord" /></a>
+    <a href="https://glide.einstack.ai/"><img src="https://img.shields.io/badge/build-view-violet%20?style=flat&logo=books&label=docs&link=https%3A%2F%2Fglide.einstack.ai%2F" alt="Glide Docs" /></a>
+    <a href="https://github.com/EinStack/glide/blob/main/LICENSE"><img src="https://img.shields.io/github/license/EinStack/glide.svg?style=flat-square&color=%233f90c8" alt="License" /></a>
+    <a href="https://artifacthub.io/packages/helm/einstack/glide"><img src="https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/einstack" alt="ArtifactHub" /></a>
+    <a href="https://app.fossa.com/projects/git%2Bgithub.com%2FEinStack%2Fglide?ref=badge_shield"><img src="https://app.fossa.com/api/projects/git%2Bgithub.com%2FEinStack%2Fglide.svg?type=shield" alt="FOSSA Status" /></a>
 </div>
 
-
-[![codecov](https://codecov.io/github/EinStack/glide/graph/badge.svg?token=F7JT39RHX9)](https://codecov.io/github/EinStack/glide)
-[![Discord](https://img.shields.io/discord/1181281407813828710)](https://discord.gg/pt53Ej7rrc)
-[![Documentation](https://img.shields.io/badge/build-view-violet%20?style=flat&logo=books&label=docs&link=https%3A%2F%2Fglide.einstack.ai%2F)](https://glide.einstack.ai/)
-[![LICENSE](https://img.shields.io/github/license/EinStack/glide.svg?style=flat-square&color=%233f90c8)](https://github.com/EinStack/glide/blob/main/LICENSE)
-[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2FEinStack%2Fglide.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2FEinStack%2Fglide?ref=badge_shield)
-
 ---
 
-Glide is your go-to cloud-native LLM gateway, delivering high-performance LLMOps in a lightweight, all-in-one package.
+**Glide** is your go-to cloud-native LLM gateway, delivering high-performance LLMOps in a lightweight, all-in-one package.
 
 We take all problems of managing and communicating with external providers out of your applications,
 so you can dive into tackling your core challenges.
 
+> [!Important]
+> Give us a star⭐ to support the project and watch👀 our repositories not to miss any update. Appriciate your interest 🙏
+
 Glide sits between your application and model providers to seamlessly handle various LLMOps tasks like
 model failover, caching, key management, etc. 
 
@@ -27,7 +29,7 @@ Take a look at the develop branch.
 Check out our [documentation](https://glide.einstack.ai)!
 
 > [!Warning]
-> Glide is under active development right now. Give us a star to support the project ✨
+> Glide is under active development right now 🛠️
 
 ## Features
 
@@ -38,35 +40,18 @@ Check out our [documentation](https://glide.einstack.ai)!
 - **Production-ready observability** via OpenTelemetry, emit metrics on models health, allows whitebox monitoring (coming soon)
 - Straightforward and simple maintenance and configuration, centralized API key control & management & rotation, etc.
 
-## Supported Providers
-
 ### Large Language Models
 
 |                                                     | Provider      | Support Status  |
 |-----------------------------------------------------|---------------|-----------------|
-| <img src="docs/images/openai.svg" width="18" />     | OpenAI        | 👍  Supported  |
+| <img src="docs/images/anthropic.svg" width="18" />  | Anthropic     | 👍  Supported  |
 | <img src="docs/images/azure.svg" width="18" />      | Azure OpenAI  | 👍  Supported  |
+| <img src="docs/images/aws-icon.png" width="18" />  | AWS Bedrock (Titan)    | 👍  Supported  |
 | <img src="docs/images/cohere.png" width="18" />     | Cohere        | 👍  Supported |
-| <img src="docs/images/octo.png" width="18" />     | OctoML        | 👍  Supported  |
-| <img src="docs/images/anthropic.svg" width="18" />  | Anthropic     | 👍  Supported  |
 | <img src="docs/images/bard.svg" width="18" />       | Google Gemini | 🏗️ Coming Soon |
-
-
-### Routers
-
-Routers are a core functionality of Glide. Think of routers as a group of models with some predefined logic. For example, the resilience router allows a user to define a set of backup models should the initial model fail. Another example, would be to leverage the least-latency router to make latency sensitive LLM calls in the most efficient manner.
-
-Detailed info on routers can be found [here](https://glide.einstack.ai/essentials/routers).
-
-#### Available Routers
-
-| Router      | Description  |
-|---------------|-----------------|
-| Priority        | When the target model fails the request is sent to the secondary model. The entire service instance keeps track of the number of failures for a specific model reducing latency upon model failure  |
-| Least Latency        | This router selects the model with the lowest average latency over time. If the least latency model becomes unhealthy, it will pick the second the best, etc.  |
-| Round Robin        | Split traffic equally among specified models. Great for A/B testing.  |
-| Weighted Round Robin | Split traffic based on weights. For example, 70% of traffic to Model A and 30% of traffic to Model B.  |
-
+| <img src="docs/images/octo.png" width="18" />     | OctoML        | 👍  Supported  |
+| <img src="docs/images/ollama.png" width="18" />     | Ollama        | 👍  Supported  |
+| <img src="docs/images/openai.svg" width="18" />     | OpenAI        | 👍  Supported  |
 
 ## Get Started
 
@@ -107,7 +92,6 @@ See [API Reference](https://glide.einstack.ai/api-reference/introduction) for mo
 
 ```json
 {
- "model": "gpt-3.5-turbo", # this is not required but can be used to specify different prompts to different models
  "message":
       {
         "role": "user",
@@ -196,7 +180,58 @@ docker pull ghcr.io/einstack/glide:latest-redhat
 
 ### Helm Chart
 
-Coming Soon
+Add the EinStack repository:
+
+```bash
+helm repo add einstack https://einstack.github.io/helm-charts
+helm repo update
+```
+
+Before installing the Helm chart, you need to create a Kubernetes secret with your API keys like:
+
+```bash
+kubectl create secret generic api-keys --from-literal=OPENAI_API_KEY=sk-abcdXYZ
+```
+
+Then, you need to create a custom values.yaml file to override the secret name like:
+
+```yaml
+# save as custom.values.yaml, for example
+glide:
+    apiKeySecret: "api-keys"
+```
+
+Finally, you should be able to install Glide's chart via:
+
+```
+helm upgrade glide-gateway einstack/glide --values custom.values.yaml --install
+```
+
+## SDKs
+
+To let you work with Glide's API with ease, we are going to provide you with SDKs that fits your tech stack:
+
+- Python (coming soon)
+- NodeJS (coming soon)
+- Golang (coming soon)
+- Rust (coming soon)
+
+## Core Concepts
+
+### Routers
+
+Routers are a core functionality of Glide. Think of routers as a group of models with some predefined logic. For example, the resilience router allows a user to define a set of backup models should the initial model fail. Another example, would be to leverage the least-latency router to make latency sensitive LLM calls in the most efficient manner.
+
+Detailed info on routers can be found [here](https://glide.einstack.ai/essentials/routers).
+
+#### Available Routers
+
+| Router      | Description  |
+|---------------|-----------------|
+| Priority        | When the target model fails the request is sent to the secondary model. The entire service instance keeps track of the number of failures for a specific model reducing latency upon model failure  |
+| Least Latency        | This router selects the model with the lowest average latency over time. If the least latency model becomes unhealthy, it will pick the second the best, etc.  |
+| Round Robin        | Split traffic equally among specified models. Great for A/B testing.  |
+| Weighted Round Robin | Split traffic based on weights. For example, 70% of traffic to Model A and 30% of traffic to Model B.  |
 
 ## Community