Skip to content

Commit

Permalink
Support for GitHub Models using the Azure AI Inference API (langchain…
Browse files Browse the repository at this point in the history
…4j#1807)

Fix langchain4j#1719

This adds GitHub Models (see https://github.com/marketplace/models )
support with the new Azure AI Inference API Java SDK.
  • Loading branch information
jdubois authored Sep 24, 2024
1 parent c9995ab commit d546c64
Show file tree
Hide file tree
Showing 33 changed files with 3,170 additions and 15 deletions.
151 changes: 151 additions & 0 deletions docs/docs/integrations/language-models/github-models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
sidebar_position: 6
---

# GitHub Models

If you want to develop a generative AI application, you can use GitHub Models to find and experiment with AI models for free.
Once you are ready to bring your application to production, you can switch to a token from a paid Azure account.

## GitHub Models Documentation

- [GitHub Models Documentation](https://docs.github.com/en/github-models)
- [GitHub Models Marketplace](https://github.com/marketplace/models)

## Maven Dependency

### Plain Java

```xml
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-github-models</artifactId>
<version>0.34.0</version>
</dependency>
```

## GitHub token

To use GitHub Models, you need to use a GitHub token for authentication.

Token are created and managed in [GitHub Developer Settings > Personal access tokens](https://github.com/settings/tokens).

Once you have a token, you can set it as an environment variable and use it in your code:

```bash
export GITHUB_TOKEN="<your-github-token-goes-here>"
```

## Creating a `GitHubModelsChatModel` with a GitHub token

### Plain Java

```java
GitHubModelsChatModel model = GitHubModelsChatModel.builder()
.gitHubToken(System.getenv("GITHUB_TOKEN"))
.modelName("gpt-4o-mini")
.build();
```

This will create an instance of `GitHubModelsChatModel`.
Model parameters (e.g. `temperature`) can be customized by providing values in the `GitHubModelsChatModel`'s builder.

### Spring Boot

Create a `GitHubModelsChatModelConfiguration` Spring Bean:

```Java
package com.example.demo.configuration.github;

import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.github.GitHubModelsChatModel;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

@Configuration
@Profile("github")
public class GitHubModelsChatModelConfiguration {

@Value("${GITHUB_TOKEN}")
private String gitHubToken;

@Bean
ChatLanguageModel gitHubModelsChatLanguageModel() {
return GitHubModelsChatModel.builder()
.gitHubToken(gitHubToken)
.modelName("gpt-4o-mini")
.logRequestsAndResponses(true)
.build();
}
}
```

This configuration will create an `GitHubModelsChatModel` bean,
which can be either used by an [AI Service](https://docs.langchain4j.dev/tutorials/spring-boot-integration/#langchain4j-spring-boot-starter)
or autowired where needed, for example:

```java
@RestController
class ChatLanguageModelController {

ChatLanguageModel chatLanguageModel;

ChatLanguageModelController(ChatLanguageModel chatLanguageModel) {
this.chatLanguageModel = chatLanguageModel;
}

@GetMapping("/model")
public String model(@RequestParam(value = "message", defaultValue = "Hello") String message) {
return chatLanguageModel.generate(message);
}
}
```

## Creating a `GitHubModelsStreamingChatModel` with a GitHub token

### Plain Java

```java
GitHubModelsStreamingChatModel model = GitHubModelsStreamingChatModel.builder()
.gitHubToken(System.getenv("GITHUB_TOKEN"))
.modelName("gpt-4o-mini")
.logRequestsAndResponses(true)
.build();
```

### Spring Boot

Create a `GitHubModelsStreamingChatModelConfiguration` Spring Bean:
```Java
package com.example.demo.configuration.github;

import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.github.GitHubModelsChatModel;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

@Configuration
@Profile("github")
public class GitHubModelsStreamingChatModelConfiguration {

@Value("${GITHUB_TOKEN}")
private String gitHubToken;

@Bean
GitHubModelsStreamingChatModel gitHubModelsStreamingChatLanguageModel() {
return GitHubModelsStreamingChatModel.builder()
.gitHubToken(System.getenv("GITHUB_TOKEN"))
.modelName("gpt-4o-mini")
.logRequestsAndResponses(true)
.build();
}
}
```

## Examples

- [GitHub Models Examples](https://github.com/langchain4j/langchain4j-examples/tree/main/github-models-examples/src/main/java)
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/google-ai-gemini.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 6
sidebar_position: 7
---

# Google AI Gemini
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/google-palm.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 8
sidebar_position: 9
---

# Google Vertex AI PaLM 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 7
sidebar_position: 8
---

# Google Vertex AI Gemini
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/hugging-face.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 9
sidebar_position: 10
---

# Hugging Face
Expand Down
1 change: 1 addition & 0 deletions docs/docs/integrations/language-models/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ sidebar_position: 0
| [Azure OpenAI](/integrations/language-models/azure-open-ai) |||| text, image || | | |
| [ChatGLM](/integrations/language-models/chatglm) | | | | text | | | | |
| [DashScope](/integrations/language-models/dashscope) ||| | text, image, audio || | | |
| [GitHub Models](/integrations/language-models/github-models) |||| text || | | |
| [Google AI Gemini](/integrations/language-models/google-ai-gemini) | ||| text, image, audio, video, PDF || | | |
| [Google Vertex AI Gemini](/integrations/language-models/google-vertex-ai-gemini) |||| text, image, audio, video, PDF || | | |
| [Google Vertex AI PaLM 2](/integrations/language-models/google-palm) | | | | text | | || |
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/jlama.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 10
sidebar_position: 11
---

# Jlama
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/local-ai.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 11
sidebar_position: 12
---

# LocalAI
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/mistral-ai.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 12
sidebar_position: 13
---

# MistralAI
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/ollama.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 13
sidebar_position: 14
---

# Ollama
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/open-ai.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 14
sidebar_position: 15
---

# OpenAI
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/qianfan.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 15
sidebar_position: 16
---

# Qianfan
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/workers-ai.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 16
sidebar_position: 17
---

# Cloudflare Workers AI
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/integrations/language-models/zhipu-ai.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 17
sidebar_position: 18
---

# Zhipu AI
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -148,10 +148,12 @@ public AzureOpenAiStreamingChatModel(String endpoint,
Map<String, String> customHeaders) {

this(deploymentName, tokenizer, maxTokens, temperature, topP, logitBias, user, n, stop, presencePenalty, frequencyPenalty, dataSources, enhancements, seed, responseFormat, listeners);
if(useAsyncClient)
if(useAsyncClient) {
this.asyncClient = setupAsyncClient(endpoint, serviceVersion, apiKey, timeout, maxRetries, proxyOptions, logRequestsAndResponses, userAgentSuffix, customHeaders);
else
this.client = setupSyncClient(endpoint, serviceVersion, apiKey, timeout, maxRetries, proxyOptions, logRequestsAndResponses, userAgentSuffix, customHeaders); }
} else {
this.client = setupSyncClient(endpoint, serviceVersion, apiKey, timeout, maxRetries, proxyOptions, logRequestsAndResponses, userAgentSuffix, customHeaders);
}
}

public AzureOpenAiStreamingChatModel(String endpoint,
String serviceVersion,
Expand Down
Loading

0 comments on commit d546c64

Please sign in to comment.