-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
community[minor]: Addded ChatTogetherAI integration #4215
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
--- | ||
sidebar_label: TogetherAI | ||
--- | ||
|
||
import CodeBlock from "@theme/CodeBlock"; | ||
|
||
# ChatTogetherAI | ||
|
||
## Setup | ||
|
||
1. Create a TogetherAI account and get your API key [here](http://api.together.ai/). | ||
2. Export or set your API key inline. The ChatTogetherAI class defaults to `process.env.TOGETHER_AI_API_KEY`. | ||
|
||
```bash | ||
export TOGETHER_AI_API_KEY=your-api-key | ||
``` | ||
|
||
You can use models provided by TogetherAI as follows: | ||
|
||
import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx"; | ||
|
||
<IntegrationInstallTooltip></IntegrationInstallTooltip> | ||
|
||
```bash npm2yarn | ||
npm install @langchain/community | ||
``` | ||
|
||
import TogetherAI from "@examples/models/chat/integration_togetherai.ts"; | ||
|
||
<CodeBlock language="typescript">{TogetherAI}</CodeBlock> | ||
|
||
Behind the scenes, TogetherAI uses the OpenAI SDK and OpenAI compatible API, with some caveats: | ||
|
||
- Certain properties are not supported by the TogetherAI API, see [here](https://docs.together.ai/reference/chat-completions). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
import { ChatTogetherAI } from "@langchain/community/chat_models/togetherai"; | ||
import { HumanMessage } from "@langchain/core/messages"; | ||
|
||
const model = new ChatTogetherAI({ | ||
temperature: 0.9, | ||
// In Node.js defaults to process.env.TOGETHER_AI_API_KEY | ||
togetherAIApiKey: "YOUR-API-KEY", | ||
}); | ||
|
||
console.log(await model.invoke([new HumanMessage("Hello there!")])); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
78 changes: 78 additions & 0 deletions
78
libs/langchain-community/src/chat_models/tests/chattogetherai.int.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
import { describe, test } from "@jest/globals"; | ||
import { ChatMessage, HumanMessage } from "@langchain/core/messages"; | ||
import { | ||
PromptTemplate, | ||
ChatPromptTemplate, | ||
AIMessagePromptTemplate, | ||
HumanMessagePromptTemplate, | ||
SystemMessagePromptTemplate, | ||
} from "@langchain/core/prompts"; | ||
import { ChatTogetherAI } from "../togetherai.js"; | ||
|
||
describe("ChatTogetherAI", () => { | ||
test("invoke", async () => { | ||
const chat = new ChatTogetherAI(); | ||
const message = new HumanMessage("Hello!"); | ||
const res = await chat.invoke([message]); | ||
console.log({ res }); | ||
expect(res.content.length).toBeGreaterThan(10); | ||
}); | ||
|
||
test("generate", async () => { | ||
const chat = new ChatTogetherAI(); | ||
const message = new HumanMessage("Hello!"); | ||
const res = await chat.generate([[message]]); | ||
console.log(JSON.stringify(res, null, 2)); | ||
expect(res.generations[0][0].text.length).toBeGreaterThan(10); | ||
}); | ||
|
||
test("custom messages", async () => { | ||
const chat = new ChatTogetherAI(); | ||
const res = await chat.invoke([new ChatMessage("Hello!", "user")]); | ||
console.log({ res }); | ||
expect(res.content.length).toBeGreaterThan(10); | ||
}); | ||
|
||
test("prompt templates", async () => { | ||
const chat = new ChatTogetherAI(); | ||
|
||
// PaLM doesn't support translation yet | ||
const systemPrompt = PromptTemplate.fromTemplate( | ||
"You are a helpful assistant who must always respond like a {job}." | ||
); | ||
|
||
const chatPrompt = ChatPromptTemplate.fromMessages([ | ||
new SystemMessagePromptTemplate(systemPrompt), | ||
HumanMessagePromptTemplate.fromTemplate("{text}"), | ||
]); | ||
|
||
const responseA = await chat.generatePrompt([ | ||
await chatPrompt.formatPromptValue({ | ||
job: "pirate", | ||
text: "What would be a good company name a company that makes colorful socks?", | ||
}), | ||
]); | ||
|
||
console.log(responseA.generations); | ||
expect(responseA.generations[0][0].text.length).toBeGreaterThan(10); | ||
}); | ||
|
||
test("longer chain of messages", async () => { | ||
const chat = new ChatTogetherAI(); | ||
|
||
const chatPrompt = ChatPromptTemplate.fromMessages([ | ||
HumanMessagePromptTemplate.fromTemplate(`Hi, my name is Joe!`), | ||
AIMessagePromptTemplate.fromTemplate(`Nice to meet you, Joe!`), | ||
HumanMessagePromptTemplate.fromTemplate("{text}"), | ||
]); | ||
|
||
const responseA = await chat.generatePrompt([ | ||
await chatPrompt.formatPromptValue({ | ||
text: "What did I just say my name was?", | ||
}), | ||
]); | ||
|
||
console.log(responseA.generations); | ||
expect(responseA.generations[0][0].text.length).toBeGreaterThan(10); | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
import type { BaseChatModelParams } from "@langchain/core/language_models/chat_models"; | ||
import { | ||
type OpenAIClient, | ||
type ChatOpenAICallOptions, | ||
type OpenAIChatInput, | ||
type OpenAICoreRequestOptions, | ||
ChatOpenAI, | ||
} from "@langchain/openai"; | ||
|
||
import { getEnvironmentVariable } from "@langchain/core/utils/env"; | ||
|
||
type TogetherAIUnsupportedArgs = | ||
| "frequencyPenalty" | ||
| "presencePenalty" | ||
| "logitBias" | ||
| "functions"; | ||
|
||
type TogetherAIUnsupportedCallOptions = "functions" | "function_call" | "tools"; | ||
|
||
export type ChatTogetherAICallOptions = Partial< | ||
Omit<ChatOpenAICallOptions, TogetherAIUnsupportedCallOptions> | ||
>; | ||
|
||
export interface ChatTogetherAIInput | ||
extends Omit<OpenAIChatInput, "openAIApiKey" | TogetherAIUnsupportedArgs>, | ||
BaseChatModelParams { | ||
/** | ||
* The TogetherAI API key to use for requests. | ||
* @default process.env.TOGETHER_AI_API_KEY | ||
*/ | ||
togetherAIApiKey?: string; | ||
} | ||
|
||
/** | ||
* Wrapper around TogetherAI API for large language models fine-tuned for chat | ||
* | ||
* TogetherAI API is compatible to the OpenAI API with some limitations. View the | ||
* full API ref at: | ||
* @link {https://docs.together.ai/reference/chat-completions} | ||
* | ||
* To use, you should have the `TOGETHER_AI_API_KEY` environment variable set. | ||
* @example | ||
* ```typescript | ||
* const model = new ChatTogetherAI({ | ||
* temperature: 0.9, | ||
* togetherAIApiKey: process.env.TOGETHER_AI_API_KEY, | ||
* }); | ||
* | ||
* const response = await model.invoke([new HumanMessage("Hello there!")]); | ||
* console.log(response); | ||
* ``` | ||
*/ | ||
export class ChatTogetherAI extends ChatOpenAI<ChatTogetherAICallOptions> { | ||
static lc_name() { | ||
return "ChatTogetherAI"; | ||
} | ||
|
||
_llmType() { | ||
return "togetherAI"; | ||
} | ||
|
||
get lc_secrets(): { [key: string]: string } | undefined { | ||
return { | ||
togetherAIApiKey: "TOGETHER_AI_API_KEY", | ||
}; | ||
} | ||
|
||
lc_serializable = true; | ||
|
||
constructor( | ||
fields?: Partial< | ||
Omit<OpenAIChatInput, "openAIApiKey" | TogetherAIUnsupportedArgs> | ||
> & | ||
BaseChatModelParams & { togetherAIApiKey?: string } | ||
) { | ||
const togetherAIApiKey = | ||
fields?.togetherAIApiKey || getEnvironmentVariable("TOGETHER_AI_API_KEY"); | ||
|
||
if (!togetherAIApiKey) { | ||
throw new Error( | ||
`TogetherAI API key not found. Please set the TOGETHER_AI_API_KEY environment variable or provide the key into "togetherAIApiKey"` | ||
); | ||
} | ||
|
||
super({ | ||
...fields, | ||
modelName: fields?.modelName || "mistralai/Mixtral-8x7B-Instruct-v0.1", | ||
openAIApiKey: togetherAIApiKey, | ||
configuration: { | ||
baseURL: "https://api.together.xyz/v1/", | ||
}, | ||
}); | ||
} | ||
|
||
toJSON() { | ||
const result = super.toJSON(); | ||
|
||
if ( | ||
"kwargs" in result && | ||
typeof result.kwargs === "object" && | ||
result.kwargs != null | ||
) { | ||
delete result.kwargs.openai_api_key; | ||
delete result.kwargs.configuration; | ||
} | ||
|
||
return result; | ||
} | ||
|
||
async completionWithRetry( | ||
request: OpenAIClient.Chat.ChatCompletionCreateParamsStreaming, | ||
options?: OpenAICoreRequestOptions | ||
): Promise<AsyncIterable<OpenAIClient.Chat.Completions.ChatCompletionChunk>>; | ||
|
||
async completionWithRetry( | ||
request: OpenAIClient.Chat.ChatCompletionCreateParamsNonStreaming, | ||
options?: OpenAICoreRequestOptions | ||
): Promise<OpenAIClient.Chat.Completions.ChatCompletion>; | ||
|
||
/** | ||
* Calls the TogetherAI API with retry logic in case of failures. | ||
* @param request The request to send to the TogetherAI API. | ||
* @param options Optional configuration for the API call. | ||
* @returns The response from the TogetherAI API. | ||
*/ | ||
async completionWithRetry( | ||
request: | ||
| OpenAIClient.Chat.ChatCompletionCreateParamsStreaming | ||
| OpenAIClient.Chat.ChatCompletionCreateParamsNonStreaming, | ||
options?: OpenAICoreRequestOptions | ||
): Promise< | ||
| AsyncIterable<OpenAIClient.Chat.Completions.ChatCompletionChunk> | ||
| OpenAIClient.Chat.Completions.ChatCompletion | ||
> { | ||
delete request.frequency_penalty; | ||
delete request.presence_penalty; | ||
delete request.logit_bias; | ||
delete request.functions; | ||
|
||
if (request.stream === true) { | ||
return super.completionWithRetry(request, options); | ||
} | ||
|
||
return super.completionWithRetry(request, options); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there! I noticed that the recent code changes explicitly handle the environment variable for the TogetherAI API key. I've flagged this for your review to ensure it aligns with the intended functionality. Keep up the great work!