Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Heurist API Integration as New Model Provider #335

Merged
merged 10 commits into from
Nov 15, 2024
Merged

feat: Add Heurist API Integration as New Model Provider #335

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
910e9d0
add Heurist API
tsubasakong Nov 15, 2024
ff46598
Merge branch 'ai16z:main' into main
tsubasakong Nov 15, 2024
a350e62
update Heurist chat models
tsubasakong Nov 15, 2024
3d23f7a
update chat models
tsubasakong Nov 15, 2024
a702d2e
add Heurist API key option
tsubasakong Nov 15, 2024
a40ceb6
Merge branch 'ai16z:main' into main
tsubasakong Nov 15, 2024
b828be4
Merge branch 'ai16z:main' into main
tsubasakong Nov 15, 2024
3c2bedb
clean
tsubasakong Nov 15, 2024
03cd5cc
clean log
tsubasakong Nov 15, 2024
2e3d507
back to default model provider
tsubasakong Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update Heurist chat models
  • Loading branch information
@tsubasakong
tsubasakong committed Nov 15, 2024
commit a350e62a7b9568c3b4d4ea0d5a88479e8f4c8bb7
2 changes: 1 addition & 1 deletion packages/core/src/defaultCharacter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ export const defaultCharacter: Character = {
name: "Eliza",
plugins: [],
clients: [],
modelProvider: ModelProviderName.OPENAI,
modelProvider: ModelProviderName.HEURIST,
settings: {
secrets: {},
voice: {
Expand Down
26 changes: 24 additions & 2 deletions packages/core/src/generation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ export async function generateText({
const max_response_length = models[provider].settings.maxOutputTokens;

const apiKey = runtime.token;
console.log("runtime",runtime);

try {
elizaLogger.debug(
Expand Down Expand Up @@ -289,6 +290,28 @@ export async function generateText({
console.debug("Received response from Ollama model.");
break;

case ModelProviderName.HEURIST: {
elizaLogger.debug("Initializing Heurist model.");
console.log("apiKey", apiKey);
const API_KEY = "loadtesting#loadtesting-bot"
console.log("API_KEY", API_KEY);
const heurist = createOpenAI({ apiKey: API_KEY, baseURL: endpoint });

const { text: heuristResponse } = await aiGenerateText({
model: heurist.languageModel(model),
prompt: context,
system: runtime.character.system ?? settings.SYSTEM_PROMPT ?? undefined,
temperature: temperature,
maxTokens: max_response_length,
frequencyPenalty: frequency_penalty,
presencePenalty: presence_penalty,
});

response = heuristResponse;
elizaLogger.debug("Received response from Heurist model.");
break;
}

default: {
const errorMessage = `Unsupported provider: ${provider}`;
elizaLogger.error(errorMessage);
Expand Down Expand Up @@ -679,8 +702,7 @@ export const generateImage = async (

const model = getModel(runtime.character.modelProvider, ModelClass.IMAGE);
const modelSettings = models[runtime.character.modelProvider].imageSettings;
const apiKey = runtime.token ?? runtime.getSetting("HEURIST_API_KEY") ?? runtime.getSetting("TOGETHER_API_KEY") ?? runtime.getSetting("OPENAI_API_KEY");

const apiKey = runtime.token ?? runtime.getSetting("HEURIST_API_KEY") ?? runtime.getSetting("TOGETHER_API_KEY") ?? runtime.getSetting("OPENAI_API_KEY");
try {
if (runtime.character.modelProvider === ModelProviderName.HEURIST) {
const response = await fetch('http://sequencer.heurist.xyz/submit_job', {
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/models.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ const models: Models = {
},
endpoint: "https://llm-gateway.heurist.xyz",
model: {
[ModelClass.SMALL]: "dolphin-2.9-llama3-8b",
[ModelClass.SMALL]: "meta-llama/llama-3-70b-instruct",
[ModelClass.MEDIUM]: "meta-llama/llama-3-70b-instruct",
[ModelClass.LARGE]: "meta-llama/llama-3.1-405b-instruct",
[ModelClass.EMBEDDING]: "" , //Add later,
Expand Down