-
Notifications
You must be signed in to change notification settings - Fork 569
Closed
Labels
inference@huggingface/inference related@huggingface/inference related
Milestone
Description
- Provide individual exports for each function, they can have an additional
accessTokenparameter - Better typing:
HfInference.endpoint(...)should removemodelfrom the arg of the functions
We'll still keep the doc as is (new HfInference.xxx()), but individual functions will enable tree-shaking if they're imported directly, and it'll be more in line with @huggingface/hub
For example:
import { textGenerationStream, HfInference } from "@huggingface/inference";
// ok
await textGenerationStream({
model: "google/flan-t5-xxl", // can also be an endpoint url
inputs: "hi"
});
// ok
await textGenerationStream({
model: "google/flan-t5-xxl",
inputs: "hi",
accessToken: "hf_...",
});
const inference = new HfInference("hf_...");
// ok
await inference.textGenerationStream({
model: "google/flan-t5-xxl", // can also be an endpoint url
inputs: "hi"
});
// error, accessToken not possible
await inference.textGenerationStream({
model: "google/flan-t5-xxl",
inputs: "hi",
accessToken: "hf_..."
});
const endpoint = inference.endpoint(...);
// ok
await endpoint.textGenerationStream({
inputs: "hi"
});
// error, model not possible
await endpoint.textGenerationStream({
model: "google/flan-t5-xxl",
inputs: "hi",
});Metadata
Metadata
Assignees
Labels
inference@huggingface/inference related@huggingface/inference related