Skip to content

Refacto exports #146

@coyotte508

Description

@coyotte508
  • Provide individual exports for each function, they can have an additional accessToken parameter
  • Better typing: HfInference.endpoint(...) should remove model from the arg of the functions

We'll still keep the doc as is (new HfInference.xxx()), but individual functions will enable tree-shaking if they're imported directly, and it'll be more in line with @huggingface/hub

For example:

import { textGenerationStream, HfInference } from "@huggingface/inference";

// ok
await textGenerationStream({
  model: "google/flan-t5-xxl", // can also be an endpoint url
  inputs: "hi"
}); 

// ok
await textGenerationStream({
  model: "google/flan-t5-xxl", 
  inputs: "hi",
  accessToken: "hf_...",
}); 

const inference = new HfInference("hf_...");

// ok
await inference.textGenerationStream({
  model: "google/flan-t5-xxl", // can also be an endpoint url
  inputs: "hi"
});

// error, accessToken not possible
await inference.textGenerationStream({
  model: "google/flan-t5-xxl", 
  inputs: "hi",
  accessToken: "hf_..."
});

const endpoint = inference.endpoint(...);

// ok 
await endpoint.textGenerationStream({
  inputs: "hi"
});

// error, model not possible
await endpoint.textGenerationStream({
  model: "google/flan-t5-xxl", 
  inputs: "hi",
});

Metadata

Metadata

Assignees

Labels

inference@huggingface/inference related

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions