Skip to content

[Provider] Finish remaining OpenAI-compatible provider batch: Chutes, Featherless, Novita #572

Closed
#652
#653
Closed
[Provider] Finish remaining OpenAI-compatible provider batch: Chutes, Featherless, Novita#572
#652
#653
Assignees
olddonkey
Labels
catalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or requestproviderProvider adapters, OpenAI-compat presets, upstream API quirksroadmapAccepted long-term work item; needs its own project cycletoolstool_calls, MCP, web-search / sidecar tools

Description

@olddonkey
olddonkey
opened on Jul 27, 2026

Area

Provider registry / model discovery / catalog / CLI / Dashboard / documentation

Goal

Finish the remaining providers from the accepted OpenAI-compatible provider-promotion batch.

The original umbrella contained twelve providers. The shared discovery/security contract and nine provider presets have already landed on dev.

This issue now tracks only the three remaining candidates:

  • chutes
  • featherless
  • novita

Keep this umbrella open until each remaining candidate is either merged or explicitly deferred with the reason recorded.

Delivered work

Shared discovery/security contract

#652 landed the common provider-discovery contract, including:

  • registry-owned discovery destination/query policy;
  • bounded response bytes and accepted row counts;
  • declarative eligibility filtering;
  • redirect refusal;
  • outbound destination-policy enforcement;
  • fixed-preset host protection;
  • same-named custom-provider destination preservation;
  • fixture-based regression coverage rather than live-provider CI dependencies.

Provider presets already delivered

The following original candidates are now first-class providers on dev:

These providers are complete for this umbrella and should not be reimplemented here.

Remaining work

Chutes

Tracked by draft PR #1315.

Expected preset:

id: chutes
base URL: https://llm.chutes.ai/v1
adapter: openai-chat
auth: Bearer API key

The implementation must retain the established safety properties:

  • registry-owned fixed destination;
  • existing same-named custom destinations must not be silently retargeted;
  • bounded public-model discovery;
  • admit only models with provider evidence for the required chat/tool capability;
  • provider-native model IDs preserved;
  • no false claim that the public model catalog validates the supplied API key;
  • fixture-only tests;
  • explicit security review for the credential/destination contract.

Do not broaden this preset to arbitrary user-hosted Chutes endpoints or unrelated Chutes APIs.

Featherless

Tracked by draft PR #1317.

Expected preset:

id: featherless
base URL: https://api.featherless.ai/v1
adapter: openai-chat
auth: Bearer API key

The major catalog constraint remains important: Featherless exposes a very large model catalog.

Discovery must therefore remain explicitly bounded and fail closed rather than downloading or admitting the complete catalog.

The current implementation direction should preserve:

  • bounded response size and row count;
  • first-page / bounded discovery behavior;
  • current-plan eligibility;
  • chat-model eligibility;
  • tool-use eligibility;
  • exclusion of gated/ineligible rows;
  • conservative capability metadata;
  • authenticated key-validation boundary where supported;
  • same-named custom-provider destination preservation;
  • fixture-only tests;
  • explicit maintainer security review.

Novita AI

Tracked by draft PR #1318.

Expected preset:

id: novita
base URL: https://api.novita.ai/openai/v1
adapter: openai-chat
auth: Bearer API key

Earlier review blocked canonical promotion on operating-entity/provider evidence rather than on the wire protocol itself.

The current follow-up contains updated evidence and should be judged on that record rather than the older blocker.

The implementation should preserve:

  • fixed registry-owned destination;
  • same-named custom-provider destination preservation;
  • bounded model discovery;
  • admission only for rows positively identified as compatible chat models;
  • provider-native slash-containing model IDs;
  • conservative capability claims;
  • public model-list success must not be treated as proof that an API key is valid;
  • fixture-only tests;
  • explicit maintainer/security review of the canonical-provider evidence.

Common acceptance contract

Each remaining provider must:

  • be an explicit canonical registry allowlist entry;
  • use the documented fixed OpenAI-compatible endpoint;
  • preserve an existing same-named custom provider's destination;
  • bound discovery response bytes before unbounded parsing/caching;
  • bound accepted raw model rows;
  • exclude models that cannot serve the supported chat/tool path;
  • preserve provider-native model IDs, including embedded /;
  • avoid claiming unsupported reasoning, parallel-tool, multimodal, or structured-output capabilities;
  • expose the preset consistently through CLI, Dashboard/provider workspace, and documentation;
  • use fixture-based deterministic tests rather than live provider endpoints in CI;
  • avoid logging or exposing API-key material;
  • receive required security review for new credential destinations;
  • merge into dev, or be explicitly deferred with the reason recorded here.

Active PRs

All three intentionally relate to this umbrella rather than auto-closing it individually.

Close condition

Close this issue as completed when all three remaining providers have either:

  1. landed on dev; or
  2. been explicitly declined/deferred with a documented reason and no remaining implementation is expected under this umbrella.

Do not reopen completed providers from the original batch here. Future providers with materially different authentication, transport, or discovery contracts should use separate issues.

Activity

  1. github-actions
    added
    enhancementNew feature or request
    on Jul 27, 2026

  2. github-actions commented on Jul 27, 2026

    @github-actions
    github-actionsbot
    on Jul 27, 2026 – with GitHub Actions
    Contributor

    Automated translation bookkeeping — detected language: English.

  3. Ingwannu commented on Jul 28, 2026

    @Ingwannu
    Ingwannu
    on Jul 28, 2026
    Owner

    Thanks for the unusually thorough proposal and endpoint verification. The direction is useful, but this should remain an umbrella issue rather than become one review-sized change containing all twelve providers.

    Please proceed in phases against dev:

    1. Land the shared discovery contract first: registry-derived discovery URL/query/filtering, bounded response bytes and accepted row count, chat-model eligibility filtering, representative metadata parsing, and same-named custom-provider destination preservation. Keep tests fixture-based; CI must not depend on live provider endpoints.
    2. Add providers in small reviewable batches (roughly 2–4 with the same discovery shape), with registry/CLI/dashboard/docs parity and provider-specific fixtures in each PR.
    3. Keep providers requiring a different transport, authentication scheme, or arbitrary prediction schema out of this issue. Together AI remains the existing baseline and must not be re-added.

    The safety requirements in the report are acceptance criteria, especially preventing an existing custom provider/key from being silently retargeted and preventing oversized mixed catalogs from entering the Codex catalog. A provider should not claim streaming, tools, structured output, or multimodal support beyond what its selected upstream models and official documentation actually establish.

    I am assigning this to you. You do not need to implement every provider in one PR; reference this umbrella issue from each phased PR and target dev.

  4. Wibias
    assigned
    olddonkey
    on Jul 28, 2026
  5. Wibias
    linked a pull request that will close this issuefeat(providers): add Baseten Model APIs preset #653on Jul 29, 2026
  6. Wibias
    linked a pull request that will close this issuefeat(providers): add bounded model discovery contract #652on Jul 29, 2026

  7. lidge-jun commented on Jul 29, 2026

    @lidge-jun
    lidge-jun
    on Jul 29, 2026
    Owner

    Phase 1 landed: #652 merged as 48f2e8362.

    That is the shared discovery contract you were asked to land first — registry-owned endpoint/query policy, declarative eligibility filters, bounded response bytes and row count, redirect: "error" so credentials cannot follow a redirect, destination-policy checks before fetch, OAuth host pinning for fixed presets, and same-named custom-provider collision preservation.

    Verified on the merged dev tip rather than from the PR branch: bun x tsc --noEmit clean, and 151 tests pass across the discovery-contract, provider-registry-parity, and codex-catalog suites.

    Keeping this umbrella open for the remaining provider batches. Next up is #653 (Baseten pilot), which no longer has the contract as a blocker. Its windows-latest job is currently failing on the Test step — I confirmed that is a genuine failure rather than a cancelled run, and I have dispatched a rerun now that the prerequisite is on dev. Worth noting for the follow-on batches: the same PR's focused suites pass locally here (38 pass, typecheck clean), so if the Windows failure persists it is more likely a runner-level problem than a defect in the preset itself — the earlier #610 investigation hit EEXIST: epoll_ctl from the Bun test runner on Linux CI, and I could not find a matching upstream Bun issue to pin it to.

    Thanks for structuring this as phases. Landing the contract before the twelve presets is what made #652 reviewable as one security-boundary change instead of twelve.

  8. Wibias
    added
    catalogModel catalog, slugs, visibility, routed entries
    toolstool_calls, MCP, web-search / sidecar tools
    providerProvider adapters, OpenAI-compat presets, upstream API quirks
    on Aug 1, 2026
  9. lidge-jun
    added
    roadmapAccepted long-term work item; needs its own project cycle
    on Aug 3, 2026
  10. lidge-jun
    added 3 commits that reference this issue on Aug 5, 2026

    docs(devlog): move three inferred PR-issue links out of the literal l…

    23ef7d0

    docs(devlog): plan the #572 provider-preset stack landing

    ce4d2b4

    docs(devlog): record the #572 provider-preset stack outcome

    6a864de

  11. olddonkey commented on Aug 8, 2026

    @olddonkey
    olddonkey
    on Aug 8, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Progress update (2026-08-08):

    After #1315, the original list has two candidates left:

    1. featherless — needs a bounded, chat/tool-capable discovery query for its very large catalog.
    2. novita — the API shape was previously verified, but canonical-preset operating-entity evidence is still the blocker.

    All batch PRs use “Relates to #572”; none closes this umbrella issue. I will keep #572 open until the remaining candidates are either implemented or explicitly deferred with the reason recorded here.

  12. olddonkey commented on Aug 8, 2026

    @olddonkey
    olddonkey
    on Aug 8, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Progress update (2026-08-08):

    • feat(providers): add Chutes preset #1315 (Chutes) is now ready for review. The first review round's three actionable threads were addressed in 8f4201f5 and are resolved: the test-only private-network opt-in was removed, canonical-host path near-miss coverage was added, and sparse tool-model metadata fallback is now pinned.
    • feat(providers): add Featherless preset #1317 (Featherless) is open as a draft. It uses an authenticated, first-page-only query capped at 128 KiB / 100 raw rows, then independently fail-closes on current-plan availability, no Hugging Face gate, and explicit tool support. The PR includes canonical technical/operator evidence, fixture-only coverage, translated docs, and an explicit security-review request.

    That leaves novita as the only candidate from the original list not implemented or under review. Its API/discovery shape remains viable, but canonical-preset operating-entity evidence is still the blocker; I am not promoting it until that evidence is authoritative enough to name who operates the API and under what developer terms.

    Both open PRs use “Relates to #572”; neither will close this umbrella issue.

  13. olddonkey commented on Aug 8, 2026

    @olddonkey
    olddonkey
    on Aug 8, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Follow-up to the earlier 2026-08-08 progress note:

    A fresh evidence check found that Novita's official Terms were materially revised on 2026-08-05, after the #747 evidence decision. They now expressly cover AI inference APIs, third-party Model Providers, customer Input/Output processing, and Delaware law. The official USPTO TSDR record connects the NOVITA AI mark to Hivemind Labs, Inc. (with the abandoned-application status disclosed), and Hugging Face independently lists Novita as a chat/VLM Inference Providers partner.

    On that updated record, #1318 is now open and ready for review. The PR asks maintainers/security to confirm the operator and routing evidence explicitly rather than assuming it. Its implementation also corrects the old draft design: the live model list is public, so key validation is unknown; catalog admission requires both model_type: chat and chat/completions; discovery is bounded to 512 KiB / 256 raw rows.

    Current remaining review queue for this umbrella:

    Every provider in the original list is now either merged or represented by one of these open review-ready PRs. All three use “Relates to #572”; none closes this umbrella. Please keep #572 open until the outstanding PRs are merged or explicitly declined.

  14. Wibias commented on Aug 8, 2026

    @Wibias
    Wibias
    on Aug 8, 2026
    Contributor

    Tracking policy

    This issue is the umbrella for the explicitly listed OpenAI-compatible provider batch.

    Provider-specific PRs/issues that implement one of the providers listed above should reference this issue as their parent/umbrella.

    Do not fold providers with materially different authentication or transport contracts into this issue.

  15. Wibias
    changed the title [-][Provider] Promote a verified batch of OpenAI-compatible inference providers[/-] [+][Provider] Finish remaining OpenAI-compatible provider batch: Chutes, Featherless, Novita[/+] on Aug 9, 2026
  16. Wibias
    closed this as completedon Aug 10, 2026
  17. agentHits
    added 3 commits that reference this issue on Sep 17, 2026

    docs(devlog): move three inferred PR-issue links out of the literal l…

    78a1f60

    docs(devlog): plan the lidge-jun#572 provider-preset stack landing

    015b667

    docs(devlog): record the lidge-jun#572 provider-preset stack outcome

    8645bbc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

catalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or requestproviderProvider adapters, OpenAI-compat presets, upstream API quirksroadmapAccepted long-term work item; needs its own project cycletoolstool_calls, MCP, web-search / sidecar tools

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions