Skip to content

feat: validate custom NVIDIA NIM model availability - #492

Draft
jyje wants to merge 1 commit into
langchain-ai:mainfrom
jyje:feat/nvidia-model-availability-validation
Draft

feat: validate custom NVIDIA NIM model availability#492
jyje wants to merge 1 commit into
langchain-ai:mainfrom
jyje:feat/nvidia-model-availability-validation

Conversation

@jyje

@jyje jyje commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Validate selected models against a custom NVIDIA_BASE_URL NIM-compatible endpoint before inference starts.
  • Treat a model missing from that endpoint's /models response as unavailable.
  • Preserve inference when credentials are missing, the lookup fails, or the default NVIDIA hosted endpoint is used.

Why

A custom NVIDIA NIM deployment exposes the models actually loaded by that endpoint. Checking its model catalogue before inference gives users an actionable error instead of waiting for the first inference request to fail.

The default NVIDIA hosted endpoint remains non-authoritative for this check, so its catalogue cannot block inference without stronger entitlement guarantees.

Validation

  • pnpm run format:check
  • pnpm run lint:check
  • pnpm test
    • 174 test files passed, 2 skipped
    • 2,330 tests passed, 3 skipped
  • Fresh live GitHub Actions verification after conflict resolution: NVIDIA availability contract
    • verified PR head: d94ed3cd78dcd74ac2d290b9db97a796917fcc2c
    • positive: the selected NVIDIA model returned available
    • negative: a nonexistent model returned unavailable
    • fallbacks: missing credentials, the hosted default endpoint, and a lookup failure returned unknown
pr_head_sha=d94ed3cd78dcd74ac2d290b9db97a796917fcc2c
verification_sha=167804f283f08bddcb31c2a65f0f047233e75d5f
provider=nvidia
endpoint_mode=custom-nim-compatible
selected_model=nvidia/nemotron-3-super-120b-a12b
case=positive provider=nvidia model=nvidia/nemotron-3-super-120b-a12b status=available
case=negative provider=nvidia model=openwiki-live-verification-nonexistent-model status=unavailable
case=no-credential provider=nvidia status=unknown
case=hosted-default provider=nvidia status=unknown
case=lookup-failure provider=nvidia status=unknown
Test Files 1 passed (1)
Tests 5 passed (5)

The temporary live-verification workflow is not included in this PR branch.

Related issue

This is the NVIDIA provider follow-up for #490 and reuses the shared availability contract introduced in #491.

@jyje
jyje force-pushed the feat/nvidia-model-availability-validation branch from f86ae36 to d94ed3c Compare August 13, 2026 12:44
@changeset-bot

changeset-bot Bot commented Aug 13, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d94ed3c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
openwiki Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant