Skip to content

docs(MDA): add Harbor evals guide for Managed Deep Agents#4923

Open
Christian Bromann (christian-bromann) wants to merge 9 commits into
mainfrom
cb/docs-mda-evals
Open

docs(MDA): add Harbor evals guide for Managed Deep Agents#4923
Christian Bromann (christian-bromann) wants to merge 9 commits into
mainfrom
cb/docs-mda-evals

Conversation

@christian-bromann

@christian-bromann Christian Bromann (christian-bromann) commented Jul 15, 2026

Copy link
Copy Markdown
Member

Overview

Adds Managed Deep Agents docs for Harbor-style evals.

  • New page: Evals — scaffold Harbor tasks (instruction.md + tests/), mda evals compile handoff under .mda/evals/, then run trials with Harbor
  • Updates nav in src/docs.json and refreshes cross-links across CLI, overview, how-it-works, tutorial, identity, and related snippets
  • Reflects the Harbor-only pivot: MDA does not run trials (check / run removed); authors edit harbor-job.json and invoke harbor run

Type of change

Type: New documentation page

Related issues/PRs

Test plan

  • Preview the new Evals page in Mintlify/docs preview
  • Confirm sidebar entry under Managed Deep Agents (after Identity)
  • Spot-check links from CLI, overview, next-steps, and identity cards
  • Confirm docs describe mda evals init + mda evals compile + Harbor (no check / run / defineEvaluator)

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for opening a docs PR, Christian Bromann (@christian-bromann)! When it's ready for review, please add the relevant reviewers:

  • @katmayb or @fjmorris (LangSmith)

@christian-bromann

Copy link
Copy Markdown
Member Author

Updated to call out --environment langsmith (remote Harbor trials) alongside local Docker.

Document scaffolding and running basic Docker/Harbor eval tasks with mda eval, and wire the page into nav and cross-links.
Mention --environment langsmith alongside Docker for Harbor trials.
Document the plural form as the primary CLI surface; note mda eval as an alias.
Keep init/run guidance generic instead of listing starter suite tasks.
Focus the Verifier section on Harbor's reward contract instead of scaffold test runners.
Explain /app, /tests, and /logs/verifier without Harbor-internal wording.
MDA now only scaffolds tasks and compiles a Harbor handoff; trials are run with Harbor. Drop check/run flags and document harbor-job.json.
@github-actions

Copy link
Copy Markdown
Contributor

Mintlify preview branch generated: preview-cbdocs-1784230442-77c063c

Site preview: https://langchain-5e9cc07a-preview-cbdocs-1784230442-77c063c.mintlify.site

Important

Preview links may take a few minutes to start working while the deployment finishes.

Changed documentation pages (preview deep links):

Only the top 5 changed markdown files by diff size are listed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal langsmith For docs changes to LangSmith

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant