feat(agentos-server): HTTP telemetry ingest endpoint + Mongo projection

Add POST /agentos/api/ingest/events so the Python SDK posts telemetry
instead of writing Mongo directly; the server now owns the projection.

- routes/ingest.ts + ingest-auth.ts: batch ingest with a bearer guard
  (open if AGENTOS_INGEST_TOKEN unset; boot warning in that case). Mounted
  before the global 1mb parser + requireAuth with a 5mb body limit.
- stores/telemetry-projection.ts: project each event into agent_registry,
  agent_logs, sessions, chat_sessions, agent_messages. Writes the canonical
  chat_sessions row (fixes library sessions missing from the dashboard list)
  and drops the dead slack_threads write. session_started is the sole creator
  of the sessions doc (no stub corruption on dropped/reordered events).
  Idempotent on event_id. + agent-source.ts (model-strip + library/inline
  source builders, ported from the Python sink).
- Extend agent-log-store.ts (optional rollup fields + idempotent _id), widen
  SessionDoc/MessageDoc, mount + boot warning in index.ts.
- vitest projection tests incl. out-of-order regression; CLAUDE.md updated to
  the HTTP-ingest model.

Also snapshots in-progress AgentOS SPA, observability, and engine work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: abhi-bhat-lyzr

CLAUDE.md

@@ -89,20 +89,21 @@ Optional GitHub repo **variables** (build-time baked into the SPA bundle):
 
 ### 2.3 MongoDB Atlas — AgentOS runtime backend
 
-The Mongo cluster is the source of truth for AgentOS. The Python SDK's `AgentRegistrySink` + `MongoMessageSink` and the AgentOS server both connect here.
+The Mongo cluster is the source of truth for AgentOS. **Only the `agentos-server` connects to Mongo.** As of the post-0.2.1 dev build, the Python SDK no longer writes Mongo directly — it POSTs telemetry to the server's ingest endpoint (`AgentOSHttpSink` → `POST /agentos/api/ingest/events`), and the server owns all writes. (The old `AgentRegistrySink` + `MongoMessageSink` and the `motor` dep were removed — see the Python SDK history below.)
 
-**Collections (database = `AGENTOS_MONGO_DB`):**
-- `agent_registry` — one doc per registered agent (the SDK writes `source.type="library"` for harness-mode agents → AgentOS UI hides the chat-sandbox button for those, see commit `8d829b8`)
+**Collections (database = the server's `MONGO_DATABASE`):**
+- `agent_registry` — one doc per registered agent (the server writes `source.type="library"` for harness-mode agents → AgentOS UI hides the chat-sandbox button for those, see commit `8d829b8`)
 - `agent_logs` — one doc per conversation (one `ComputerAgent` instance = one log row, multi-turn collapses correctly since the 0.2.0 session-id refactor)
-- `sessions` — ordered chat transcript (one doc per session_id, entries appended in order)
+- `sessions` — ordered chat transcript (one doc per session_id, entries appended in order; **`session_started` is the sole creator** of the doc, so a dropped/reordered start can't stub it)
+- `chat_sessions` — the session-index row (`{_id, agent, createdAt, lastMessageAt}`) the dashboard's session list + per-agent `sessionCount`/`lastActivity` read. The server projection writes this so library-mode sessions show up (the old Python sink omitted it).
 - `agent_messages` — per-event audit trail (every assistant_message / tool_use / tool_result lands here)
-- `slack_threads` — chat-channel state
+- `slack_threads` — Slack-bot chat-channel state only; **not** written by the ingest projection (it was dead/legacy for library agents).
 
-**Credentials required (runtime env, anywhere the SDK or server runs):**
-- `AGENTOS_MONGO_URL` — `mongodb+srv://<user>:<pass>@<cluster>.mongodb.net`
-- `AGENTOS_MONGO_DB` — usually `computeragent` / `computeragent-prod` / `computeragent-test` per env
+**Credentials required:**
+- On the **SDK** side: `AGENTOS_INGEST_URL` (e.g. `https://<host>/agentos/api/ingest/events`) + optional `AGENTOS_INGEST_TOKEN` (sent as `Authorization: Bearer …`). No Mongo creds.
+- On the **server** side: `MONGO_URL` + `MONGO_DATABASE` (this is the DB the collections above live in).
 
-**Behaviour:** when `AGENTOS_MONGO_URL` is set, the default telemetry pipeline auto-attaches **both** the registry sink and the message sink. Pre-0.2.0 only the registry sink auto-attached and `agent_messages` was empty — that bug is fixed.
+**Behaviour:** when `AGENTOS_INGEST_URL` is set, the SDK's default telemetry pipeline auto-attaches `AgentOSHttpSink` (gated on the `[agentos]` extra, which is now `httpx`-based). Each event carries a stable `event_id` so the server's writes are idempotent on retry. ⚠️ When the server's `AGENTOS_INGEST_TOKEN` is unset the ingest route is **open** (anonymous writes) — set it on any network-exposed deployment.
 
 ---
 
@@ -253,7 +254,7 @@ Top-level directories (`pnpm` workspace, `turbo` for the build graph):
                            └──────────────────┘
 ```
 
-The Python SDK (`computer-agent-py`) re-implements the harness layer in Python with the same four orthogonal axes. It writes to the same MongoDB collections via `AgentRegistrySink` + `MongoMessageSink` so library-mode Python agents show up in the AgentOS UI alongside TS harness-server-hosted ones.
+The Python SDK (`computer-agent-py`) re-implements the harness layer in Python with the same four orthogonal axes. It POSTs telemetry to the AgentOS server (`AgentOSHttpSink` → `POST /agentos/api/ingest/events`), which projects it into the same MongoDB collections so library-mode Python agents show up in the AgentOS UI alongside TS harness-server-hosted ones. (Through 0.2.x the SDK wrote Mongo directly via `AgentRegistrySink` + `MongoMessageSink`; that was removed in favour of HTTP ingest so the SDK needs no Mongo creds and the schema lives server-side.)
 
 ---
 
@@ -271,9 +272,13 @@ ANTHROPIC_API_KEY=sk-ant-...
 GITCLAW_MODEL_BASE_URL=https://api.lyzr.ai/v1
 OPENAI_API_KEY=sk-...
 
-# AgentOS Mongo (auto-attaches both sinks when set)
-AGENTOS_MONGO_URL=mongodb+srv://user:pass@cluster.mongodb.net
-AGENTOS_MONGO_DB=computeragent
+# AgentOS persistence — SDK POSTs telemetry to the server; the server writes Mongo.
+# On the SDK (library/worker) side:
+AGENTOS_INGEST_URL=https://<agentos-host>/agentos/api/ingest/events
+AGENTOS_INGEST_TOKEN=<shared-secret>          # optional; must match the server's
+# On the agentos-server side (NOT the SDK):
+MONGO_URL=mongodb+srv://user:pass@cluster.mongodb.net
+MONGO_DATABASE=computeragent
 
 # OTel → New Relic
 OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net
@@ -356,6 +361,7 @@ pnpm build && pnpm start                  # node dist/index.js
 | `COOKIE_SECURE` | derived from `NODE_ENV` | Force `true` / `false` explicitly |
 | `AGENTOS_SESSION_SECRET` | random per boot | Cookie-session secret. **Set to a stable value in prod** or sessions are invalidated on restart |
 | `API_AUTH_USER` + `API_AUTH_PASS` | unset | Basic-auth gate on the API. When unset the API is open (relies on network policy) |
+| `AGENTOS_INGEST_TOKEN` | unset | Bearer token guarding `POST /agentos/api/ingest/events` (the Python SDK's telemetry ingest). When unset the route is **open** (anonymous writes to registry/logs/sessions) — set it on any network-exposed pod. The SDK must send the same value as `AGENTOS_INGEST_TOKEN`. |
 | `AGENTOS_RUNTIME` | unset | Default substrate name used by the "Register agent" form (`local` / `bwrap` / `e2b` / `vzvm`) |
 | `AGENTOS_SEED_DEFAULT` | unset | Set to `1` to auto-seed a default agent into the registry on first boot |
 | `AGENTOS_DEFAULT_SOURCE` | `github.com/shreyas-lyzr/general-agent` | Used by the seed agent |

agentos/package.json

@@ -34,6 +34,7 @@
     "react-hook-form": "^7.76.1",
     "react-markdown": "^10.1.0",
     "react-resizable-panels": "^4.11.2",
+    "react-router-dom": "^7",
     "recharts": "^3.8.1",
     "remark-gfm": "^4.0.1",
     "sonner": "^2.0.7",

agentos/src/App.tsx

@@ -1,90 +1,34 @@
-import { useCallback, useEffect, useState } from "react";
 import { Home as HomeIcon, Activity, Shield, Boxes } from "lucide-react";
-import { api, type Agent } from "./api.ts";
-import { LogsTab } from "./components/LogsTab.tsx";
-import { WorkspaceTab } from "./components/WorkspaceTab.tsx";
-import { SchedulesTab } from "./components/SchedulesTab.tsx";
+import { NavLink, Navigate, Outlet, Route, Routes, useLocation, useNavigate } from "react-router-dom";
 import { HomePage } from "./components/HomePage.tsx";
-import { PolicyTab } from "./components/PolicyTab.tsx";
 import { PoliciesPage } from "./components/PoliciesPage.tsx";
 import { ObservabilityTab } from "./components/observability/ObservabilityTab.tsx";
 import { RegistryPage } from "./components/RegistryPage.tsx";
-import { Tabs, TabsList, TabsTrigger } from "./components/ui/tabs.tsx";
-import { Badge } from "./components/ui/badge.tsx";
+import { AgentDashboard } from "./components/AgentDashboard.tsx";
 import { Separator } from "./components/ui/separator.tsx";
-import { PageHeader } from "./components/composite/PageHeader.tsx";
+import { useAgents } from "./context/AgentsContext.tsx";
 import { cn } from "./lib/cn.ts";
 
-type Tab = "chat" | "schedules" | "policy" | "logs";
-type View = "home" | "observability" | "policies" | "registry" | "dashboard";
-
-const NAME_OVERRIDES: Record<string, string> = {
-  "general-agent": "General Agent",
-  "agentos-builder": "AgentOS Builder",
-  "gap-promoter": "GAP Promoter",
-  "framework-translator-agent": "Framework Translator",
-};
-function agentNameFromSource(source: string): string {
-  const slug = source.split("/").pop() ?? source;
-  return NAME_OVERRIDES[slug] ?? slug.replace(/[-_]+/g, " ").replace(/\b\w/g, (c) => c.toUpperCase());
-}
-
-function typeLogo(harness: string): string | null {
-  if (harness === "gitagent") return "/logos/gitagent.png";
-  if (harness === "claude-agent-sdk") return "/logos/claude.svg";
-  if (harness === "deepagents") return "/logos/langchain.svg";
-  return null;
-}
-
-function TypeBadge({ agent, className = "" }: { agent: Agent; className?: string }) {
-  const logo = typeLogo(agent.harness);
+export default function App() {
   return (
-    <Badge variant="secondary" className={cn("gap-1.5 pl-1 pr-2 py-0.5 text-[10px] font-normal", className)}>
-      {logo && (
-        <span className="h-3.5 w-3.5 grid place-items-center rounded-sm bg-background shrink-0">
-          <img src={logo} alt="" className="h-2.5 w-2.5 object-contain" />
-        </span>
-      )}
-      <span className="truncate">{agent.label}</span>
-    </Badge>
+    <Routes>
+      <Route element={<Layout />}>
+        <Route index element={<Navigate to="/home" replace />} />
+        <Route path="home" element={<HomeRoute />} />
+        <Route path="registry" element={<RegistryRoute />} />
+        <Route path="observability" element={<ObservabilityTab />} />
+        <Route path="policies" element={<PoliciesPage />} />
+        <Route path="agents/:name" element={<AgentDashboard />} />
+        <Route path="*" element={<Navigate to="/home" replace />} />
+      </Route>
+    </Routes>
   );
 }
 
-export default function App() {
-  const [view, setView] = useState<View>("home");
-  const [agents, setAgents] = useState<Agent[]>([]);
-  const [agentsLoaded, setAgentsLoaded] = useState(false);
-  const [selected, setSelected] = useState<string | null>(null);
-  const [tab, setTab] = useState<Tab>("chat");
-  const [err, setErr] = useState<string | null>(null);
-  const [launchMessage, setLaunchMessage] = useState<string | null>(null);
-
-  const reloadAgents = useCallback(() => {
-    api
-      .agents()
-      .then(setAgents)
-      .catch((e) => setErr(String(e)))
-      .finally(() => setAgentsLoaded(true));
-  }, []);
-
-  useEffect(() => {
-    reloadAgents();
-  }, [reloadAgents]);
-
-  const agent = agents.find((a) => a.name === selected) ?? null;
-
-  const openAgent = (name: string) => {
-    setSelected(name);
-    setTab("chat");
-    setView("dashboard");
-  };
-
-  const launchFromHome = (agentName: string, message: string) => {
-    setSelected(agentName);
-    setTab("chat");
-    setLaunchMessage(message);
-    setView("dashboard");
-  };
+function Layout() {
+  const { pathname } = useLocation();
+  // "Agent Registry" stays active for both the registry list and any open agent.
+  const registryActive = pathname.startsWith("/registry") || pathname.startsWith("/agents");
 
   return (
     <div className="flex h-full bg-background text-foreground">
@@ -101,30 +45,10 @@ export default function App() {
         </div>
 
         <nav className="px-2 pt-2 space-y-1">
-          <RailButton
-            icon={HomeIcon}
-            label="Home"
-            active={view === "home"}
-            onClick={() => setView("home")}
-          />
-          <RailButton
-            icon={Boxes}
-            label="Agent Registry"
-            active={view === "registry" || view === "dashboard"}
-            onClick={() => setView("registry")}
-          />
-          <RailButton
-            icon={Activity}
-            label="Observability"
-            active={view === "observability"}
-            onClick={() => setView("observability")}
-          />
-          <RailButton
-            icon={Shield}
-            label="Policies"
-            active={view === "policies"}
-            onClick={() => setView("policies")}
-          />
+          <RailLink to="/home" icon={HomeIcon} label="Home" />
+          <RailLink to="/registry" icon={Boxes} label="Agent Registry" active={registryActive} />
+          <RailLink to="/observability" icon={Activity} label="Observability" />
+          <RailLink to="/policies" icon={Shield} label="Policies" />
         </nav>
 
         <div className="flex-1" />
@@ -135,103 +59,68 @@ export default function App() {
 
       {/* Main */}
       <main className="flex-1 flex flex-col min-w-0">
-        {view === "policies" ? (
-          <PoliciesPage />
-        ) : view === "home" ? (
-          <HomePage agents={agents} onLaunch={launchFromHome} onOpenDashboard={() => agents[0] && openAgent(agents[0].name)} />
-        ) : view === "observability" ? (
-          <ObservabilityTab />
-        ) : view === "registry" ? (
-          <RegistryPage
-            agents={agents}
-            loaded={agentsLoaded}
-            err={err}
-            selected={selected}
-            onOpenAgent={openAgent}
-            onReload={reloadAgents}
-          />
-        ) : agent ? (
-          <>
-            <PageHeader
-              title={
-                <span className="flex items-center gap-2">
-                  {agentNameFromSource(agent.sourceUrl ?? "")}
-                  <TypeBadge agent={agent} />
-                </span>
-              }
-              description={
-                <>
-                  {agent.harness} · {agent.model ?? "default model"}
-                  {!agent.sandboxCapable && (
-                    <span className="ml-2 text-warning">one-shot · no memory across turns</span>
-                  )}
-                </>
-              }
-              actions={
-                <Tabs value={tab} onValueChange={(v) => setTab(v as Tab)}>
-                  <TabsList>
-                    <TabsTrigger value="chat">Chat</TabsTrigger>
-                    <TabsTrigger value="schedules">Schedules</TabsTrigger>
-                    <TabsTrigger value="policy">Policy</TabsTrigger>
-                    <TabsTrigger value="logs">Logs</TabsTrigger>
-                  </TabsList>
-                </Tabs>
-              }
-            />
-            <section className="flex-1 min-h-0">
-              {tab === "chat" && (
-                <WorkspaceTab
-                  key={agent.name}
-                  agent={agent.name}
-                  sandboxCapable={agent.sandboxCapable}
-                  liveChatCapable={agent.liveChatCapable !== false}
-                  initialMessage={launchMessage}
-                  onConsumedInitial={() => setLaunchMessage(null)}
-                />
-              )}
-              {tab === "schedules" && <SchedulesTab key={agent.name} agent={agent.name} agentLabel={agent.label} />}
-              {tab === "policy" && (
-                <PolicyTab
-                  key={agent.name}
-                  agent={agent.name}
-                  agentLabel={agent.label}
-                  onManagePolicies={() => setView("policies")}
-                />
-              )}
-              {tab === "logs" && <LogsTab key={agent.name} agent={agent.name} />}
-            </section>
-          </>
-        ) : (
-          <div className="flex-1 grid place-items-center text-muted-foreground">
-            {err ? <span className="text-destructive">{err}</span> : "Select an agent"}
-          </div>
-        )}
+        <Outlet />
       </main>
     </div>
   );
 }
 
-function RailButton({
+// ── Route wrappers — inject navigate-based callbacks so the page components
+//    keep their existing prop contracts and need no router awareness. ──
+
+function HomeRoute() {
+  const { agents } = useAgents();
+  const navigate = useNavigate();
+  return (
+    <HomePage
+      agents={agents}
+      onLaunch={(name, message) => navigate(`/agents/${encodeURIComponent(name)}`, { state: { message } })}
+      onOpenDashboard={() => navigate(agents[0] ? `/agents/${encodeURIComponent(agents[0].name)}` : "/registry")}
+    />
+  );
+}
+
+function RegistryRoute() {
+  const { agents, loaded, err, reload } = useAgents();
+  const navigate = useNavigate();
+  return (
+    <RegistryPage
+      agents={agents}
+      loaded={loaded}
+      err={err}
+      selected={null}
+      onOpenAgent={(name) => navigate(`/agents/${encodeURIComponent(name)}`)}
+      onReload={reload}
+    />
+  );
+}
+
+function RailLink({
+  to,
   icon: Icon,
   label,
   active,
-  onClick,
 }: {
+  to: string;
   icon: React.ComponentType<{ className?: string }>;
   label: string;
-  active: boolean;
-  onClick: () => void;
+  /** Optional override; defaults to react-router's own active match. */
+  active?: boolean;
 }) {
   return (
-    <button
-      onClick={onClick}
-      className={cn(
-        "w-full text-left rounded-md px-3 py-2 text-sm transition-colors flex items-center gap-2 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring",
-        active ? "bg-muted ring-1 ring-primary/40 text-foreground" : "hover:bg-muted/60 text-muted-foreground",
-      )}
+    <NavLink
+      to={to}
+      className={({ isActive }) =>
+        cn(
+          "w-full text-left rounded-md px-3 py-2 text-sm transition-colors flex items-center gap-2 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring",
+          (active ?? isActive)
+            ? "bg-muted ring-1 ring-primary/40 text-foreground"
+            : "hover:bg-muted/60 text-muted-foreground",
+        )
+      }
     >
       <Icon className="h-4 w-4" />
       <span>{label}</span>
-    </button>
+    </NavLink>
   );
 }