feat: Add tracing configuration and instances documentation

docs/adding-custom-metadata.md

@@ -0,0 +1,116 @@
+# Adding Custom Metadata to Traces
+
+## Automatic Metadata from RequestContext
+
+Instead of manually adding metadata to each span, you can configure Mastra to automatically extract values from RequestContext and attach them as metadata to all spans in a trace.
+
+### Configuration-Level Extraction
+
+Define which RequestContext keys to extract in your tracing configuration:
+
+```typescript
+export const mastra = new Mastra({
+  observability: new Observability({
+    configs: {
+      default: {
+        serviceName: "my-service",
+        requestContextKeys: ["userId", "environment", "tenantId"],
+        exporters: [new DefaultExporter()],
+      },
+    },
+  }),
+});
+```
+
+### Per-Request Additions
+
+You can add trace-specific keys using `tracingOptions.requestContextKeys`:
+
+```typescript
+const requestContext = new RequestContext();
+requestContext.set("userId", "user-123");
+requestContext.set("environment", "production");
+requestContext.set("experimentId", "exp-789");
+
+const result = await agent.generate({
+  messages: [{ role: "user", content: "Hello" }],
+  requestContext,
+  tracingOptions: {
+    requestContextKeys: ["experimentId"], // Adds to configured keys
+  },
+});
+```
+
+### Nested Value Extraction
+
+Use dot notation to extract nested values from RequestContext:
+
+```typescript
+export const mastra = new Mastra({
+  observability: new Observability({
+    configs: {
+      default: {
+        requestContextKeys: ["user.id", "session.data.experimentId"],
+        exporters: [new DefaultExporter()],
+      },
+    },
+  }),
+});
+```
+
+## Adding Tags to Traces
+
+Tags are string labels that help you categorize and filter traces. Use `tracingOptions.tags`:
+
+```typescript
+const result = await agent.generate({
+  messages: [{ role: "user", content: "Hello" }],
+  tracingOptions: {
+    tags: ["production", "experiment-v2", "user-request"],
+  },
+});
+```
+
+## Manual Metadata Addition
+
+You can add metadata to any span using the tracing context:
+
+```typescript
+execute: async ({ inputData, tracingContext }) => {
+  const startTime = Date.now();
+  const response = await fetch(inputData.endpoint);
+
+  // Add custom metadata to the current span
+  tracingContext.currentSpan?.update({
+    metadata: {
+      apiStatusCode: response.status,
+      endpoint: inputData.endpoint,
+      responseTimeMs: Date.now() - startTime,
+      userTier: inputData.userTier,
+      region: process.env.AWS_REGION,
+    },
+  });
+
+  return await response.json();
+};
+```
+
+## Child Spans and Metadata Extraction
+
+When creating child spans within tools or workflow steps, you can pass the `requestContext` parameter to enable metadata extraction:
+
+```typescript
+execute: async ({ tracingContext, requestContext }) => {
+  // Create child span WITH requestContext - gets metadata extraction
+  const dbSpan = tracingContext.currentSpan?.createChildSpan({
+    type: "generic",
+    name: "database-query",
+    requestContext, // Pass to enable metadata extraction
+  });
+
+  const results = await db.query("SELECT * FROM users");
+  dbSpan?.end({ output: results });
+
+  return results;
+};
+```

docs/creating-child-spans.md

@@ -0,0 +1,75 @@
+# Creating Child Spans
+
+Child spans allow you to track fine-grained operations within your workflow steps or tools. They provide visibility into sub-operations like database queries, API calls, file operations, or complex calculations. This hierarchical structure helps you identify performance bottlenecks and understand the exact sequence of operations.
+
+## Creating Child Spans
+
+Create child spans inside a tool call or workflow step to track specific operations:
+
+```typescript
+execute: async ({ inputData, tracingContext }) => {
+  // Create another child span for the main database operation
+  const querySpan = tracingContext.currentSpan?.createChildSpan({
+    type: "generic",
+    name: "database-query",
+    input: { query: inputData.query },
+    metadata: { database: "production" },
+  });
+
+  try {
+    const results = await db.query(inputData.query);
+    querySpan?.end({
+      output: results.data,
+      metadata: {
+        rowsReturned: results.length,
+        queryTimeMs: results.executionTime,
+        cacheHit: results.fromCache,
+      },
+    });
+    return results;
+  } catch (error) {
+    querySpan?.error({
+      error,
+      metadata: { retryable: isRetryableError(error) },
+    });
+    throw error;
+  }
+};
+```
+
+Child spans automatically inherit the trace context from their parent, maintaining the relationship hierarchy in your observability platform.
+
+## Child Span Options
+
+When creating child spans, you can specify:
+
+- `type`: The span type ("generic", "llm", "tool", etc.)
+- `name`: A descriptive name for the operation
+- `input`: Input data for the operation
+- `metadata`: Additional context information
+- `requestContext`: For automatic metadata extraction (optional)
+
+## Error Handling in Child Spans
+
+Use the `error()` method to record errors in child spans:
+
+```typescript
+querySpan?.error({
+  error,
+  metadata: { retryable: isRetryableError(error) },
+});
+```
+
+## Ending Child Spans
+
+Always end child spans with the `end()` method, providing output data and final metadata:
+
+```typescript
+querySpan?.end({
+  output: results,
+  metadata: {
+    rowsReturned: results.length,
+    executionTime: Date.now() - startTime,
+  },
+});
+```

docs/opentelemetry-semantic-conventions.md

@@ -0,0 +1,22 @@
+# OpenTelemetry Semantic Conventions
+
+## Span Naming
+
+• LLM Operations: `chat {model}`
+• Tool Execution: `execute_tool {tool_name}`
+• Agent Runs: `invoke_agent {agent_id}`
+• Workflow Runs: `invoke_workflow {workflow_id}`
+
+## Key Attributes
+
+• `gen_ai.operation.name` - Operation type (chat, tool.execute, etc.)
+• `gen_ai.provider.name` - AI provider (openai, anthropic, etc.)
+• `gen_ai.request.model` - Model identifier
+• `gen_ai.input.messages` - Chat history provided to the model
+• `gen_ai.output.messages` - Messages returned by the model
+• `gen_ai.usage.input_tokens` - Number of input tokens
+• `gen_ai.usage.output_tokens` - Number of output tokens
+• `gen_ai.request.temperature` - Sampling temperature
+• `gen_ai.response.finish_reasons` - Completion reasons
+
+The exporter follows [OpenTelemetry Semantic Conventions for GenAI v1.38.0](https://github.com/open-telemetry/semantic-conventions/tree/v1.38.0/docs/gen-ai), ensuring compatibility with observability platforms.

docs/tracing-configuration-reference.md

@@ -0,0 +1,128 @@
+# Mastra Tracing Configuration Reference
+
+## ObservabilityRegistryConfig
+
+```typescript
+interface ObservabilityRegistryConfig {
+  default?: { enabled?: boolean };
+  configs?: Record<string, Omit<ObservabilityInstanceConfig, "name"> | ObservabilityInstance>;
+  configSelector?: ConfigSelector;
+}
+```
+
+## ObservabilityInstanceConfig
+
+```typescript
+interface ObservabilityInstanceConfig {
+  name: string;
+  serviceName: string;
+  sampling?: SamplingStrategy;
+  exporters?: ObservabilityExporter[];
+  spanOutputProcessors?: SpanOutputProcessor[];
+  includeInternalSpans?: boolean;
+  requestContextKeys?: string[];
+  serializationOptions?: SerializationOptions;
+}
+```
+
+## SerializationOptions
+
+Options for controlling how span data is serialized before export.
+
+```typescript
+interface SerializationOptions {
+  maxStringLength?: number;
+  maxDepth?: number;
+  maxArrayLength?: number;
+  maxObjectKeys?: number;
+}
+```
+
+## SamplingStrategy
+
+```typescript
+type SamplingStrategy =
+  | { type: "always" }
+  | { type: "never" }
+  | { type: "ratio"; probability: number }
+  | { type: "custom"; sampler: (options?: TracingOptions) => boolean };
+```
+
+## ConfigSelector
+
+```typescript
+type ConfigSelector = (
+  options: ConfigSelectorOptions,
+  availableConfigs: ReadonlyMap<string, ObservabilityInstance>,
+) => string | undefined;
+```
+
+## ConfigSelectorOptions
+
+```typescript
+interface ConfigSelectorOptions {
+  requestContext?: RequestContext;
+}
+```
+
+## Registry Methods
+
+### registerInstance
+```typescript
+registerInstance(name: string, instance: ObservabilityInstance, isDefault?: boolean): void;
+```
+Registers an observability instance in the registry.
+
+### getInstance
+```typescript
+getInstance(name: string): ObservabilityInstance | undefined;
+```
+Retrieves an observability instance by name.
+
+### getDefaultInstance
+```typescript
+getDefaultInstance(): ObservabilityInstance | undefined;
+```
+Returns the default observability instance.
+
+### getSelectedInstance
+```typescript
+getSelectedInstance(options: ConfigSelectorOptions): ObservabilityInstance | undefined;
+```
+Returns the observability instance selected by the config selector or default.
+
+### listInstances
+```typescript
+listInstances(): ReadonlyMap<string, ObservabilityInstance>;
+```
+Returns all registered observability instances.
+
+### hasInstance
+```typescript
+hasInstance(name: string): boolean;
+```
+Checks if an observability instance exists.
+
+### setConfigSelector
+```typescript
+setConfigSelector(selector: ConfigSelector): void;
+```
+Sets the config selector function.
+
+### unregisterInstance
+```typescript
+unregisterInstance(name: string): boolean;
+```
+Removes an observability instance from the registry.
+
+### clear
+```typescript
+clear(): void;
+```
+Clears all instances without shutdown.
+
+### shutdown
+```typescript
+async shutdown(): Promise<void>;
+```
+Shuts down all observability instances and clears the registry.