Skip to content

feat: add a few metrics #8910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jif-oai merged 3 commits into from
Jan 8, 2026
Merged

feat: add a few metrics #8910

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9940d2c
feat: add a few metrics
jif-oai Jan 8, 2026
6c213b4
nit
jif-oai Jan 8, 2026
2d223e1
prettier
jif-oai Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions codex-rs/core/src/codex.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -645,6 +645,18 @@ impl Session {
session_configuration.session_source.clone(),
);
config.features.emit_metrics(&otel_manager);
otel_manager.counter(
"codex.session.started",
1,
&[(
"is_git",
if get_git_repo_root(&session_configuration.cwd).is_some() {
"true"
} else {
"false"
},
)],
);

otel_manager.conversation_starts(
config.model_provider.name.as_str(),
Expand Down Expand Up @@ -1757,6 +1769,7 @@ mod handlers {
use codex_protocol::protocol::TurnAbortReason;
use codex_protocol::protocol::WarningEvent;

use crate::context_manager::is_user_turn_boundary;
use codex_protocol::user_input::UserInput;
use codex_rmcp_client::ElicitationAction;
use codex_rmcp_client::ElicitationResponse;
Expand Down Expand Up @@ -2111,6 +2124,18 @@ mod handlers {
.terminate_all_processes()
.await;
info!("Shutting down Codex instance");
let turn_count = sess
.clone_history()
.await
.get_history()
.iter()
.filter(|item| is_user_turn_boundary(item))
.count();
sess.services.otel_manager.counter(
"conversation.turn.count",
Copy link
Contributor

@gt-oai gt-oai Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does is_user_turn_boundary effectively count user turns? Would conversation.user_turn.count be more explicit in that case?

If "turn" is accepted convention for user turn then ignore me.

Copy link
Collaborator Author

@jif-oai jif-oai Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is :)

i64::try_from(turn_count).unwrap_or(0),
&[],
);

// Gracefully flush and shutdown rollout recorder on session end so tests
// that inspect the rollout file do not race with the background writer.
Expand Down Expand Up @@ -2834,6 +2859,7 @@ pub(super) fn get_last_assistant_message_from_turn(responses: &[ResponseItem]) -
#[cfg(test)]
pub(crate) use tests::make_session_and_context;

use crate::git_info::get_git_repo_root;
#[cfg(test)]
pub(crate) use tests::make_session_and_context_with_rx;

Expand Down
2 changes: 1 addition & 1 deletion codex-rs/core/src/context_manager/history.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ fn is_session_prefix(text: &str) -> bool {
lowered.starts_with("<environment_context>")
}

fn is_user_turn_boundary(item: &ResponseItem) -> bool {
pub(crate) fn is_user_turn_boundary(item: &ResponseItem) -> bool {
let ResponseItem::Message { role, content, .. } = item else {
return false;
};
Expand Down
1 change: 1 addition & 0 deletions codex-rs/core/src/context_manager/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ mod history;
mod normalize;

pub(crate) use history::ContextManager;
pub(crate) use history::is_user_turn_boundary;
18 changes: 10 additions & 8 deletions codex-rs/core/src/tasks/compact.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,18 @@ impl SessionTask for CompactTask {
session.as_ref(),
&ctx.client.get_provider(),
) {
let _ = session
.services
.otel_manager
.counter("codex.task.compact.remote", 1, &[]);
let _ = session.services.otel_manager.counter(
"codex.task.compact",
1,
&[("type", "remote")],
);
crate::compact_remote::run_remote_compact_task(session, ctx).await
} else {
let _ = session
.services
.otel_manager
.counter("codex.task.compact.local", 1, &[]);
let _ = session.services.otel_manager.counter(
"codex.task.compact",
1,
&[("type", "local")],
);
crate::compact::run_compact_task(session, ctx, input).await
}

Expand Down
39 changes: 22 additions & 17 deletions docs/telemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,25 @@ This section list all the metrics exported by Codex when locally installed.

## Metrics catalog

Each metric includes the required fields plus the global context above.

| Metric | Type | Fields | Description |
| ------------------------- | --------- | ------------------------------------- | ------------------------------------------------------------------------------- |
| `approval.requested` | counter | `tool`, `approved` | Tool approval request result (`approved`: `yes` or `no`). |
| `auth.completed` | counter | `status` | Authentication completed (only for ChatGPT authentication). |
| `conversation.compact` | counter | `status`, `number` | Compaction event including the status and the compaction number in the session. |
| `conversation.turn.count` | counter | `role` | User/assistant turns per session. |
| `feature.duration_ms` | histogram | `feature`, `status` | End-to-end feature latency. |
| `feature.used` | counter | `feature` | Feature usage through `/` (e.g., `/undo`, `/review`, ...). |
| `features.state` | counter | `key`, `value` | Feature values that differ from defaults (emit one row per non-default). |
| `mcp.call` | counter | `status` | MCP tool invocation result (`ok` or error string). |
| `model.call.duration_ms` | histogram | `provider`, `status`, `attempt` | Model API request duration. |
| `session.started` | counter | `is_git` | New session created. |
| `tool.call` | counter | `tool`, `status` | Tool invocation result (`ok` or error string). |
| `tool.call.duration_ms` | histogram | `tool`, `status` | Tool execution time. |
| `user.feedback.submitted` | counter | `category`, `include_logs`, `success` | Feedback submission via `/feedback`. |
Each metric includes the required fields plus the global context above. Every metrics are prefixed by `codex.`.

| Metric | Type | Fields | Description |
| ----------------- | ------- | -------------- | ------------------------------------------------------------------------ |
| `features.state` | counter | `key`, `value` | Feature values that differ from defaults (emit one row per non-default). |
| `session.started` | counter | `is_git` | New session created. |
| `task.compact` | counter | `type` | Number of compaction per type (`remote` or `local`) |
| `task.user_shell` | counter | | Number of user shell actions (`!` in the TUI for example) |
| `task.review` | counter | | Number of reviews triggered |
| `task.undo` | counter | | Number of undo made |

### Metrics to be added

| Metric | Type | Fields | Description |
| ------------------------- | --------- | ------------------------------------- | --------------------------------------------------------- |
| `approval.requested` | counter | `tool`, `approved` | Tool approval request result (`approved`: `yes` or `no`). |
| `conversation.turn.count` | counter | | User/assistant turns per session. |
| `mcp.call` | counter | `status` | MCP tool invocation result (`ok` or error string). |
| `model.call.duration_ms` | histogram | `provider`, `status`, `attempt` | Model API request duration. |
| `tool.call` | counter | `tool`, `status` | Tool invocation result (`ok` or error string). |
| `tool.call.duration_ms` | histogram | `tool`, `status` | Tool execution time. |
| `user.feedback.submitted` | counter | `category`, `include_logs`, `success` | Feedback submission via `/feedback`. |
Loading