Skip to content

[WIP] add events for dart mcp server tool invocations #2112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jakemac53
Copy link
Contributor

This is more a proposal for potential new analytics events than a PR intended to be merged right now.

Specifically, we should consider whether we want a more general purpose event type, or specific events for different MCP server interactions.

For example we do expose runtime errors as both a tool and a resource currently, depending on the client support. Do we want a separate event to capture access to resources, or should we try and make this more general so we can add that kind of event later on without changes to this package?

Copy link

Package publishing

Package Version Status Publish tag (post-merge)
package:bazel_worker 1.1.3 already published at pub.dev
package:benchmark_harness 2.4.0-wip WIP (no publish necessary)
package:boolean_selector 2.1.2 already published at pub.dev
package:browser_launcher 1.1.3 already published at pub.dev
package:cli_config 0.2.1-wip WIP (no publish necessary)
package:cli_util 0.4.2 already published at pub.dev
package:clock 1.1.3-wip WIP (no publish necessary)
package:code_builder 4.10.2-wip WIP (no publish necessary)
package:coverage 1.14.1 already published at pub.dev
package:csslib 1.0.2 already published at pub.dev
package:extension_discovery 2.1.0 already published at pub.dev
package:file 7.0.2-wip WIP (no publish necessary)
package:file_testing 3.1.0-wip WIP (no publish necessary)
package:glob 2.1.3 already published at pub.dev
package:graphs 2.3.3-wip WIP (no publish necessary)
package:html 0.15.6 already published at pub.dev
package:io 1.1.0-wip WIP (no publish necessary)
package:json_rpc_2 4.0.0 already published at pub.dev
package:markdown 7.3.1-wip WIP (no publish necessary)
package:mime 2.0.0 already published at pub.dev
package:oauth2 2.0.4-wip WIP (no publish necessary)
package:package_config 2.3.0-wip WIP (no publish necessary)
package:pool 1.5.2-wip WIP (no publish necessary)
package:process 5.0.4 already published at pub.dev
package:pub_semver 2.2.0 already published at pub.dev
package:pubspec_parse 1.5.0 already published at pub.dev
package:source_map_stack_trace 2.1.3-wip WIP (no publish necessary)
package:source_maps 0.10.14-wip WIP (no publish necessary)
package:source_span 1.10.1 already published at pub.dev
package:sse 4.1.8 already published at pub.dev
package:stack_trace 1.12.1 already published at pub.dev
package:stream_channel 2.1.4 already published at pub.dev
package:stream_transform 2.1.2-wip WIP (no publish necessary)
package:string_scanner 1.4.1 already published at pub.dev
package:term_glyph 1.2.3-wip WIP (no publish necessary)
package:test_reflective_loader 0.3.0 ready to publish test_reflective_loader-v0.3.0
package:timing 1.0.2 already published at pub.dev
package:unified_analytics 8.0.1 already published at pub.dev
package:watcher 1.1.2 already published at pub.dev
package:yaml 3.1.3 already published at pub.dev
package:yaml_edit 2.2.2 already published at pub.dev

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

Copy link

github-actions bot commented Jun 17, 2025

PR Health

Breaking changes ⚠️
Package Change Current Version New Version Needed Version Looking good?
unified_analytics Breaking 8.0.1 8.0.1 9.0.0
Got "8.0.1" expected >= "9.0.0" (breaking changes)
⚠️

This check can be disabled by tagging the PR with skip-breaking-check.

Changelog Entry
Package Changed Files
package:unified_analytics pkgs/unified_analytics/lib/src/enums.dart
pkgs/unified_analytics/lib/src/event.dart

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

Coverage ⚠️
File Coverage
pkgs/unified_analytics/lib/src/enums.dart 💚 100 %
pkgs/unified_analytics/lib/src/event.dart 💔 94 % ⬇️ 4 %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ⚠️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbols
unified_analytics Condition
PersistedSurvey
GAClient
UserProperty

This check can be disabled by tagging the PR with skip-leaking-check.

License Headers ✔️
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/bazel_worker/benchmark/benchmark.dart
pkgs/bazel_worker/example/client.dart
pkgs/bazel_worker/example/worker.dart
pkgs/benchmark_harness/integration_test/perf_benchmark_test.dart
pkgs/boolean_selector/example/example.dart
pkgs/clock/lib/clock.dart
pkgs/clock/lib/src/clock.dart
pkgs/clock/lib/src/default.dart
pkgs/clock/lib/src/stopwatch.dart
pkgs/clock/lib/src/utils.dart
pkgs/clock/test/clock_test.dart
pkgs/clock/test/default_test.dart
pkgs/clock/test/stopwatch_test.dart
pkgs/clock/test/utils.dart
pkgs/coverage/lib/src/coverage_options.dart
pkgs/html/example/main.dart
pkgs/html/lib/dom.dart
pkgs/html/lib/dom_parsing.dart
pkgs/html/lib/html_escape.dart
pkgs/html/lib/parser.dart
pkgs/html/lib/src/constants.dart
pkgs/html/lib/src/encoding_parser.dart
pkgs/html/lib/src/html_input_stream.dart
pkgs/html/lib/src/list_proxy.dart
pkgs/html/lib/src/query_selector.dart
pkgs/html/lib/src/token.dart
pkgs/html/lib/src/tokenizer.dart
pkgs/html/lib/src/treebuilder.dart
pkgs/html/lib/src/utils.dart
pkgs/html/test/dom_test.dart
pkgs/html/test/parser_feature_test.dart
pkgs/html/test/parser_test.dart
pkgs/html/test/query_selector_test.dart
pkgs/html/test/selectors/level1_baseline_test.dart
pkgs/html/test/selectors/level1_lib.dart
pkgs/html/test/selectors/selectors.dart
pkgs/html/test/support.dart
pkgs/html/test/tokenizer_test.dart
pkgs/html/test/trie_test.dart
pkgs/html/tool/generate_trie.dart
pkgs/pubspec_parse/test/git_uri_test.dart
pkgs/stack_trace/example/example.dart
pkgs/watcher/test/custom_watcher_factory_test.dart
pkgs/yaml_edit/example/example.dart

@kenzieschmoll
Copy link
Contributor

Cross linking tracking issue: #2112

@kenzieschmoll
Copy link
Contributor

Specifically, we should consider whether we want a more general purpose event type, or specific events for different MCP server interactions.

For example we do expose runtime errors as both a tool and a resource currently, depending on the client support. Do we want a separate event to capture access to resources, or should we try and make this more general so we can add that kind of event later on without changes to this package?

We could add a parameter type instead where type is 'tool', 'resource', 'prompt', etc. That would make it easier to tweak analytics events down the road without needing to make a change to this package. We took a more general approach for DevTools analytics sent through the unified_analytics package and it made development much simpler to not have to change & publish this package just to add a new event.

@jakemac53
Copy link
Contributor Author

We could add a parameter type instead where type is 'tool', 'resource', 'prompt', etc. That would make it easier to tweak analytics events down the road without needing to make a change to this package. We took a more general approach for DevTools analytics sent through the unified_analytics package and it made development much simpler to not have to change & publish this package just to add a new event.

Yeah, I just don't know if we need more specific types or not, if generic events are fine I would lean towards that direction.

@kenzieschmoll
Copy link
Contributor

Generic events are fine to use. We can just pass some event label in the data and then use that to look it up in queries. Take a look at devtoolsEvent for an example. We send every DevTools analytics event through this single unified analytics event.

@jakemac53
Copy link
Contributor Author

Ok, I made the events more generic and also added a field for the MCP server version.

@jakemac53
Copy link
Contributor Author

(note that I will update tests once we agree on a structure for these events)

@jakemac53
Copy link
Contributor Author

jakemac53 commented Jun 17, 2025

Fwiw, when I tested using these events in the actual MCP server, I am getting errors because of the CustomMetrics field.

It doesn't appear that Maps are actually allowed, so this whole CustomMetrics class seems like it needs to be deprecated.

We could encode all the event data as a String... although that seems less than ideal. May want to move as many shared components out as possible (like elapsed milliseconds).

Nevermind, I see the DevTools strategy is just to flatten all the fields directly into the parent map, that works.

Copy link
Contributor

@kenzieschmoll kenzieschmoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once this is approved from a product perspective @anderdobo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants