dotnet-counters does not report values from multiple Metrics Meters with distinct tags

[PJB3005]

Description

I was adding built-in support for System.Diagnostics.Metrics to a library I maintain. In the library I want it to be possible to report separate metrics for each instance of a network socket, so I made each object instantiate a Meter and have an API for passing in a list of tags to said meter to distinguish them. Then I can just instantiate a new observable counter/gauge for each property I want to report and it looks pretty nice. For reference, here is the code.

However, dotnet counters only seems to report the metrics from one meter. In this example I have a separate meter with bind=:: and bind=0.0.0.0 created, but I only get results for bind=::.

Name                                                               Current Value
[Lidgren.Network.NetPeer]
    messages_dropped (Count)
        bind=::                                                            0
    messages_received (Count)
        bind=::                                                       97,375
    messages_resent (Count)
        bind=::,drop_reason=delay                                         20
        bind=::,drop_reason=hole                                         481
    messages_sent (Count)
        bind=::                                                      106,354
    packets_received (Count)
        bind=::                                                       82,033
    packets_sent (Count)
        bind=::                                                      102,192
    recycle_pool_bytes (bytes)
        bind=::                                                      922,600
    sent_bytes (bytes)
        bind=::                                                    8,875,636
    storage_allocated_bytes (bytes)
        bind=::                                                      932,688

Cross-referencing against prometheus-net¹, these values are correctly reported however.

> curl localhost:44880/metrics --silent | rg -i lidgren
# HELP lidgren_network_netpeer_packets_sent Number of packets sent by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_packets_sent gauge
lidgren_network_netpeer_packets_sent{bind="0.0.0.0"} 93498
lidgren_network_netpeer_packets_sent{bind="::"} 94293
# HELP lidgren_network_netpeer_packets_received Number of packets received by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_packets_received gauge
lidgren_network_netpeer_packets_received{bind="0.0.0.0"} 74491
lidgren_network_netpeer_packets_received{bind="::"} 75749
# HELP lidgren_network_netpeer_messages_sent Number of messages sent by this NetPeer. (ObservableCounter`1)
# TYPE lidgren_network_netpeer_messages_sent gauge
lidgren_network_netpeer_messages_sent{bind="0.0.0.0"} 97988
lidgren_network_netpeer_messages_sent{bind="::"} 98752

I already noticed that the same thing happened if I created two separate instruments with identical name but different tags, but I figured that one was at least somewhat reasonable. Instead I ended up just using callback-form that returns multiple measurements for the counter that needed to report tags.

Is this intended behavior or a bug? If it's intended behavior, I suppose I should make a static Meter instance and use it for all peers, but that would be rather annoying and inflexible IMO.

Configuration

Tool: dotnet-counters, version 8.0.510501+8c08c89a0643d31db91e119b1adb463be3e0ffe5
.NET runtime I'm testing against: .NET 8.0.3
OS: Windows 11 Pro, 23H2, build 22631.3296
Architecture: x64
Environment: just running stuff directly from my IDE/terminal, no containers or whatever.

dotnet info

> dotnet --info
.NET SDK:
 Version:           8.0.103
 Commit:            6a90b4b4bc
 Workload version:  8.0.100-manifests.4e94be9c

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.103\

.NET workloads installed:
 Workload version: 8.0.100-manifests.4e94be9c
There are no installed workloads to display.

Host:
  Version:      9.0.0-preview.1.24080.9
  Architecture: x64
  Commit:       1d1bf92fcf

.NET SDKs installed:
  8.0.100 [C:\Program Files\dotnet\sdk]
  8.0.103 [C:\Program Files\dotnet\sdk]
  9.0.100-preview.1.24101.2 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0-preview.1.24081.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0-preview.1.24080.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 9.0.0-preview.1.24081.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  E:\ss14\space-station-14\RobustToolbox\global.json

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Regression?

Meter/instrument tags are a new feature in .NET 8, so I doubt it. I had trouble before because the tags wouldn't show up at all, but that was fixed by just making sure my dotnet-counters was at the latest version.

Footnotes

1. I just made a PR to make prometheus-net respect meter/instrument tags. ↩

[noahfalk]

I'm pretty sure are hitting a known issue tracked here and the current plan is to fix it during .NET 9 for the scenario involving dotnet-counters. I wouldn't take that as a promise, but I think its likely.

Just so you are aware OpenTelemetry.NET also doesn't support multiple Meters distinguished solely by tags and at the moment the OTel spec officially says that isn't supported. Others have also encountered this issue and asked them to please support it in the future. I believe they are looking into it but if you care about OpenTelmetry support you can always open an issue in OTel.NET repo. The changes needed to make dotnet-counters and OTel support this are fully independent of one another.

Hope that helps explain where things are!
-Noah

[PJB3005]

at the moment the OTel spec officially says that isn't supported.

Damn, that's kind of annoying. Guess I have to change my library code then. Thanks though!

[tommcdon]

Closing via #4938