[main][stats beat] implement stats beat in application insights #2489

siyuniu-ms · 2025-03-05T02:16:07Z

No description provided.

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

siyuniu-ms · 2025-03-05T02:20:46Z

Current StatsBeat Logic
Definition & Creation in Core: StatsBeat is defined and instantiated in the core without a channel, making it compatible with both AI and 1DS.
Initialization in Sender: The sender initializes StatsBeat and assigns the appropriate channel for sending telemetry.
Pros
✔ Prevents infinite loops: Eliminates the risk of recursive creation between StatsBeat and Sender.
✔ Unified implementation: Both AI and 1DS share the same StatsBeat class, reducing duplication.

Cons
❌ Always included in the codebase: Since StatsBeat is created in the core, it remains part of the code even if the user disables it.

StatsBeat Data Handling
StatsBeat data is stored within the payload.
When the final result is ready to be sent, the data is extracted before transmission.

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

channels/applicationinsights-channel-js/src/Sender.ts

shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IXHROverride.ts

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

channels/applicationinsights-channel-js/src/Sender.ts

lzchen · 2025-03-24T19:03:29Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+                    _networkCounter.totalRequest++;
+                    _networkCounter.requestDuration += utcNow() - payloadData["statsBeatData"]["startTime"];
+                }
+                let retryArray = [401, 403, 408, 429, 500, 503];


Missing 504 in retry. From https://github.com/aep-health-and-standards/Telemetry-Collection-Spec/blob/main/ApplicationInsights/sdk_behavior_breeze.md

I followed the status code from https://github.com/aep-health-and-standards/Telemetry-Collection-Spec/blob/main/ApplicationInsights/statsbeat/statsbeat.md, I could update the code there and specify its for js use.

https://github.com/aep-health-and-standards/Telemetry-Collection-Spec/pull/450

It's not specifically for js use, all languages should be doing so. Let's merge the spec change and then you can make the change here too.

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

lzchen · 2025-03-24T19:07:15Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+                } else if (throttleArray.includes(status)) {
+                    _networkCounter.throttle++;
+                } else if (status !== 307 && status !== 308) {
+                    _networkCounter.failure[status] = (_networkCounter.failure[status] || 0) + 1;


Missing logic for partial success 206?

in the new update, I am treating 206 neither a success or a failure (it would not be counted in any category except totalReuqeustCounter)

Interesting, Python is treating 206 as success, I guess we did not explicitly specify what this is. @JacksonWeber @TimothyMothra @jeanbisutti do we know how each language treats 206 in terms of success/failure?

@lzchen we treat 206 as retriable and persist the failed envelopes to disk.

I would expect a 206 to be treated as a success as it most likely means that breeze has either

applied server side sampling

As our data is going to be mixed in with theirs (the single transport) we may be getting throttled, so when we extend this to client side details, we would also want to treat this as success -- unless we get told which ones failed and we add code to "parse" out the dropped events

To be a little more simplistic, I would expect any 200 (200-299) response to be treated as a success, as a 204 response (no content) is a valid response that we get from the 1DS Collector.

Yeah, it looks like Node.js is out of spec treating these as retriable. I'll create a work item.

lzchen · 2025-03-24T19:10:29Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+
+
+
+            _self.countException = (endpoint: string, exceptionType: string) => {


Not super clear on this logic, should this be only incremented on exception?

lzchen · 2025-03-24T19:12:53Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+
+            function _setupTimer() {
+                if (!_timeoutHandle) {
+                    _timeoutHandle = scheduleTimeout(() => {


@MSNev

Confirm that scheduleTimeout returns ms?

yes, based on the comment in ts-utils
* @param timeout - The time, in milliseconds that the timer should wait before the specified

lzchen · 2025-03-24T19:23:46Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+                _cikey = ikey;
+                _language = STATSBEAT_LANGUAGE;
+                _os = STATSBEAT_TYPE;
+                _runTimeVersion = STATSBEAT_TYPE;


Is there a more useful value we can track for runtime version for web?

Might be able to parse the userAgent to get this information. This might be good default information though if none can be retrieved from that header.

As this component is in the core we don't really have access to the existing code that parses out the user-agent, we do a bunch of stuff in the 1DS code and some additional in the OsPlugin.

We could look at enhancing this in a later release -- probably as part of adding the OsPlugin by default (which we are planning in a future release) - not sure of the timing now though.

lzchen · 2025-03-24T19:25:24Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+                    "runtimeVersion": _runTimeVersion,
+                    "os": _os,
+                    "language": _language,
+                    "version": _sdkVersion,


For the other sdks, we do not populate this with sdkVersion but rather JUST the version of the SDK.

lzchen · 2025-03-24T19:32:18Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+                    iKey: INSTRUMENTATION_KEY,
+                    name: name,
+                    baseData: {
+                        name: name,


Is this the correct way to build the TelemetryItem? In Python we have type(TelemetryItem.baseData) = MetricsData and type(MetricsData.metrics) = List[MetricDataPoint].

yes, as we will do some envelop transform before we sent out telemetry, I will post some screenshot of how the telemetry will look like when go out of door here later

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

siyuniu-ms · 2025-03-25T22:48:08Z

here is the example of request_duration and request_success_count telemetry that send out

JacksonWeber

LGTM

MSNev · 2025-03-28T16:42:24Z

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts

+        let _os: string;
+        let _runTimeVersion: string;
+        dynamicProto(Statsbeat, this, (_self, _base) => {
+            _self.initialize = (core: IAppInsightsCore, ikey: string, endpoint: string, version?: string) => {


Lets update the signature to pass the values as a config object (iKey, endpoint, version) as I'm feeling that we might want to update this in the future.

We could also then make these dynamic, so we could "update" then for things like passing in the browser version etc. So basically follow the normal type of "onConfigChange()" with setting defaults -- the code "supports" defining this for internal types (not part of the global config object (IConfiguration)), but it may need some signature updates or we fudge it.

For now just create an interface with these 3 and pass them in (ie. don't treat as dynamic or use the onConfigChange)

MSNev · 2025-03-28T16:43:36Z

tools/chrome-debug-extension/manifest.json

@@ -2,8 +2,8 @@
    "name": "Telemetry Viewer - M3",
    "short_name": "Telemetry Viewer M3",
    "description": "A browser extension that provides a real time view of what's happening in Application Insights including what telemetry is being logged by the web application",
-    "version": "0.7.5",
-    "version_name": "0.7.5",
+    "version": "0.7.6",


Changes to this file really shouldn't be in this PR.

MSNev · 2025-03-28T16:51:47Z

channels/applicationinsights-channel-js/src/Sender.ts

@@ -283,6 +288,11 @@ export class Sender extends BaseTelemetryPlugin implements IChannelControls {
                        }
                    }

+                    if (!config.disableStatsBeat && _statsBeat && !_statsBeat.isInitialized()) {


You need to "set" a default otherwise disableStatsBeat will be undefined and not dynamic.

I'm also in 2 minds on this as we currently don't have any throttling in place, so once someone turns this one (by looking at the code or the tests), this will spam (overload) our statsbeat subscription with no option to stop it....

So we "might" want to check this code in with all of the "enable" commented out so there is zero chance of someone accidently enabling this. And then as part of the follow up PR to add the throttling we then add the config and support for "enabling" / "disabling".

Basically, we don't want someone turning this on a killing ALL of our statsbeat metrics. As the end-user (customers) subscription won't see any of this data.

basic structure

acc2e43

siyuniu-ms requested a review from a team as a code owner March 5, 2025 02:16

github-advanced-security bot found potential problems Mar 5, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

siyuniu-ms and others added 2 commits March 4, 2025 18:36

Merge branch 'main' into siyu/statsbeats

65be711

imp[ort

bfe5965

JacksonWeber reviewed Mar 5, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Outdated Show resolved Hide resolved

test locally works good

5182765

github-advanced-security bot found potential problems Mar 6, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

siyuniu-ms added 2 commits March 5, 2025 16:25

Update StatsBeat.ts

3f5ae6c

only check statsbeat before send it

0711bf7

siyuniu-ms requested review from lzchen and a team March 6, 2025 00:38

refactor

94a090e

github-advanced-security bot found potential problems Mar 11, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

siyuniu-ms added 2 commits March 17, 2025 15:28

remove statsbeatdata from function call

5cbac35

remove

9913e66

github-advanced-security bot found potential problems Mar 17, 2025

View reviewed changes

refactor statsbeat

31811e7

github-advanced-security bot found potential problems Mar 17, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

siyuniu-ms added 4 commits March 17, 2025 22:52

count in a different way

a6ba0fb

call oncomplete manually

0006c9b

Update Sender.ts

11b7e26

format change

ff8e0b6

github-advanced-security bot found potential problems Mar 18, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Fixed Show fixed Hide fixed

siyuniu-ms added 5 commits March 19, 2025 14:47

rename var and remove set initilize

2190c12

change class to interface

28f2696

set up timer

c81ea69

avoid duplicate init

eb2392b

rename exception

6dd7528

siyuniu-ms added 2 commits March 20, 2025 14:14

Update testVersionConflict.html

96d5089

add some comment

6131e9f

github-advanced-security bot found potential problems Mar 20, 2025

View reviewed changes

channels/applicationinsights-channel-js/src/Sender.ts Fixed Show fixed Hide fixed

channels/applicationinsights-channel-js/src/Sender.ts Fixed Show fixed Hide fixed

channels/applicationinsights-channel-js/src/Sender.ts Fixed Show fixed Hide fixed

lzchen reviewed Mar 24, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Outdated Show resolved Hide resolved

lzchen reviewed Mar 24, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/StatsBeat.ts Show resolved Hide resolved

siyuniu-ms added 4 commits March 24, 2025 15:40

add count check

b24c9fd

throttle count

ea2b96a

fix the ugilify not compile problem

41cb9d7

status code change

d475568

siyuniu-ms added 4 commits March 25, 2025 15:54

add basic test

fae3295

add counter test

c94a752

Update Sender.ts

ee5faa9

Merge branch 'main' into siyu/statsbeats

13ec43c

siyuniu-ms requested review from lzchen and JacksonWeber March 27, 2025 20:44

JacksonWeber approved these changes Mar 28, 2025

View reviewed changes

MSNev reviewed Mar 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[main][stats beat] implement stats beat in application insights #2489

[main][stats beat] implement stats beat in application insights #2489

siyuniu-ms commented Mar 5, 2025

siyuniu-ms commented Mar 5, 2025

lzchen Mar 24, 2025

siyuniu-ms Mar 24, 2025

siyuniu-ms Mar 25, 2025

lzchen Mar 26, 2025

lzchen Mar 24, 2025

siyuniu-ms Mar 25, 2025

lzchen Mar 26, 2025

JacksonWeber Mar 26, 2025

MSNev Mar 28, 2025

MSNev Mar 28, 2025

JacksonWeber Mar 28, 2025

lzchen Mar 24, 2025

lzchen Mar 24, 2025

siyuniu-ms Mar 25, 2025

lzchen Mar 24, 2025

JacksonWeber Mar 24, 2025

MSNev Mar 28, 2025 •

edited

Loading

lzchen Mar 24, 2025

lzchen Mar 24, 2025

siyuniu-ms Mar 25, 2025

siyuniu-ms commented Mar 25, 2025

JacksonWeber left a comment

MSNev Mar 28, 2025

MSNev Mar 28, 2025

MSNev Mar 28, 2025




		_self.countException = (endpoint: string, exceptionType: string) => {

[main][stats beat] implement stats beat in application insights #2489

Are you sure you want to change the base?

[main][stats beat] implement stats beat in application insights #2489

Conversation

siyuniu-ms commented Mar 5, 2025

siyuniu-ms commented Mar 5, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MSNev Mar 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siyuniu-ms commented Mar 25, 2025

JacksonWeber left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MSNev Mar 28, 2025 •

edited

Loading