Skip to content

Conversation

@jiashengguo
Copy link
Member

  • telemetry tracking for VSCode extension

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 26, 2025

📝 Walkthrough

Walkthrough

Adds a VS Code telemetry module, instruments extension activation and sign-in flows, updates ZModel preview to cache and save generated markdown with a fixed temp filename and context-key toggling, and adds a build step to copy preview resources and replace a telemetry token.

Changes

Cohort / File(s) Summary of Changes
Telemetry core module
packages/schema/src/vscode/vscode-telemetry.ts
New Mixpanel-backed telemetry helper. Adds TelemetryEvents type, VSCodeTelemetry class, device-id derivation, environment context, track and identify methods, and default singleton export.
Extension lifecycle & auth instrumentation
packages/schema/src/extension.ts
Switches imports to ./vscode/*; emits telemetry events for activation and sign-in lifecycle (extension:activate, `extension:signin:show
Auth provider changes
packages/schema/src/vscode/zenstack-auth-provider.ts
Imports telemetry and calls telemetry.identify(claims.email!) after decoding JWT; removes generateState() method.
ZModel preview feature & telemetry
packages/schema/src/vscode/zmodel-preview.ts, packages/schema/src/vscode/res/zmodel-preview-release-notes.html
Adds telemetry for preview/save, toggles zenstack.isMarkdownPreview on tab change, caches lastGeneratedMarkdown, uses fixed preview temp filename, changes openMarkdownPreview to accept content only, adds saveZModelDocumentation command, and updates release notes wording.
Build pipeline: bundle post-processing
packages/schema/build/bundle.js
Adds replaceTelemetryTokenInBundle that reads VSCODE_TELEMETRY_TRACKING_TOKEN, replaces placeholder in built bundle, copies src/res and src/vscode/res into bundle/res, enforces token presence (exit code 1), and adds top-level error handling.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant VSCode as VS Code
  participant Ext as Extension
  participant Tele as Telemetry
  participant Auth as Auth Provider
  participant IdP as Identity Provider

  VSCode->>Ext: activate()
  Ext->>Tele: track("extension:activate")

  Ext->>VSCode: requireAuth() prompts
  Ext->>Tele: track("extension:signin:show")
  VSCode->>Ext: user clicks "Sign in"
  Ext->>Tele: track("extension:signin:start")
  Ext->>Auth: start auth flow
  Auth->>IdP: open auth URL
  IdP-->>Auth: callback with token
  Auth->>Auth: decode claims (email)
  Auth->>Tele: identify(email)
  Auth-->>Ext: success
  Ext->>Tele: track("extension:signin:complete")
  Note over Ext,Tele: On error: track("extension:signin:error", payload)
Loading
sequenceDiagram
  autonumber
  participant User
  participant ZPrev as ZModelPreview
  participant Tele as Telemetry
  participant FS as File System
  participant VSCode as VS Code UI

  User->>ZPrev: Preview ZModel
  ZPrev->>Tele: track("extension:zmodel-preview")
  ZPrev->>ZPrev: generate markdown & cache lastGeneratedMarkdown
  ZPrev->>FS: write temp preview.md (fixed name)
  ZPrev->>VSCode: open markdown preview

  Note over VSCode,ZPrev: Tab change updates context key

  User->>ZPrev: Save documentation (command)
  ZPrev->>ZPrev: ensure lastGeneratedMarkdown exists
  ZPrev->>VSCode: show save dialog (.md)
  VSCode-->>ZPrev: selected path
  ZPrev->>FS: write saved file
  ZPrev->>VSCode: open & close saved doc (refresh preview)
  ZPrev->>Tele: track("extension:zmodel-save")
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: preview zmodel doc #2239 — Touches the same auth flow and ZModel preview areas; likely overlaps with or precedes these telemetry and preview changes.

Suggested reviewers

  • ymc9

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Title Check ✅ Passed The title “feat: support save markdown preview file” describes the newly introduced ability to save the generated markdown preview, which is indeed part of the changeset, but it omits the more substantial telemetry instrumentation enhancements that form the core of this pull request.
Description Check ✅ Passed The description “telemetry tracking for VSCode extension” accurately reflects the major telemetry instrumentation work added throughout the extension and is therefore on-topic for this pull request.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jiasheng-markdown

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9ee7e62 and 08d048e.

📒 Files selected for processing (2)
  • packages/schema/build/bundle.js (2 hunks)
  • packages/schema/src/vscode/vscode-telemetry.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/schema/build/bundle.js
🧰 Additional context used
🧬 Code graph analysis (1)
packages/schema/src/vscode/vscode-telemetry.ts (3)
packages/schema/bin/post-install.js (3)
  • Mixpanel (6-6)
  • os (8-8)
  • payload (16-28)
packages/schema/src/utils/machine-id-utils.ts (1)
  • getMachineId (63-74)
packages/schema/src/global.d.ts (1)
  • TELEMETRY_TRACKING_TOKEN (2-2)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: build-test (20.x)
  • GitHub Check: dependency-review
  • GitHub Check: build-test (20.x)
  • GitHub Check: OSSAR-Scan
  • GitHub Check: build-test (20.x)
🔇 Additional comments (1)
packages/schema/src/vscode/vscode-telemetry.ts (1)

65-71: Restore distinct_id on $identify payload.

The Mixpanel Node SDK drops $identify events that don’t ship a non-empty distinct_id, logging Invalid track call - distinct_id must be a non-empty string. That means we never merge the anonymous device with the signed-in account, so identity stitching fails. Please add distinct_id (same value as $identified_id) before we ship this.

         if (this.mixpanel) {
             this.mixpanel.track('$identify', {
+                distinct_id: userId,
                 $identified_id: userId,
                 $anon_id: this.deviceId,
                 token: TELEMETRY_TRACKING_TOKEN,
             });
         }

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (4)
packages/schema/src/vscode-telemetry.ts (1)

29-35: Telemetry enablement fallback (optional)

env.isTelemetryEnabled may not exist in older VS Code. Optionally, fall back to telemetry.telemetryLevel !== 'off'.

- if (vscode.env.isTelemetryEnabled) {
+ const isEnabled =
+   (vscode.env as any).isTelemetryEnabled ??
+   vscode.workspace.getConfiguration('telemetry').get('telemetryLevel') !== 'off';
+ if (isEnabled) {
packages/schema/src/extension.ts (1)

23-35: Sanitize error telemetry payloads

Sending raw error messages can include URLs, tokens, or PII. Prefer minimal fields (name/code) and truncate length.

- telemetry.track('extension:signin:error', { error: e instanceof Error ? e.message : String(e) });
+ const err = e instanceof Error ? { name: e.name, message: String(e.message).slice(0, 200) } : { message: String(e).slice(0, 200) };
+ telemetry.track('extension:signin:error', err);
packages/schema/src/zmodel-preview.ts (2)

290-334: Refine Save UX and robustness

  • Opening then immediately closing the saved file is jarring and doesn’t refresh the preview (which points to the temp file).
  • Consider notifying the user and offering to open the saved file.
-            // Open and close the saved file to refresh the shown markdown preview
-            await vscode.commands.executeCommand('vscode.open', saveUri);
-            await vscode.commands.executeCommand('workbench.action.closeActiveEditor');
+            // Notify user and optionally open the saved file
+            const action = await vscode.window.showInformationMessage(
+                `Documentation saved: ${saveUri.fsPath}`,
+                'Open File'
+            );
+            if (action === 'Open File') {
+                await vscode.commands.executeCommand('vscode.open', saveUri);
+            }

268-271: Optional: include source filename in default save name

If feasible, derive defaultFilePath from the active ZModel (e.g., -doc.md). You’d need to persist the originating document name when generating.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 80ebb55 and afdb47d.

⛔ Files ignored due to path filters (1)
  • packages/schema/package.json is excluded by !**/*.json
📒 Files selected for processing (5)
  • packages/schema/src/extension.ts (3 hunks)
  • packages/schema/src/res/zmodel-preview-release-notes.html (1 hunks)
  • packages/schema/src/vscode-telemetry.ts (1 hunks)
  • packages/schema/src/zenstack-auth-provider.ts (2 hunks)
  • packages/schema/src/zmodel-preview.ts (7 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
packages/schema/src/vscode-telemetry.ts (3)
packages/schema/bin/post-install.js (3)
  • Mixpanel (6-6)
  • os (8-8)
  • payload (16-28)
packages/schema/src/global.d.ts (1)
  • TELEMETRY_TRACKING_TOKEN (2-2)
packages/schema/src/utils/machine-id-utils.ts (1)
  • getMachineId (63-74)
packages/schema/src/extension.ts (1)
packages/schema/src/zenstack-auth-provider.ts (1)
  • AUTH_PROVIDER_ID (11-11)
packages/schema/src/zmodel-preview.ts (1)
packages/schema/src/documentation-cache.ts (1)
  • DocumentationCache (15-152)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)
🔇 Additional comments (6)
packages/schema/src/zenstack-auth-provider.ts (1)

2-2: LGTM: central telemetry import

Importing the shared telemetry singleton here aligns usage across the extension.

packages/schema/src/extension.ts (1)

45-45: LGTM: activation telemetry

Activation event is tracked once on startup and gated by telemetry settings.

packages/schema/src/zmodel-preview.ts (4)

18-21: Preview file naming approach is sound

Zero-width space avoids collisions; consistent with the tab detection logic.


96-96: LGTM: preview event telemetry

Event is emitted at start of preview generation.


128-133: Good: cache last generated content for Save

This enables Save without re-calling the API.


268-284: Temp preview creation is fine

Using os.tmpdir and markdown.showPreviewToSide is appropriate.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
packages/schema/src/zmodel-preview.ts (1)

301-309: Offer contextual default Save name (optional improvement).

Right now every save dialog defaults to zmodel-doc.md, which makes it easy to overwrite the wrong file when you’re documenting several models. We already know the source .zmodel that produced lastGeneratedMarkdown, so we could capture its stem during preview and reuse it here to prefill something like <model>-documentation.md.

-        let defaultFilePath = `zmodel-doc.md`;
+        const defaultFileName = `${this.lastPreviewedDocumentName ?? 'zmodel-doc'}.md`;
+        let defaultFilePath = defaultFileName;
@@
-            defaultFilePath = path.join(workspacePath, defaultFilePath);
+            defaultFilePath = path.join(workspacePath, defaultFileName);
// near other fields
private lastPreviewedDocumentName: string | null = null;

// in previewZModelFile() when generation succeeds
this.lastPreviewedDocumentName = path.parse(document.fileName).name;
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between afdb47d and dc4aee9.

⛔ Files ignored due to path filters (1)
  • packages/schema/package.json is excluded by !**/*.json
📒 Files selected for processing (2)
  • packages/schema/src/vscode-telemetry.ts (1 hunks)
  • packages/schema/src/zmodel-preview.ts (7 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/schema/src/vscode-telemetry.ts
🧰 Additional context used
🧬 Code graph analysis (1)
packages/schema/src/zmodel-preview.ts (1)
packages/schema/src/documentation-cache.ts (1)
  • DocumentationCache (15-152)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: dependency-review
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)
  • GitHub Check: OSSAR-Scan

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dc4aee9 and 1cbc5c1.

⛔ Files ignored due to path filters (1)
  • packages/schema/tsconfig.json is excluded by !**/*.json
📒 Files selected for processing (6)
  • packages/schema/build/bundle.js (1 hunks)
  • packages/schema/src/extension.ts (3 hunks)
  • packages/schema/src/vscode/res/zmodel-preview-release-notes.html (1 hunks)
  • packages/schema/src/vscode/vscode-telemetry.ts (1 hunks)
  • packages/schema/src/vscode/zenstack-auth-provider.ts (2 hunks)
  • packages/schema/src/vscode/zmodel-preview.ts (7 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
packages/schema/src/vscode/zmodel-preview.ts (1)
packages/schema/src/vscode/documentation-cache.ts (1)
  • DocumentationCache (15-152)
packages/schema/src/extension.ts (1)
packages/schema/src/vscode/zenstack-auth-provider.ts (1)
  • AUTH_PROVIDER_ID (11-11)
packages/schema/src/vscode/vscode-telemetry.ts (3)
packages/schema/bin/post-install.js (3)
  • Mixpanel (6-6)
  • os (8-8)
  • payload (16-28)
packages/schema/src/global.d.ts (1)
  • TELEMETRY_TRACKING_TOKEN (2-2)
packages/schema/src/utils/machine-id-utils.ts (1)
  • getMachineId (63-74)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: build-test (20.x)
  • GitHub Check: OSSAR-Scan
  • GitHub Check: dependency-review
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1cbc5c1 and abc8efb.

📒 Files selected for processing (1)
  • packages/schema/build/bundle.js (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)
  • GitHub Check: dependency-review
  • GitHub Check: OSSAR-Scan
  • GitHub Check: build-test (20.x)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between abc8efb and 9ee7e62.

📒 Files selected for processing (1)
  • packages/schema/build/bundle.js (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: dependency-review
  • GitHub Check: build-test (20.x)
  • GitHub Check: OSSAR-Scan
  • GitHub Check: build-test (20.x)
  • GitHub Check: build-test (20.x)

@ymc9 ymc9 merged commit ada3469 into dev Sep 28, 2025
12 checks passed
@ymc9 ymc9 deleted the jiasheng-markdown branch September 28, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants