Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: lineage refactor #1212

Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
4bd8d9c
feat: lineage v2
saravmajestic Jun 13, 2024
b0d6aaa
fix: lineage interactions
saravmajestic Jun 13, 2024
fb048de
chore: help button
saravmajestic Jun 13, 2024
1da70b0
fix: action widget buttons
saravmajestic Jun 14, 2024
0bc6223
revert: unnecessary changes
saravmajestic Jun 14, 2024
b6ba6f1
chore: updated components
saravmajestic Jun 14, 2024
bee9382
fix: styles
saravmajestic Jun 14, 2024
d1fcb2d
Merge branch 'master' into feat/lineage-v2
saravmajestic Jun 14, 2024
9382b8f
fix: views code
saravmajestic Jun 14, 2024
da68973
Merge branch 'master' into feat/lineage-v2
saravmajestic Jun 17, 2024
e0c3649
fix: updated components
saravmajestic Jun 17, 2024
b4bbbc0
chore: cleanup
saravmajestic Jun 17, 2024
9fe65ea
chore: cleanup
saravmajestic Jun 17, 2024
13dcce1
Merge branch 'master' into feat/lineage-v2
saravmajestic Jul 8, 2024
fbec974
fix: static lineage integration
saravmajestic Jul 8, 2024
b3d8b2f
Merge branch 'master' into feat/lineage-v2
saravmajestic Jul 9, 2024
d7e3061
fix: merge issues
saravmajestic Jul 9, 2024
c8c38ad
fix: sql visualizer ui issues
saravmajestic Jul 9, 2024
2f82023
Merge branch 'feat/lineage-v2' of https://github.com/AltimateAI/vscod…
saravmajestic Jul 9, 2024
d82bd80
fix: update lib version
saravmajestic Jul 9, 2024
17e599b
fix: image cleanup
saravmajestic Jul 10, 2024
5143e9a
revert: lib changes
saravmajestic Jul 17, 2024
2df32ef
Merge branch 'master' into feat/lineage-v2
saravmajestic Jul 17, 2024
1aa310c
fix: lineage components (#1311)
AdiGajbhiye Jul 18, 2024
f9910a4
fix: update components
saravmajestic Jul 18, 2024
96be3e5
fix: minor issues
saravmajestic Jul 18, 2024
1ef1ba4
fix: expand popover ui
saravmajestic Jul 18, 2024
3eff7ba
fix: lineae demo
saravmajestic Jul 19, 2024
b5fde33
fix: lineage sidebar updates
saravmajestic Jul 19, 2024
6f6c6b6
fix: feedback btn for sql lineage
saravmajestic Jul 19, 2024
e2d7702
fix: code block type error
AdiGajbhiye Jul 19, 2024
c3a4989
fix: sql lineage render
AdiGajbhiye Jul 23, 2024
72c638b
fix: open url
AdiGajbhiye Jul 23, 2024
02e407e
feat: added final builds
AdiGajbhiye Jul 23, 2024
edf489a
feat: added final builds
AdiGajbhiye Jul 23, 2024
6284060
Merge branch 'master' into feat/lineage-v2
anandgupta42 Jul 24, 2024
c5fb15e
Merge branch 'master' into feat/lineage-v2
saravmajestic Jul 25, 2024
7f268e2
fix: default expansion
AdiGajbhiye Jul 31, 2024
8aa3e75
fix: reset
AdiGajbhiye Aug 5, 2024
dff48f1
Merge branch 'master' into feat/lineage-v2
AdiGajbhiye Aug 6, 2024
16232b1
Merge branch 'master' into feat/lineage-v2
saravmajestic Aug 13, 2024
f685076
Merge branch 'master' into feat/lineage-v2
saravmajestic Sep 23, 2024
67fd4f8
fix: ts error and update lib
saravmajestic Sep 24, 2024
ce8762d
fix: OOM
saravmajestic Sep 24, 2024
d27a9b8
Merge branch 'master' into feat/lineage-v2
AdiGajbhiye Oct 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -1117,7 +1117,7 @@
"scripts": {
"prepare": "husky install",
"vscode:prepublish": "npm run panel:webviews && npm run panel:lineage && webpack --mode production",
"webpack": "npm run panel:webviews && npm run panel:lineage && webpack --mode development",
"webpack": "npm run panel:webviews && webpack --mode development",
"webpack-dev": "npm run panel:webviews && npm run panel:lineage && webpack --mode development --watch",
"watch": "concurrently \"npm:watch --prefix ./new_lineage_panel\" \"npm:watch --prefix ./webview_panels\" \"webpack --mode development --watch\"",
"test-compile": "tsc -p ./",
Expand Down
13 changes: 13 additions & 0 deletions src/webview_provider/altimateWebviewProvider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -473,6 +473,17 @@ export class AltimateWebviewProvider implements WebviewViewProvider {
),
),
);
const LineageGif = webview.asWebviewUri(
Uri.file(
path.join(
extensionUri.fsPath,
"webview_panels",
"dist",
"assets",
"lineage.gif",
),
),
);
const codiconsUri = webview.asWebviewUri(
Uri.joinPath(
extensionUri,
Expand Down Expand Up @@ -505,9 +516,11 @@ export class AltimateWebviewProvider implements WebviewViewProvider {
<body>
<div id="root"></div>
<div id="sidebar"></div>
<div id="modal"></div>
<script nonce="${nonce}" >
window.viewPath = "${this.viewPath}";
var spinnerUrl = "${SpinnerUrl}"
var lineageGif = "${LineageGif}"
</script>

<script nonce="${nonce}" type="module" src="${indexJs}"></script>
Expand Down
108 changes: 70 additions & 38 deletions src/webview_provider/newLineagePanel.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
/* eslint-disable prettier/prettier */
import { readFileSync, writeFileSync } from "fs";
import * as path from "path";
import {
Expand All @@ -9,12 +10,12 @@ import {
TextEditor,
Uri,
Webview,
WebviewOptions,
WebviewView,
WebviewViewResolveContext,
window,
workspace,
env,
WebviewOptions,
} from "vscode";
import { AltimateRequest, ModelNode } from "../altimate";
import {
Expand All @@ -33,6 +34,10 @@ import { TelemetryService } from "../telemetry";
import { PythonException } from "python-bridge";
import { AbortError } from "node-fetch";
import { DBTTerminal } from "../dbt_client/dbtTerminal";
import { AltimateWebviewProvider } from "./altimateWebviewProvider";
import { QueryManifestService } from "../services/queryManifestService";
import { SharedStateService } from "../services/sharedStateService";
import { UsersService } from "../services/usersService";

type Table = {
label: string;
Expand Down Expand Up @@ -70,19 +75,52 @@ class DerivedCancellationTokenSource extends CancellationTokenSource {
}

@provideSingleton(NewLineagePanel)
export class NewLineagePanel implements LineagePanelView {
private _panel: WebviewView | undefined;
private eventMap: Map<string, ManifestCacheProjectAddedEvent> = new Map();
export class NewLineagePanel
extends AltimateWebviewProvider
implements LineagePanelView
{
protected viewPath = "/lineage";
protected panelDescription = "Lineage panel";
protected _panel: WebviewView | undefined;
protected eventMap: Map<string, ManifestCacheProjectAddedEvent> = new Map();
// since lineage can be cancelled from 2 places: progress bar and panel actions
private cancellationTokenSource: DerivedCancellationTokenSource | undefined;
private cllProgressResolve: () => void = () => {};

public constructor(
private dbtProjectContainer: DBTProjectContainer,
protected dbtProjectContainer: DBTProjectContainer,
private altimate: AltimateRequest,
private telemetry: TelemetryService,
protected telemetry: TelemetryService,
private terminal: DBTTerminal,
) {}
private eventEmitterService: SharedStateService,
protected queryManifestService: QueryManifestService,
protected usersService: UsersService,
) {
super(
dbtProjectContainer,
altimate,
telemetry,
eventEmitterService,
terminal,
queryManifestService,
usersService,
);

this._disposables.push(
workspace.onDidChangeConfiguration(
(e) => {
if (!e.affectsConfiguration("dbt.enableLineagePanelV2")) {
return;
}
if (this._panel) {
this.renderWebviewView(this._panel.webview);
}
},
this,
this._disposables,
),
);
}

public changedActiveTextEditor(event: TextEditor | undefined) {
if (event === undefined) {
Expand Down Expand Up @@ -130,23 +168,9 @@ export class NewLineagePanel implements LineagePanelView {
});
}

resolveWebviewView(
panel: WebviewView,
context: WebviewViewResolveContext<unknown>,
_token: CancellationToken,
): void | Thenable<void> {
this.terminal.debug(
"newLineagePanel:resolveWebviewView",
"onResolveWebviewView",
);
this._panel = panel;
this.setupWebviewOptions(context);
this.renderWebviewView(context);
}

async handleCommand(message: { command: string; args: any }): Promise<void> {
const { command, args } = message;
const { id, params } = args;
async handleCommand(message: { command: string; args: any, syncRequestId?: string }): Promise<void> {
const { command, args = {}, syncRequestId } = message;
const { id = syncRequestId, params } = args;

if (command === "openProblemsTab") {
commands.executeCommand("workbench.action.problems.focus");
Expand All @@ -156,7 +180,7 @@ export class NewLineagePanel implements LineagePanelView {
const body = await this.getUpstreamTables(params);
this._panel?.webview.postMessage({
command: "response",
args: { id, body, status: true },
args: { id, syncRequestId, body, status: true },
});
return;
}
Expand All @@ -165,7 +189,7 @@ export class NewLineagePanel implements LineagePanelView {
const body = await this.getDownstreamTables(params);
this._panel?.webview.postMessage({
command: "response",
args: { id, body, status: true },
args: { id, syncRequestId, body, status: true },
});
return;
}
Expand All @@ -174,7 +198,7 @@ export class NewLineagePanel implements LineagePanelView {
const body = await this.getColumns(params);
this._panel?.webview.postMessage({
command: "response",
args: { id, body, status: true },
args: { id, syncRequestId, body, status: true },
});
return;
}
Expand All @@ -183,7 +207,7 @@ export class NewLineagePanel implements LineagePanelView {
const body = await this.getExposureDetails(params);
this._panel?.webview.postMessage({
command: "response",
args: { id, body, status: true },
args: { id, syncRequestId, body, status: true },
});
return;
}
Expand All @@ -192,7 +216,7 @@ export class NewLineagePanel implements LineagePanelView {
const body = await this.getConnectedColumns(params);
this._panel?.webview.postMessage({
command: "response",
args: { id, body, status: !!body },
args: { id, syncRequestId, body, status: !!body },
});
return;
}
Expand All @@ -207,12 +231,12 @@ export class NewLineagePanel implements LineagePanelView {
});
this._panel?.webview.postMessage({
command: "response",
args: { id, status: true },
args: { id, syncRequestId, status: true },
});
} catch (error) {
this._panel?.webview.postMessage({
command: "response",
args: { id, status: false },
args: { id, syncRequestId, status: false },
});
window.showErrorMessage(
extendErrorWithSupportLinks(
Expand Down Expand Up @@ -253,6 +277,7 @@ export class NewLineagePanel implements LineagePanelView {
command: "response",
args: {
id,
syncRequestId,
status: true,
body: {
showSelectEdges: config.get("showSelectEdges", true),
Expand All @@ -273,6 +298,7 @@ export class NewLineagePanel implements LineagePanelView {
command: "response",
args: {
id,
syncRequestId,
status: true,
body: { ok: true },
},
Expand All @@ -285,6 +311,7 @@ export class NewLineagePanel implements LineagePanelView {
"Unsupported command",
message,
);
super.handleCommand(message);
}

private async handleColumnLineage({ event }: { event: CllEvents }) {
Expand Down Expand Up @@ -927,14 +954,19 @@ export class NewLineagePanel implements LineagePanelView {
return { node, aiEnabled };
}

private setupWebviewOptions(context: WebviewViewResolveContext) {
this._panel!.description =
"Show table level and column level lineage SQL queries";
this._panel!.webview.options = <WebviewOptions>{ enableScripts: true };
}
protected renderWebviewView(webview: Webview) {
const enableLineagePanelV2 = workspace
.getConfiguration("dbt")
.get<boolean>("enableLineagePanelV2", false);

private renderWebviewView(context: WebviewViewResolveContext) {
const webview = this._panel!.webview!;
if (enableLineagePanelV2){
this._panel!.webview.html = super.getHtml(
webview,
this.dbtProjectContainer.extensionUri,
);
return;
}
this._panel!.webview.options = <WebviewOptions>{ enableScripts: true };
this._panel!.webview.html = getHtml(
webview,
this.dbtProjectContainer.extensionUri,
Expand Down
2 changes: 2 additions & 0 deletions webview_panels/src/AppConstants.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import DocumentationProvider from "@modules/documentationEditor/DocumentationPro
import DataPilotPanel from "@modules/dataPilot";
import DbtDocsView from "@modules/dbtDocs/DbtDocsView";
import QueryPanelProvider from "@modules/queryPanel/QueryPanelProvider";
import LineageView from "@modules/lineage/LineageView";

// TODO: lazy loading breaks loading dynamic webviews when having css because of vite dynamic loading
// research on how to fix that and then use lazy loading
Expand All @@ -25,4 +26,5 @@ export const AvailableRoutes = {
},
"/dbt-docs": { component: <DbtDocsView /> },
"/query-panel": { component: <QueryPanelProvider /> },
"/lineage": { component: <LineageView /> },
};
10 changes: 10 additions & 0 deletions webview_panels/src/assets/icons/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ export { default as EmptySquareIcon } from "./square.svg?react";
export { default as CheckedSquareIcon } from "./checked-square.svg?react";
export { default as TestsIcon } from "./tests.svg?react";
export { default as FolderIcon } from "./folder.svg?react";
export { default as PlayCircleIcon } from "./play-circle.svg?react";
import LoadingSpinnerUrl from "./spinner.gif";
import LineageGif from "./lineage.gif";
import "./styles.css";

interface Props {
Expand Down Expand Up @@ -124,6 +126,14 @@ export const LoadingSpinner = (): JSX.Element => (
/>
);

export const LineageDemo = (): JSX.Element => (
<img
// @ts-expect-error added in altimateWebViewProvider
src={(window.lineageGif as string) ?? LineageGif}
alt="Lineage demo"
/>
);
Comment on lines +133 to +139
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review the LineageDemo component implementation.

The new LineageDemo component is well-structured and follows the existing patterns in the file. However, there are a few points to consider:

  1. The fallback mechanism (window.lineageGif as string) ?? LineageGif is a good practice for flexibility.
  2. The alt text "Lineage demo" is appropriate and descriptive.

Potential issues:

  1. The TypeScript error suppression comment (@ts-expect-error) might hide underlying type issues. Consider addressing the root cause instead of suppressing the error.
  2. The use of window.lineageGif assumes it's defined somewhere else, which could lead to runtime errors if not properly managed. Ensure that this global variable is consistently set where needed.

Consider refactoring to avoid the need for the TypeScript error suppression:

export const LineageDemo = (): JSX.Element => {
  const gifSrc = typeof window !== 'undefined' && 'lineageGif' in window
    ? (window as any).lineageGif as string
    : LineageGif;

  return <img src={gifSrc} alt="Lineage demo" />;
};

This approach type-checks the window object and uses a type assertion only when necessary, reducing the risk of runtime errors.


export const LoadingIcon = (
props: HTMLAttributes<HTMLElement>,
): JSX.Element => <Icon icon="loading" className="rotate" {...props} />;
Binary file added webview_panels/src/assets/icons/lineage.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 12 additions & 0 deletions webview_panels/src/assets/icons/play-circle.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading