Language server #117
Language server #117
Conversation
Language Server
---------------
1. Identifies links and imports so the editor can mark
them as such. Ctrl+click on the links will attempt
to launch what the link points to.
2. 'Find All References' will show a list (as presented
by vscode) of all references of the given node.
3. 'Goto Definition' will do one of two things. If there
are multiple definitions, it will show a list similar
to 'Find All References'. If there is only one option
for the definition, then the cursor will jump to that
option.
4. 'Rename...' will rename a node. If there are multiple
nodes (a.style... / a.shape... / a.lable... / etc.),
then a rename will change all locations that contain
that node.
5. Auto Completion:
a) Typing a '@' will show a list of d2 files that are contained in the d2 documents current folder and all child folders from there.
b) Typing a '.' that is along a valid path, will bring up the proper list for that position in the path.
c) Typing a ':' will bring up a list of possible values for that position in the current path. If there is no list, in the case of a freeform field such as 'label', then no list is shown.
d) The 'Tab' key will commit the auto completion.
6. When a preview is not present, then there is no error
messages generated, the language server will show any
errors that are created during the AST phase of d2
compliation.
New items in the PR
To facilitate a better edit/build/debug process, the extenstion was moved to a 'client' directory, the language server to a 'server' directory. **_Note:_** This may break the nightly build process and will need to be ajusted.
1. Identifies links and imports so the editor can mark
them as such. Ctrl+click on the links will attempt
to launch what the link points to.
2. 'Find All References' will show a list (as presented
by vscode) of all references of the given node.
3. 'Goto Definition' will do one of two things. If there
are multiple definitions, it will show a list similar
to 'Find All References'. If there is only one option
for the definition, then the cursor will jump to that
option.
4. 'Rename...' will rename a node. If there are multiple
nodes (a.style... / a.shape... / a.label... / etc.),
then a rename will change all locations that contain
that node.
5. Auto Completion:
a) Typing a '@' will show a list of d2 files that are contained in the d2 documents current folder and all child folders from there.
b) Typing a '.' that is along a valid path, will bring up the proper list for that position in the path.
c) Typing a ':' will bring up a list of possible values for that position in the current path. If there is no list, in the case of a freeform field such as 'label', then no list is shown.
d) The 'Tab' key will commit the auto completion.
6. When a preview is not present, then there is no error messages generated, the language server will show any errors that are created during the AST phase of d2 compliation.
|
(apologies for the delay on this, team is super busy with some d2 studio stuff w/ deadlines, very looking forward to digging into this and trying it soon) |
There was a problem hiding this comment.
Note: we should add yarn installdep as a first step to the readme
ran into this issue when using an older d2 version, looks like it has errors trying to parse the svg response:
***************************
D2 Language Server Starting
Client: Visual Studio Code
Version: 1.85.0
PID: 11423
***************************
[Info - 8:09:44 PM] Language Server D2 path: d2
SyntaxError: Unexpected token '<', "<?xml vers"... is not valid JSON
at JSON.parse (<anonymous>)
at new AstReader (.../terrastruct.d2-0.8.7/server/dist/d2Ast.js:18:28)
at .../terrastruct.d2-0.8.7/server/dist/server.js:97:15
at CallbackList.invoke (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/events.js:55:39)
at Emitter.fire (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/events.js:117:36)
at .../terrastruct.d2-0.8.7/server/node_modules/vscode-languageserver/lib/common/textDocuments.js:122:38
at handleNotification (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/connection.js:640:25)
at handleMessage (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/connection.js:342:13)
at processMessageQueue (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/connection.js:362:17)
at Immediate._onImmediate (.../terrastruct.d2-0.8.7/server/node_modules/vscode-jsonrpc/lib/common/connection.js:334:13)
[Error - 8:09:47 PM] Request textDocument/documentLink failed.
Message: Request textDocument/documentLink failed with message: Cannot read properties of undefined (reading 'LinksAndImports')
Code: -32603
|
go to definition worked for objects at the outermost level but doesn't seem to work for nested objects 
|
|
@alixander and @gavin-ts Ping. |
There was a problem hiding this comment.
@BarryNolte I'm running into this error when I try to install. Any ideas?
Error at the end: ERROR Extension entrypoint(s) missing.
~/dev/d2-vscode LanguageServer ✔ 31d22h ⍉ 31d22h ⍉
▶ npm run dev
> d2@0.8.8 dev
> code --uninstall-extension terrastruct.d2; yarn gen && yarn pkg && code --install-extension d2.vsix
Extension 'terrastruct.d2' is not installed.
Make sure you use the full extension ID, including the publisher, e.g.: ms-dotnettools.csharp
yarn run v1.22.21
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ yq -o=json . syntaxes/d2.tmLanguage.yaml > syntaxes/d2.tmLanguage.json
✨ Done in 0.04s.
yarn run v1.22.21
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ vsce package --out d2.vsix
(node:99072) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
INFO Detected presence of yarn.lock. Using 'yarn' instead of 'npm' (to override this pass '--no-yarn' on the command line).
Executing prepublish script 'yarn run vscode:prepublish'...
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ yarn run package
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ yarn run package_client && yarn run package_server
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ webpack --config ./client/webpack.config.js --mode production --devtool hidden-source-map
[webpack-cli] Compiler starting...
[webpack-cli] Compiler is using config: '/Users/nca/dev/d2-vscode/client/webpack.config.js'
asset extension.js 362 KiB [compared for emit] [minimized] (name: main) 1 related asset
asset ../client/dist/utility.d.ts 1.17 KiB [compared for emit]
asset ../client/dist/extension.d.ts 880 bytes [compared for emit]
asset ../client/dist/docToPreviewGenerator.d.ts 814 bytes [compared for emit]
asset ../client/dist/browserWindow.d.ts 603 bytes [compared for emit]
asset ../client/dist/refreshTimer.d.ts 527 bytes [compared for emit]
asset ../client/dist/outputChannel.d.ts 453 bytes [compared for emit]
asset ../client/dist/taskRunner.d.ts 453 bytes [compared for emit]
asset ../client/dist/tasks.d.ts 430 bytes [compared for emit]
asset ../client/dist/themePicker.d.ts 411 bytes [compared for emit]
asset ../client/dist/layoutPicker.d.ts 251 bytes [compared for emit]
runtime modules 88 bytes 1 module
modules by path ./client/node_modules/ 890 KiB 129 modules
modules by path ./client/src/*.ts 33.7 KiB
./client/src/extension.ts 9.97 KiB [built] [code generated]
./client/src/docToPreviewGenerator.ts 3.55 KiB [built] [code generated]
./client/src/utility.ts 3.95 KiB [built] [code generated]
./client/src/outputChannel.ts 1.18 KiB [built] [code generated]
./client/src/layoutPicker.ts 1.6 KiB [built] [code generated]
+ 5 modules
+ 8 modules
WARNING in ./client/node_modules/vscode-languageserver-types/lib/umd/main.js 3:24-31
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
@ ./client/node_modules/vscode-languageserver-protocol/lib/common/api.js 23:13-51
@ ./client/node_modules/vscode-languageserver-protocol/lib/node/main.js 24:13-37
@ ./client/node_modules/vscode-languageclient/lib/common/api.js 22:13-54
@ ./client/node_modules/vscode-languageclient/lib/node/main.js 34:13-37
@ ./client/node_modules/vscode-languageclient/node.js 7:0-43
@ ./client/src/extension.ts 18:15-52
1 warning has detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.
webpack 5.88.2 compiled with 1 warning in 2940 ms
[webpack-cli] Compiler finished
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ webpack --config ./server/webpack.config.js --mode production --devtool hidden-source-map
[webpack-cli] Compiler starting...
[webpack-cli] Compiler is using config: '/Users/nca/dev/d2-vscode/server/webpack.config.js'
asset server.js 200 KiB [compared for emit] [minimized] (name: main) 1 related asset
asset ../server/dist/dataContainers.d.ts 4.21 KiB [compared for emit]
asset ../server/dist/d2Ast.d.ts 2.06 KiB [compared for emit]
asset ../server/dist/completionTree.d.ts 1.38 KiB [compared for emit]
asset ../server/dist/server.d.ts 763 bytes [compared for emit]
asset ../server/dist/completionHelpers.d.ts 649 bytes [compared for emit]
asset ../server/dist/utility.d.ts 311 bytes [compared for emit]
runtime modules 88 bytes 1 module
modules by path ./server/node_modules/ 469 KiB
cacheable modules 468 KiB 72 modules
./server/node_modules/vscode-languageserver-textdocument/lib/umd/ sync 160 bytes [built] [code generated]
./server/node_modules/vscode-languageserver-types/lib/umd/ sync 160 bytes [built] [code generated]
modules by path ./server/src/*.ts 46.2 KiB
./server/src/server.ts 8.19 KiB [built] [code generated]
./server/src/d2Ast.ts 5.42 KiB [built] [code generated]
./server/src/completionHelpers.ts 5.06 KiB [built] [code generated]
./server/src/dataContainers.ts 14.1 KiB [built] [code generated]
./server/src/utility.ts 1.82 KiB [built] [code generated]
./server/src/completionTree.ts 11.6 KiB [built] [code generated]
+ 10 modules
WARNING in ./server/node_modules/vscode-languageserver-textdocument/lib/umd/main.js 12:24-31
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
@ ./server/src/server.ts 10:45-90
WARNING in ./server/node_modules/vscode-languageserver-types/lib/umd/main.js 3:24-31
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
@ ./server/node_modules/vscode-languageserver-protocol/lib/common/api.js 23:13-51
@ ./server/node_modules/vscode-languageserver-protocol/lib/node/main.js 24:13-37
@ ./server/node_modules/vscode-languageserver/lib/common/api.js 25:13-55
@ ./server/node_modules/vscode-languageserver/lib/node/main.js 29:13-37
@ ./server/src/completionHelpers.ts 7:32-64
@ ./server/src/server.ts 13:28-58
2 warnings have detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.
webpack 5.88.2 compiled with 2 warnings in 2339 ms
[webpack-cli] Compiler finished
INFO Detected presence of yarn.lock. Using 'yarn' instead of 'npm' (to override this pass '--no-yarn' on the command line).
ERROR Extension entrypoint(s) missing. Make sure these files exist and aren't ignored by '.vscodeignore':
extension/client/dist/extension.js
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
I moved a ton of files around to separate the client from the server, so this could be leftovers from the previous directory structure. I'd delete the whole directory, re-clone the repository and build from there. As I moved things around, I ran into numerous issues just like this that were fixed by deep cleaning the node modules directories and deleting ./dist/... directories. |
There was a problem hiding this comment.
from a completely fresh clone to the branch, i still get the error.
can you try these commands as well?
/dev
▶ git clone -b LanguageServer git@github.com:BarryNolte/d2-vscode.git barry-vscode
Cloning into 'barry-vscode'...
remote: Enumerating objects: 1323, done.
remote: Counting objects: 100% (775/775), done.
remote: Compressing objects: 100% (292/292), done.
remote: Total 1323 (delta 540), reused 676 (delta 462), pack-reused 548
Receiving objects: 100% (1323/1323), 5.58 MiB | 4.00 MiB/s, done.
Resolving deltas: 100% (791/791), done.
~/dev
▶ cd barry-vscode
~/dev/barry-vscode LanguageServer ✔ 35d0h 35d0h
▶ yarn installdep
yarn run v1.22.21
warning d2@0.8.8: The engine "vscode" appears to be invalid.
$ yarn install && cd client && yarn install && cd ../server && yarn install && cd ..
[1/5] 🔍 Validating package.json...
warning d2@0.8.8: The engine "vscode" appears to be invalid.
[2/5] 🔍 Resolving packages...
[3/5] 🚚 Fetching packages...
[4/5] 🔗 Linking dependencies...
[5/5] 🔨 Building fresh packages...
[1/5] 🔍 Validating package.json...
warning d2-extension-client@0.8.7: The engine "vscode" appears to be invalid.
[2/5] 🔍 Resolving packages...
[3/5] 🚚 Fetching packages...
warning Pattern ["wrap-ansi@^7.0.0"] is trying to unpack in the same destination "/Users/nca/Library/Caches/Yarn/v6/npm-wrap-ansi-cjs-7.0.0-integrity/node_modules/wrap-ansi-cjs" as pattern ["wrap-ansi-cjs@npm:wrap-ansi@^7.0.0"]. This could result in non-deterministic behavior, skipping
.
warning vscode-languageclient@9.0.1: The engine "vscode" appears to be invalid.
[4/5] 🔗 Linking dependencies...
[5/5] 🔨 Building fresh packages...
[1/5] 🔍 Validating package.json...
[2/5] 🔍 Resolving packages...
[3/5] 🚚 Fetching packages...
[4/5] 🔗 Linking dependencies...
[5/5] 🔨 Building fresh packages...
✨ Done in 4.52s.
~/dev/barry-vscode LanguageServer ✔ 35d0h 35d0h
▶ npm run dev
...rest of output is same as above
|
it must be that it's building as |
There was a problem hiding this comment.
It was looking for .js files, then just weren't being generated. I updated package.json to fix that. Since I run "npm run watch", it continuously compiles and I didn't notice it not happening when I ran "npm run dev", since everything was already compiled. Should work now.
|
I checked out d2:master and it appears to be behind my d2:LSP-Fixes branch. If you can checkout upstream/master, it should have my changes and the error spew will go away. All new features are described in the PR Entry #117 |
There was a problem hiding this comment.
Was able to test it all finally, feels really nice, especially the autocomplete list and import/link goto's 🚀 .
The implementation is a little flaky though. You want to defer as much of the interpretation of D2 to D2. You're already deferring parsing, but I see you've implemented finding references. This is actually quite cumbersome with many edge cases, for example, D2's underscore _ mechanic that lets you reference parents and such. D2 actually has an an IR (intermediate representation). Though it's not specifically made for LSPs, it's designed with LSPs in mind from the start.
It sits in between the parser and compiler. (d2 file -> parser -> ir -> compile -> render)
https://github.com/terrastruct/d2/tree/master/d2ir
Here's an example output of what d2ir spits out for x.y: https://github.com/terrastruct/d2/blob/master/testdata/d2ir/TestCompile/fields/primary/nested.exp.json
As you can see, it's got the list of fields and edges, and for each of those, all of its references.
- It seems to treat URLs as files.
Kapture.2024-02-21.at.13.57.22.mp4
2,3,4. It works shallowly but it doesn't distinguish that a nested obj is different than one a level above.
5b. Is there any way for the list for "shape" to persist after a space is pressed after the colon?
Kapture.2024-02-21.at.14.01.40.mp4
|
When I started this, I was passing the IR to the server, but then I figured out how to synthesize what I needed from the AST. However, you found a limit that I should go back to using the IR for references and nested objects. I'll work on that. The autocomplete of ':' vs. ': ', I also thought was a good idea, and had it implemented. It didn't quite feel right, at least to me. So I tried a bunch of different languages, and every one of them auto completes on the token delimiter ('.', ':', etc). The vscode autocomplete system is also geared toward this implementation. But, since I know I want a space in there, when you pick an item to autocompletion, it will put the space in. |
|
Can we split out the smaller ones from this large PR and have this just be focused on the language server? @BarryNolte |
New items in the PR
Resolves #51
Must pull terrastruct/d2#1534 for this to work.
Project Layout
To facilitate a better edit/build/debug process, the extension was moved to a 'client' directory, the language server to a 'server' directory. Note: This may break the nightly build process and will need to be adjusted.
Language Server
Identifies links and imports so the editor can mark
them as such. Ctrl+click on the links will attempt
to launch what the link points to.
'Find All References' will show a list (as presented
by vscode) of all references of the given node.
'Goto Definition' will do one of two things. If there
are multiple definitions, it will show a list similar
to 'Find All References'. If there is only one option
for the definition, then the cursor will jump to that
option.
'Rename...' will rename a node. If there are multiple
nodes (a.style... / a.shape... / a.label... / etc.),
then a rename will change all locations that contain
that node.
Auto Completion:
a) Typing a '@' will show a list of d2 files that are contained in the d2 documents current folder and all child folders from there.
b) Typing a '.' that is along a valid path, will bring up the proper list for that position in the path.
c) Typing a ':' will bring up a list of possible values for that position in the current path. If there is no list, in the case of a free-form field such as 'label', then no list is shown.
d) The 'Tab' key will commit the auto completion.
When a preview is not present, then there is no error messages generated, the language server will show any errors that are created during the AST phase of d2 compilation.