Sourcekit-lsp go-to-def not working for modules built from diff toolchain version

[jkl0619]

Swift version

swift 5

Platform

Linux 5.19

Editor

Visual studio with swift-tools-version 5.9

Does the issue reproduce with Swift 6?

I didn’t try

Description

Creating this issue as I was asked here

Hello, this is using VSCode + sourcekit-lsp

I have a swift library that is built using toolchain sdk version of 13.4.

In this code, I have import modules, import A and import B

The code builds, and I am also able to find the go-to-defs for items in A. However, I am unable to find anything for items in B when using go-to-defs. I've took a look at the LSP logging, and can verify that both output directory of the lib modules are included in the key.compilerargs with the -XCC flag

B uses sdk version of 15.2, A uses version 13.0

Is there something with the sdk versions that causes sourcekit-lsp to be able to not link the symbols correctly? Is this a known issue? Or am I totally on the wrong track here and the issue is elsewhere?

Thanks in advance.

Steps to Reproduce

Unfortunately I don't really have a simple way to reproduce this, as I can't share this project.

Logging

Unfortunately contains internal information, but if you'd like specific information I can do my best to send it

[ahoppen]

Synced to Apple’s issue tracker as rdar://133965202

[ahoppen]

Is this a SwiftPM project, a project with compile_commands.json or a build server?

Did you build your project when you’re seeing this issue? SourceKit bundled with Swift 5.9 doesn’t support background indexing and we need a build to update the index.

B uses sdk version of 15.2, A uses version 13.0

Are they just built with different SDKs or also different toolchains / Swift compilers?

[jkl0619]

Is this a SwiftPM project, a project with compile_commands.json or a build server?

build server

Did you build your project when you’re seeing this issue? SourceKit bundled with Swift 5.9 doesn’t support background indexing and we need a build to update the index.

Yes, I am able to build it. What's weird is that for the module where the go-to-defs don't work, clicking on the import B go-to-def will take me to a .swiftinterface in /tmp/sourcekit-lsp/GeneratedInterfaces/. However, clicking on the actual symbols from B will lead to no-definition-found

Are they just built with different SDKs or also different toolchains / Swift compilers?

Different SDK Versions as far as I am aware. They all use the same build server.

Thanks for looking into

[ahoppen]

I assume that we are missing index data, which causes cross-module jump-to-definition not to work. Jumping to definition on the import itself uses a slightly different code path because we don’t need to figure out in which module a type was defined.

Could you ensure that you:

• Set -index-store-path in the compiler invocations that build your project
• Make sure that we have the index store and index database path by either
  • Return the path to the index store as indexStorePath from your build server and returning indexDatabasePath to any directory in which SourceKit-LSP can build indexstore-db (

    sourcekit-lsp/Sources/BuildSystemIntegration/BuildServerBuildSystem.swift

    Lines 175 to 181 in a3bb2d7

    	// see if index store was set as part of the server metadata
    	if let indexDbPath = readReponseDataKey(data: response.data, key: "indexDatabasePath") {
    	self.indexDatabasePath = try AbsolutePath(validating: indexDbPath, relativeTo: self.projectRoot)
    	}
    	if let indexStorePath = readReponseDataKey(data: response.data, key: "indexStorePath") {
    	self.indexStorePath = try AbsolutePath(validating: indexStorePath, relativeTo: self.projectRoot)
    	}

    )
  • Alternatively, you can set these options as -index-store-path and -index-db-path when launching sourcekit-lsp.

[jkl0619]

Hey thanks for your response

Set -index-store-path in the compiler invocations that build your project

I'm not sure if I understand this correctly. Should I be overriding it to my current projects path?
or should it be something like
-index-store-path ${SOURCEKIT_LSP_CMAKE_BINARY_DIR}/index?

[ahoppen]

It’s hard to tell without knowing what your build server does. Do you have an example of build settings that are provided by your build server?

[jkl0619]

For the build setting, are you referring to the project, or the sourcekit-lsp?

Just as a general info:
So I dug a little bit deeper into the build etc

We use the buck build system

The sourcekit-lsp is built via MAKE that has a specific .mk file to specify configuration, looks like its built using swift bunch of commands and configurations

It does not have an index-store-path in the set of configurations from what I can tell, but I assume it just uses some sort of default set path if thats the case.

The other modules (aka the imports / project) are built using buck (which underlying I believe uses swift compiler, uses the apple_library macro)

The reason why I was under the impression that the index store is included is because I have
the following from the sourcekit logging when opening the problem project file

sourcekit: [2:handleRequest-before:49.4170] {
  key.request: source.request.editor.open,
  key.name: "/some/directory/problemProject.swift",
  key.enablesyntaxmap: 0,
  key.enablesubstructure: 0,
  key.enablediagnostics: 0,
  key.syntactic_only: 1,
  key.compilerargs: [
  ...

`"-Xcc", "buck-out/.swift-vscode/blah/blah-blah/B/swift-extended_symlink_tree"`,
`"-Xcc", "buck-out/.swift-vscode/blah/blah-blah/A/exported_symlink_tree"`,
...

With that being said, I examined both directories and they both had their respective A-Swift.h and B-Swift.h files in those places. Below are few other settings that may be(?) relevant, so including it here. They are still part of the key.compilerargs

    "-Xfrontend",
    "-emit-clang-header-nonmodular-includes",
    "-Xfrontend",
    "-disable-cxx-interop-requirement-at-import",
    "-Xfrontend",
    "-enable-upcoming-feature",
    "-Xfrontend",
    "ImportObjcForwardDeclarations",
    "-Xfrontend",
    "-explicit-swift-module-map-file",
    "-Xfrontend",
    "buck-out/.swift-vscode/projectFileName.swift_module_map.json",
    ```

[ahoppen]

For the build setting, are you referring to the project, or the sourcekit-lsp?

I am referring to your project’s build settings.

It does not have an index-store-path in the set of configurations from what I can tell, but I assume it just uses some sort of default set path if thats the case.

You need to pass a -index-store-path so that we can build an index of your project during a build, which can then be used by SourceKit-LSP to provide cross-module functionality. Opening the file in sourcekitd doesn’t use the index and provides mostly functionality that’s contained within a single module. sourcekitd needs to co-operate with the index to provide cross-module functionality.

[jkl0619]

I will look into being able to do that and report back when I can.

That does bring me another question however.

Clicking on a symbol from B will still take me to the generated interface file of .swiftinterface (Even without generating it first via clicking on import B go to def)

Are there any guesses as to why symbols from B will go to generated-interface, but say no definition found for A symbols, even if go-to-def for that import will take me to a .swiftinterface file as well? I'm trying to follow through in the sourcekit-lsp code but can't seem to understand why such case might happen

[ahoppen]

I haven’t fully understood that part yet. It is a little hard to reason about without a reproducing example project. But I’m pretty sure that the lack of index data is the root cause here.

[jkl0619]

Looks like buck model and index store is very incompatible and they are currently working on coming up with a solution for it, so I guess at the time being the cross-module jump functionality is kind of thrown out the window. I do apologize that I'm unable to just provide everything (e.g. reproducible example project) so that you can also easily see whats going on.

I'd still like to discover why the go-to-def works for some symbols but not others to at least direct uesrs to .swiftinterface files. I see #747 seems to have added the functionality for it, but I guess it must be failing for some while succeeding for others. I guess I'll have to keep digging, but yeah, my initial thought was something with the toolchain version since they are both in the key.compilerargs: ....