Native Test Explorer pane unable to link with source code

[clipitar]

Checklist

• The issue is about this extension and NOT about a fork.
• Check the known issues list.
• The latest version of the extension was used.
• It is not related to remote-vscode or I checked the following issue

Describe the bug

Since 4.0.0, with the use of the native Test Explorer pane, I can no longer access my source code from the Test Explorer pane. It handles the relative path to my sources as an absolute path, and therefore can't find the source file.

I'm not sure if this is a bug or if there's some configuration missing on my part... but this used to work before 4.0.0.

Here's my config in settings.json:

"testMate.cpp.test.advancedExecutables": [
		{
			"pattern": "bin/Debug/**/*validate_*",
			"env": {
				"LD_LIBRARY_PATH": "${os_env:HOME}/Vault/ubuntu18_gcc9/poco/poco_1.9.3/lib"
			},
			"sourceFileMap": {
				"/home/user/git/std_c": "${os_env:HOME}/git/std_c"
			}
		}
	],

To Reproduce
Having a similar configuration to mine, as far as test executables' location is concerned...

Screenshots

Link to source not working:

Link to source working:

Desktop

• Extension Version: 4.0.6
• VS Code Version: 1.62.3
• Catch2 / Google Test / DOCTest Version: GoogleTest 1.8
• OS Type and Version: Linux x64 5.4.0-90-generic, Ubuntu 20.04
• Using remote-ssh/docker/wsl?: none of these, working locally on Ubuntu vm.

Regression bug?
It's kind of a regression, since it used to work with the old Test Explorer pane...

• Last extension version in which the feature were working: 3.6.33

Log (optional but recommended)
I have attached the output log when running a test.

log-4.0.6.txt

[matepek]

More logs/details pls.
Maybe try to explain the issue with other words. I'm not sure I fully got it.

[clipitar]

I apologize for not being clear...

Before 4.0.0:

When I run a unit test that fails, I get a hyperlink to the source code of the failing test in the Output window. Clicking on the hyperlink opens the source code in the editor window.

Below is the image of the working hyperlink:

After 4.0.0:

When I run a unit test that fails, nothing is shown in the output window, apart from an error message "Unable to open test_file.cpp. Unable to read file '/wrong/path/to/source/test_file.cpp'.
Below is an image of the error message:

Does this gives a better idea of the problem...?

[justdanpo]

I have the same issue

[matepek]

Not really, sorry.

When I run a unit test that fails, nothing is shown in the output window,

Unlikely.

"Unable to open test_file.cpp. Unable to read file '/wrong/path/to/source/test_file.cpp'.

That's not an output window.

Would you include a FULL log?

I have one idea though unlikely it will help. Try to uninstalling the "Test Explorer".

Do you have the navigate to source button on the line of the tests?

[clipitar]

After uninstalling Test Explorer, I have the same problem.

No, I do not have the "navigate to source" button on the line of the tests. Only "Run Test" and "Debug Test".

Here's the log generated by testmate:
testmate.log

I've browsed through the other logs (~/.config/Code/logs), and the only one that seems of interest is the renderer's (also shown in the Output panel when selecting the "Log (Window)" in the Output window's drop-down menu :

renderer1.log

Hopefully this is what you meant by a FULL log...?

[matepek]

Okay, from this it is pretty clear that there is no file and line information provided:

[2021-12-02 15:24:22.640] [DEBUG] testGrouping {
  testId: 'TestEnvironment_init_TC1Auto',
  file: undefined,
...

Also

[2021-12-02 15:24:22.637] [INFO] Couldn't parse output file. Possibly it is an older version of Google Test framework. It is trying to parse the output

What is your output for ./gtest2.exe --gtest_list_tests --gtest_output=xml:a.xml
And do you have a.xml file created? What is the content?

I would say the problem is with google test but you state that it works with the legacy extension. I'm confused...
Maybe I would see a log of that too.

[matepek]

Okay I see now. In the legacy extension the hyperlink is working in the output window but no navigation button either.
Ah..

Very simple. Update your google test framework. :)
Otherwise it wont provide the necessary information.

There is output window for the new extension too. There you should see the working hyperlinks.

[matepek]

I’m still not sure that you’re talking about the same thing.

[justdanpo]

Let me show what I get:

3.x:

• native diasbled:
  • Running tests from a context menu in a test source code works fine, it executes test and opens output view with results of the current test being edited.
  • Single click on a failed test in the left panel opens output of the test, double click opens the test in editor.
• native enabled:
  • Running tests from a context menu in a test source code doesn't work.
  • Running tests by clicking on icon near line number sometimes works but mostly does not.
  • Clicking on a failed test in the left panel does nothing if the test is already selected. "Show output" button opens terminal view a single line "> Test run finished at 03/12/2021, 01:07:13 <".
  • Clicking on a failed test if it's not selected opens the test in the editor and displays popup over that with the test output and execution report tree where clicking on a test name navigates the editor to the test, clicking on a failure message displays error message: Unable to open 'source.cpp': Unable to read file 'vscode-remote://ssh-remote+remotehostname/workspaserelativesourcepath/source.cpp' (Error: Unable to resolve nonexistent file 'vscode-remote://ssh-remote+remotehostname/workspaserelativesourcepath/source.cpp').

4.x:

• native enabled or disabled:
  • Running tests from a context menu in a test source code doesn't work.
  • When I click failed test in the left panel vscode opens tab with a source file in the title, immediately closes it and displays error message: Unable to open 'source.cpp': Unable to read file 'vscode-remote://ssh-remote+remotehostname/workspaserelativesourcepath/source.cpp' (Error: Unable to resolve nonexistent file 'vscode-remote://ssh-remote+remotehostname/workspaserelativesourcepath/source.cpp'). (the file exists and it's accessible).
  • When I click "show output" button it opens output view with results of all tests while I need only selected one.
    • Absolute) sourcefilepath:line of tests are clickable but they open search window with path where slashes are replaced with backslashes so the file cannot be opened.
    • Relative sourcefilepath:line of fails work fine though.

So at the moment I prefer 3.x with disabled native testing.

[clipitar]

Indeed, upgrading to the latest googletest has added the "Go to Test" button on the test line, as well as the testing context menus in test files. And the "Go to Test" functionnality works. Yay, thank you for that!

But I still get the pop-up when I run a test that fails:
12-C++ TestMate.log

What I see is pretty much the same as described by @justdanpo , except for the part in 4.x where running tests from a context menu in a test source code. This works for me...

[matepek]

I still get the pop-up when I run a test that fails

So you get this when you click on the last/location part of the line bellow:

$7| | Faulire @ ../../test_pgms_console_validate_test_pgms_console/test_RemoteTabs.cpp:43

?

[clipitar]

No, I get that when click on a failed test in the Testing panel, I don't even need to re-run it:

out.mp4

[stefanhaller]

This is easy to reproduce. It happens whenever the build system references the source files using relative paths.

Here's an easy repro (on Mac):

Create a file main.cpp at workspace root with this content:

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

TEST_CASE("T")
{
  CHECK(false);
}

Compile using this command:

clang++ -std=c++17 main.cpp -o mainTest

Make sure the setting "Testing: Automatically Open Peek View" is set to "failureAnywhere".

Run the test. You should see an error message similar to the one shown above.

If you instead build using the following command, all is fine:

clang++ -std=c++17 `realpath main.cpp` -o mainTest

--

The problem is that in ExpressionProcessor::end(), this.attribs.filename contains a relative path. It seems there's a call to resolveAndFindSourceFilePath missing somewhere, maybe inside of TestResultBuilder::addMessage (et al)?

[matepek]

Thank you for the clarification.

[matepek]

Fixed in v4.0.10

[clipitar]

Fix confirmed (in v4.0.11, because of #313)
Thanks!