Regression of #17414: Folders named 'py' result in "Test result not found"

[orbelico]

Type: Bug

Behaviour

As described in #17414, if I have a directory named "py" somewhere in the path of pytest tests, they will report "Test result not found". Renaming the folder fixes the issue. #17414 is closed and I cannot comment there, so I am raising a new issue.

Steps to reproduce:

1. Create a python project with a pytest-based test
2. Have a folder named "py" somewhere in the path leading to the tests. This can be a higher folder (in my case, it is the project folder itself), does not have to be the folder directly containing the tests.
3. Discover and run the tests through VS Code UI. The tests will run and pass correctly(!) - visible under "Output -> Python Test Log". However, all tests are marked as red in the UI and in the "Test Results" pane, we get

Total number of tests expected to run: 1
Total number of tests run: 0
Total number of tests passed: 0
Total number of tests failed: 0
Total number of tests failed with errors: 0
Total number of tests skipped: 0
Total number of tests with no result data: 1
Finished running tests!

4. Rename the "py" folder to anything else (including "python"!) - and everything works.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

Extension version: 2024.22.1
VS Code version: Code 1.95.3 (f1a4fb101478ce6ec82fe9627c43efbf9e98c813, 2024-11-13T14:50:04.152Z)
OS version: Linux x64 5.15.0-127-generic
Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Installed Extensions

Extension Name	Extension Id	Version
autoDocstring - Python Docstring Generator	njpwerner.autodocstring	0.6.1
Bash IDE	mads-hartmann.bash-ide-vscode	1.43.0
Dev Containers	ms-vscode-remote.remote-containers	0.394.0
Docker	ms-azuretools.vscode-docker	1.29.3
DotENV	mikestead.dotenv	1.0.1
Eva Theme	fisheva.eva-theme	2.7.3
Even Better TOML	tamasfe.even-better-toml	0.21.2
Git Graph	mhutchie.git-graph	1.30.0
GitHub Copilot	GitHub.copilot	1.250.0
GitHub Copilot Chat	GitHub.copilot-chat	0.22.4
GitLab Workflow	GitLab.gitlab-workflow	5.25.1
Gitmoji	Vtrois.gitmoji-vscode	1.0.9
Home Assistant Config Helper	keesschollaart.vscode-home-assistant	1.43.0
JavaScript Debugger	ms-vscode.js-debug	1.95.3
JavaScript Debugger Companion Extension	ms-vscode.js-debug-companion	1.1.3
Jupyter	ms-toolsai.jupyter	2024.10.0
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Keymap	ms-toolsai.jupyter-keymap	1.1.2
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.21
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
LaTeX Workshop	James-Yu.latex-workshop	10.7.1
LTeX – LanguageTool grammar/spell checking	valentjn.vscode-ltex	13.1.0
Markdown All in One	yzhang.markdown-all-in-one	3.6.2
Markdown Table	TakumiI.markdowntable	0.12.0
markdownlint	DavidAnson.vscode-markdownlint	0.57.0
Pylance	ms-python.vscode-pylance	2024.12.1
Python	ms-python.python	2024.22.1
Python Debugger	ms-python.debugpy	2024.14.0
Python Indent	KevinRose.vsc-python-indent	1.19.0
Ruff	charliermarsh.ruff	2024.56.0
ShellCheck	timonwong.shellcheck	0.37.1
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.10
Thunder Client	rangav.vscode-thunder-client	2.33.2
Visual Studio Dark Theme	Adrien.VisualStudioDarkTheme	1.0.1

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz (4 x 2799)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off webnn: disabled_off
Load (avg)	1, 1, 1
Memory (System)	7.66GB (3.08GB free)
Process Argv	.
Screen Reader	no
VM	0%
DESKTOP_SESSION	plasma
XDG_CURRENT_DESKTOP	KDE
XDG_SESSION_DESKTOP	KDE
XDG_SESSION_TYPE	x11

[eleanorjboyd]

ah interesting- I see why this happens as it interferes with how we parse the tests and build the test tree. I will investigate a fix, thanks

[eleanorjboyd]

Hi! I was able to get this working- see screenshot. Do I have a part of my setup wrong? Also are you on the rewrite experiment? You can check if its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up. To set to trace use theDeveloper: set log level command in the command palette. Thanks

[orbelico]

Hey @eleanorjboyd,

thanks for following up and sorry for the late response. Unfortunately, I cannot see the screenshot you are talking about?

Anyway, I checked the Python log and I have:
2025-01-21 23:44:37.184 [info] Experiments are disabled, only manually opted experiments are active.

I also checked my settings and I have not opted into any experiment.

However, I don't quite get the log message above, because I also have this in settings 🤔

[orbelico]

Ok, I played around a little and found out the following additional info, that I hope might help:

• Experiments were disabled, because I had a deprecated setting in my json file to disable telemetry entirely.
• When I enabled telemetry, I became part of the pythonTestAdapter experiment. With that on, I was not able to check if the problem still occurs however, because it was not even able to collect the tests (not matter if I had a py folder in the directory tree or not). But that is probably a separate problem, I guess.

[eleanorjboyd]

@orbelico do you have anything in your logs that might explain it?