Problem running debugpy in embedded python interpreter

[nicelifeBS]

Environment data

• debugpy version: 1.0.0b9
• OS and version: Windows 10
• Python version (& distribution if applicable, e.g. Anaconda): Blender 2.82, Nuke 11, Maya2018
• Using VS Code or Visual Studio: 1.45.0

Actual behavior

Remote debugging works running it in a python environment in a shell. But when run through an application with embedded python interpreter it starts a new instance of this application and it cannot connect.

I think the problem is that sys.executable is not returning the path to the python interpreter but the path of the application executable itself. And because of this the arguments passed with subprocess are not recognized.

debugpy/src/debugpy/server/api.py

Line 103 in 7a2891b

sys.executable,

If I replace sys.executable with the path to the embedded interpreter remote debugging works as expected.

An older version of ptvsd worked out of the box btw: microsoft/ptvsd@0825dbe

[tanant]

dumb question - do we need the specific sys.executable that the process/app running is using, or for the purposes of this, do we just need a valid python interpreter to mediate the comms back and forth?

[karthiknadig]

@tanant In the case mentioned above it is needed for comms. For the above case, attach using process id would probably work. Essentially start debug adapter in server mode.

python  /path/to/debugpy/adapter --host 0.0.0.0 --port 5678

Then from VSCode, use the following to connect:

        {
            "name": "Remote Attach",
            "type": "python",
            "request": "attach",
            "debugServer": 5678,
            "processId": "", // <---- put process id of the process that has the embedded python
            "pathMappings": [{
                "localRoot": "${workspaceFolder}",
                "remoteRoot": "."
            }]
        }

This way you will use the python executable for comms with VS Code, and it should be able to load the debugger where the embedded python. This uses a different pathway where we don't try to start the python executable.

[puremourning]

I think the problem is that sys.executable is not returning the path to the python interpreter but the path of the application executable itself. And because of this the arguments passed with subprocess are not recognized.

I found the same thing. This was my solution. It's absolutely not art: https://github.com/puremourning/vimspector/blob/master/python3/vimspector/developer.py#L23

[fabioz]

Well, I think that as @karthiknadig mentioned, the approach currently supported in this case is the attach by pid.

I think that we could extend that to accept a python_exe in debugpy.listen (which would default to sys.executable) so that the user could customize it without fiddling with sys.executable as @puremourning mentioned.

One thing to note though is that we don't support automatically attaching to subprocesess in this case (we check if the executable basename has python on it so that we don't inadvertently attach to some non-python process such as calc and crash it).

[int19h]

This kind of stuff should go into debugpy.configure() if we do it.

What's not clear to me is why the adapter running under a custom host in this case. It's a separate process from the debugged app, and it kinda expects to be run using a regular interpreter. But it then applies "python" from launch.json to start the debuggee (and only falls back to sys.executable if it wasn't specified).

[puremourning]

What's not clear to me is why the adapter running under a custom host in this case

we're calling import debugpy; debugpy.listen() from within a host program that embeds the python interp (in my case, the host program is vim). Then debugpy is shelling out to start another interpreter using sys.executable

applies "python" from launch.json to start the debuggee

This is an "attach" scenario.

[int19h]

Ahh, that makes perfect sense now. Yes, we should just make this configurable.

[int19h]

What we want here is really the same thing as "python" config property in the launch scenario, except it needs to be applied before the client has a chance to connect and transmit the configuration in the "attach" request. That's exactly why we added debugpy.configure() - to handle "subProcess" in the same manner.

So we should just support "python" in configure().

[dalmago]

A possible workaround for Windows is to use sys.exec_prefix and concatenate it with \\python.exe. I'm replacing sys.executable temporarily, before calling debugpy.listen. As said above, "it's absolutely not art".

[ghaefele]

I have the same problem with ArcGIS Pro and Python Toolbox Tools.
The adapter times out ("timed out waiting for adapter to connect") whatever I try (changing sys.executable, attach using listen, attach, attach using process id...).
If I don't change sys.executable the debugger starts a new ArcGIS Pro instance.

Until June 2020 I used ptvsd with "attach" and that worked fine.

Environment:

• Windows 10
• ArcGIS Pro 2.6 with embedded Python 3.6.10 (custom Miniconda environment)
• VS Code 1.48.2
• debugpy 1.0.0rc2