Test failure of `test_py_info_cache_invalidation_on_py_info_change`

[frenzymadness]

I'm bulding the latest version 20.33.0 as an RPM package in Fedora Linux with Python 3.14 and the test mentioned in the subject fails with:

    def test_py_info_cache_invalidation_on_py_info_change(mocker, session_app_data):
        # 1. Get a PythonInfo object for the current executable, this will cache it.
        PythonInfo.from_exe(sys.executable, session_app_data)
    
        # 2. Spy on _run_subprocess
        spy = mocker.spy(cached_py_info, "_run_subprocess")
    
        # 3. Modify the content of py_info.py
        py_info_script = Path(cached_py_info.__file__).parent / "py_info.py"
        original_content = py_info_script.read_text(encoding="utf-8")

        try:
            # 4. Clear the in-memory cache
            mocker.patch.dict(cached_py_info._CACHE, {}, clear=True)  # noqa: SLF001
            py_info_script.write_text(original_content + "\n# a comment", encoding="utf-8")
    
            # 5. Get the PythonInfo object again
            info = PythonInfo.from_exe(sys.executable, session_app_data)

            # 6. Assert that _run_subprocess was called again
            if is_macos_brew(info):
                assert spy.call_count in {2, 3}
            else:
>               assert spy.call_count == 2
E               assert 1 == 2
E                +  where 1 = <function _run_subprocess at 0x7fd7655e5d20>.call_count

I've debugged it, and it seems that the spy.call_count is increased only once (0 → 1) in step 5.

I don't understand why the spy.call_count should be 2 when the spy is created after step one, which should mean that the first call to _run_subprocess in step one is not recorded by the mocker.

[gaborbernat]

cc @esafak

[esafak]

@gaborbernat knows best, but I see possibly two probes here, if isinstance(proposed, PythonInfo) and resolve_to_host via _resolve_to_system:

virtualenv/src/virtualenv/discovery/py_info.py

Lines 441 to 459 in 3163a11

	def from_exe( # noqa: PLR0913
	cls,
	exe,
	app_data=None,
	raise_on_error=True, # noqa: FBT002
	ignore_cache=False, # noqa: FBT002
	resolve_to_host=True, # noqa: FBT002
	env=None,
	):
	"""Given a path to an executable get the python information."""
	# this method is not used by itself, so here and called functions can import stuff locally
	from virtualenv.discovery.cached_py_info import from_exe # noqa: PLC0415
	env = os.environ if env is None else env
	proposed = from_exe(cls, app_data, exe, env=env, raise_on_error=raise_on_error, ignore_cache=ignore_cache)
	if isinstance(proposed, PythonInfo) and resolve_to_host:
	try:
	proposed = proposed._resolve_to_system(app_data, proposed) # noqa: SLF001

I'm guessing isinstance(proposed, PythonInfo) failed in your case.

[frenzymadness]

I've tested the patch from #2940 and it does not solve the issue. Let me take a look at what is happening in the from_exe function.

[frenzymadness]

if isinstance(proposed, PythonInfo) and resolve_to_host: is True in both cases – step 1 and 5 in the test and the proposed is returned from from_exe.

But when I stepped into the from_exe function in PDB for the second time, I got:

*** WARNING: file '/builddir/build/BUILD/python-virtualenv-20.33.0-build/BUILDROOT/usr/lib/python3.14/site-packages/virtualenv/discovery/py_info.py' was edited, running stale code until the program is rerun

I'm not sure whether this might be related or not.

[esafak]

That PR is for a different issue; the correct PR is #2938. I see belatedly there are two identical issues (cf. #2933)

A bug in Github causes stale links not to be removed.

Please try #2938 and report the result.

[frenzymadness]

I'm building 20.33.1, which includes the fix merged in #2938, and I still see the same test failure.

[esafak]

Is this a 3.14 thing? We don't support it anyway; it's a release candidate. I just fired up Python 3.13.5 on Fedora 42 and the test passed. What architecture did you test on? I'm on aarch64.

[frenzymadness]

Is this a 3.14 thing?

No. I've tried the same RPM build procedure with Python 3.13 on Fedora 42 and it failed as well.

What value should the spy.call_count have during the test execution? The assert expects that a single call to PythonInfo.from_exe calls _run_subprocess twice, right?

[esafak]

Are you running the test in a virtual environment? If not you miss one call:

if target.executable != target.system_executable:
    target = cls.from_exe(target.system_executable, app_data, cache=cache)

https://github.com/esafak/virtualenv/blob/78ebc61f8f30ff3ab2e175418ac69fe112002ff5/src/virtualenv/discovery/py_info.py#L520-L521

[frenzymadness]

You are right! The condition you've mentioned is false in our environment, where we run tests without an active virtual environment. I guess there is nothing to do on your side so I'm going to close this issue, and we'll ignore the test in the RPM build process.

[esafak]

I'm hardening the test to fix that discrepancy.

[frenzymadness]

It seems we have a similar issue with the latest version 20.35.0:

___________________________ test_py_info_cache_clear ___________________________

mocker = <pytest_mock.plugin.MockerFixture object at 0x7fdcdd5eaa90>

    def test_py_info_cache_clear(mocker):
        spy = mocker.spy(cached_py_info, "_run_subprocess")
        app_data, cache = MockAppData(), MockCache()
        result = PythonInfo.from_exe(sys.executable, app_data, cache)
        assert result is not None
        count = 1 if result.executable == sys.executable else 2  # at least two, one for the venv, one more for the host
        assert spy.call_count >= count
        PythonInfo.clear_cache()
        assert PythonInfo.from_exe(sys.executable, app_data, cache) is not None
>       assert spy.call_count >= 2 * count
E       assert 1 >= (2 * 1)
E        +  where 1 = <function _run_subprocess at 0x7fdcdcdf6c40>.call_count

I can disable or patch it on our side, but since you fixed the previous problem, I thought you might want to be aware of a similar issue in the latest version.

Feel free to close this if you are not interested in this type of issues anymore.