I've been facing an issue in a repo with a structure like:

/app
  __init__.py
  main.py
  whatever.py
  # etc
/tests
  __init__.py
  something/__init__.py
  something/else.py
  another_thing/__init__.py
  another_thing/problem_file.py
  # etc
/pyproject.toml

with the contents of pyproject.toml being:

[tool.isort]
profile = "black"
src_paths = ["app", "tests"]

and /tests/another_thing/problem_file.py containing:

from tests.something.else import some_function

What I was seeing was inconsistent results - when running isort tests --check -v, about half the time I would see tests.something.else module correctly being identified as FIRSTPARTY in its own absolute imports, and the other half of the time, it would be identified as THIRDPARTY.

My clue was the fact that, if you run with --show-config a number of times, you'll see that all of the list orders are nondeterministic between runs. The issue looks to be here:

_src_path is attempting to recurse to figure out if the module is on one of the src_paths or not - but, if the ordering for my run happened to place app first, auto_identify_namespace_packages defaults to True and there's an __init__,py in the app directory, so _is_namespace_package returns True, so it recurses into the app path. But because that's done via return - it never considers the tests entry.

If I change my pyproject.toml to either include auto_identify_namespace_packages = false or use known_local_folder instead of src_paths, I no longer get inconsistent results (and always get what I expect).