bug: RecursionError when subclass is attribute

[patrick-kidger]

Description of the bug

Code of the following form produces a RecursionError in griffe:

class X:
    pass

class Y(X):
    pass

X.Y = Y

To Reproduce

https://github.com/patrick-kidger/griffe-error

FWIW I'm providing this via mkdocs, as I didn't quite piece together the correct griffe.load(...) invocation to obtain this error. A simple griffe.load("mylib", force_inspection=True) does not raise the error.

Full traceback

Full traceback

INFO    -  Building documentation...
INFO    -  Cleaning site directory
ERROR   -  Error reading page 'foo.md': maximum recursion depth exceeded
Traceback (most recent call last):
  File ".../.venv/lib/python3.12/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocs/__main__.py", line 272, in serve_command
    serve.serve(**kwargs)
  File ".../.venv/lib/python3.12/site-packages/mkdocs/commands/serve.py", line 85, in serve
    builder(config)
  File ".../.venv/lib/python3.12/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, serve_url=None if is_clean else serve_url, dirty=is_dirty)
  File ".../.venv/lib/python3.12/site-packages/mkdocs/commands/build.py", line 310, in build
    _populate_page(file.page, config, files, dirty)
  File ".../.venv/lib/python3.12/site-packages/mkdocs/commands/build.py", line 167, in _populate_page
    page.render(config, files)
  File ".../.venv/lib/python3.12/site-packages/mkdocs/structure/pages.py", line 285, in render
    self.content = md.convert(self.markdown)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/markdown/core.py", line 357, in convert
    root = self.parser.parseDocument(self.lines).getroot()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/markdown/blockparser.py", line 117, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File ".../.venv/lib/python3.12/site-packages/markdown/blockparser.py", line 136, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File ".../.venv/lib/python3.12/site-packages/markdown/blockparser.py", line 158, in parseBlocks
    if processor.run(parent, blocks) is not False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings/_internal/extension.py", line 126, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings/_internal/extension.py", line 201, in _process_block
    rendered = handler.render(data, options)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/_internal/handler.py", line 294, in render
    return template.render(
           ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
  File ".../.venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/templates/material/class.html.jinja", line 1, in top-level template code
    {% extends "_base/class.html.jinja" %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/templates/material/_base/class.html.jinja", line 120, in top-level template code
    {% block contents scoped %}
^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/templates/material/_base/class.html.jinja", line 220, in block 'contents'
    {% block children scoped %}


... some thousand lines from jinja templating removed for brevity ...


  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/templates/material/children.html.jinja", line 1, in top-level template code
    {% extends "_base/children.html.jinja" %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/mkdocstrings_handlers/python/templates/material/_base/children.html.jinja", line 13, in top-level template code
    {% if obj.all_members %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/jinja2/environment.py", line 490, in getattr
    return getattr(obj, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/mixins.py", line 299, in all_members
    if self.is_class:  # type: ignore[attr-defined]
       ^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 1345, in is_class
    return self.final_target.is_class
           ^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 1693, in final_target
    if target.path in paths_seen:
       ^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 1152, in path
    return f"{self.parent.path}.{self.name}"  # type: ignore[union-attr]
              ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 1152, in path
    return f"{self.parent.path}.{self.name}"  # type: ignore[union-attr]
              ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 1152, in path
    return f"{self.parent.path}.{self.name}"  # type: ignore[union-attr]
              ^^^^^^^^^^^^^^^^
  [Previous line repeated 134 more times]
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 856, in path
    return self.canonical_path
           ^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 868, in canonical_path
    return f"{self.parent.path}.{self.name}"
              ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 856, in path
    return self.canonical_path
           ^^^^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 868, in canonical_path
    return f"{self.parent.path}.{self.name}"
              ^^^^^^^^^^^^^^^^
  File ".../.venv/lib/python3.12/site-packages/_griffe/models.py", line 856, in path
    return self.canonical_path
           ^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded

Expected behavior

No crash :)

Environment information

- __System__: macOS-15.3.2-arm64-arm-64bit
- __Python__: cpython 3.12.0 (.../.venv/bin/python3)
- __Environment variables__:
- __Installed packages__:
  - `griffe` v1.6.2

[pawamoy]

Thanks for all the reports @patrick-kidger! I was able to reproduce this one too. Looks like Y is attached as a member of itself during dynamic analysis.

[pawamoy]

Hmm yep, by assigning Y to X, Y inherits it too 😆 I'll try to break the recursion there, when using inspect.getmembers to get the members of an object.

[pawamoy]

Hmmm actually the recursion is already handled in Griffe but not in mkdocstrings-python. I think it's best to fix it in mkdocstrings-python 🤔

[pawamoy]

It was more difficult than I anticipated. X.Y and Y are seen as distinct classes, making it hard to detect later that Y.Y is actually Y. Ideally we would detect that X.Y is Y, but that's not easy either with dynamic analysis. When we inspect X, we see the Y attribute, so we add it as member. Later, we see Y, which should tell us to update X.Y to point at it. It means we somehow have to keep a global mapping of object ids (id(obj)) to reduce duplicates and use aliases instead. Even then, with dynamic analysis there's no way to distinguish these two cases:

class X: ...
class Y: ...
X.Y = Y

class X:
    class Y: ...
Y = X.Y

Essentially, we never know which of Y or X.Y is the original class and which is the alias. That means we won't be able to "help" mkdocstrings-python from Griffe this way.

I've found a solution anyway: when selecting members to render, I make sure none of them is an alias that points to any of its parent, or any of its parents' target.

[patrick-kidger]

Super happy this is fixed - and so quickly - thank you for your efforts. I'll give it a go on my own use-case.

On your point about distinguishing those two cases, I think f"{Y.__module__}.{Y.__qualname__}" would define the path to the original class, and the other would be the alias. Other than this, the two cases are essentially identical to the runtime interpreter, after all -- arguably there isn't any difference by definition.

(And this would be true even if __module__ and __qualname__ are reassigned; this indicates a user's intent to declare which should be thought of as the original.)

[pawamoy]

Ah, right, thanks for mentionning __qualname__. This will be useful if I ever do such "backtracking" to remove duplicates 🙂

New version not released yet, but soon!

[patrick-kidger]

I've just tried mkdocstrings-python at commit eba28e5c4ba3205837b34877eae002b9e9fff226 and unfortunately whilst it does fix the (clearly too simplistic) MWE I provided, the RecursionError is still raised in my real world use-case :/

I've just pushed an updated version to my repo above. In still seems to be interacting badly with the use of a Griffe extension.

I don't know of a simple way to include an extension in a MWE (Griffe seems to rely on entry-points to locate these, I think, which adds a lot of boilerplate to the setup) so as we've discussed it already then I've taken the simplest route of just adding hippogriffe as the extension. It's not doing anything fancy, the recusion error is just arising when calling .all_members, same as before.

[pawamoy]

Griffe seems to rely on entry-points to locate these

It relies on the package name (or a script path), no need for entry-points 😄

So, the nature of your code means there is infinite recursion going on in its structure, since Y inherits itself (as attribute) from X. The thing however is that inherited objects are aliased to their parent ones, so Y.foo, Y.Y.foo and Y.Y.Y.foo are different objects, which means Y.Y.Y.foo is still processed while Y.Y.foo is already in the seen set. The solution here would be to either:

• change Griffe so that it ignores Y.Y (... and not member is self.obj and ...)
• change your extension so that it adds the path of objects / alias final targets to the seen set instead of the instance themselves (seen.add(final_item.path) if I read correctly)

WDYT?