Skip to content

Commit

Permalink
pybind/mgr: maximum recursion depth exceeded in comparison
Browse files Browse the repository at this point in the history
Original issue from python/typing_extensions#10

In case typing-extensions is upgrade to 4.1.1
(https://pypi.org/project/typing-extensions/4.1.1/, the final Python 3.6
supported version), Ceph MGR will failed with pg_autoscaler as below:

    [root@cp-nightsky ~]# pip list | grep typing-extensions
    typing-extensions      4.1.1

    [root@cp-nightsky ~]# systemctl restart ceph-mgr@cp-nightsky.service

    [root@cp-nightsky ~]# journalctl -xef -u ceph-mgr@cp-nightsky.service
    May 12 06:16:39 cp-nightsky.novalocal systemd[1]: Started Ceph cluster manager daemon.
    -- Subject: Unit ceph-mgr@cp-nightsky.service has finished start-up
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit ceph-mgr@cp-nightsky.service has finished starting up.
    --
    -- The start-up result is done.
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.181+0000 7fe3eb521700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'pg_autoscaler' while running on mgr.cp-nightsky: maximum recursion depth exceeded in comparison
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.182+0000 7fe3eb521700 -1 pg_autoscaler.serve:
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.182+0000 7fe3eb521700 -1 Traceback (most recent call last):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 206, in serve
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self._maybe_adjust()
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 423, in _maybe_adjust
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: ps, root_map, pool_root = self._get_pool_status(osdmap, pools)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 325, in _get_pool_status
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self.log.debug('skipping empty subtree %s', cr_name)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1296, in debug
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: self._log(DEBUG, msg, args, **kwargs)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1443, in _log
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: exc_info, func, extra, sinfo)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 1413, in makeRecord
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: sinfo)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/logging/__init__.py", line 277, in __init__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if (args and len(args) == 1 and isinstance(args[0], collections.Mapping)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 193, in __instancecheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: return cls.__subclasscheck__(subclass)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/typing.py", line 1154, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: return super().__subclasscheck__(cls)
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib64/python3.6/abc.py", line 228, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 426, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if issubclass(subclass, scls):
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: [Previous line repeated 239 more times]
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: File "/lib/python3.6/site-packages/typing_extensions.py", line 421, in __subclasscheck__
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: if self.__extra__ in subclass.__mro__:
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: RecursionError: maximum recursion depth exceeded in comparison
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists
    May 12 06:16:51 cp-nightsky.novalocal ceph-mgr[1719]: 2022-05-12T06:16:51.291+0000 7fe3de3c7700 -1 client.0 error registering admin socket command: (17) File exists

Solution provided by
python/typing_extensions#10 (comment)
not working:

    self.log.debug('skipping empty subtree %s', str(cr_name))

But implement with `.format` could help:

    self.log.debug("skipping empty subtree {0}".format(cr_name))

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit 8ff8e09)
  • Loading branch information
hswong3i committed Aug 20, 2022
1 parent f2877ae commit fd02e7f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/pybind/mgr/pg_autoscaler/module.py
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ def _get_pool_status(

capacity = root_map[root_id].capacity
if capacity == 0:
self.log.debug('skipping empty subtree %s', cr_name)
self.log.debug("skipping empty subtree {0}".format(cr_name))
continue

raw_used_rate = osdmap.pool_raw_used_rate(pool_id)
Expand Down

0 comments on commit fd02e7f

Please sign in to comment.