Inverted enum.Flag

[GrafLearnt]

Bug report

Checklist

• I am confident this is a bug in CPython, not a bug in a third-party project
• I have searched the CPython issue tracker, and am confident this bug has not been reported before

A clear and concise description of the bug

Adding inverted option to enum.Flag brakes logic of enum.
I assume it caused by ambiguous implementation of bitwise NOT in Flag which processes ~value option different inside class scope and outside class scope unlike bitwise OR

import enum

class X(enum.Flag):
  a = enum.auto()
  b = enum.auto()
  c = a | b


assert list(X) == [X.a, X.b]
assert ~X.a == X.b
assert list(~X.a) == [X.b]


class Y(enum.Flag):
  a = enum.auto()
  b = enum.auto()
  c = a | b
  d = ~a  # this line brakes the code


assert list(Y) == [Y.a, Y.b]
assert ~Y.a == Y.b  # AssertionError
assert list(~Y.a) == [Y.b]  # ValueError: -2 is not a positive integer

Traceback:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1482, in __iter__
  yield from self._iter_member_(self._value_)
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1359, in _iter_member_by_value_
  for val in _iter_bits_lsb(value & cls._flag_mask_):
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 122, in _iter_bits_lsb
  raise ValueError('%r is not a positive integer' % original)
ValueError: -2 is not a positive integer

Your environment

• CPython versions tested on: Clang 14.0.3 (clang-1403.0.22.14.1)
• Operating system and architecture: 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64

Linked PRs

• gh-107538: [Enum] fix handling of inverted/negative values #132273

[Eclips4]

Cannot reproduce on current main. Seems that this case fixed by #105542

[ethanfurman]

It's not failing, and the first three work correctly, but Y.d should be an alias for Y.b, not -2.

[GrafLearnt]

@ethanfurman may be you are right but I find following quite confusing:
~a changed while a remaining the same

class X(enum.Flag):
    a = enum.auto()
    b = enum.auto()
    c = enum.auto()

>>> list(X)
[<X.a: 1>, <X.b: 2>, <X.c: 4>]
>>> ~X.a
<X.b|c: 6>
>>> list(~X.a)
[<X.b: 2>, <X.c: 4>]

class Y(enum.Flag):
    a = enum.auto()
    b = enum.auto()
    c = enum.auto()
    d = ~a

>>> list(Y)
[<Y.a: 1>, <Y.b: 2>, <Y.c: 4>]
>>> ~Y.a
<Y.d: -2>
>>> list(~Y.a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1482, in __iter__
    yield from self._iter_member_(self._value_)
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1359, in _iter_member_by_value_
    for val in _iter_bits_lsb(value & cls._flag_mask_):
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 122, in _iter_bits_lsb
    raise ValueError('%r is not a positive integer' % original)
ValueError: -2 is not a positive integer

[ethanfurman]

@GrafLearnt, in your latest example, d should be 6 (i.e. b|c). Thanks for the reminder, this will be a more involved fix than I was originally thinking.

[GrafLearnt]

@ethanfurman d = b | c will work perfectly fine until someone add new member without adding it to d

[ethanfurman]

Well, presumably, if they write d = b|c that's what they want; while d = ~a means every member that's not a.

[GrafLearnt]

@ethanfurman
I love it:

class X(enum.Flag):
    a = enum.auto()
    b = enum.auto()
    c = enum.auto()

>>> ~X.a
<X.b|c: 6>

But this doesn't look correct to me:

class Y(enum.Flag):
    a = enum.auto()
    b = enum.auto()
    c = enum.auto()
    d = ~a

>>> ~Y.a
<Y.d: -2>

Kindly ask to clarify if I am allowed to define inverted.
It brakes iteration of ~Y.a
Also I get following:

>>> ~Y.b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1542, in __invert__
    self._inverted_ = self.__class__(self._flag_mask_ ^ self._value_)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 711, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1136, in __new__
    raise exc
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1113, in __new__
    result = cls._missing_(value)
             ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/enum.py", line 1399, in _missing_
    raise ValueError(
ValueError: <flag 'Y'> invalid value -3
    given 0b1 01
  allowed 0b1 11

[ethanfurman]

Defining ~a as a member is what is not working correctly.

I believe that last error is fixed in 3.11.5.

[GrafLearnt]

@ethanfurman is defining inverted restricted or will be fixed?

Works fine in Python 3.10.12:

>>> ~Y.b
<Y.c|a: 5>

[ethanfurman]

Will be fixed.