Mailing List Archive

[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
New submission from Kevin Chen <kevinjchen94@gmail.com>:

Here's a code sample:

```
import time
from enum import Flag, auto


class MyFlag(Flag):
NONE = 0
FLAG_1 = auto()
FLAG_2 = auto()
FLAG_3 = auto()
FLAG_4 = auto()
FLAG_5 = auto()
FLAG_6 = auto()
#
# NOT_FLAG_1_OR_2 = ~FLAG_1 & ~FLAG_2


def test_flag():
f = MyFlag.NONE
inverted = (
~MyFlag.FLAG_1
& ~MyFlag.FLAG_2
& ~MyFlag.FLAG_3
& ~MyFlag.FLAG_4
& ~MyFlag.FLAG_5
& ~MyFlag.FLAG_6
)
return f & inverted


INVERTED = (
~MyFlag.FLAG_1
& ~MyFlag.FLAG_2
& ~MyFlag.FLAG_3
& ~MyFlag.FLAG_4
& ~MyFlag.FLAG_5
& ~MyFlag.FLAG_6
)


def test_flag_cached():
f = MyFlag.NONE
return f & INVERTED


if __name__ == "__main__":
start_time = time.time()
for _ in range(10_000):
test_flag()

elapsed = time.time() - start_time
print(f"Took normal {elapsed:2f} seconds.")

start_time = time.time()
for _ in range(10_000):
test_flag_cached()

elapsed = time.time() - start_time
print(f"Took cached {elapsed:2f} seconds.")
```

And its outputs:
```
Took normal 1.799731 seconds.
Took cached 0.009488 seconds.
```

Basically, bitwise negation is very very slow. From what I can tell, it seems that a lot of time is spent here computing powers of two. I've read elsewhere that flag values are cached, and it looks like negated Flag values can't be cached? This seems related to the second issue, which is that any negated Flag value being defined results in `RecursionError: maximum recursion depth exceeded` as it searches for a good name for Flag.

Obviously, the simple workaround is just to define a constant variable elsewhere with the negated value, but it isn't very obvious anywhere that this is necessary, and I wanted to raise this to see if anyone has knowledge of the implementation details of Flag for possibly resolving this in the class itself.

----------
messages: 384983
nosy: aspin2
priority: normal
severity: normal
status: open
title: enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
versions: Python 3.8

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value [ In reply to ]
Change by Guido van Rossum <guido@python.org>:


----------
nosy: +ethan.furman

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value [ In reply to ]
Ethan Furman <ethan@stoneleaf.us> added the comment:

I just finished a rewrite of Flag for 3.10. Using your test below I was able to tweak the rewrite so the final numbers are:

Took normal 0.148092 seconds.
Took cached 0.017438 seconds.

Your original post had a ratio of nearly 200 -- it is now 8.7ish.

----------
assignee: -> ethan.furman

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value [ In reply to ]
Kevin Chen <kevinjchen94@gmail.com> added the comment:

Awesome thanks! Does the rewrite fix the issue with creating negated flags as well?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value [ In reply to ]
Ethan Furman <ethan@stoneleaf.us> added the comment:

Yes.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue42915] enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value [ In reply to ]
Ethan Furman <ethan@stoneleaf.us> added the comment:

Fixed in 3.10 in issue38250.

Also fixed in my 3rd-party library, aenum 3.0:

(https://pypi.org/project/aenum/)

----------
resolution: -> wont fix
stage: -> resolved
status: open -> closed
superseder: -> enum.Flag should be more set-like
type: -> behavior

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42915>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com