New submission from Yonatan Goldschmidt <yon.goldschmidt@gmail.com>:
Key-sharing dictionaries, defined by https://www.python.org/dev/peps/pep-0412/, require that any resizing of the shared dictionary keys will happen before a second instance of the class is created.
cached_property inserts its resolved result into the instance dict after it is called. This is likely to happen *after* a second instance has been created, and it is also likely to cause a resize of the dict, as demonstrated by this snippet:
from functools import cached_property
import sys
def dict_size(o):
return sys.getsizeof(o.__dict__)
class X:
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
self.d = 4
self.e = 5
@cached_property
def f(self):
return id(self)
x1 = X()
x2 = X()
print(dict_size(x1))
print(dict_size(x2))
x1.f
print(dict_size(x1))
print(dict_size(x2))
x3 = X()
print(dict_size(x3))
Essentially it means that types using cached_property are less likely to enjoy the benefits of shared keys. It may also incur a certain performance hit, because a resize + unshare will happen every time.
A simple way I've thought of to let cached_property play more nicely with shared keys, is to first create a single object of the class, and set the cached_property attribute to some value (so the key is added to the shared dict). In the snippet above, if you add "x0 = X(); x0.f = None" before creating x1 and x2, you'll see that the cached_property resolving does not unshare the dicts.
But I wonder if there's a way to do so without requiring user code changes.
----------
components: Library (Lib)
messages: 379439
nosy: Yonatan Goldschmidt
priority: normal
severity: normal
status: open
title: functools.cached_property possibly disables key-sharing instance dictionaries
type: performance
versions: Python 3.10
_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42127>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
Key-sharing dictionaries, defined by https://www.python.org/dev/peps/pep-0412/, require that any resizing of the shared dictionary keys will happen before a second instance of the class is created.
cached_property inserts its resolved result into the instance dict after it is called. This is likely to happen *after* a second instance has been created, and it is also likely to cause a resize of the dict, as demonstrated by this snippet:
from functools import cached_property
import sys
def dict_size(o):
return sys.getsizeof(o.__dict__)
class X:
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
self.d = 4
self.e = 5
@cached_property
def f(self):
return id(self)
x1 = X()
x2 = X()
print(dict_size(x1))
print(dict_size(x2))
x1.f
print(dict_size(x1))
print(dict_size(x2))
x3 = X()
print(dict_size(x3))
Essentially it means that types using cached_property are less likely to enjoy the benefits of shared keys. It may also incur a certain performance hit, because a resize + unshare will happen every time.
A simple way I've thought of to let cached_property play more nicely with shared keys, is to first create a single object of the class, and set the cached_property attribute to some value (so the key is added to the shared dict). In the snippet above, if you add "x0 = X(); x0.f = None" before creating x1 and x2, you'll see that the cached_property resolving does not unshare the dicts.
But I wonder if there's a way to do so without requiring user code changes.
----------
components: Library (Lib)
messages: 379439
nosy: Yonatan Goldschmidt
priority: normal
severity: normal
status: open
title: functools.cached_property possibly disables key-sharing instance dictionaries
type: performance
versions: Python 3.10
_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42127>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com