Hi there,
I've spent quite some time on memory profiling for Python now. I'm
struggling to get more information from the allocated memory right now
for what looks like a sad reason. :(
Supposedly PyObject_Malloc() returns some memory space to store a
PyObject. If that was true all the time, that would allow anyone to
introspect the allocated memory and understand why it's being used.
Unfortunately, this is not the case. Objects whose types are tracked by
the GC go through _PyObject_GC_Alloc() which changes the underlying
memory structure to be (PyGC_HEAD + PyObject).
This is a bummer as there then no safe way that I can think of to know
if an allocated memory space is gc-tracked or gc-untracked. It makes it
therefore impossible to introspect the memory allocated by
PyObject_Malloc().
There are multiple ways to solve this, but I don't see any way right now
of doing this without slightly changing CPython.
Has anyone any idea on how to workaround this or the kind of change that
could be acceptable to mitigate the issue?
Thanks!
Cheers,
--
Julien Danjou
# Free Software hacker
# https://julien.danjou.info
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QRX6U5XBXMHMT6YKIXERS3UT64ALYV27/
Code of Conduct: http://python.org/psf/codeofconduct/
I've spent quite some time on memory profiling for Python now. I'm
struggling to get more information from the allocated memory right now
for what looks like a sad reason. :(
Supposedly PyObject_Malloc() returns some memory space to store a
PyObject. If that was true all the time, that would allow anyone to
introspect the allocated memory and understand why it's being used.
Unfortunately, this is not the case. Objects whose types are tracked by
the GC go through _PyObject_GC_Alloc() which changes the underlying
memory structure to be (PyGC_HEAD + PyObject).
This is a bummer as there then no safe way that I can think of to know
if an allocated memory space is gc-tracked or gc-untracked. It makes it
therefore impossible to introspect the memory allocated by
PyObject_Malloc().
There are multiple ways to solve this, but I don't see any way right now
of doing this without slightly changing CPython.
Has anyone any idea on how to workaround this or the kind of change that
could be acceptable to mitigate the issue?
Thanks!
Cheers,
--
Julien Danjou
# Free Software hacker
# https://julien.danjou.info
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QRX6U5XBXMHMT6YKIXERS3UT64ALYV27/
Code of Conduct: http://python.org/psf/codeofconduct/