Mailing List Archive

Method __func__ objects are unpicklable
During some sophisticated pickling I noticed that method `__func__`
objects are unpicklable, because they share the name with the bound
method object itself.


```
from pickle import dumps

class A:
@classmethod
def b(cls):
pass

print(A.b) # <bound method A.b of <class '__main__.A'>>
print(A.b.__func__) # <function A.b at 0x7f5574570bf8>

dumps(A.b) # works
dumps(A.b.__func__) # breaks
# >Traceback (most recent call last):
# > File "<stdin>", line 1, in <module>
# >_pickle.PicklingError: Can't pickle <function A.b at 0x7fd1d50d8bf8>:
# >it's not the same object as __main__.A.b
```


The last call compains that global symbol "A.b" is not the same object
as `A.b.__func__`.

Everything would work if the `__func__` objects had the suffix
".__func__" in their qualname.

Actually setting the qualname of the `__func__` object makes it
picklable, but then the bound method object is unpicklable, as it
inherits the name from the `__func__`. It would be good if they were
separate.

This is an attempt at emulating the desired behavior:


```
from pickle import dumps

class A:
pass

def c(cls):
pass

A.c = lambda: c(A)
A.c.__qualname__ = "A.c"
A.c.__name__ = "c"
A.c.__self__ = A
A.c.__func__ = c
A.c.__func__.__qualname__ = "A.c.__func__"

print(A.c) # <function A.c at 0x7f83ceabcbf8>
print(A.c.__func__) # <function A.c.__func__ at 0x7f83ceabcb70>

dumps(A.c) # works
dumps(A.c.__func__) # works
```


Can we make the `__func__` objects picklable this way?


haael

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N2J6MVLURTBAB5PJLJG3LO6U4PTPJMU2/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Method __func__ objects are unpicklable [ In reply to ]
This is not just about the content of the name attributes: ;bound methods
do contain a reference to the specific instance
they are bound too - which is not retrievable (or maybe is through some
dunder attribute) - this instance has to be pickled, transported
and its reference updated on unpickling.

On Fri, Jun 10, 2022 at 9:00 AM haael <haael@interia.pl> wrote:

>
>
> During some sophisticated pickling I noticed that method `__func__`
> objects are unpicklable, because they share the name with the bound
> method object itself.
>
>
> ```
> from pickle import dumps
>
> class A:
> @classmethod
> def b(cls):
> pass
>
> print(A.b) # <bound method A.b of <class '__main__.A'>>
> print(A.b.__func__) # <function A.b at 0x7f5574570bf8>
>
> dumps(A.b) # works
> dumps(A.b.__func__) # breaks
> # >Traceback (most recent call last):
> # > File "<stdin>", line 1, in <module>
> # >_pickle.PicklingError: Can't pickle <function A.b at 0x7fd1d50d8bf8>:
> # >it's not the same object as __main__.A.b
> ```
>
>
> The last call compains that global symbol "A.b" is not the same object
> as `A.b.__func__`.
>
> Everything would work if the `__func__` objects had the suffix
> ".__func__" in their qualname.
>
> Actually setting the qualname of the `__func__` object makes it
> picklable, but then the bound method object is unpicklable, as it
> inherits the name from the `__func__`. It would be good if they were
> separate.
>
> This is an attempt at emulating the desired behavior:
>
>
> ```
> from pickle import dumps
>
> class A:
> pass
>
> def c(cls):
> pass
>
> A.c = lambda: c(A)
> A.c.__qualname__ = "A.c"
> A.c.__name__ = "c"
> A.c.__self__ = A
> A.c.__func__ = c
> A.c.__func__.__qualname__ = "A.c.__func__"
>
> print(A.c) # <function A.c at 0x7f83ceabcbf8>
> print(A.c.__func__) # <function A.c.__func__ at 0x7f83ceabcb70>
>
> dumps(A.c) # works
> dumps(A.c.__func__) # works
> ```
>
>
> Can we make the `__func__` objects picklable this way?
>
>
> haael
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/N2J6MVLURTBAB5PJLJG3LO6U4PTPJMU2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: Method __func__ objects are unpicklable [ In reply to ]
I believe a lot of people use https://pypi.org/project/cloudpickle/ thanks
to its support for pickling functions.

On Fri, Jun 10, 2022 at 4:56 AM haael <haael@interia.pl> wrote:

>
>
> During some sophisticated pickling I noticed that method `__func__`
> objects are unpicklable, because they share the name with the bound
> method object itself.
>
>
> ```
> from pickle import dumps
>
> class A:
> @classmethod
> def b(cls):
> pass
>
> print(A.b) # <bound method A.b of <class '__main__.A'>>
> print(A.b.__func__) # <function A.b at 0x7f5574570bf8>
>
> dumps(A.b) # works
> dumps(A.b.__func__) # breaks
> # >Traceback (most recent call last):
> # > File "<stdin>", line 1, in <module>
> # >_pickle.PicklingError: Can't pickle <function A.b at 0x7fd1d50d8bf8>:
> # >it's not the same object as __main__.A.b
> ```
>
>
> The last call compains that global symbol "A.b" is not the same object
> as `A.b.__func__`.
>
> Everything would work if the `__func__` objects had the suffix
> ".__func__" in their qualname.
>
> Actually setting the qualname of the `__func__` object makes it
> picklable, but then the bound method object is unpicklable, as it
> inherits the name from the `__func__`. It would be good if they were
> separate.
>
> This is an attempt at emulating the desired behavior:
>
>
> ```
> from pickle import dumps
>
> class A:
> pass
>
> def c(cls):
> pass
>
> A.c = lambda: c(A)
> A.c.__qualname__ = "A.c"
> A.c.__name__ = "c"
> A.c.__self__ = A
> A.c.__func__ = c
> A.c.__func__.__qualname__ = "A.c.__func__"
>
> print(A.c) # <function A.c at 0x7f83ceabcbf8>
> print(A.c.__func__) # <function A.c.__func__ at 0x7f83ceabcb70>
>
> dumps(A.c) # works
> dumps(A.c.__func__) # works
> ```
>
>
> Can we make the `__func__` objects picklable this way?
>
>
> haael
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/N2J6MVLURTBAB5PJLJG3LO6U4PTPJMU2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>