Mailing List Archive: Restrict the type of __slots__

Restrict the type of __slots__

storchaka at gmail

Mar 18, 2022, 2:29 AM

Post #1 of 16 (1356 views)

Currently __slots__ can be either string or an iterable of strings.

1. If it is a string, it is a name of a single slot. Third-party code
which iterates __slots__ will be confused.

2. If it is an iterable, it should emit names of slots. Note that
non-reiterable iterators are accepted too, but it causes weird bugs if
__slots__ is iterated more than once. For example it breaks default
pickling and copying.

I propose to restrict the type of __slots__. Require it always been a
tuple of strings. Most __slots__ in real code are tuples. It is rarely
we need only single slot and set __slots__ as a string.

It will break some code (there are 2 occurrences in the stdlib an 1 in
scripts), but that code can be easily fixed.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

storchaka at gmail

Mar 18, 2022, 2:42 AM

Post #2 of 16 (1356 views)

18.03.22 11:29, Serhiy Storchaka ????:
> It will break some code (there are 2 occurrences in the stdlib an 1 in
> scripts), but that code can be easily fixed.

Actually it will break more code, because initializing __slots__ with a
list is pretty common. Maybe restrict it to tuple or list? But having an
immutable __slots__ may be more reliable.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HCZHJN2LSD2NXFLSFAO7VVOMEZRTLDBQ/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

python-dev at python

Mar 18, 2022, 2:51 AM

Post #3 of 16 (1356 views)

> On 18 Mar 2022, at 10:29, Serhiy Storchaka <storchaka@gmail.com> wrote:
>
> Currently __slots__ can be either string or an iterable of strings.
>
> 1. If it is a string, it is a name of a single slot. Third-party code which iterates __slots__ will be confused.
>
> 2. If it is an iterable, it should emit names of slots. Note that non-reiterable iterators are accepted too, but it causes weird bugs if __slots__ is iterated more than once. For example it breaks default pickling and copying.
>
> I propose to restrict the type of __slots__. Require it always been a tuple of strings. Most __slots__ in real code are tuples. It is rarely we need only single slot and set __slots__ as a string.
>
> It will break some code (there are 2 occurrences in the stdlib an 1 in scripts), but that code can be easily fixed.

Pydoc supports __slots__ that is a dict, and will use the values in the dict als documentation for the slots. I’ve also seen code using ``__slots__ = “field1 field2”.split()``. I don’t particularly like this code pattern, but your proposal would break this.

Also note that __slots__ only has a side effect during class definition, changing it afterwards is possible but has no effect (“class Foo: pass; Foo.__slots__ = 42”). This surprised my recently and I have no idea if this feature is ever used.

Ronald

>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
> Code of Conduct: http://python.org/psf/codeofconduct/

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

Re: Restrict the type of __slots__ [ In reply to ]

jsbueno at python

Mar 18, 2022, 6:37 AM

Post #4 of 16 (1356 views)

IMO this is a purism that have little, if any place in language
restrictions.
I see that not allowing. "run once" iterables could indeed void attempts to
write
"deliberatly non cooperative code" - but can it even be reliably detected?

The other changes seem just to break backwards compatibility for little or
no gain at all.

On Fri, Mar 18, 2022 at 6:57 AM Ronald Oussoren via Python-Dev <
python-dev@python.org> wrote:

>
>
> On 18 Mar 2022, at 10:29, Serhiy Storchaka <storchaka@gmail.com> wrote:
>
> Currently __slots__ can be either string or an iterable of strings.
>
> 1. If it is a string, it is a name of a single slot. Third-party code
> which iterates __slots__ will be confused.
>
> 2. If it is an iterable, it should emit names of slots. Note that
> non-reiterable iterators are accepted too, but it causes weird bugs if
> __slots__ is iterated more than once. For example it breaks default
> pickling and copying.
>
> I propose to restrict the type of __slots__. Require it always been a
> tuple of strings. Most __slots__ in real code are tuples. It is rarely we
> need only single slot and set __slots__ as a string.
>
> It will break some code (there are 2 occurrences in the stdlib an 1 in
> scripts), but that code can be easily fixed.
>
>
> Pydoc supports __slots__ that is a dict, and will use the values in the
> dict als documentation for the slots. I’ve also seen code using
> ``__slots__ = “field1 field2”.split()``. I don’t particularly like this
> code pattern, but your proposal would break this.
>
> Also note that __slots__ only has a side effect during class definition,
> changing it afterwards is possible but has no effect (“class Foo: pass;
> Foo.__slots__ = 42”). This surprised my recently and I have no idea if this
> feature is ever used.
>
> Ronald
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> —
>
> Twitter / micro.blog: @ronaldoussoren
> Blog: https://blog.ronaldoussoren.net/
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YQUWR7CYKNM65HR5FZQ3BANR5SNNK6N6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Re: Restrict the type of __slots__ [ In reply to ]

python-dev at python

Mar 18, 2022, 7:01 AM

Post #5 of 16 (1356 views)

> On 18 Mar 2022, at 14:37, Joao S. O. Bueno <jsbueno@python.org.br> wrote:

Please don’t toppost when responding to a normally threaded message. That makes it unnecessary hard to follow the conversation.

>
> IMO this is a purism that have little, if any place in language restrictions.
> I see that not allowing. "run once" iterables could indeed void attempts to write
> "deliberatly non cooperative code" - but can it even be reliably detected?
>
> The other changes seem just to break backwards compatibility for little or no gain at all.

It may not be worth the trouble to fix this, but Serhiy’s proposal does try to fix a ward.

It may be better to rely on linter’s here, but one way to do this with few backward compatibility concerns:

- if __slots__ is a dict keep it as is
- Otherwise use tuple(__slots__) while constructing the class and store that value in the __slots__ attribute of the class

That way the value of the attribute reflects the slots that were created while not breaking code that uses __slots__ and doesn’t change the value after class creation.

Ronald

>
>
>
> On Fri, Mar 18, 2022 at 6:57 AM Ronald Oussoren via Python-Dev <python-dev@python.org <mailto:python-dev@python.org>> wrote:
>
>
>> On 18 Mar 2022, at 10:29, Serhiy Storchaka <storchaka@gmail.com <mailto:storchaka@gmail.com>> wrote:
>>
>> Currently __slots__ can be either string or an iterable of strings.
>>
>> 1. If it is a string, it is a name of a single slot. Third-party code which iterates __slots__ will be confused.
>>
>> 2. If it is an iterable, it should emit names of slots. Note that non-reiterable iterators are accepted too, but it causes weird bugs if __slots__ is iterated more than once. For example it breaks default pickling and copying.
>>
>> I propose to restrict the type of __slots__. Require it always been a tuple of strings. Most __slots__ in real code are tuples. It is rarely we need only single slot and set __slots__ as a string.
>>
>> It will break some code (there are 2 occurrences in the stdlib an 1 in scripts), but that code can be easily fixed.
>
> Pydoc supports __slots__ that is a dict, and will use the values in the dict als documentation for the slots. I’ve also seen code using ``__slots__ = “field1 field2”.split()``. I don’t particularly like this code pattern, but your proposal would break this.
>
> Also note that __slots__ only has a side effect during class definition, changing it afterwards is possible but has no effect (“class Foo: pass; Foo.__slots__ = 42”). This surprised my recently and I have no idea if this feature is ever used.
>
> Ronald
>
>>
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org>
>> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org>
>> https://mail.python.org/mailman3/lists/python-dev.python.org/ <https://mail.python.org/mailman3/lists/python-dev.python.org/>
>> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/ <https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/>
>> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>
>
> —
>
> Twitter / micro.blog: @ronaldoussoren
> Blog: https://blog.ronaldoussoren.net/ <https://blog.ronaldoussoren.net/>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org>
> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org>
> https://mail.python.org/mailman3/lists/python-dev.python.org/ <https://mail.python.org/mailman3/lists/python-dev.python.org/>
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YQUWR7CYKNM65HR5FZQ3BANR5SNNK6N6/ <https://mail.python.org/archives/list/python-dev@python.org/message/YQUWR7CYKNM65HR5FZQ3BANR5SNNK6N6/>
> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

Re: Restrict the type of __slots__ [ In reply to ]

eric at trueblade

Mar 18, 2022, 7:38 AM

Post #6 of 16 (1356 views)

On 3/18/2022 10:01 AM, Ronald Oussoren via Python-Dev wrote:
>
>
>> On 18 Mar 2022, at 14:37, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
>>
>> IMO this is a purism that have little, if any place in language
>> restrictions.
>> I see that not allowing. "run once" iterables could indeed void
>> attempts to write
>> "deliberatly non cooperative code" - but can it even be reliably
>> detected?
>>
>> The other changes seem just to break backwards compatibility for
>> little or no gain at all.
There a practical need for these changes. See
https://bugs.python.org/issue46382 for a case where dataclasses needs
__slots__ to be iterated over after the class has been created. And it
would be good if __slots__ accurately reflected the slots that were
actually created.
>
> It may not be worth the trouble to fix this, but Serhiy’s proposal
> does try to fix a ward.
>
> It may be better to rely on linter’s here, but one way to do this with
> few backward compatibility concerns:
>
> - if __slots__ is a dict keep it as is
> - Otherwise use tuple(__slots__) while constructing the class and
> store that value in the __slots__ attribute of the class
>
> That way the value of the attribute reflects the slots that were
> created while not breaking code that uses __slots__ and doesn’t change
> the value after class creation.

I like this approach, too.

Eric

>
> Ronald
>
>>
>>
>>
>> On Fri, Mar 18, 2022 at 6:57 AM Ronald Oussoren via Python-Dev
>> <python-dev@python.org> wrote:
>>
>>
>>
>>> On 18 Mar 2022, at 10:29, Serhiy Storchaka <storchaka@gmail.com>
>>> wrote:
>>>
>>> Currently __slots__ can be either string or an iterable of strings.
>>>
>>> 1. If it is a string, it is a name of a single slot. Third-party
>>> code which iterates __slots__ will be confused.
>>>
>>> 2. If it is an iterable, it should emit names of slots. Note
>>> that non-reiterable iterators are accepted too, but it causes
>>> weird bugs if __slots__ is iterated more than once. For example
>>> it breaks default pickling and copying.
>>>
>>> I propose to restrict the type of __slots__. Require it always
>>> been a tuple of strings. Most __slots__ in real code are tuples.
>>> It is rarely we need only single slot and set __slots__ as a string.
>>>
>>> It will break some code (there are 2 occurrences in the stdlib
>>> an 1 in scripts), but that code can be easily fixed.
>>
>> Pydoc supports __slots__ that is a dict, and will use the values
>> in the dict als documentation for the slots. I’ve also seen
>> code using ``__slots__ = “field1 field2”.split()``. I don’t
>> particularly like this code pattern, but your proposal would
>> break this.
>>
>> Also note that __slots__ only has a side effect during class
>> definition, changing it afterwards is possible but has no effect
>> (“class Foo: pass; Foo.__slots__ = 42”). This surprised my
>> recently and I have no idea if this feature is ever used.
>>
>> Ronald
>>
>>>
>>> _______________________________________________
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-leave@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> —
>>
>> Twitter / micro.blog: @ronaldoussoren
>> Blog: https://blog.ronaldoussoren.net/
>>
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/YQUWR7CYKNM65HR5FZQ3BANR5SNNK6N6/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> —
>
> Twitter / micro.blog: @ronaldoussoren
> Blog: https://blog.ronaldoussoren.net/
>
>
> _______________________________________________
> Python-Dev mailing list --python-dev@python.org
> To unsubscribe send an email topython-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived athttps://mail.python.org/archives/list/python-dev@python.org/message/FZFRSHSJ3HQU37V6RFZNHMFGJXUPJ32X/
> Code of Conduct:http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

pbryan at anode

Mar 18, 2022, 8:58 AM

Post #7 of 16 (1356 views)

You've proposed a "what", but I don't see a "why".

Indeed, it will break some code.

I've been (currently legally) expressing __slots__ as sets, which is
arguably more consistent with its purpose, and in testing appeared to
perform better than tuples.

So, I would request that you amend the allowed types to include sets.

Also, people are using dicts to provide docstrings on attributes, so
that would need to be addressed as well.

On Fri, 2022-03-18 at 11:42 +0200, Serhiy Storchaka wrote:
> 18.03.22 11:29, Serhiy Storchaka ????:
> > It will break some code (there are 2 occurrences in the stdlib an 1
> > in
> > scripts), but that code can be easily fixed.
>
> Actually it will break more code, because initializing __slots__ with
> a
> list is pretty common. Maybe restrict it to tuple or list? But having
> an
> immutable __slots__ may be more reliable.
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HCZHJN2LSD2NXFLSFAO7VVOMEZRTLDBQ/
> Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

guido at python

Mar 18, 2022, 9:35 AM

Post #8 of 16 (1356 views)

The motivation has been explained already.

What on earth did your test do that got a speedup by using sets? Was it
repeatedly checking whether a variable was in a slot? The other thing this
does is rearrange the order in which slots appear from run to run (since
set order is affected by hash randomization) and you might have gotten
lucky with a popular attribute being moved to the front, where it's more
likely to be in the memory cache already (cache lines being 64 bytes and
pointers being 8 bytes nowadays).

I agree that dicts are a use case to preserve.

On Fri, Mar 18, 2022 at 08:59 Paul Bryan <pbryan@anode.ca> wrote:

> You've proposed a "what", but I don't see a "why".
>
> Indeed, it will break some code.
>
> I've been (currently legally) expressing __slots__ as sets, which is
> arguably more consistent with its purpose, and in testing appeared to
> perform better than tuples.
>
> So, I would request that you amend the allowed types to include sets.
>
> Also, people are using dicts to provide docstrings on attributes, so that
> would need to be addressed as well.
>
> On Fri, 2022-03-18 at 11:42 +0200, Serhiy Storchaka wrote:
>
> 18.03.22 11:29, Serhiy Storchaka ????:
>
> It will break some code (there are 2 occurrences in the stdlib an 1 in
> scripts), but that code can be easily fixed.
>
>
> Actually it will break more code, because initializing __slots__ with a
> list is pretty common. Maybe restrict it to tuple or list? But having an
> immutable __slots__ may be more reliable.
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HCZHJN2LSD2NXFLSFAO7VVOMEZRTLDBQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/6AKHUOVUQ2SQRPE4HAGWAEW2VYRKPFCL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Re: Restrict the type of __slots__ [ In reply to ]

pbryan at anode

Mar 18, 2022, 9:41 AM

Post #9 of 16 (1356 views)

On Fri, 2022-03-18 at 09:35 -0700, Guido van Rossum wrote:
> The motivation has been explained already.

In this thread?

> What on earth did your test do that got a speedup by using sets? Was
> it repeatedly checking whether a variable was in a slot? The other
> thing this does is rearrange the order in which slots appear from run
> to run (since set order is affected by hash randomization) and you
> might have gotten lucky with a popular attribute being moved to the
> front, where it's more likely to be in the memory cache already
> (cache lines being 64 bytes and pointers being 8 bytes nowadays).

I created objects in a tight loop, populating attributes, noting the
elapsed time.

> I agree that dicts are a use case to preserve.
>
> On Fri, Mar 18, 2022 at 08:59 Paul Bryan <pbryan@anode.ca> wrote:
> > You've proposed a "what", but I don't see a "why".
> >
> > Indeed, it will break some code.
> >
> > I've been (currently legally) expressing __slots__ as sets, which
> > is arguably more consistent with its purpose, and in testing
> > appeared to perform better than tuples.
> >
> > So, I would request that you amend the allowed types to include
> > sets.
> >
> > Also, people are using dicts to provide docstrings on attributes,
> > so that would need to be addressed as well.
> >
> > On Fri, 2022-03-18 at 11:42 +0200, Serhiy Storchaka wrote:
> > > 18.03.22 11:29, Serhiy Storchaka ????:
> > > > It will break some code (there are 2 occurrences in the stdlib
> > > > an 1 in
> > > > scripts), but that code can be easily fixed.
> > >
> > > Actually it will break more code, because initializing __slots__
> > > with a
> > > list is pretty common. Maybe restrict it to tuple or list? But
> > > having an
> > > immutable __slots__ may be more reliable.
> > >
> > > _______________________________________________
> > > Python-Dev mailing list -- python-dev@python.org
> > > To unsubscribe send an email to python-dev-leave@python.org
> > > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > > Message archived at
> > > https://mail.python.org/archives/list/python-dev@python.org/message/HCZHJN2LSD2NXFLSFAO7VVOMEZRTLDBQ/
> > > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> > https://mail.python.org/archives/list/python-dev@python.org/message/6AKHUOVUQ2SQRPE4HAGWAEW2VYRKPFCL/
> > Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

raymond.hettinger at gmail

Mar 18, 2022, 10:07 AM

Post #10 of 16 (1356 views)

> I propose to restrict the type of __slots__.

-1 for adding a restriction. This breaks code for no good reason. This
API has been around for a very long time. I've seen lists, tuples, dicts,
single strings, and occasionally something more exotic. Why wreck stable
code?

Also, the inspect module will detect whether __slots__ is a dictionary and
will use it to display docstrings. In the database world, data
dictionaries have proven value, so it would be a bummer to kill off this
functionality which is used in much the same way as docstrings for
properties. It is still rarely used, but I'm hoping it will catch on (just
like people are slowly growing more aware that they can add docstringa to
fields in named tuples).

Raymond

On Fri, Mar 18, 2022 at 4:33 AM Serhiy Storchaka <storchaka@gmail.com>
wrote:

> Currently __slots__ can be either string or an iterable of strings.
>
> 1. If it is a string, it is a name of a single slot. Third-party code
> which iterates __slots__ will be confused.
>
> 2. If it is an iterable, it should emit names of slots. Note that
> non-reiterable iterators are accepted too, but it causes weird bugs if
> __slots__ is iterated more than once. For example it breaks default
> pickling and copying.
>
> I propose to restrict the type of __slots__. Require it always been a
> tuple of strings. Most __slots__ in real code are tuples. It is rarely
> we need only single slot and set __slots__ as a string.
>
> It will break some code (there are 2 occurrences in the stdlib an 1 in
> scripts), but that code can be easily fixed.
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/E32BRLAWOU5GESMZ5MLAOIYPXSL37HOI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Re: Restrict the type of __slots__ [ In reply to ]

steve.dower at python

Mar 18, 2022, 10:14 AM

Post #11 of 16 (1356 views)

On 3/18/2022 2:38 PM, Eric V. Smith wrote:
> There a practical need for these changes. See
> https://bugs.python.org/issue46382 for a case where dataclasses needs
> __slots__ to be iterated over after the class has been created. And it
> would be good if __slots__ accurately reflected the slots that were
> actually created.

It seems totally sufficient for dataclasses to require that base class
__slots__ are a string or iterables of strings. Which is apparently what
would happen today, since anything else would cause incorrect behaviour.

A single special case doesn't have to become a language-wide backwards
compatibility break, even if it perhaps would've been better to have had
the restriction from the start.

Cheers,
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/R5PBNN36BMUM72QPRJG4DAHSD6UKUY3O/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

guido at python

Mar 18, 2022, 1:58 PM

Post #12 of 16 (1355 views)

On Fri, Mar 18, 2022 at 9:40 AM Paul Bryan <pbryan@anode.ca> wrote:

> On Fri, 2022-03-18 at 09:35 -0700, Guido van Rossum wrote:
>
> The motivation has been explained already.
>
>
> In this thread?
>

Yes, Eric's message.

> What on earth did your test do that got a speedup by using sets? Was it
> repeatedly checking whether a variable was in a slot? The other thing this
> does is rearrange the order in which slots appear from run to run (since
> set order is affected by hash randomization) and you might have gotten
> lucky with a popular attribute being moved to the front, where it's more
> likely to be in the memory cache already (cache lines being 64 bytes and
> pointers being 8 bytes nowadays).
>
>
> I created objects in a tight loop, populating attributes, noting the
> elapsed time.
>

It does not make sense that that is correlated to the type of __slots__,
since __slots__ is not used for instance creation at all (it is only used
to create the class). I stick to my likely explanation.

Regarding Serhiy's proposal, I'm +0 on disallowing strings, and +0 on
disallowing things that can't be reiterated (but it's not a problem in
practice). Given other responses the status quo is likely best.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

Re: Restrict the type of __slots__ [ In reply to ]

alex.waygood at gmail

Mar 18, 2022, 2:12 PM

Post #13 of 16 (1355 views)

I'm +1 on disallowing strings specifically. The unexpected behaviour that ensues if you try to iterate over a class's slots is a footgun that I've trodden on a few times, and it's always been annoying.

But I don't see any particular reason to disallow any other kind of iterable, and I'm definitely -1 on disallowing using a dictionary for __slots__. Using a dictionary for __slots__ as a way to document attributes is a feature I very much like.

Best,
Alex

> On 18 Mar 2022, at 20:59, Guido van Rossum <guido@python.org> wrote:
>
> ?
>> On Fri, Mar 18, 2022 at 9:40 AM Paul Bryan <pbryan@anode.ca> wrote:
>>> On Fri, 2022-03-18 at 09:35 -0700, Guido van Rossum wrote:
>>> The motivation has been explained already.
>>
>> In this thread?
>
> Yes, Eric's message.
>
>>> What on earth did your test do that got a speedup by using sets? Was it repeatedly checking whether a variable was in a slot? The other thing this does is rearrange the order in which slots appear from run to run (since set order is affected by hash randomization) and you might have gotten lucky with a popular attribute being moved to the front, where it's more likely to be in the memory cache already (cache lines being 64 bytes and pointers being 8 bytes nowadays).
>>
>> I created objects in a tight loop, populating attributes, noting the elapsed time.
>
> It does not make sense that that is correlated to the type of __slots__, since __slots__ is not used for instance creation at all (it is only used to create the class). I stick to my likely explanation.
>
> Regarding Serhiy's proposal, I'm +0 on disallowing strings, and +0 on disallowing things that can't be reiterated (but it's not a problem in practice). Given other responses the status quo is likely best.
>
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EYZ32GBU4QUTRPU26BDCYYMKG4GX633N/
> Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

eric at trueblade

Mar 18, 2022, 2:29 PM

Post #14 of 16 (1355 views)

On 3/18/2022 4:58 PM, Guido van Rossum wrote:
> On Fri, Mar 18, 2022 at 9:40 AM Paul Bryan <pbryan@anode.ca> wrote:
>
> On Fri, 2022-03-18 at 09:35 -0700, Guido van Rossum wrote:
>> The motivation has been explained already.
>
> In this thread?
>
>
> Yes, Eric's message.
>
>> What on earth did your test do that got a speedup by using sets?
>> Was it repeatedly checking whether a variable was in a slot? The
>> other thing this does is rearrange the order in which slots
>> appear from run to run (since set order is affected by hash
>> randomization) and you might have gotten lucky with a popular
>> attribute being moved to the front, where it's more likely to be
>> in the memory cache already (cache lines being 64 bytes and
>> pointers being 8 bytes nowadays).
>
> I created objects in a tight loop, populating attributes, noting
> the elapsed time.
>
>
> It does not make sense that that is correlated to the type of
> __slots__, since __slots__ is not used for instance creation at all
> (it is only used to create the class). I stick to my likely explanation.
>
> Regarding Serhiy's proposal, I'm +0 on disallowing strings, and +0 on
> disallowing things that can't be reiterated (but it's not a problem in
> practice). Given other responses the status quo is likely best.

The PR for the issue I mentioned (https://bugs.python.org/issue46382)
allows strings and iterables (but not iterators). And it uses a match
statement, no less! I think that's good enough for dataclasses.

Eric

Re: Restrict the type of __slots__ [ In reply to ]

eryksun at gmail

Mar 19, 2022, 9:48 AM

Post #15 of 16 (1355 views)

On 3/18/22, Ronald Oussoren via Python-Dev <python-dev@python.org> wrote:
>
> - if __slots__ is a dict keep it as is
> - Otherwise use tuple(__slots__) while constructing the class and store that
> value in the __slots__ attribute of the class

If this is just for pydoc, then it can be updated with new behavior.
For example, if the given __slots__ is a dict, set it as something
like __slots_doc__, and rewrite __slots__ as a tuple of the keys.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y6RDNJZGO4FN3DCNTWJM7YA7WFLKIXDV/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Restrict the type of __slots__ [ In reply to ]

eryksun at gmail

Mar 19, 2022, 10:50 AM

Post #16 of 16 (1355 views)

On 3/19/22, Eryk Sun <eryksun@gmail.com> wrote:
> On 3/18/22, Ronald Oussoren via Python-Dev <python-dev@python.org> wrote:
>>
>> - if __slots__ is a dict keep it as is
>> - Otherwise use tuple(__slots__) while constructing the class and store
>> that
>> value in the __slots__ attribute of the class
>
> If this is just for pydoc, then it can be updated with new behavior.
> For example, if the given __slots__ is a dict, set it as something
> like __slots_doc__, and rewrite __slots__ as a tuple of the keys.

Or extend pydoc to support a mappingproxy for __slots__.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NPIZQIZ5OPYU6V7YIGK5TZ75Z6WIOK5O/
Code of Conduct: http://python.org/psf/codeofconduct/