Mailing List Archive

On 1/8/21 4:31 PM, Mats Wichmann wrote:
>
>
> Someone reported a testsuite break on stuff I work on (scons) with
> 3.10a4, and it looks similar to this which appears in the changelog at
>
> https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>
> bpo-23898: Fix inspect.classify_class_attrs() to support attributes with
> overloaded __eq__ and __bool__. Patch by Mike Bayer.
>
> Except when I go look at that BPO issue, it's old - closed 2015.  Is
> this appearing in the 3.10 changelog in error?  Sorry - confused !!!

okay, that was silly, I didn't realize the changelog was cumulative over
many versions, so that entry was not for 3.10 at all (teach me to do
searching in browser window, where it just flies right past any section
headings so I miss it was for a different version :) ).

> The test in question does indeed touch a class which overrides __bool_
> in order to raise an exception (to say "don't do that"), and in the test
> run the (expected) exception is not raised.

So updated information: the test in question is checking if a class (A)
has an attribute using a truth test, where the attribute's value is an
instance of another class (B) and expecting that that will cause the
__bool__ method to be called. [.aside: this test is done to validate that
a class which really doesn't want this kind of test indeed rejects it]
That apparently no longer happens, if it's wrapped in a try block ???
Distilled down to simple case:

class A:
pass

class B:
def __bool__(self):
raise AttributeError("don't do that!")

a = A()
b = B()
a.b = b
# expect this to cause b.__bool__ to be called
if a.b:
print("Found it!")

and it raises the exception. But when protected:

try:
if a.b:
pass
except AttributeError:
print("Got expected exception")
else:
print("Missed expected exception")

it won't trigger. But if I add a "real" statement in the block following
the "if", then it's back to the pre-3.10 behavior of calling __bool__:

try:
if a.b:
dummy = True
except AttributeError:
print("Got expected exception")
else:
print("Missed expected exception")


Any thoughts on this?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VC6VRTL7LKE4PVFQBYJW4HYJX6D6TJVM/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>
>
> On 1/8/21 4:31 PM, Mats Wichmann wrote:
> >
> >
> > Someone reported a testsuite break on stuff I work on (scons) with
> > 3.10a4, and it looks similar to this which appears in the changelog at
> >
> > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
> >
> > bpo-23898: Fix inspect.classify_class_attrs() to support attributes with
> > overloaded __eq__ and __bool__. Patch by Mike Bayer.
> >
> > Except when I go look at that BPO issue, it's old - closed 2015. Is
> > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>
> okay, that was silly, I didn't realize the changelog was cumulative over
> many versions, so that entry was not for 3.10 at all (teach me to do
> searching in browser window, where it just flies right past any section
> headings so I miss it was for a different version :) ).
>
> > The test in question does indeed touch a class which overrides __bool_
> > in order to raise an exception (to say "don't do that"), and in the test
> > run the (expected) exception is not raised.
>
> So updated information: the test in question is checking if a class (A)
> has an attribute using a truth test, where the attribute's value is an
> instance of another class (B) and expecting that that will cause the
> __bool__ method to be called. [.aside: this test is done to validate that
> a class which really doesn't want this kind of test indeed rejects it]
> That apparently no longer happens, if it's wrapped in a try block ???
> Distilled down to simple case:
>
> class A:
> pass
>
> class B:
> def __bool__(self):
> raise AttributeError("don't do that!")
>
> a = A()
> b = B()
> a.b = b
> # expect this to cause b.__bool__ to be called
> if a.b:
> print("Found it!")
>
> and it raises the exception. But when protected:
>
> try:
> if a.b:
> pass
> except AttributeError:
> print("Got expected exception")
> else:
> print("Missed expected exception")
>
> it won't trigger. But if I add a "real" statement in the block following
> the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>
> try:
> if a.b:
> dummy = True
> except AttributeError:
> print("Got expected exception")
> else:
> print("Missed expected exception")
>
>
> Any thoughts on this?

Oooh interesting. I tried on a build of 3.10 from October and:
1) The unguarded version bombed out with an exception
2) The "if... pass" version reported that it got the exception
3) The "if... dummy" version reported that it got the exception

ie every one of them did indeed raise. But on a fresh build from the
master branch, I got the same results you did. That means the change
happened some time between commit 497126f7ea and commit ace008c531, an
800ish commit span.

I'll start bisecting to try to track this down. It looks like "if a.b:
pass" is getting partially optimized out; the disassembly shows a
being loaded, its attribute b being looked up, and then it just jumps
to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
bit of actual code in there.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
Code of Conduct: http://python.org/psf/codeofconduct/

This may be related to the changes in https://bugs.python.org/issue42246.
Could you open a new issue and add Mark Shannon to it if that turns to be
the case?

Pablo

On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
> >
> >
> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
> > >
> > >
> > > Someone reported a testsuite break on stuff I work on (scons) with
> > > 3.10a4, and it looks similar to this which appears in the changelog at
> > >
> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
> > >
> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
> with
> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
> > >
> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
> >
> > okay, that was silly, I didn't realize the changelog was cumulative over
> > many versions, so that entry was not for 3.10 at all (teach me to do
> > searching in browser window, where it just flies right past any section
> > headings so I miss it was for a different version :) ).
> >
> > > The test in question does indeed touch a class which overrides __bool_
> > > in order to raise an exception (to say "don't do that"), and in the
> test
> > > run the (expected) exception is not raised.
> >
> > So updated information: the test in question is checking if a class (A)
> > has an attribute using a truth test, where the attribute's value is an
> > instance of another class (B) and expecting that that will cause the
> > __bool__ method to be called. [.aside: this test is done to validate that
> > a class which really doesn't want this kind of test indeed rejects it]
> > That apparently no longer happens, if it's wrapped in a try block ???
> > Distilled down to simple case:
> >
> > class A:
> > pass
> >
> > class B:
> > def __bool__(self):
> > raise AttributeError("don't do that!")
> >
> > a = A()
> > b = B()
> > a.b = b
> > # expect this to cause b.__bool__ to be called
> > if a.b:
> > print("Found it!")
> >
> > and it raises the exception. But when protected:
> >
> > try:
> > if a.b:
> > pass
> > except AttributeError:
> > print("Got expected exception")
> > else:
> > print("Missed expected exception")
> >
> > it won't trigger. But if I add a "real" statement in the block following
> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
> >
> > try:
> > if a.b:
> > dummy = True
> > except AttributeError:
> > print("Got expected exception")
> > else:
> > print("Missed expected exception")
> >
> >
> > Any thoughts on this?
>
> Oooh interesting. I tried on a build of 3.10 from October and:
> 1) The unguarded version bombed out with an exception
> 2) The "if... pass" version reported that it got the exception
> 3) The "if... dummy" version reported that it got the exception
>
> ie every one of them did indeed raise. But on a fresh build from the
> master branch, I got the same results you did. That means the change
> happened some time between commit 497126f7ea and commit ace008c531, an
> 800ish commit span.
>
> I'll start bisecting to try to track this down. It looks like "if a.b:
> pass" is getting partially optimized out; the disassembly shows a
> being loaded, its attribute b being looked up, and then it just jumps
> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
> bit of actual code in there.
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

All that said (I agree it's surprising that 3.10 seems backwards
incompatible here) I would personally not raise AttributeError but
TypeError in the `__bool__()` method.

On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado <pablogsal@gmail.com>
wrote:

> This may be related to the changes in https://bugs.python.org/issue42246.
> Could you open a new issue and add Mark Shannon to it if that turns to be
> the case?
>
> Pablo
>
> On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:
>
>> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>> >
>> >
>> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
>> > >
>> > >
>> > > Someone reported a testsuite break on stuff I work on (scons) with
>> > > 3.10a4, and it looks similar to this which appears in the changelog at
>> > >
>> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>> > >
>> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
>> with
>> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
>> > >
>> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
>> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>> >
>> > okay, that was silly, I didn't realize the changelog was cumulative over
>> > many versions, so that entry was not for 3.10 at all (teach me to do
>> > searching in browser window, where it just flies right past any section
>> > headings so I miss it was for a different version :) ).
>> >
>> > > The test in question does indeed touch a class which overrides __bool_
>> > > in order to raise an exception (to say "don't do that"), and in the
>> test
>> > > run the (expected) exception is not raised.
>> >
>> > So updated information: the test in question is checking if a class (A)
>> > has an attribute using a truth test, where the attribute's value is an
>> > instance of another class (B) and expecting that that will cause the
>> > __bool__ method to be called. [.aside: this test is done to validate that
>> > a class which really doesn't want this kind of test indeed rejects it]
>> > That apparently no longer happens, if it's wrapped in a try block ???
>> > Distilled down to simple case:
>> >
>> > class A:
>> > pass
>> >
>> > class B:
>> > def __bool__(self):
>> > raise AttributeError("don't do that!")
>> >
>> > a = A()
>> > b = B()
>> > a.b = b
>> > # expect this to cause b.__bool__ to be called
>> > if a.b:
>> > print("Found it!")
>> >
>> > and it raises the exception. But when protected:
>> >
>> > try:
>> > if a.b:
>> > pass
>> > except AttributeError:
>> > print("Got expected exception")
>> > else:
>> > print("Missed expected exception")
>> >
>> > it won't trigger. But if I add a "real" statement in the block following
>> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>> >
>> > try:
>> > if a.b:
>> > dummy = True
>> > except AttributeError:
>> > print("Got expected exception")
>> > else:
>> > print("Missed expected exception")
>> >
>> >
>> > Any thoughts on this?
>>
>> Oooh interesting. I tried on a build of 3.10 from October and:
>> 1) The unguarded version bombed out with an exception
>> 2) The "if... pass" version reported that it got the exception
>> 3) The "if... dummy" version reported that it got the exception
>>
>> ie every one of them did indeed raise. But on a fresh build from the
>> master branch, I got the same results you did. That means the change
>> happened some time between commit 497126f7ea and commit ace008c531, an
>> 800ish commit span.
>>
>> I'll start bisecting to try to track this down. It looks like "if a.b:
>> pass" is getting partially optimized out; the disassembly shows a
>> being loaded, its attribute b being looked up, and then it just jumps
>> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
>> bit of actual code in there.
>>
>> ChrisA
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7DMQXD3EF6CIAUIHFORMXEOUDZGUO2YW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

I've raised: https://bugs.python.org/issue42899.

The triggering commit was: c71581c7a4192e6ba9a79eccc583aaadab300efa
bpo-42615: Delete redundant jump instructions that only bypass empty blocks
(GH-23733)




On Mon, Jan 11, 2021 at 8:04 PM Guido van Rossum <guido@python.org> wrote:

> All that said (I agree it's surprising that 3.10 seems backwards
> incompatible here) I would personally not raise AttributeError but
> TypeError in the `__bool__()` method.
>
> On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado <
> pablogsal@gmail.com> wrote:
>
>> This may be related to the changes in https://bugs.python.org/issue42246.
>> Could you open a new issue and add Mark Shannon to it if that turns to be
>> the case?
>>
>> Pablo
>>
>> On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:
>>
>>> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>>> >
>>> >
>>> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
>>> > >
>>> > >
>>> > > Someone reported a testsuite break on stuff I work on (scons) with
>>> > > 3.10a4, and it looks similar to this which appears in the changelog
>>> at
>>> > >
>>> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>>> > >
>>> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
>>> with
>>> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
>>> > >
>>> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
>>> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>>> >
>>> > okay, that was silly, I didn't realize the changelog was cumulative
>>> over
>>> > many versions, so that entry was not for 3.10 at all (teach me to do
>>> > searching in browser window, where it just flies right past any section
>>> > headings so I miss it was for a different version :) ).
>>> >
>>> > > The test in question does indeed touch a class which overrides
>>> __bool_
>>> > > in order to raise an exception (to say "don't do that"), and in the
>>> test
>>> > > run the (expected) exception is not raised.
>>> >
>>> > So updated information: the test in question is checking if a class (A)
>>> > has an attribute using a truth test, where the attribute's value is an
>>> > instance of another class (B) and expecting that that will cause the
>>> > __bool__ method to be called. [aside: this test is done to validate
>>> that
>>> > a class which really doesn't want this kind of test indeed rejects it]
>>> > That apparently no longer happens, if it's wrapped in a try block ???
>>> > Distilled down to simple case:
>>> >
>>> > class A:
>>> > pass
>>> >
>>> > class B:
>>> > def __bool__(self):
>>> > raise AttributeError("don't do that!")
>>> >
>>> > a = A()
>>> > b = B()
>>> > a.b = b
>>> > # expect this to cause b.__bool__ to be called
>>> > if a.b:
>>> > print("Found it!")
>>> >
>>> > and it raises the exception. But when protected:
>>> >
>>> > try:
>>> > if a.b:
>>> > pass
>>> > except AttributeError:
>>> > print("Got expected exception")
>>> > else:
>>> > print("Missed expected exception")
>>> >
>>> > it won't trigger. But if I add a "real" statement in the block
>>> following
>>> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>>> >
>>> > try:
>>> > if a.b:
>>> > dummy = True
>>> > except AttributeError:
>>> > print("Got expected exception")
>>> > else:
>>> > print("Missed expected exception")
>>> >
>>> >
>>> > Any thoughts on this?
>>>
>>> Oooh interesting. I tried on a build of 3.10 from October and:
>>> 1) The unguarded version bombed out with an exception
>>> 2) The "if... pass" version reported that it got the exception
>>> 3) The "if... dummy" version reported that it got the exception
>>>
>>> ie every one of them did indeed raise. But on a fresh build from the
>>> master branch, I got the same results you did. That means the change
>>> happened some time between commit 497126f7ea and commit ace008c531, an
>>> 800ish commit span.
>>>
>>> I'll start bisecting to try to track this down. It looks like "if a.b:
>>> pass" is getting partially optimized out; the disassembly shows a
>>> being loaded, its attribute b being looked up, and then it just jumps
>>> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
>>> bit of actual code in there.
>>>
>>> ChrisA
>>> _______________________________________________
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-leave@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/7DMQXD3EF6CIAUIHFORMXEOUDZGUO2YW/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VQ5FDBBCCF7FB3D2AH4J2LN6WQWJSBAJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On 1/11/21 1:00 PM, Guido van Rossum wrote:
> All that said (I agree it's surprising that 3.10 seems backwards
> incompatible here) I would personally not raise AttributeError but
> TypeError in the `__bool__()` method.

eh, that was just me picking a cheap something to demo it. the program
raises an application-specific error that I didn't feel like defining to
keep the repro as short as possible.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
Code of Conduct: http://python.org/psf/codeofconduct/

Ah never mind. Seems to be a real bug -- thanks for reporting!

On Mon, Jan 11, 2021 at 2:57 PM Mats Wichmann <mats@wichmann.us> wrote:

> On 1/11/21 1:00 PM, Guido van Rossum wrote:
> > All that said (I agree it's surprising that 3.10 seems backwards
> > incompatible here) I would personally not raise AttributeError but
> > TypeError in the `__bool__()` method.
>
> eh, that was just me picking a cheap something to demo it. the program
> raises an application-specific error that I didn't feel like defining to
> keep the repro as short as possible.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

Hi everyone,

Should the optimizer eliminate tests that it can prove have no effect on
the control flow of the program, even if that may eliminate some side
effects in __bool__()?

For several years we have converted

if a and b:
...

to

if a:
if b:
...

which are equivalent, unless bool(a) has side effects the second time it
is called.

In master we convert `if x: pass` to `pass` which is equivalent, unless
bool(x) has side effects the first time it is called. This is a recent
change.

This is one of those "easy to fix, if we can decide on the semantics" bugs.


Submit your thoughts to https://bugs.python.org/issue42899, please.

Cheers,
Mark.


On 12/01/2021 12:45 am, Guido van Rossum wrote:
> Ah never mind. Seems to be a real bug -- thanks for reporting!
>
> On Mon, Jan 11, 2021 at 2:57 PM Mats Wichmann <mats@wichmann.us
> <mailto:mats@wichmann.us>> wrote:
>
> On 1/11/21 1:00 PM, Guido van Rossum wrote:
> > All that said (I agree it's surprising that 3.10 seems backwards
> > incompatible here) I would personally not raise AttributeError but
> > TypeError in the `__bool__()` method.
>
> eh, that was just me picking a cheap something to demo it.  the program
> raises an application-specific error that I didn't feel like
> defining to
> keep the repro as short as possible.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> <mailto:python-dev@python.org>
> To unsubscribe send an email to python-dev-leave@python.org
> <mailto:python-dev-leave@python.org>
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> /Pronouns: he/him //(why is my pronoun here?)/
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BK4IUDXCZDDQCRSX3QGY7XUHOKMIDPG4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DFX7HMZ7RFUQJMJI7MABHKEK4EOYHR4A/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 10:53 AM, Mark Shannon wrote:
> Hi everyone,
>
> Should the optimizer eliminate tests that it can prove have no effect
> on the control flow of the program, even if that may eliminate some
> side effects in __bool__()?
>
> For several years we have converted
>
>     if a and b:
>         ...
>
> to
>
>     if a:
>         if b:
>             ...
>
> which are equivalent, unless bool(a) has side effects the second time
> it is called.
>
> In master we convert `if x: pass` to `pass` which is equivalent,
> unless bool(x) has side effects the first time it is called. This is a
> recent change.
>
> This is one of those "easy to fix, if we can decide on the semantics"
> bugs.
>
>
> Submit your thoughts to https://bugs.python.org/issue42899, please.
>
> Cheers,
> Mark.

One key point about 'and' and 'or' is that those operators are defined
to be 'short circuiting', i.e. that  with a and b, that if a is false,
then b is not evaluated at all. This can be important if a is a
condition that test if the expression b is 'safe' to evaluate. In fact,
isn't it true that 'a and b' is defined to be the equivalent to:  'a if
not a else b' so b doesn't need to be evaluated unless a is truthy.

I know that I would be very surpised if a statement like


if foo():

    pass

ended up optimizing itself and not calling foo(), which is only one step
away from the quoted case of

if b:

    pass

which is the equivalent to

if b.__bool__():

    pass


Yes, perhaps there is more of an expectation that __bool__() is
innocuous, but is that a an assumption that should be baked into the
language.

--
Richard Damon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HODNO42SFFRFX4IJY5K562YHT2MTIHIQ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 4:24 AM Richard Damon <Richard@damon-family.org> wrote:
>
> On 1/12/21 10:53 AM, Mark Shannon wrote:
> > Hi everyone,
> >
> > Should the optimizer eliminate tests that it can prove have no effect
> > on the control flow of the program, even if that may eliminate some
> > side effects in __bool__()?
> >
> > For several years we have converted
> >
> > if a and b:
> > ...
> >
> > to
> >
> > if a:
> > if b:
> > ...
> >
> > which are equivalent, unless bool(a) has side effects the second time
> > it is called.
> >
> > In master we convert `if x: pass` to `pass` which is equivalent,
> > unless bool(x) has side effects the first time it is called. This is a
> > recent change.
> >
> > This is one of those "easy to fix, if we can decide on the semantics"
> > bugs.
> >
> >
> > Submit your thoughts to https://bugs.python.org/issue42899, please.
> >
> > Cheers,
> > Mark.
>
> One key point about 'and' and 'or' is that those operators are defined
> to be 'short circuiting', i.e. that with a and b, that if a is false,
> then b is not evaluated at all. This can be important if a is a
> condition that test if the expression b is 'safe' to evaluate. In fact,
> isn't it true that 'a and b' is defined to be the equivalent to: 'a if
> not a else b' so b doesn't need to be evaluated unless a is truthy.

Yes, the shortcircuiting behaviour isn't in question. But consider:

class A:
def __bool__(self):
print("A().__bool__")
return False

def f():
print("if A() and A()")
if A() and A(): x = 1
print("cond = A() and A()")
cond = A() and A()
if cond: x = 1

f()

And once you've run the code, disassemble the function for extra insight.

There's a very definite difference here, and it's the same difference as:

for x in thing:

and

for x in iter(thing):

I'd be fine with documenting that __bool__ is (guaranteed to be)
called only if the interpreter needs to know the result, leaving open
the option for things like this to be optimized out. That'd leave open
the option for "foo() if x else foo()" to be optimized down to just
"foo()", although I don't think that particular one is needed.

> I know that I would be very surpised if a statement like
>
>
> if foo():
>
> pass
>
> ended up optimizing itself and not calling foo(), which is only one step
> away from the quoted case of
>
> if b:
>
> pass
>
> which is the equivalent to
>
> if b.__bool__():
>
> pass

Yes, they do look similar. The difference is that calling __bool__ is
under the interpreter's control, and it's easier to document that it
be assumed to be side-effect-free.

> Yes, perhaps there is more of an expectation that __bool__() is
> innocuous, but is that a an assumption that should be baked into the
> language.
>

I think so. Consider that sometimes other dunders won't be called, if
the interpreter believes it's not necessary:

class A(float):
def __index__(self):
print("A().__index__")
return 10

class B(int):
def __index__(self):
print("B().__index__")
return 10

print(range(20)[A():B()])

If it's a subclass of float, slicing will call __index__, but if it's
a subclass of int, Python knows already that it can use the internal
integer value.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GHKDF6YE3D43JPWS7GVG34FVPJNYE5SO/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Jan 12, 2021 at 9:51 AM Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Jan 13, 2021 at 4:24 AM Richard Damon <Richard@damon-family.org>
> wrote:
> >
> > On 1/12/21 10:53 AM, Mark Shannon wrote:
> > > Hi everyone,
> > >
> > > Should the optimizer eliminate tests that it can prove have no effect
> > > on the control flow of the program, even if that may eliminate some
> > > side effects in __bool__()?
> > >
> > > For several years we have converted
> > >
> > > if a and b:
> > > ...
> > >
> > > to
> > >
> > > if a:
> > > if b:
> > > ...
> > >
> > > which are equivalent, unless bool(a) has side effects the second time
> > > it is called.
> > >
> > > In master we convert `if x: pass` to `pass` which is equivalent,
> > > unless bool(x) has side effects the first time it is called. This is a
> > > recent change.
> > >
> > > This is one of those "easy to fix, if we can decide on the semantics"
> > > bugs.
> > >
> > >
> > > Submit your thoughts to https://bugs.python.org/issue42899, please.
> > >
> > > Cheers,
> > > Mark.
> >
> > One key point about 'and' and 'or' is that those operators are defined
> > to be 'short circuiting', i.e. that with a and b, that if a is false,
> > then b is not evaluated at all. This can be important if a is a
> > condition that test if the expression b is 'safe' to evaluate. In fact,
> > isn't it true that 'a and b' is defined to be the equivalent to: 'a if
> > not a else b' so b doesn't need to be evaluated unless a is truthy.
>

https://snarky.ca/unravelling-boolean-operations/ if you want the gory
details.


>
> Yes, the shortcircuiting behaviour isn't in question. But consider:
>
> class A:
> def __bool__(self):
> print("A().__bool__")
> return False
>
> def f():
> print("if A() and A()")
> if A() and A(): x = 1
> print("cond = A() and A()")
> cond = A() and A()
> if cond: x = 1
>
> f()
>
> And once you've run the code, disassemble the function for extra insight.
>
> There's a very definite difference here, and it's the same difference as:
>
> for x in thing:
>
> and
>
> for x in iter(thing):
>
> I'd be fine with documenting that __bool__ is (guaranteed to be)
> called only if the interpreter needs to know the result, leaving open
> the option for things like this to be optimized out. That'd leave open
> the option for "foo() if x else foo()" to be optimized down to just
> "foo()", although I don't think that particular one is needed.
>

I think that's the key question here: it is a language change, but is it
one we want (specifically in this case and in general)? It will require a
language reference change at least, and as it has been shown, some people
don't expect Python to take shortcuts in execution knowing full well that
CPython dutifully executes things. So saying we only call __bool__() *if
necessary* is definitely a change. But then this begs the question of
whether we want to do this more widely in the language in the name of
optimization, or not in the name of consistency that a user can easily
reason about.


>
> > I know that I would be very surpised if a statement like
> >
> >
> > if foo():
> >
> > pass
> >
> > ended up optimizing itself and not calling foo(), which is only one step
> > away from the quoted case of
> >
> > if b:
> >
> > pass
> >
> > which is the equivalent to
> >
> > if b.__bool__():
> >
> > pass
>
> Yes, they do look similar. The difference is that calling __bool__ is
> under the interpreter's control, and it's easier to document that it
> be assumed to be side-effect-free.
>
> > Yes, perhaps there is more of an expectation that __bool__() is
> > innocuous, but is that a an assumption that should be baked into the
> > language.
> >
>
> I think so. Consider that sometimes other dunders won't be called, if
> the interpreter believes it's not necessary:
>
> class A(float):
> def __index__(self):
> print("A().__index__")
> return 10
>
> class B(int):
> def __index__(self):
> print("B().__index__")
> return 10
>
> print(range(20)[A():B()])
>
> If it's a subclass of float, slicing will call __index__, but if it's
> a subclass of int, Python knows already that it can use the internal
> integer value.
>

https://snarky.ca/unravelling-not-in-python/ ???? But basically,
https://docs.python.org/3.8/reference/datamodel.html#object.__index__ says
"called *if *conversion to an int is necessary" which isn't the case when
something is already an int.


>
> ChrisA
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GHKDF6YE3D43JPWS7GVG34FVPJNYE5SO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:

> That'd leave open
> the option for "foo() if x else foo()" to be optimized down to just
> "foo()", although I don't think that particular one is needed.

That would be an unsafe optimization. Not all objets are representable
as truthy/falsey values, e.g. numpy arrays.

>>> bool(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element
is ambiguous. Use a.any() or a.all()

Not even all builtin values. This is in Python 3.9:

>>> bool(NotImplemented)
<stdin>:1: DeprecationWarning: NotImplemented should not be used in
a boolean context
True

I believe that 3.10 makes it an error. If not 3.10, then it will surely
happen soon. But even without the change to NotImplemented, it has never
been the case that *every* object is guaranteed to be either truthy or
falsey. At least not since the Python 1.x `__nonzero__` dunder was put
into the language.



--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7Q76EALFDEI2S6L3NFXSJS4TNNEWMO6L/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano <steve@pearwood.info> wrote:
>
> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
>
> > That'd leave open
> > the option for "foo() if x else foo()" to be optimized down to just
> > "foo()", although I don't think that particular one is needed.
>
> That would be an unsafe optimization. Not all objets are representable
> as truthy/falsey values, e.g. numpy arrays.
>
> >>> bool(a)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ValueError: The truth value of an array with more than one element
> is ambiguous. Use a.any() or a.all()
>
> Not even all builtin values. This is in Python 3.9:
>
> >>> bool(NotImplemented)
> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
> a boolean context
> True
>
> I believe that 3.10 makes it an error. If not 3.10, then it will surely
> happen soon. But even without the change to NotImplemented, it has never
> been the case that *every* object is guaranteed to be either truthy or
> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
> into the language.
>

But this is exactly where we started: with a boolification that fails,
in a context where the result wouldn't actually change anything.
Actually calling bool() on something will continue to have the
behaviour you're describing, but if the truthiness or falsiness would
make no difference, is the interpreter required to find out?

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IMG437FNV552XUWOQJW76A7SFAJQULEH/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 10:37 PM, Chris Angelico wrote:
> On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote:
>>
>> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
>>
>>> That'd leave open
>>> the option for "foo() if x else foo()" to be optimized down to just
>>> "foo()", although I don't think that particular one is needed.
>>
>> That would be an unsafe optimization. Not all objets are representable
>> as truthy/falsey values

>> Not even all builtin values. This is in Python 3.9:
>>
>> >>> bool(NotImplemented)
>> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
>> a boolean context
>> True
>>
>> I believe that 3.10 makes it an error. If not 3.10, then it will surely
>> happen soon. But even without the change to NotImplemented, it has never
>> been the case that *every* object is guaranteed to be either truthy or
>> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
>> into the language.
>
> But this is exactly where we started: with a boolification that fails,
> in a context where the result wouldn't actually change anything.
> Actually calling bool() on something will continue to have the
> behaviour you're describing, but if the truthiness or falsiness would
> make no difference, is the interpreter required to find out?

Yes.

Optimizations are an implementation detail, and implementation details should not change the language.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JFOAJSXWH5KO32YVBN2LKZCPWAFMG63G/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman <ethan@stoneleaf.us> wrote:
>
> On 1/12/21 10:37 PM, Chris Angelico wrote:
> > On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote:
> >>
> >> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
> >>
> >>> That'd leave open
> >>> the option for "foo() if x else foo()" to be optimized down to just
> >>> "foo()", although I don't think that particular one is needed.
> >>
> >> That would be an unsafe optimization. Not all objets are representable
> >> as truthy/falsey values
>
> >> Not even all builtin values. This is in Python 3.9:
> >>
> >> >>> bool(NotImplemented)
> >> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
> >> a boolean context
> >> True
> >>
> >> I believe that 3.10 makes it an error. If not 3.10, then it will surely
> >> happen soon. But even without the change to NotImplemented, it has never
> >> been the case that *every* object is guaranteed to be either truthy or
> >> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
> >> into the language.
> >
> > But this is exactly where we started: with a boolification that fails,
> > in a context where the result wouldn't actually change anything.
> > Actually calling bool() on something will continue to have the
> > behaviour you're describing, but if the truthiness or falsiness would
> > make no difference, is the interpreter required to find out?
>
> Yes.
>
> Optimizations are an implementation detail, and implementation details should not change the language.
>

The language can also be defined in an optimization-friendly way,
though. Consider how we got positional-only arguments in Python: first
they existed in C-implemented functions in CPython, even though they
couldn't exist in pure Python code, and then the functionality got
added to the language definition, thus permitting the optimization.

Or consider dictionary lookup. Most people treat it as "find a key
which is equal to the one you're looking for", but the actual
definition is "find a key which is identical to, or equal to, the one
you're looking for". That's partly to ensure that weird cases like NaN
don't break too badly, but also it means that x.__eq__(x) doesn't have
to be called all the time.

The topic under discussion is a language definition. Choosing to
permit the optimization doesn't mean that the implementation detail
changes the language. Choosing to deny it means there won't be an
optimization.

I personally don't see any reason to force Python to calculate
something unnecessarily, given that this is *already* happening in
other situations (see the "if a and b:" optimization, which doesn't
boolify twice).

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SAUNDT7XTTFWREABUXPE2OKWI6KBQECO/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Wed, 13 Jan 2021 18:27:07 +1100
Chris Angelico <rosuav@gmail.com> wrote:

[]

> > Optimizations are an implementation detail, and implementation
> > details should not change the language.
>
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

In this case, the culprit seems to be that __bool__() and many other
"special" methods may have side effects. So, just as "const" annotation
would be useful for variables, "pure" (side-effect free) annotation
would be useful for functions.

But that alone won't help due to too-dynamic nature of Python. It's not
much of an optimization if, at runtime, at each call-site, you need to
check whether a function is pure to decide if you have to call it, or
may skip it. Instead, that rather be done at compile time.

But checking which method (pure or impure) will be called when is again
complicated statically in Python. What to do about that? Well, we can
introduce, ahem, some "strict" mode, in which *all* __bool__(),
__gt__(), and friends must be declared pure, and for good measure,
returning bool. Then we know we can skip any such method call.

You see, all heresy starts from small things...

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 11:27 PM, Chris Angelico wrote:
> On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman wrote:

>> Optimizations are an implementation detail, and implementation details should not change the language.
>>
>
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

1. What optimization?
2. Did the language change because of the optimization?

> Or consider dictionary lookup. Most people treat it as "find a key
> which is equal to the one you're looking for", but the actual
> definition is "find a key which is identical to, or equal to, the one
> you're looking for".

Exactly. The definition, i.e. language spec, says identity, then equality.

> The topic under discussion is a language definition. Choosing to
> permit the optimization doesn't mean that the implementation detail
> changes the language. Choosing to deny it means there won't be an
> optimization.

There are, I am sure, many optimizations that are not possible because of Python's dynamism. `if <something>` is
supposed to evaluate `bool(something)`, regardless of what comes after.

> I personally don't see any reason to force Python to calculate
> something unnecessarily, given that this is *already* happening in
> other situations (see the "if a and b:" optimization, which doesn't
> boolify twice).

Sure, and I agree that calling `bool()` a second time is wasteful, as well as possibly confusing -- Python already has
the answer from the first `bool()` call, so why would it need to do it again? That seems a matter of correctness, not
optimization.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MNP25T5WJ5YIVIOJJFTU5QSO5ME6BUED/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 8:02 PM Ethan Furman <ethan@stoneleaf.us> wrote:
>
> On 1/12/21 11:27 PM, Chris Angelico wrote:
> > On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman wrote:
>
> >> Optimizations are an implementation detail, and implementation details should not change the language.
> >>
> >
> > The language can also be defined in an optimization-friendly way,
> > though. Consider how we got positional-only arguments in Python: first
> > they existed in C-implemented functions in CPython, even though they
> > couldn't exist in pure Python code, and then the functionality got
> > added to the language definition, thus permitting the optimization.
>
> 1. What optimization?
> 2. Did the language change because of the optimization?

It's a lot faster for C-implemented functions to require positional
parameters. A number of functions had help text that implied that they
accepted keyword-or-positional args, but if you tried to call them
with keyword args, they'd error out.

And yes. PEP 457 and then PEP 570 introduced real positional-only
arguments, and part of the debate surrounded the question "should we
require that all C-implemented functions support keyword args?". Why
have a performance hit on all C functions just for the sake of an
arbitrary consistency? Instead, the language definition was changed to
make this possible - and now we have, for instance, "len(obj, /)",
which clearly does not accept len(obj="x"), but does accept len("x").

> > Or consider dictionary lookup. Most people treat it as "find a key
> > which is equal to the one you're looking for", but the actual
> > definition is "find a key which is identical to, or equal to, the one
> > you're looking for".
>
> Exactly. The definition, i.e. language spec, says identity, then equality.

Right. The definition is set to permit the optimization. Therefore the
optimization cannot be in violation of the spec.

> > The topic under discussion is a language definition. Choosing to
> > permit the optimization doesn't mean that the implementation detail
> > changes the language. Choosing to deny it means there won't be an
> > optimization.
>
> There are, I am sure, many optimizations that are not possible because of Python's dynamism. `if <something>` is
> supposed to evaluate `bool(something)`, regardless of what comes after.

It doesn't, though. There is a difference between:

if a and b: ...

and

x = a and b
if x: ...

The first one will boolify a only once; the second will figure out
what bool(a) is before the assignment, and then (if it's false)
boolify it again in the 'if' statement.

Current builds of Python 3.10 optimize this away too:

if a or 1: print(1)

Regardless of the truthiness of a, the print call will happen.

(Note that a is still *evaluated*. It's just not queried for
truthiness. For instance, "if f() or 1:" will still call the function;
it just won't call __bool__() on the return value.)

> > I personally don't see any reason to force Python to calculate
> > something unnecessarily, given that this is *already* happening in
> > other situations (see the "if a and b:" optimization, which doesn't
> > boolify twice).
>
> Sure, and I agree that calling `bool()` a second time is wasteful, as well as possibly confusing -- Python already has
> the answer from the first `bool()` call, so why would it need to do it again? That seems a matter of correctness, not
> optimization.
>

Is it? It means that there's a visible difference between putting it
in a variable and doing it inline. If it's okay to do that, then why
is it not okay to remove the bool check when it can't affect the
result?

In theory, "x = (expr)" followed by "if x:" should perform the exact
same checks as "if (expr):"; if it's okay to violate that principle
for "if a and b:", then why not for "if a: pass"? Either way, the
interpreter knows that any sane __bool__ function cannot affect the
result.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RK36KL2EH5I54W76INZ64OBFJ5BBY5L7/
Code of Conduct: http://python.org/psf/codeofconduct/

Even if you define __bool__() as returning a bool, and error/undefined
behavior otherwise, that doesn't eliminate side effects. Is it even
possible to nail down a definition to the point that you can say, "Thou
shalt not mutate or cause anything" and have it meaningfully enforced in
the compiler or interpreter?

-Em

On Wed, Jan 13, 2021 at 12:18 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:

> Hello,
>
> On Wed, 13 Jan 2021 18:27:07 +1100
> Chris Angelico <rosuav@gmail.com> wrote:
>
> []
>
> > > Optimizations are an implementation detail, and implementation
> > > details should not change the language.
> >
> > The language can also be defined in an optimization-friendly way,
> > though. Consider how we got positional-only arguments in Python: first
> > they existed in C-implemented functions in CPython, even though they
> > couldn't exist in pure Python code, and then the functionality got
> > added to the language definition, thus permitting the optimization.
>
> In this case, the culprit seems to be that __bool__() and many other
> "special" methods may have side effects. So, just as "const" annotation
> would be useful for variables, "pure" (side-effect free) annotation
> would be useful for functions.
>
> But that alone won't help due to too-dynamic nature of Python. It's not
> much of an optimization if, at runtime, at each call-site, you need to
> check whether a function is pure to decide if you have to call it, or
> may skip it. Instead, that rather be done at compile time.
>
> But checking which method (pure or impure) will be called when is again
> complicated statically in Python. What to do about that? Well, we can
> introduce, ahem, some "strict" mode, in which *all* __bool__(),
> __gt__(), and friends must be declared pure, and for good measure,
> returning bool. Then we know we can skip any such method call.
>
> You see, all heresy starts from small things...
>
> []
>
> --
> Best regards,
> Paul mailto:pmiscml@gmail.com
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Wed, Jan 13, 2021 at 9:08 PM Emily Bowman <silverbacknet@gmail.com> wrote:
>
> Even if you define __bool__() as returning a bool, and error/undefined behavior otherwise, that doesn't eliminate side effects. Is it even possible to nail down a definition to the point that you can say, "Thou shalt not mutate or cause anything" and have it meaningfully enforced in the compiler or interpreter?
>

Yes - just say "will be called when the interpreter needs to know the
booleanness of the object", and don't mandate that it be called if the
interpreter already knows. If someone writes a __bool__ function that
decrements a counter and returns True if it's still above zero, then
they're shooting themselves in the foot, and we don't have to protect
them.

It's like having __iter__ return an object on which __iter__ doesn't
return self. There's no protection in the language to stop you from
creating such a monster, but you'll run into problems sooner or later
if you do, and it's not a bug in Python.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RN7SAZ3X4YM2KEHNVMAXDNPLQFWVY7NJ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 11:27 PM, Chris Angelico wrote:
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

I wouldn't describe positional-only parameters as an "optimization". 
They were created to add expressiveness to Python code, not to make
Python code faster.  The classic example is the dict constructor:
technically, you couldn't implement it correctly in pure Python code,
because it had a parameter (the iterable) that could not be a named
parameter.  Any name you gave it would preclude passing in that name as
a name=value argument.

Parsing positional-only parameters is often faster than parsing named
parameters, for obvious reasons.  But this is really small stuff,
drowned out by almost any other variation in your code.


Cheers,


//arry/

Hello,

On Wed, 13 Jan 2021 02:08:01 -0800
Emily Bowman <silverbacknet@gmail.com> wrote:

> Even if you define __bool__() as returning a bool, and
> error/undefined behavior otherwise, that doesn't eliminate side
> effects. Is it even possible to nail down a definition to the point
> that you can say, "Thou shalt not mutate or cause anything" and have
> it meaningfully enforced in the compiler or interpreter?

Yes, sure, "pure" annotation on a function would (ideally/eventually)
need "typechecking". We could start with simple rules like: a) a pure
function should not mutate any objects which it didn't create; b) for
"mutation" also counts passing objects to (other) non-pure functions.
The hardest part is again to know which of these other functions is pure
or impure. Runtime checking should not be the aim to shoot for. There
should be a way to preserve as much as possible of Python's dynamic
nature that we all love, yet avoid runtime lookups.

The above "purity" conditions are also too restrictive, so would need
to be extended/elaborated (which would certainly require more detailed
annotations; fortunately, we all already accepted annotations to be a
part of Python's nature, so wanting to use them more shouldn't come as
surprise).

>
> -Em
>
> On Wed, Jan 13, 2021 at 12:18 AM Paul Sokolovsky <pmiscml@gmail.com>
> wrote:
>
> > Hello,
> >
> > On Wed, 13 Jan 2021 18:27:07 +1100
> > Chris Angelico <rosuav@gmail.com> wrote:
> >
> > []
> >
> > > > Optimizations are an implementation detail, and implementation
> > > > details should not change the language.
> > >
> > > The language can also be defined in an optimization-friendly way,
> > > though. Consider how we got positional-only arguments in Python:
> > > first they existed in C-implemented functions in CPython, even
> > > though they couldn't exist in pure Python code, and then the
> > > functionality got added to the language definition, thus
> > > permitting the optimization.
> >
> > In this case, the culprit seems to be that __bool__() and many other
> > "special" methods may have side effects. So, just as "const"
> > annotation would be useful for variables, "pure" (side-effect free)
> > annotation would be useful for functions.
> >
> > But that alone won't help due to too-dynamic nature of Python. It's
> > not much of an optimization if, at runtime, at each call-site, you
> > need to check whether a function is pure to decide if you have to
> > call it, or may skip it. Instead, that rather be done at compile
> > time.
> >
> > But checking which method (pure or impure) will be called when is
> > again complicated statically in Python. What to do about that?
> > Well, we can introduce, ahem, some "strict" mode, in which *all*
> > __bool__(), __gt__(), and friends must be declared pure, and for
> > good measure, returning bool. Then we know we can skip any such
> > method call.
> >
> > You see, all heresy starts from small things...
> >
> > []
> >
> > --
> > Best regards,
> > Paul mailto:pmiscml@gmail.com
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> > https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >



--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TCG7237FJILE4MVKBXSTTQEIR6CX7VFJ/
Code of Conduct: http://python.org/psf/codeofconduct/

Hi Rob,

On 13/01/2021 12:18 pm, Rob Cliffe wrote:
>
>
> On 12/01/2021 15:53, Mark Shannon wrote:
>>
>> In master we convert `if x: pass` to `pass` which is equivalent,
>> unless bool(x) has side effects the first time it is called. This is a
>> recent change.
> Can you please confirm that this optimisation ONLY applies to bare
> names, i.e. NOT
>
>     if x(): pass
>     if x.y: pass
>     if x+y: pass


The optimization doesn't apply to the expression, but the test.
The optimizer (might) transform

if x+y: pass

to

x+y

But the expression is still evaluated.
Sorry for the confusion, I should have been clearer in my example.
It is the call to `bool()` that *might* be eliminated.

Cheers,
Mark.



>
> Thanks
> Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YZMA3ABVUKJ3PDFO4BS56QRSL4YCC3DJ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 12/01/2021 15:53, Mark Shannon wrote:
> Hi everyone,
>
>
>
> In master we convert `if x: pass` to `pass` which is equivalent,
> unless bool(x) has side effects the first time it is called. This is a
> recent change.
>
Suppose x is not a currently valid variable name at runtime.  Will the
NameError still be "optimised" away?
Thanks
Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JK4QENGPZV56RNGI5YHLWCL4OF4AEGSP/
Code of Conduct: http://python.org/psf/codeofconduct/