Mailing List Archive

On 1/8/21 4:31 PM, Mats Wichmann wrote:
>
>
> Someone reported a testsuite break on stuff I work on (scons) with
> 3.10a4, and it looks similar to this which appears in the changelog at
>
> https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>
> bpo-23898: Fix inspect.classify_class_attrs() to support attributes with
> overloaded __eq__ and __bool__. Patch by Mike Bayer.
>
> Except when I go look at that BPO issue, it's old - closed 2015.  Is
> this appearing in the 3.10 changelog in error?  Sorry - confused !!!

okay, that was silly, I didn't realize the changelog was cumulative over
many versions, so that entry was not for 3.10 at all (teach me to do
searching in browser window, where it just flies right past any section
headings so I miss it was for a different version :) ).

> The test in question does indeed touch a class which overrides __bool_
> in order to raise an exception (to say "don't do that"), and in the test
> run the (expected) exception is not raised.

So updated information: the test in question is checking if a class (A)
has an attribute using a truth test, where the attribute's value is an
instance of another class (B) and expecting that that will cause the
__bool__ method to be called. [.aside: this test is done to validate that
a class which really doesn't want this kind of test indeed rejects it]
That apparently no longer happens, if it's wrapped in a try block ???
Distilled down to simple case:

class A:
pass

class B:
def __bool__(self):
raise AttributeError("don't do that!")

a = A()
b = B()
a.b = b
# expect this to cause b.__bool__ to be called
if a.b:
print("Found it!")

and it raises the exception. But when protected:

try:
if a.b:
pass
except AttributeError:
print("Got expected exception")
else:
print("Missed expected exception")

it won't trigger. But if I add a "real" statement in the block following
the "if", then it's back to the pre-3.10 behavior of calling __bool__:

try:
if a.b:
dummy = True
except AttributeError:
print("Got expected exception")
else:
print("Missed expected exception")


Any thoughts on this?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VC6VRTL7LKE4PVFQBYJW4HYJX6D6TJVM/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>
>
> On 1/8/21 4:31 PM, Mats Wichmann wrote:
> >
> >
> > Someone reported a testsuite break on stuff I work on (scons) with
> > 3.10a4, and it looks similar to this which appears in the changelog at
> >
> > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
> >
> > bpo-23898: Fix inspect.classify_class_attrs() to support attributes with
> > overloaded __eq__ and __bool__. Patch by Mike Bayer.
> >
> > Except when I go look at that BPO issue, it's old - closed 2015. Is
> > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>
> okay, that was silly, I didn't realize the changelog was cumulative over
> many versions, so that entry was not for 3.10 at all (teach me to do
> searching in browser window, where it just flies right past any section
> headings so I miss it was for a different version :) ).
>
> > The test in question does indeed touch a class which overrides __bool_
> > in order to raise an exception (to say "don't do that"), and in the test
> > run the (expected) exception is not raised.
>
> So updated information: the test in question is checking if a class (A)
> has an attribute using a truth test, where the attribute's value is an
> instance of another class (B) and expecting that that will cause the
> __bool__ method to be called. [.aside: this test is done to validate that
> a class which really doesn't want this kind of test indeed rejects it]
> That apparently no longer happens, if it's wrapped in a try block ???
> Distilled down to simple case:
>
> class A:
> pass
>
> class B:
> def __bool__(self):
> raise AttributeError("don't do that!")
>
> a = A()
> b = B()
> a.b = b
> # expect this to cause b.__bool__ to be called
> if a.b:
> print("Found it!")
>
> and it raises the exception. But when protected:
>
> try:
> if a.b:
> pass
> except AttributeError:
> print("Got expected exception")
> else:
> print("Missed expected exception")
>
> it won't trigger. But if I add a "real" statement in the block following
> the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>
> try:
> if a.b:
> dummy = True
> except AttributeError:
> print("Got expected exception")
> else:
> print("Missed expected exception")
>
>
> Any thoughts on this?

Oooh interesting. I tried on a build of 3.10 from October and:
1) The unguarded version bombed out with an exception
2) The "if... pass" version reported that it got the exception
3) The "if... dummy" version reported that it got the exception

ie every one of them did indeed raise. But on a fresh build from the
master branch, I got the same results you did. That means the change
happened some time between commit 497126f7ea and commit ace008c531, an
800ish commit span.

I'll start bisecting to try to track this down. It looks like "if a.b:
pass" is getting partially optimized out; the disassembly shows a
being loaded, its attribute b being looked up, and then it just jumps
to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
bit of actual code in there.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
Code of Conduct: http://python.org/psf/codeofconduct/

This may be related to the changes in https://bugs.python.org/issue42246.
Could you open a new issue and add Mark Shannon to it if that turns to be
the case?

Pablo

On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
> >
> >
> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
> > >
> > >
> > > Someone reported a testsuite break on stuff I work on (scons) with
> > > 3.10a4, and it looks similar to this which appears in the changelog at
> > >
> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
> > >
> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
> with
> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
> > >
> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
> >
> > okay, that was silly, I didn't realize the changelog was cumulative over
> > many versions, so that entry was not for 3.10 at all (teach me to do
> > searching in browser window, where it just flies right past any section
> > headings so I miss it was for a different version :) ).
> >
> > > The test in question does indeed touch a class which overrides __bool_
> > > in order to raise an exception (to say "don't do that"), and in the
> test
> > > run the (expected) exception is not raised.
> >
> > So updated information: the test in question is checking if a class (A)
> > has an attribute using a truth test, where the attribute's value is an
> > instance of another class (B) and expecting that that will cause the
> > __bool__ method to be called. [.aside: this test is done to validate that
> > a class which really doesn't want this kind of test indeed rejects it]
> > That apparently no longer happens, if it's wrapped in a try block ???
> > Distilled down to simple case:
> >
> > class A:
> > pass
> >
> > class B:
> > def __bool__(self):
> > raise AttributeError("don't do that!")
> >
> > a = A()
> > b = B()
> > a.b = b
> > # expect this to cause b.__bool__ to be called
> > if a.b:
> > print("Found it!")
> >
> > and it raises the exception. But when protected:
> >
> > try:
> > if a.b:
> > pass
> > except AttributeError:
> > print("Got expected exception")
> > else:
> > print("Missed expected exception")
> >
> > it won't trigger. But if I add a "real" statement in the block following
> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
> >
> > try:
> > if a.b:
> > dummy = True
> > except AttributeError:
> > print("Got expected exception")
> > else:
> > print("Missed expected exception")
> >
> >
> > Any thoughts on this?
>
> Oooh interesting. I tried on a build of 3.10 from October and:
> 1) The unguarded version bombed out with an exception
> 2) The "if... pass" version reported that it got the exception
> 3) The "if... dummy" version reported that it got the exception
>
> ie every one of them did indeed raise. But on a fresh build from the
> master branch, I got the same results you did. That means the change
> happened some time between commit 497126f7ea and commit ace008c531, an
> 800ish commit span.
>
> I'll start bisecting to try to track this down. It looks like "if a.b:
> pass" is getting partially optimized out; the disassembly shows a
> being loaded, its attribute b being looked up, and then it just jumps
> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
> bit of actual code in there.
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

All that said (I agree it's surprising that 3.10 seems backwards
incompatible here) I would personally not raise AttributeError but
TypeError in the `__bool__()` method.

On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado <pablogsal@gmail.com>
wrote:

> This may be related to the changes in https://bugs.python.org/issue42246.
> Could you open a new issue and add Mark Shannon to it if that turns to be
> the case?
>
> Pablo
>
> On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:
>
>> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>> >
>> >
>> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
>> > >
>> > >
>> > > Someone reported a testsuite break on stuff I work on (scons) with
>> > > 3.10a4, and it looks similar to this which appears in the changelog at
>> > >
>> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>> > >
>> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
>> with
>> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
>> > >
>> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
>> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>> >
>> > okay, that was silly, I didn't realize the changelog was cumulative over
>> > many versions, so that entry was not for 3.10 at all (teach me to do
>> > searching in browser window, where it just flies right past any section
>> > headings so I miss it was for a different version :) ).
>> >
>> > > The test in question does indeed touch a class which overrides __bool_
>> > > in order to raise an exception (to say "don't do that"), and in the
>> test
>> > > run the (expected) exception is not raised.
>> >
>> > So updated information: the test in question is checking if a class (A)
>> > has an attribute using a truth test, where the attribute's value is an
>> > instance of another class (B) and expecting that that will cause the
>> > __bool__ method to be called. [.aside: this test is done to validate that
>> > a class which really doesn't want this kind of test indeed rejects it]
>> > That apparently no longer happens, if it's wrapped in a try block ???
>> > Distilled down to simple case:
>> >
>> > class A:
>> > pass
>> >
>> > class B:
>> > def __bool__(self):
>> > raise AttributeError("don't do that!")
>> >
>> > a = A()
>> > b = B()
>> > a.b = b
>> > # expect this to cause b.__bool__ to be called
>> > if a.b:
>> > print("Found it!")
>> >
>> > and it raises the exception. But when protected:
>> >
>> > try:
>> > if a.b:
>> > pass
>> > except AttributeError:
>> > print("Got expected exception")
>> > else:
>> > print("Missed expected exception")
>> >
>> > it won't trigger. But if I add a "real" statement in the block following
>> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>> >
>> > try:
>> > if a.b:
>> > dummy = True
>> > except AttributeError:
>> > print("Got expected exception")
>> > else:
>> > print("Missed expected exception")
>> >
>> >
>> > Any thoughts on this?
>>
>> Oooh interesting. I tried on a build of 3.10 from October and:
>> 1) The unguarded version bombed out with an exception
>> 2) The "if... pass" version reported that it got the exception
>> 3) The "if... dummy" version reported that it got the exception
>>
>> ie every one of them did indeed raise. But on a fresh build from the
>> master branch, I got the same results you did. That means the change
>> happened some time between commit 497126f7ea and commit ace008c531, an
>> 800ish commit span.
>>
>> I'll start bisecting to try to track this down. It looks like "if a.b:
>> pass" is getting partially optimized out; the disassembly shows a
>> being loaded, its attribute b being looked up, and then it just jumps
>> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
>> bit of actual code in there.
>>
>> ChrisA
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7DMQXD3EF6CIAUIHFORMXEOUDZGUO2YW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

I've raised: https://bugs.python.org/issue42899.

The triggering commit was: c71581c7a4192e6ba9a79eccc583aaadab300efa
bpo-42615: Delete redundant jump instructions that only bypass empty blocks
(GH-23733)




On Mon, Jan 11, 2021 at 8:04 PM Guido van Rossum <guido@python.org> wrote:

> All that said (I agree it's surprising that 3.10 seems backwards
> incompatible here) I would personally not raise AttributeError but
> TypeError in the `__bool__()` method.
>
> On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado <
> pablogsal@gmail.com> wrote:
>
>> This may be related to the changes in https://bugs.python.org/issue42246.
>> Could you open a new issue and add Mark Shannon to it if that turns to be
>> the case?
>>
>> Pablo
>>
>> On Mon, 11 Jan 2021 at 19:36, Chris Angelico <rosuav@gmail.com> wrote:
>>
>>> On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann <mats@wichmann.us> wrote:
>>> >
>>> >
>>> > On 1/8/21 4:31 PM, Mats Wichmann wrote:
>>> > >
>>> > >
>>> > > Someone reported a testsuite break on stuff I work on (scons) with
>>> > > 3.10a4, and it looks similar to this which appears in the changelog
>>> at
>>> > >
>>> > > https://docs.python.org/3.10/whatsnew/changelog.html#changelog
>>> > >
>>> > > bpo-23898: Fix inspect.classify_class_attrs() to support attributes
>>> with
>>> > > overloaded __eq__ and __bool__. Patch by Mike Bayer.
>>> > >
>>> > > Except when I go look at that BPO issue, it's old - closed 2015. Is
>>> > > this appearing in the 3.10 changelog in error? Sorry - confused !!!
>>> >
>>> > okay, that was silly, I didn't realize the changelog was cumulative
>>> over
>>> > many versions, so that entry was not for 3.10 at all (teach me to do
>>> > searching in browser window, where it just flies right past any section
>>> > headings so I miss it was for a different version :) ).
>>> >
>>> > > The test in question does indeed touch a class which overrides
>>> __bool_
>>> > > in order to raise an exception (to say "don't do that"), and in the
>>> test
>>> > > run the (expected) exception is not raised.
>>> >
>>> > So updated information: the test in question is checking if a class (A)
>>> > has an attribute using a truth test, where the attribute's value is an
>>> > instance of another class (B) and expecting that that will cause the
>>> > __bool__ method to be called. [aside: this test is done to validate
>>> that
>>> > a class which really doesn't want this kind of test indeed rejects it]
>>> > That apparently no longer happens, if it's wrapped in a try block ???
>>> > Distilled down to simple case:
>>> >
>>> > class A:
>>> > pass
>>> >
>>> > class B:
>>> > def __bool__(self):
>>> > raise AttributeError("don't do that!")
>>> >
>>> > a = A()
>>> > b = B()
>>> > a.b = b
>>> > # expect this to cause b.__bool__ to be called
>>> > if a.b:
>>> > print("Found it!")
>>> >
>>> > and it raises the exception. But when protected:
>>> >
>>> > try:
>>> > if a.b:
>>> > pass
>>> > except AttributeError:
>>> > print("Got expected exception")
>>> > else:
>>> > print("Missed expected exception")
>>> >
>>> > it won't trigger. But if I add a "real" statement in the block
>>> following
>>> > the "if", then it's back to the pre-3.10 behavior of calling __bool__:
>>> >
>>> > try:
>>> > if a.b:
>>> > dummy = True
>>> > except AttributeError:
>>> > print("Got expected exception")
>>> > else:
>>> > print("Missed expected exception")
>>> >
>>> >
>>> > Any thoughts on this?
>>>
>>> Oooh interesting. I tried on a build of 3.10 from October and:
>>> 1) The unguarded version bombed out with an exception
>>> 2) The "if... pass" version reported that it got the exception
>>> 3) The "if... dummy" version reported that it got the exception
>>>
>>> ie every one of them did indeed raise. But on a fresh build from the
>>> master branch, I got the same results you did. That means the change
>>> happened some time between commit 497126f7ea and commit ace008c531, an
>>> 800ish commit span.
>>>
>>> I'll start bisecting to try to track this down. It looks like "if a.b:
>>> pass" is getting partially optimized out; the disassembly shows a
>>> being loaded, its attribute b being looked up, and then it just jumps
>>> to the else - there's no POP_JUMP_IF_FALSE as there is when there's a
>>> bit of actual code in there.
>>>
>>> ChrisA
>>> _______________________________________________
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-leave@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/K2LD2L5RF2ZFUYEXQ3Z5U4TY5QBRFPCQ/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/7DMQXD3EF6CIAUIHFORMXEOUDZGUO2YW/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VQ5FDBBCCF7FB3D2AH4J2LN6WQWJSBAJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On 1/11/21 1:00 PM, Guido van Rossum wrote:
> All that said (I agree it's surprising that 3.10 seems backwards
> incompatible here) I would personally not raise AttributeError but
> TypeError in the `__bool__()` method.

eh, that was just me picking a cheap something to demo it. the program
raises an application-specific error that I didn't feel like defining to
keep the repro as short as possible.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
Code of Conduct: http://python.org/psf/codeofconduct/

Ah never mind. Seems to be a real bug -- thanks for reporting!

On Mon, Jan 11, 2021 at 2:57 PM Mats Wichmann <mats@wichmann.us> wrote:

> On 1/11/21 1:00 PM, Guido van Rossum wrote:
> > All that said (I agree it's surprising that 3.10 seems backwards
> > incompatible here) I would personally not raise AttributeError but
> > TypeError in the `__bool__()` method.
>
> eh, that was just me picking a cheap something to demo it. the program
> raises an application-specific error that I didn't feel like defining to
> keep the repro as short as possible.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

Hi everyone,

Should the optimizer eliminate tests that it can prove have no effect on
the control flow of the program, even if that may eliminate some side
effects in __bool__()?

For several years we have converted

if a and b:
...

to

if a:
if b:
...

which are equivalent, unless bool(a) has side effects the second time it
is called.

In master we convert `if x: pass` to `pass` which is equivalent, unless
bool(x) has side effects the first time it is called. This is a recent
change.

This is one of those "easy to fix, if we can decide on the semantics" bugs.


Submit your thoughts to https://bugs.python.org/issue42899, please.

Cheers,
Mark.


On 12/01/2021 12:45 am, Guido van Rossum wrote:
> Ah never mind. Seems to be a real bug -- thanks for reporting!
>
> On Mon, Jan 11, 2021 at 2:57 PM Mats Wichmann <mats@wichmann.us
> <mailto:mats@wichmann.us>> wrote:
>
> On 1/11/21 1:00 PM, Guido van Rossum wrote:
> > All that said (I agree it's surprising that 3.10 seems backwards
> > incompatible here) I would personally not raise AttributeError but
> > TypeError in the `__bool__()` method.
>
> eh, that was just me picking a cheap something to demo it.  the program
> raises an application-specific error that I didn't feel like
> defining to
> keep the repro as short as possible.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> <mailto:python-dev@python.org>
> To unsubscribe send an email to python-dev-leave@python.org
> <mailto:python-dev-leave@python.org>
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVGFN4DCDN462QVVMHY45IKH2XL4GVRD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> /Pronouns: he/him //(why is my pronoun here?)/
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BK4IUDXCZDDQCRSX3QGY7XUHOKMIDPG4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DFX7HMZ7RFUQJMJI7MABHKEK4EOYHR4A/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 10:53 AM, Mark Shannon wrote:
> Hi everyone,
>
> Should the optimizer eliminate tests that it can prove have no effect
> on the control flow of the program, even if that may eliminate some
> side effects in __bool__()?
>
> For several years we have converted
>
>     if a and b:
>         ...
>
> to
>
>     if a:
>         if b:
>             ...
>
> which are equivalent, unless bool(a) has side effects the second time
> it is called.
>
> In master we convert `if x: pass` to `pass` which is equivalent,
> unless bool(x) has side effects the first time it is called. This is a
> recent change.
>
> This is one of those "easy to fix, if we can decide on the semantics"
> bugs.
>
>
> Submit your thoughts to https://bugs.python.org/issue42899, please.
>
> Cheers,
> Mark.

One key point about 'and' and 'or' is that those operators are defined
to be 'short circuiting', i.e. that  with a and b, that if a is false,
then b is not evaluated at all. This can be important if a is a
condition that test if the expression b is 'safe' to evaluate. In fact,
isn't it true that 'a and b' is defined to be the equivalent to:  'a if
not a else b' so b doesn't need to be evaluated unless a is truthy.

I know that I would be very surpised if a statement like


if foo():

    pass

ended up optimizing itself and not calling foo(), which is only one step
away from the quoted case of

if b:

    pass

which is the equivalent to

if b.__bool__():

    pass


Yes, perhaps there is more of an expectation that __bool__() is
innocuous, but is that a an assumption that should be baked into the
language.

--
Richard Damon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HODNO42SFFRFX4IJY5K562YHT2MTIHIQ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 4:24 AM Richard Damon <Richard@damon-family.org> wrote:
>
> On 1/12/21 10:53 AM, Mark Shannon wrote:
> > Hi everyone,
> >
> > Should the optimizer eliminate tests that it can prove have no effect
> > on the control flow of the program, even if that may eliminate some
> > side effects in __bool__()?
> >
> > For several years we have converted
> >
> > if a and b:
> > ...
> >
> > to
> >
> > if a:
> > if b:
> > ...
> >
> > which are equivalent, unless bool(a) has side effects the second time
> > it is called.
> >
> > In master we convert `if x: pass` to `pass` which is equivalent,
> > unless bool(x) has side effects the first time it is called. This is a
> > recent change.
> >
> > This is one of those "easy to fix, if we can decide on the semantics"
> > bugs.
> >
> >
> > Submit your thoughts to https://bugs.python.org/issue42899, please.
> >
> > Cheers,
> > Mark.
>
> One key point about 'and' and 'or' is that those operators are defined
> to be 'short circuiting', i.e. that with a and b, that if a is false,
> then b is not evaluated at all. This can be important if a is a
> condition that test if the expression b is 'safe' to evaluate. In fact,
> isn't it true that 'a and b' is defined to be the equivalent to: 'a if
> not a else b' so b doesn't need to be evaluated unless a is truthy.

Yes, the shortcircuiting behaviour isn't in question. But consider:

class A:
def __bool__(self):
print("A().__bool__")
return False

def f():
print("if A() and A()")
if A() and A(): x = 1
print("cond = A() and A()")
cond = A() and A()
if cond: x = 1

f()

And once you've run the code, disassemble the function for extra insight.

There's a very definite difference here, and it's the same difference as:

for x in thing:

and

for x in iter(thing):

I'd be fine with documenting that __bool__ is (guaranteed to be)
called only if the interpreter needs to know the result, leaving open
the option for things like this to be optimized out. That'd leave open
the option for "foo() if x else foo()" to be optimized down to just
"foo()", although I don't think that particular one is needed.

> I know that I would be very surpised if a statement like
>
>
> if foo():
>
> pass
>
> ended up optimizing itself and not calling foo(), which is only one step
> away from the quoted case of
>
> if b:
>
> pass
>
> which is the equivalent to
>
> if b.__bool__():
>
> pass

Yes, they do look similar. The difference is that calling __bool__ is
under the interpreter's control, and it's easier to document that it
be assumed to be side-effect-free.

> Yes, perhaps there is more of an expectation that __bool__() is
> innocuous, but is that a an assumption that should be baked into the
> language.
>

I think so. Consider that sometimes other dunders won't be called, if
the interpreter believes it's not necessary:

class A(float):
def __index__(self):
print("A().__index__")
return 10

class B(int):
def __index__(self):
print("B().__index__")
return 10

print(range(20)[A():B()])

If it's a subclass of float, slicing will call __index__, but if it's
a subclass of int, Python knows already that it can use the internal
integer value.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GHKDF6YE3D43JPWS7GVG34FVPJNYE5SO/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Jan 12, 2021 at 9:51 AM Chris Angelico <rosuav@gmail.com> wrote:

> On Wed, Jan 13, 2021 at 4:24 AM Richard Damon <Richard@damon-family.org>
> wrote:
> >
> > On 1/12/21 10:53 AM, Mark Shannon wrote:
> > > Hi everyone,
> > >
> > > Should the optimizer eliminate tests that it can prove have no effect
> > > on the control flow of the program, even if that may eliminate some
> > > side effects in __bool__()?
> > >
> > > For several years we have converted
> > >
> > > if a and b:
> > > ...
> > >
> > > to
> > >
> > > if a:
> > > if b:
> > > ...
> > >
> > > which are equivalent, unless bool(a) has side effects the second time
> > > it is called.
> > >
> > > In master we convert `if x: pass` to `pass` which is equivalent,
> > > unless bool(x) has side effects the first time it is called. This is a
> > > recent change.
> > >
> > > This is one of those "easy to fix, if we can decide on the semantics"
> > > bugs.
> > >
> > >
> > > Submit your thoughts to https://bugs.python.org/issue42899, please.
> > >
> > > Cheers,
> > > Mark.
> >
> > One key point about 'and' and 'or' is that those operators are defined
> > to be 'short circuiting', i.e. that with a and b, that if a is false,
> > then b is not evaluated at all. This can be important if a is a
> > condition that test if the expression b is 'safe' to evaluate. In fact,
> > isn't it true that 'a and b' is defined to be the equivalent to: 'a if
> > not a else b' so b doesn't need to be evaluated unless a is truthy.
>

https://snarky.ca/unravelling-boolean-operations/ if you want the gory
details.


>
> Yes, the shortcircuiting behaviour isn't in question. But consider:
>
> class A:
> def __bool__(self):
> print("A().__bool__")
> return False
>
> def f():
> print("if A() and A()")
> if A() and A(): x = 1
> print("cond = A() and A()")
> cond = A() and A()
> if cond: x = 1
>
> f()
>
> And once you've run the code, disassemble the function for extra insight.
>
> There's a very definite difference here, and it's the same difference as:
>
> for x in thing:
>
> and
>
> for x in iter(thing):
>
> I'd be fine with documenting that __bool__ is (guaranteed to be)
> called only if the interpreter needs to know the result, leaving open
> the option for things like this to be optimized out. That'd leave open
> the option for "foo() if x else foo()" to be optimized down to just
> "foo()", although I don't think that particular one is needed.
>

I think that's the key question here: it is a language change, but is it
one we want (specifically in this case and in general)? It will require a
language reference change at least, and as it has been shown, some people
don't expect Python to take shortcuts in execution knowing full well that
CPython dutifully executes things. So saying we only call __bool__() *if
necessary* is definitely a change. But then this begs the question of
whether we want to do this more widely in the language in the name of
optimization, or not in the name of consistency that a user can easily
reason about.


>
> > I know that I would be very surpised if a statement like
> >
> >
> > if foo():
> >
> > pass
> >
> > ended up optimizing itself and not calling foo(), which is only one step
> > away from the quoted case of
> >
> > if b:
> >
> > pass
> >
> > which is the equivalent to
> >
> > if b.__bool__():
> >
> > pass
>
> Yes, they do look similar. The difference is that calling __bool__ is
> under the interpreter's control, and it's easier to document that it
> be assumed to be side-effect-free.
>
> > Yes, perhaps there is more of an expectation that __bool__() is
> > innocuous, but is that a an assumption that should be baked into the
> > language.
> >
>
> I think so. Consider that sometimes other dunders won't be called, if
> the interpreter believes it's not necessary:
>
> class A(float):
> def __index__(self):
> print("A().__index__")
> return 10
>
> class B(int):
> def __index__(self):
> print("B().__index__")
> return 10
>
> print(range(20)[A():B()])
>
> If it's a subclass of float, slicing will call __index__, but if it's
> a subclass of int, Python knows already that it can use the internal
> integer value.
>

https://snarky.ca/unravelling-not-in-python/ ???? But basically,
https://docs.python.org/3.8/reference/datamodel.html#object.__index__ says
"called *if *conversion to an int is necessary" which isn't the case when
something is already an int.


>
> ChrisA
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GHKDF6YE3D43JPWS7GVG34FVPJNYE5SO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:

> That'd leave open
> the option for "foo() if x else foo()" to be optimized down to just
> "foo()", although I don't think that particular one is needed.

That would be an unsafe optimization. Not all objets are representable
as truthy/falsey values, e.g. numpy arrays.

>>> bool(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element
is ambiguous. Use a.any() or a.all()

Not even all builtin values. This is in Python 3.9:

>>> bool(NotImplemented)
<stdin>:1: DeprecationWarning: NotImplemented should not be used in
a boolean context
True

I believe that 3.10 makes it an error. If not 3.10, then it will surely
happen soon. But even without the change to NotImplemented, it has never
been the case that *every* object is guaranteed to be either truthy or
falsey. At least not since the Python 1.x `__nonzero__` dunder was put
into the language.



--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7Q76EALFDEI2S6L3NFXSJS4TNNEWMO6L/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano <steve@pearwood.info> wrote:
>
> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
>
> > That'd leave open
> > the option for "foo() if x else foo()" to be optimized down to just
> > "foo()", although I don't think that particular one is needed.
>
> That would be an unsafe optimization. Not all objets are representable
> as truthy/falsey values, e.g. numpy arrays.
>
> >>> bool(a)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ValueError: The truth value of an array with more than one element
> is ambiguous. Use a.any() or a.all()
>
> Not even all builtin values. This is in Python 3.9:
>
> >>> bool(NotImplemented)
> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
> a boolean context
> True
>
> I believe that 3.10 makes it an error. If not 3.10, then it will surely
> happen soon. But even without the change to NotImplemented, it has never
> been the case that *every* object is guaranteed to be either truthy or
> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
> into the language.
>

But this is exactly where we started: with a boolification that fails,
in a context where the result wouldn't actually change anything.
Actually calling bool() on something will continue to have the
behaviour you're describing, but if the truthiness or falsiness would
make no difference, is the interpreter required to find out?

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IMG437FNV552XUWOQJW76A7SFAJQULEH/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 10:37 PM, Chris Angelico wrote:
> On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote:
>>
>> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
>>
>>> That'd leave open
>>> the option for "foo() if x else foo()" to be optimized down to just
>>> "foo()", although I don't think that particular one is needed.
>>
>> That would be an unsafe optimization. Not all objets are representable
>> as truthy/falsey values

>> Not even all builtin values. This is in Python 3.9:
>>
>> >>> bool(NotImplemented)
>> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
>> a boolean context
>> True
>>
>> I believe that 3.10 makes it an error. If not 3.10, then it will surely
>> happen soon. But even without the change to NotImplemented, it has never
>> been the case that *every* object is guaranteed to be either truthy or
>> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
>> into the language.
>
> But this is exactly where we started: with a boolification that fails,
> in a context where the result wouldn't actually change anything.
> Actually calling bool() on something will continue to have the
> behaviour you're describing, but if the truthiness or falsiness would
> make no difference, is the interpreter required to find out?

Yes.

Optimizations are an implementation detail, and implementation details should not change the language.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JFOAJSXWH5KO32YVBN2LKZCPWAFMG63G/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman <ethan@stoneleaf.us> wrote:
>
> On 1/12/21 10:37 PM, Chris Angelico wrote:
> > On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote:
> >>
> >> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote:
> >>
> >>> That'd leave open
> >>> the option for "foo() if x else foo()" to be optimized down to just
> >>> "foo()", although I don't think that particular one is needed.
> >>
> >> That would be an unsafe optimization. Not all objets are representable
> >> as truthy/falsey values
>
> >> Not even all builtin values. This is in Python 3.9:
> >>
> >> >>> bool(NotImplemented)
> >> <stdin>:1: DeprecationWarning: NotImplemented should not be used in
> >> a boolean context
> >> True
> >>
> >> I believe that 3.10 makes it an error. If not 3.10, then it will surely
> >> happen soon. But even without the change to NotImplemented, it has never
> >> been the case that *every* object is guaranteed to be either truthy or
> >> falsey. At least not since the Python 1.x `__nonzero__` dunder was put
> >> into the language.
> >
> > But this is exactly where we started: with a boolification that fails,
> > in a context where the result wouldn't actually change anything.
> > Actually calling bool() on something will continue to have the
> > behaviour you're describing, but if the truthiness or falsiness would
> > make no difference, is the interpreter required to find out?
>
> Yes.
>
> Optimizations are an implementation detail, and implementation details should not change the language.
>

The language can also be defined in an optimization-friendly way,
though. Consider how we got positional-only arguments in Python: first
they existed in C-implemented functions in CPython, even though they
couldn't exist in pure Python code, and then the functionality got
added to the language definition, thus permitting the optimization.

Or consider dictionary lookup. Most people treat it as "find a key
which is equal to the one you're looking for", but the actual
definition is "find a key which is identical to, or equal to, the one
you're looking for". That's partly to ensure that weird cases like NaN
don't break too badly, but also it means that x.__eq__(x) doesn't have
to be called all the time.

The topic under discussion is a language definition. Choosing to
permit the optimization doesn't mean that the implementation detail
changes the language. Choosing to deny it means there won't be an
optimization.

I personally don't see any reason to force Python to calculate
something unnecessarily, given that this is *already* happening in
other situations (see the "if a and b:" optimization, which doesn't
boolify twice).

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SAUNDT7XTTFWREABUXPE2OKWI6KBQECO/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Wed, 13 Jan 2021 18:27:07 +1100
Chris Angelico <rosuav@gmail.com> wrote:

[]

> > Optimizations are an implementation detail, and implementation
> > details should not change the language.
>
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

In this case, the culprit seems to be that __bool__() and many other
"special" methods may have side effects. So, just as "const" annotation
would be useful for variables, "pure" (side-effect free) annotation
would be useful for functions.

But that alone won't help due to too-dynamic nature of Python. It's not
much of an optimization if, at runtime, at each call-site, you need to
check whether a function is pure to decide if you have to call it, or
may skip it. Instead, that rather be done at compile time.

But checking which method (pure or impure) will be called when is again
complicated statically in Python. What to do about that? Well, we can
introduce, ahem, some "strict" mode, in which *all* __bool__(),
__gt__(), and friends must be declared pure, and for good measure,
returning bool. Then we know we can skip any such method call.

You see, all heresy starts from small things...

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 11:27 PM, Chris Angelico wrote:
> On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman wrote:

>> Optimizations are an implementation detail, and implementation details should not change the language.
>>
>
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

1. What optimization?
2. Did the language change because of the optimization?

> Or consider dictionary lookup. Most people treat it as "find a key
> which is equal to the one you're looking for", but the actual
> definition is "find a key which is identical to, or equal to, the one
> you're looking for".

Exactly. The definition, i.e. language spec, says identity, then equality.

> The topic under discussion is a language definition. Choosing to
> permit the optimization doesn't mean that the implementation detail
> changes the language. Choosing to deny it means there won't be an
> optimization.

There are, I am sure, many optimizations that are not possible because of Python's dynamism. `if <something>` is
supposed to evaluate `bool(something)`, regardless of what comes after.

> I personally don't see any reason to force Python to calculate
> something unnecessarily, given that this is *already* happening in
> other situations (see the "if a and b:" optimization, which doesn't
> boolify twice).

Sure, and I agree that calling `bool()` a second time is wasteful, as well as possibly confusing -- Python already has
the answer from the first `bool()` call, so why would it need to do it again? That seems a matter of correctness, not
optimization.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MNP25T5WJ5YIVIOJJFTU5QSO5ME6BUED/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 8:02 PM Ethan Furman <ethan@stoneleaf.us> wrote:
>
> On 1/12/21 11:27 PM, Chris Angelico wrote:
> > On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman wrote:
>
> >> Optimizations are an implementation detail, and implementation details should not change the language.
> >>
> >
> > The language can also be defined in an optimization-friendly way,
> > though. Consider how we got positional-only arguments in Python: first
> > they existed in C-implemented functions in CPython, even though they
> > couldn't exist in pure Python code, and then the functionality got
> > added to the language definition, thus permitting the optimization.
>
> 1. What optimization?
> 2. Did the language change because of the optimization?

It's a lot faster for C-implemented functions to require positional
parameters. A number of functions had help text that implied that they
accepted keyword-or-positional args, but if you tried to call them
with keyword args, they'd error out.

And yes. PEP 457 and then PEP 570 introduced real positional-only
arguments, and part of the debate surrounded the question "should we
require that all C-implemented functions support keyword args?". Why
have a performance hit on all C functions just for the sake of an
arbitrary consistency? Instead, the language definition was changed to
make this possible - and now we have, for instance, "len(obj, /)",
which clearly does not accept len(obj="x"), but does accept len("x").

> > Or consider dictionary lookup. Most people treat it as "find a key
> > which is equal to the one you're looking for", but the actual
> > definition is "find a key which is identical to, or equal to, the one
> > you're looking for".
>
> Exactly. The definition, i.e. language spec, says identity, then equality.

Right. The definition is set to permit the optimization. Therefore the
optimization cannot be in violation of the spec.

> > The topic under discussion is a language definition. Choosing to
> > permit the optimization doesn't mean that the implementation detail
> > changes the language. Choosing to deny it means there won't be an
> > optimization.
>
> There are, I am sure, many optimizations that are not possible because of Python's dynamism. `if <something>` is
> supposed to evaluate `bool(something)`, regardless of what comes after.

It doesn't, though. There is a difference between:

if a and b: ...

and

x = a and b
if x: ...

The first one will boolify a only once; the second will figure out
what bool(a) is before the assignment, and then (if it's false)
boolify it again in the 'if' statement.

Current builds of Python 3.10 optimize this away too:

if a or 1: print(1)

Regardless of the truthiness of a, the print call will happen.

(Note that a is still *evaluated*. It's just not queried for
truthiness. For instance, "if f() or 1:" will still call the function;
it just won't call __bool__() on the return value.)

> > I personally don't see any reason to force Python to calculate
> > something unnecessarily, given that this is *already* happening in
> > other situations (see the "if a and b:" optimization, which doesn't
> > boolify twice).
>
> Sure, and I agree that calling `bool()` a second time is wasteful, as well as possibly confusing -- Python already has
> the answer from the first `bool()` call, so why would it need to do it again? That seems a matter of correctness, not
> optimization.
>

Is it? It means that there's a visible difference between putting it
in a variable and doing it inline. If it's okay to do that, then why
is it not okay to remove the bool check when it can't affect the
result?

In theory, "x = (expr)" followed by "if x:" should perform the exact
same checks as "if (expr):"; if it's okay to violate that principle
for "if a and b:", then why not for "if a: pass"? Either way, the
interpreter knows that any sane __bool__ function cannot affect the
result.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RK36KL2EH5I54W76INZ64OBFJ5BBY5L7/
Code of Conduct: http://python.org/psf/codeofconduct/

Even if you define __bool__() as returning a bool, and error/undefined
behavior otherwise, that doesn't eliminate side effects. Is it even
possible to nail down a definition to the point that you can say, "Thou
shalt not mutate or cause anything" and have it meaningfully enforced in
the compiler or interpreter?

-Em

On Wed, Jan 13, 2021 at 12:18 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:

> Hello,
>
> On Wed, 13 Jan 2021 18:27:07 +1100
> Chris Angelico <rosuav@gmail.com> wrote:
>
> []
>
> > > Optimizations are an implementation detail, and implementation
> > > details should not change the language.
> >
> > The language can also be defined in an optimization-friendly way,
> > though. Consider how we got positional-only arguments in Python: first
> > they existed in C-implemented functions in CPython, even though they
> > couldn't exist in pure Python code, and then the functionality got
> > added to the language definition, thus permitting the optimization.
>
> In this case, the culprit seems to be that __bool__() and many other
> "special" methods may have side effects. So, just as "const" annotation
> would be useful for variables, "pure" (side-effect free) annotation
> would be useful for functions.
>
> But that alone won't help due to too-dynamic nature of Python. It's not
> much of an optimization if, at runtime, at each call-site, you need to
> check whether a function is pure to decide if you have to call it, or
> may skip it. Instead, that rather be done at compile time.
>
> But checking which method (pure or impure) will be called when is again
> complicated statically in Python. What to do about that? Well, we can
> introduce, ahem, some "strict" mode, in which *all* __bool__(),
> __gt__(), and friends must be declared pure, and for good measure,
> returning bool. Then we know we can skip any such method call.
>
> You see, all heresy starts from small things...
>
> []
>
> --
> Best regards,
> Paul mailto:pmiscml@gmail.com
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Wed, Jan 13, 2021 at 9:08 PM Emily Bowman <silverbacknet@gmail.com> wrote:
>
> Even if you define __bool__() as returning a bool, and error/undefined behavior otherwise, that doesn't eliminate side effects. Is it even possible to nail down a definition to the point that you can say, "Thou shalt not mutate or cause anything" and have it meaningfully enforced in the compiler or interpreter?
>

Yes - just say "will be called when the interpreter needs to know the
booleanness of the object", and don't mandate that it be called if the
interpreter already knows. If someone writes a __bool__ function that
decrements a counter and returns True if it's still above zero, then
they're shooting themselves in the foot, and we don't have to protect
them.

It's like having __iter__ return an object on which __iter__ doesn't
return self. There's no protection in the language to stop you from
creating such a monster, but you'll run into problems sooner or later
if you do, and it's not a bug in Python.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RN7SAZ3X4YM2KEHNVMAXDNPLQFWVY7NJ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/12/21 11:27 PM, Chris Angelico wrote:
> The language can also be defined in an optimization-friendly way,
> though. Consider how we got positional-only arguments in Python: first
> they existed in C-implemented functions in CPython, even though they
> couldn't exist in pure Python code, and then the functionality got
> added to the language definition, thus permitting the optimization.

I wouldn't describe positional-only parameters as an "optimization". 
They were created to add expressiveness to Python code, not to make
Python code faster.  The classic example is the dict constructor:
technically, you couldn't implement it correctly in pure Python code,
because it had a parameter (the iterable) that could not be a named
parameter.  Any name you gave it would preclude passing in that name as
a name=value argument.

Parsing positional-only parameters is often faster than parsing named
parameters, for obvious reasons.  But this is really small stuff,
drowned out by almost any other variation in your code.


Cheers,


//arry/

Hello,

On Wed, 13 Jan 2021 02:08:01 -0800
Emily Bowman <silverbacknet@gmail.com> wrote:

> Even if you define __bool__() as returning a bool, and
> error/undefined behavior otherwise, that doesn't eliminate side
> effects. Is it even possible to nail down a definition to the point
> that you can say, "Thou shalt not mutate or cause anything" and have
> it meaningfully enforced in the compiler or interpreter?

Yes, sure, "pure" annotation on a function would (ideally/eventually)
need "typechecking". We could start with simple rules like: a) a pure
function should not mutate any objects which it didn't create; b) for
"mutation" also counts passing objects to (other) non-pure functions.
The hardest part is again to know which of these other functions is pure
or impure. Runtime checking should not be the aim to shoot for. There
should be a way to preserve as much as possible of Python's dynamic
nature that we all love, yet avoid runtime lookups.

The above "purity" conditions are also too restrictive, so would need
to be extended/elaborated (which would certainly require more detailed
annotations; fortunately, we all already accepted annotations to be a
part of Python's nature, so wanting to use them more shouldn't come as
surprise).

>
> -Em
>
> On Wed, Jan 13, 2021 at 12:18 AM Paul Sokolovsky <pmiscml@gmail.com>
> wrote:
>
> > Hello,
> >
> > On Wed, 13 Jan 2021 18:27:07 +1100
> > Chris Angelico <rosuav@gmail.com> wrote:
> >
> > []
> >
> > > > Optimizations are an implementation detail, and implementation
> > > > details should not change the language.
> > >
> > > The language can also be defined in an optimization-friendly way,
> > > though. Consider how we got positional-only arguments in Python:
> > > first they existed in C-implemented functions in CPython, even
> > > though they couldn't exist in pure Python code, and then the
> > > functionality got added to the language definition, thus
> > > permitting the optimization.
> >
> > In this case, the culprit seems to be that __bool__() and many other
> > "special" methods may have side effects. So, just as "const"
> > annotation would be useful for variables, "pure" (side-effect free)
> > annotation would be useful for functions.
> >
> > But that alone won't help due to too-dynamic nature of Python. It's
> > not much of an optimization if, at runtime, at each call-site, you
> > need to check whether a function is pure to decide if you have to
> > call it, or may skip it. Instead, that rather be done at compile
> > time.
> >
> > But checking which method (pure or impure) will be called when is
> > again complicated statically in Python. What to do about that?
> > Well, we can introduce, ahem, some "strict" mode, in which *all*
> > __bool__(), __gt__(), and friends must be declared pure, and for
> > good measure, returning bool. Then we know we can skip any such
> > method call.
> >
> > You see, all heresy starts from small things...
> >
> > []
> >
> > --
> > Best regards,
> > Paul mailto:pmiscml@gmail.com
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> > https://mail.python.org/archives/list/python-dev@python.org/message/4P52JORZ4WBCPC6DERRITDRETEFWT73H/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >



--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TCG7237FJILE4MVKBXSTTQEIR6CX7VFJ/
Code of Conduct: http://python.org/psf/codeofconduct/

Hi Rob,

On 13/01/2021 12:18 pm, Rob Cliffe wrote:
>
>
> On 12/01/2021 15:53, Mark Shannon wrote:
>>
>> In master we convert `if x: pass` to `pass` which is equivalent,
>> unless bool(x) has side effects the first time it is called. This is a
>> recent change.
> Can you please confirm that this optimisation ONLY applies to bare
> names, i.e. NOT
>
>     if x(): pass
>     if x.y: pass
>     if x+y: pass


The optimization doesn't apply to the expression, but the test.
The optimizer (might) transform

if x+y: pass

to

x+y

But the expression is still evaluated.
Sorry for the confusion, I should have been clearer in my example.
It is the call to `bool()` that *might* be eliminated.

Cheers,
Mark.



>
> Thanks
> Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YZMA3ABVUKJ3PDFO4BS56QRSL4YCC3DJ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 12/01/2021 15:53, Mark Shannon wrote:
> Hi everyone,
>
>
>
> In master we convert `if x: pass` to `pass` which is equivalent,
> unless bool(x) has side effects the first time it is called. This is a
> recent change.
>
Suppose x is not a currently valid variable name at runtime.  Will the
NameError still be "optimised" away?
Thanks
Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JK4QENGPZV56RNGI5YHLWCL4OF4AEGSP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 08:27:46PM +1100, Chris Angelico wrote:

> It's a lot faster for C-implemented functions to require positional
> parameters. A number of functions had help text that implied that they
> accepted keyword-or-positional args, but if you tried to call them
> with keyword args, they'd error out.

That's an unfortunate limitation of the docs, not a language feature.
One might even call it a documentation bug.


> And yes. PEP 457 and then PEP 570 introduced real positional-only
> arguments, and part of the debate surrounded the question "should we
> require that all C-implemented functions support keyword args?". Why
> have a performance hit on all C functions just for the sake of an
> arbitrary consistency? Instead, the language definition was changed to
> make this possible - and now we have, for instance, "len(obj, /)",
> which clearly does not accept len(obj="x"), but does accept len("x").

Sorry to be pedantic here... oh who am I fooling, I'm not sorry at all
*wink*

I think that if we are to be precise, and we should be, Python the
language has always supported positional-only arguments. What we lacked
was syntax for declaring named positional-only **parameters**.

There has always been at least two ways to implement positional-only
arguments:

- write your function in C;

- or use `*args` and process them yourself.

So that's not a new feature. The new feature gained in 3.8 was syntax to
define names for positional-only parameters in def statements.

This may not have been really obvious until after PEPs 457 and 570
because we were (and still are) fairly blas? about the difference
between arguments and parameters, and of calling things "positional"
regardless of whether they were actually positional-or-keyword or
positional-only.

In any case, this is very much a red herring. Function call syntax is
not an optimization, it is a syntactic feature of the language. The
ability to pass an argument by position, position-or-keyword, or keyword
is an element of language design. The choice of which ones to support
may be influenced by concerns about efficiency and speed, but that's as
far as it goes.

I think your earlier example of short-cutting equality tests with
identity tests is more relevant here.


[...]
> In theory, "x = (expr)" followed by "if x:" should perform the exact
> same checks as "if (expr):"; if it's okay to violate that principle
> for "if a and b:", then why not for "if a: pass"? Either way, the
> interpreter knows that any sane __bool__ function cannot affect the
> result.

Of course they can: the method can fail and raise an exception. That
could be due to a bug, or by design, as in numpy arrays and (soon)
NotImplemented.

Sane `__bool__` methods can have useful side-effects. More than once
I've debugged code by sticking some logging inside a method I knew would
be called, and that method could as easily be `__bool__` as any other.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RH2SYWCCCGZGDMPXWHJ5VMIKK3EQJIVZ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 02:08:01AM -0800, Emily Bowman wrote:
> Even if you define __bool__() as returning a bool, and error/undefined
> behavior otherwise, that doesn't eliminate side effects. Is it even
> possible to nail down a definition to the point that you can say, "Thou
> shalt not mutate or cause anything" and have it meaningfully enforced in
> the compiler or interpreter?

No, I don't think it is possible to enforce lack of side-effects. Not
without rebuilding the language from the ground up with a clear,
definitive and enforcable distinction between pure and impure functions.

(I think Haskell does something like that. But trying to retrofit that
into Python would probably be about as hard as building a restricted
mode, and for similar reasons.)

Besides, we probably don't want to prohibit side-effects in `__bool__`.
That would prohibit useful tricks such as putting logging calls into a
method you are trying to debug.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/L3M6RLVJ44VN4S34RYJTROSAZMUEPGWM/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Thu, 14 Jan 2021 03:29:35 +1100
Steven D'Aprano <steve@pearwood.info> wrote:

> On Wed, Jan 13, 2021 at 02:08:01AM -0800, Emily Bowman wrote:
> > Even if you define __bool__() as returning a bool, and
> > error/undefined behavior otherwise, that doesn't eliminate side
> > effects. Is it even possible to nail down a definition to the point
> > that you can say, "Thou shalt not mutate or cause anything" and
> > have it meaningfully enforced in the compiler or interpreter?
>
> No, I don't think it is possible to enforce lack of side-effects. Not
> without rebuilding the language from the ground up with a clear,
> definitive and enforcable distinction between pure and impure
> functions.
>
> (I think Haskell does something like that. But trying to retrofit
> that into Python would probably be about as hard as building a
> restricted mode, and for similar reasons.)
>
> Besides, we probably don't want to prohibit side-effects in
> `__bool__`. That would prohibit useful tricks such as putting logging
> calls into a method you are trying to debug.

Surely, if we do that, we wouldn't use Haskell's definition of
purity ;-). Rather, a practical definition of purity, Python-style. For
example, print() would be considered "pure", as its purpose is to
provide program output, not arbitrarily change program state (so, all of
__str__, __repr__, __format__ would need to be pure, but if someone
overrode print() to shoot at random modules/data at them, then they
shoot themselves in the feet, nothing else).

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/44IGY7CWR3OPM2RN64GJHXOML4ZMW3IX/
Code of Conduct: http://python.org/psf/codeofconduct/

On 14/01/21 5:29 am, Steven D'Aprano wrote:
> No, I don't think it is possible to enforce lack of side-effects. Not
> without rebuilding the language from the ground up with a clear,
> definitive and enforcable distinction between pure and impure functions.
>
> (I think Haskell does something like that.

All functions are pure in Haskell -- not sure if that counts
as "doing something like that". Retrofitting it into
Python would turn it into a very different language.

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7BOWX7N54CA6FPGJDCQSEH73MQZKJEEE/
Code of Conduct: http://python.org/psf/codeofconduct/

On 14/01/21 6:17 am, Paul Sokolovsky wrote:
> For
> example, print() would be considered "pure", as its purpose is to
> provide program output, not arbitrarily change program state

That definition of purity wouldn't really help with optimisations,
though, because optimising away a print() call would still cause
a visible change in the program's behaviour.

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/B6VDVU7UOB5QNILEJ44J7JOSVBF7WDBF/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Thu, 14 Jan 2021 12:45:11 +1300
Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

> On 14/01/21 6:17 am, Paul Sokolovsky wrote:
> > For
> > example, print() would be considered "pure", as its purpose is to
> > provide program output, not arbitrarily change program state
>
> That definition of purity wouldn't really help with optimisations,
> though, because optimising away a print() call would still cause
> a visible change in the program's behaviour.

But nobody talked about optimizing away generic "pure"-annotated
functions (which would differ from "mathematical" definition of
purity), only about optimizing "pure" *dunder* methods (which are by
definition special methods, which are called implicitly; or not called,
which is the aim we discuss).

"Pure" annotation on other functions are drawn only to verify that a
dunder itself is indeed "pure".

And "pure" is of course a pretty rough term meaning "not causing
*non-local* side effects". print() in that regard causes only localized
side effect of output appearing on terminal. print() also can be
redefined to output to io.StringIO, and it still will be "pure", as
side effects will be localized to just that io.StringIO object (which
has dedicated purpose to be print's buffer in the first place). But if
user redefines print() to e.g. manipulate sys.path, it's now causes
non-local side effects, and thus breaks API contract re: "purity".

> --
> Greg

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/B6E2NZYIURXDRZ336TBAVCANI666KH4A/
Code of Conduct: http://python.org/psf/codeofconduct/

On 14/01/21 1:13 pm, Paul Sokolovsky wrote:
> But nobody talked about optimizing away generic "pure"-annotated
> functions (which would differ from "mathematical" definition of
> purity), only about optimizing "pure" *dunder* methods

The same thing applies. If we decide that print() is pure,
then a __bool__ that calls print() is also pure, so there's
nothing wrong with optimising it away, right?

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KFP2TZXVJAQROMILOOM3CJT24DJESBHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/13/2021 8:56 PM, Greg Ewing wrote:
> On 14/01/21 1:13 pm, Paul Sokolovsky wrote:
>> But nobody talked about optimizing away generic "pure"-annotated
>> functions (which would differ from "mathematical" definition of
>> purity), only about optimizing "pure" *dunder* methods
>
> The same thing applies. If we decide that print() is pure,
> then a __bool__ that calls print() is also pure, so there's
> nothing wrong with optimising it away, right?

I say 'yes', because the purpose of logging is to document what happens,
and if nothing happens, there is nothing to document. Wrapping a
.__bool__ in a logging decorator might be part of testing it.


--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NRJT2YZ2U73ZYFZPAKKPXFBKGLJMBG3M/
Code of Conduct: http://python.org/psf/codeofconduct/

On 14/01/21 3:32 pm, Terry Reedy wrote:
> I say 'yes', because the purpose of logging is to document what happens,
> and if nothing happens, there is nothing to document.  Wrapping a
> .__bool__ in a logging decorator might be part of testing it.

Or it might be something else.

It would be fine to *define* __bool__ as something the compiler
is allowed to optimise away. But that's a different thing from
"purity" (which is hard to pin down in an imperative language).

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FKQSB6SVOAO2Y3BOQSXSHEN5LM2KBETR/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Thu, 14 Jan 2021 23:00:06 +1300
Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

> On 14/01/21 3:32 pm, Terry Reedy wrote:
> > I say 'yes', because the purpose of logging is to document what
> > happens, and if nothing happens, there is nothing to document.
> > Wrapping a .__bool__ in a logging decorator might be part of
> > testing it.
>
> Or it might be something else.
>
> It would be fine to *define* __bool__ as something the compiler
> is allowed to optimise away.

Yes, and the talk was exactly about how to *proceed* with (re)defining
__bool__ (and other dunders, then other runtime inefficiencies) as
something the compiler is allowed to optimise away. To retrace the
outline:

1. Current semantics doesn't say that __bool__ can be optimized away,
and it stays that way. (Compare that to the parallel discussion of
PEP649 - it tries to change existing semantics by disallowing variable
annotations inside the "if" statement, and that's raised as a concern).
2. Instead, a new semantic mode is introduced, which needs to be
explicitly enabled.
3. Now the problem comes up about compatibility of the existing source
code with the new mode.
4. It's resolved by requiring __bool__ and other dunders to be
annotated "pure" (tentative code-name). The code which doesn't have all
__bool__, etc. methods annotated as such cannot be run in the new
semantic mode (and benefit from any optimizations it may offer), and has
to continue be run in the old mode. (And note that annotated code
still can run in the old mode either.)
5. Now, stamping annotations is nice, but the code should correspond to
them, so there should be ability to "typecheck" such annotated code
(more like "behavior-check", though effects tracking is also part of
the type theory). It can be handled in the same way as typechecking
existing type annotations happen - by external tools.

> But that's a different thing from
> "purity" (which is hard to pin down in an imperative language).

At the beginning of this sub-thread, example of Haskell was given. It
went the different way - of banning any mutability and side-effects at
all, then making heroic efforts of introducing concepts which would
make writing programs under such model still surmountable, like monads,
etc. That was hard. It didn't stop them. We here talk about
evolutionary, and clearly delinated adjustments to Python semantics,
based on the experience with efficiently executing Python code.
Certainly, much less hard than what Haskell did.

> --
> Greg

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2AI7DRSQ6AJUNQSVXU672DQ3OOWVQNV7/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 13, 2021 at 08:17:26PM +0300, Paul Sokolovsky wrote:

> > Besides, we probably don't want to prohibit side-effects in
> > `__bool__`. That would prohibit useful tricks such as putting logging
> > calls into a method you are trying to debug.
>
> Surely, if we do that, we wouldn't use Haskell's definition of
> purity ;-). Rather, a practical definition of purity, Python-style.

It's not *Haskell's* definition, it's pretty much *everyone's*
definition of pure.

https://en.wikipedia.org/wiki/Pure_function



> For example, print() would be considered "pure", as its purpose is to
> provide program output, not arbitrarily change program state


Is writing to an external file a change to program state? Keep in mind
that programs can get state from external files.

I would say that what you are describing is not a distinction between
pure and impure functions, based on whether or not they can be safely
optimised away without a change in visible behaviour. The distinction
you are proposing is into two arbitrary sets of functions, some of which
can be optimized away and some which can't, regardless of whether or not
that optimisation leads to change in visible behaviour.

If you want to argue for that, fine, do so, but don't call impure
functions "pure".


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FKSAUJVPPP2JV56T4BIB7W3B6GD63HGE/
Code of Conduct: http://python.org/psf/codeofconduct/

Hello,

On Thu, 14 Jan 2021 22:05:37 +1100
Steven D'Aprano <steve@pearwood.info> wrote:

> On Wed, Jan 13, 2021 at 08:17:26PM +0300, Paul Sokolovsky wrote:
>
> > > Besides, we probably don't want to prohibit side-effects in
> > > `__bool__`. That would prohibit useful tricks such as putting
> > > logging calls into a method you are trying to debug.
> >
> > Surely, if we do that, we wouldn't use Haskell's definition of
> > purity ;-). Rather, a practical definition of purity,
> > Python-style.
>
> It's not *Haskell's* definition, it's pretty much *everyone's*
> definition of pure.
>
> https://en.wikipedia.org/wiki/Pure_function
>
>
>
> > For example, print() would be considered "pure", as its purpose is
> > to provide program output, not arbitrarily change program state
>
>
> Is writing to an external file a change to program state? Keep in
> mind that programs can get state from external files.

Please reread the reply about "not having non-local side
effects". NotHavingNonLocalSideEffects is a bit longish for an
annotation identifier though...

>
> I would say that what you are describing is not a distinction between
> pure and impure functions,

You say that as if you propose some better identifier for
annotation ;-). I personally would be fine if the annotation is:

def print(*Any) -> PythoniclyPure: ...

I'm talking about small-scale semantic distinctions required for capture
needed large-scale semantic effect. Naming, when it comes to it, will
be subject of long, looooooong bikeshedding. (Which already started,
thanks!)

> based on whether or not they can be safely
> optimised away without a change in visible behaviour. The distinction
> you are proposing is into two arbitrary sets of functions, some of
> which can be optimized away and some which can't, regardless of
> whether or not that optimisation leads to change in visible behaviour.
>
> If you want to argue for that, fine, do so, but don't call impure
> functions "pure".

[]

--
Best regards,
Paul mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QBWTRDJZHDXYAOY42DZMETRNPOLHPFZH/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Jan 15, 2021 at 10:13 PM Rob Cliffe via Python-Dev
<python-dev@python.org> wrote:
>
>
>
> On 12/01/2021 15:53, Mark Shannon wrote:
> > Hi everyone,
> >
> >
> >
> > In master we convert `if x: pass` to `pass` which is equivalent,
> > unless bool(x) has side effects the first time it is called. This is a
> > recent change.
> >
> Suppose x is not a currently valid variable name at runtime. Will the
> NameError still be "optimised" away?

No, it won't. The expression still gets fully evaluated. The ONLY part
that gets optimized away is the check "is this thing true?".

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KQ2QH3VB5Q6LYCBE4SUWZKRMML54TOXS/
Code of Conduct: http://python.org/psf/codeofconduct/