Mailing List Archive

On Wed, 13 Jan 2021, 12:35 pm Larry Hastings, <larry@hastings.org> wrote:

>
> On 1/12/21 5:28 PM, Brett Cannon wrote:
>
> The other thing to keep in mind is we are talking about every module,
> class, and function getting 64 bytes ... which I bet isn't that much.
>
> Actually it's only every module and class. Functions don't have this
> problem because they've always stored __annotations__ internally--meaning,
> peeking in their __dict__ doesn't work, and they don't support inheritance
> anyway. So the number is even smaller than that.
>
> If we can just make __annotations__ default to an empty dict on classes
> and modules, and not worry about the memory consumption, that goes a long
> way to cleaning up the semantics.
>

Could you get the best of both worlds by making __annotations__ an
auto-populating descriptor on "type", the way it is on functions?

Continue to add a non-empty annotations dict to the class dict eagerly, but
only add the empty dict when "cls.__annotations__" is accessed.

Then your co_annotations PEP would only be changing the way the non-empty
case was handled, rather than introducing the descriptor in the first place.

Cheers,
Nick.

>

On 1/16/21 8:41 AM, Nick Coghlan wrote:
> Could you get the best of both worlds by making __annotations__ an
> auto-populating descriptor on "type", the way it is on functions?
>
> Continue to add a non-empty annotations dict to the class dict
> eagerly, but only add the empty dict when "cls.__annotations__" is
> accessed.


I think that'll work though it's a little imprecise.  Consider the best
practice for getting class annotations, example here from
Lib/dataclasses.py:

cls_annotations = cls.__dict__.get('__annotations__', {})

What happens when that current best practice code meets your proposed
"lazy-populate the empty dict" approach?

* If a class has annotations set, cls.__dict__['__annotations__'] will
be set, so the code works fine.

* If a class doesn't have annotations set, then
cls.__dict__['__annotations__'] won't be set yet.  So people peering
in cls.__dict__['__annotations__'] will get the right /answer/, that
no annotations are set.  But they'll see the wrong /specifics/:
they'll think annotations are unset, when in fact it has an empty
dict as its value.

So the code will continue to work, even though it's arguably a little
misguided.  If anybody distinguished between "annotations are unset" and
"annotations are set to an empty dict", that code would fail, but I
expect nobody ever does that.

Two notes about this idea.  First, I think most people who use this
best-practices code above use it for modules as well as classes.  (They
have two code paths: one for functions, the other for not-functions.) 
But everything I said above is true for both classes and modules.

Second, I think this is only sensible if, at the same time, we make it
illegal to delete cls.__annotations__.  If we lazy-populate the empty
dict, and a user deletes cls.__annotations__, and we don't remember some
extra state, we'd just re-"lazy" create the empty dict the next time
they asked for it.  Which is actually what functions do, just
lazy-repopulate the empty annotations dict every time, and I'm not keen
to bring those semantics to classes and modules.


Cheers,


//arry/

On 17/01/21 12:31 pm, Larry Hastings wrote:
>
> Consider the best
> practice for getting class annotations, example here from
> Lib/dataclasses.py:
>
> cls_annotations = cls.__dict__.get('__annotations__', {})

Isn't that going to get broken anyway? It won't trigger the
calling of __co_annotations__.

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPDBI3J4SUJ6NCWRORWYAOKYW4IM3HMQ/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/16/21 4:09 PM, Greg Ewing wrote:
> On 17/01/21 12:31 pm, Larry Hastings wrote:
>>
>> Consider the best practice for getting class annotations, example
>> here from Lib/dataclasses.py:
>>
>>     cls_annotations = cls.__dict__.get('__annotations__', {})
>
> Isn't that going to get broken anyway? It won't trigger the
> calling of __co_annotations__.


I proposed these as two separate conversations, because I wanted to
clean up the semantics of annotations whether or not PEP 649 was
accepted.  But, yes, if PEP 649 is accepted (in some form), this
current-best-practice would no longer work, and the new best practice
would likely become much more complicated.


Cheers,


//arry/

On Sat, Jan 16, 2021 at 3:32 PM Larry Hastings <larry@hastings.org> wrote:

> [...] If anybody distinguished between "annotations are unset" and
> "annotations are set to an empty dict", that code would fail, but I expect
> nobody ever does that.
>

I agree, since I can't think of differing semantics. Given that
`__annotations__` is filled from annotated class variables, the only reason
someone might care about the difference would be if they are aware of code
that manually *sets* `X.__annotations__ = {}` and they have some kind of
shared understanding that that means something special. I find that highly
unlikely, and frankly, if someone needs such a shared understanding, let
them just pick a unique key to set.

I do worry about the best practice getting worse if your PEP 649 is
accepted.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 1/18/21 12:16 PM, Guido van Rossum wrote:
> I do worry about the best practice getting worse if your PEP 649 is
> accepted.


A good part of what motivated me to start this second thread ("Let's Fix
...") was how much worse best practice would become if PEP 649 is
accepted.  But if we accept PEP 649, /and/ take steps to fix the
semantics of annotations, I think the resulting best practice will be
excellent in the long-run.

Let's assume for a minute that PEP 649 is accepted more-or-less like it
is now.  (The name resolution approach is clearly going to change but
that won't affect the discussion here.)  And let's assume that we also
change the semantics so annotations are always defined (you can't delete
them) and they're guaranteed to be either a dict or None.  (Permitting
__annotations__ to be None isn't settled yet, but it's most similar to
current semantics, so let's just assume it for now.)

Because the current semantics are kind of a mess, most people who
examine annotations already have a function that gets the annotations
for them.  Given that, I really do think the best approach is to gate
the code on version 3.10, like I've described before:

if python version >= 3.10:
    def get_annotations(o):
        return o.__annotations__
else:
    def get_annotations(o):
        if isinstance(o, (type, types.ModuleType)):
            return o.__dict__.get("__annotations__", None)
        else:
            return o.__annotations__

This assumes returning None is fine.  If it had to always return a valid
dict, I'd add "or {}" to the end of every return statement.

Given that it already has to be a function, I think this approach is
readable and performant.  And, at some future time when the code can
drop support for Python < 3.10, we can throw away the if statement and
the whole else block, keeping just the one-line function.  At which
point maybe we'd refactor away the function and just use
"o.__annotations__" everywhere.


I concede that, in the short term, now we've got nine lines and two if
statements to do something that /should/ be relatively
straightforward--accessing the annotations on an object.  But that's
where we find ourselves.  Current best practice is kind of a mess, and
unfortunately PEP 649 breaks current best practice anyway.  My goal is
to fix the semantics so that long-term best practice is sensible, easy,
and obvious.


Cheers,


//arry/

Hm. It's unfortunate that this would break code using what is *currently*
the best practice.

The saving grace seems that for *many* use cases the best practice is to
call typing.get_type_hints(). This is particularly useful for classes
because it includes annotations from base classes.

Also, for functions and modules I would recommend `getattr(o,
"__annotations__", None)` (perhaps with `or {}` added).

I would also honestly discount what dataclasses.py and typing.py have to
do. But what do 3rd party packages do when they don't want to use
get_type_hints() and they want to get it right for classes? That would give
an indication of how serious we should take breaking current best practice.

On Mon, Jan 18, 2021 at 1:10 PM Larry Hastings <larry@hastings.org> wrote:

>
> On 1/18/21 12:16 PM, Guido van Rossum wrote:
>
> I do worry about the best practice getting worse if your PEP 649 is
> accepted.
>
>
> A good part of what motivated me to start this second thread ("Let's Fix
> ...") was how much worse best practice would become if PEP 649 is
> accepted. But if we accept PEP 649, *and* take steps to fix the
> semantics of annotations, I think the resulting best practice will be
> excellent in the long-run.
>
> Let's assume for a minute that PEP 649 is accepted more-or-less like it is
> now. (The name resolution approach is clearly going to change but that
> won't affect the discussion here.) And let's assume that we also change
> the semantics so annotations are always defined (you can't delete them) and
> they're guaranteed to be either a dict or None. (Permitting
> __annotations__ to be None isn't settled yet, but it's most similar to
> current semantics, so let's just assume it for now.)
>
> Because the current semantics are kind of a mess, most people who examine
> annotations already have a function that gets the annotations for them.
> Given that, I really do think the best approach is to gate the code on
> version 3.10, like I've described before:
>
> if python version >= 3.10:
> def get_annotations(o):
> return o.__annotations__
> else:
> def get_annotations(o):
> if isinstance(o, (type, types.ModuleType)):
> return o.__dict__.get("__annotations__", None)
> else:
> return o.__annotations__
>
> This assumes returning None is fine. If it had to always return a valid
> dict, I'd add "or {}" to the end of every return statement.
>
> Given that it already has to be a function, I think this approach is
> readable and performant. And, at some future time when the code can drop
> support for Python < 3.10, we can throw away the if statement and the whole
> else block, keeping just the one-line function. At which point maybe we'd
> refactor away the function and just use "o.__annotations__" everywhere.
>
>
> I concede that, in the short term, now we've got nine lines and two if
> statements to do something that *should* be relatively
> straightforward--accessing the annotations on an object. But that's where
> we find ourselves. Current best practice is kind of a mess, and
> unfortunately PEP 649 breaks current best practice anyway. My goal is to
> fix the semantics so that long-term best practice is sensible, easy, and
> obvious.
>
>
> Cheers,
>
>
> */arry*
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 1/18/21 2:39 PM, Guido van Rossum wrote:
> Hm. It's unfortunate that this would break code using what is
> *currently* the best practice.

I can't figure out how to avoid it.  The problem is, current best
practice sidesteps the class and goes straight to the dict.  How do we
intercept that and run the code to lazy-calculate the annotations?

I mean, let's consider something crazy.  What if we change cls.__dict__
from a normal dict to a special dict that handles the __co_annotations__
machinery?  That might work, except, we literally allow users to supply
their own cls.__dict__ via __prepare__.  So we can't rely on our special
dict.

What if we change cls.__dict__ to a getset?  The user is allowed to set
cls.__dict__, but when you get __dict__, we wrap the actual internal
dict object with a special object that intercepts accesses to
__annotations__ and handles the __co_annotations__ mechanism.  That
might work but it's really crazy and unfortunate.  And it's remotely
possible that a user might override __dict__ as a property, in a way
that breaks this mechanism too.  So it's not guaranteed to always work.

I'm not suggesting we should do these things, I'm just trying to
illustrate how hard I think the problem is.  If someone has a good idea
how we can add the __co_annotations__ machinery without breaking current
best practice I'd love to hear it.


> Also, for functions and modules I would recommend `getattr(o,
> "__annotations__", None)` (perhaps with `or {}` added).

For functions you don't need to bother; fn.__annotations__ is guaranteed
to always be set, and be either a dict or None. (Python will only ever
set it to a dict, but the user is permitted to set it to None.)

I agree with your suggested best practice for modules as it stands today.

And actually, let me walk back something I've said before.  I believe
I've said several times that "people treat classes and modules the
same".  Actually that's wrong.

* Lib/typing.py treats functions and modules the same; it uses
getattr(o, '__annotations__', None).  It treats classes separately
and uses cls.__dict__.get('__annotations__', {}).
* Lib/dataclasses.py uses fn.__annotations__ for functions and
cls.__dict__.get('__annotations__', {}) for classes.  It doesn't
handle modules at all.
* Lib/inspect.py calls Lib/typing.py to get annotations.  Which in
retrospect I think is a bug, because annotations and type hints
aren't the same thing.  (typing.get_type_hints changes None to
type(None), it evaluates strings, etc).

So, for what it's worth, I literally have zero examples of people
treating classes and modules the same when it comes to annotations. 
Sorry for the confusion!


> I would also honestly discount what dataclasses.py and typing.py have
> to do. But what do 3rd party packages do when they don't want to use
> get_type_hints() and they want to get it right for classes? That would
> give an indication of how serious we should take breaking current best
> practice.

I'm not sure how to figure that out.  Off the top of my head, the only
current third-party packages I can think of that uses annotations are
mypy and attrs.  I took a quick look at mypy but I can't figure out what
it's doing.

attrs does something a little kooky.  It access __annotations__ using a
function called _has_own_attributes(), which detects whether or not the
object is inheriting an attribute.  But it doesn't peek in __dict__,
instead it walks the mro and sees if any of its base classes have the
same (non-False) value for that attribute.

https://github.com/python-attrs/attrs/blob/a025629e36440dcc27aee0ee5b04d6523bcc9931/src/attr/_make.py#L343

Happily, that seems like it would continue to work even if PEP 649 is
accepted.  That's good news!


Cheers,


//arry/

On Mon, Jan 18, 2021 at 4:34 PM Larry Hastings <larry@hastings.org> wrote:

>
> On 1/18/21 2:39 PM, Guido van Rossum wrote:
>
> Hm. It's unfortunate that this would break code using what is *currently*
> the best practice.
>
> I can't figure out how to avoid it. The problem is, current best practice
> sidesteps the class and goes straight to the dict. How do we intercept
> that and run the code to lazy-calculate the annotations?
>
> I mean, let's consider something crazy. What if we change cls.__dict__
> from a normal dict to a special dict that handles the __co_annotations__
> machinery? That might work, except, we literally allow users to supply
> their own cls.__dict__ via __prepare__. So we can't rely on our special
> dict.
>

There's a secret though. `cls.__dict__` is not actually a dict -- is a
mappingproxy. The proxy exists because we want to be able to intercept
changes to class attributes such as `__add__` or `__getattribute__` in
order to manipulate the C-level wrappers that implement such overloads.

So *perhaps* we could expand the mappingproxy class to trap read access to
`__annotations__` as a key to do your bidding. (The trick might be exposed
by things like .keys() but that doesn't bother me as much.)

I honestly don't know how the mappingproxy and `__prepare__` interact. I
have to admit I've never used the latter. Presumably the mappingproxy still
plays a role because we'd still want to intercept e.g. `cls.__add__ = <some


> What if we change cls.__dict__ to a getset? The user is allowed to set
> cls.__dict__, but when you get __dict__, we wrap the actual internal dict
> object with a special object that intercepts accesses to __annotations__
> and handles the __co_annotations__ mechanism. That might work but it's
> really crazy and unfortunate. And it's remotely possible that a user might
> override __dict__ as a property, in a way that breaks this mechanism too.
> So it's not guaranteed to always work.
>

Maybe such guarantees are overrated; in any case it looks like a rare
second-order effect (and we're already talking about esoteric usage
patterns).

I'm not suggesting we should do these things, I'm just trying to illustrate
> how hard I think the problem is. If someone has a good idea how we can add
> the __co_annotations__ machinery without breaking current best practice I'd
> love to hear it.
>
> Also, for functions and modules I would recommend `getattr(o,
> "__annotations__", None)` (perhaps with `or {}` added).
>
> For functions you don't need to bother; fn.__annotations__ is guaranteed
> to always be set, and be either a dict or None. (Python will only ever set
> it to a dict, but the user is permitted to set it to None.)
>
> I agree with your suggested best practice for modules as it stands today.
>
> And actually, let me walk back something I've said before. I believe I've
> said several times that "people treat classes and modules the same".
> Actually that's wrong.
>
> - Lib/typing.py treats functions and modules the same; it uses
> getattr(o, '__annotations__', None). It treats classes separately and uses
> cls.__dict__.get('__annotations__', {}).
> - Lib/dataclasses.py uses fn.__annotations__ for functions and
> cls.__dict__.get('__annotations__', {}) for classes. It doesn't handle
> modules at all.
> - Lib/inspect.py calls Lib/typing.py to get annotations. Which in
> retrospect I think is a bug, because annotations and type hints aren't the
> same thing. (typing.get_type_hints changes None to type(None), it
> evaluates strings, etc).
>
> So, for what it's worth, I literally have zero examples of people treating
> classes and modules the same when it comes to annotations. Sorry for the
> confusion!
>

Yeah, that part felt fishy -- basically classes are the only complicated
case here, because in order to construct the full set of annotations you
must walk the MRO.

Honestly *if* you are walking the MRO anyways, it probably doesn't matter
much if you use cls.__dict__.get('__annotations__') or getattr(cls,
'__annotations__') -- you might see some duplicates but you should
generally end up with the same overall set of annotations (though
presumably one could construct a counter-example using multiple
inheritance).

>
> I would also honestly discount what dataclasses.py and typing.py have to
> do. But what do 3rd party packages do when they don't want to use
> get_type_hints() and they want to get it right for classes? That would give
> an indication of how serious we should take breaking current best practice.
>
> I'm not sure how to figure that out. Off the top of my head, the only
> current third-party packages I can think of that uses annotations are mypy
> and attrs. I took a quick look at mypy but I can't figure out what it's
> doing.
>

Mypy is irrelevant because it reads your source code -- it doesn't ever run
your code to inspect `__annotations__`.

attrs does something a little kooky. It access __annotations__ using a
> function called _has_own_attributes(), which detects whether or not the
> object is inheriting an attribute. But it doesn't peek in __dict__,
> instead it walks the mro and sees if any of its base classes have the same
> (non-False) value for that attribute.
>
>
> https://github.com/python-attrs/attrs/blob/a025629e36440dcc27aee0ee5b04d6523bcc9931/src/attr/_make.py#L343
>
> Happily, that seems like it would continue to work even if PEP 649 is
> accepted. That's good news!
>

I wonder how much pain it cost to develop that.

Another example of a well-known library that presumably does something
clever with annotations at runtime is Pydantic. I've not looked into it
more.

There are people who routinely search many GitHub repos for various
patterns. Maybe one of them can help? (I've never tried this but IIRC Irit
showed me some examples.)

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 1/18/21 5:33 PM, Guido van Rossum wrote:

> There's a secret though. `cls.__dict__` is not actually a dict -- is a mappingproxy. The proxy exists because we want to
> be able to intercept changes to class attributes such as `__add__` or `__getattribute__` in order to manipulate the
> C-level wrappers that implement such overloads.
>
> So *perhaps* we could expand the mappingproxy class to trap read access to `__annotations__` as a key to do your
> bidding. (The trick might be exposed by things like .keys()  but that doesn't bother me as much.)
>
> I honestly don't know how the mappingproxy and `__prepare__` interact.

`__prepare__` returns a dict-like namespace that is used as is. `EnumMeta` uses `__prepare__` to return an instance of
`_EnumDict`.

When `type.__new__` is called, whatever the namespace used to be is then converted into a normal Python dict, and a
mappingproxy is returned for all further `cls.__dict__` requests.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4RO6ZBRBPXCR6R5OTYCJXCO7PZWDGNEK/
Code of Conduct: http://python.org/psf/codeofconduct/

On 1/18/21 5:53 PM, Ethan Furman wrote:

> `__prepare__` returns a dict-like namespace that is used as is.  `EnumMeta` uses `__prepare__` to return an instance of
> `_EnumDict`.
>
> When `type.__new__` is called, whatever the namespace used to be is then converted into a normal Python dict, and a
> mappingproxy is returned for all further `cls.__dict__` requests.

To be more precise, when `__prepare__` is called, there is no class, and therefore no `class.__dict__`.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ABLK4FKI357SPCSKCYNG4BDLKD5UJHPU/
Code of Conduct: http://python.org/psf/codeofconduct/

As long as I'm gravedigging old conversations...!  Remember this one,
also from January of this year?  Here's a link to the thread in the
c.l.p-d Mailman archive.  The first message in the thread is a good
overview of the problem:

https://mail.python.org/archives/list/python-dev@python.org/thread/AWKVI3NRCHKPIDPCJYGVLW4HBYTEOQYL/


Here's kind of where we left it:

On 1/12/21 7:48 PM, Guido van Rossum wrote:
> On Tue, Jan 12, 2021 at 6:35 PM Larry Hastings <larry@hastings.org
> <mailto:larry@hastings.org>> wrote:
>
> On 1/12/21 5:28 PM, Brett Cannon wrote:
>> The other thing to keep in mind is we are talking about every
>> module, class, and function getting 64 bytes ... which I bet
>> isn't that much.
>
> Actually it's only every module and class.  Functions don't have
> this problem because they've always stored __annotations__
> internally--meaning, peeking in their __dict__ doesn't work, and
> they don't support inheritance anyway.  So the number is even
> smaller than that.
>
> If we can just make __annotations__ default to an empty dict on
> classes and modules, and not worry about the memory consumption,
> that goes a long way to cleaning up the semantics.
>
>
> I would like that very much. And the exception for functions is
> especially helpful.


First of all, I've proposed a function that should also help a lot:

https://bugs.python.org/issue43817

The function will be called inspect.get_annotations(o).  It's like
typing.get_type_hints(o) except less opinionated.  This function would
become the best practice for everybody who wants annotations**, like so:

import inspect
if hasattr(inspect, "get_annotations"):
    how_i_get_annotations = inspect.get_annotations
else:
    # do whatever it was I did in Python 3.9 and before...


** Everybody who specifically wants /type hints/ should instead call
typing.get_type_hints(), and good news!, /that/ function has existed for
several versions now.  So they probably already /do/ call it.


I'd still like to add a default empty __annotations__ dict to all
classes and modules for Python 3.10, for everybody who doesn't switch to
using this as-yet-unwritten inspect.get_annotations() function.  The
other changes I propose in that thread (e.g. deleting __annotations__
always throws TypeError) would be nice, but honestly they aren't high
priority.  They can wait until after Python 3.10.  Just these these two
things (inspect.get_annotations() and always populating __annotations__
for classes and modules) would go a long way to cleaning up how people
examine annotations.

Long-term, hopefully we can fold the desirable behaviors of
inspect.get_annotations() into the language itself, at which point we
could probably deprecate the function.  That wouldn't be until a long
time from now of course.


Does this need a lot of discussion, or can I just go ahead with the bpo
and PR and such?  I mean, I'd JFDI, as Barry always encourages, but
given how much debate we've had over annotations in the last two weeks,
I figured I should first bring it up here.


Happy two-weeks'-notice,


//arry/

p.s. I completely forgot about this until just now--sorry.  At least I
remembered before Python 3.10b1!

This is happening, right? Adding a default `__annotations = {}` to modules
and classes. (Though https://bugs.python.org/issue43901 seems temporarily
stuck.)

On Mon, Apr 19, 2021 at 10:10 PM Larry Hastings <larry@hastings.org> wrote:

>
>
> As long as I'm gravedigging old conversations...! Remember this one, also
> from January of this year? Here's a link to the thread in the c.l.p-d
> Mailman archive. The first message in the thread is a good overview of the
> problem:
>
>
> https://mail.python.org/archives/list/python-dev@python.org/thread/AWKVI3NRCHKPIDPCJYGVLW4HBYTEOQYL/
>
>
> Here's kind of where we left it:
>
> On 1/12/21 7:48 PM, Guido van Rossum wrote:
>
> On Tue, Jan 12, 2021 at 6:35 PM Larry Hastings <larry@hastings.org> wrote:
>
>> On 1/12/21 5:28 PM, Brett Cannon wrote:
>>
>> The other thing to keep in mind is we are talking about every module,
>> class, and function getting 64 bytes ... which I bet isn't that much.
>>
>> Actually it's only every module and class. Functions don't have this
>> problem because they've always stored __annotations__ internally--meaning,
>> peeking in their __dict__ doesn't work, and they don't support inheritance
>> anyway. So the number is even smaller than that.
>>
>> If we can just make __annotations__ default to an empty dict on classes
>> and modules, and not worry about the memory consumption, that goes a long
>> way to cleaning up the semantics.
>>
>
> I would like that very much. And the exception for functions is especially
> helpful.
>
>
> First of all, I've proposed a function that should also help a lot:
>
> https://bugs.python.org/issue43817
>
> The function will be called inspect.get_annotations(o). It's like
> typing.get_type_hints(o) except less opinionated. This function would
> become the best practice for everybody who wants annotations**, like so:
>
> import inspect
> if hasattr(inspect, "get_annotations"):
> how_i_get_annotations = inspect.get_annotations
> else:
> # do whatever it was I did in Python 3.9 and before...
>
>
> ** Everybody who specifically wants *type hints* should instead call
> typing.get_type_hints(), and good news!, *that* function has existed for
> several versions now. So they probably already *do* call it.
>
>
> I'd still like to add a default empty __annotations__ dict to all classes
> and modules for Python 3.10, for everybody who doesn't switch to using this
> as-yet-unwritten inspect.get_annotations() function. The other changes I
> propose in that thread (e.g. deleting __annotations__ always throws
> TypeError) would be nice, but honestly they aren't high priority. They can
> wait until after Python 3.10. Just these these two things
> (inspect.get_annotations() and always populating __annotations__ for
> classes and modules) would go a long way to cleaning up how people examine
> annotations.
>
> Long-term, hopefully we can fold the desirable behaviors of
> inspect.get_annotations() into the language itself, at which point we could
> probably deprecate the function. That wouldn't be until a long time from
> now of course.
>
>
> Does this need a lot of discussion, or can I just go ahead with the bpo
> and PR and such? I mean, I'd JFDI, as Barry always encourages, but given
> how much debate we've had over annotations in the last two weeks, I figured
> I should first bring it up here.
>
>
> Happy two-weeks'-notice,
>
>
> */arry*
>
> p.s. I completely forgot about this until just now--sorry. At least I
> remembered before Python 3.10b1!
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/J4LZEIZTYZQWGIM5VZGNMQPWB5ZWVEXP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 4/23/21 9:26 PM, Guido van Rossum wrote:
> This is happening, right? Adding a default `__annotations = {}` to
> modules and classes. (Though https://bugs.python.org/issue43901
> <https://bugs.python.org/issue43901> seems temporarily stuck.)


It's happening, and I wouldn't say it's stuck.  I'm actively working on
it--currently puzzling my way through some wild unit test failures.  I
expect to ship my first PR over the weekend.


Cheers,


//arry/

I've hit a conceptual snag in this.

What I thought I needed to do: set __annotations__= {} in the module
dict, and set __annotations__= {} in user class dicts.  The latter was
more delicate than the former but I think I figured out a good spot for
both.  I have this much working, including fixing the test suite.

But now I realize (*head-slap* here): if *every* class is going to have
annotations, does that mean builtin classes too?  StructSequence classes
like float? Bare-metal type objects like complex?  Heck, what about type
itself?!

My knee-jerk initial response: yes, those too.  Which means adding a new
getsetdef to the type object.  But that's slightly complicated.  The
point of doing this is to preserve the existing best-practice of peeking
in the class dict for __annotations__, to avoid inheriting it.  If I'm
to preserve that, the get/set for __annotations__ on a type object would
need to get/set it on tp_dict if tp_dict was not NULL, and use internal
storage somewhere if there is no tp_dict.

It's worth noticing that builtin types don't currently have
__annotations__ set, and you can't set them. (Or, at least, float,
complex, and type didn't have them set, and wouldn't let me set
annotations on them.)  So presumably people using current best
practice--peek in the class dict--aren't having problems.

So I now suspect that my knee-jerk answer is wrong.  Am I going too far
down the rabbit hole?  Should I /just/ make the change for user classes
and leave builtin classes untouched?  What do you think?


Cheers,


//arry/

On Sat, 24 Apr 2021, 5:53 pm Larry Hastings, <larry@hastings.org> wrote:

>
> So I now suspect that my knee-jerk answer is wrong. Am I going too far
> down the rabbit hole? Should I *just* make the change for user classes
> and leave builtin classes untouched? What do you think?
>

I'd suggest kicking the can down the road: leave builtin classes alone for
now, but file a ticket to reconsider the question for 3.11.

In the meantime, inspect.get_annotations can help hide the discrepancy.

Cheers,
Nick.


>
> Cheers,
>
>
> */arry*
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IK7IWUCTESD5OZE47J45EY3FRVM7GEKM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On 24. 04. 21 9:52, Larry Hastings wrote:
> I've hit a conceptual snag in this.
>
> What I thought I needed to do: set __annotations__= {} in the module
> dict, and set __annotations__= {} in user class dicts.  The latter was
> more delicate than the former but I think I figured out a good spot for
> both.  I have this much working, including fixing the test suite.
>
> But now I realize (*head-slap* here): if *every* class is going to have
> annotations, does that mean builtin classes too?  StructSequence classes
> like float? Bare-metal type objects like complex?  Heck, what about type
> itself?!
>
> My knee-jerk initial response: yes, those too.  Which means adding a new
> getsetdef to the type object.  But that's slightly complicated.  The
> point of doing this is to preserve the existing best-practice of peeking
> in the class dict for __annotations__, to avoid inheriting it.  If I'm
> to preserve that, the get/set for __annotations__ on a type object would
> need to get/set it on tp_dict if tp_dict was not NULL, and use internal
> storage somewhere if there is no tp_dict.
>
> It's worth noticing that builtin types don't currently have
> __annotations__ set, and you can't set them. (Or, at least, float,
> complex, and type didn't have them set, and wouldn't let me set
> annotations on them.)  So presumably people using current best
> practice--peek in the class dict--aren't having problems.
>
> So I now suspect that my knee-jerk answer is wrong.  Am I going too far
> down the rabbit hole?  Should I /just/ make the change for user classes
> and leave builtin classes untouched?  What do you think?

Beware of adding mutable state to bulit-in (C static) type objects:
these are shared across interpreters, so changing them can “pollute”
unwanted contexts.

This has been so for a long time [0]. There are some subinterpreter
efforts underway that might eventually lead to making __annotations__ on
static types easier to add, but while you're certainly welcome to
explore the neighboring rabbit hole as well, I do think you're going in
too far for now :)

[0]
https://mail.python.org/archives/list/python-dev@python.org/message/KLCZIA6FSDY3S34U7A72CPSBYSOMGZG3/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3GIZO2R2IRIN47THXRWAZKEQ5JBFRITP/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/24/21 7:11 AM, Nick Coghlan wrote:
> On Sat, 24 Apr 2021, 5:53 pm Larry Hastings, <larry@hastings.org
> <mailto:larry@hastings.org>> wrote:
>
>
> So I now suspect that my knee-jerk answer is wrong.  Am I going
> too far down the rabbit hole? Should I /just/ make the change for
> user classes and leave builtin classes untouched?  What do you think?
>
>
> I'd suggest kicking the can down the road: leave builtin classes alone
> for now, but file a ticket to reconsider the question for 3.11.
>
> In the meantime, inspect.get_annotations can help hide the discrepancy.


The good news: inspect.get_annotations() absolutely can handle it. 
inspect.get_annotations() is so paranoid about examining the object you
pass in, I suspect you could pass in an old boot and it would pull out
the annotations--if it had any.

Cheers,


//arry/

On 4/24/21 8:09 AM, Petr Viktorin wrote:
> On 24. 04. 21 9:52, Larry Hastings wrote:
>> I've hit a conceptual snag in this.
>>
>> What I thought I needed to do: set __annotations__= {} in the module
>> dict, and set __annotations__= {} in user class dicts. The latter was
>> more delicate than the former but I think I figured out a good spot
>> for both.  I have this much working, including fixing the test suite.
>>
>> But now I realize (*head-slap* here): if *every* class is going to
>> have annotations, does that mean builtin classes too? StructSequence
>> classes like float? Bare-metal type objects like complex?  Heck, what
>> about type itself?!
>>
>> My knee-jerk initial response: yes, those too.  Which means adding a
>> new getsetdef to the type object.  But that's slightly complicated. 
>> The point of doing this is to preserve the existing best-practice of
>> peeking in the class dict for __annotations__, to avoid inheriting
>> it.  If I'm to preserve that, the get/set for __annotations__ on a
>> type object would need to get/set it on tp_dict if tp_dict was not
>> NULL, and use internal storage somewhere if there is no tp_dict.
>>
>> It's worth noticing that builtin types don't currently have
>> __annotations__ set, and you can't set them. (Or, at least, float,
>> complex, and type didn't have them set, and wouldn't let me set
>> annotations on them.)  So presumably people using current best
>> practice--peek in the class dict--aren't having problems.
>>
>> So I now suspect that my knee-jerk answer is wrong.  Am I going too
>> far down the rabbit hole?  Should I /just/ make the change for user
>> classes and leave builtin classes untouched?  What do you think?
>
> Beware of adding mutable state to bulit-in (C static) type objects:
> these are shared across interpreters, so changing them can “pollute”
> unwanted contexts.
>
> This has been so for a long time [0]. There are some subinterpreter
> efforts underway that might eventually lead to making __annotations__
> on static types easier to add, but while you're certainly welcome to
> explore the neighboring rabbit hole as well, I do think you're going
> in too far for now :)
>
> [0]
> https://mail.python.org/archives/list/python-dev@python.org/message/KLCZIA6FSDY3S34U7A72CPSBYSOMGZG3/


That's a good point!  The sort of detail one forgets in the rush of the
moment.

Given that the lack of annotations on builtin types already isn't a
problem, and given this wrinkle, and generally given the "naw you don't
have to" vibe I got from you and Nick (and the lack of "yup you gotta" I
got from anybody else), I'm gonna go with not polluting the builtin
types for now.

This is not to say that, in the fullness of time, those objects should
never have annotations.  Even in the three random types I picked in my
example, there's at least one example: float.imag is a data member and
might theoretically be annotated.  But we can certainly kick this can
down the road too.  Maybe by the time we get around to it, we'll have a
read-only dictionary we can use for the purpose.


Cheers,


//arry/

On Sat, Apr 24, 2021 at 2:25 PM Larry Hastings <larry@hastings.org> wrote:

> This is not to say that, in the fullness of time, those objects should
> never have annotations. Even in the three random types I picked in my
> example, there's at least one example: float.imag is a data member and
> might theoretically be annotated. But we can certainly kick this can down
> the road too. Maybe by the time we get around to it, we'll have a
> read-only dictionary we can use for the purpose.
>

We already have one -- the mappingproxy type you get back from a class'
__dict__ attribute. (Though in the fullness of times, type objects
presumably won't be shared between multiple interpreters, which solves the
problem in a different way.)

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>