Mailing List Archive

The current state of typing PEPs
Does PEP 563 or 649 satisfy static and dynamic typing needs?

In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs.

Defer decision on PEP 563 and 649 in 3.11

As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users.

Calling for help from static and dynamic typing users

This is also a call to you, the folks who both care deeply about typing in Python, and have a solid understanding of the subject matter (or willingness to learn) to help us! There are use cases we don’t know about, and unclear requirements from both the static and dynamic typing users. We need help in defining those requirements clearly, in uncovering the use cases we’re not aware of, and most importantly, being an interface to typing enthusiasts to help build consensus with either of the proposed PEPs, or some other solution that doesn’t yet have a PEP (such as suggestions made by ?ukasz in his blog post here: https://lukasz.langa.pl/61df599c-d9d8-4938-868b-36b67fdb4448/).

Volunteer to be a typing PEP shepherd

We’re looking for someone(s) who can be neutral as to the solution, knowledgeable enough to understand all sides of the problem, willing to serve as not quite a PEP author, not a PEP Delegate, but more than a PEP shepherd. If you are that person, please let us know!

Cheers,
-Barry (on behalf of the Python Steering Council)

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On 11/17/2021 5:47 PM, Barry Warsaw wrote:
> Does PEP 563 or 649 satisfy static and dynamic typing needs?
>
> In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs.
>
> Defer decision on PEP 563 and 649 in 3.11
>
> As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users.

"In the face of ambiguity, refuse the temptation to guess."

I appreciate the carefully articulated reasoning being given with these
PEP decisions.

--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/S46JI4E3OZJUUSVZLDPAP5A5MOMAMD5O/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
Thanks to Barry and the SC for giving us this update.

> In the interest of full transparency, we want to let the Python community
know that the Steering Council continues to discuss PEP 563 (Postponed
Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations
Using Descriptors).

<snip>

I want to draw attention to this point:

> all the use cases and requirements across the static and dynamic typing
constituents.

TL;DR:

Annotations can be, and are, used for other things than "typing". I just
noticed that PEP 563 apparently deprecated those other uses (well, sort of:
"uses for annotations incompatible with the aforementioned PEPs should be
considered deprecated"), but if the SC is reconsidering PEP 563, then it
would be nice to be clear about whether non-typing uses of annotations are
indeed deprecated. If not, then the challenge is to come up with a way
forward that not only supports both static and dynamic typing, but also
other potentially arbitrary use cases.

And now onto more detail ...

One example is a use case of mine -- I have built a hierarchical object
system, built on dataclasses, in which the annotation absolutely has to be
an actual type(class) object. PEP 563 will very much break this use case.
Of course, there are other ways to write that code, but I'm pretty sure it
would require a complete change to the API. I'm not entirely sure if PEP
646 would work for my use case -- likely it would, at least with modest
modifications, rather than a new API.

I have no idea how many other similar use cases are out in the wild, but we
know that the Pydantic project has concerns.

To be fair -- I wrote all my code after PEP 563 was accepted. And I only
just now read it carefully, and found that it says:

"With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated."

If that's the case, then that's the case, and I'll deal, but it is
disappointing.

But I suspect I'm not alone in not really noticing that statement, so I
think it's worth it for the SC to explicitly reiterate at this point that
all non-typing uses of annotations are depreciated, if indeed, that is
still the intent. Also, that statement is not entirely clear. In fact, I
think my use case IS compatible with the "aforementioned PEPs", it's just
not compatible with PEP 653 itself. It all depends on what is meant by
"compatible".

I would like to provide a bit of perspective on this. I was criticised on
this list a while back for, essentially, not paying attention and then
complaining later. After all, PEP 563 was accepted over four years ago.
Fair enough.

But the fact is that I, among others, have been a bit uncomfortable about
the focus on typing in Python for years. But when issues are raised, we
have been repeatedly told that typing is, and always will remain, optional.
In short, it was made clear that anyone not interested in typing could
safely ignore the discussions about it. And thus, a number of PEPs came and
went, and those among us that did not choose to involve ourselves in the
conversation did not pay attention to the details.

And thus we didn't notice that buried in what seemed like a typing PEP,
was, in fact, a depreciation of any non-typing uses of an existing Python
feature. (also to be fair, the title of the PEP is "Postponed Evaluation of
Annotations" -- so should have caught the attention of anyone interested in
annotations for any use.

In my personal case, I didn't notice it at the time, as I wasn't using
annotations for anything at all until fairly recently. In fact, it was the
introduction of dataclasses, which I think are the first use of annotations
in the standard library, that drew my attention. dataclasses, as I'm sure
you all know, use the presence of an annotation on a class attribute to
define a field.

This is a pretty nifty use of annotations as an API for defining the
fields. I liked that, and decided to use it.

dataclasses create a Field object, which has an attribute called "type",
that gets populated with the annotation's value.

PEP 563 would very much change what gets put in the "type" attribute of a
Field.

Eric V. Smith has indicated that he never intended that to actually be a
type, but rather was intended to be whatever the annotation was, so that
PEP 563 wouldn't change anything. I'm happy to take his word for it (and
PEP 567 says that the actual type is not used or modified by dataclasses),
but as users, all we have to go on is the documentation and the choice of
names, which very much gave the impression that a Field.type would be a
type -- at least if a type was set in the annotation.

I also notice that PEP 567 (dataclasses) and PEP 563 were both developed
and approved at about the same time. So I suspect the impact of PEP 563 was
not considered when designing or documenting dataclasses. However, PEP 567
does make a number of references to typing, and was clearly written with
the idea in mind that the annotations would, in fact, be used for type
analysis. But again, no mention of PEP 563, or what might be in the
field.type attribute -- one might wonder why it was there at all if it was
not intended to be used.

We need help in defining those requirements clearly, in uncovering the use
> cases we’re not aware of, and most importantly, being an interface to
> typing enthusiasts


Again -- is it only "typing enthusiasts" that you want to engage? Or "users
of annotations"? -- maybe it is, but it would be nice if that was a clear
statement from the SC.

- CHB


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Re: The current state of typing PEPs [ In reply to ]
On Sun, 21 Nov 2021 at 07:50, Christopher Barker <pythonchb@gmail.com> wrote:
>
> TL;DR:
>
> Annotations can be, and are, used for other things than "typing". I just noticed that PEP 563 apparently deprecated those other uses (well, sort of: "uses for annotations incompatible with the aforementioned PEPs should be considered deprecated"), but if the SC is reconsidering PEP 563, then it would be nice to be clear about whether non-typing uses of annotations are indeed deprecated. If not, then the challenge is to come up with a way forward that not only supports both static and dynamic typing, but also other potentially arbitrary use cases.

I agree with the points made in this post. It's becoming harder and
harder for people not particularly interested in static typing to
simply ignore it, and any use of annotations to affect runtime
behaviour is in a weird grey area. And as a library author, I'm now
finding that I'm getting requests to add typing to my code "for my
users" (i.e., using types is no longer just a choice I make for my
project, it's an API design issue).

So I too would appreciate clarity on where annotations, and more
generally typing, stand as Python language features¹.

Paul

¹ In particular, typing features seem to change so rapidly that
supporting a wide range of Python versions is a real pain (in
practical terms - not getting syntax errors is straightforward, but
writing type annotations that work well across versions very
definitely isn't, at least for someone who doesn't really care about
static typing).
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AMGOAG22UJXCFPQ2QCGH57RNB3UTAX7H/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On 21/11/2021 11:04, Paul Moore wrote:
> On Sun, 21 Nov 2021 at 07:50, Christopher Barker <pythonchb@gmail.com> wrote:
> It's becoming harder and harder for people not particularly interested
> in static typing to simply ignore it
(Raises hand.)  +1
Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YMZ4UE2TMRH5WNF6Y3KWGJS2R2YJQAZI/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
Executive summary:

The typing-suspicious crowd has a valid complaint about PEPs 563 and
649, but it's not that they weren't warned.

Christopher Barker writes:

> Annotations can be, and are, used for other things than "typing". I
> just noticed that PEP 563 apparently deprecated those other uses
> (well, sort of: "uses for annotations incompatible with the
> aforementioned PEPs should be considered deprecated")

Not a PEP proponent (or even a typing user), but I thought this had
been made clear long ago. My understanding is that optional,
incremental type hints are and have always been considered the primary
use case for annotations by the BDFL and AFAICT the SC following the
BDFL. If compatibility with typing is an issue, then the burden of
implementing that is on the other application. Typing *might* do
something to help, but it's not obligated to do so.

> One example is a use case of mine -- I have built a hierarchical
> object system, built on dataclasses, in which the annotation
> absolutely has to be an actual type(class) object. PEP 563 will
> very much break this use case. [...]

> I'm not entirely sure if PEP 649 would work for my use case --
> likely it would, at least with modest modifications, rather than a
> new API.

If PEP 649 works for you, I expect you could work with PEP 563 by
moving Larry's data descriptor concept out of the compiler and into
your library code.

You could also appeal to the "might do something" caveat above. Yury
suggested providing both the immediately evaluated object (if there
were no forward references, I guess) and the string representation,
which would be backward compatible with your use case. The idea was
rejected on the basis that it doesn't help with the primary use in
typing, and doesn't remove the need for a __future__ import during the
transition period. You could ask that that decision be revisited,
since it would ensure that uses of __annotations__ like yours continue
to work.

> But I suspect I'm not alone in not really noticing that statement,

I'm not surprised people haven't noticed that statement. I am
surprised that a lot of folks haven't noticed that this comes up
occasionally and the answer every time is "we're not going to go out
of our way to break other use cases, but typing *is* the primary use
case and will take precedence if the question comes up".

I will grant that restricting the type of compiled annotations from
typing.Any to "string serializing an object's AST" could reasonably be
said to be "going out of your way to break other use cases". You have
a valid beef here, although it's not obvious to me how it should be
resolved.

So I'm not saying that you don't have an interest in the future
semantics of this dunder. I am +1 on Python providing relief for your
use case.

> But the fact is that I, among others, have been a bit uncomfortable
> about the focus on typing in Python for years.

You mean y'all are uncomfortable with the popularity of typing. You
wouldn't care if it wasn't used outside of hugely complex proprietary
codebases you'll never see.

Thing is, reasoning about programs is What We Do as programmers, and
efforts to make it easier to do that for complex programs, and to
provide software to help with that task, are here to stay and they
will be used in mixed company. Even in Python. (And that's why, as
someone who doesn't use typing, I support typing. Anything that helps
*other people* to write better code, I can get behind. My code, I'm
going to write as incompetently as ever.[1] ;-)

> But when issues are raised, we have been repeatedly told that
> typing is, and always will remain, optional.

This has always been in the context of *your* source code. For
example, nobody ever claimed you could keep it out of anybody else's
source code (although that's clearly what a lot of those "raising
issues" want!) Still, features like stub files were provided so
typists could work and play nicely with non-typists if they wanted to
(and many typists want that, themselves).

But dunders are the property of the language (or sometimes the
implementation), and they always have been. If you use them in a way
that's not documented to work, you're at risk even if it happens to
work now. PEP 3107 didn't document that anything would work. ;-)


Footnotes:
[1] Barry, do not tell Abhilash I wrote that.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2O3JHKJ2CM7OFNGT664WXLN2DKU3YGIS/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On 25/11/2021 15:15, Stephen J. Turnbull wrote:
> Executive summary:
>
> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>
>
“As you will no doubt be aware, the plans for development of the
outlying regions of the Galaxy require the building of a hyperspatial
express route through your star system, and regrettably your planet is
one of those scheduled for demolition. The process will take slightly
less than two of your Earth minutes. Thank you.”

Well we were warned, so everything's fine.
Rob Cliffe
Re: The current state of typing PEPs [ In reply to ]
On Sat, Nov 20, 2021 at 11:46:56PM -0800, Christopher Barker wrote:

> One example is a use case of mine -- I have built a hierarchical object
> system, built on dataclasses, in which the annotation absolutely has to be
> an actual type(class) object. PEP 563 will very much break this use case.

I don't think that's an insurmountable problem. I think that all you
need is a small class decorator to evaluate the stringified annotations
back to the real things. The PEP tells us the right way to evaluate
annotations, so all(?) you need is a decorator to do that to each method
in your class, and Bob's your uncle.

Maybe PEP 563 could include a decorator in the typing module to
destringify all the annotations in a class or function?

As far as I can see from a brief scan of the PEP, and based on knowing
next to nothing about your use-case, the only hypothetical problem might
be this line in the PEP:

"Consequently, using local state in annotations is no longer possible in
general."

but it's not entirely clear to me what ?ukasz means by that, or whether
it will affect your use-case.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3RYHEEARXUCK3S6WIZALAU75PEPG7GR2/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
First:

At this point, I am not advocating anything in particular. I am just asking
that the SC makes a clear statement about the intention at this point.

From Barry's email:
"""
This is also a call to you, the folks who both care deeply about typing in
Python, and have a solid understanding of the subject matter (or
willingness to learn) to help us! There are use cases we don’t know about,
and unclear requirements from both the static and dynamic typing users.
"""
I don't care deeply about typing, and I'm not a static or dynamic typing
user. So this implies that a decision has been made that annotations are
for typing, and typing only.

(not that you're not allowed to use for anything else, of course you are,
but that other uses won;t be taken into account when designing the new
interface)

But I have never seen that clearly stated anywhere. The closest is from PEP
563, where it says:

"""
With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.
"""

Which pretty much makes the point, but it's a bit subtle -- what does
"incompatible' mean?

On Thu, Nov 25, 2021 at 7:15 AM Stephen J. Turnbull <
stephenjturnbull@gmail.com> wrote:

> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>

Well, "typing-suspicious" is part of it, but the real problem is folks that
might be using annotations for non-typing purposes.

As for warning: yes and no -- what's new here is that as far as I know,
this is the first time since the py3 transition that a __future__ import
would be made standard behavior. And you'd have to be paying a lot of
attention to know that was going to happen. And I suspect that only folks
involved in the development of typing tools were paying that much attention.

In the case of the py3 transition -- it was very well known and documented
that lots of things would be changing, and we had a very long transition
period.

But the challenge with this is that there is no way to raise a depreciation
warning for not using a __future__ import. So while we all should have been
testing our code with:

from __future__ import annotations

For the last few versions, I doubt that very many people not developing
typing tools were doing that.

> Annotations can be, and are, used for other things than "typing". I
> > just noticed that PEP 563 apparently deprecated those other uses
> > (well, sort of: "uses for annotations incompatible with the
> > aforementioned PEPs should be considered deprecated")
>
> Not a PEP proponent (or even a typing user), but I thought this had
> been made clear long ago. My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL and AFAICT the SC following the
> BDFL.


well no -- not "always" -- look at PEP 3107, when annotations were first
introduced:

"By itself, Python does not attach any particular meaning or significance
to annotations."

and it goes on to give non-typing examples.

Since then, the shift from "nothing more than a way of associating
arbitrary Python expressions with various parts of a function at
compile-time" to a system for defining types has been gradual, with a
number of PEPs, and, as far as i can tell, PEP 563 is the first time that
a change was proposed that would affect other uses for annotations.

Anyway, that is water under the bridge. But the reason I bring it up is
because of all the many thousands of Python users, I pay probably more
attention to changes than most -- I have followed Python-ideas and
Python-dev for years. So I'm guessing that there are other folks out there
that will be surprised if and when the behavior of annotations changes.


> If compatibility with typing is an issue, then the burden of
> implementing that is on the other application.


That does indeed seem to be the current intent. And I'm happy to deal with
that -- I just want it to be clear that I (and others) have to wait and see
how it shakes out, and then figure out what to do, or if I should
contribute to the conversation now. If non-typing use cases of annotations
are still considered important, then we should encourage folks like me to
be part of the conversation.


> > I'm not entirely sure if PEP 649 would work for my use case --
> > likely it would, at least with modest modifications, rather than a
> > new API.
>
> If PEP 649 works for you, I expect you could work with PEP 563 by
> moving Larry's data descriptor concept out of the compiler and into
> your library code.
>

That would indeed require maybe not a change to the API, but certainly some
complex code. But yeah, probably doable one way or another.


> > But I suspect I'm not alone in not really noticing that statement,
>
> I am
> surprised that a lot of folks haven't noticed that this comes up
> occasionally and the answer every time is "we're not going to go out
> of our way to break other use cases, but typing *is* the primary use
> case and will take precedence if the question comes up".
>

I think the "breaking other use cases" has been subtle to those not paying
attention to typing.


> > But the fact is that I, among others, have been a bit uncomfortable
> > about the focus on typing in Python for years.
>
> You mean y'all are uncomfortable with the popularity of typing. You
> wouldn't care if it wasn't used outside of hugely complex proprietary
> codebases you'll never see.
>

well, yes. The issue is that, intended or not, typing is making it's way
into Python culture. As an instructor of beginning python users, I am
unsure at this point when to introduce type annotations.

What is their role? Up to today, I have treated them as an advanced
feature, useful for "complex codebases". But there are any number of
examples springing up on the internet, to the point where many students now
think they are "best practice", if not actually required.

But that's getting pretty OT here.

> Anything that helps

> *other people* to write better code, I can get behind. My code, I'm
> going to write as incompetently as ever.[1] ;-)
>

me too -- actually, not quite. I write a lot of "scripts" that I will
continue to keep simple, and I also develop complex systems -- and those
require more care.


> > But when issues are raised, we have been repeatedly told that
> > typing is, and always will remain, optional.
>
> This has always been in the context of *your* source code.


Sure -- my point with that is that if I don't want to use typing in my
code, then I didn't have to pay attention to the development of the typing
systems.

But my point here is that what changed with PEP 563 is that while typing
is still optional, this is the first time that the language itself may be
changed to accommodate typing -- so it can no longer be completely ignored.

But dunders are the property of the language (or sometimes the
> implementation), and they always have been. If you use them in a way
> that's not documented to work, you're at risk even if it happens to
> work now. PEP 3107 didn't document that anything would work. ;-)
>

no, but it did document that annotations were "arbitrary Python
expressions" -- that is changed in PEP 563 -- intentionally, but it is
changing something about the language that was documented.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Re: The current state of typing PEPs [ In reply to ]
On 26/11/21 4:15 am, Stephen J. Turnbull wrote:
> My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL

I'm not sure that's true. The way I remember it, back when
annotations were first introduced, the BDFL didn't say that,
or at least didn't say it very clearly. It sounded more like
type hints were just one of many possible uses, and he
encouraged people to experiment. There were even discussions
about coming up with a convention to manage conflicting uses
of annotations in the same code. That wouldn't have happened
if typing were considered the only supported use.

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/R5TWU6HTOKLCTY2QZDGEAR74TGHZHQQ4/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Thu, 25 Nov 2021 at 15:16, Stephen J. Turnbull
<stephenjturnbull@gmail.com> wrote:
>
> Executive summary:
>
> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>
> Christopher Barker writes:
>
> > Annotations can be, and are, used for other things than "typing". I
> > just noticed that PEP 563 apparently deprecated those other uses
> > (well, sort of: "uses for annotations incompatible with the
> > aforementioned PEPs should be considered deprecated")
>
> Not a PEP proponent (or even a typing user), but I thought this had
> been made clear long ago. My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL and AFAICT the SC following the
> BDFL. If compatibility with typing is an issue, then the burden of
> implementing that is on the other application. Typing *might* do
> something to help, but it's not obligated to do so.

This was not my understanding of annotations when they were introduced e.g.:
https://www.python.org/dev/peps/pep-3107/#use-cases

As I remember it, a decision about the purpose of annotations was
*explicitly* not made when they were introduced. It was clear that
typing was a major potential use and then at some point (around about
the introduction of the typing module) there seemed to be a shift in
people's understanding of what annotations were for. Eventually that
reached the point that people who were particularly interested in
typing had no memory of the fact that the purpose of annotations had
not really been specified as being about typing in the first place.

It looks to me like Chris has identified in PEP 563 what is
potentially the earliest reference (in an accepted PEP) to the idea
that non-typing uses of annotations are to be discouraged.

--
Oscar
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BERGEZTXG4XSEIJQHQWC3IZLXYEZPZ6C/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
My 2¢ from a position of *extreme ignorance*, not to mention zero
interest in annotations (all welcome to shoot me down in flames, or
treat me with contemptuous silence.  But *occasionall*y the "idiot's"
point of view is worth considering, so here goes).

ISTM that typing/annotations have been allowed to drift without a clear
end in view, rather like a ship that is steered in one direction by one
person, then in another by someone else, with nobody knowing what the
destination port is.  As witness the conflicting views in this thread. 
At the risk of being rude, the phrase "headless chickens" comes to mind.
ISTM that it is time to take a step back and decide on a definite
policy.  Perhaps a definitive pronouncement from Guido.  Or the SC. Or a
discussion from all parties that reaches an acceptable conclusion.  Then
stick to it.  Even if it causes some backward incompatibility - ISTM
that things have changed so rapidly that this really would be an
acceptable evil if it results in clarity for the future.  Meanwhile, do
not change anything, do not approve any PEPs, until this conclusion is
reached.
I apologise if I am being presumptuous in commenting on a subject I know
almost nothing about.

/I'm just a soul whose intentions are good,//
//Oh Lord, please don't let me be misunderstood.//
/
Best wishes
Rob Cliffe


On 25/11/2021 23:51, Oscar Benjamin wrote:
> On Thu, 25 Nov 2021 at 15:16, Stephen J. Turnbull
> <stephenjturnbull@gmail.com> wrote:
>> Executive summary:
>>
>> The typing-suspicious crowd has a valid complaint about PEPs 563 and
>> 649, but it's not that they weren't warned.
>>
>> Christopher Barker writes:
>>
>> > Annotations can be, and are, used for other things than "typing". I
>> > just noticed that PEP 563 apparently deprecated those other uses
>> > (well, sort of: "uses for annotations incompatible with the
>> > aforementioned PEPs should be considered deprecated")
>>
>> Not a PEP proponent (or even a typing user), but I thought this had
>> been made clear long ago. My understanding is that optional,
>> incremental type hints are and have always been considered the primary
>> use case for annotations by the BDFL and AFAICT the SC following the
>> BDFL. If compatibility with typing is an issue, then the burden of
>> implementing that is on the other application. Typing *might* do
>> something to help, but it's not obligated to do so.
> This was not my understanding of annotations when they were introduced e.g.:
> https://www.python.org/dev/peps/pep-3107/#use-cases
>
> As I remember it, a decision about the purpose of annotations was
> *explicitly* not made when they were introduced. It was clear that
> typing was a major potential use and then at some point (around about
> the introduction of the typing module) there seemed to be a shift in
> people's understanding of what annotations were for. Eventually that
> reached the point that people who were particularly interested in
> typing had no memory of the fact that the purpose of annotations had
> not really been specified as being about typing in the first place.
>
> It looks to me like Chris has identified in PEP 563 what is
> potentially the earliest reference (in an accepted PEP) to the idea
> that non-typing uses of annotations are to be discouraged.
>
> --
> Oscar
> _______________________________________________
> Python-Dev mailing list --python-dev@python.org
> To unsubscribe send an email topython-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived athttps://mail.python.org/archives/list/python-dev@python.org/message/BERGEZTXG4XSEIJQHQWC3IZLXYEZPZ6C/
> Code of Conduct:http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Thu, Nov 25, 2021 at 3:49 PM Greg Ewing <greg.ewing@canterbury.ac.nz>
wrote:

> On 26/11/21 4:15 am, Stephen J. Turnbull wrote:
> > My understanding is that optional,
> > incremental type hints are and have always been considered the primary
> > use case for annotations by the BDFL
>
> I'm not sure that's true. The way I remember it, back when
> annotations were first introduced, the BDFL didn't say that,
> or at least didn't say it very clearly. It sounded more like
> type hints were just one of many possible uses, and he
> encouraged people to experiment. There were even discussions
> about coming up with a convention to manage conflicting uses
> of annotations in the same code. That wouldn't have happened
> if typing were considered the only supported use.
>

My memory is also hazy, but I'm quite sure that *in my mind* annotations
were intended as a compromise between conflicting proposals for *typing*.
We didn't have agreement on the syntax or semantics, but we did know we
wanted to do something with types eventually. Some folks wanted to enforce
types at runtime. Others wanted to use them to generate faster code. Yet
others wanted types to be checked by the compiler. The term "gradual
typing" wasn't invented (or hadn't reached our community) yet, and offline
static type checking wasn't something we had thought of either (I think).
But it was clear that typing would have to be optional.

Fortunately the current situation is that there are few people who want to
use annotations for non-typing, but there are still conflicting use cases,
in particular offline static type checking on the one hand (which started
with mypy and PEP 484) and some runtime use of types on the other hand (
pydantic <https://pydantic-docs.helpmanual.io/> being the main example
IIUC).

Tools like pydantic seem to have adopted the *notation* of PEP 484 fully,
but their *interpretation* is different. This was explicitly provided for
by PEP 484 (the design there carefully ensures that annotations are fully
introspectable) but it wasn't anticipated to be a big use case, hence the
mistaken belief at the time PEP 563 was accepted that stringifying
annotations would be sufficient for the introspection use case.

So I'm in agreement with Stephen, but since he left out any mention of
offline static checking, his observation isn't helping much.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Re: The current state of typing PEPs [ In reply to ]
On Fri, Nov 26, 2021 at 12:15:10AM +0900, Stephen J. Turnbull wrote:

> I will grant that restricting the type of compiled annotations from
> typing.Any to "string serializing an object's AST" could reasonably be
> said to be "going out of your way to break other use cases". You have
> a valid beef here, although it's not obvious to me how it should be
> resolved.

I don't think that's what PEP 563 says. Annotations are not *restricted*
to only be strings, it is merely that when the function or class object
is built, the annotations are left as strings.

So we can post-process annotations and destringify them:

>>> from __future__ import annotations
>>> def func(arg:float) -> int:
... pass
...
>>> func.__annotations__
{'arg': 'float', 'return': 'int'}
>>> func.__annotations__['return'] = int
>>> func.__annotations__['arg'] = float
>>> func.__annotations__
{'arg': <class 'float'>, 'return': <class 'int'>}


There may be scoping issues to be sorted out, but I don't think they are
insurmountable.


--
Steven
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/24PKPJTI5SIES32C2KZVF6R6D3RLF6BI/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
Steven D'Aprano writes:

> I don't think that's what PEP 563 says. Annotations are not
> *restricted* to only be strings, it is merely that when the
> function or class object is built, the annotations are left as
> strings.

You're right, I was imprecise. I meant as PEP 563 is implemented now,
I doubt it can handle non-strings nicely.

> So we can post-process annotations and destringify them:

True, but you face the dread "double-decode problem": if the
annotation happens to destringify to a string, then the "post-
processor" may try to destringify again. I guess we can mark each
annotation as already destringified or not. As long as this is done
by some sort of API, you're probably OK (but it makes me nervous since
__annotations__ is completely exposed to the program, as in your
example).

> There may be scoping issues to be sorted out, but I don't think they are
> insurmountable.

But isn't there still the issue of forward reference, which motivates
these PEPs in the first place?

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LX7EOMA7RPVJUTS5ADMLBIK2CZN2TMLQ/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Fri, 26 Nov 2021 at 05:14, Guido van Rossum <guido@python.org> wrote:
>
> My memory is also hazy, but I'm quite sure that *in my mind* annotations were intended as a compromise between conflicting proposals for *typing*. We didn't have agreement on the syntax or semantics, but we did know we wanted to do something with types eventually.

More hazy memories here, but I think the original proposal left open
the possibility of annotations not being types at all - for example,
being docstrings for the arguments, or option names for a "function
call to CLI" tool, etc. I think that some libraries took this approach
(in particular the "CLI builder" idea). At some point (probably around
PEP 484) it became clear that the expectation was that annotations
would be *types*, and in particular would be the expected/intended
type of the annotated value. I think there was some discussion at that
point about "what do we do about the CLI builders" but the conclusion
was that they had relatively low adoption rates, and could adapt (or
more accurately, abandon the annotation approach). That was the point,
in my recollection, where the "annotations are for types" principle
was established. However, I feel that many people got (or were given)
the impression that static type checkers were the core use case
(probably as a result of the emphasis on the message that "type
annotations will never be mandatory"), for better or worse.

I think that by now, it's long been understood by most people that
annotations are types. This may actually be one reason why people are
so uncomfortable with the idea of stringified annotations, because it
violates that assumption - personally, I have the same discomfort
about using explicit string annotations for forward references, it
feels like I'm not declaring a "proper type".

If what I say above is right, the debate here isn't about whether
annotations "are for types", but rather about whether reading the
types in annotations and using them to affect behaviour *at runtime*
is a legitimate use of annotations. That is the use case that
stringifying annotations makes more difficult, and which doesn't seem
to have a strong enough voice in the direction of typing proposals. I
lurk on the typing-sig, and from an outsider's perspective, the
participants seem to be almost entirely designers or heavy users of
static type checkers. That gives a certain emphasis to the proposals
coming from that group.

I'd therefore interpret Barry's plea as being for *anyone* with a use
for annotations to provide their feedback (at least, anyone who
accepts that annotations are types), with particular emphasis on
people who want to use the types declared in annotations to affect
runtime behaviour, as that's the most under-represented group at the
moment (and it's not clear whether it's under-represented because
there aren't many such uses, or because the users aren't being heard
from).

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5N5GTSWQPVALCTJDWZ4G4JSBPDS6324V/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Thu, 25 Nov 2021 at 21:45, Christopher Barker <pythonchb@gmail.com> wrote:
> The issue is that, intended or not, typing is making it's way into Python culture. As an instructor of beginning python users, I am unsure at this point when to introduce type annotations.
>
> What is their role? Up to today, I have treated them as an advanced feature, useful for "complex codebases". But there are any number of examples springing up on the internet, to the point where many students now think they are "best practice", if not actually required.

Agreed this is somewhat OT, but I also think the messaging around
annotations needs to be reviewed. I suspect that students will also
not clearly understand the fact that annotations aren't checked at
runtime. And IMO this is particularly difficult to get across in the
case of non-typechecker uses of annotations.

The following, for example, is extremely non-intuitive to me:

>>> from dataclasses import dataclass
>>>
>>> @dataclass
... class A:
... n: int
...
>>> a = A("didn't expect this!")
>>> a
A(n="didn't expect this!")

Even though the "typing is optional" message is well-understood, my
instincts still lead me to expect that declaring the type of n will
result in a runtime type-check, in the constructor at least. (If I
check the code with mypy, it does raise an error, so that's good.
Although the more I think about it, given that I believe dataclasses
use eval "under the hood", the less I understand *how* it manages to
do that without special-case knowledge of the dataclass decorator...)

I'd like to see a clearer statement from "somewhere" about how APIs
should use annotations at runtime, such that Python users have a much
clearer intuition about APIs like the dataclass one, and library
designers can build their APIs based on a clear "common understanding"
of what to expect when annotations are used.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NBE7YL6EBVBB3NQ33DJI4KY3NSYL2Q5W/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
Am 26.11.21 um 01:48 schrieb Rob Cliffe via Python-Dev:
> ISTM that typing/annotations have been allowed to drift without a
> clear end in view, rather like a ship that is steered in one direction
> by one person, then in another by someone else, with nobody knowing
> what the destination port is.

I believe the problem is less within the typing community itself
(whether static or runtime), but a fundamental conflict between the
typing community and what Stephen called the "typing-suspicious crowd".
typing in Python has always been hampered by sometimes valid, often
unfounded fears that typing is "taking over" and somehow becomes a
requirement. For example, PEP 484 made no changes to Python or the
stdlib, except the introduction of a new module, often to the detriment
of readability and usability of type annotations. And while progress is
made slowly (for example with generics in standard collections or the
new callable syntax proposal), I'm sure the typing community would love
to progress faster and with less obstacles.

 - Sebastian

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HC43AFLA2RCTBXZ3K5O2IIYGPCMYQXOX/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Thu, 25 Nov 2021 23:51:58 +0000
Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
> >
> > Not a PEP proponent (or even a typing user), but I thought this had
> > been made clear long ago. My understanding is that optional,
> > incremental type hints are and have always been considered the primary
> > use case for annotations by the BDFL and AFAICT the SC following the
> > BDFL. If compatibility with typing is an issue, then the burden of
> > implementing that is on the other application. Typing *might* do
> > something to help, but it's not obligated to do so.
>
> This was not my understanding of annotations when they were introduced e.g.:
> https://www.python.org/dev/peps/pep-3107/#use-cases
>
> As I remember it, a decision about the purpose of annotations was
> *explicitly* not made when they were introduced.

This is also what I remember from the discussions at the time of PEP
3107. Annotations were purposefully use case-agnostic, and there was
no stated desire to push for one use case or another. I don't think
gradual typing was even on the radar, not in public comments anyway.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DD2OTSIWZ3KMR6PDK3CSLOMDJQMQSUHL/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Fri, Nov 26, 2021 at 1:37 AM Paul Moore <p.f.moore@gmail.com> wrote:

> On Thu, 25 Nov 2021 at 21:45, Christopher Barker <pythonchb@gmail.com>
> wrote:
> > The issue is that, intended or not, typing is making it's way into
> Python culture. As an instructor of beginning python users, I am unsure at
> this point when to introduce type annotations.
> >
> > What is their role? Up to today, I have treated them as an advanced
> feature, useful for "complex codebases". But there are any number of
> examples springing up on the internet, to the point where many students now
> think they are "best practice", if not actually required.
>
> Agreed this is somewhat OT, but I also think the messaging around
> annotations needs to be reviewed. I suspect that students will also
> not clearly understand the fact that annotations aren't checked at
> runtime. And IMO this is particularly difficult to get across in the
> case of non-typechecker uses of annotations.
>
> The following, for example, is extremely non-intuitive to me:
>
> >>> from dataclasses import dataclass
> >>>
> >>> @dataclass
> ... class A:
> ... n: int
> ...
> >>> a = A("didn't expect this!")
> >>> a
> A(n="didn't expect this!")
>
> Even though the "typing is optional" message is well-understood, my
> instincts still lead me to expect that declaring the type of n will
> result in a runtime type-check, in the constructor at least. (If I
> check the code with mypy, it does raise an error, so that's good.
> Although the more I think about it, given that I believe dataclasses
> use eval "under the hood", the less I understand *how* it manages to
> do that without special-case knowledge of the dataclass decorator...)
>

Static checkers special-case the @dataclass decorator. Eric Traut has a
proposal to generalize this support (sorry, I'm in a rush, otherwise I'd
dig up the link, but it's in the typing-sig archives).


> I'd like to see a clearer statement from "somewhere" about how APIs
> should use annotations at runtime, such that Python users have a much
> clearer intuition about APIs like the dataclass one, and library
> designers can build their APIs based on a clear "common understanding"
> of what to expect when annotations are used.
>

Note that @dataclass itself is very careful not to use the annotations, it
only looks for their *presence*. With one exception for ClassVar.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Re: The current state of typing PEPs [ In reply to ]
On Fri, 26 Nov 2021 at 17:13, Guido van Rossum <guido@python.org> wrote:

>> Although the more I think about it, given that I believe dataclasses
>> use eval "under the hood", the less I understand *how* it manages to
>> do that without special-case knowledge of the dataclass decorator...)
>
> Static checkers special-case the @dataclass decorator. Eric Traut has a proposal to generalize this support (sorry, I'm in a rush, otherwise I'd dig up the link, but it's in the typing-sig archives).

:-( That's what I suspected, but it does mean that dataclasses has a
privilege that other libraries (like attrs, I guess?) don't get.

>> I'd like to see a clearer statement from "somewhere" about how APIs
>> should use annotations at runtime, such that Python users have a much
>> clearer intuition about APIs like the dataclass one, and library
>> designers can build their APIs based on a clear "common understanding"
>> of what to expect when annotations are used.
>
> Note that @dataclass itself is very careful not to use the annotations, it only looks for their *presence*. With one exception for ClassVar.

Understood. What I'm suggesting is that it would be good to have a
clear "common understanding" about whether libraries should be careful
like this, or whether it's OK to base runtime behaviour on type
annotations. And if it is OK, then what are good patterns of design
and behaviour? This is where the proposal to store annotations as
strings hit issues, because it appears to take the view that libraries
*shouldn't* be looking at the actual types specified by annotations
(or maybe that they should only do so via something like
`typing.get_type_hints`). There are other subtleties here (runtime
code needs to deal with the fact that int and "int" should be treated
the same) that there's no guidance on, again possibly because no-one
is really considering that use case.

Paul

PS I've never written code myself that does runtime introspection of
type annotations - so it's quite possible that there *is* guidance
that I've just missed. But it wasn't obvious to me from a quick search
- the "introspection helpers" section of the typing module docs is
pretty basic...
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y4WTMMHZ5XR6PCPKRP4FTIX4R2YJCS7L/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: The current state of typing PEPs [ In reply to ]
On Fri, Nov 26, 2021 at 11:58 AM Paul Moore <p.f.moore@gmail.com> wrote:

> On Fri, 26 Nov 2021 at 17:13, Guido van Rossum <guido@python.org> wrote:
>
> >> Although the more I think about it, given that I believe dataclasses
> >> use eval "under the hood", the less I understand *how* it manages to
> >> do that without special-case knowledge of the dataclass decorator...)
> >
> > Static checkers special-case the @dataclass decorator. Eric Traut has a
> proposal to generalize this support (sorry, I'm in a rush, otherwise I'd
> dig up the link, but it's in the typing-sig archives).
>
> :-( That's what I suspected, but it does mean that dataclasses has a
> privilege that other libraries (like attrs, I guess?) don't get.
>

Actually both are done through plugins (since what they do just doesn't fit
in the PEP 484 type system) and both have the same status, at least in
mypy. (In fact we have twice as many lines of code dedicated to attrs than
to dataclasses. https://github.com/python/mypy/tree/master/mypy/plugins)
The proposal I mentioned by Eric Traut (pyright's author) would make it
easier to support many similar libraries across all static type checkers. (
https://mail.python.org/archives/list/typing-sig@python.org/thread/TXL5LEHYX5ZJAZPZ7YHZU7MVFXMVUVWL/
)


> >> I'd like to see a clearer statement from "somewhere" about how APIs
> >> should use annotations at runtime, such that Python users have a much
> >> clearer intuition about APIs like the dataclass one, and library
> >> designers can build their APIs based on a clear "common understanding"
> >> of what to expect when annotations are used.
> >
> > Note that @dataclass itself is very careful not to use the annotations,
> it only looks for their *presence*. With one exception for ClassVar.
>
> Understood. What I'm suggesting is that it would be good to have a
> clear "common understanding" about whether libraries should be careful
> like this, or whether it's OK to base runtime behaviour on type
> annotations. And if it is OK, then what are good patterns of design
> and behaviour? This is where the proposal to store annotations as
> strings hit issues, because it appears to take the view that libraries
> *shouldn't* be looking at the actual types specified by annotations
> (or maybe that they should only do so via something like
> `typing.get_type_hints`). There are other subtleties here (runtime
> code needs to deal with the fact that int and "int" should be treated
> the same) that there's no guidance on, again possibly because no-one
> is really considering that use case.
>

You are hitting the nail on the head here. I'd say that so far the
recommendation has been "use typing.get_type_hints(x) rather than
x.__annotations__" -- this handles the equivalence between int and "int"
(and hence forward references as well as 'from __future__ import
annotations'). There's now also inspect.get_annotations() which has roughly
the same functionality without quite so much bias towards typing, so
perhaps we should recommend it over typing.get_type_hints() -- though
before 3.10 inspect.get_annotations() didn't exist so you might have to
fall back on using the other. (There are some semantic differences between
the two that I'm glossing over here, because I'm not sure about what
exactly they are. :-)

We now also have a document that recommends best practices (
https://docs.python.org/3/howto/annotations.html) although it's very new --
it appears Larry Hastings wrote it while he was pushing for PEP 649 (but it
received buy-in from the static typing community as well).

So perhaps it isn't as bad as it seems, *if* you know where to look?

That said, I don't think the current static typing infrastructure would be
prepared for an onslaught of modules that use type introspection at runtime
to modify the behavior of classes a la attrs and dataclasses. I don't know
enough about pydantic to say whether it also falls in this category. But
it's definitely difficult to write code that makes use of type annotations
at runtime that *also* passes static type checks by mypy etc.; you
certainly shouldn't attempt to do so without having CI jobs to run the
static checker and test the runtime-introspecting framework you're using.
(It's not quite like trying to write code that's valid Python and Fortran
at the same time, but it's not trivial. :-) There are also features of our
static type system that tend not to be supported by the
runtime-introspecting frameworks -- in particular, I'd expect generics and
callable types to be hard to deal with at runtime. It's easy enough to do
something at runtime with `def f(a: list[int]) -> int`. It's not so simple
to handle `def f(a: Sequence[T]) -> T` or `def f(cb: (T) -> tuple[str, T],
Sequence[T]) -> Mapping[str, T]`. Presumably frameworks like pydantic just
don't support such things and tell the user not to do that. (Someone should
look where pydantic draws the line and report back here.)



>
> Paul
>
> PS I've never written code myself that does runtime introspection of
> type annotations - so it's quite possible that there *is* guidance
> that I've just missed. But it wasn't obvious to me from a quick search
> - the "introspection helpers" section of the typing module docs is
> pretty basic...
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Re: The current state of typing PEPs [ In reply to ]
> There may > be scoping issues to be sorted out,

> Yes, the scoping issues are the main problem.


> > but I don't think they are
> > insurmountable.

Probably not — and LLukas Langa has some good ideas that the SC is
considering.

(Sorry I can’t get your name right in a phone)

Maybe inspect.get_annotations() does, or will be able to, solve many of
these issues.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Re: The current state of typing PEPs [ In reply to ]
On Fri, Nov 26, 2021 at 1:47 PM Guido van Rossum <guido@python.org> wrote
> It's easy enough to do something at runtime with `def f(a: list[int]) ->
int`. It's not so simple to handle `def f(a: Sequence[T]) -> T` or `def
f(cb: (T) -> tuple[str, T], Sequence[T]) -> Mapping[str, T]`. Presumably
frameworks like pydantic just don't support such things and tell the user
not to do that.

I don’t know about Pydantic, but that’s exactly my use case: the type needs
to be an actual intstantiatable type.

So list and tuple is fine, and Sequence[t] won’t work. But it’s fine that
this use case is restricted.

In fact, other than the basic core types, you need to use specialized types
with this system anyway.

-CHB
--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Re: The current state of typing PEPs [ In reply to ]
So does your library work both with and without ‘from __future__ import
annotations’? If it does, you shouldn’t have to worry. If it doesn’t, it
would be useful if you could post some detailed examples of what goes wrong.

On Fri, Nov 26, 2021 at 14:24 Christopher Barker <pythonchb@gmail.com>
wrote:

>
>
> On Fri, Nov 26, 2021 at 1:47 PM Guido van Rossum <guido@python.org> wrote
> > It's easy enough to do something at runtime with `def f(a: list[int])
> -> int`. It's not so simple to handle `def f(a: Sequence[T]) -> T` or `def
> f(cb: (T) -> tuple[str, T], Sequence[T]) -> Mapping[str, T]`. Presumably
> frameworks like pydantic just don't support such things and tell the user
> not to do that.
>
> I don’t know about Pydantic, but that’s exactly my use case: the type
> needs to be an actual intstantiatable type.
>
> So list and tuple is fine, and Sequence[t] won’t work. But it’s fine
> that this use case is restricted.
>
> In fact, other than the basic core types, you need to use specialized
> types with this system anyway.
>
> -CHB
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>
--
--Guido (mobile)

1 2 3  View All