Mailing List Archive

On 11/17/2021 5:47 PM, Barry Warsaw wrote:
> Does PEP 563 or 649 satisfy static and dynamic typing needs?
>
> In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs.
>
> Defer decision on PEP 563 and 649 in 3.11
>
> As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users.

"In the face of ambiguity, refuse the temptation to guess."

I appreciate the carefully articulated reasoning being given with these
PEP decisions.

--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/S46JI4E3OZJUUSVZLDPAP5A5MOMAMD5O/
Code of Conduct: http://python.org/psf/codeofconduct/

Thanks to Barry and the SC for giving us this update.

> In the interest of full transparency, we want to let the Python community
know that the Steering Council continues to discuss PEP 563 (Postponed
Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations
Using Descriptors).

<snip>

I want to draw attention to this point:

> all the use cases and requirements across the static and dynamic typing
constituents.

TL;DR:

Annotations can be, and are, used for other things than "typing". I just
noticed that PEP 563 apparently deprecated those other uses (well, sort of:
"uses for annotations incompatible with the aforementioned PEPs should be
considered deprecated"), but if the SC is reconsidering PEP 563, then it
would be nice to be clear about whether non-typing uses of annotations are
indeed deprecated. If not, then the challenge is to come up with a way
forward that not only supports both static and dynamic typing, but also
other potentially arbitrary use cases.

And now onto more detail ...

One example is a use case of mine -- I have built a hierarchical object
system, built on dataclasses, in which the annotation absolutely has to be
an actual type(class) object. PEP 563 will very much break this use case.
Of course, there are other ways to write that code, but I'm pretty sure it
would require a complete change to the API. I'm not entirely sure if PEP
646 would work for my use case -- likely it would, at least with modest
modifications, rather than a new API.

I have no idea how many other similar use cases are out in the wild, but we
know that the Pydantic project has concerns.

To be fair -- I wrote all my code after PEP 563 was accepted. And I only
just now read it carefully, and found that it says:

"With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated."

If that's the case, then that's the case, and I'll deal, but it is
disappointing.

But I suspect I'm not alone in not really noticing that statement, so I
think it's worth it for the SC to explicitly reiterate at this point that
all non-typing uses of annotations are depreciated, if indeed, that is
still the intent. Also, that statement is not entirely clear. In fact, I
think my use case IS compatible with the "aforementioned PEPs", it's just
not compatible with PEP 653 itself. It all depends on what is meant by
"compatible".

I would like to provide a bit of perspective on this. I was criticised on
this list a while back for, essentially, not paying attention and then
complaining later. After all, PEP 563 was accepted over four years ago.
Fair enough.

But the fact is that I, among others, have been a bit uncomfortable about
the focus on typing in Python for years. But when issues are raised, we
have been repeatedly told that typing is, and always will remain, optional.
In short, it was made clear that anyone not interested in typing could
safely ignore the discussions about it. And thus, a number of PEPs came and
went, and those among us that did not choose to involve ourselves in the
conversation did not pay attention to the details.

And thus we didn't notice that buried in what seemed like a typing PEP,
was, in fact, a depreciation of any non-typing uses of an existing Python
feature. (also to be fair, the title of the PEP is "Postponed Evaluation of
Annotations" -- so should have caught the attention of anyone interested in
annotations for any use.

In my personal case, I didn't notice it at the time, as I wasn't using
annotations for anything at all until fairly recently. In fact, it was the
introduction of dataclasses, which I think are the first use of annotations
in the standard library, that drew my attention. dataclasses, as I'm sure
you all know, use the presence of an annotation on a class attribute to
define a field.

This is a pretty nifty use of annotations as an API for defining the
fields. I liked that, and decided to use it.

dataclasses create a Field object, which has an attribute called "type",
that gets populated with the annotation's value.

PEP 563 would very much change what gets put in the "type" attribute of a
Field.

Eric V. Smith has indicated that he never intended that to actually be a
type, but rather was intended to be whatever the annotation was, so that
PEP 563 wouldn't change anything. I'm happy to take his word for it (and
PEP 567 says that the actual type is not used or modified by dataclasses),
but as users, all we have to go on is the documentation and the choice of
names, which very much gave the impression that a Field.type would be a
type -- at least if a type was set in the annotation.

I also notice that PEP 567 (dataclasses) and PEP 563 were both developed
and approved at about the same time. So I suspect the impact of PEP 563 was
not considered when designing or documenting dataclasses. However, PEP 567
does make a number of references to typing, and was clearly written with
the idea in mind that the annotations would, in fact, be used for type
analysis. But again, no mention of PEP 563, or what might be in the
field.type attribute -- one might wonder why it was there at all if it was
not intended to be used.

We need help in defining those requirements clearly, in uncovering the use
> cases we’re not aware of, and most importantly, being an interface to
> typing enthusiasts


Again -- is it only "typing enthusiasts" that you want to engage? Or "users
of annotations"? -- maybe it is, but it would be nice if that was a clear
statement from the SC.

- CHB


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Sun, 21 Nov 2021 at 07:50, Christopher Barker <pythonchb@gmail.com> wrote:
>
> TL;DR:
>
> Annotations can be, and are, used for other things than "typing". I just noticed that PEP 563 apparently deprecated those other uses (well, sort of: "uses for annotations incompatible with the aforementioned PEPs should be considered deprecated"), but if the SC is reconsidering PEP 563, then it would be nice to be clear about whether non-typing uses of annotations are indeed deprecated. If not, then the challenge is to come up with a way forward that not only supports both static and dynamic typing, but also other potentially arbitrary use cases.

I agree with the points made in this post. It's becoming harder and
harder for people not particularly interested in static typing to
simply ignore it, and any use of annotations to affect runtime
behaviour is in a weird grey area. And as a library author, I'm now
finding that I'm getting requests to add typing to my code "for my
users" (i.e., using types is no longer just a choice I make for my
project, it's an API design issue).

So I too would appreciate clarity on where annotations, and more
generally typing, stand as Python language features¹.

Paul

¹ In particular, typing features seem to change so rapidly that
supporting a wide range of Python versions is a real pain (in
practical terms - not getting syntax errors is straightforward, but
writing type annotations that work well across versions very
definitely isn't, at least for someone who doesn't really care about
static typing).
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AMGOAG22UJXCFPQ2QCGH57RNB3UTAX7H/
Code of Conduct: http://python.org/psf/codeofconduct/

On 21/11/2021 11:04, Paul Moore wrote:
> On Sun, 21 Nov 2021 at 07:50, Christopher Barker <pythonchb@gmail.com> wrote:
> It's becoming harder and harder for people not particularly interested
> in static typing to simply ignore it
(Raises hand.)  +1
Rob Cliffe
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YMZ4UE2TMRH5WNF6Y3KWGJS2R2YJQAZI/
Code of Conduct: http://python.org/psf/codeofconduct/

Executive summary:

The typing-suspicious crowd has a valid complaint about PEPs 563 and
649, but it's not that they weren't warned.

Christopher Barker writes:

> Annotations can be, and are, used for other things than "typing". I
> just noticed that PEP 563 apparently deprecated those other uses
> (well, sort of: "uses for annotations incompatible with the
> aforementioned PEPs should be considered deprecated")

Not a PEP proponent (or even a typing user), but I thought this had
been made clear long ago. My understanding is that optional,
incremental type hints are and have always been considered the primary
use case for annotations by the BDFL and AFAICT the SC following the
BDFL. If compatibility with typing is an issue, then the burden of
implementing that is on the other application. Typing *might* do
something to help, but it's not obligated to do so.

> One example is a use case of mine -- I have built a hierarchical
> object system, built on dataclasses, in which the annotation
> absolutely has to be an actual type(class) object. PEP 563 will
> very much break this use case. [...]

> I'm not entirely sure if PEP 649 would work for my use case --
> likely it would, at least with modest modifications, rather than a
> new API.

If PEP 649 works for you, I expect you could work with PEP 563 by
moving Larry's data descriptor concept out of the compiler and into
your library code.

You could also appeal to the "might do something" caveat above. Yury
suggested providing both the immediately evaluated object (if there
were no forward references, I guess) and the string representation,
which would be backward compatible with your use case. The idea was
rejected on the basis that it doesn't help with the primary use in
typing, and doesn't remove the need for a __future__ import during the
transition period. You could ask that that decision be revisited,
since it would ensure that uses of __annotations__ like yours continue
to work.

> But I suspect I'm not alone in not really noticing that statement,

I'm not surprised people haven't noticed that statement. I am
surprised that a lot of folks haven't noticed that this comes up
occasionally and the answer every time is "we're not going to go out
of our way to break other use cases, but typing *is* the primary use
case and will take precedence if the question comes up".

I will grant that restricting the type of compiled annotations from
typing.Any to "string serializing an object's AST" could reasonably be
said to be "going out of your way to break other use cases". You have
a valid beef here, although it's not obvious to me how it should be
resolved.

So I'm not saying that you don't have an interest in the future
semantics of this dunder. I am +1 on Python providing relief for your
use case.

> But the fact is that I, among others, have been a bit uncomfortable
> about the focus on typing in Python for years.

You mean y'all are uncomfortable with the popularity of typing. You
wouldn't care if it wasn't used outside of hugely complex proprietary
codebases you'll never see.

Thing is, reasoning about programs is What We Do as programmers, and
efforts to make it easier to do that for complex programs, and to
provide software to help with that task, are here to stay and they
will be used in mixed company. Even in Python. (And that's why, as
someone who doesn't use typing, I support typing. Anything that helps
*other people* to write better code, I can get behind. My code, I'm
going to write as incompetently as ever.[1] ;-)

> But when issues are raised, we have been repeatedly told that
> typing is, and always will remain, optional.

This has always been in the context of *your* source code. For
example, nobody ever claimed you could keep it out of anybody else's
source code (although that's clearly what a lot of those "raising
issues" want!) Still, features like stub files were provided so
typists could work and play nicely with non-typists if they wanted to
(and many typists want that, themselves).

But dunders are the property of the language (or sometimes the
implementation), and they always have been. If you use them in a way
that's not documented to work, you're at risk even if it happens to
work now. PEP 3107 didn't document that anything would work. ;-)


Footnotes:
[1] Barry, do not tell Abhilash I wrote that.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2O3JHKJ2CM7OFNGT664WXLN2DKU3YGIS/
Code of Conduct: http://python.org/psf/codeofconduct/

On 25/11/2021 15:15, Stephen J. Turnbull wrote:
> Executive summary:
>
> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>
>
“As you will no doubt be aware, the plans for development of the
outlying regions of the Galaxy require the building of a hyperspatial
express route through your star system, and regrettably your planet is
one of those scheduled for demolition. The process will take slightly
less than two of your Earth minutes. Thank you.”

Well we were warned, so everything's fine.
Rob Cliffe

On Sat, Nov 20, 2021 at 11:46:56PM -0800, Christopher Barker wrote:

> One example is a use case of mine -- I have built a hierarchical object
> system, built on dataclasses, in which the annotation absolutely has to be
> an actual type(class) object. PEP 563 will very much break this use case.

I don't think that's an insurmountable problem. I think that all you
need is a small class decorator to evaluate the stringified annotations
back to the real things. The PEP tells us the right way to evaluate
annotations, so all(?) you need is a decorator to do that to each method
in your class, and Bob's your uncle.

Maybe PEP 563 could include a decorator in the typing module to
destringify all the annotations in a class or function?

As far as I can see from a brief scan of the PEP, and based on knowing
next to nothing about your use-case, the only hypothetical problem might
be this line in the PEP:

"Consequently, using local state in annotations is no longer possible in
general."

but it's not entirely clear to me what ?ukasz means by that, or whether
it will affect your use-case.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3RYHEEARXUCK3S6WIZALAU75PEPG7GR2/
Code of Conduct: http://python.org/psf/codeofconduct/

First:

At this point, I am not advocating anything in particular. I am just asking
that the SC makes a clear statement about the intention at this point.

From Barry's email:
"""
This is also a call to you, the folks who both care deeply about typing in
Python, and have a solid understanding of the subject matter (or
willingness to learn) to help us! There are use cases we don’t know about,
and unclear requirements from both the static and dynamic typing users.
"""
I don't care deeply about typing, and I'm not a static or dynamic typing
user. So this implies that a decision has been made that annotations are
for typing, and typing only.

(not that you're not allowed to use for anything else, of course you are,
but that other uses won;t be taken into account when designing the new
interface)

But I have never seen that clearly stated anywhere. The closest is from PEP
563, where it says:

"""
With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.
"""

Which pretty much makes the point, but it's a bit subtle -- what does
"incompatible' mean?

On Thu, Nov 25, 2021 at 7:15 AM Stephen J. Turnbull <
stephenjturnbull@gmail.com> wrote:

> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>

Well, "typing-suspicious" is part of it, but the real problem is folks that
might be using annotations for non-typing purposes.

As for warning: yes and no -- what's new here is that as far as I know,
this is the first time since the py3 transition that a __future__ import
would be made standard behavior. And you'd have to be paying a lot of
attention to know that was going to happen. And I suspect that only folks
involved in the development of typing tools were paying that much attention.

In the case of the py3 transition -- it was very well known and documented
that lots of things would be changing, and we had a very long transition
period.

But the challenge with this is that there is no way to raise a depreciation
warning for not using a __future__ import. So while we all should have been
testing our code with:

from __future__ import annotations

For the last few versions, I doubt that very many people not developing
typing tools were doing that.

> Annotations can be, and are, used for other things than "typing". I
> > just noticed that PEP 563 apparently deprecated those other uses
> > (well, sort of: "uses for annotations incompatible with the
> > aforementioned PEPs should be considered deprecated")
>
> Not a PEP proponent (or even a typing user), but I thought this had
> been made clear long ago. My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL and AFAICT the SC following the
> BDFL.


well no -- not "always" -- look at PEP 3107, when annotations were first
introduced:

"By itself, Python does not attach any particular meaning or significance
to annotations."

and it goes on to give non-typing examples.

Since then, the shift from "nothing more than a way of associating
arbitrary Python expressions with various parts of a function at
compile-time" to a system for defining types has been gradual, with a
number of PEPs, and, as far as i can tell, PEP 563 is the first time that
a change was proposed that would affect other uses for annotations.

Anyway, that is water under the bridge. But the reason I bring it up is
because of all the many thousands of Python users, I pay probably more
attention to changes than most -- I have followed Python-ideas and
Python-dev for years. So I'm guessing that there are other folks out there
that will be surprised if and when the behavior of annotations changes.


> If compatibility with typing is an issue, then the burden of
> implementing that is on the other application.


That does indeed seem to be the current intent. And I'm happy to deal with
that -- I just want it to be clear that I (and others) have to wait and see
how it shakes out, and then figure out what to do, or if I should
contribute to the conversation now. If non-typing use cases of annotations
are still considered important, then we should encourage folks like me to
be part of the conversation.


> > I'm not entirely sure if PEP 649 would work for my use case --
> > likely it would, at least with modest modifications, rather than a
> > new API.
>
> If PEP 649 works for you, I expect you could work with PEP 563 by
> moving Larry's data descriptor concept out of the compiler and into
> your library code.
>

That would indeed require maybe not a change to the API, but certainly some
complex code. But yeah, probably doable one way or another.


> > But I suspect I'm not alone in not really noticing that statement,
>
> I am
> surprised that a lot of folks haven't noticed that this comes up
> occasionally and the answer every time is "we're not going to go out
> of our way to break other use cases, but typing *is* the primary use
> case and will take precedence if the question comes up".
>

I think the "breaking other use cases" has been subtle to those not paying
attention to typing.


> > But the fact is that I, among others, have been a bit uncomfortable
> > about the focus on typing in Python for years.
>
> You mean y'all are uncomfortable with the popularity of typing. You
> wouldn't care if it wasn't used outside of hugely complex proprietary
> codebases you'll never see.
>

well, yes. The issue is that, intended or not, typing is making it's way
into Python culture. As an instructor of beginning python users, I am
unsure at this point when to introduce type annotations.

What is their role? Up to today, I have treated them as an advanced
feature, useful for "complex codebases". But there are any number of
examples springing up on the internet, to the point where many students now
think they are "best practice", if not actually required.

But that's getting pretty OT here.

> Anything that helps

> *other people* to write better code, I can get behind. My code, I'm
> going to write as incompetently as ever.[1] ;-)
>

me too -- actually, not quite. I write a lot of "scripts" that I will
continue to keep simple, and I also develop complex systems -- and those
require more care.


> > But when issues are raised, we have been repeatedly told that
> > typing is, and always will remain, optional.
>
> This has always been in the context of *your* source code.


Sure -- my point with that is that if I don't want to use typing in my
code, then I didn't have to pay attention to the development of the typing
systems.

But my point here is that what changed with PEP 563 is that while typing
is still optional, this is the first time that the language itself may be
changed to accommodate typing -- so it can no longer be completely ignored.

But dunders are the property of the language (or sometimes the
> implementation), and they always have been. If you use them in a way
> that's not documented to work, you're at risk even if it happens to
> work now. PEP 3107 didn't document that anything would work. ;-)
>

no, but it did document that annotations were "arbitrary Python
expressions" -- that is changed in PEP 563 -- intentionally, but it is
changing something about the language that was documented.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On 26/11/21 4:15 am, Stephen J. Turnbull wrote:
> My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL

I'm not sure that's true. The way I remember it, back when
annotations were first introduced, the BDFL didn't say that,
or at least didn't say it very clearly. It sounded more like
type hints were just one of many possible uses, and he
encouraged people to experiment. There were even discussions
about coming up with a convention to manage conflicting uses
of annotations in the same code. That wouldn't have happened
if typing were considered the only supported use.

--
Greg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/R5TWU6HTOKLCTY2QZDGEAR74TGHZHQQ4/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, 25 Nov 2021 at 15:16, Stephen J. Turnbull
<stephenjturnbull@gmail.com> wrote:
>
> Executive summary:
>
> The typing-suspicious crowd has a valid complaint about PEPs 563 and
> 649, but it's not that they weren't warned.
>
> Christopher Barker writes:
>
> > Annotations can be, and are, used for other things than "typing". I
> > just noticed that PEP 563 apparently deprecated those other uses
> > (well, sort of: "uses for annotations incompatible with the
> > aforementioned PEPs should be considered deprecated")
>
> Not a PEP proponent (or even a typing user), but I thought this had
> been made clear long ago. My understanding is that optional,
> incremental type hints are and have always been considered the primary
> use case for annotations by the BDFL and AFAICT the SC following the
> BDFL. If compatibility with typing is an issue, then the burden of
> implementing that is on the other application. Typing *might* do
> something to help, but it's not obligated to do so.

This was not my understanding of annotations when they were introduced e.g.:
https://www.python.org/dev/peps/pep-3107/#use-cases

As I remember it, a decision about the purpose of annotations was
*explicitly* not made when they were introduced. It was clear that
typing was a major potential use and then at some point (around about
the introduction of the typing module) there seemed to be a shift in
people's understanding of what annotations were for. Eventually that
reached the point that people who were particularly interested in
typing had no memory of the fact that the purpose of annotations had
not really been specified as being about typing in the first place.

It looks to me like Chris has identified in PEP 563 what is
potentially the earliest reference (in an accepted PEP) to the idea
that non-typing uses of annotations are to be discouraged.

--
Oscar
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BERGEZTXG4XSEIJQHQWC3IZLXYEZPZ6C/
Code of Conduct: http://python.org/psf/codeofconduct/

My 2¢ from a position of *extreme ignorance*, not to mention zero
interest in annotations (all welcome to shoot me down in flames, or
treat me with contemptuous silence.  But *occasionall*y the "idiot's"
point of view is worth considering, so here goes).

ISTM that typing/annotations have been allowed to drift without a clear
end in view, rather like a ship that is steered in one direction by one
person, then in another by someone else, with nobody knowing what the
destination port is.  As witness the conflicting views in this thread. 
At the risk of being rude, the phrase "headless chickens" comes to mind.
ISTM that it is time to take a step back and decide on a definite
policy.  Perhaps a definitive pronouncement from Guido.  Or the SC. Or a
discussion from all parties that reaches an acceptable conclusion.  Then
stick to it.  Even if it causes some backward incompatibility - ISTM
that things have changed so rapidly that this really would be an
acceptable evil if it results in clarity for the future.  Meanwhile, do
not change anything, do not approve any PEPs, until this conclusion is
reached.
I apologise if I am being presumptuous in commenting on a subject I know
almost nothing about.

/I'm just a soul whose intentions are good,//
//Oh Lord, please don't let me be misunderstood.//
/
Best wishes
Rob Cliffe


On 25/11/2021 23:51, Oscar Benjamin wrote:
> On Thu, 25 Nov 2021 at 15:16, Stephen J. Turnbull
> <stephenjturnbull@gmail.com> wrote:
>> Executive summary:
>>
>> The typing-suspicious crowd has a valid complaint about PEPs 563 and
>> 649, but it's not that they weren't warned.
>>
>> Christopher Barker writes:
>>
>> > Annotations can be, and are, used for other things than "typing". I
>> > just noticed that PEP 563 apparently deprecated those other uses
>> > (well, sort of: "uses for annotations incompatible with the
>> > aforementioned PEPs should be considered deprecated")
>>
>> Not a PEP proponent (or even a typing user), but I thought this had
>> been made clear long ago. My understanding is that optional,
>> incremental type hints are and have always been considered the primary
>> use case for annotations by the BDFL and AFAICT the SC following the
>> BDFL. If compatibility with typing is an issue, then the burden of
>> implementing that is on the other application. Typing *might* do
>> something to help, but it's not obligated to do so.
> This was not my understanding of annotations when they were introduced e.g.:
> https://www.python.org/dev/peps/pep-3107/#use-cases
>
> As I remember it, a decision about the purpose of annotations was
> *explicitly* not made when they were introduced. It was clear that
> typing was a major potential use and then at some point (around about
> the introduction of the typing module) there seemed to be a shift in
> people's understanding of what annotations were for. Eventually that
> reached the point that people who were particularly interested in
> typing had no memory of the fact that the purpose of annotations had
> not really been specified as being about typing in the first place.
>
> It looks to me like Chris has identified in PEP 563 what is
> potentially the earliest reference (in an accepted PEP) to the idea
> that non-typing uses of annotations are to be discouraged.
>
> --
> Oscar
> _______________________________________________
> Python-Dev mailing list --python-dev@python.org
> To unsubscribe send an email topython-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived athttps://mail.python.org/archives/list/python-dev@python.org/message/BERGEZTXG4XSEIJQHQWC3IZLXYEZPZ6C/
> Code of Conduct:http://python.org/psf/codeofconduct/

On Thu, Nov 25, 2021 at 3:49 PM Greg Ewing <greg.ewing@canterbury.ac.nz>
wrote:

> On 26/11/21 4:15 am, Stephen J. Turnbull wrote:
> > My understanding is that optional,
> > incremental type hints are and have always been considered the primary
> > use case for annotations by the BDFL
>
> I'm not sure that's true. The way I remember it, back when
> annotations were first introduced, the BDFL didn't say that,
> or at least didn't say it very clearly. It sounded more like
> type hints were just one of many possible uses, and he
> encouraged people to experiment. There were even discussions
> about coming up with a convention to manage conflicting uses
> of annotations in the same code. That wouldn't have happened
> if typing were considered the only supported use.
>

My memory is also hazy, but I'm quite sure that *in my mind* annotations
were intended as a compromise between conflicting proposals for *typing*.
We didn't have agreement on the syntax or semantics, but we did know we
wanted to do something with types eventually. Some folks wanted to enforce
types at runtime. Others wanted to use them to generate faster code. Yet
others wanted types to be checked by the compiler. The term "gradual
typing" wasn't invented (or hadn't reached our community) yet, and offline
static type checking wasn't something we had thought of either (I think).
But it was clear that typing would have to be optional.

Fortunately the current situation is that there are few people who want to
use annotations for non-typing, but there are still conflicting use cases,
in particular offline static type checking on the one hand (which started
with mypy and PEP 484) and some runtime use of types on the other hand (
pydantic <https://pydantic-docs.helpmanual.io/> being the main example
IIUC).

Tools like pydantic seem to have adopted the *notation* of PEP 484 fully,
but their *interpretation* is different. This was explicitly provided for
by PEP 484 (the design there carefully ensures that annotations are fully
introspectable) but it wasn't anticipated to be a big use case, hence the
mistaken belief at the time PEP 563 was accepted that stringifying
annotations would be sufficient for the introspection use case.

So I'm in agreement with Stephen, but since he left out any mention of
offline static checking, his observation isn't helping much.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On Fri, Nov 26, 2021 at 12:15:10AM +0900, Stephen J. Turnbull wrote:

> I will grant that restricting the type of compiled annotations from
> typing.Any to "string serializing an object's AST" could reasonably be
> said to be "going out of your way to break other use cases". You have
> a valid beef here, although it's not obvious to me how it should be
> resolved.

I don't think that's what PEP 563 says. Annotations are not *restricted*
to only be strings, it is merely that when the function or class object
is built, the annotations are left as strings.

So we can post-process annotations and destringify them:

>>> from __future__ import annotations
>>> def func(arg:float) -> int:
... pass
...
>>> func.__annotations__
{'arg': 'float', 'return': 'int'}
>>> func.__annotations__['return'] = int
>>> func.__annotations__['arg'] = float
>>> func.__annotations__
{'arg': <class 'float'>, 'return': <class 'int'>}


There may be scoping issues to be sorted out, but I don't think they are
insurmountable.


--
Steven
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/24PKPJTI5SIES32C2KZVF6R6D3RLF6BI/
Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano writes:

> I don't think that's what PEP 563 says. Annotations are not
> *restricted* to only be strings, it is merely that when the
> function or class object is built, the annotations are left as
> strings.

You're right, I was imprecise. I meant as PEP 563 is implemented now,
I doubt it can handle non-strings nicely.

> So we can post-process annotations and destringify them:

True, but you face the dread "double-decode problem": if the
annotation happens to destringify to a string, then the "post-
processor" may try to destringify again. I guess we can mark each
annotation as already destringified or not. As long as this is done
by some sort of API, you're probably OK (but it makes me nervous since
__annotations__ is completely exposed to the program, as in your
example).

> There may be scoping issues to be sorted out, but I don't think they are
> insurmountable.

But isn't there still the issue of forward reference, which motivates
these PEPs in the first place?

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LX7EOMA7RPVJUTS5ADMLBIK2CZN2TMLQ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, 26 Nov 2021 at 05:14, Guido van Rossum <guido@python.org> wrote:
>
> My memory is also hazy, but I'm quite sure that *in my mind* annotations were intended as a compromise between conflicting proposals for *typing*. We didn't have agreement on the syntax or semantics, but we did know we wanted to do something with types eventually.

More hazy memories here, but I think the original proposal left open
the possibility of annotations not being types at all - for example,
being docstrings for the arguments, or option names for a "function
call to CLI" tool, etc. I think that some libraries took this approach
(in particular the "CLI builder" idea). At some point (probably around
PEP 484) it became clear that the expectation was that annotations
would be *types*, and in particular would be the expected/intended
type of the annotated value. I think there was some discussion at that
point about "what do we do about the CLI builders" but the conclusion
was that they had relatively low adoption rates, and could adapt (or
more accurately, abandon the annotation approach). That was the point,
in my recollection, where the "annotations are for types" principle
was established. However, I feel that many people got (or were given)
the impression that static type checkers were the core use case
(probably as a result of the emphasis on the message that "type
annotations will never be mandatory"), for better or worse.

I think that by now, it's long been understood by most people that
annotations are types. This may actually be one reason why people are
so uncomfortable with the idea of stringified annotations, because it
violates that assumption - personally, I have the same discomfort
about using explicit string annotations for forward references, it
feels like I'm not declaring a "proper type".

If what I say above is right, the debate here isn't about whether
annotations "are for types", but rather about whether reading the
types in annotations and using them to affect behaviour *at runtime*
is a legitimate use of annotations. That is the use case that
stringifying annotations makes more difficult, and which doesn't seem
to have a strong enough voice in the direction of typing proposals. I
lurk on the typing-sig, and from an outsider's perspective, the
participants seem to be almost entirely designers or heavy users of
static type checkers. That gives a certain emphasis to the proposals
coming from that group.

I'd therefore interpret Barry's plea as being for *anyone* with a use
for annotations to provide their feedback (at least, anyone who
accepts that annotations are types), with particular emphasis on
people who want to use the types declared in annotations to affect
runtime behaviour, as that's the most under-represented group at the
moment (and it's not clear whether it's under-represented because
there aren't many such uses, or because the users aren't being heard
from).

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5N5GTSWQPVALCTJDWZ4G4JSBPDS6324V/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, 25 Nov 2021 at 21:45, Christopher Barker <pythonchb@gmail.com> wrote:
> The issue is that, intended or not, typing is making it's way into Python culture. As an instructor of beginning python users, I am unsure at this point when to introduce type annotations.
>
> What is their role? Up to today, I have treated them as an advanced feature, useful for "complex codebases". But there are any number of examples springing up on the internet, to the point where many students now think they are "best practice", if not actually required.

Agreed this is somewhat OT, but I also think the messaging around
annotations needs to be reviewed. I suspect that students will also
not clearly understand the fact that annotations aren't checked at
runtime. And IMO this is particularly difficult to get across in the
case of non-typechecker uses of annotations.

The following, for example, is extremely non-intuitive to me:

>>> from dataclasses import dataclass
>>>
>>> @dataclass
... class A:
... n: int
...
>>> a = A("didn't expect this!")
>>> a
A(n="didn't expect this!")

Even though the "typing is optional" message is well-understood, my
instincts still lead me to expect that declaring the type of n will
result in a runtime type-check, in the constructor at least. (If I
check the code with mypy, it does raise an error, so that's good.
Although the more I think about it, given that I believe dataclasses
use eval "under the hood", the less I understand *how* it manages to
do that without special-case knowledge of the dataclass decorator...)

I'd like to see a clearer statement from "somewhere" about how APIs
should use annotations at runtime, such that Python users have a much
clearer intuition about APIs like the dataclass one, and library
designers can build their APIs based on a clear "common understanding"
of what to expect when annotations are used.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NBE7YL6EBVBB3NQ33DJI4KY3NSYL2Q5W/
Code of Conduct: http://python.org/psf/codeofconduct/

Am 26.11.21 um 01:48 schrieb Rob Cliffe via Python-Dev:
> ISTM that typing/annotations have been allowed to drift without a
> clear end in view, rather like a ship that is steered in one direction
> by one person, then in another by someone else, with nobody knowing
> what the destination port is.

I believe the problem is less within the typing community itself
(whether static or runtime), but a fundamental conflict between the
typing community and what Stephen called the "typing-suspicious crowd".
typing in Python has always been hampered by sometimes valid, often
unfounded fears that typing is "taking over" and somehow becomes a
requirement. For example, PEP 484 made no changes to Python or the
stdlib, except the introduction of a new module, often to the detriment
of readability and usability of type annotations. And while progress is
made slowly (for example with generics in standard collections or the
new callable syntax proposal), I'm sure the typing community would love
to progress faster and with less obstacles.

 - Sebastian

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HC43AFLA2RCTBXZ3K5O2IIYGPCMYQXOX/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, 25 Nov 2021 23:51:58 +0000
Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
> >
> > Not a PEP proponent (or even a typing user), but I thought this had
> > been made clear long ago. My understanding is that optional,
> > incremental type hints are and have always been considered the primary
> > use case for annotations by the BDFL and AFAICT the SC following the
> > BDFL. If compatibility with typing is an issue, then the burden of
> > implementing that is on the other application. Typing *might* do
> > something to help, but it's not obligated to do so.
>
> This was not my understanding of annotations when they were introduced e.g.:
> https://www.python.org/dev/peps/pep-3107/#use-cases
>
> As I remember it, a decision about the purpose of annotations was
> *explicitly* not made when they were introduced.

This is also what I remember from the discussions at the time of PEP
3107. Annotations were purposefully use case-agnostic, and there was
no stated desire to push for one use case or another. I don't think
gradual typing was even on the radar, not in public comments anyway.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DD2OTSIWZ3KMR6PDK3CSLOMDJQMQSUHL/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Nov 26, 2021 at 1:37 AM Paul Moore <p.f.moore@gmail.com> wrote:

> On Thu, 25 Nov 2021 at 21:45, Christopher Barker <pythonchb@gmail.com>
> wrote:
> > The issue is that, intended or not, typing is making it's way into
> Python culture. As an instructor of beginning python users, I am unsure at
> this point when to introduce type annotations.
> >
> > What is their role? Up to today, I have treated them as an advanced
> feature, useful for "complex codebases". But there are any number of
> examples springing up on the internet, to the point where many students now
> think they are "best practice", if not actually required.
>
> Agreed this is somewhat OT, but I also think the messaging around
> annotations needs to be reviewed. I suspect that students will also
> not clearly understand the fact that annotations aren't checked at
> runtime. And IMO this is particularly difficult to get across in the
> case of non-typechecker uses of annotations.
>
> The following, for example, is extremely non-intuitive to me:
>
> >>> from dataclasses import dataclass
> >>>
> >>> @dataclass
> ... class A:
> ... n: int
> ...
> >>> a = A("didn't expect this!")
> >>> a
> A(n="didn't expect this!")
>
> Even though the "typing is optional" message is well-understood, my
> instincts still lead me to expect that declaring the type of n will
> result in a runtime type-check, in the constructor at least. (If I
> check the code with mypy, it does raise an error, so that's good.
> Although the more I think about it, given that I believe dataclasses
> use eval "under the hood", the less I understand *how* it manages to
> do that without special-case knowledge of the dataclass decorator...)
>

Static checkers special-case the @dataclass decorator. Eric Traut has a
proposal to generalize this support (sorry, I'm in a rush, otherwise I'd
dig up the link, but it's in the typing-sig archives).


> I'd like to see a clearer statement from "somewhere" about how APIs
> should use annotations at runtime, such that Python users have a much
> clearer intuition about APIs like the dataclass one, and library
> designers can build their APIs based on a clear "common understanding"
> of what to expect when annotations are used.
>

Note that @dataclass itself is very careful not to use the annotations, it
only looks for their *presence*. With one exception for ClassVar.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On Fri, 26 Nov 2021 at 17:13, Guido van Rossum <guido@python.org> wrote:

>> Although the more I think about it, given that I believe dataclasses
>> use eval "under the hood", the less I understand *how* it manages to
>> do that without special-case knowledge of the dataclass decorator...)
>
> Static checkers special-case the @dataclass decorator. Eric Traut has a proposal to generalize this support (sorry, I'm in a rush, otherwise I'd dig up the link, but it's in the typing-sig archives).

:-( That's what I suspected, but it does mean that dataclasses has a
privilege that other libraries (like attrs, I guess?) don't get.

>> I'd like to see a clearer statement from "somewhere" about how APIs
>> should use annotations at runtime, such that Python users have a much
>> clearer intuition about APIs like the dataclass one, and library
>> designers can build their APIs based on a clear "common understanding"
>> of what to expect when annotations are used.
>
> Note that @dataclass itself is very careful not to use the annotations, it only looks for their *presence*. With one exception for ClassVar.

Understood. What I'm suggesting is that it would be good to have a
clear "common understanding" about whether libraries should be careful
like this, or whether it's OK to base runtime behaviour on type
annotations. And if it is OK, then what are good patterns of design
and behaviour? This is where the proposal to store annotations as
strings hit issues, because it appears to take the view that libraries
*shouldn't* be looking at the actual types specified by annotations
(or maybe that they should only do so via something like
`typing.get_type_hints`). There are other subtleties here (runtime
code needs to deal with the fact that int and "int" should be treated
the same) that there's no guidance on, again possibly because no-one
is really considering that use case.

Paul

PS I've never written code myself that does runtime introspection of
type annotations - so it's quite possible that there *is* guidance
that I've just missed. But it wasn't obvious to me from a quick search
- the "introspection helpers" section of the typing module docs is
pretty basic...
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y4WTMMHZ5XR6PCPKRP4FTIX4R2YJCS7L/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Nov 26, 2021 at 11:58 AM Paul Moore <p.f.moore@gmail.com> wrote:

> On Fri, 26 Nov 2021 at 17:13, Guido van Rossum <guido@python.org> wrote:
>
> >> Although the more I think about it, given that I believe dataclasses
> >> use eval "under the hood", the less I understand *how* it manages to
> >> do that without special-case knowledge of the dataclass decorator...)
> >
> > Static checkers special-case the @dataclass decorator. Eric Traut has a
> proposal to generalize this support (sorry, I'm in a rush, otherwise I'd
> dig up the link, but it's in the typing-sig archives).
>
> :-( That's what I suspected, but it does mean that dataclasses has a
> privilege that other libraries (like attrs, I guess?) don't get.
>

Actually both are done through plugins (since what they do just doesn't fit
in the PEP 484 type system) and both have the same status, at least in
mypy. (In fact we have twice as many lines of code dedicated to attrs than
to dataclasses. https://github.com/python/mypy/tree/master/mypy/plugins)
The proposal I mentioned by Eric Traut (pyright's author) would make it
easier to support many similar libraries across all static type checkers. (
https://mail.python.org/archives/list/typing-sig@python.org/thread/TXL5LEHYX5ZJAZPZ7YHZU7MVFXMVUVWL/
)


> >> I'd like to see a clearer statement from "somewhere" about how APIs
> >> should use annotations at runtime, such that Python users have a much
> >> clearer intuition about APIs like the dataclass one, and library
> >> designers can build their APIs based on a clear "common understanding"
> >> of what to expect when annotations are used.
> >
> > Note that @dataclass itself is very careful not to use the annotations,
> it only looks for their *presence*. With one exception for ClassVar.
>
> Understood. What I'm suggesting is that it would be good to have a
> clear "common understanding" about whether libraries should be careful
> like this, or whether it's OK to base runtime behaviour on type
> annotations. And if it is OK, then what are good patterns of design
> and behaviour? This is where the proposal to store annotations as
> strings hit issues, because it appears to take the view that libraries
> *shouldn't* be looking at the actual types specified by annotations
> (or maybe that they should only do so via something like
> `typing.get_type_hints`). There are other subtleties here (runtime
> code needs to deal with the fact that int and "int" should be treated
> the same) that there's no guidance on, again possibly because no-one
> is really considering that use case.
>

You are hitting the nail on the head here. I'd say that so far the
recommendation has been "use typing.get_type_hints(x) rather than
x.__annotations__" -- this handles the equivalence between int and "int"
(and hence forward references as well as 'from __future__ import
annotations'). There's now also inspect.get_annotations() which has roughly
the same functionality without quite so much bias towards typing, so
perhaps we should recommend it over typing.get_type_hints() -- though
before 3.10 inspect.get_annotations() didn't exist so you might have to
fall back on using the other. (There are some semantic differences between
the two that I'm glossing over here, because I'm not sure about what
exactly they are. :-)

We now also have a document that recommends best practices (
https://docs.python.org/3/howto/annotations.html) although it's very new --
it appears Larry Hastings wrote it while he was pushing for PEP 649 (but it
received buy-in from the static typing community as well).

So perhaps it isn't as bad as it seems, *if* you know where to look?

That said, I don't think the current static typing infrastructure would be
prepared for an onslaught of modules that use type introspection at runtime
to modify the behavior of classes a la attrs and dataclasses. I don't know
enough about pydantic to say whether it also falls in this category. But
it's definitely difficult to write code that makes use of type annotations
at runtime that *also* passes static type checks by mypy etc.; you
certainly shouldn't attempt to do so without having CI jobs to run the
static checker and test the runtime-introspecting framework you're using.
(It's not quite like trying to write code that's valid Python and Fortran
at the same time, but it's not trivial. :-) There are also features of our
static type system that tend not to be supported by the
runtime-introspecting frameworks -- in particular, I'd expect generics and
callable types to be hard to deal with at runtime. It's easy enough to do
something at runtime with `def f(a: list[int]) -> int`. It's not so simple
to handle `def f(a: Sequence[T]) -> T` or `def f(cb: (T) -> tuple[str, T],
Sequence[T]) -> Mapping[str, T]`. Presumably frameworks like pydantic just
don't support such things and tell the user not to do that. (Someone should
look where pydantic draws the line and report back here.)



>
> Paul
>
> PS I've never written code myself that does runtime introspection of
> type annotations - so it's quite possible that there *is* guidance
> that I've just missed. But it wasn't obvious to me from a quick search
> - the "introspection helpers" section of the typing module docs is
> pretty basic...
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

> There may > be scoping issues to be sorted out,

> Yes, the scoping issues are the main problem.


> > but I don't think they are
> > insurmountable.

Probably not — and LLukas Langa has some good ideas that the SC is
considering.

(Sorry I can’t get your name right in a phone)

Maybe inspect.get_annotations() does, or will be able to, solve many of
these issues.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Fri, Nov 26, 2021 at 1:47 PM Guido van Rossum <guido@python.org> wrote
> It's easy enough to do something at runtime with `def f(a: list[int]) ->
int`. It's not so simple to handle `def f(a: Sequence[T]) -> T` or `def
f(cb: (T) -> tuple[str, T], Sequence[T]) -> Mapping[str, T]`. Presumably
frameworks like pydantic just don't support such things and tell the user
not to do that.

I don’t know about Pydantic, but that’s exactly my use case: the type needs
to be an actual intstantiatable type.

So list and tuple is fine, and Sequence[t] won’t work. But it’s fine that
this use case is restricted.

In fact, other than the basic core types, you need to use specialized types
with this system anyway.

-CHB
--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

So does your library work both with and without ‘from __future__ import
annotations’? If it does, you shouldn’t have to worry. If it doesn’t, it
would be useful if you could post some detailed examples of what goes wrong.

On Fri, Nov 26, 2021 at 14:24 Christopher Barker <pythonchb@gmail.com>
wrote:

>
>
> On Fri, Nov 26, 2021 at 1:47 PM Guido van Rossum <guido@python.org> wrote
> > It's easy enough to do something at runtime with `def f(a: list[int])
> -> int`. It's not so simple to handle `def f(a: Sequence[T]) -> T` or `def
> f(cb: (T) -> tuple[str, T], Sequence[T]) -> Mapping[str, T]`. Presumably
> frameworks like pydantic just don't support such things and tell the user
> not to do that.
>
> I don’t know about Pydantic, but that’s exactly my use case: the type
> needs to be an actual intstantiatable type.
>
> So list and tuple is fine, and Sequence[t] won’t work. But it’s fine
> that this use case is restricted.
>
> In fact, other than the basic core types, you need to use specialized
> types with this system anyway.
>
> -CHB
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>
--
--Guido (mobile)

Paul Moore wrote:
> More hazy memories here, but I think the original proposal left open
> the possibility of annotations not being types at all - for example,
> being docstrings for the arguments, or option names for a "function
> call to CLI" tool, etc.

Absolutely.

While it was clear that Guido's own use cases were about typing, annotations were explicitly not limited to typing, which is one reason why some of the later changes have felt to some people like bait and switch. Maybe it is already too late to avoid that.

> ... the expectation was that annotations
> would be *types*,

Even from the start, it was assumed that they would be objects. (Specifically types was expected to be common, but not universal.) The particular way strings are being substituted for evaluated objects has sometimes reminded me of raising a string instead of an exception class/object. It will work, but it can seem sloppy, and it can be annoying if you were assuming otherwise and suddenly have to add a bunch of evals. (That said, I haven't yet been sufficiently motivated to even tease out exactly what the problems are, let alone to propose an alternative that also satisfies the typing fans -- in part because it feels like the obvious optimization is to just not run typing, and it isn't clear what middle grounds are generally worthwhile.)

>... personally, I have the same discomfort
> about using explicit string annotations for forward references, it
> feels like I'm not declaring a "proper type".
> If what I say above is right, the debate here isn't about whether
> annotations "are for types", but rather about whether reading the
> types in annotations and using them to affect behaviour *at runtime*
> is a legitimate use of annotations.

I see that as a second dispute, which I had previously missed. I think you're right, though. On the other hand, I'm not sure the solution to both isn't just a helper function that does the 2nd-pass resolution -- preferably without requiring that all the rest of typing be imported, since even the people who want to use the typing package agree that importing it is not lightweight.

> ... I lurk on the typing-sig, and from an outsider's perspective, the
> participants seem to be almost entirely designers or heavy users of
> static type checkers. That gives a certain emphasis to the proposals
> coming from that group.

At times, it sort of reminds me of OWL and "Semantic Web". There are plenty of people who will want to use annotations as a tool, but won't be willing to wade through what can feel like "How many angels can dance on the head of a pin?" discussions. That said, I'm not sure how to best reach people who just want a rough-and-ready usually-good-enough tool.

-jJ
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JL3AQ4UVEVWXYTRFAVGVHNT23W2NCUDI/
Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano wrote:
> On Sat, Nov 20, 2021 at 11:46:56PM -0800, Christopher Barker wrote:

> Maybe PEP 563 could include a decorator in the typing module to
> destringify all the annotations in a class or function?

If it were in an annotations module, that would probably be sufficient.

If it is in typing, then it is a very heavyweight dependency -- heavy enough that even the people actually using that module for development (and not for production runs) are worried about the costs. If the costs of the typing module are that high, it is not acceptable to impose them on people not otherwise using the module.

-jJ
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZQHP24T2PKRTDBGZ36KLBHLLOKITP5ON/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Nov 26, 2021 at 5:47 PM Jim J. Jewett <jimjjewett@gmail.com> wrote:

> Steven D'Aprano wrote:
> > Maybe PEP 563 could include a decorator in the typing module to
> > destringify all the annotations in a class or function?
>
> If it were in an annotations module, that would probably be sufficient.
>
> If it is in typing, then it is a very heavyweight dependency --


As of Py 3.10 there is:

inspect.get_annotations

one could write a decorator around that, but I think the real question is
when do you want the annotations to be "finalized"?

-CHB



heavy enough that even the people actually using that module for
> development (and not for production runs) are worried about the costs. If
> the costs of the typing module are that high, it is not acceptable to
> impose them on people not otherwise using the module.
>
> -jJ
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZQHP24T2PKRTDBGZ36KLBHLLOKITP5ON/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

I hope my understanding of where the SC’s debate about this currently sits isn’t a misrepresentation, but since I sent the email on behalf of the SC, let me try to clarify what I was trying to say.

Aside: A little insight into how the SC works. For communications like this, after whatever debate we have in our live weekly meetings (and sometimes off-line discussions in Slack and email), one of us volunteers to write a draft of the email. We then all comment and wordsmith until we’re in unanimous agreement that it’s ready to go out. Then usually the person who wrote it will send the email. We all have slightly different styles, but I usually sign it with my name “on behalf of the Steering Council”. That’s my way of saying that the communication comes from all of us and represents the SC’s opinion, but it generally has the written style of the original drafter.

> On Nov 25, 2021, at 13:41, Christopher Barker <pythonchb@gmail.com> wrote:
> From Barry's email:
> """
> This is also a call to you, the folks who both care deeply about typing in Python, and have a solid understanding of the subject matter (or willingness to learn) to help us! There are use cases we don’t know about, and unclear requirements from both the static and dynamic typing users.
> """
> I don't care deeply about typing, and I'm not a static or dynamic typing user. So this implies that a decision has been made that annotations are for typing, and typing only.

Practically speaking, I think that’s current reality: annotations are for typing, be they static type checking or dynamic runtime uses. I was inelegantly saying that even with that narrower interpretation of annotations, there are use cases, interactions, side-effects, semantics, and implications that we do not fully understand. PEP 563 and 649 have visible effects that even within that domain can have important side effects. For example, PEP 563’s loss of local scope, which even “de-stringify-ing” can’t recover. This is what we need help with.

> (not that you're not allowed to use for anything else, of course you are, but that other uses won;t be taken into account when designing the new interface)
>
> But I have never seen that clearly stated anywhere. The closest is from PEP 563, where it says:
>
> """
> With this in mind, uses for annotations incompatible with the aforementioned PEPs should be considered deprecated.
> """
>
> Which pretty much makes the point, but it's a bit subtle -- what does "incompatible' mean?

You make a good point. I agree that while all the signs are there for “annotations are for typing”, this has never been explicitly or sufficiently codified, and I’ve been proposing that Someone write a PEP that makes this an official pronouncement. That someone may very well be a person on the SC, but given the timing of elections, it’s likely to fall to the next SC, if they still agree with that position! :-D

My recollection of the history of annotations falls somewhere between Greg’s and Guido’s. Annotations as a feature were inspired by the typing use case (with no decision at the time whether those were to be static or runtime checks), but at the same time allowing for experimentation for other use cases. Over time, annotations-for-typing clearly won the mindset and became the predominant use case. Personally, I was strongly against type annotations because of my experience in other languages, but over time I was also won over. From library documentation, to complex code bases, to the transient nature of contributors, type annotations are pretty compelling, and most (but not all) of my worries really didn’t come to fruition.

We can lament the non-typing use of annotations, but I think that horse is out of the barn and I don’t know how you would resolve conflicts of use for typing and non-typing annotations. It’s been a slow boil, with no definitive pronouncement, and that needs to be fixed, but I think it's just acknowledging reality. That’s my personal opinion.

> well, yes. The issue is that, intended or not, typing is making it's way into Python culture. As an instructor of beginning python users, I am unsure at this point when to introduce type annotations.
>
> What is their role? Up to today, I have treated them as an advanced feature, useful for "complex codebases". But there are any number of examples springing up on the internet, to the point where many students now think they are "best practice", if not actually required.

This is an important observation. As much as I’m in the "type annotations are good” crowd now, I still think they should always be optional. Python’s use is so broad these days, I for one don’t want to have to add annotations to every bit of Python I write.

-Barry

On Nov 26, 2021, at 01:13, Paul Moore <p.f.moore@gmail.com> wrote:

> I'd therefore interpret Barry's plea as being for *anyone* with a use
> for annotations to provide their feedback (at least, anyone who
> accepts that annotations are types), with particular emphasis on
> people who want to use the types declared in annotations to affect
> runtime behaviour, as that's the most under-represented group at the
> moment (and it's not clear whether it's under-represented because
> there aren't many such uses, or because the users aren't being heard
> from).

Spot on.

-Barry

On 11/29/21 2:56 PM, Barry Warsaw wrote:
> PEP 563 and 649 have visible effects that even within that domain can have important side effects. For example, PEP 563’s loss of local scope, which even “de-stringify-ing” can’t recover. This is what we need help with.

Well, sure.  If PEP 563 and 649 didn't have visible effects, there'd be
no point in doing them.  That said, I suggest 649 does a lovely job of
avoiding /undesirable/ side-effects.

Sure, 649 has observable side effects.  For example, you can detect
whether or not 649 is active by rebinding a name after it's used in an
annotation but before examining that annotation at runtime.  This seems
harmless--unlikely to happen in production code, and easily remedied if
someone did trip over it.

A more credible side effect: if you use an undefined name in an
annotation, you won't notice at compile-time.  Now, this is actually
649's major feature!  But there are also scenarios where this feature
could cover up a bug, like if you misspell a name--you won't notice
until you examine the annotation at runtime (or, more likely, until you
run your static analyzer).  563 has this same behavior--and it wasn't
enough to prevent 563 being accepted.  So I assume this wouldn't be
enough to prevent accepting 649 either.

649 has effects on memory usage and performance, but honestly I'm not
worried about these.  I don't think the memory usage and performance of
the prototype were particularly bad.  Anyway, as I've said many times:
we should figure out the semantics we want first, and then we can worry
about optimization.  The Python core dev community has no end of smart
people who love optimizing things--I'm sure if 649 was accepted and
merged, the optimizations would start rolling in.

Then of course there are also things 649 simply doesn't do, e.g. resolve
the "if TYPE_CHECKING" situation.  But it's not appropriate to call that
a "side effect" per se.

And that's my list.  If anybody knows of other visible side effects from
649, naturally you should contact the SC.  And/or me, if you think we
could change 649 to mitigate it without losing its major features.


Happy holidays,


//arry/

Here's a use case for runtime type annotations: dataclasses use
annotations to define fields. With the exception of ClassVar, the actual
type is ignored. There's code in dataclasses.py to deal with stringized
annotations, specifically just looking for ClassVar. I'd like to see
this special case go away, assuming 649 is accepted.

And while dataclasses itself doesn't care about the types, apparently
users do, and want to be able to call get_type_hints on the annotations.
Many people actually want dataclasses to call this for them, but I've
resisted. See https://bugs.python.org/issue36643

There are plenty of bugs around this issue, some of which have been
fixed. But there are also at least a couple that haven't been fixed, and
I'm not sure they will be. I'm hoping that 649 gets accepted and I don't
have to worry about the problem at all. Here's an example of an open bug
around get_type_hints with dataclasses: https://bugs.python.org/issue45524

Eric

On 11/29/2021 6:00 PM, Barry Warsaw wrote:
> On Nov 26, 2021, at 01:13, Paul Moore <p.f.moore@gmail.com> wrote:
>
>> I'd therefore interpret Barry's plea as being for *anyone* with a use
>> for annotations to provide their feedback (at least, anyone who
>> accepts that annotations are types), with particular emphasis on
>> people who want to use the types declared in annotations to affect
>> runtime behaviour, as that's the most under-represented group at the
>> moment (and it's not clear whether it's under-represented because
>> there aren't many such uses, or because the users aren't being heard
>> from).
> Spot on.
>
> -Barry
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4522JDS2ZOYXXFGPVXRGJ67IB3YLCEFM/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ENVEVQUEMEDKT2YUI2LQOBZDJC2YMNQ5/
Code of Conduct: http://python.org/psf/codeofconduct/

On 11/29/2021 5:56 PM, Barry Warsaw wrote:

> On Nov 25, 2021, at 13:41, Christopher Barker <pythonchb@gmail.com> wrote:
>> What is their role? Up to today, I have treated them as an advanced feature, useful for "complex codebases". But there are any number of examples springing up on the internet, to the point where many students now think they are "best practice", if not actually required.

> This is an important observation. As much as I’m in the "type annotations are good” crowd now, I still think they should always be optional. Python’s use is so broad these days, I for one don’t want to have to add annotations to every bit of Python I write.

Maybe it should be reiterated with whatever decision comes forth that

>>> def muladd(x, y, z):
... return x * (y+z)
...
>>> muladd(3.1459, 87.33, 2.7e2)
1124.124447
>>> muladd(3, 5, 7)
36
>>> from fractions import Fraction as Fr
>>> muladd(Fr(22, 7), 87, Fr(2714, 100))
Fraction(62777, 175)
>>> muladd(3, 'hel', 'lo ')
'hello hello hello '

and other duck-typed code will always be legal, idiomatic, and even
expected as good practice for beginner, informal, exploratory, and
similar python code.

--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/R2HLTW5DZQGKCMJTTFTAKHX7EYQWNROG/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Nov 29, 2021 at 07:56:16PM -0500, Terry Reedy wrote:

> Maybe it should be reiterated with whatever decision comes forth that
>
> >>> def muladd(x, y, z):
> ... return x * (y+z)
> ...

[...]

> and other duck-typed code will always be legal, idiomatic, and even
> expected as good practice for beginner, informal, exploratory, and
> similar python code.

Many of the typing-related PEP comes with such a disclaimer, listed as
"Non-goals". For example:

https://www.python.org/dev/peps/pep-0484/#non-goals

?ukasz's stringified annotations PEP has a non-goals section:

https://www.python.org/dev/peps/pep-0563/#non-goals

Larry's deferred evaluation PEP does not:

https://www.python.org/dev/peps/pep-0649/

but I don't think we should hold it against PEP-649. It's not trying to
sneakily sneak mandatory static typechecking in by the back door like
some sort of sneaking sneak :-)

--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XRBNYBEK6ZBZUC2KIDBGEYKYXZ2CBF4C/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Nov 29, 2021 at 5:01 PM Terry Reedy <tjreedy@udel.edu> wrote:

> On 11/29/2021 5:56 PM, Barry Warsaw wrote:
>
> > On Nov 25, 2021, at 13:41, Christopher Barker <pythonchb@gmail.com>
> wrote:
> >> What is their role? Up to today, I have treated them as an advanced
> feature, useful for "complex codebases". But there are any number of
> examples springing up on the internet, to the point where many students now
> think they are "best practice", if not actually required.
>
> > This is an important observation. As much as I’m in the "type
> annotations are good” crowd now, I still think they should always be
> optional. Python’s use is so broad these days, I for one don’t want to
> have to add annotations to every bit of Python I write.
>
> Maybe it should be reiterated with whatever decision comes forth that
>
> >>> def muladd(x, y, z):
> ... return x * (y+z)
> ...
> >>> muladd(3.1459, 87.33, 2.7e2)
> 1124.124447
> >>> muladd(3, 5, 7)
> 36
> >>> from fractions import Fraction as Fr
> >>> muladd(Fr(22, 7), 87, Fr(2714, 100))
> Fraction(62777, 175)
> >>> muladd(3, 'hel', 'lo ')
> 'hello hello hello '
>
> and other duck-typed code will always be legal, idiomatic, and even
> expected as good practice for beginner, informal, exploratory, and
> similar python code.
>

Why would it need to be reiterated? Are there really people who believe
that such code would become invalid? AFAIK *everybody* here agrees that
this should stay valid. So who would we be reiterating it for?

(Yes, several static type checkers have options that cause the checker to
complain about unannotated code. But that's not the default behavior and
running a static checker is a choice, like running a linter. Three-space
indents or capitalized function names are never going to be disallowed
either, even though PEP 8 says that's not how you ought to code.)

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 11/29/2021 8:16 PM, Guido van Rossum wrote:
> On Mon, Nov 29, 2021 at 5:01 PM Terry Reedy <tjreedy@udel.edu
> <mailto:tjreedy@udel.edu>> wrote:
>
> On 11/29/2021 5:56 PM, Barry Warsaw wrote:
>
> > On Nov 25, 2021, at 13:41, Christopher Barker
> <pythonchb@gmail.com <mailto:pythonchb@gmail.com>> wrote:
> >> What is their role? Up to today, I have treated them as an
> advanced feature, useful for "complex codebases".


> But there are any
> number of examples springing up on the internet, to the point where
> many students now think they are "best practice", if not actually
> required.

Chris, can you say anything more about why people would get such a
mis-impression?

> > This is an important observation.  As much as I’m in the "type
> annotations are good” crowd now, I still think they should always be
> optional.  Python’s use is so broad these days, I for one don’t want
> to have to add annotations to every bit of Python I write.
>
> Maybe it should be reiterated with whatever decision comes forth that
>
>  >>> def muladd(x, y, z):
> ...     return x * (y+z)
> ...
>  >>> muladd(3.1459, 87.33, 2.7e2)
> 1124.124447
>  >>> muladd(3, 5, 7)
> 36
>  >>> from fractions import Fraction as Fr
>  >>> muladd(Fr(22, 7), 87, Fr(2714, 100))
> Fraction(62777, 175)
>  >>> muladd(3, 'hel', 'lo ')
> 'hello hello hello '
>
> and other duck-typed code will always be legal, idiomatic, and even
> expected as good practice for beginner, informal, exploratory, and
> similar python code.
>

> Why would it need to be reiterated? Are there really people who believe
> that such code would become invalid? AFAIK *everybody* here agrees that
> this should stay valid. So who would we be reiterating it for?

The people that Chris B. referred to, and their 'sources'. But we need
more info to know how to best counter such wrong beliefs.

--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KLJ7UCGAWQ7X5EW6WPHBIH2XC3DIVEFZ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Nov 29, 2021, at 15:57, Larry Hastings <larry@hastings.org> wrote:
>
> On 11/29/21 2:56 PM, Barry Warsaw wrote:
>> PEP 563 and 649 have visible effects that even within that domain can have important side effects. For example, PEP 563’s loss of local scope, which even “de-stringify-ing” can’t recover. This is what we need help with.
> Well, sure. If PEP 563 and 649 didn't have visible effects, there'd be no point in doing them. That said, I suggest 649 does a lovely job of avoiding undesirable side-effects.
>
> Sure, 649 has observable side effects. For example, you can detect whether or not 649 is active by rebinding a name after it's used in an annotation but before examining that annotation at runtime. This seems harmless--unlikely to happen in production code, and easily remedied if someone did trip over it.

PEP 563 also has visible side effects, but the real implication of them wasn’t well understood at the time the PEP was approved. Or maybe it was and we underestimated the impact on certain important users and use cases. I’m particularly wary about getting into that same (um) pickle again.

> A more credible side effect: if you use an undefined name in an annotation, you won't notice at compile-time. Now, this is actually 649's major feature! But there are also scenarios where this feature could cover up a bug, like if you misspell a name--you won't notice until you examine the annotation at runtime (or, more likely, until you run your static analyzer). 563 has this same behavior--and it wasn't enough to prevent 563 being accepted. So I assume this wouldn't be enough to prevent accepting 649 either.

How common is it for code to examine annotations at run time, outside of specialized libraries that have to be more defensive anyway (especially given they already have to be defensive in light of 563)?

> 649 has effects on memory usage and performance, but honestly I'm not worried about these. I don't think the memory usage and performance of the prototype were particularly bad. Anyway, as I've said many times: we should figure out the semantics we want first, and then we can worry about optimization. The Python core dev community has no end of smart people who love optimizing things--I'm sure if 649 was accepted and merged, the optimizations would start rolling in.

I agree about that priority (semantics first, optimization later), as long as those semantics don’t make such future optimizations impossible, or severely degrade important performance constraints such as import times.

> Then of course there are also things 649 simply doesn't do, e.g. resolve the "if TYPE_CHECKING" situation. But it's not appropriate to call that a "side effect" per se.

Although, it would be nice to have a comprehensive solution to this and other situations too, even if they happen outside of PEP 563/649 discussion…

> And that's my list. If anybody knows of other visible side effects from 649, naturally you should contact the SC. And/or me, if you think we could change 649 to mitigate it without losing its major features.

That list is exactly what I'm (we’re?) looking for.

-Barry

On 11/30/2021 1:16 AM, Guido van Rossum wrote:
> Why would it need to be reiterated? Are there really people who believe
> that such code would become invalid? AFAIK *everybody* here agrees that
> this should stay valid. So who would we be reiterating it for?
>
> (Yes, several static type checkers have options that cause the checker
> to complain about unannotated code. But that's not the default behavior
> and running a static checker is a choice, like running a linter.
> Three-space indents or capitalized function names are never going to be
> disallowed either, even though PEP 8 says that's not how you ought to code.)

I know it sounds like a straw-man argument, but I don't think it is. I
just think the non-straw-men making the argument aren't "important"
enough to call out specifically - they are "random individual
programmers who have gotten the wrong impression from another language
and applied it to Python" rather than "influencers trying to change how
we code".[1]

And we can always say that the *language* does not require running the
type checker, but the reality for many developers is that the projects
and teams they are part of will require it. And often, they require it
because it's some kind of global "Best Practice" rather than because it
makes sense for their particular situation.

For example, I saw a snippet fly by on Twitter the other day (again, not
the fault of the individual who posted it, so I'm not
naming-and-shaming) showing how to use Black to format a big directory
of Jupyter notebooks. Convenient, perhaps, but the vast majority of
people with "big directories of Jupyter notebooks" are going to be much
happier if you just leave them alone! And yet, "The Community" is
suggesting we should format all of them in bulk.

THAT'S the kind of thing that also has been happening with typing, and
why some of us feel the need to publicly re-state things that are all
agreed upon within this group, but are struggling to be heard over the
public discourse on the topics. And this kind of reiteration is easier
when our official documents (PEPs, etc.) state it explicitly.

Cheers,
Steve


[1]: Okay, I will call out the Python Bytes podcast - which I've been on
a couple of times - as one that regularly opines on topics based on very
little information and says things more definitively than they've been
said anywhere else. They have a much bigger (and more easily influenced)
audience than python-dev.

I'll also say that what I see from LWN is generally very good in this
respect, their writers handle things well. Though I don't tend to read
or listen to either of these that often.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SFLYLQO4YVQGV3BUKQO7KS6N367HNG62/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Nov 18, 2021 at 8:00 AM Barry Warsaw <barry@python.org> wrote:
>
> Does PEP 563 or 649 satisfy static and dynamic typing needs?
>
> In the interest of full transparency, we want to let the Python community know that the Steering Council continues to discuss PEP 563 (Postponed Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). We have concluded that we lack enough detailed information to make a decision in favor of either PEP. As we see it, adopting either PEP 563 or PEP 649 as the default would be insufficient. They will not fully resolve the existing problems these PEPs intend to fix, will break some existing code, and likely don’t address all the use cases and requirements across the static and dynamic typing constituents. We are also uncertain as to the best migration path from the current state of affairs.
>
> Defer decision on PEP 563 and 649 in 3.11
>
> As such, at this time, the only reasonable path forward that the SC sees is to defer the decision in Python 3.11 again, essentially keeping the 3.10 status quo. We know that this is far from ideal, but it’s also the safest path since we can’t clearly make the situation better, and we don’t have confidence that either PEP solves the problems once and for all. Pragmatically, we don’t want to make the situation worse, and we really don’t want to find ourselves back here again in a couple of releases because we overlooked an important requirement for a set of users.
>

In my opinion, we can agree that we can not make PEP 563 default in
the future because it will break too many use cases.
Anyone against making a statement that "PEP 563 will never be the
default behavior"?

Then, we do not need to decide "PEP 563 or 649".
We can focus on whether we can replace "stock semantics + opt-in PEP
563" with PEP 649 or not.

Regards,
--
Inada Naoki <songofacandy@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KCZKREGT66VE67MPT7NNJ5UPCACY6X7A/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Nov 29, 2021 at 8:17 PM Guido van Rossum <guido@python.org> wrote:

> Why would it need to be reiterated? Are there really people who believe
> that such code would become invalid? AFAIK *everybody* here agrees that
> this should stay valid. So who would we be reiterating it for?
>

I'm certainly not alone, among people on this list, in regard to the
following. But I teach a lot of people who are coders, but not necessarily
senior (in Python, or otherwise). I also mentor/lead such junior
programmers.

I find it quite common when people who haven't known Python for 20 years
see type annotations, they have trouble getting the optional and gradual
nature of them. Explaining that is certainly not impossible, nor even all
that difficult. However, it DOES need to be explained. Especially when
less experienced Pythonistas have worked with statically typed languages,
they often see code with type annotations and overgeneralize to their
requirement.

I don't know exactly what that means about *where* this needs to be
explained. Maybe just by trainers and authors. But maybe things in
official Python documentation as well, which seem more definitive.

--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.

On 11/29/21 7:10 PM, Inada Naoki wrote:
> Anyone against making a statement that "PEP 563 will never be the
> default behavior"?

I think only the SC is empowered to make such a statement.


> Then, we do not need to decide "PEP 563 or 649".
> We can focus on whether we can replace "stock semantics + opt-in PEP
> 563" with PEP 649 or not.

I doubt the current status quo--keeping PEP 563 as optional behavior,
permanently--is viable long-term.  It causes problems for most runtime
uses of annotations, which we were collectively basically ignoring until
now.  As runtime use of annotations /and/ typing both become more
pervasive, these problems are only going to grow.


//arry/

I don't think the issue of how typing is making its way intot he community
is particular relevant to this thread, but I was directly asked a question,
so I'll answer it:

I don't know exactly where the impression is coming from that typing is a
best practice, but it's certainly creeping into example code, etc. So folks
will see examples of code that have nothing to do with typing, and probably
have no need for it, and there will be type annotations. Many (most?) folks
learn from examples more than targeted instruction, so as those examples
grow, so will the impression that it's an inherent part of Python.

David Mertz' experience is similar to mine as an instructor.

Another anecdotal example: I've been teaching an intro to Python class for
years, based on the same material, that we try to keep up to date. But
typing is not yet included. However, about a year ago, one of the other
instructors (that had not previously developed materials for the class)
added a new example / exercise to the "intro to classes" lesson. And that
example had type annotations. I ended up removing them, as I thought it
could be pretty confusing to students to see them for the first time with
no explanation (would they think it was something specific to classes?). I
wish I'd discussed it with the other instructor, I don't know why that was
done -- was it standard practice in their workplace? Seemed like a best
practice? I have no idea.

Another observation is that more and more library authors are being asked
to add type annotations -- maybe that IS a best practice for widely used
libraries, but that's one more piece of data.

Also: I know this isn't intentional, but Guido has an enormous influence on
the Python community -- so I suspect that the fact that he's taken an
active role in developing static typing has influenced how it's being
perceived.

I do think this is a topic for the community to grapple with, but not
really a Python-dev responsibility.

-CHB


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Tue, 30 Nov 2021 at 02:52, Steve Dower <steve.dower@python.org> wrote:
>
> THAT'S the kind of thing that also has been happening with typing, and
> why some of us feel the need to publicly re-state things that are all
> agreed upon within this group, but are struggling to be heard over the
> public discourse on the topics. And this kind of reiteration is easier
> when our official documents (PEPs, etc.) state it explicitly.

This is very definitely the case. There's a subtle (maybe not so
subtle, actually) and increasing pressure on projects to add typing.
Often with little or no justification beyond "you should", as if
having typing is a sort of "obvious best practice". Sometimes "because
it will make it easier for your users who use typing" is given as a
justification, but while that's fair, it's also a disturbing gradual
pressure for typing to extend everywhere, manifesting by making it
feel like not adding typing is somehow "not caring about your users".

Also, related to the question Terry raised, IMO it would be useful to
have a clear statement on code that *does* use type annotations, but
violates them at runtime. To be specific, is the following considered
as an error?

>>> def muladd(x: int, y: int, z: int) -> int:
... return x * (y+z)
...
>>> muladd(3.1459, 87.33, 2.7e2)
1124.124447

If (as in the current interpreter) typing is optional, this isn't an
error. Certainly, running mypy over it fails, but the point here is
that mypy is an external tool - the Python interpreter itself (and the
language definition) allows it. I think it would be useful to have a
clear message that this is intended, and won't be altered lightly.
Remember that the "correct" annotation here is either muladd(x: Any,
y: Any, z: Any) -> Any, or muladd(x: SomeMultiplyProtocol, y:
SomeAddProtocol, z: SomeAddProtocol) -> Any, neither of which is
particularly helpful...

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E36ILTCKFVW3MNFU753PKKWIFY2FWQXU/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Nov 30, 2021 at 8:19 PM Paul Moore <p.f.moore@gmail.com> wrote:
> Also, related to the question Terry raised, IMO it would be useful to
> have a clear statement on code that *does* use type annotations, but
> violates them at runtime. To be specific, is the following considered
> as an error?
>
> >>> def muladd(x: int, y: int, z: int) -> int:
> ... return x * (y+z)
> ...
> >>> muladd(3.1459, 87.33, 2.7e2)
> 1124.124447
>

My understanding is that it's precisely as wrong, or not wrong, as this:

def add(x, y):
"""Multiply two numbers"""
return x / y

To my mind, annotations are machine-readable metadata, with no
inherent "correctness" to them (from the language's point of view),
other than syntactically.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/54VRD5SJ3RQT6W4AX5WPIW2OUWZNWPZO/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, 30 Nov 2021 at 09:37, Chris Angelico <rosuav@gmail.com> wrote:
>
> On Tue, Nov 30, 2021 at 8:19 PM Paul Moore <p.f.moore@gmail.com> wrote:
> > Also, related to the question Terry raised, IMO it would be useful to
> > have a clear statement on code that *does* use type annotations, but
> > violates them at runtime. To be specific, is the following considered
> > as an error?
> >
> > >>> def muladd(x: int, y: int, z: int) -> int:
> > ... return x * (y+z)
> > ...
> > >>> muladd(3.1459, 87.33, 2.7e2)
> > 1124.124447
> >
>
> My understanding is that it's precisely as wrong, or not wrong, as this:
>
> def add(x, y):
> """Multiply two numbers"""
> return x / y
>
> To my mind, annotations are machine-readable metadata, with no
> inherent "correctness" to them (from the language's point of view),
> other than syntactically.

That makes sense to me, and I'd support that being clearly stated as
the "official position".

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LUY2SXBGX4TQKTX6PE2FHYHABDCYY2IN/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, 26 Nov 2021 at 09:14, Paul Moore <p.f.moore@gmail.com> wrote:

> I'd therefore interpret Barry's plea as being for *anyone* with a use
> for annotations to provide their feedback (at least, anyone who
> accepts that annotations are types), with particular emphasis on
> people who want to use the types declared in annotations to affect
> runtime behaviour, as that's the most under-represented group at the
> moment (and it's not clear whether it's under-represented because
> there aren't many such uses, or because the users aren't being heard
> from).


I don't know if this helps, but I'm a maintainer/creator of a Python library
that uses type annotations at runtime. The library is used to create GraphQL
APIs[1]. We took a lot of inspiration from dataclasses, our API looks like
this:

```python
@strawberry.type
class User:
name: str
```

The `strawberry.type` decorator reads the annotations and creates an object
with a list
of fields and types, but it doesn't try to resolve them (that happens later
to prevent issues
with circular dependencies and so on).

I remember the discussion around PEP 563 and how it would break libraries
using type annotations,
in our case things are fine since the types we define should be defined in
the global namespace[2]
anyway. I think the only issue we have with type annotations is when you
have circular references
inside your python code (where you'd use `if TYPE_CHECKING: import X`), we
"solved" that with
a custom class called `LazyType[type, module]`, that allows to specify a
type and where it comes
from, so our library can resolve the proper type, even if it was not
imported in the namespace of the
class using that type.

Sorry for the digression into some of the details of how the library works,
I hope it helps a bit
understanding the use case.

I think there's definitely use cases for type hints being used at runtime.
I personally love
the fact that we are allowed to do this, in other languages (like
TypeScript) you don't have
access to the types at runtime and it does prevent the creation of
libraries like dataclasses
or mine, which is a bit of a shame (you'd need to use code generation to
get a similar experience).

There's also other libraries that use type hints at runtime (not for
runtime type validation[3]):

- Typer - http://typer.tiangolo.com
- SQLModel - http://sqlmodel.tiangolo.com
- Beanie - https://github.com/roman-right/beanie/
- Pydantic - http://pydantic-docs.helpmanual.io
- odmantic - https://github.com/art049/odmantic/
- singledispatch -
https://docs.python.org/3/library/functools.html#functools.singledispatch


[1] If you're not familiar with GraphQL there's some info here:
https://graphql.org
[2] I did ask if there was a way to resolve types when having them
encapsulated into functions,
but that was a specific use case with tests and I was/am fine with
using global there

https://mail.python.org/archives/list/typing-sig@python.org/thread/SNKJB2U5S74TWGDWVD6FMXOP63WVIGDR/#A5JFR7RDE7NNKESM2DWYM3ZT5ZYGOLCF

[3] while I see runtime validation being something useful, I think it is
worth taking a look at
different use cases for type hints :)

--
Patrick Arminio

On 29 Nov 2021, at 23:56, Barry Warsaw wrote:

> [...]
>> (not that you're not allowed to use for anything else, of course you
>> are, but that other uses won;t be taken into account when designing
>> the new interface)
>>
>> But I have never seen that clearly stated anywhere. The closest is
>> from PEP 563, where it says:
>>
>> """
>> With this in mind, uses for annotations incompatible with the
>> aforementioned PEPs should be considered deprecated.
>> """
>>
>> Which pretty much makes the point, but it's a bit subtle -- what does
>> "incompatible' mean?
>
> You make a good point. I agree that while all the signs are there for
> “annotations are for typing”, this has never been explicitly or
> sufficiently codified, and I’ve been proposing that Someone write a
> PEP that makes this an official pronouncement. That someone may very
> well be a person on the SC, but given the timing of elections, it’s
> likely to fall to the next SC, if they still agree with that position!
> :-D
>
> My recollection of the history of annotations falls somewhere between
> Greg’s and Guido’s. Annotations as a feature were inspired by the
> typing use case (with no decision at the time whether those were to be
> static or runtime checks), but at the same time allowing for
> experimentation for other use cases. Over time,
> annotations-for-typing clearly won the mindset and became the
> predominant use case. Personally, I was strongly against type
> annotations because of my experience in other languages, but over time
> I was also won over. From library documentation, to complex code
> bases, to the transient nature of contributors, type annotations are
> pretty compelling, and most (but not all) of my worries really
> didn’t come to fruition.
>
> We can lament the non-typing use of annotations, but I think that
> horse is out of the barn and I don’t know how you would resolve
> conflicts of use for typing and non-typing annotations. It’s been a
> slow boil, with no definitive pronouncement, and that needs to be
> fixed, but I think it's just acknowledging reality. That’s my
> personal opinion.

But isn't that the reason why we have `typing.Annotated`, so that
annotations used for typing and annotations used for other purposes can
coexist?

An annotation used for typing only is:

```
>>> def f(x: int):
... return x
...
>>> f.__annotations__['x']
<class 'int'>
```

An annotation used for something else is:

```
>>> def f(x: 'something not typing related'):
... return x
...
>>> f.__annotations__['x']
'something not typing related'
```

`typing.Annotated` gives us both:

```
>>> from typing import *
>>> def f(x: Annotated[int, 'something not typing related']):
... return x
...
>>> f.__annotations__['x'].__args__[0]
<class 'int'>
>>> f.__annotations__['x'].__metadata__[0]
'something not typing related'
```

Granted, for someone who only wants to use annotations for their own
purpose, digging out the original not typing related stuff from
`typing.Annotated` is more work, but doable.

Or am I missing something here?

> [...]

Servus,
Walter

On Tue, Nov 30, 2021 at 09:17:13AM +0000, Paul Moore wrote:

> Also, related to the question Terry raised, IMO it would be useful to
> have a clear statement on code that *does* use type annotations, but
> violates them at runtime. To be specific, is the following considered
> as an error?
>
> >>> def muladd(x: int, y: int, z: int) -> int:
> ... return x * (y+z)
> ...
> >>> muladd(3.1459, 87.33, 2.7e2)
> 1124.124447

Isn't that just duck-typing? You've got a function that is documented as
taking arguments of one type (int), but actually will work with any
numeric type (and some non-numeric types) that quacks like an int.

muladd(2, 'ab', 'cd') # returns 'abcdabcd'

If you replaced the type annotations with a docstring that listed the
parameters x, y, z as int, would duck-typing be wrong? Maybe.

I don't think we can make any concrete claims here. Sometimes
duck-typing is perfectly fine. Sometimes its not. It depends on the
function's implementation and its semantics. Sometimes calling a
function with a duck-typed value seems to work fine but the result is
meaningless junk. (Garbage In, Garbage Out.)

I guess all we can say is that the Python language is agnostic and
neutral on this matter.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/W24VGDUH6F43C6VT4OUGERNEZFW223MP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, 30 Nov 2021 at 09:23, Paul Moore <p.f.moore@gmail.com> wrote:
>
> On Tue, 30 Nov 2021 at 02:52, Steve Dower <steve.dower@python.org> wrote:
> >
> > THAT'S the kind of thing that also has been happening with typing, and
> > why some of us feel the need to publicly re-state things that are all
> > agreed upon within this group, but are struggling to be heard over the
> > public discourse on the topics. And this kind of reiteration is easier
> > when our official documents (PEPs, etc.) state it explicitly.
>
> This is very definitely the case. There's a subtle (maybe not so
> subtle, actually) and increasing pressure on projects to add typing.
> Often with little or no justification beyond "you should", as if
> having typing is a sort of "obvious best practice". Sometimes "because
> it will make it easier for your users who use typing" is given as a
> justification, but while that's fair, it's also a disturbing gradual
> pressure for typing to extend everywhere, manifesting by making it
> feel like not adding typing is somehow "not caring about your users".

I suspect that a big source of this is editors that use typing for
autocompletion etc. Speaking as a vim user this is something that has
passed me by except that I have students who I see adding type hints
even though I deliberately haven't taught them anything about type
hints. I can only presume that some editor is doing this for them or
telling them that they need to do this (students often can't tell the
difference between editor warnings and actual errors).

Others have mentioned the pressure on libraries to adopt typing and
I've certainly noticed this with SymPy. I think type hints could be
good for SymPy for internal use but it seems that a lot of users want
it for external reasons that I don't always understand but that also
seems to come partly from editors as well.

Some people apparently want to add type hints that look completely
useless to me like
def f() -> Union[MyClass, Any]
As I understand it this does not give any meaningful information to a
type checker but it apparently makes vscode work better:
https://github.com/sympy/sympy/pull/22180#discussion_r718917577

Here in SymPy you can see a suggestion that type hints are needed
because otherwise vscode is slow and uses 4GB of memory just to load
documentation for SymPy functions:
https://github.com/sympy/sympy/issues/17945#issuecomment-928798977

Not to do with editors but also indicative is this PR whose title
implies that SymPy is "incompatible" with PEP 561 (a PEP that I had
never heard of before seeing the PR):
https://github.com/sympy/sympy/pull/22337

There are other open "issues" like this for SymPy where the
presumption is that not having type hints is now to be considered a
deficiency of the library regardless of whether the hints have any
benefit for internal use. I don't object to adding the hints but it's
a huge amount of work that someone would have to do and I don't want
to add useless/inaccurate hints temporarily (as some have suggested to
do).

--
Oscar
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YQ773X24AICPLPLIN742B25WNVZJHMRO/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, 30 Nov 2021 at 12:39, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:

> There are other open "issues" like this for SymPy where the
> presumption is that not having type hints is now to be considered a
> deficiency of the library regardless of whether the hints have any
> benefit for internal use. I don't object to adding the hints but it's
> a huge amount of work that someone would have to do and I don't want
> to add useless/inaccurate hints temporarily (as some have suggested to
> do).

This is precisely the sort of concern I have. Although it is awfully
tempting to passive-aggressively annotate everything as Any, just to
shut people up :-(

And to be clear, it's often very non-obvious how to annotate something
- in https://github.com/pfmoore/editables I basically gave up because
I couldn't work out how to write a maintainable annotation for an
argument that is "a Path, or something that can be passed to the Path
constructor to create a Path" (it's essentially impossible without
copy/pasting the argument annotation for the Path constructor). I also
spent a lot of time trying to deal with a typeshed bug. And even
worse, if I put the types in a .pyi file (which I want to do, because
I don't want the library to import typing at runtime because it's a
significant overhead for something that's supposed to be lightweight),
apparently mypy won't check them so I gain no benefit for the project
itself.

Anyway, we're *way* off topic now, and I doubt there's much that the
SC or python-dev can do to change the community view on typing,
anyway.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4DUJM7MNJSJ3TOYYFILQCG4RT4CU6DME/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Nov 30, 2021 at 02:30:18PM +0000, Paul Moore wrote:

> And to be clear, it's often very non-obvious how to annotate something
> - in https://github.com/pfmoore/editables I basically gave up because
> I couldn't work out how to write a maintainable annotation for an
> argument that is "a Path, or something that can be passed to the Path
> constructor to create a Path" (it's essentially impossible without
> copy/pasting the argument annotation for the Path constructor).

I thought that type inference was supposed to solve that sort of
problem? If the typechecker can see that an argument is passed to the
Path constructor, it should be able to infer that it must be the same
types as accepted by Path.

Aside: I'm a little disappointed in the way the typing ecosystem has
developed. What I understood was that we'd get type inference like ML or
Haskell use, so we wouldn't need to annotate *everything*, only the bits
needed to resolve ambiguity. But what we seem to have got is typing like
C, Pascal and Java, except gradual. Am I being unreasonable to be
disappointed? I'm not a heavy mypy user, I just dabble with it
occasionally, so maybe I've missed something.



> Anyway, we're *way* off topic now, and I doubt there's much that the
> SC or python-dev can do to change the community view on typing,
> anyway.

Heh. We could update PEP 8 to ban type annotations, then watch as the
people who over-zealously apply PEP 8 to everything AND over-zealously
insist on adding type annotations to everything have their heads
explode.


--
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y6GOWIZV5JCOG5TP4ZZ4SVLXL4BGDJTI/
Code of Conduct: http://python.org/psf/codeofconduct/

This is great Patrick, thanks.

My use case is similar to Patrick's, except it builds on dataclasses
directly:

It's part of a larger system, but I've just pulled it out into its own
poorly documented and poorly tested package:

https://github.com/PythonCHB/flexi

(I think it's generally useful, so may one day make a proper package out of
it)

Anyway, the principle is this:

The goal is to have a flexible data model that enforces the structure, but
not necessarily the validity of values, or even types. This is useful
because we need to store / edit / import complex hierarchy of data, and we
very much don't want to have all of it to have to be valid in order to even
store it. An example is a dataset we recently imported (from
spreadsheets--arrgghh), where in a thousand records, they had put "too
viscous to measure" instead of a value for interfacial tension. That, of
course, would be helpful to a person reading the spreadsheet, but would
have been a pain if we had enforced that that field had to be a float > 0.
Instead, it imported fine, and then our validation code flagged the issue
to be fixed later. And all the rest of the data could be used in the
meantime.

Anyway, that was a digression (but explains why we don't use Pydantic).
Functionally, we need the "nodes" in the data model to be able to convert
themselves to/from JSON compatible Python, and validate themselves, etc.
This is done by using the actual type object, as stored in
dataclasses.Field.type

This has worked nicely for us, and dataclasses saved us a lot of
boilerplate code.

if we do

from future import Annotations

then the system breaks, as we have a string instead of the actual type that
is needed.

Solutions:

I *think* PEP 649 would work fine for us, as the annotation would get
evaluated when it was added to the field object.

But inspect.get_annotations isn't really helpful for this use, as the
Field.type attributes aren't annotations anymore. So we would have to
eval() the strings ourselves, and I think it would be unclear what
namespaces to use for that.

If dataclasses were to use

inspect.get_annotations(cls, eval_str=rue)

in order to extract the types to put into the Field objects, then I think
all would be good for us. And as I understand it, dataclasses itself
doesn't do anything with that attribute anyway.

If that's not decided to be a good thing for dataclasses, then I think we'd
have to re-implement a lot ourselves, though it could probably be hacked
in - I haven't tried that yet.

BTW, I don't think we've heard from the attrs (and especially cattrs) folks
in this thread. A search of the repo issues indicates that there has been
some discussion of PEP 563's impact, but it's not totally clear to me if
it's been resolved.

-CHB


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On 11/26/21 1:13 AM, Paul Moore wrote:
> On Fri, 26 Nov 2021 at 05:14, Guido van Rossum wrote:
>>
>> My memory is also hazy, but I'm quite sure that *in my mind* annotations were
>> intended as a compromise between conflicting proposals for *typing*. We didn't
>> have agreement on the syntax or semantics, but we did know we wanted to do
>> something with types eventually.
>
> More hazy memories here, but I think the original proposal left open
> the possibility of annotations not being types at all - for example,
> being docstrings for the arguments, or option names for a "function
> call to CLI" tool, etc. I think that some libraries took this approach
> (in particular the "CLI builder" idea).

I have one such tool. Fortunately for me, I strongly dislike having the annotations inside the function header as it
quickly gets difficult for me to read, so I was already using decorator syntax for the annotations. When PEP 484 was
accepted I just changed where the annotations were being stored from __annotations__ to my own dunder.

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VC6NSJ4MXJOTNBKO7OOBSV7JHQ5JBPLO/
Code of Conduct: http://python.org/psf/codeofconduct/

Am 30.11.21 um 13:39 schrieb Oscar Benjamin:
> Others have mentioned the pressure on libraries to adopt typing and
> I've certainly noticed this with SymPy. I think type hints could be
> good for SymPy for internal use but it seems that a lot of users want
> it for external reasons that I don't always understand but that also
> seems to come partly from editors as well.

Please note that users of you library usually won't care that the
library uses type hints. It's more important that there are type hints
for the API, which can also be supplied using a stub file.

> Some people apparently want to add type hints that look completely
> useless to me like
> def f() -> Union[MyClass, Any]
> As I understand it this does not give any meaningful information to a
> type checker but it apparently makes vscode work better:
> https://github.com/sympy/sympy/pull/22180#discussion_r718917577

We have several resources that can help with questions like these:

* https://github.com/python/typing/discussions is a forum for
questions about typing.
* The forum also has a separate section where users can ask for
reviews of type annotations, for example in stub files.
* https://gitter.im/python/typing is a chat for typing-related questions.

We are also working on a documentation hub at
https://typing.readthedocs.io/, but there is not much to see at the moment.

> There are other open "issues" like this for SymPy where the
> presumption is that not having type hints is now to be considered a
> deficiency of the library regardless of whether the hints have any
> benefit for internal use. I don't object to adding the hints but it's
> a huge amount of work that someone would have to do and I don't want
> to add useless/inaccurate hints temporarily (as some have suggested to
> do).

Please note that users of you library usually won't care that the
library uses type hints. It's more important that there are type hints
for the API, which can also be supplied using a stub file. This is
usually quite a bit easier than typing an existing code base.
Alternatively, just adding type annotations to the public API, but not
using type checking yourself, can be useful for your users.

 - Sebastian

On Wed, 1 Dec 2021 at 12:08, Sebastian Rittau <srittau@rittau.biz> wrote:

> Please note that users of you library usually won't care that the library uses type hints. It's more important that there are type hints for the API, which can also be supplied using a stub file.

I tried that route, but I was informed that if I do that, mypy will
not check my stubs against the source code. Which means that there's
no easy way of testing that the stubs are correct - is that accurate?

Paul

PS I appreciate the links you posted to various typing forums, but IMO
the most critical missing resource is a central set of typing
documentation that includes examples, FAQs and best practices as well
as reference materials. Typically when I hit an issue with types, I
want to research my own solution, not ask a question, which requires
me to bother other people, as well as interrupting my flow while I
wait for a response. I'd much rather see work being done on
documenting what we currently have in typing, rather than yet more
changes which while no doubt useful, is effectively just churn that
makes maintaining a codebase that has to support multiple Python
versions harder.

PPS Sorry if this sounds negative. TBH, I'd quite happily not use
typing if I didn't want to and stay quiet. A lot of the frustration I
see being expressed here (including my own) seems to come from the
fact that it's so difficult to actually take that sort of "I can
ignore it if I don't use it" attitude, whether that's because of
community pressure, tool requirements, or whatever.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LAR7R5MCAC74N7H7NQXJUGXG7LCOA4CB/
Code of Conduct: http://python.org/psf/codeofconduct/

On 2021-11-30 09:01, Steven D'Aprano wrote:
>
> Heh. We could update PEP 8 to ban type annotations, then watch as the
> people who over-zealously apply PEP 8 to everything AND over-zealously
> insist on adding type annotations to everything have their heads
> explode.
>

Captain Kirk would be proud:

https://memory-alpha.fandom.com/wiki/Induced_self-destruction

-Mike


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/D464YHCBENUCEF6E7E77EKVLPG2JY5WT/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, 1 Dec 2021 at 12:12, Sebastian Rittau <srittau@rittau.biz> wrote:
>
> Am 30.11.21 um 13:39 schrieb Oscar Benjamin:
>
>> Others have mentioned the pressure on libraries to adopt typing and
>> I've certainly noticed this with SymPy. I think type hints could be
>> good for SymPy for internal use but it seems that a lot of users want
>> it for external reasons that I don't always understand but that also
>> seems to come partly from editors as well.
>
> Please note that users of you library usually won't care that the library uses type hints. It's more important that there are type hints for the API, which can also be supplied using a stub file.

The distinction between internal code and API is not always that clear
in SymPy but in any case it is still very hard to type many of SymPy's
public API functions. You are right that there is a distinction here
because when I imagine that type hints could be useful for SymPy I
think of adding hints in places that users definitely don't care
about. Adding hints for public API is more immediately useful for
users but also much harder because many public APIs are quite
liberal/flexible in what they accept and return.

>> Some people apparently want to add type hints that look completely
>> useless to me like
>> def f() -> Union[MyClass, Any]
>> As I understand it this does not give any meaningful information to a
>> type checker but it apparently makes vscode work better:
>> https://github.com/sympy/sympy/pull/22180#discussion_r718917577
>
> We have several resources that can help with questions like these:
>
> https://github.com/python/typing/discussions is a forum for questions about typing.
> The forum also has a separate section where users can ask for reviews of type annotations, for example in stub files.
> https://gitter.im/python/typing is a chat for typing-related questions.
> We are also working on a documentation hub at https://typing.readthedocs.io/, but there is not much to see at the moment.

I'm not sure what your point is here. Are you suggesting that I use
these forums or that I redirect other people there?

>> There are other open "issues" like this for SymPy where the
>> presumption is that not having type hints is now to be considered a
>> deficiency of the library regardless of whether the hints have any
>> benefit for internal use. I don't object to adding the hints but it's
>> a huge amount of work that someone would have to do and I don't want
>> to add useless/inaccurate hints temporarily (as some have suggested to
>> do).
>
> Please note that users of you library usually won't care that the library uses type hints. It's more important that there are type hints for the API, which can also be supplied using a stub file. This is usually quite a bit easier than typing an existing code base. Alternatively, just adding type annotations to the public API, but not using type checking yourself, can be useful for your users.

My previous statement still stands: it's a huge amount of work and I
do not want to add bogus hints as a stopgap. If the hints are to be
added then someone needs to do that hard work to make sure that they
are accurate.

--
Oscar
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UWG2SVMAHLPOPR25DESC4GCD6TKSQPFB/
Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Nov 30, 2021 at 5:05 PM Steven D'Aprano <steve@pearwood.info> wrote:

> On Tue, Nov 30, 2021 at 02:30:18PM +0000, Paul Moore wrote:
> [...]
> Aside: I'm a little disappointed in the way the typing ecosystem has
> developed. What I understood was that we'd get type inference like ML or
> Haskell use, so we wouldn't need to annotate *everything*, only the bits
> needed to resolve ambiguity. But what we seem to have got is typing like
> C, Pascal and Java, except gradual. Am I being unreasonable to be
> disappointed? I'm not a heavy mypy user, I just dabble with it
> occasionally, so maybe I've missed something.
>
> You might be more comfortable with pytype, which I understand is much more
inferential than pypy.

>
> > Anyway, we're *way* off topic now, and I doubt there's much that the
> > SC or python-dev can do to change the community view on typing,
> > anyway.
>
> Heh. We could update PEP 8 to ban type annotations, then watch as the
> people who over-zealously apply PEP 8 to everything AND over-zealously
> insist on adding type annotations to everything have their heads
> explode.
>
> Cruel, but funny.

Am 01.12.21 um 13:36 schrieb Paul Moore:
> On Wed, 1 Dec 2021 at 12:08, Sebastian Rittau <srittau@rittau.biz> wrote:
>
>> Please note that users of you library usually won't care that the library uses type hints. It's more important that there are type hints for the API, which can also be supplied using a stub file.
> I tried that route, but I was informed that if I do that, mypy will
> not check my stubs against the source code. Which means that there's
> no easy way of testing that the stubs are correct - is that accurate?

mypy includes a "stubtest" tool that compares a stub to introspected
runtime data. We use it in typeshed as part of the CI process and it's a
very powerful tool to help keeping stubs up-to-date and correct.


> PS I appreciate the links you posted to various typing forums, but IMO
> the most critical missing resource is a central set of typing
> documentation that includes examples, FAQs and best practices as well
> as reference materials.

Completely agree. This is what typing.readthedocs.io is supposed to
become, but time constraints mean that it's very incomplete at the moment.

> PPS Sorry if this sounds negative. TBH, I'd quite happily not use
> typing if I didn't want to and stay quiet. A lot of the frustration I
> see being expressed here (including my own) seems to come from the
> fact that it's so difficult to actually take that sort of "I can
> ignore it if I don't use it" attitude, whether that's because of
> community pressure, tool requirements, or whatever.

I completely understand this pressure, especially for library authors.
Providing high quality stubs and the best user experience is not easy.
But I believe that referring people to typeshed can help. While we of
course prefer high quality stubs or type annotations shipped with the
package in question, typeshed can provide a fairly low barrier of entry
for projects that don't have the resources to maintain type annotations
themselves. It can also be used as an "incubator", where stubs are
created and improved iteratively, until they are deemed ready for
inclusion in an upstream package.

 - Sebastian

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3Z7MCNNQUH3S5IWXA5MXVAUKPYYWKFZO/
Code of Conduct: http://python.org/psf/codeofconduct/

>for library authors.

> Providing high quality stubs and the best user experience is not easy.
> But I believe that referring people to typeshed can help.


This is actually very helpful. It provides an answer for open source
projects for which do users want typing.

One can say to the users that want type stubs for the library that they are
encouraged to contribute those stubs to type shed themselves and once they
have been tested and vetted they can be included in the project.

it’s not unlike any other feature request. The developer of an open source
project is under no obligation to provide any feature asked for, but a
well-managed project will encourage useful contributions from users.

-CHB
--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

I assume you accidentally pressed Send prematurely.
Still, maybe you have inadvertently listed everything that is agreed
about typing PEPs. ????
Rob Cliffe

On 02/12/2021 23:20, Christopher Barker wrote:
>
> >for library authors.
>
> Providing high quality stubs and the best user experience is not
> easy.
> But I believe that referring people to typeshed can help.
>
>
> This is actually very helpful. It provides an answer for open source
> projects for which do users want typing.
>
> One can say to the users that want type stubs for the library that
> they are encouraged to contribute those stubs to type shed themselves
> and once they have been tested and vetted they can be included in the
> project.
>
> it’s not unlike any other feature request. The developer of an open
> source project is under no obligation to provide any feature asked
> for, but a well-managed project will encourage useful contributions
> from users.
>
> -CHB
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
>
> _______________________________________________
> Python-Dev mailing list --python-dev@python.org
> To unsubscribe send an email topython-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived athttps://mail.python.org/archives/list/python-dev@python.org/message/ASO2PUTECTOB25HC7DURFOGOHOUQOCD5/
> Code of Conduct:http://python.org/psf/codeofconduct/

@Paul

> ... missing resource is a central set of typing documentation that
includes examples, FAQs and best practices as well as reference materials

Like Sebastian, I agree, and this is something we're making progress on.

> ... easy way of testing that the stubs are correct

mypy ships with a tool called stubtest. It's what typeshed uses to validate
itself against CPython. I recently wrote up documentation for it, see here:
https://mypy.readthedocs.io/en/latest/stubtest.html

> A lot of the frustration I see being expressed here (including my own)
seems to come from the fact that it's so difficult to actually take that
sort of "I can ignore it if I don't use it"

Thanks for not staying quiet and helping us make typing better. In
particular, I think one takeaway from this thread is that a lot of our
documentation is focussed on "how to get started with typing my code", and
not much addressed at library authors with limited time to deal with typing
questions (like "what is a PEP 561" and "if my type hints are incomplete
does that cause problems for my users")

On Thu, 2 Dec 2021 at 15:34, Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:

> I assume you accidentally pressed Send prematurely.
> Still, maybe you have inadvertently listed everything that is agreed about
> typing PEPs. ????
> Rob Cliffe
>
> On 02/12/2021 23:20, Christopher Barker wrote:
>
>
> >for library authors.
>
>> Providing high quality stubs and the best user experience is not easy.
>> But I believe that referring people to typeshed can help.
>
>
> This is actually very helpful. It provides an answer for open source
> projects for which do users want typing.
>
> One can say to the users that want type stubs for the library that they
> are encouraged to contribute those stubs to type shed themselves and once
> they have been tested and vetted they can be included in the project.
>
> it’s not unlike any other feature request. The developer of an open source
> project is under no obligation to provide any feature asked for, but a
> well-managed project will encourage useful contributions from users.
>
> -CHB
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.orghttps://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ASO2PUTECTOB25HC7DURFOGOHOUQOCD5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7AF22G3S23S67FAHH7BI4XUNHWOU4YXT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

resent -- damn iPhone!

On Thu, Dec 2, 2021 at 3:20 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> >for library authors.
> >
> > Providing high quality stubs and the best user experience is not easy.
> > But I believe that referring people to typeshed can help.
>
>
> This is actually very helpful. It provides an answer for open source
> projects for which do users want typing.
>
> One can say to the users that want type stubs for the library that they
> are encouraged to contribute those stubs to type shed themselves and once
> they have been tested and vetted they can be included in the project.
>
> it’s not unlike any other feature request. The developer of an open source
> project is under no obligation to provide any feature asked for, but a
> well-managed project will encourage useful contributions from users.
>
> -CHB



> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Dec 2, 2021 at 3:35 PM Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:

> I assume you accidentally pressed Send prematurely.
>

Actually, it was my phone making the text white -- what the heck?

why is it so hard to send plain text email?

resent now.

-CHB



> Still, maybe you have inadvertently listed everything that is agreed about
> typing PEPs. ????
> Rob Cliffe
>
> On 02/12/2021 23:20, Christopher Barker wrote:
>
>
> >for library authors.
>
>> Providing high quality stubs and the best user experience is not easy.
>> But I believe that referring people to typeshed can help.
>
>
> This is actually very helpful. It provides an answer for open source
> projects for which do users want typing.
>
> One can say to the users that want type stubs for the library that they
> are encouraged to contribute those stubs to type shed themselves and once
> they have been tested and vetted they can be included in the project.
>
> it’s not unlike any other feature request. The developer of an open source
> project is under no obligation to provide any feature asked for, but a
> well-managed project will encourage useful contributions from users.
>
> -CHB
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.orghttps://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ASO2PUTECTOB25HC7DURFOGOHOUQOCD5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7AF22G3S23S67FAHH7BI4XUNHWOU4YXT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

So maybe this is my time to chime in. I have used annotations for runtime
behavior. My primary use case is an injection library that I wrote. It allows
something along the lines of:

class IMyService(IService):
pass

@inject
def call_something(arg1: str, arg2: int, svc: IMyService = None):
assert svc is not None
# do stuff...

At runtime the `inject()` decorator will scan the function signature and
insert an instance of `IMyService` based on a registry lookup. (We use
zope.component utilities, but that's an implementation detail.) When testing,
one can simply pass a mock version of the service.

Using the mypy-zope plugin, the above also passes all mypy type checking.

It is a simple pattern in terms of annotation but a very effective pattern
that mimics other injection systems in Java and TypeScript (Angular). We have
used it for a few years now and like it a lot.

Regards,
Stephan

On Monday, November 29, 2021 6:00:04 PM EST Barry Warsaw wrote:
> On Nov 26, 2021, at 01:13, Paul Moore <p.f.moore@gmail.com> wrote:
> > I'd therefore interpret Barry's plea as being for *anyone* with a use
> > for annotations to provide their feedback (at least, anyone who
> > accepts that annotations are types), with particular emphasis on
> > people who want to use the types declared in annotations to affect
> > runtime behaviour, as that's the most under-represented group at the
> > moment (and it's not clear whether it's under-represented because
> > there aren't many such uses, or because the users aren't being heard
> > from).
>
> Spot on.
>
> -Barry


--
Stephan Richter
Entrepreneur & Geek


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XRJJ4ZMOQJXOTRFT3FQ25QXHXOKBVXKP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Dec 2, 2021 at 6:57 PM Stephan Richter <stephan.richter@gmail.com>
wrote:

> So maybe this is my time to chime in. I have used annotations for runtime
> behavior. My primary use case is an injection library that I wrote.


Does it work with __future__ annotations?

-CHB





> It allows
> something along the lines of:
>
> class IMyService(IService):
> pass
>
> @inject
> def call_something(arg1: str, arg2: int, svc: IMyService = None):
> assert svc is not None
> # do stuff...
>
> At runtime the `inject()` decorator will scan the function signature and
> insert an instance of `IMyService` based on a registry lookup. (We use
> zope.component utilities, but that's an implementation detail.) When
> testing,
> one can simply pass a mock version of the service.
>
> Using the mypy-zope plugin, the above also passes all mypy type checking.
>
> It is a simple pattern in terms of annotation but a very effective pattern
> that mimics other injection systems in Java and TypeScript (Angular). We
> have
> used it for a few years now and like it a lot.
>
> Regards,
> Stephan
>
> On Monday, November 29, 2021 6:00:04 PM EST Barry Warsaw wrote:
> > On Nov 26, 2021, at 01:13, Paul Moore <p.f.moore@gmail.com> wrote:
> > > I'd therefore interpret Barry's plea as being for *anyone* with a use
> > > for annotations to provide their feedback (at least, anyone who
> > > accepts that annotations are types), with particular emphasis on
> > > people who want to use the types declared in annotations to affect
> > > runtime behaviour, as that's the most under-represented group at the
> > > moment (and it's not clear whether it's under-represented because
> > > there aren't many such uses, or because the users aren't being heard
> > > from).
> >
> > Spot on.
> >
> > -Barry
>
>
> --
> Stephan Richter
> Entrepreneur & Geek
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XRJJ4ZMOQJXOTRFT3FQ25QXHXOKBVXKP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Fri, 3 Dec 2021 at 00:10, Shantanu Jain <hauntsaninja@gmail.com> wrote:
>
> @Paul
>
> > ... missing resource is a central set of typing documentation that includes examples, FAQs and best practices as well as reference materials
>
> Like Sebastian, I agree, and this is something we're making progress on.

That's great to know - it's easy for me to say "we need more docs" but
I know it's hard to produce them. I'm glad it's on the radar :-)

> > ... easy way of testing that the stubs are correct
>
> mypy ships with a tool called stubtest. It's what typeshed uses to validate itself against CPython. I recently wrote up documentation for it, see here: https://mypy.readthedocs.io/en/latest/stubtest.html

Thanks, I wasn't aware of that. Although TBH I like the suggestion of
using typeshed as a kind of staging point for interested users to
develop annotations, which can then be moved back into the library
once they've been proven stable.

I'm still surprised that mypy doesn't check stub files though. If I
write stubs for my external API, and then later decide I want to add
types to my internal functions, to do static checking on my library,
am I therefore required to move the annotations out of the stub file
into the main code? What if I want to avoid the runtime overhead of
importing the typing module? Or I want to keep the source code clear,
by not making function definitions into multi-line monsters because of
type declarations?

> > A lot of the frustration I see being expressed here (including my own) seems to come from the fact that it's so difficult to actually take that sort of "I can ignore it if I don't use it"
>
> Thanks for not staying quiet and helping us make typing better. In particular, I think one takeaway from this thread is that a lot of our documentation is focussed on "how to get started with typing my code", and not much addressed at library authors with limited time to deal with typing questions (like "what is a PEP 561" and "if my type hints are incomplete does that cause problems for my users")

That's a good point that I hadn't spotted - thanks for picking up on it!
Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CRBIOVH4K5VFIIPB2D6EJCIVWJXGITEU/
Code of Conduct: http://python.org/psf/codeofconduct/