Mailing List Archive

On Thu, Jun 03, 2021 at 10:10:53AM -0700, Guido van Rossum wrote:
> This is not a complete thought yet, but it occurred to me that while we have
> deprecated APIs (which will eventually go away), and provisional APIs (which
> must mature a little before they're declared stable), and stable APIs (which
> everyone can rely on), it might be good to also have something like *unstable*
> APIs, which will continually change without ever going away or stabilizing.

The first grey area will between Provisional API vs Unstable API.

Do developers consider provisional APIs as stable and start relying upon
heavily? I am not sure.

I also lack the experience for the use-cases that you are thinking
about.

--
Senthil
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XOMDLGMXWFRBETG7FWI5PB2Y5YNGNXP3/
Code of Conduct: http://python.org/psf/codeofconduct/

In practice, provisional APIs have been quite stable. The term
"provisional" was introduced for PEPs that introduce new modules, where we
wanted to allow some wiggle room for changes based on experience with using
the new module during the first release cycle where it's made available.
You can think of it as a sort of extended beta period for that module only.
Generally provisional status only lasts for one release cycle.

"Unstable" has a different meaning -- it's for APIs (including modules)
that are likely to change in every release (or most releases, anyway).
Users are not discouraged from using these, but they *must* be mindful of
their code breaking with every new release.

I could imagine some unstability to allow incompatible changes in bugfix
releases, though for my main use case it would be sufficient to only allow
those in minor releases.

On Thu, Jun 3, 2021 at 10:32 AM Senthil Kumaran <senthil@python.org> wrote:

> On Thu, Jun 03, 2021 at 10:10:53AM -0700, Guido van Rossum wrote:
> > This is not a complete thought yet, but it occurred to me that while we
> have
> > deprecated APIs (which will eventually go away), and provisional APIs
> (which
> > must mature a little before they're declared stable), and stable APIs
> (which
> > everyone can rely on), it might be good to also have something like
> *unstable*
> > APIs, which will continually change without ever going away or
> stabilizing.
>
> The first grey area will between Provisional API vs Unstable API.
>
> Do developers consider provisional APIs as stable and start relying upon
> heavily? I am not sure.
>
> I also lack the experience for the use-cases that you are thinking
> about.
>
> --
> Senthil
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

Overall agreement. Your list of ast and code objects and bytecode
instructions are things that I'd _hope_ people already consider unstable so
declaring them as such just makes sense if we're not doing that already.

The ideal way to declare an API as unstable is to constantly change it in a
breaking manner. With every release and potentially even within some patch
releases when the point really needs to be made. Even when you didn't have
a reason to change anything. If you don't do that, people are going to
find it convenient, discover stability, assume it exists, depend on it, and
complain about breakage no matter what was stated.
https://www.hyrumslaw.com/

One obvious now in hindsight question: Why are any of these APIs even
public? They all deserve underscore prefixed names to highlight their
private-ness and potential instability.

-gps

On Thu, Jun 3, 2021 at 10:46 AM Guido van Rossum <guido@python.org> wrote:

> In practice, provisional APIs have been quite stable. The term
> "provisional" was introduced for PEPs that introduce new modules, where we
> wanted to allow some wiggle room for changes based on experience with using
> the new module during the first release cycle where it's made available.
> You can think of it as a sort of extended beta period for that module only.
> Generally provisional status only lasts for one release cycle.
>
> "Unstable" has a different meaning -- it's for APIs (including modules)
> that are likely to change in every release (or most releases, anyway).
> Users are not discouraged from using these, but they *must* be mindful of
> their code breaking with every new release.
>
> I could imagine some unstability to allow incompatible changes in bugfix
> releases, though for my main use case it would be sufficient to only allow
> those in minor releases.
>
> On Thu, Jun 3, 2021 at 10:32 AM Senthil Kumaran <senthil@python.org>
> wrote:
>
>> On Thu, Jun 03, 2021 at 10:10:53AM -0700, Guido van Rossum wrote:
>> > This is not a complete thought yet, but it occurred to me that while we
>> have
>> > deprecated APIs (which will eventually go away), and provisional APIs
>> (which
>> > must mature a little before they're declared stable), and stable APIs
>> (which
>> > everyone can rely on), it might be good to also have something like
>> *unstable*
>> > APIs, which will continually change without ever going away or
>> stabilizing.
>>
>> The first grey area will between Provisional API vs Unstable API.
>>
>> Do developers consider provisional APIs as stable and start relying upon
>> heavily? I am not sure.
>>
>> I also lack the experience for the use-cases that you are thinking
>> about.
>>
>> --
>> Senthil
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZUTAVE3S3QMWNIBGTBDOTJ7M62CTO57R/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Thu, Jun 3, 2021 at 11:11 Gregory P. Smith <greg@krypto.org> wrote:

> One obvious now in hindsight question: Why are any of these APIs even
> public? They all deserve underscore prefixed names to highlight their
> private-ness and potential instability.
>

Because they are still useful when manipulating or introspecting Python
code. I think stable and public are orthogonal axes.
--
--Guido (mobile)

I think it makes sense, and I do see a difference between Provisional and Unstable. Is this anything more than a documentation label?

-Barry

> On Jun 3, 2021, at 13:10, Guido van Rossum <guido@python.org> wrote:
>
> This is not a complete thought yet, but it occurred to me that while we have deprecated APIs (which will eventually go away), and provisional APIs (which must mature a little before they're declared stable), and stable APIs (which everyone can rely on), it might be good to also have something like *unstable* APIs, which will continually change without ever going away or stabilizing. Examples would be the ast module (since the AST structure changes every time the grammar changes) and anything to do with code objects and bytecode (since we sometimes think of better ways to execute Python).
>
> So maybe the docs should grow a standard way of saying "this is an unstable API"?
>
> Would we need a PEP to create an initial list of APIs (modules, classes, etc.) that are considered unstable?
>
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JM6SQ2YNMDAKXYD5O54QWMVR2X7QOXVL/
> Code of Conduct: http://python.org/psf/codeofconduct/

Hi Guido,

It seems like you are talking about the Python API.

In the C API, there is the internal C API which fits with your
description. To access it, you have to declare the
Py_BUILD_CORE_MODULE macro. It's not usable directly on purpose. It's
an user agreement: I know what I am doing, and I know that this API is
not supported nor stable.

I don't know if there would be a way to advertise that a Python API is
unstable. Some projects use an underscore prefix in their module names
to mark them as "private". For example, sub-modules of a package are
called "_something.py" and they exposed in package/__init__.py (or
another public module).

Victor

On Thu, Jun 3, 2021 at 7:14 PM Guido van Rossum <guido@python.org> wrote:
>
> This is not a complete thought yet, but it occurred to me that while we have deprecated APIs (which will eventually go away), and provisional APIs (which must mature a little before they're declared stable), and stable APIs (which everyone can rely on), it might be good to also have something like *unstable* APIs, which will continually change without ever going away or stabilizing. Examples would be the ast module (since the AST structure changes every time the grammar changes) and anything to do with code objects and bytecode (since we sometimes think of better ways to execute Python).
>
> So maybe the docs should grow a standard way of saying "this is an unstable API"?
>
> Would we need a PEP to create an initial list of APIs (modules, classes, etc.) that are considered unstable?
>
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JM6SQ2YNMDAKXYD5O54QWMVR2X7QOXVL/
> Code of Conduct: http://python.org/psf/codeofconduct/



--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KG4Z45UW7Y2EY5WMPZ6Z4763CJWUTXH3/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Jun 4, 2021 at 6:44 AM Barry Warsaw <barry@python.org> wrote:
>
> I think it makes sense, and I do see a difference between Provisional and Unstable. Is this anything more than a documentation label?
>

Would it be a pipe dream to hope that static checkers could be taught
to recognize them? Not a huge deal, but it would mean you could ask
something to analyze your code (I hesitate to call it a type checker,
since this is nothing to do with data types, but the same kind of
tool) and it'd tell you whether your code is (a) portable to all OSes,
(b) portable to all Pythons, and (c) stable across versions.

BTW, does "unstable" cover things like dis.dis(), which have existed
and will continue to exist for many versions, but their output can
change? In one sense, dis.dis() always does the exact same thing: it
shows you the disassembly of a piece of code. In another sense, its
output changes drastically when things change.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MRNDNHVLQWX5HC3LETQVUT4XX2TKYKVW/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, 4 Jun 2021 at 03:13, Guido van Rossum <guido@python.org> wrote:

> This is not a complete thought yet, but it occurred to me that while we
> have deprecated APIs (which will eventually go away), and provisional APIs
> (which must mature a little before they're declared stable), and stable
> APIs (which everyone can rely on), it might be good to also have something
> like *unstable* APIs, which will continually change without ever going away
> or stabilizing. Examples would be the ast module (since the AST structure
> changes every time the grammar changes) and anything to do with code
> objects and bytecode (since we sometimes think of better ways to execute
> Python).
>

Perhaps "living API" analogous to "living document". Much more positive
connotations ...

Tim Delaney

On Thu, Jun 3, 2021 at 1:33 PM Victor Stinner <vstinner@python.org> wrote:

> Hi Guido,
>
> It seems like you are talking about the Python API.
>

Primarily, yes.

In the C API, there is the internal C API which fits with your
> description. To access it, you have to declare the
> Py_BUILD_CORE_MODULE macro. It's not usable directly on purpose. It's
> an user agreement: I know what I am doing, and I know that this API is
> not supported nor stable.
>

Hm, but aren't for example all the fields of code objects (co_name,
co_argcount, etc.) in the "non-internal" API? Those (and the functions that
manipulate code objects) are a prime example of what I'd consider
"unstable".

On https://docs.python.org/3/c-api/code.html it already says about the
fields "The fields of this type are subject to change at any time." But I
think everything else on that page should be considered unstable as well.
(And why do we even have PyCode_GetNumFree()?)


> I don't know if there would be a way to advertise that a Python API is
> unstable. Some projects use an underscore prefix in their module names
> to mark them as "private". For example, sub-modules of a package are
> called "_something.py" and they exposed in package/__init__.py (or
> another public module).
>

I was primarily thinking of the docs.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On Thu, Jun 3, 2021 at 2:01 PM Chris Angelico <rosuav@gmail.com> wrote:

> On Fri, Jun 4, 2021 at 6:44 AM Barry Warsaw <barry@python.org> wrote:
> >
> > I think it makes sense, and I do see a difference between Provisional
> and Unstable. Is this anything more than a documentation label?
> >
>
> Would it be a pipe dream to hope that static checkers could be taught
> to recognize them? Not a huge deal, but it would mean you could ask
> something to analyze your code (I hesitate to call it a type checker,
> since this is nothing to do with data types, but the same kind of
> tool) and it'd tell you whether your code is (a) portable to all OSes,
> (b) portable to all Pythons, and (c) stable across versions.
>

Yeah, this could easily be taken on by any of the many linters.


> BTW, does "unstable" cover things like dis.dis(), which have existed
> and will continue to exist for many versions, but their output can
> change? In one sense, dis.dis() always does the exact same thing: it
> shows you the disassembly of a piece of code. In another sense, its
> output changes drastically when things change.
>

That's debatable. I sure hope people aren't ever parsing dis output.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On Thu, Jun 3, 2021 at 3:17 PM Tim Delaney <timothy.c.delaney@gmail.com>
wrote:

> Perhaps "living API" analogous to "living document". Much more positive
> connotations ...
>

Perhaps, but that's pretty much coining a new term, which we would then
have to explain. And since the opposite would be "dead API", what is a dead
API exactly? And doesn't it follow that all APIs are living? If we stick
with unstable, we're adopting a term that's in use by at least one other
language community (Rust), and the slightly negative connotation is welcome
-- people should think twice before using unstable APIs.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On 6/3/2021 3:34 PM, Guido van Rossum wrote:
> On Thu, Jun 3, 2021 at 3:17 PM Tim Delaney
> <timothy.c.delaney@gmail.com <mailto:timothy.c.delaney@gmail.com>> wrote:
>
> Perhaps "living API" analogous to "living document". Much more
> positive connotations ...
>
>
> Perhaps, but that's pretty much coining a new term, which we would
> then have to explain. And since the opposite would be "dead API", what
> is a dead API exactly? And doesn't it follow that all APIs are living?
> If we stick with unstable, we're adopting a term that's in use by at
> least one other language community (Rust), and the slightly negative
> connotation is welcome -- people should think twice before using
> unstable APIs.

I read somewhere that the term "stable" means "dead" in some contexts...
was it maybe a medical context?  So "living" would be "unstable" too, as
Tim suggested.

And since people know what a living document is, a living API wouldn't
be much of a stretch.

On the other hand, "unstable" carries a bit more connotation of "needs
caution".

On Fri, Jun 4, 2021 at 8:31 AM Guido van Rossum <guido@python.org> wrote:
>
> On Thu, Jun 3, 2021 at 2:01 PM Chris Angelico <rosuav@gmail.com> wrote:
>>
>> On Fri, Jun 4, 2021 at 6:44 AM Barry Warsaw <barry@python.org> wrote:
>> >
>> > I think it makes sense, and I do see a difference between Provisional and Unstable. Is this anything more than a documentation label?
>> >
>>
>> Would it be a pipe dream to hope that static checkers could be taught
>> to recognize them? Not a huge deal, but it would mean you could ask
>> something to analyze your code (I hesitate to call it a type checker,
>> since this is nothing to do with data types, but the same kind of
>> tool) and it'd tell you whether your code is (a) portable to all OSes,
>> (b) portable to all Pythons, and (c) stable across versions.
>
>
> Yeah, this could easily be taken on by any of the many linters.

Cool cool.

>> BTW, does "unstable" cover things like dis.dis(), which have existed
>> and will continue to exist for many versions, but their output can
>> change? In one sense, dis.dis() always does the exact same thing: it
>> shows you the disassembly of a piece of code. In another sense, its
>> output changes drastically when things change.
>
>
> That's debatable. I sure hope people aren't ever parsing dis output.
>

That's exactly what I mean. People shouldn't be parsing that output,
because it's human-readable. Does it count as an API change when the
human-readable output is giving different information?

Actually, I think I just talked myself out of this. The
sys.version_info tuple is going to change from one version to another
(obviously!), but it is, by definition, stable and dependable. So I
think no, it's not "unstable" based on human-readable output.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HDTPVRMBX7L5NPWFSDOHTO7UTISOUC6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

Maybe ‘version-dependent’ api? It indicates why the api is unstable (as opposed to something like test.support where the docs say ‘this is for us, we’re not bothered about keeping it stable’).

In some contexts ‘unstable’ means buggy/unreliable.

> On 4 Jun 2021, at 00:20, Glenn Linderman <v+python@g.nevcal.com> wrote:
>
> ?
> On 6/3/2021 3:34 PM, Guido van Rossum wrote:
>> On Thu, Jun 3, 2021 at 3:17 PM Tim Delaney <timothy.c.delaney@gmail.com> wrote:
>>> Perhaps "living API" analogous to "living document". Much more positive connotations ...
>>
>> Perhaps, but that's pretty much coining a new term, which we would then have to explain. And since the opposite would be "dead API", what is a dead API exactly? And doesn't it follow that all APIs are living? If we stick with unstable, we're adopting a term that's in use by at least one other language community (Rust), and the slightly negative connotation is welcome -- people should think twice before using unstable APIs.
>
> I read somewhere that the term "stable" means "dead" in some contexts... was it maybe a medical context? So "living" would be "unstable" too, as Tim suggested.
>
> And since people know what a living document is, a living API wouldn't be much of a stretch.
>
> On the other hand, "unstable" carries a bit more connotation of "needs caution".
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CS23XQX3356Q3S6WI66PISIT3NXEGMGM/
> Code of Conduct: http://python.org/psf/codeofconduct/

03.06.21 20:10, Guido van Rossum ????:
> This is not a complete thought yet, but it occurred to me that while we
> have deprecated APIs (which will eventually go away), and provisional
> APIs (which must mature a little before they're declared stable), and
> stable APIs (which everyone can rely on), it might be good to also have
> something like *unstable* APIs, which will continually change without
> ever going away or stabilizing. Examples would be the ast module (since
> the AST structure changes every time the grammar changes) and anything
> to do with code objects and bytecode (since we sometimes think of better
> ways to execute Python).
>
> So maybe the docs should grow a standard way of saying "this is an
> unstable API"?

There is already a way to specify the stable ABI (see
Doc/tools/extensions/c_annotations.py). But unfortunately this feature
is not is not used in the documentation. It needs just an amount of work
to do this, and nobody did this.

After marking all stable ABI we can extend this feature to support
halftones: provisional API, unstable API for Cython, etc.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KAOTXYRGZMTNB4TYJBQTQAEEG2PJZBDI/
Code of Conduct: http://python.org/psf/codeofconduct/

On 04. 06. 21 10:25, Serhiy Storchaka wrote:
> 03.06.21 20:10, Guido van Rossum ????:
>> This is not a complete thought yet, but it occurred to me that while we
>> have deprecated APIs (which will eventually go away), and provisional
>> APIs (which must mature a little before they're declared stable), and
>> stable APIs (which everyone can rely on), it might be good to also have
>> something like *unstable* APIs, which will continually change without
>> ever going away or stabilizing. Examples would be the ast module (since
>> the AST structure changes every time the grammar changes) and anything
>> to do with code objects and bytecode (since we sometimes think of better
>> ways to execute Python).
>>
>> So maybe the docs should grow a standard way of saying "this is an
>> unstable API"?
>
> There is already a way to specify the stable ABI (see
> Doc/tools/extensions/c_annotations.py). But unfortunately this feature
> is not is not used in the documentation. It needs just an amount of work
> to do this, and nobody did this.

It is used, and I started the work :)
See e.g.
https://docs.python.org/3.10/c-api/sequence.html#c.PySequence_Concat


> After marking all stable ABI we can extend this feature to support
> halftones: provisional API, unstable API for Cython, etc.

I don't think that's necessary for the C API; the three-tier structure
we have now (see https://devguide.python.org/c-api/ ) is, IMO, sufficient.

I don't think it can be easily adapted for the Python API, though.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4QINROTO26H5USYKUSQLRY4XTZOSV2FS/
Code of Conduct: http://python.org/psf/codeofconduct/

Am 03.06.21 um 20:11 schrieb Gregory P. Smith:
> The ideal way to declare an API as unstable is to constantly change it
> in a breaking manner.  With every release and potentially even within
> some patch releases when the point really needs to be made.  Even when
> you didn't have a reason to change anything.  If you don't do that,
> people are going to find it convenient, discover stability, assume it
> exists, depend on it, and complain about breakage no matter what was
> stated. https://www.hyrumslaw.com/ <https://www.hyrumslaw.com/>

There is certainly value in having some stability guarantees, even in
"unstable" APIs. For example, while it is expected that the ast module
breaks with each new minor Python version (3.7, 3.8 etc.), it's still
stable during each such version. This makes the ast module quite useful
for lots of applications that work on Python source code and that are
understood to need changes for each Python version anyway. Breaking the
compatibility just for the sake of it would be counterproductive.

 - Sebastian

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7BBKPPUD7BX6THFFBWZXCUK3TBIFNNQS/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Jun 4, 2021 at 12:29 AM Guido van Rossum <guido@python.org> wrote:
>> In the C API, there is the internal C API which fits with your
>> description. To access it, you have to declare the
>> Py_BUILD_CORE_MODULE macro. It's not usable directly on purpose. It's
>> an user agreement: I know what I am doing, and I know that this API is
>> not supported nor stable.
>
> Hm, but aren't for example all the fields of code objects (co_name, co_argcount, etc.) in the "non-internal" API? Those (and the functions that manipulate code objects) are a prime example of what I'd consider "unstable".
>
> On https://docs.python.org/3/c-api/code.html it already says about the fields "The fields of this type are subject to change at any time." But I think everything else on that page should be considered unstable as well. (And why do we even have PyCode_GetNumFree()?)

Hum, the C API is somehow off-topic, but let me reply anyway ;-)

As I explained in my PEP 620, the Python C API never had any design.
Things were only exposed because it was easy and technically possible,
and it was a convenient way to define a function in one file and uses
it from another file. But 30 years later, we identified that exposing
some things are causing troubles and we are trying to make the C API
more "abstract".

Exposing directly all structures is causing a lot of headaches at
every new Python 3.x release. Getter and setter functions can handle
structure changes, retrieve information from another structure, return
an error, etc. This abstractation is needed for users to not have to
update their code, and to CPython developers to be able to change
things.

If possible, I would prefer to make PyThreadState, PyCodeObject and
other structures opaque, and only go through getter and setter
functions ;-) PyCode_New() is another problem :-/ The PEP 570 first
changed it to add a new parameter. It broke Cython and other projects.
The change was reverted, and PyCode_NewWithPosOnlyArgs() was added.
The lesson is that it's possible to change PyCodeObject without
breaking PyCode_New() (which handles the "backward compatibility" for
you).

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GQI2NKFXQ4LYJ2HVFBCHL3KPMQVD4HI4/
Code of Conduct: http://python.org/psf/codeofconduct/

04.06.21 12:08, Petr Viktorin ????:
> It is used, and I started the work :)
> See e.g.
> https://docs.python.org/3.10/c-api/sequence.html#c.PySequence_Concat

Great news! Several core developers (including me) tried to solve this
problem, but a large amount of work stopped us at an early stage. Glad
there is a progress.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/44XXQIFBIUNTYTDH7XSN3XKBZ6SLYM5K/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Jun 4, 2021 at 6:15 AM Victor Stinner <vstinner@python.org> wrote:

> On Fri, Jun 4, 2021 at 12:29 AM Guido van Rossum <guido@python.org> wrote:
> >> In the C API, there is the internal C API which fits with your
> >> description. To access it, you have to declare the
> >> Py_BUILD_CORE_MODULE macro. It's not usable directly on purpose. It's
> >> an user agreement: I know what I am doing, and I know that this API is
> >> not supported nor stable.
> >
> > Hm, but aren't for example all the fields of code objects (co_name,
> co_argcount, etc.) in the "non-internal" API? Those (and the functions that
> manipulate code objects) are a prime example of what I'd consider
> "unstable".
> >
> > On https://docs.python.org/3/c-api/code.html it already says about the
> fields "The fields of this type are subject to change at any time." But I
> think everything else on that page should be considered unstable as well.
> (And why do we even have PyCode_GetNumFree()?)
>
> Hum, the C API is somehow off-topic, but let me reply anyway ;-)
>

I see the smiley but I still don't understand why you're saying this.

As I explained in my PEP 620, the Python C API never had any design.
>

That's not entirely true. *Parts* of it were definitely designed, e.g. the
existing "abstract" API (PyObject_GetItem, _GetAttr, and many more; IIRC
Zope's Jim Fulton pushed for these). Other parts are indeed not designed
much, all the PyList, PyDict, PyTuple etc. APIs (which came before the
"abstract" ones), and the many APIs that followed their example (to this
day).


> Things were only exposed because it was easy and technically possible,
> and it was a convenient way to define a function in one file and uses
> it from another file.


Historically, it was even worse -- people started embedding Python before
we even had "Py" prefixes. Try to find out about the "great renaming" (I
think it was the late '90s).


> But 30 years later, we identified that exposing
> some things are causing troubles and we are trying to make the C API
> more "abstract".
>

This time the abstraction isn't just about supporting different object
types, it's about allowing us to change other aspects of the interpreter,
in particular object layout ("object model" as it's somehow known) and
memory management.

Who knows what we'll want to change 20 years from now...


> Exposing directly all structures is causing a lot of headaches at
> every new Python 3.x release. Getter and setter functions can handle
> structure changes, retrieve information from another structure, return
> an error, etc. This abstractation is needed for users to not have to
> update their code, and to CPython developers to be able to change
> things.
>
> If possible, I would prefer to make PyThreadState, PyCodeObject and
> other structures opaque, and only go through getter and setter
> functions ;-) PyCode_New() is another problem :-/ The PEP 570 first
> changed it to add a new parameter. It broke Cython and other projects.
> The change was reverted, and PyCode_NewWithPosOnlyArgs() was added.
> The lesson is that it's possible to change PyCodeObject without
> breaking PyCode_New() (which handles the "backward compatibility" for
> you).
>

I'm afraid that won't always be possible though. At some point there just
may not be a valid meaning for the original PyCode_New() call. It was easy
in the case of positional arguments (by default don't have any) but it may
not always be that simple, and we shouldn't make guarantees here.

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

On Sat, Jun 5, 2021 at 2:12 AM Serhiy Storchaka <storchaka@gmail.com> wrote:

> 04.06.21 12:08, Petr Viktorin ????:
> > It is used, and I started the work :)
> > See e.g.
> > https://docs.python.org/3.10/c-api/sequence.html#c.PySequence_Concat
>
> Great news! Several core developers (including me) tried to solve this
> problem, but a large amount of work stopped us at an early stage. Glad
> there is a progress.
>

Yes, this is indeed great news. It also brings the existence of these
distinctions (between stable/public/cpython APIs) to the attention of their
users, which helps building mindshare for limiting API use to the stable
API. Thanks Petr for pushing this!

--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>

06.06.21 06:48, Guido van Rossum ????:
> On Fri, Jun 4, 2021 at 6:15 AM Victor Stinner <vstinner@python.org
> <mailto:vstinner@python.org>> wrote:
> If possible, I would prefer to make PyThreadState, PyCodeObject and
> other structures opaque, and only go through getter and setter
> functions ;-) PyCode_New() is another problem :-/ The PEP 570 first
> changed it to add a new parameter. It broke Cython and other projects.
> The change was reverted, and PyCode_NewWithPosOnlyArgs() was added.
> The lesson is that it's possible to change PyCodeObject without
> breaking PyCode_New() (which handles the "backward compatibility" for
> you).
>
>
> I'm afraid that won't always be possible though. At some point there
> just may not be a valid meaning for the original PyCode_New() call. It
> was easy in the case of positional arguments (by default don't have any)
> but it may not always be that simple, and we shouldn't make guarantees here.

We have already reached this limit. In 3.11 the code object needs a
table of exception handlers. Only simplest code which do not contain any
"try" or "with" can now be created with old PyCode_New() and
PyCode_NewWithPosOnlyArgs().

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/G5Y5DD4T3ZSJ4DYWFZH4TFVNHJHBN5K2/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, 4 Jun 2021, 4:17 am Gregory P. Smith, <greg@krypto.org> wrote:

> Overall agreement. Your list of ast and code objects and bytecode
> instructions are things that I'd _hope_ people already consider unstable so
> declaring them as such just makes sense if we're not doing that already.
>

Another example of a public API that explicitly declares itself unstable is
"ssl.get_default_context()" (along with any TLS-enabled API that uses it).
It was made that way so the default TLS settings could evolve with the
times, even on maintenance branches. The secrets module has a similar
caveat on its default token lengths (i.e. making them longer is considered
an acceptable API change - if consuming code can't handle that for some
reason, it should set an explicit length).

The metaprogramming APIs for customisation of class creation also arguably
qualify - we've previously imposed new obligations on metaclass developers
as a consequence of adding features like zero-arg super() and the
descriptor naming hooks.

It occurs to me that PEP 387 (the backwards compatibility policy) should
probably mention that these formally unstable APIs exist, and link to a
page in the standard library docs that:

* references the formally unstable APIs
* specifies the points where potentially incompatible changes are allowed
(normally new feature releases, but the default SSL/TLS context definition
may change in maintenance releases if necessary)

Cheers,
Nick.



>
>