Mailing List Archive: Re: [External] : Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

Hi,

(I briefly commented also on the doc regarding this)

I?m probably misinterpreting the exact goals. I read ?stable ABI for everyone? and I?m thinking ?what needs to happen to stay binary compatible and working for a couple of decades at least?. If that were the goal, I think the ideas around HPy?s handles and it?s usage of a context to get access to function pointers are most important, since that will ensure that new APIs can be added and old ones removed without breaking ABI. (This is very similar to how e.g. libraries like SDL deal with the problem that distributors want to update the SDL library and have it still work with 10 year old proprietary games).

In particular, even things like slots in types need to be opaque. An optimizing runtime may want to use varying layouts for both types and objects - allowing direct access into any runtime structures prevents that. Exposed structures should not be used for any runtime objects, only as specs for construction of those runtime objects.

Best,
Tim

From: Victor Stinner<mailto:vstinner@python.org>
Sent: Tuesday, April 5, 2022 10:54 PM
To: Petr Viktorin<mailto:encukou@gmail.com>
Cc: Python Dev<mailto:python-dev@python.org>
Subject: [External] : [Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

IMO it would be better to keep the HPy design as the long term goal:

* Refer to Python objects with opaque handles
* All structures are opaque (with a few exceptions, like PyType_Spec)

It will likely take multiple iterations (Python releases) to reach
this goal, and incompatible C API changes may need a PEP (like PEP
674), but IMO it's good to keep this goal in mind.

Otherwise, it's not easy to understand the rationale for changes like
https://urldefense.com/v3/__https://peps.python.org/pep-0674/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRmzmIsF1a$<https://urldefense.com/v3/__https:/peps.python.org/pep-0674/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRmzmIsF1a$> "PEP 674 ? Disallow using macros as
l-values".

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://urldefense.com/v3/__https://mail.python.org/mailman3/lists/python-dev.python.org/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm7Y1wy1v$<https://urldefense.com/v3/__https:/mail.python.org/mailman3/lists/python-dev.python.org/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm7Y1wy1v$>
Message archived at https://urldefense.com/v3/__https://mail.python.org/archives/list/python-dev@python.org/message/3FYHB74CF6XBADFRLUVFV6NUZKXRSBSY/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm9YsYkeW$<https://urldefense.com/v3/__https:/mail.python.org/archives/list/python-dev@python.org/message/3FYHB74CF6XBADFRLUVFV6NUZKXRSBSY/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm9YsYkeW$>
Code of Conduct: https://urldefense.com/v3/__http://python.org/psf/codeofconduct/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm9jKqd_2$<https://urldefense.com/v3/__http:/python.org/psf/codeofconduct/__;!!ACWV5N9M2RV99hQ!eClvPBBrluADcw7qiB1sfyVPEAyTXoolnPkf_c9MLV-1Ns5roXVSrOKLBqaRm9jKqd_2$>

On 06. 04. 22 9:23, Tim Felgentreff wrote:
> Hi,
>
> (I briefly commented also on the doc regarding this)
>
> I’m probably misinterpreting the exact goals. I read “stable ABI for
> everyone” and I’m thinking “what needs to happen to stay binary
> compatible and working for a couple of decades at least”.

Well, the doc is more down to the ground than this thread can be :)

> If that were
> the goal, I think the ideas around HPy’s handles and it’s usage of a
> context to get access to function pointers are most important, since
> that will ensure that new APIs can be added and old ones removed without
> breaking ABI. (This is very similar to how e.g. libraries like SDL deal
> with the problem that distributors want to update the SDL library and
> have it still work with 10 year old proprietary games).
>
> In particular, even things like slots in types need to be opaque. An
> optimizing runtime may want to use varying layouts for both types and
> objects - allowing direct access into any runtime structures prevents
> that. Exposed structures should not be used for any runtime objects,
> only as specs for construction of those runtime objects.

With Python's current general API backwards compatibility policy (PEP
387), it wouldn't make much sense to make ABI guarantees that strong. In
reality you're likely to get DeprecationWarnings and runtime exceptions,
and will need to recompile extensions well before decades+-scale ABI
incompatibilities hit you.

IIUC, adopting HPy would break API: adding a context argument to all
functions would would be so massive a change, it'd be easier to just
call that HPy rather than Python C-API. And the doc I shared is for the
C-API.
(Perhaps CPython can even move to HPy and implement C-API as a veener on
top -- but I don't think the original C-API can go away in our lifetimes.)
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZHHLESVGP6BHNK2UHJTPY34GVWJCV3KP/
Code of Conduct: http://python.org/psf/codeofconduct/