Mailing List Archive: Using the Python C API in C++

Using the Python C API in C++

Apr 27, 2022, 9:31 AM

Post #1 of 17 (1128 views)

Hi,

If you got issues with using the Python C API <Python.h> in C++,
please speak up! I'm looking for feedback :-)

Extending Python by writing C++ code is now easy with the pybind11 project:
https://pybind11.readthedocs.io/

It seems like over the last years, building C++ extensions with the
Python C API started to emit more C++ compiler warnings. One
explanation may be that converting macros to static inline functions
(PEP 670) introduce new warnings, even if the old and the new code is
exactly the same. I just discover this issue recently. C and C++
compilers treat static inline functions differently. Macros are
treated at legacy code which cannot be fixed, like system headers or
old C header files, and so many warnings are made quiet. Static inline
functions (defined in header files) are treated as regular code and
compilers are more eager to emit warnings.

I just modified the Python C API to use C++ static_cast<type>(expr)
and reinterpret_cast<type>(expr) if it's used with C++. In C, the
regular (type)expr cast (called "old-style cast" by C++ compilers ;-))
is still used as before.

I'm also working on adding an unit test to make suite that using the
Python C API works with a C++ compiler and doesn't emit compiler
warnings:

* https://github.com/python/cpython/issues/91321
* https://github.com/python/cpython/pull/32175

In terms of C++ version, it was proposed to target C++11.

In the pythoncapi-compat project, I got warnings when the NULL
constant is used in static inline functions. I modified the
pythoncapi_compat.h header file to use nullptr if used with C++ to fix
these compiler warnings. So far, I'm unable to reproduce the issue
with <Python.h>, and so I didn't try to address this issue in Python.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/73UTO4MTKVJR7XWXTHLOV3V2L2QKP2TK/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

h.vetinari at gmx

Apr 27, 2022, 8:45 PM

Post #2 of 17 (1128 views)

Permalink

> In terms of C++ version, it was proposed to target C++11.

GCC 5 has full C++14 support (one library functionality missing), and so does VS2015 onwards as well as Clang 3.4, see
https://en.cppreference.com/w/cpp/compiler_support

I doubt that any older compilers are in use _anywhere_ in reasonable numbers that this should constrain the development of CPython.

While I don't know who proposed C++11 or where, I'd therefore like to propose to move to _at least_ C++14.

Note that https://github.com/python/peps/pull/2309 already bumped the required C-standard to C11, and originally defined this as
> The C11 subset are features supported by GCC 8.5,
> clang 8.0, and MSVC of Visual Studio 2017.

If those versions should be regarded as the lower bounds of compiler support (are they - or anything lower - tested on the build bots...?), then C++17 core language support would automatically fall out of this (there are some stragglers for full stdlib support, especially in clang; but that is usually not an issue).

Best
H.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AGUI6B6W55TUXL6SA7KQQGPYYSRJNCFH/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

stephenjturnbull at gmail

Apr 28, 2022, 6:03 AM

Post #3 of 17 (1128 views)

Permalink

h.vetinari@gmx.com writes:

> While I don't know who proposed C++11 or where, I'd therefore like
> to propose to move to _at least_ C++14.

What benefits does this have for Python development?

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HBPHSH5MNHPWQKF6PC5N7VK7X2SWLKI6/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

h.vetinari at gmx

Apr 28, 2022, 6:25 AM

Post #4 of 17 (1128 views)

Permalink

> > While I don't know who proposed C++11 or where, I'd therefore like
> > to propose to move to _at least_ C++14.
>
> What benefits does this have for Python development?

Likewise I can ask what benefits choosing C++11 would have?

In general, I think standards and compilers need version hygiene like anything else, just with a larger lag. But even in the standard things get deprecated occasionally, and more importantly: new warnings may get issued, and being able to include CPython into C++ warning-free was one of the points that Victor mentioned specifically.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KGJN7LN6FWAYFF2IYH5FMWWPEIWH5OMG/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

stephenjturnbull at gmail

Apr 28, 2022, 7:40 AM

Post #5 of 17 (1128 views)

Permalink

h.vetinari@gmx.com writes:

> > > While I don't know who proposed C++11 or where, I'd therefore like
> > > to propose to move to _at least_ C++14.
> >
> > What benefits does this have for Python development?
>
> Likewise I can ask what benefits choosing C++11 would have?

Not for me to answer, I'm not a proponent of the change. I'm sure if
you read past discussions here and on Discourse you'll find answers
from the people who studied the problem carefully. I thought you
might have something to add to the conversation, but I guess not?

Steve

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2YRVSA4D3DZNPQJBX35P3KW6W7DJGDK4/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

vstinner at python

Apr 28, 2022, 8:00 AM

Post #6 of 17 (1128 views)

Permalink

Since we are you talking about tests, we can easily run the tests on
multiple C++ versions. But we have to start somewhere, so I propose to
start with C++11. More C++ versions can be tested later.

Victor

On Thu, Apr 28, 2022 at 5:54 AM <h.vetinari@gmx.com> wrote:
>
> > In terms of C++ version, it was proposed to target C++11.
>
> GCC 5 has full C++14 support (one library functionality missing), and so does VS2015 onwards as well as Clang 3.4, see
> https://en.cppreference.com/w/cpp/compiler_support
>
> I doubt that any older compilers are in use _anywhere_ in reasonable numbers that this should constrain the development of CPython.
>
> While I don't know who proposed C++11 or where, I'd therefore like to propose to move to _at least_ C++14.
>
> Note that https://github.com/python/peps/pull/2309 already bumped the required C-standard to C11, and originally defined this as
> > The C11 subset are features supported by GCC 8.5,
> > clang 8.0, and MSVC of Visual Studio 2017.
>
> If those versions should be regarded as the lower bounds of compiler support (are they - or anything lower - tested on the build bots...?), then C++17 core language support would automatically fall out of this (there are some stragglers for full stdlib support, especially in clang; but that is usually not an issue).
>
> Best
> H.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AGUI6B6W55TUXL6SA7KQQGPYYSRJNCFH/
> Code of Conduct: http://python.org/psf/codeofconduct/

--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O6GC3IWERSJCOYUUJQYEFO5NKR5DS6UI/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

antoine at python

Apr 28, 2022, 8:15 AM

Post #7 of 17 (1128 views)

Permalink

On Thu, 28 Apr 2022 22:03:25 +0900
"Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
> h.vetinari@gmx.com writes:
>
> > While I don't know who proposed C++11 or where, I'd therefore like
> > to propose to move to _at least_ C++14.
>
> What benefits does this have for Python development?

Let me second that question as well.

I work on Apache Arrow, where the C++ parts require C++11 (and we can't
go further than this for now because of R compatibility concerns). We
could say that enabling the Python bindings switches the required C++
version to C++14, but that would bring complication for no actual again
given that you're not likely to benefit from C++14 features in the
header files of a *C* project, are you?

Regards

Antoine.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BIK3SEBQVCX4Y5IX3VDXGSL72P5PWB77/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

antoine at python

Apr 28, 2022, 8:16 AM

Post #8 of 17 (1128 views)

Permalink

As a data point, I don't remember that recent versions of CPython
brought any particular pain for PyArrow, which is a set of bindings
written in Cython around some C++ core library code.

Regards

Antoine.

On Wed, 27 Apr 2022 18:31:13 +0200
Victor Stinner <vstinner@python.org> wrote:
> Hi,
>
> If you got issues with using the Python C API <Python.h> in C++,
> please speak up! I'm looking for feedback :-)
>
> Extending Python by writing C++ code is now easy with the pybind11 project:
> https://pybind11.readthedocs.io/
>
> It seems like over the last years, building C++ extensions with the
> Python C API started to emit more C++ compiler warnings. One
> explanation may be that converting macros to static inline functions
> (PEP 670) introduce new warnings, even if the old and the new code is
> exactly the same. I just discover this issue recently. C and C++
> compilers treat static inline functions differently. Macros are
> treated at legacy code which cannot be fixed, like system headers or
> old C header files, and so many warnings are made quiet. Static inline
> functions (defined in header files) are treated as regular code and
> compilers are more eager to emit warnings.
>
> I just modified the Python C API to use C++ static_cast<type>(expr)
> and reinterpret_cast<type>(expr) if it's used with C++. In C, the
> regular (type)expr cast (called "old-style cast" by C++ compilers ;-))
> is still used as before.
>
> I'm also working on adding an unit test to make suite that using the
> Python C API works with a C++ compiler and doesn't emit compiler
> warnings:
>
> * https://github.com/python/cpython/issues/91321
> * https://github.com/python/cpython/pull/32175
>
> In terms of C++ version, it was proposed to target C++11.
>
> In the pythoncapi-compat project, I got warnings when the NULL
> constant is used in static inline functions. I modified the
> pythoncapi_compat.h header file to use nullptr if used with C++ to fix
> these compiler warnings. So far, I'm unable to reproduce the issue
> with <Python.h>, and so I didn't try to address this issue in Python.
>
> Victor

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JFRJID3ETO3GZ5UHKONDA45WTUS5N7KE/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

vstinner at python

Apr 28, 2022, 8:52 AM

Post #9 of 17 (1128 views)

Permalink

Recently, a issue about C++20 compatibility was reported:

"The Python library will not compile with a C++2020 compiler because
the code uses the reserved “module” keyword"
https://github.com/python/cpython/issues/83536

In fact, after a long study, Python C API is *not* affected by this
issue. Using "module" remains valid in C++20: see the issue for
details.

Victor

On Thu, Apr 28, 2022 at 5:19 PM Antoine Pitrou <antoine@python.org> wrote:
>
> On Thu, 28 Apr 2022 22:03:25 +0900
> "Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
> > h.vetinari@gmx.com writes:
> >
> > > While I don't know who proposed C++11 or where, I'd therefore like
> > > to propose to move to _at least_ C++14.
> >
> > What benefits does this have for Python development?
>
> Let me second that question as well.
>
> I work on Apache Arrow, where the C++ parts require C++11 (and we can't
> go further than this for now because of R compatibility concerns). We
> could say that enabling the Python bindings switches the required C++
> version to C++14, but that would bring complication for no actual again
> given that you're not likely to benefit from C++14 features in the
> header files of a *C* project, are you?
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BIK3SEBQVCX4Y5IX3VDXGSL72P5PWB77/
> Code of Conduct: http://python.org/psf/codeofconduct/

--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AS2CRKHCPLXJ74CTMNURAZ5ANAKUNZ3J/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

antoine at python

Apr 28, 2022, 9:10 AM

Post #10 of 17 (1128 views)

Permalink

On Thu, 28 Apr 2022 17:52:40 +0200
Victor Stinner <vstinner@python.org> wrote:

> Recently, a issue about C++20 compatibility was reported:
>
> "The Python library will not compile with a C++2020 compiler because
> the code uses the reserved “module” keyword"
> https://github.com/python/cpython/issues/83536
>
> In fact, after a long study, Python C API is *not* affected by this
> issue. Using "module" remains valid in C++20: see the issue for
> details.

I'm not surprised. The C++ committee takes compatibility extremely
seriously...

Regards

Antoine.

>
> Victor
>
> On Thu, Apr 28, 2022 at 5:19 PM Antoine Pitrou <antoine@python.org> wrote:
> >
> > On Thu, 28 Apr 2022 22:03:25 +0900
> > "Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
> > > h.vetinari@gmx.com writes:
> > >
> > > > While I don't know who proposed C++11 or where, I'd therefore like
> > > > to propose to move to _at least_ C++14.
> > >
> > > What benefits does this have for Python development?
> >
> > Let me second that question as well.
> >
> > I work on Apache Arrow, where the C++ parts require C++11 (and we can't
> > go further than this for now because of R compatibility concerns). We
> > could say that enabling the Python bindings switches the required C++
> > version to C++14, but that would bring complication for no actual again
> > given that you're not likely to benefit from C++14 features in the
> > header files of a *C* project, are you?
> >
> > Regards
> >
> > Antoine.
> >
> >
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BIK3SEBQVCX4Y5IX3VDXGSL72P5PWB77/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LCWKZDADMICV3DRNKCLCNDHTXJXTCZ7K/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

h.vetinari at gmx

Apr 28, 2022, 3:08 PM

Post #11 of 17 (1128 views)

Permalink

> I work on Apache Arrow, where the C++ parts require C++11 (and we can't
go further than this for now because of R compatibility concerns).

Thanks for the datapoint, that's reasonable of course (though I'll note you're using abseil at least through grpc, and abseil is scheduled to remove C++11 support this year: https://abseil.io/blog/20201001-platforms).

> We could say that enabling the Python bindings switches the required C++
> version to C++14, but that would bring complication for no actual again
> given that you're not likely to benefit from C++14 features in the
> header files of a *C* project, are you?

I get your point, and I agree. My argument was to ensure compatibility with more recent standard versions, and Victor already suggested that several versions could be tested.

Best
H.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/X7KTI36ZWSYOURPEG3NFKEFO7TQ5L2WE/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

h.vetinari at gmx

Apr 28, 2022, 3:27 PM

Post #12 of 17 (1128 views)

Permalink

> Not for me to answer, I'm not a proponent of the change. I'm sure if
> you read past discussions here and on Discourse you'll find answers
> from the people who studied the problem carefully.

The opening mail proposed C++11 without rationale or references. I did search
the archives and discourse before, but nothing stood out, and I don't think an
encyclopedic knowledge of past python-dev discussions is a reasonable
requirement to comment or propose a variation on its merits.

> I thought you might have something to add to the conversation, but I guess not?

I find this tone quite out-of-place. I made a proposal (based on compiler support,
version hygiene and compatibility with newer standards), and I'd have been more
than happy to hear arguments (like Antoine's) or references for the merits of
preferring C++11 (though, again, the point became moot since Victor correctly
pointed out we can test against several versions).

Still, the insinuation (as it arrives on my end) that I shouldn't participate
seems really unnecessary.

Best,
H.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7SSI53EDU2U565O2TYRTU4CPYLVXPO5K/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

vstinner at python

Apr 29, 2022, 12:15 AM

Post #13 of 17 (1128 views)

Permalink

Hi,

The C++ version was discussed in the 2nd link that I gave in my first message:
https://github.com/python/cpython/issues/91321

Gregory wrote "If we can conditionally test new things based on C++XX
version, accumulating modern issue regression tests seems useful.
Otherwise 11 at minimum."

Another data point is that I mentioned that pybind11 is an important
project for Python ecosystem. This project name and its definition
refer to C++11: "pybind11: Seamless operability between C++11 and
Python".

So far, the only C++ code used in Python is the very recent change
that I made to use reinterpret_cast<> and static_cast<> in the Python
C API (.h header files). The C code base of Python is, well, written
only in C.

I mentioned nullptr if we get compiler warnings on the NULL constant
in static inline functions. nullptr was added by C++11.

I also mentioned C++20 "module" keyword and explained that it doesn't
affect the Python C API.

Having believes and assumptions about C++ compatibility is one thing.
My plan is more about actually *testing it* ;-)

Victor

On Fri, Apr 29, 2022 at 12:36 AM <h.vetinari@gmx.com> wrote:
>
> > Not for me to answer, I'm not a proponent of the change. I'm sure if
> > you read past discussions here and on Discourse you'll find answers
> > from the people who studied the problem carefully.
>
> The opening mail proposed C++11 without rationale or references. I did search
> the archives and discourse before, but nothing stood out, and I don't think an
> encyclopedic knowledge of past python-dev discussions is a reasonable
> requirement to comment or propose a variation on its merits.
>
> > I thought you might have something to add to the conversation, but I guess not?
>
> I find this tone quite out-of-place. I made a proposal (based on compiler support,
> version hygiene and compatibility with newer standards), and I'd have been more
> than happy to hear arguments (like Antoine's) or references for the merits of
> preferring C++11 (though, again, the point became moot since Victor correctly
> pointed out we can test against several versions).
>
> Still, the insinuation (as it arrives on my end) that I shouldn't participate
> seems really unnecessary.
>
> Best,
> H.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7SSI53EDU2U565O2TYRTU4CPYLVXPO5K/
> Code of Conduct: http://python.org/psf/codeofconduct/

--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HV6AV527HVU6REWNFABYFLNV6CHGJ6RN/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

vstinner at python

May 2, 2022, 6:21 AM

Post #14 of 17 (1128 views)

Permalink

Hi,

I merged the best basic tests to make sure that using the Python C API in
C++ does not emit compiler warnings:

* Code: https://github.com/python/cpython/blob/main/Lib/test/_testcppext.cpp
* Test: https://github.com/python/cpython/blob/main/Lib/test/test_cppext.py

The code is only built to check for compiler warnings. Later, I plan to
build this C++ extension to be able to *execute* it at runtime. For now, a
C++ compiler is required to run the Python 3.11 test suite. Maybe it should
be made optional, but so far I failed to test if distutils has an available
C++ compiler.

Slowly, more and more tests can be added. For example, this change fixing
compiler warnings when passing "const PyObject*" to Python C API functions
adds tests to _testcppext.cpp:
https://github.com/python/cpython/pull/92138

Victor

Re: Using the Python C API in C++ [ In reply to ]

steve.dower at python

May 2, 2022, 11:43 AM

Post #15 of 17 (1128 views)

Permalink

On 5/2/2022 2:21 PM, Victor Stinner wrote:
> Maybe it
> should be made optional, but so far I failed to test if distutils has an
> available C++ compiler.

Considering the current status of distutils, you should probably put
that check in configure or the Makefile and build the extension from
there. Then the test suite can check whether it was built or not.

Cheers,
Steve

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/46C4ZJZZ5KSPHS4WRICHEOEQZWHXHWWT/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

antoine at python

May 3, 2022, 12:21 AM

Post #16 of 17 (1112 views)

Permalink

On Mon, 2 May 2022 15:21:24 +0200
Victor Stinner <vstinner@python.org> wrote:
>
> Slowly, more and more tests can be added. For example, this change fixing
> compiler warnings when passing "const PyObject*" to Python C API functions
> adds tests to _testcppext.cpp:
> https://github.com/python/cpython/pull/92138

Doesn't passing "const PyObject*" leak implementation details, for
example that the reference count does not change? It seems to go
counter the objective of making the C API more abstract and more stable.

(C++ has the "mutable" keyword for such situat but I don't think C has
it yet)

Regards

Antoine.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YFD3WOOR6PRORTTKLTSSOSDNHRYAA2N7/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: Using the Python C API in C++ [ In reply to ]

vstinner at python

May 3, 2022, 2:57 AM

Post #17 of 17 (1112 views)

Permalink

Hi Antoine,

I'm in favor of making the C API more strict. The limited C API of Python
3.11 no longer cast arguments to PyObject* ! The caller must now cast to
PyObject*: see PEP 670 "convert macros to functions".

Converting macros to static inline functions (PEP 670) is already an
important change. I prefer to not change too many things at the same time.

Maybe in Python 3.12 (or later), we can consider requiring to not pass
"const PyObject*" in C++: remove the cast dropping constness of the
argument.

My long term goal is to only use opaque "PyObject*" "handles" in the C API.
But for practical reasons, changing the C API must be done slowly, step by
step. To reduce the maintenance burden in third party C and C+ extensions.

--

IMO it's a bad idea to use "const PyObject*" in the C API. Maybe today, a
function doesn't modify the memory. But maybe tomorrow, the implementation
will deeply change and will need to modify the memory. In C, using "const"
(for something other than const char*) is just not convenient.

Victor

On Tue, May 3, 2022 at 9:22 AM Antoine Pitrou <antoine@python.org> wrote:

> On Mon, 2 May 2022 15:21:24 +0200
> Victor Stinner <vstinner@python.org> wrote:
> >
> > Slowly, more and more tests can be added. For example, this change fixing
> > compiler warnings when passing "const PyObject*" to Python C API
> functions
> > adds tests to _testcppext.cpp:
> > https://github.com/python/cpython/pull/92138
>
> Doesn't passing "const PyObject*" leak implementation details, for
> example that the reference count does not change? It seems to go
> counter the objective of making the C API more abstract and more stable.
>
> (C++ has the "mutable" keyword for such situat but I don't think C has
> it yet)
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YFD3WOOR6PRORTTKLTSSOSDNHRYAA2N7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

--
Night gathers, and now my watch begins. It shall not end until my death.