Hi all,
for quite some time I've been working on a Python implementation of a protocol called NTS which requires access to an API in OpenSSL which is not provided by the Python ssl module. I added a patch for that which unfortunately for me the maintainer did not want to accept. Some comments were made of a possible future way to give more generic access to the openssl library via ctypes/cffi but I have been unable to find more information about that. I was home sick last week and decided to take a shot at it and have built something that I feel is a bit ugly but does seem to work. I'd like to some feedback on this approach.
My patches can be found on github, based on the Python 3.11 tag:
https://github.com/python/cpython/compare/3.11...wingel:cpython:main
Here's a short description of each patch on this branch:
"bpo-37952: SSL: add support for export_keying_material" is my old patch which adds the method I need to the ssl library just for reference.
The other commits add the necessary infrastructure with some example code. These commits are not ready for submission but hopefully they show what I have in mind.
"Add CRYPTO_DLL_PATH and SSL_DLL_PATH to the _ssl module. "
This commit adds two constants to the "_ssl" C module with the paths to libcrypto and libssl respectively. On Linux dladdr and on Windows GetModuleHandle/GetModuleFilename are used on a symbol in each library to find the path to the corresponding DLL. I've verified that this works Debian Bulleye and on Windows 10 with Visual Studio 2017. I don't own a Mac so I haven't been able to test this on macOS, but I believe dladdr is available on modern macOS so it might work out of the box. With the paths it's possible to use ctypes or cffi get a handle to these libraries.
"Add API to get the address of the SSL structure" then adds an API to an SSLSocket which returns the address of the corresponding "SSL" C structure. This address can be used by ctypes/cffi. One would probably want to expose SSL_CTX, SSL_SESSION and BIO too but I started with just SSL since that's what my code needs right now.
"Add a small test program" is a small test program that uses the infrastructure from the two above commits to call C functions in libssl/libcrypto using both ctypes and cffi. It's a bit ugly but hopefully it's not too hard to understand.
"Example of how to extend the ssl library using ctypes" is an example of how a Python module that extends the SSL library using ctypes could look. First get a handle to libssl using ctypes, set up ctypes with the correct API for the export_keying_material function, wrap it in a more Pythonic function and then extend SSLSocket with the new function. A simplified version looks like this:
import ssl, ctypes
ssl_lib = ctypes.CDLL(ssl._ssl.SSL_DLL_PATH)
ssl_lib.SSL_export_keying_material.argtypes = (
ctypes.c_void_p, # SSL pointer
ctypes.c_void_p, ctypes.c_size_t, # out pointer, out length
ctypes.c_void_p, ctypes.c_size_t, # label buffer, label length
ctypes.c_void_p, ctypes.c_size_t, # context, context length
ctypes.c_int) # use context flag
ssl_lib.SSL_export_keying_material.restype = ctypes.c_int
def SSL_export_keying_material(self, label, key_len, context = None):
c_key = ctypes.create_string_buffer(key_len)
c_label = ctypes.create_string_buffer(len(label))
c_context = ctypes.create_string_buffer(context, len(context))
if ssl_lib.SSL_export_keying_material(
self._sslobj.get_internal_addr(),
c_key, key_len,
c_label, len(label),
c_context, len(context), 1);
return bytes(c_key)
ssl.SSLSocket.export_keying_material = SSL_export_keying_material
There's a final commit "Expose more OPENSSL_ variables" which add some more constants to the ssl module which expose the cflags and build information from OpenSSL. This patch is not really necessary, but it might be a good idea to compare these constants with the corresponding constants retrieved using ctypes/cffi to make sure that exactly the same version of the openssl library is used.
Does this seem like a good idea? As I said, I feel that it is a bit ugly, but it does mean that if someone wants to use some SSL_really_obscure_function in libcrypto or libssl they can do that without having to rebuild all of CPython themselves. Or if they want to integrate with some other C library that wants a raw pointer to a SSL socket. Hopefully this would reduce the burden on the ssl module maintainers a bit.
Anyway, if you think this is a good approach I could clean up my patches, add support for SSL_CTX/SSL_SESSION/BIO, document all of this and make it into a proper pull request.
/Christer
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JFUMGTPXEGV63DIY5BNZRRCNADPEQC2O/
Code of Conduct: http://python.org/psf/codeofconduct/
for quite some time I've been working on a Python implementation of a protocol called NTS which requires access to an API in OpenSSL which is not provided by the Python ssl module. I added a patch for that which unfortunately for me the maintainer did not want to accept. Some comments were made of a possible future way to give more generic access to the openssl library via ctypes/cffi but I have been unable to find more information about that. I was home sick last week and decided to take a shot at it and have built something that I feel is a bit ugly but does seem to work. I'd like to some feedback on this approach.
My patches can be found on github, based on the Python 3.11 tag:
https://github.com/python/cpython/compare/3.11...wingel:cpython:main
Here's a short description of each patch on this branch:
"bpo-37952: SSL: add support for export_keying_material" is my old patch which adds the method I need to the ssl library just for reference.
The other commits add the necessary infrastructure with some example code. These commits are not ready for submission but hopefully they show what I have in mind.
"Add CRYPTO_DLL_PATH and SSL_DLL_PATH to the _ssl module. "
This commit adds two constants to the "_ssl" C module with the paths to libcrypto and libssl respectively. On Linux dladdr and on Windows GetModuleHandle/GetModuleFilename are used on a symbol in each library to find the path to the corresponding DLL. I've verified that this works Debian Bulleye and on Windows 10 with Visual Studio 2017. I don't own a Mac so I haven't been able to test this on macOS, but I believe dladdr is available on modern macOS so it might work out of the box. With the paths it's possible to use ctypes or cffi get a handle to these libraries.
"Add API to get the address of the SSL structure" then adds an API to an SSLSocket which returns the address of the corresponding "SSL" C structure. This address can be used by ctypes/cffi. One would probably want to expose SSL_CTX, SSL_SESSION and BIO too but I started with just SSL since that's what my code needs right now.
"Add a small test program" is a small test program that uses the infrastructure from the two above commits to call C functions in libssl/libcrypto using both ctypes and cffi. It's a bit ugly but hopefully it's not too hard to understand.
"Example of how to extend the ssl library using ctypes" is an example of how a Python module that extends the SSL library using ctypes could look. First get a handle to libssl using ctypes, set up ctypes with the correct API for the export_keying_material function, wrap it in a more Pythonic function and then extend SSLSocket with the new function. A simplified version looks like this:
import ssl, ctypes
ssl_lib = ctypes.CDLL(ssl._ssl.SSL_DLL_PATH)
ssl_lib.SSL_export_keying_material.argtypes = (
ctypes.c_void_p, # SSL pointer
ctypes.c_void_p, ctypes.c_size_t, # out pointer, out length
ctypes.c_void_p, ctypes.c_size_t, # label buffer, label length
ctypes.c_void_p, ctypes.c_size_t, # context, context length
ctypes.c_int) # use context flag
ssl_lib.SSL_export_keying_material.restype = ctypes.c_int
def SSL_export_keying_material(self, label, key_len, context = None):
c_key = ctypes.create_string_buffer(key_len)
c_label = ctypes.create_string_buffer(len(label))
c_context = ctypes.create_string_buffer(context, len(context))
if ssl_lib.SSL_export_keying_material(
self._sslobj.get_internal_addr(),
c_key, key_len,
c_label, len(label),
c_context, len(context), 1);
return bytes(c_key)
ssl.SSLSocket.export_keying_material = SSL_export_keying_material
There's a final commit "Expose more OPENSSL_ variables" which add some more constants to the ssl module which expose the cflags and build information from OpenSSL. This patch is not really necessary, but it might be a good idea to compare these constants with the corresponding constants retrieved using ctypes/cffi to make sure that exactly the same version of the openssl library is used.
Does this seem like a good idea? As I said, I feel that it is a bit ugly, but it does mean that if someone wants to use some SSL_really_obscure_function in libcrypto or libssl they can do that without having to rebuild all of CPython themselves. Or if they want to integrate with some other C library that wants a raw pointer to a SSL socket. Hopefully this would reduce the burden on the ssl module maintainers a bit.
Anyway, if you think this is a good approach I could clean up my patches, add support for SSL_CTX/SSL_SESSION/BIO, document all of this and make it into a proper pull request.
/Christer
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JFUMGTPXEGV63DIY5BNZRRCNADPEQC2O/
Code of Conduct: http://python.org/psf/codeofconduct/