Mailing List Archive: First impressions from OpenSSL 3.0.0 and httpd 2.4.45

First impressions from OpenSSL 3.0.0 and httpd 2.4.45

Aug 1, 2020, 8:44 AM

Post #1 of 3 (157 views)

Hi there,

during release testing for 2.4.45 I also built and tested using OpenSSL
3.0.0alpha5 on the server. Overall first results are pretty good:

- a few deprecation warnings during compilation:

modules/ssl/ssl_engine_config.c:610:5: warning: 'ENGINE_by_id' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:612:9: warning: 'ENGINE_free' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:617:9: warning: 'ENGINE_get_first' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:619:13: warning: 'ENGINE_get_id' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:620:42: warning: 'ENGINE_get_name' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:623:13: warning: 'ENGINE_get_next' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:457:9: warning: 'ENGINE_by_id' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:467:13: warning: 'ENGINE_ctrl' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:471:9: warning: 'ENGINE_set_default' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:482:9: warning: 'ENGINE_free' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2611:9: warning: 'HMAC_Init_ex' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2632:9: warning: 'HMAC_Init_ex' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_log.c:90:5: warning: 'ERR_peek_error_line_data'
is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:856:5: warning: 'ENGINE_by_id' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:864:5: warning: 'ENGINE_init' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:877:9: warning:
'ENGINE_ctrl_cmd_string' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:886:9: warning: 'ENGINE_ctrl_cmd' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:896:5: warning:
'ENGINE_load_private_key' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:904:5: warning: 'ENGINE_finish' is
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:905:5: warning: 'ENGINE_free' is
deprecated [-Wdeprecated-declarations]

- a few const warnings

modules/ssl/ssl_engine_kernel.c:608:55: warning: passing argument 2 of
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:627:61: warning: passing argument 2 of
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:638:57: warning: passing argument 2 of
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:1039:49: warning: passing argument 2 of
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
[-Wdiscarded-qualifiers]

and unit tests show two problems, one will be fixed in OpenSSL itself:

- during unit test preparation, our test script create a PKCS12 store
with default encoding params. That's known to be broken in alpha5. So
the "-configure" step of "t/TEST" should be run before the actual
testing with a stable version of OpenSSL.
https://github.com/openssl/openssl/pull/12540
https://github.com/openssl/openssl/issues/11672

- independent of OpenSSL 3.0.0: to work around the previous observation
I tried using the env var "APACHE_TEST_OPENSSL_CMD". Unfortunately this
is slightly broken, because it tests for the existence using the "which"
function in TestConfig.pm and that function is broken when used for a
command containing a path component. I temporarily fixed it using:

@@ -1782,6 +1782,11 @@

return undef unless $program;

+ # No need to search PATH components
+ # if $program already contains a path
+ return $program if !OSX and !WINFU and
+ $program =~ /\// and -f $program and -x $program;
+
my @dirs = File::Spec->path();

require Config;

- when testing with client >= OpenSSL 1.1.0 against 3.0.0alpha5, only
t/ssl/proxy.t shows failures, especially in eat_post but already during
TLS handshake:

[ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
AH01964: Connection to child 82 established (server localhost:8532)

[ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
AH02276: Certificate Verification: Error (3): unable to get certificate
CRL [.subject:
emailAddress=test-dev@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San
Francisco,ST=California,C=US / issuer:
emailAddress=test-dev@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San
Francisco,ST=California,C=US / serial: 0C / notbefore: Jul 30 23:29:05
2020 GMT / notafter: Jul 30 23:29:05 2021 GMT]

[ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
AH02008: SSL library error 1 in handshake (server localhost:8532)

[ssl:info] [pid 9162:tid 140326149928720] SSL Library Error:
error:0A000418:SSL routines::tlsv1 alert unknown ca (SSL alert number 48)

[ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
AH01998: Connection closed to child 82 with abortive shutdown (server
localhost:8532)

[example_hooks:notice] [pid 9162:tid 140326149928720] x_create_request()
[ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
AH02003: SSL Proxy connect failed

[ssl:info] [pid 9162:tid 140326166714128] SSL Library Error:
error:0A000086:SSL routines::certificate verify failed

[ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
AH01998: Connection closed to child 0 with abortive shutdown (server
localhost:8563)

[ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
AH01997: SSL handshake failed: sending 502

[proxy:error] [pid 9162:tid 140326166714128] (20014)Internal error
(specific information not available): [client 127.0.0.1:49283] AH01084:
pass request body failed to [::1]:8532 (localhost)

[proxy:error] [pid 9162:tid 140326166714128] [client 127.0.0.1:49283]
AH00898: Error during SSL Handshake with remote server returned by /eat_post

[proxy_http:error] [pid 9162:tid 140326166714128] [client
127.0.0.1:49283] AH01097: pass request body failed to [::1]:8532
(localhost) from 127.0.0.1 ()

- when testing with OpenSSL 0.9.8zh as a client, mostly all TLS tests
fail. I guess, one would have to load the legacy provider for the server
side OpenSSL to allow handshakes with the old version of TLS supported
by 0.9.8. I have not verified this yet.

Regards,

Rainer

Re: First impressions from OpenSSL 3.0.0 and httpd 2.4.45 [ In reply to ]

wrowe at rowe-clan

Aug 4, 2020, 6:53 AM

Post #2 of 3 (156 views)

Permalink

On Sat, Aug 1, 2020 at 10:44 AM Rainer Jung <rainer.jung@kippdata.de> wrote:

> Hi there,
>
> during release testing for 2.4.45 I also built and tested using OpenSSL
> 3.0.0alpha5 on the server. Overall first results are pretty good:
>

This is pretty awesome, thanks! Beyond the usual current packages, I'm also
doing a snapshots against released maintenance branch snapshots all our
dependencies when available (e.g. the working branch for lua is not
apparent),
so 2.4.46 is validated against;

apr_rev=1878356
apr_srcpath=https://svn.apache.org/repos/asf/apr/apr/branches/1.7.x
apr_ver=1.7.x-1878356
apriconv_rev=1872196
apriconv_srcpath=https://svn.apache.org/repos/asf/apr/apr-iconv/trunk
apriconv_ver=1.3.x-1872196
aprutil_rev=1880287
aprutil_srcpath=https://svn.apache.org/repos/asf/apr/apr-util/branches/1.7.x
aprutil_ver=1.7.x-1880287
brotli_rev=0c5603e
brotli_srcpath=https://github.com/google/brotli
brotli_ver=master
curl_rev=2b6b843bb
curl_srcpath=https://github.com/curl/curl
curl_ver=master
expat_rev=990e3d0
expat_srcpath=https://github.com/libexpat/libexpat
expat_ver=master
httpd_pkg=httpd-2.4.46.tar.bz2
httpd_srcpath=https://httpd.apache.org/dev/dist
httpd_ver=2.4.46
httpdtest_rev=1880510
httpdtest_srcpath=
https://svn.apache.org/repos/asf/httpd/test/framework/trunk
httpdtest_ver=1880510
jansson_rev=73ccec0
jansson_srcpath=https://github.com/akheron/jansson
jansson_ver=master
libxml2_rev=21ca8829
libxml2_srcpath=https://github.com/gnome/libxml2
libxml2_ver=master
lua_pkg=lua-5.4.0.tar.gz
lua_srcpath=https://www.lua.org/ftp
lua_ver=5.4.0
nghttp2_rev=4ecdc290
nghttp2_srcpath=https://github.com/nghttp2/nghttp2
nghttp2_ver=master
openldap_rev=63d6f680b
openldap_srcdir=openldap-2.x-63d6f680b
openldap_srcpath=https://git.openldap.org/openldap/openldap.git
openldap_ver=master
openssl_rev=6328d3673f
openssl_srcpath=https://github.com/openssl/openssl
openssl_ver=OpenSSL_1_1_1-stable
pcre_rev=1763
pcre_srcpath=svn://vcs.exim.org/pcre/code/trunk
pcre_ver=8.x-1763
zlib_rev=cacf7f1
zlib_srcpath=https://github.com/madler/zlib
zlib_ver=master

Based on your progress, I'm promoting openssl 3.0.0 branch from bleed to
snapshot status today as the maintenance branch of that dependency, thanks
to your validations!

Appreciated, and cheers,

Bill

(See https://github.com/appsuite/oss-httpd-build/ for reproducing such
speculative snapshot or bleed builds.)

Re: First impressions from OpenSSL 3.0.0 and httpd 2.4.45 [ In reply to ]

rainer.jung at kippdata

Aug 4, 2020, 2:00 PM

Post #3 of 3 (156 views)

Permalink

Concerning the failures with OpenSSL 3.0.0 in t/ssl/proxy.t, this should
be gone with the next alpha or beta of OpenSSL 3.0.0.

The culprit is indeed:

> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH02276: Certificate Verification: Error (3): unable to get certificate
> CRL [subject:
>
emailAddress=test-dev@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San

> Francisco,ST=California,C=US / issuer:
> emailAddress=test-dev@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San
> Francisco,ST=California,C=US / serial: 0C / notbefore: Jul 30 23:29:05
> 2020 GMT / notafter: Jul 30 23:29:05 2021 GMT]

The reason is, that lib/Apache/TestSSLCA.pm does not use the injected
"APACHE_TEST_OPENSSL_CMD" in one line, where it uses "`openssl ...`"
instead of "`$openssl ...`". And this happens exactly when the hash file
for ca-bundle.crt gets created. So instead of the older 1.1.1 openssl I
inject during configure, the new 3.0.0 gets used to create the hash
file. That would be fine, but OpenSSL 3.0.0 has a bug just fixed very
recently (not yet released), that "openssl crl" can not read from STDIN.
Which is what we do.

I'll commit the "$openssl" instead of "openssl" in backticks for
lib/Apache/TestSSLCA.pm to make its behavior more consistent.

Concerning the failures when the test client uses OpenSSL 0.9.8 I was
able to provide OpenSSL 3.0.0 in the server with a auto-loaded
openssl.cnf which contained the lines to load the legacy provider. The
provider got loaded, but still the handshakes with the old OpenSSL fail.
Don't know why. Probably not the biggest problem, because 0.9.8 based
clients should really not matter when thinking about 3.0.0 support in
the server.

Regards,

Rainer

Am 01.08.2020 um 17:44 schrieb Rainer Jung:
> Hi there,
>
> during release testing for 2.4.45 I also built and tested using OpenSSL
> 3.0.0alpha5 on the server. Overall first results are pretty good:
>
> - a few deprecation warnings during compilation:
>
> modules/ssl/ssl_engine_config.c:610:5: warning: 'ENGINE_by_id' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_config.c:612:9: warning: 'ENGINE_free' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_config.c:617:9: warning: 'ENGINE_get_first' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_config.c:619:13: warning: 'ENGINE_get_id' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_config.c:620:42: warning: 'ENGINE_get_name' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_config.c:623:13: warning: 'ENGINE_get_next' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_init.c:457:9: warning: 'ENGINE_by_id' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_init.c:467:13: warning: 'ENGINE_ctrl' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_init.c:471:9: warning: 'ENGINE_set_default' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_init.c:482:9: warning: 'ENGINE_free' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_kernel.c:2611:9: warning: 'HMAC_Init_ex' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_kernel.c:2632:9: warning: 'HMAC_Init_ex' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_log.c:90:5: warning: 'ERR_peek_error_line_data'
> is deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:856:5: warning: 'ENGINE_by_id' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:864:5: warning: 'ENGINE_init' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:877:9: warning:
> 'ENGINE_ctrl_cmd_string' is deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:886:9: warning: 'ENGINE_ctrl_cmd' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:896:5: warning:
> 'ENGINE_load_private_key' is deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:904:5: warning: 'ENGINE_finish' is
> deprecated [-Wdeprecated-declarations]
> modules/ssl/ssl_engine_pphrase.c:905:5: warning: 'ENGINE_free' is
> deprecated [-Wdeprecated-declarations]
>
> - a few const warnings
>
> modules/ssl/ssl_engine_kernel.c:608:55: warning: passing argument 2 of
> 'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
> [-Wdiscarded-qualifiers]
> modules/ssl/ssl_engine_kernel.c:627:61: warning: passing argument 2 of
> 'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
> [-Wdiscarded-qualifiers]
> modules/ssl/ssl_engine_kernel.c:638:57: warning: passing argument 2 of
> 'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
> [-Wdiscarded-qualifiers]
> modules/ssl/ssl_engine_kernel.c:1039:49: warning: passing argument 2 of
> 'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type
> [-Wdiscarded-qualifiers]
>
> and unit tests show two problems, one will be fixed in OpenSSL itself:
>
> - during unit test preparation, our test script create a PKCS12 store
> with default encoding params. That's known to be broken in alpha5. So
> the "-configure" step of "t/TEST" should be run before the actual
> testing with a stable version of OpenSSL.
> https://github.com/openssl/openssl/pull/12540
> https://github.com/openssl/openssl/issues/11672
>
> - independent of OpenSSL 3.0.0: to work around the previous observation
> I tried using the env var "APACHE_TEST_OPENSSL_CMD". Unfortunately this
> is slightly broken, because it tests for the existence using the "which"
> function in TestConfig.pm and that function is broken when used for a
> command containing a path component. I temporarily fixed it using:
>
> @@ -1782,6 +1782,11 @@
>
>      return undef unless $program;
>
> +    # No need to search PATH components
> +    # if $program already contains a path
> +    return $program if !OSX and !WINFU and
> +        $program =~ /\// and -f $program and -x $program;
> +
>      my @dirs = File::Spec->path();
>
>      require Config;
>
>
> - when testing with client >= OpenSSL 1.1.0 against 3.0.0alpha5, only
> t/ssl/proxy.t shows failures, especially in eat_post but already during
> TLS handshake:
>
> [ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
> AH01964: Connection to child 82 established (server localhost:8532)
>
> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH02276: Certificate Verification: Error (3): unable to get certificate
> CRL [.subject:
> emailAddress=test-dev@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San
> Francisco,ST=California,C=US / issuer:
> emailAddress=test-dev@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San
> Francisco,ST=California,C=US / serial: 0C / notbefore: Jul 30 23:29:05
> 2020 GMT / notafter: Jul 30 23:29:05 2021 GMT]
>
> [ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
> AH02008: SSL library error 1 in handshake (server localhost:8532)
>
> [ssl:info] [pid 9162:tid 140326149928720] SSL Library Error:
> error:0A000418:SSL routines::tlsv1 alert unknown ca (SSL alert number 48)
>
> [ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312]
> AH01998: Connection closed to child 82 with abortive shutdown (server
> localhost:8532)
>
> [example_hooks:notice] [pid 9162:tid 140326149928720] x_create_request()
> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH02003: SSL Proxy connect failed
>
> [ssl:info] [pid 9162:tid 140326166714128] SSL Library Error:
> error:0A000086:SSL routines::certificate verify failed
>
> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH01998: Connection closed to child 0 with abortive shutdown (server
> localhost:8563)
>
> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH01997: SSL handshake failed: sending 502
>
> [proxy:error] [pid 9162:tid 140326166714128] (20014)Internal error
> (specific information not available): [client 127.0.0.1:49283] AH01084:
> pass request body failed to [::1]:8532 (localhost)
>
> [proxy:error] [pid 9162:tid 140326166714128] [client 127.0.0.1:49283]
> AH00898: Error during SSL Handshake with remote server returned by
> /eat_post
>
> [proxy_http:error] [pid 9162:tid 140326166714128] [client
> 127.0.0.1:49283] AH01097: pass request body failed to [::1]:8532
> (localhost) from 127.0.0.1 ()
>
>
> - when testing with OpenSSL 0.9.8zh as a client, mostly all TLS tests
> fail. I guess, one would have to load the legacy provider for the server
> side OpenSSL to allow handshakes with the old version of TLS supported
> by 0.9.8. I have not verified this yet.
>
>
> Regards,
>
> Rainer