Mailing List Archive

Bundling SSL modules in core [was: PSC #093: 2023-01-13]
On Sat, 14 Jan 2023 01:59:46 +0100
Graham Knop <haarg@haarg.org> wrote:

> On Fri, Jan 13, 2023 at 7:04 PM Paul "LeoNerd" Evans
> <leonerd@leonerd.org.uk> wrote:
> > Things bundled into cpan/ are built by perl's own build system while
> > basically ignoring the actual configure- and build-time
> > instructions of that module. This makes it very complicated to do
> > things like optional configurations, etc.. It would be difficult to
> > get Net::SSLeay to work with that.
>
> This seems like a misunderstanding or something. The configure and
> build instructions of dists in cpan/ are not ignored. If a dist in
> cpan/ includes a Makefile.PL, that file is used to configure and build
> the dist. I'm pretty sure that doesn't have anything to do with why
> Net::SSLeay is not included in core.

Ah, perhaps I have misremembered then. I remember when I wanted to add
a bunch of symbols to Socket so that IO::Socket::IP could use them
while being moved to core, it was all very complicated. Socket's own
Makefile.PL couldn't perform configure-like tests; it had to rely on
core's own configure mechanisms to tell it such answers as whether the
system C library has an `inet_pton()` function.

I may therefore be under the misapprehension that all bundled modules
have that restriction. If they don't, then great - it sounds like it
would be even easier to bundle Net::SSLeay into core than I had first
imagined. :)

If that reason isn't in the way, then I don't see why we can't at least
have an attempt at just bundling Net::SSLeay + IO::Socket::SSL with
perl, and see if it works.

Has anyone thought to just try it? ;)

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Sat, Jan 14, 2023 at 3:47 PM Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> On Sat, 14 Jan 2023 01:59:46 +0100
> Graham Knop <haarg@haarg.org> wrote:
>
> > On Fri, Jan 13, 2023 at 7:04 PM Paul "LeoNerd" Evans
> > <leonerd@leonerd.org.uk> wrote:
> > > Things bundled into cpan/ are built by perl's own build system while
> > > basically ignoring the actual configure- and build-time
> > > instructions of that module. This makes it very complicated to do
> > > things like optional configurations, etc.. It would be difficult to
> > > get Net::SSLeay to work with that.
> >
> > This seems like a misunderstanding or something. The configure and
> > build instructions of dists in cpan/ are not ignored. If a dist in
> > cpan/ includes a Makefile.PL, that file is used to configure and build
> > the dist. I'm pretty sure that doesn't have anything to do with why
> > Net::SSLeay is not included in core.
>
> Ah, perhaps I have misremembered then. I remember when I wanted to add
> a bunch of symbols to Socket so that IO::Socket::IP could use them
> while being moved to core, it was all very complicated. Socket's own
> Makefile.PL couldn't perform configure-like tests; it had to rely on
> core's own configure mechanisms to tell it such answers as whether the
> system C library has an `inet_pton()` function.
>
> I may therefore be under the misapprehension that all bundled modules
> have that restriction. If they don't, then great - it sounds like it
> would be even easier to bundle Net::SSLeay into core than I had first
> imagined. :)
>
> If that reason isn't in the way, then I don't see why we can't at least
> have an attempt at just bundling Net::SSLeay + IO::Socket::SSL with
> perl, and see if it works.
>
> Has anyone thought to just try it? ;)
>

You can stick it into cpan/ and it should Just Work™, but what to do when
Net::SSLeay doesn't build, most obviously because the system doesn't have
openssl. There's some prior art for this kind of scenario in the *DBM
modules, but this would be a tricker case as openssl is much more version
sensitive.

Leon
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Sun, 15 Jan 2023 13:42:38 +0100
Leon Timmermans <fawaka@gmail.com> wrote:

> You can stick it into cpan/ and it should Just Work™, but what to do
> when Net::SSLeay doesn't build, most obviously because the system
> doesn't have openssl. There's some prior art for this kind of
> scenario in the *DBM modules, but this would be a tricker case as
> openssl is much more version sensitive.

Personally, I think that would be fine. If it doesn't work then the
user is no worse off than they would be right now when we don't bundle
it at all. It is OK for it to be optionally provided "if possible".

The aim of the exercise isn't to create a perfect 100% guaranteed "you
will definitely have SSL". It's to make it easier for users who want to
- and know how to - have SSL working "out of the box", to arrange for
it to happen. This would cut down a whole bunch of steps and make it
easier for them.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Sun, Jan 15, 2023 at 3:44 PM Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> On Sun, 15 Jan 2023 13:42:38 +0100
> Leon Timmermans <fawaka@gmail.com> wrote:
>
> > You can stick it into cpan/ and it should Just Work™, but what to do
> > when Net::SSLeay doesn't build, most obviously because the system
> > doesn't have openssl. There's some prior art for this kind of
> > scenario in the *DBM modules, but this would be a tricker case as
> > openssl is much more version sensitive.
>
> Personally, I think that would be fine. If it doesn't work then the
> user is no worse off than they would be right now when we don't bundle
> it at all. It is OK for it to be optionally provided "if possible".
>
> The aim of the exercise isn't to create a perfect 100% guaranteed "you
> will definitely have SSL". It's to make it easier for users who want to
> - and know how to - have SSL working "out of the box", to arrange for
> it to happen. This would cut down a whole bunch of steps and make it
> easier for them.
>

What about a configure option that toggles TLS (please don't call it SSL in
2023, SSL v3.0 is dead since mid 2015 https://www.rfc-editor.org/rfc/rfc7568
)?
If enabled a Perl build should fail if the TLS support can't be built, else
it would go missing unnoticed in automated builds.


>
> --
> Paul "LeoNerd" Evans
>
> leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
> http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
>
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Mon, 16 Jan 2023 09:04:57 +0100
Alexander Hartmaier <alex.hartmaier@gmail.com> wrote:

> What about a configure option that toggles TLS (please don't call it
> SSL in 2023, SSL v3.0 is dead since mid 2015
> https://www.rfc-editor.org/rfc/rfc7568 )?

Mmm. On-the-wire the protocol is indeed called TLS and not SSL. But in
practice in people's minds, "the ability to have a secure socket" is so
tightly associated with the name SSL that it's hard to change that
name. Additionally, the acronym "SSL" has basically no other meanings,
whereas "TLS" is also the name for "thread-local storage" which
sometimes causes confusion.

This is one of those times when yes "TLS" would be more technically
correct but in practice "SSL" causes less confusion.

> If enabled a Perl build should fail if the TLS support can't be
> built, else it would go missing unnoticed in automated builds.

Mmm yes that sounds like a useful option indeed. It would be nice not
to have it silently skip when you really did expect it to happen. Maybe
a three-way choice of

Don't bother
Optimistically try but skip if we can't
Definitely attempt it and abort if we can't

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Mon, Jan 16, 2023 at 12:03 PM Paul "LeoNerd" Evans <
leonerd@leonerd.org.uk> wrote:

> On Mon, 16 Jan 2023 09:04:57 +0100
> Alexander Hartmaier <alex.hartmaier@gmail.com> wrote:
>
> > What about a configure option that toggles TLS (please don't call it
> > SSL in 2023, SSL v3.0 is dead since mid 2015
> > https://www.rfc-editor.org/rfc/rfc7568 )?
>
> Mmm. On-the-wire the protocol is indeed called TLS and not SSL. But in
> practice in people's minds, "the ability to have a secure socket" is so
> tightly associated with the name SSL that it's hard to change that
> name. Additionally, the acronym "SSL" has basically no other meanings,
> whereas "TLS" is also the name for "thread-local storage" which
> sometimes causes confusion.
>
> This is one of those times when yes "TLS" would be more technically
> correct but in practice "SSL" causes less confusion.
>
> > If enabled a Perl build should fail if the TLS support can't be
> > built, else it would go missing unnoticed in automated builds.
>
> Mmm yes that sounds like a useful option indeed. It would be nice not
> to have it silently skip when you really did expect it to happen. Maybe
> a three-way choice of
>
> Don't bother
> Optimistically try but skip if we can't
> Definitely attempt it and abort if we can't
>
> In the security space where I'm at work, having Perl in the news for
'adding SSL support out of the box' would just increase the view of Perl
being a dead language.
As it's nowadays not even SSLv3 plus TLS 1.x but only TLS, I'd call it that
way.

I've convinced Jonathan quite easliy for Cro::SSL -> Cro::TLS:
https://github.com/croservices/cro-tls/commit/8ffb0aaca930ed53db4bf248317bff0450cf3ae3

Best regards, Alex


> --
> Paul "LeoNerd" Evans
>
> leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
> http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
>
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Mon, 16 Jan 2023 12:30:21 +0100
Alexander Hartmaier <alex.hartmaier@gmail.com> wrote:

> > In the security space where I'm at work, having Perl in the news
> > for
> 'adding SSL support out of the box' would just increase the view of
> Perl being a dead language.
> As it's nowadays not even SSLv3 plus TLS 1.x but only TLS, I'd call
> it that way.
>
> I've convinced Jonathan quite easliy for Cro::SSL -> Cro::TLS:
> https://github.com/croservices/cro-tls/commit/8ffb0aaca930ed53db4bf248317bff0450cf3ae3

Mmm well we can try. The main task is to bring Net::SSLeay and
IO::Socket::SSL into core though. Do you suggest we attempt to get
those modules to rename as well? ;)

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
Op 16-01-2023 om 12:30 schreef Alexander Hartmaier:
>
> In the security space where I'm at work, having Perl in the news for
> 'adding SSL support out of the box' would just increase the view of
> Perl being a dead language.
> As it's nowadays not even SSLv3 plus TLS 1.x but only TLS, I'd call it
> that way.


I see both viewpoints, but I heartily agree with Alexander here. If we
do not start changing today, we're only putting it off and Perl will be
more of a dead language every day we delay.


M4
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Mon, Jan 16, 2023 at 12:47 PM Paul "LeoNerd" Evans <
leonerd@leonerd.org.uk> wrote:

> On Mon, 16 Jan 2023 12:30:21 +0100
> Alexander Hartmaier <alex.hartmaier@gmail.com> wrote:
>
> > > In the security space where I'm at work, having Perl in the news
> > > for
> > 'adding SSL support out of the box' would just increase the view of
> > Perl being a dead language.
> > As it's nowadays not even SSLv3 plus TLS 1.x but only TLS, I'd call
> > it that way.
> >
> > I've convinced Jonathan quite easliy for Cro::SSL -> Cro::TLS:
> >
> https://github.com/croservices/cro-tls/commit/8ffb0aaca930ed53db4bf248317bff0450cf3ae3
>
> Mmm well we can try. The main task is to bring Net::SSLeay and
> IO::Socket::SSL into core though. Do you suggest we attempt to get
> those modules to rename as well? ;)
>
Cro(::SSL) was young, so this was an option. In case of Net::SSLeay which
name follows Open/LibreSSL, that doesn't make sense.
IO::Socket::TLS could be an additional module that limits the wire protocol
to TLS, but I'm not suggesting to rename IO::Socket::SSL.

Best regards, Alex

>
> --
> Paul "LeoNerd" Evans
>
> leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
> http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
>
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Sun, Jan 15, 2023 at 8:44 AM Paul "LeoNerd" Evans
<leonerd@leonerd.org.uk> wrote:
>
> On Sun, 15 Jan 2023 13:42:38 +0100
> Leon Timmermans <fawaka@gmail.com> wrote:
>
> > You can stick it into cpan/ and it should Just Work™, but what to do
> > when Net::SSLeay doesn't build, most obviously because the system
> > doesn't have openssl. There's some prior art for this kind of
> > scenario in the *DBM modules, but this would be a tricker case as
> > openssl is much more version sensitive.
>
> Personally, I think that would be fine. If it doesn't work then the
> user is no worse off than they would be right now when we don't bundle
> it at all. It is OK for it to be optionally provided "if possible".
>
> The aim of the exercise isn't to create a perfect 100% guaranteed "you
> will definitely have SSL". It's to make it easier for users who want to
> - and know how to - have SSL working "out of the box", to arrange for
> it to happen. This would cut down a whole bunch of steps and make it
> easier for them.

Porting/Glossary says:

extensions (Extensions.U):
This variable holds a list of all extension files (both XS and
non-xs) installed with the package. It is propagated to Config.pm
and is typically used to test whether a particular extension
is available.

The problem is that C<extensions> is populated at Configure time. I
have been assuming that the logic in Net::SSLeay's Makefile.PL to
probe for (Open|Libre)SSL will need to be reproduced in Configure so
we know before the build whether we're including it. Hopefully
someone smarter than me will think of why this assumption doesn't hold
and there is a better way to do it.

But it seems wrong to me to say that we have Net::SSLeay when we don't
by listing it in C<extensions> willy nilly before we know we can build
it, and equally wrong to provide it without saying we have it such
that someone testing for it in the traditional way thinks it's absent
when it is really present. It briefly flitted through my mind that we
could do a post-processing step to check whether we built it
successfully and regenerate Config.pm near the end of the build, but I
think touching Config.pm triggers a rebuild of Perl, so that's a
circular dependency. Touching the timestamp of Config.pm to make it
appear older than it really is seems hackish and prone to causing
weird build problems.

So, how hard would it be to add C<netssleay_openssl_libdir> and
C<netssleay_openssl_incdir> variables to Configure and only add
Net::SSLeay to C<extensions> when we are able to populate them? It
would be hard for me, but Unix shell scripting in general and
Configure in particular are not really my strong suit. But it seems
possible. Yes, we'd probably be patching weird cases under hints/ for
the foreseeable future, but that happens with a lot of things already.
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Sun, 15 Jan 2023 13:42:38 +0100
Leon Timmermans <fawaka@gmail.com> wrote:

> You can stick it into cpan/

OK, so lets just try it and see?

I have to admit I don't know what is the process for "just stick it in
cpan/". Is there a porting script or an automated button of some kind
we can press to make it happen, or is it a totally manual process of
"grab tarball, tar -x, git add a bunch of stuff, etc etc..."

If I knew how to proceed with that process I would have done it
probably twenty times by now already, and these two SSL modules would
have been in years ago.

Can we get a bit of a description on this?

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Fri, Feb 10, 2023 at 11:21 AM Paul "LeoNerd" Evans
<leonerd@leonerd.org.uk> wrote:
>
> On Sun, 15 Jan 2023 13:42:38 +0100
> Leon Timmermans <fawaka@gmail.com> wrote:
>
> > You can stick it into cpan/
>
> OK, so lets just try it and see?

After being inspired by this thread, I did in fact try this and opened
a PR with my attempt: https://github.com/Perl/perl5/pull/20739

> I have to admit I don't know what is the process for "just stick it in
> cpan/". Is there a porting script or an automated button of some kind
> we can press to make it happen, or is it a totally manual process of
> "grab tarball, tar -x, git add a bunch of stuff, etc etc..."
>
> If I knew how to proceed with that process I would have done it
> probably twenty times by now already, and these two SSL modules would
> have been in years ago.
>
> Can we get a bit of a description on this?
>

What appears to be the routine is to edit Porting/Maintainers.pl, add
the module to the list, run Porting/sync-with-cpan, and handle the
porting tests that may fail. I'm not positive of that process because
by the time I found it, I had already pulled down the tarball and done
half of what sync-with-cpan does. Running sync-with-cpan did help me
in correcting some of the porting tests I was having issues with and
made correcting the PR easier.

The inclusion for Net::SSLeay was/will be a bit more involved since
there has to be additions to Configure to selectively disable it.

-Jon Gentle
Re: Bundling SSL modules in core [was: PSC #093: 2023-01-13] [ In reply to ]
On Fri, 10 Feb 2023 12:03:16 -0500
Jon Gentle <atrodo@atrodo.org> wrote:

> On Fri, Feb 10, 2023 at 11:21 AM Paul "LeoNerd" Evans
> <leonerd@leonerd.org.uk> wrote:
> >
> > On Sun, 15 Jan 2023 13:42:38 +0100
> > Leon Timmermans <fawaka@gmail.com> wrote:
> >
> > > You can stick it into cpan/
> >
> > OK, so lets just try it and see?
>
> After being inspired by this thread, I did in fact try this and opened
> a PR with my attempt: https://github.com/Perl/perl5/pull/20739

Ahyes, somehow that managed to escape my attention before now.

I've given it a nudge into running the CI tests. Initial results appear
that it has a problem on Windows - we should discuss further on the PR
thread.

> > Can we get a bit of a description on this?
>
> What appears to be the routine is to edit Porting/Maintainers.pl, add
> the module to the list, run Porting/sync-with-cpan, and handle the
> porting tests that may fail. I'm not positive of that process because
> by the time I found it, I had already pulled down the tarball and done
> half of what sync-with-cpan does. Running sync-with-cpan did help me
> in correcting some of the porting tests I was having issues with and
> made correcting the PR easier.

Hmmm I see.

> The inclusion for Net::SSLeay was/will be a bit more involved since
> there has to be additions to Configure to selectively disable it.

Righty.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/