Mailing List Archive

> On Tuesday, 23 November 2021, 18:55:55 CET, Neil Bowers <neilb@neilb.org>
> wrote:
>
> When you install Perl, although it comes with HTTP::Tiny, it doesn't support 
> https because HTTP::Tiny requires IO::Socket::SSL (which isn’t bundled with
> Perl). This means that CPAN.pm and other tools can't make https mandatory,
> as when bootstrapping a new box you'll typically install perl then run cpan to
> first install Net::SSLeay and IO::Socket::SSL.

I love this idea and as a standalone, I support it as is, if feasible. However, if we do this, can we improve the overall experience of Perl?

One of the issue that I think Perl has is that we our presence is almost entirely in sysadmin tasks and web development. I can't comment on the former, but the latter also requires a few more modules to make Perl more natively useful. If nothing else, a fast JSON tool would be welcome (not the JSON::PP that is in core. That's very slow)

Also, some proper native readline support would be nice for a useful debugger. I assume Term::ReadLine::Gnu isn't bundled because of the licence, but my debugger is always broken when I first use it.

Best,
Ovid

Op 23-11-2021 om 19:18 schreef Ovid via perl5-porters:
>> On Tuesday, 23 November 2021, 18:55:55 CET, Neil Bowers <neilb@neilb.org>
>> wrote:
>>
>> When you install Perl, although it comes with HTTP::Tiny, it doesn't support
>> https because HTTP::Tiny requires IO::Socket::SSL (which isn’t bundled with
>> Perl). This means that CPAN.pm and other tools can't make https mandatory,
>> as when bootstrapping a new box you'll typically install perl then run cpan to
>> first install Net::SSLeay and IO::Socket::SSL.
> I love this idea and as a standalone, I support it as is, if feasible.


+1


> However, if we do this, can we improve the overall experience of Perl?
>
> One of the issue that I think Perl has is that we our presence is almost entirely in sysadmin tasks and web development. I can't comment on the former, but the latter also requires a few more modules to make Perl more natively useful. If nothing else, a fast JSON tool would be welcome (not the JSON::PP that is in core. That's very slow)


Standard CSV parsing is another one. I use a lot of Python only because
that does support CSV parsing out of the box.


>
> Also, some proper native readline support would be nice for a useful debugger. I assume Term::ReadLine::Gnu isn't bundled because of the licence, but my debugger is always broken when I first use it.


+10 ;)


HTH,

M4

* Neil Bowers <neilb@neilb.org> [2021-11-23 17:55:28 +0000]:

> When you install Perl, although it comes with HTTP::Tiny, it doesn't support https because HTTP::Tiny requires IO::Socket::SSL (which isn?t bundled with Perl). This means that CPAN.pm and other tools can't make https mandatory, as when bootstrapping a new box you'll typically install perl then run cpan to first install Net::SSLeay and IO::Socket::SSL.
>
> I think we should make Perl support https "out of the box", so that CPAN.pm could make https mandatory, and other modules could then support https while relying only on core modules.
>
> One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I don't know if that's the best / right solution?
>
> One problem with bundling these modules is that we are then committed to ensuring that they work on all supported platforms / configurations. Is it ok to support https out-of-the-box on some supported platforms but not all of them? I think so.

That last point is the rub. I've on several occasions run into issues with
Net::SSLeay not building on upgrade Ubuntu because of a lag in supporting
some breaking ABI change in libssl. In addition to that, there might be some
issues with OSes that distribute perl even including the necessary development
libraries/headers out of the box. Otherwise, I think this is a nice idea -
just might require a large coordinated effort - particularly from "downstream"
packages, ports, and base OS maintainers. I'm none of those, but I do feel your
pain..

Brett

>
> Neil

--
--
oodler@cpan.org
oodler577@sdf-eu.org
SDF-EU Public Access UNIX System - http://sdfeu.org
irc.perl.org #openmp #pdl #native

Awesome as SSL is a modern standard.

23 noy 2021, Ç.a. 22:30 tarixind? Oodler 577 via perl5-porters <
perl5-porters@perl.org> yazd?:

> * Neil Bowers <neilb@neilb.org> [2021-11-23 17:55:28 +0000]:
>
> > When you install Perl, although it comes with HTTP::Tiny, it doesn't
> support https because HTTP::Tiny requires IO::Socket::SSL (which isn?t
> bundled with Perl). This means that CPAN.pm and other tools can't make
> https mandatory, as when bootstrapping a new box you'll typically install
> perl then run cpan to first install Net::SSLeay and IO::Socket::SSL.
> >
> > I think we should make Perl support https "out of the box", so that
> CPAN.pm could make https mandatory, and other modules could then support
> https while relying only on core modules.
> >
> > One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I don't
> know if that's the best / right solution?
> >
> > One problem with bundling these modules is that we are then committed to
> ensuring that they work on all supported platforms / configurations. Is it
> ok to support https out-of-the-box on some supported platforms but not all
> of them? I think so.
>
> That last point is the rub. I've on several occasions run into issues with
> Net::SSLeay not building on upgrade Ubuntu because of a lag in supporting
> some breaking ABI change in libssl. In addition to that, there might be
> some
> issues with OSes that distribute perl even including the necessary
> development
> libraries/headers out of the box. Otherwise, I think this is a nice idea -
> just might require a large coordinated effort - particularly from
> "downstream"
> packages, ports, and base OS maintainers. I'm none of those, but I do feel
> your
> pain..
>
> Brett
>
> >
> > Neil
>
> --
> --
> oodler@cpan.org
> oodler577@sdf-eu.org
> SDF-EU Public Access UNIX System - http://sdfeu.org
> irc.perl.org #openmp #pdl #native
>

??, 23 ????. 2021 ?. ? 20:55, Neil Bowers <neilb@neilb.org>:
> One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I don't know if that's the best / right solution?
>
> One problem with bundling these modules is that we are then committed to ensuring that they work on all supported platforms / configurations. Is it ok to support https out-of-the-box on some supported platforms but not all of them? I think so.

There exists (at least) one huge gotcha. openssl major api bump &
rolling distros like arch/void. There will be situations when the
distro already has an api-incompatible library, but even bleadperl
still doesn't have an updated bindings, not starting to speak about
x-1 release. So for a successful experience, both in-core tests and
http layer over it should be ready for a complete compilation failures
and subsequent module absence.

I'd say that it should be it's own separate level of guarantees for
those modules and their availability despite being bundled with core.

Best regards,
Sergey Aleynikov

On Tue, 23 Nov 2021 17:55:28 +0000
Neil Bowers <neilb@neilb.org> wrote:

> I think we should make Perl support https "out of the box", so that
> CPAN.pm could make https mandatory, and other modules could then
> support https while relying only on core modules.

I don't think anyone objects to the idea of better (i.e. any at all)
support of SSL/TLS out-of-the-box, but in case it needed being said:

+1 from me too.

> One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I
> don't know if that's the best / right solution?
>
> One problem with bundling these modules is that we are then committed
> to ensuring that they work on all supported platforms /
> configurations. Is it ok to support https out-of-the-box on some
> supported platforms but not all of them? I think so.

I feel surely this isn't a problem that Perl uniquely is trying to
solve. It feels like this should be easily solved by "doing what the
other languages do" - i.e. could someone more familiar with some of the
others report back on how e.g. Python or Ruby handle this kind of
thing? If it works for them then surely we can do the same?

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On Tue, Nov 23, 2021 at 1:19 PM Ovid via perl5-porters <
perl5-porters@perl.org> wrote:

> > On Tuesday, 23 November 2021, 18:55:55 CET, Neil Bowers <neilb@neilb.org
> >
> > wrote:
> >
> > When you install Perl, although it comes with HTTP::Tiny, it doesn't
> support
> > https because HTTP::Tiny requires IO::Socket::SSL (which isn’t bundled
> with
> > Perl). This means that CPAN.pm and other tools can't make https
> mandatory,
> > as when bootstrapping a new box you'll typically install perl then run
> cpan to
> > first install Net::SSLeay and IO::Socket::SSL.
>

I agree that this is a huge problem and we should make significant effort
to try to solve it. However it's not yet clear how we could best solve it.
Bundling Net::SSLeay and IO::Socket::SSL is of course the easiest in terms
of immediate effort

An alternative is to link against SSL libs within Perl's own build process
like some other languages do, which would have the potential for greater
flexibility/portability. But it means some duplication of effort, and that
it can't be upgraded from CPAN like Net::SSLeay (important for any SSL
functionality - though it could be implemented in a blead-first dual-life
module to make this possible), and that IO::Socket::SSL (probably) would
need to be updated to make use of the new functionality. And foremost, this
would need a volunteer well versed in SSL libraries.


> I love this idea and as a standalone, I support it as is, if feasible.
> However, if we do this, can we improve the overall experience of Perl?
>
> One of the issue that I think Perl has is that we our presence is almost
> entirely in sysadmin tasks and web development. I can't comment on the
> former, but the latter also requires a few more modules to make Perl more
> natively useful. If nothing else, a fast JSON tool would be welcome (not
> the JSON::PP that is in core. That's very slow)
>
> Also, some proper native readline support would be nice for a useful
> debugger. I assume Term::ReadLine::Gnu isn't bundled because of the
> licence, but my debugger is always broken when I first use it.
>

Not sure what either of these have to do with supporting HTTPS out of the
box. Perhaps suitable for other threads?

-Dan

On Tue, 23 Nov 2021 17:55:28 +0000
Neil Bowers <neilb@neilb.org> wrote:

> When you install Perl, although it comes with HTTP::Tiny, it doesn't support https because HTTP::Tiny requires IO::Socket::SSL (which isn?t bundled with Perl). This means that CPAN.pm and other tools can't make https mandatory, as when bootstrapping a new box you'll typically install perl then run cpan to first install Net::SSLeay and IO::Socket::SSL.
>
> I think we should make Perl support https "out of the box", so that CPAN.pm could make https mandatory, and other modules could then support https while relying only on core modules.
>
> One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I don't know if that's the best / right solution?
>
> One problem with bundling these modules is that we are then committed to ensuring that they work on all supported platforms / configurations. Is it ok to support https out-of-the-box on some supported platforms but not all of them? I think so.

We have a few optional core modules with external dependencies. For
example, we don't build DB_File when BerkeleyDB isn't installed. I think
it would be acceptable to do the same with the SSL modules.

In the future we might want to extend IO::Socket::SSL with additional
non-OpenSSL backends (e.g. pure-perl, SChannel for Windows), but that
shouldn't block its inclusion in core.

2021-11-24 2:55 Neil Bowers <neilb@neilb.org> wrote:

> When you install Perl, although it comes with HTTP::Tiny, it doesn't
> support https because HTTP::Tiny requires IO::Socket::SSL (which isn’t
> bundled with Perl). This means that CPAN.pm and other tools can't make
> https mandatory, as when bootstrapping a new box you'll typically install
> perl then run cpan to first install Net::SSLeay and IO::Socket::SSL.
>
> I think we should make Perl support https "out of the box", so that
> CPAN.pm could make https mandatory, and other modules could then support
> https while relying only on core modules.
>
> One approach is to bundle Net::SSLeay and IO::Socket::SSL, but I don't
> know if that's the best / right solution?
>
> One problem with bundling these modules is that we are then committed to
> ensuring that they work on all supported platforms / configurations. Is it
> ok to support https out-of-the-box on some supported platforms but not all
> of them? I think so.
>
> Neil
>

I have a question about technical matters.

Net::SSLeay is a Perl binding of the OpenSSL library.

Normally, we can install Net::SSLeay only if the OpenSSL library is already
installed and the tests of Net::SSLeay are successful.

Suppose the Perl core contains Net::SSLeay.

In this case, Net::SSLeay is already installed without the OpenSSL library
and the tests of Net::SSLeay are not yet done.

How do we test for correctness of behavior of Net::SSLeay?

Do we need a conversation with the OpenSSL developers?

On Wed, Dec 1, 2021, at 8:08 PM, Yuki Kimoto wrote:
> Suppose the Perl core contains Net::SSLeay.
>
> In this case, Net::SSLeay is already installed without the OpenSSL library and the tests of Net::SSLeay are not yet done.
>
> How do we test for correctness of behavior of Net::SSLeay?

We would only build, test, and install Net::SSLeay if openssl was available.

--
rjbs

2021-12-2 10:55 Ricardo Signes <perl.p5p@rjbs.manxome.org> wrote:

> On Wed, Dec 1, 2021, at 8:08 PM, Yuki Kimoto wrote:
>
> Suppose the Perl core contains Net::SSLeay.
>
> In this case, Net::SSLeay is already installed without the OpenSSL library
> and the tests of Net::SSLeay are not yet done.
>
> How do we test for correctness of behavior of Net::SSLeay?
>
>
> We would only build, test, and install Net::SSLeay if openssl was
> available.
>
> --
> rjbs
>

I've read the article that Perl is required to build OpenSSL.

If this is true, can OpenSSL exist before Perl is installed?

On Wed, Dec 1, 2021 at 7:55 PM Ricardo Signes <perl.p5p@rjbs.manxome.org> wrote:
>
> On Wed, Dec 1, 2021, at 8:08 PM, Yuki Kimoto wrote:
>
> Suppose the Perl core contains Net::SSLeay.
>
> In this case, Net::SSLeay is already installed without the OpenSSL library and the tests of Net::SSLeay are not yet done.
>
> How do we test for correctness of behavior of Net::SSLeay?
>
>
> We would only build, test, and install Net::SSLeay if openssl was available.

I agree this is what we would want to do, with the "extensions"
Configure variable just not including Net::SSLeay if we couldn't
locate the SSL library at Configure time. It might be a trickier case
than any we've had before, though. The Makefile.PL for Net::SSLeay:

https://github.com/radiator-software/p5-net-ssleay/blob/master/Makefile.PL

goes through all sorts of gyrations to locate the OpenSSL or LibreSSL
libraries and headers in their platform-specific hiding places, which
tend to move around with library and platform releases. It would be a
shame to have to copy and then maintain separately all of that logic
in Configure, but offhand I'm not sure how else we would do it.

Also note that Net::SSLeay requires Perl to be built with the same
compiler and options as (Open|Libre)SSL. Which would probably be
uneventful in many cases but could lead to a new flavor of bug report
in others.

Not trying to throw sand in the gears, but just trying to identify the
sand that will get in anyway.

On Thu, Dec 2, 2021 at 12:28 AM Yuki Kimoto <kimoto.yuki@gmail.com> wrote:

> I've read the article that Perl is required to build OpenSSL.
>
> If this is true, can OpenSSL exist before Perl is installed?

A previous version of Perl should be adequate to build OpenSSL, but if
you are really building everything from scratch, you would need to
build a Perl without SSL support, build OpenSSL, and then rebuild
Perl.

2021-12-2 22:41 Craig A. Berry <craig.a.berry@gmail.com> wrote:

> On Thu, Dec 2, 2021 at 12:28 AM Yuki Kimoto <kimoto.yuki@gmail.com> wrote:
>
> > I've read the article that Perl is required to build OpenSSL.
> >
> > If this is true, can OpenSSL exist before Perl is installed?
>
> A previous version of Perl should be adequate to build OpenSSL, but if
> you are really building everything from scratch, you would need to
> build a Perl without SSL support, build OpenSSL, and then rebuild
> Perl.
>

Craig

OK. thanks!

I've tried to re-read all the different threads related to this,
but Spark mail gets confused, and reading this all via nntp.perl.org
ain't user friendly either.

The motives are, at least, the following:
 - almost all APIs are https these days
 - and increasingly all websites
 - it will enable CPAN.pm to do https on a fresh install
 (it has other options for https, but those might not be available)

options:
 1. bundle modules, look for openssl (and poss other libs)
 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
 3. Go with Curl, as it can work with a range of SSL libraries
 4. work with various SSL programs (wget, curl, etc)
 5. anything else?

I suspect that the path of least resistance right now is to take
the first option:
 - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
 - Configure would look for OpenSSL or LibreSSL
 - presumably it would have to make sure it's a recent enough version
 - possibly refuse to proceed with certain versions that are now
 known to have serious security issues
 - cope with a later version of OpenSSL that's incompatible

We'd have to approach the current maintainers of those modules to
see if they're happy for them to be bundle, explaining the implications.

Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
out compatibility. One possible path is that someone familiar with
that stuff works with someone familiar with metaconfig, so that
when installed with Perl, Configure is doing the heavy lifting,
but it can still be installed from CPAN with Makefile.PL doing its thing.

This doesn't preclude people starting parallel efforts to experiment
with other approaches, but my sense is that more effort has been put
into Net::SSLeay + OpenSSL, so we should start off on those shoulders.

There are a number of pieces to this pie, so I'm guessing this will
probably need a small group of people to work on it.

I wonder if a good first step would be a proof-of-concept:
a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
b. if Net::SSLeay builds and tests successfully, then install
    Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
c. Contact the maintainers of those modules to sound them out on
    openness to dual-lifing.
d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.

Going with Curl is appealing, but unless we've got someone prepared
to take that on, it feels more complex and less well-known than the
approach outlined above.

Neil

Awesome, totally agree on the first approach.

I hope to see these three distributions in core for Perl 5.38.

These three modules are already heavily used on almost all Perl
applications as de-facto SSL standard, so why not dual-life them finally?

Usually installing the IO-Socket-SSL package pulls the two others as
dependencies on Unix systems used for HTTPS support in Perl.

Curl is PHP's approach to HTTP(S) from the beginning. It's already too late
to consider Curl as an option I believe. Because it's an alien in the Perl
world, and not as mature and integrated as the previous three.

These are my thoughts stemming from my personal experience with Perl on
Unix for many years.

Thanks.



14 iyn 2022, Ç.a. 01:11 tarixind? Neil Bowers <neilb@neilb.org> yazd?:

> I've tried to re-read all the different threads related to this,
> but Spark mail gets confused, and reading this all via nntp.perl.org
> ain't user friendly either.
>
> The motives are, at least, the following:
> - almost all APIs are https these days
> - and increasingly all websites
> - it will enable CPAN.pm to do https on a fresh install
> (it has other options for https, but those might not be available)
>
> options:
> 1. bundle modules, look for openssl (and poss other libs)
> 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> 3. Go with Curl, as it can work with a range of SSL libraries
> 4. work with various SSL programs (wget, curl, etc)
> 5. anything else?
>
> I suspect that the path of least resistance right now is to take
> the first option:
> - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> - Configure would look for OpenSSL or LibreSSL
> - presumably it would have to make sure it's a recent enough version
> - possibly refuse to proceed with certain versions that are now
> known to have serious security issues
> - cope with a later version of OpenSSL that's incompatible
>
> We'd have to approach the current maintainers of those modules to
> see if they're happy for them to be bundle, explaining the implications.
>
> Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
> out compatibility. One possible path is that someone familiar with
> that stuff works with someone familiar with metaconfig, so that
> when installed with Perl, Configure is doing the heavy lifting,
> but it can still be installed from CPAN with Makefile.PL doing its thing.
>
> This doesn't preclude people starting parallel efforts to experiment
> with other approaches, but my sense is that more effort has been put
> into Net::SSLeay + OpenSSL, so we should start off on those shoulders.
>
> There are a number of pieces to this pie, so I'm guessing this will
> probably need a small group of people to work on it.
>
> I wonder if a good first step would be a proof-of-concept:
> a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
> b. if Net::SSLeay builds and tests successfully, then install
> Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> c. Contact the maintainers of those modules to sound them out on
> openness to dual-lifing.
> d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.
>
> Going with Curl is appealing, but unless we've got someone prepared
> to take that on, it feels more complex and less well-known than the
> approach outlined above.
>
> Neil
>

On 6/13/22 17:11, Neil Bowers wrote:
> options:
> 1. bundle modules, look for openssl (and poss other libs)
> 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> 3. Go with Curl, as it can work with a range of SSL libraries
> 4. work with various SSL programs (wget, curl, etc)
> 5. anything else?
>
> I suspect that the path of least resistance right now is to take
> the first option:
> - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> - Configure would look for OpenSSL or LibreSSL
> - presumably it would have to make sure it's a recent enough version
> - possibly refuse to proceed with certain versions that are now
> known to have serious security issues
> - cope with a later version of OpenSSL that's incompatible
>
> We'd have to approach the current maintainers of those modules to
> see if they're happy for them to be bundle, explaining the implications.

+1 for this approach, it keeps the ball rolling.


I would like to add that the Docker perl image now supports HTTPS by
default and also ships with cpm.

Cheers,
Wesley


--
Wesley Schwengle

Dual-life of Mozilla::CA seems problematic for the same reason as bundling a TLS library: the need for updates.

Most OpenSSL installs seem to include a root bundle; could we defer to that instead?

A short C snippet that shows how to get that directory:
https://stackoverflow.com/a/4141173/586723

-FG


> On Jun 13, 2022, at 19:56, Elvin Aslanov <rwp.primary@gmail.com> wrote:
>
> Awesome, totally agree on the first approach.
>
> I hope to see these three distributions in core for Perl 5.38.
>
> These three modules are already heavily used on almost all Perl applications as de-facto SSL standard, so why not dual-life them finally?
>
> Usually installing the IO-Socket-SSL package pulls the two others as dependencies on Unix systems used for HTTPS support in Perl.
>
> Curl is PHP's approach to HTTP(S) from the beginning. It's already too late to consider Curl as an option I believe. Because it's an alien in the Perl world, and not as mature and integrated as the previous three.
>
> These are my thoughts stemming from my personal experience with Perl on Unix for many years.
>
> Thanks.
>
>
>
> 14 iyn 2022, Ç.a. 01:11 tarixind? Neil Bowers <neilb@neilb.org> yazd?:
> I've tried to re-read all the different threads related to this,
> but Spark mail gets confused, and reading this all via nntp.perl.org
> ain't user friendly either.
>
> The motives are, at least, the following:
> - almost all APIs are https these days
> - and increasingly all websites
> - it will enable CPAN.pm to do https on a fresh install
> (it has other options for https, but those might not be available)
>
> options:
> 1. bundle modules, look for openssl (and poss other libs)
> 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> 3. Go with Curl, as it can work with a range of SSL libraries
> 4. work with various SSL programs (wget, curl, etc)
> 5. anything else?
>
> I suspect that the path of least resistance right now is to take
> the first option:
> - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> - Configure would look for OpenSSL or LibreSSL
> - presumably it would have to make sure it's a recent enough version
> - possibly refuse to proceed with certain versions that are now
> known to have serious security issues
> - cope with a later version of OpenSSL that's incompatible
>
> We'd have to approach the current maintainers of those modules to
> see if they're happy for them to be bundle, explaining the implications.
>
> Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
> out compatibility. One possible path is that someone familiar with
> that stuff works with someone familiar with metaconfig, so that
> when installed with Perl, Configure is doing the heavy lifting,
> but it can still be installed from CPAN with Makefile.PL doing its thing.
>
> This doesn't preclude people starting parallel efforts to experiment
> with other approaches, but my sense is that more effort has been put
> into Net::SSLeay + OpenSSL, so we should start off on those shoulders.
>
> There are a number of pieces to this pie, so I'm guessing this will
> probably need a small group of people to work on it.
>
> I wonder if a good first step would be a proof-of-concept:
> a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
> b. if Net::SSLeay builds and tests successfully, then install
> Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> c. Contact the maintainers of those modules to sound them out on
> openness to dual-lifing.
> d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.
>
> Going with Curl is appealing, but unless we've got someone prepared
> to take that on, it feels more complex and less well-known than the
> approach outlined above.
>
> Neil

yeah but `cpan Mozilla::CA` isn't hard to do to update the module and it
won't break with newer Perl versions as well since it's just plaintext
non-code certificates bundle

On Tue, Jun 14, 2022 at 7:52 PM Felipe Gasper <felipe@felipegasper.com>
wrote:

> Dual-life of Mozilla::CA seems problematic for the same reason as bundling
> a TLS library: the need for updates.
>
> Most OpenSSL installs seem to include a root bundle; could we defer to
> that instead?
>
> A short C snippet that shows how to get that directory:
> https://stackoverflow.com/a/4141173/586723
>
> -FG
>
>
> > On Jun 13, 2022, at 19:56, Elvin Aslanov <rwp.primary@gmail.com> wrote:
> >
> > Awesome, totally agree on the first approach.
> >
> > I hope to see these three distributions in core for Perl 5.38.
> >
> > These three modules are already heavily used on almost all Perl
> applications as de-facto SSL standard, so why not dual-life them finally?
> >
> > Usually installing the IO-Socket-SSL package pulls the two others as
> dependencies on Unix systems used for HTTPS support in Perl.
> >
> > Curl is PHP's approach to HTTP(S) from the beginning. It's already too
> late to consider Curl as an option I believe. Because it's an alien in the
> Perl world, and not as mature and integrated as the previous three.
> >
> > These are my thoughts stemming from my personal experience with Perl on
> Unix for many years.
> >
> > Thanks.
> >
> >
> >
> > 14 iyn 2022, Ç.a. 01:11 tarixind? Neil Bowers <neilb@neilb.org> yazd?:
> > I've tried to re-read all the different threads related to this,
> > but Spark mail gets confused, and reading this all via nntp.perl.org
> > ain't user friendly either.
> >
> > The motives are, at least, the following:
> > - almost all APIs are https these days
> > - and increasingly all websites
> > - it will enable CPAN.pm to do https on a fresh install
> > (it has other options for https, but those might not be available)
> >
> > options:
> > 1. bundle modules, look for openssl (and poss other libs)
> > 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> > 3. Go with Curl, as it can work with a range of SSL libraries
> > 4. work with various SSL programs (wget, curl, etc)
> > 5. anything else?
> >
> > I suspect that the path of least resistance right now is to take
> > the first option:
> > - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> > - Configure would look for OpenSSL or LibreSSL
> > - presumably it would have to make sure it's a recent enough version
> > - possibly refuse to proceed with certain versions that are now
> > known to have serious security issues
> > - cope with a later version of OpenSSL that's incompatible
> >
> > We'd have to approach the current maintainers of those modules to
> > see if they're happy for them to be bundle, explaining the implications.
> >
> > Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
> > out compatibility. One possible path is that someone familiar with
> > that stuff works with someone familiar with metaconfig, so that
> > when installed with Perl, Configure is doing the heavy lifting,
> > but it can still be installed from CPAN with Makefile.PL doing its thing.
> >
> > This doesn't preclude people starting parallel efforts to experiment
> > with other approaches, but my sense is that more effort has been put
> > into Net::SSLeay + OpenSSL, so we should start off on those shoulders.
> >
> > There are a number of pieces to this pie, so I'm guessing this will
> > probably need a small group of people to work on it.
> >
> > I wonder if a good first step would be a proof-of-concept:
> > a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
> > b. if Net::SSLeay builds and tests successfully, then install
> > Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> > c. Contact the maintainers of those modules to sound them out on
> > openness to dual-lifing.
> > d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.
> >
> > Going with Curl is appealing, but unless we've got someone prepared
> > to take that on, it feels more complex and less well-known than the
> > approach outlined above.
> >
> > Neil
>
>
>

If we’re OK with that then we could ship mbedTLS and have TLS always “just work”.

I’ve been under the impression that’s unacceptable to depend on users to update TLS stuff manually, be it the root store or the code.

-FG

> On Jun 14, 2022, at 12:18, Elvin Aslanov <rwp.primary@gmail.com> wrote:
>
> yeah but `cpan Mozilla::CA` isn't hard to do to update the module and it won't break with newer Perl versions as well since it's just plaintext non-code certificates bundle
>
> On Tue, Jun 14, 2022 at 7:52 PM Felipe Gasper <felipe@felipegasper.com> wrote:
> Dual-life of Mozilla::CA seems problematic for the same reason as bundling a TLS library: the need for updates.
>
> Most OpenSSL installs seem to include a root bundle; could we defer to that instead?
>
> A short C snippet that shows how to get that directory:
> https://stackoverflow.com/a/4141173/586723
>
> -FG
>
>
> > On Jun 13, 2022, at 19:56, Elvin Aslanov <rwp.primary@gmail.com> wrote:
> >
> > Awesome, totally agree on the first approach.
> >
> > I hope to see these three distributions in core for Perl 5.38.
> >
> > These three modules are already heavily used on almost all Perl applications as de-facto SSL standard, so why not dual-life them finally?
> >
> > Usually installing the IO-Socket-SSL package pulls the two others as dependencies on Unix systems used for HTTPS support in Perl.
> >
> > Curl is PHP's approach to HTTP(S) from the beginning. It's already too late to consider Curl as an option I believe. Because it's an alien in the Perl world, and not as mature and integrated as the previous three.
> >
> > These are my thoughts stemming from my personal experience with Perl on Unix for many years.
> >
> > Thanks.
> >
> >
> >
> > 14 iyn 2022, Ç.a. 01:11 tarixind? Neil Bowers <neilb@neilb.org> yazd?:
> > I've tried to re-read all the different threads related to this,
> > but Spark mail gets confused, and reading this all via nntp.perl.org
> > ain't user friendly either.
> >
> > The motives are, at least, the following:
> > - almost all APIs are https these days
> > - and increasingly all websites
> > - it will enable CPAN.pm to do https on a fresh install
> > (it has other options for https, but those might not be available)
> >
> > options:
> > 1. bundle modules, look for openssl (and poss other libs)
> > 2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> > 3. Go with Curl, as it can work with a range of SSL libraries
> > 4. work with various SSL programs (wget, curl, etc)
> > 5. anything else?
> >
> > I suspect that the path of least resistance right now is to take
> > the first option:
> > - dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> > - Configure would look for OpenSSL or LibreSSL
> > - presumably it would have to make sure it's a recent enough version
> > - possibly refuse to proceed with certain versions that are now
> > known to have serious security issues
> > - cope with a later version of OpenSSL that's incompatible
> >
> > We'd have to approach the current maintainers of those modules to
> > see if they're happy for them to be bundle, explaining the implications.
> >
> > Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
> > out compatibility. One possible path is that someone familiar with
> > that stuff works with someone familiar with metaconfig, so that
> > when installed with Perl, Configure is doing the heavy lifting,
> > but it can still be installed from CPAN with Makefile.PL doing its thing.
> >
> > This doesn't preclude people starting parallel efforts to experiment
> > with other approaches, but my sense is that more effort has been put
> > into Net::SSLeay + OpenSSL, so we should start off on those shoulders.
> >
> > There are a number of pieces to this pie, so I'm guessing this will
> > probably need a small group of people to work on it.
> >
> > I wonder if a good first step would be a proof-of-concept:
> > a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
> > b. if Net::SSLeay builds and tests successfully, then install
> > Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> > c. Contact the maintainers of those modules to sound them out on
> > openness to dual-lifing.
> > d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.
> >
> > Going with Curl is appealing, but unless we've got someone prepared
> > to take that on, it feels more complex and less well-known than the
> > approach outlined above.
> >
> > Neil
>
>

* Neil Bowers <neilb@neilb.org> [2022-06-13 22:11:11 +0100]:

> I've tried to re-read all the different threads related to this,
> but Spark mail gets confused, and reading this all via?nntp.perl.org
> ain't user friendly either.
>
> The motives are, at least, the following:
> ?- almost all APIs are https these days
> ?- and increasingly all websites
> ?- it will enable CPAN.pm to do https on a fresh install
> ?(it has other options for https, but those might not be available)
>
> options:
> ?1. bundle modules, look for openssl (and poss other libs)
> ?2. bundle an SSL lib and modules such as Mbed TLS, WolfSSL
> ?3. Go with Curl, as it can work with a range of SSL libraries
> ?4. work with various SSL programs (wget, curl, etc)
> ?5. anything else?

As a user, concur that #1 seems to be the most reasonable, with the caveat
that IO::Socket::SSL provided by MAJOR packages be factored in somehow. I suspect
distros like Debian/Ubuntu would necessarily want to strip this out somehow
given they strip out things like perldoc into separate packages. But, knowing
that my opinion means nothing, I'd place much more emphasis on opinions pkg folks
from Debian, OpenBSD, FreeBSD, and pkgsrc (NetBSD), etc (I just can't name them
all out of ignorance other wise I would have). So my only request is to please
approach them somehow and get their opinions. I am thankful for the many and robust
distribution channels perl has, and I just think their opinions matter more
than (e.g.,) mine. And I think a lot of these problems are well understood,
if not solved outright.

Cheers,
Brett

>
> I suspect that the path of least resistance right now is to take
> the first option:
> ?- dual-life Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> ?- Configure would look for OpenSSL or LibreSSL
> ?- presumably it would have to make sure it's a recent enough version
> ?- possibly refuse to proceed with certain versions that are now
> ?known to have serious security issues
> ?- cope with a later version of OpenSSL that's incompatible
>
> We'd have to approach the current maintainers of those modules to
> see if they're happy for them to be bundle, explaining the implications.
>
> Net-SSLeay's Makefile.PL has stuff to to look for OpenSSL and work
> out compatibility. One possible path is that someone familiar with
> that stuff works with someone familiar with metaconfig, so that
> when installed with Perl, Configure is doing the heavy lifting,
> but it can still be installed from CPAN with Makefile.PL doing its thing.
>
> This doesn't preclude people starting parallel efforts to experiment
> with other approaches, but my sense is that more effort has been put
> into Net::SSLeay + OpenSSL, so we should start off on those shoulders.
>
> There are a number of pieces to this pie, so I'm guessing this will
> probably need a small group of people to work on it.
>
> I wonder if a good first step would be a proof-of-concept:
> a. hack together a metaconfig unit based on Net::SSLeay's Makefile.PL
> b. if Net::SSLeay builds and tests successfully, then install
> ?? ?Net::SSLeay, IO::Socket::SSL, and Mozilla::CA
> c. Contact the maintainers of those modules to sound them out on
> ?? ?openness to dual-lifing.
> d. See if this can be made to work on Linux, *BSD, MacOS, and Windows.
>
> Going with Curl is appealing, but unless we've got someone prepared
> to take that on, it feels more complex and less well-known than the
> approach outlined above.
>
> Neil

--
--
oodler@cpan.org
oodler577@sdf-eu.org
SDF-EU Public Access UNIX System - http://sdfeu.org
irc.perl.org #openmp #pdl #native

On Tue, Jun 14, 2022 at 6:18 PM Elvin Aslanov <rwp.primary@gmail.com> wrote:

> yeah but `cpan Mozilla::CA` isn't hard to do to update the module and it
> won't break with newer Perl versions as well since it's just plaintext
> non-code certificates bundle
>

I'd prefer if the stack used the OS trusted CAs by default instead of
having its own list.
This should only be the default and overrideable for private CA use-cases.

On Wed, Jun 15, 2022 at 2:23 AM Alexander Hartmaier
<alex.hartmaier@gmail.com> wrote:
>
> On Tue, Jun 14, 2022 at 6:18 PM Elvin Aslanov <rwp.primary@gmail.com> wrote:
>>
>> yeah but `cpan Mozilla::CA` isn't hard to do to update the module and it won't break with newer Perl versions as well since it's just plaintext non-code certificates bundle
>
>
> I'd prefer if the stack used the OS trusted CAs by default instead of having its own list.
> This should only be the default and overrideable for private CA use-cases.

But that is a *massively* more difficult portability problem than just
"where do I find OpenSSL or LibreSSL?". Do you know where the OS
trusted CAs are for every platform and distribution on which Perl
runs? Or if there even is such a thing as an "OS trusted CA" on all of
them? Or what format they are in? Or whether they even exist on the
filesystem or are in some proprietary data store?

> On Jun 15, 2022, at 18:22, Craig A. Berry <craig.a.berry@gmail.com> wrote:
>
> On Wed, Jun 15, 2022 at 2:23 AM Alexander Hartmaier
> <alex.hartmaier@gmail.com> wrote:
>>
>> On Tue, Jun 14, 2022 at 6:18 PM Elvin Aslanov <rwp.primary@gmail.com> wrote:
>>>
>>> yeah but `cpan Mozilla::CA` isn't hard to do to update the module and it won't break with newer Perl versions as well since it's just plaintext non-code certificates bundle
>>
>>
>> I'd prefer if the stack used the OS trusted CAs by default instead of having its own list.
>> This should only be the default and overrideable for private CA use-cases.
>
> But that is a *massively* more difficult portability problem than just
> "where do I find OpenSSL or LibreSSL?". Do you know where the OS
> trusted CAs are for every platform and distribution on which Perl
> runs? Or if there even is such a thing as an "OS trusted CA" on all of
> them? Or what format they are in? Or whether they even exist on the
> filesystem or are in some proprietary data store?

Clarification: by “OS-trusted CAs” I believe Alexander refers specifically to OpenSSL’s roots, which are easily discoverable via the mechanism I mentioned earlier in this thread.

The idea is: if there’s OpenSSL, use it; if not, no out-of-the-box TLS.

-FG