Mailing List Archive: Fwd: ldirectord fails to test HTTPS real servers.

Fwd: ldirectord fails to test HTTPS real servers.

Dec 3, 2013, 3:57 AM

Post #1 of 5 (3889 views)

Hi guys!

I've posted bug report regarding ldirectord, can you please review it and
commit, if possible?

https://github.com/ClusterLabs/resource-agents/issues/361

Ldirectord is using LWP for it's negotiate checks for the HTTP/HTTPS sites.
Since LWP 6.0 by default it verifies the correspondence of the SSL
certificate and the server hostname. In 99.9% of the cases this is the VIP
hostname and RIP are identified by their internal hostnames or, most common
- by their IP addresses.

That breaks hostname verification and hence - marks HTTPS backends as
invalid and kicks them off the pool. This problem did hit me in the
production when we've upgraded from Debian squeeze to Debian wheezy, which
brought newer version of LWP.

http://search.cpan.org/~gaas/LWP-Protocol-https-6.04/lib/LWP/Protocol/https.pm

Luckily, the fix to the problem is easy:

--- ldirectord.orig 2013-12-03 11:59:11.114983525 +0100
+++ ldirectord 2013-12-03 11:59:34.703026282 +0100
@@ -2834,7 +2834,7 @@
&ld_debug(2, "check_http: url=\"$$r{url}\" "
. "virtualhost=\"$virtualhost\"");

- my $ua = new LWP::UserAgent();
+ my $ua = new LWP::UserAgent(ssl_opts => { verify_hostname => 0 });

my $h = undef;
if ($$v{service} eq "http_proxy") {

I haven't verified that with older version of LWP, but I believe it should
just ignore unknown parameters to the constructor.

With best regards,
Timur Bakeyev.

Re: Fwd: ldirectord fails to test HTTPS real servers. [ In reply to ]

lars.ellenberg at linbit

Dec 13, 2013, 4:37 AM

Post #2 of 5 (3761 views)

Permalink

On Tue, Dec 03, 2013 at 12:57:24PM +0100, Timur I. Bakeyev wrote:
> Hi guys!
>
> I've posted bug report regarding ldirectord, can you please review it and
> commit, if possible?
>
> https://github.com/ClusterLabs/resource-agents/issues/361
>
> Ldirectord is using LWP for it's negotiate checks for the HTTP/HTTPS sites.
> Since LWP 6.0 by default it verifies the correspondence of the SSL
> certificate and the server hostname. In 99.9% of the cases this is the VIP
> hostname and RIP are identified by their internal hostnames or, most common
> - by their IP addresses.
>
> That breaks hostname verification and hence - marks HTTPS backends as
> invalid and kicks them off the pool. This problem did hit me in the
> production when we've upgraded from Debian squeeze to Debian wheezy, which
> brought newer version of LWP.
>
> http://search.cpan.org/~gaas/LWP-Protocol-https-6.04/lib/LWP/Protocol/https.pm
>
> Luckily, the fix to the problem is easy:
>
> --- ldirectord.orig 2013-12-03 11:59:11.114983525 +0100
> +++ ldirectord 2013-12-03 11:59:34.703026282 +0100
> @@ -2834,7 +2834,7 @@
> &ld_debug(2, "check_http: url=\"$$r{url}\" "
> . "virtualhost=\"$virtualhost\"");
>
> - my $ua = new LWP::UserAgent();
> + my $ua = new LWP::UserAgent(ssl_opts => { verify_hostname => 0 });
>
> my $h = undef;
> if ($$v{service} eq "http_proxy") {
>
> I haven't verified that with older version of LWP, but I believe it should
> just ignore unknown parameters to the constructor.

It does ignore, but warn, which will cause all sorts of complaints, I guess.

The more recent LWP::UserAgent states:

"verify_hostname" => $bool
When TRUE LWP will for secure protocol schemes ensure it connects
to servers that have a valid certificate matching the expected
hostname. If FALSE no checks are made and you can't be sure that
you communicate with the expected peer. The no checks behaviour
was the default for libwww-perl-5.837 and earlier releases.

This option is initialized from the PERL_LWP_SSL_VERIFY_HOSTNAME
environment variable. If this environment variable isn't set;
then "verify_hostname" defaults to 1.

So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere
is the more elegant solution?

Lars

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/

Re: Fwd: ldirectord fails to test HTTPS real servers. [ In reply to ]

timur at com

Dec 20, 2013, 3:31 AM

Post #3 of 5 (3757 views)

Permalink

On Fri, Dec 13, 2013 at 1:37 PM, Lars Ellenberg
<lars.ellenberg@linbit.com>wrote:

> On Tue, Dec 03, 2013 at 12:57:24PM +0100, Timur I. Bakeyev wrote:
> > Hi guys!
> >
> > I've posted bug report regarding ldirectord, can you please review it and
> > commit, if possible?
> >
> > https://github.com/ClusterLabs/resource-agents/issues/361
> >
> > Ldirectord is using LWP for it's negotiate checks for the HTTP/HTTPS
> sites.
> > Since LWP 6.0 by default it verifies the correspondence of the SSL
> > certificate and the server hostname. In 99.9% of the cases this is the
> VIP
> > hostname and RIP are identified by their internal hostnames or, most
> common
> > - by their IP addresses.
> >
> > That breaks hostname verification and hence - marks HTTPS backends as
> > invalid and kicks them off the pool. This problem did hit me in the
> > production when we've upgraded from Debian squeeze to Debian wheezy,
> which
> > brought newer version of LWP.
> >
> >
> http://search.cpan.org/~gaas/LWP-Protocol-https-6.04/lib/LWP/Protocol/https.pm
> >
> > Luckily, the fix to the problem is easy:
> >
> > --- ldirectord.orig 2013-12-03 11:59:11.114983525 +0100
> > +++ ldirectord 2013-12-03 11:59:34.703026282 +0100
> > @@ -2834,7 +2834,7 @@
> > &ld_debug(2, "check_http: url=\"$$r{url}\" "
> > . "virtualhost=\"$virtualhost\"");
> >
> > - my $ua = new LWP::UserAgent();
> > + my $ua = new LWP::UserAgent(ssl_opts => { verify_hostname => 0
> });
> >
> > my $h = undef;
> > if ($$v{service} eq "http_proxy") {
> >
> > I haven't verified that with older version of LWP, but I believe it
> should
> > just ignore unknown parameters to the constructor.
>
> It does ignore, but warn, which will cause all sorts of complaints, I
> guess.
>
> The more recent LWP::UserAgent states:
>
> "verify_hostname" => $bool
> When TRUE LWP will for secure protocol schemes ensure it connects
> to servers that have a valid certificate matching the expected
> hostname. If FALSE no checks are made and you can't be sure that
> you communicate with the expected peer. The no checks behaviour
> was the default for libwww-perl-5.837 and earlier releases.
>
> This option is initialized from the PERL_LWP_SSL_VERIFY_HOSTNAME
> environment variable. If this environment variable isn't set;
> then "verify_hostname" defaults to 1.
>
> So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere
> is the more elegant solution?
>

That was my original idea, but playing with ENV variables within ldirectord
sounds a bit dangerous. We can play with $ua->ssl_opts( $key => $value );
instead.

Not sure, if it exists in the previous versions of LWP, so, possibly, have
to be verified with 'can' call.

Also, it seems, that similar code was committed to the repository 2 months
ago:
https://github.com/ClusterLabs/resource-agents/commit/b4bb7444455ec4fcfb2c5900282d81f9fa88bf86

With best regards,
Timur Bakeyev.

Re: Fwd: ldirectord fails to test HTTPS real servers. [ In reply to ]

lars.ellenberg at linbit

Dec 23, 2013, 5:04 AM

Post #4 of 5 (3750 views)

Permalink

On Fri, Dec 20, 2013 at 12:31:05PM +0100, Timur I. Bakeyev wrote:
> > So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere
> > is the more elegant solution?
> >
>
>
> That was my original idea, but playing with ENV variables within ldirectord
> sounds a bit dangerous. We can play with $ua->ssl_opts( $key => $value );
> instead.
>
> Not sure, if it exists in the previous versions of LWP, so, possibly, have
> to be verified with 'can' call.
>
> Also, it seems, that similar code was committed to the repository 2 months
> ago:
> https://github.com/ClusterLabs/resource-agents/commit/b4bb7444455ec4fcfb2c5900282d81f9fa88bf86

In that case, your issue 361 does no longer apply,
because already fixed, right?

And we could wait for the next issue complaining about too noisy
warnings on older LWP versions.

Lars
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/

Re: Fwd: ldirectord fails to test HTTPS real servers. [ In reply to ]

timur at com

Dec 23, 2013, 5:47 AM

Post #5 of 5 (3753 views)

Permalink

On Mon, Dec 23, 2013 at 2:04 PM, Lars Ellenberg
<lars.ellenberg@linbit.com>wrote:

> On Fri, Dec 20, 2013 at 12:31:05PM +0100, Timur I. Bakeyev wrote:
> > > So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere
> > > is the more elegant solution?
> > >
> >
> >
> > That was my original idea, but playing with ENV variables within
> ldirectord
> > sounds a bit dangerous. We can play with $ua->ssl_opts( $key => $value );
> > instead.
> >
> > Not sure, if it exists in the previous versions of LWP, so, possibly,
> have
> > to be verified with 'can' call.
> >
> > Also, it seems, that similar code was committed to the repository 2
> months
> > ago:
> >
> https://github.com/ClusterLabs/resource-agents/commit/b4bb7444455ec4fcfb2c5900282d81f9fa88bf86
>
> In that case, your issue 361 does no longer apply,
> because already fixed, right?
>

I guess so. Thanks to Mori-san for pointing onto this.

>
> And we could wait for the next issue complaining about too noisy
> warnings on older LWP versions.
>

Let's hope they'll upgrade :)