Mailing List Archive

Occasionally heartbeat doesn't start...
Hi,

I've been investigating an HA configuration for a customer. One time in
testing heartbeat didn't start, because rpcbind had stolen its reserved
port. Restarting rpcbind made it choose a different random port. This
is definitely an interesting problem - even if it doesn't happen very often.

The best solution to this, AFAIK is to make a file
/etc/portreserve/heartbeat with this one line in it:
694/udp

and then add portrelease heartbeat to the init script.

Thoughts?

--
Alan Robertson<alanr@unix.sh> - @OSSAlanR

"Openness is the foundation and preservative of friendship... Let me claim from you at all times your undisguised opinions." - William Wilberforce
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: Occasionally heartbeat doesn't start... [ In reply to ]
On Fri, Mar 09, 2012 at 11:52:56AM -0700, Alan Robertson wrote:
> Hi,
>
> I've been investigating an HA configuration for a customer. One
> time in testing heartbeat didn't start, because rpcbind had stolen
> its reserved port. Restarting rpcbind made it choose a different
> random port. This is definitely an interesting problem - even if it
> doesn't happen very often.
>
> The best solution to this, AFAIK is to make a file
> /etc/portreserve/heartbeat with this one line in it:
> 694/udp
>
> and then add portrelease heartbeat to the init script.

"rpcbind" used to be "portmap".

You would need the portreserve daemon available, installed,
and started at the right time during your boot sequence.
So that's only a hackish workaround.

On Debian (Ubuntu, other derivatives) you'd simply add a line
to /etc/bindresvport.blacklist. But that may fail as well,
there have been reports where this was ignored for some reason.
So that again is just a workaround.

If you know exactly what will register with portmap (rpcbind),
you can tell those services to request fixed ports instead.

Typically you do, and those are just a few nfs related services.
So just edit /etc/sysconfig/* or /etc/defaults/*
to e.g. include -o and -p options for rpc.statd, and similar.

This really is a fix, as long as you know all services
that are started before heartbeat, and can tell them
to use specific ports.

--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: Occasionally heartbeat doesn't start... [ In reply to ]
In this case, it was actually rpcbind that grabbed the port. AFAIK,
there is no way to tell it to use or not use a particular port - except
to grab it first. That's what portreserve does. If it is run first,
and given the right config files, it _will_ keep anyone else from using
that port.

It makes sense to put port 694 in /etc/portreserve/heartbeat as part of
our package and include that invocation.

If someone chooses a different port they can always edit that file.

Redhat provides portreserve and starts it by default before rpcbind.

If other distros don't provide it or use it - no harm comes from
installing the file and attempting to run portrelease.

But for those that provide it, it is a help.



On 03/12/2012 05:43 AM, Lars Ellenberg wrote:
> On Fri, Mar 09, 2012 at 11:52:56AM -0700, Alan Robertson wrote:
>> Hi,
>>
>> I've been investigating an HA configuration for a customer. One
>> time in testing heartbeat didn't start, because rpcbind had stolen
>> its reserved port. Restarting rpcbind made it choose a different
>> random port. This is definitely an interesting problem - even if it
>> doesn't happen very often.
>>
>> The best solution to this, AFAIK is to make a file
>> /etc/portreserve/heartbeat with this one line in it:
>> 694/udp
>>
>> and then add portrelease heartbeat to the init script.
> "rpcbind" used to be "portmap".
>
> You would need the portreserve daemon available, installed,
> and started at the right time during your boot sequence.
> So that's only a hackish workaround.
>
> On Debian (Ubuntu, other derivatives) you'd simply add a line
> to /etc/bindresvport.blacklist. But that may fail as well,
> there have been reports where this was ignored for some reason.
> So that again is just a workaround.
>
> If you know exactly what will register with portmap (rpcbind),
> you can tell those services to request fixed ports instead.
>
> Typically you do, and those are just a few nfs related services.
> So just edit /etc/sysconfig/* or /etc/defaults/*
> to e.g. include -o and -p options for rpc.statd, and similar.
>
> This really is a fix, as long as you know all services
> that are started before heartbeat, and can tell them
> to use specific ports.
>


--
Alan Robertson<alanr@unix.sh> - @OSSAlanR

"Openness is the foundation and preservative of friendship... Let me claim from you at all times your undisguised opinions." - William Wilberforce
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/