Mailing List Archive: be warned: emerge -u world: SSH login fails:hard restart after system update

be warned: emerge -u world: SSH login fails:hard restart after system update - keep your console open to fix

Oct 20, 2004, 9:30 AM

Post #1 of 4 (485 views)

Hi

I would like to notify gentoo system administrators. I do not know
what cause this, but I have 3 examples from my remote servers,
when latest update to system (emerge -up world) results in
SSH problems.

Easy fix (run this under 'root' user):

(killall sshd ; killall -9 sshd ; /etc/init.d/sshd zap ;
/etc/init.d/sshd start)

Simple restart of sshd /etc/init.d/sshd restart - does not help because
old SSH
server works in background.

Please be warned, especially administrators of remote serves - if you skip
this step, SSH server will be kept in background, but refusing any users (
after all keys or passwords it just fails with Connection Closed error).

This happend on all 3(!) servers where I made updates, so I belive this is
common. Now, before upgrading system I am always keeping one console
to remote server open, in case of such problems.

Here is list of packages which definitely result in SSH error (if somebody
points out which exactly package update causes this problem?):

1098263971: >>> emerge (1 of 13) sys-libs/cracklib-2.7-r9 to /
1098263988: >>> emerge (2 of 13) sys-devel/autoconf-2.59-r5 to /
1098264010: >>> emerge (3 of 13) net-www/apache-2.0.52 to /
1098264374: >>> emerge (4 of 13) sys-apps/baselayout-1.9.4-r5 to /
1098264395: >>> emerge (5 of 13) sys-libs/glibc-2.3.4.20040808-r1 to /
1098266195: >>> emerge (6 of 13) sys-devel/make-3.80-r1 to /
1098266227: >>> emerge (7 of 13) sys-fs/reiserfsprogs-3.6.18 to /
1098266264: >>> emerge (8 of 13) sys-apps/shadow-4.0.4.1-r4 to /
1098266341: >>> emerge (9 of 13) dev-lang/python-2.3.4 to /
1098266539: >>> emerge (10 of 13) dev-util/subversion-1.0.9 to /
1098266814: >>> emerge (11 of 13) sys-apps/net-tools-1.60-r9 to /
1098266840: >>> emerge (12 of 13) net-misc/whois-4.6.13-r1 to /
1098266852: >>> emerge (13 of 13)
sys-kernel/gentoo-dev-sources-2.6.8-r10 to /

P.S. After this update I had to call my colocation provider asking them
to reboot server.

Regards.

Alex

--
gentoo-user@gentoo.org mailing list

Re: be warned: emerge -u world: SSH login fails:hard restart after system update - keep your console open to fix [ In reply to ]

jgonzalez at opentechnet

Oct 20, 2004, 9:42 AM

Post #2 of 4 (488 views)

Permalink

Alex V. Koval wrote:

> Hi
>
> I would like to notify gentoo system administrators. I do not know
> what cause this, but I have 3 examples from my remote servers,
> when latest update to system (emerge -up world) results in
> SSH problems.
>
> Easy fix (run this under 'root' user):
>
> (killall sshd ; killall -9 sshd ; /etc/init.d/sshd zap ;
> /etc/init.d/sshd start)
>
> Simple restart of sshd /etc/init.d/sshd restart - does not help
> because old SSH
> server works in background.

Well, I know most of us don't do it, but whenever upgrading a
service (or any package a service depends on) we should stop the service
and restart it after the upgrade has finished. In extreme cases I forgot
to stop a service I was deinstalling and after that I had to stop the
service killing it.

>
> Please be warned, especially administrators of remote serves - if you
> skip
> this step, SSH server will be kept in background, but refusing any
> users (
> after all keys or passwords it just fails with Connection Closed error).
>
> This happend on all 3(!) servers where I made updates, so I belive
> this is
> common. Now, before upgrading system I am always keeping one console
> to remote server open, in case of such problems.
>
> Here is list of packages which definitely result in SSH error (if
> somebody
> points out which exactly package update causes this problem?):
>
> 1098263971: >>> emerge (1 of 13) sys-libs/cracklib-2.7-r9 to /
> 1098263988: >>> emerge (2 of 13) sys-devel/autoconf-2.59-r5 to /
> 1098264010: >>> emerge (3 of 13) net-www/apache-2.0.52 to /
> 1098264374: >>> emerge (4 of 13) sys-apps/baselayout-1.9.4-r5 to /
> 1098264395: >>> emerge (5 of 13) sys-libs/glibc-2.3.4.20040808-r1 to /
> 1098266195: >>> emerge (6 of 13) sys-devel/make-3.80-r1 to /
> 1098266227: >>> emerge (7 of 13) sys-fs/reiserfsprogs-3.6.18 to /
> 1098266264: >>> emerge (8 of 13) sys-apps/shadow-4.0.4.1-r4 to /
> 1098266341: >>> emerge (9 of 13) dev-lang/python-2.3.4 to /
> 1098266539: >>> emerge (10 of 13) dev-util/subversion-1.0.9 to /
> 1098266814: >>> emerge (11 of 13) sys-apps/net-tools-1.60-r9 to /
> 1098266840: >>> emerge (12 of 13) net-misc/whois-4.6.13-r1 to /
> 1098266852: >>> emerge (13 of 13)
> sys-kernel/gentoo-dev-sources-2.6.8-r10 to /

I would bet on cracklib or shadow

>
> P.S. After this update I had to call my colocation provider asking them
> to reboot server.
>
> Regards.
>
> Alex
>
Best regards
Jose

--
gentoo-user@gentoo.org mailing list

Re: be warned: emerge -u world: SSH login fails:hard restart after system update - keep your console open to fix [ In reply to ]

alex at halogen-dg

Oct 20, 2004, 10:17 AM

Post #3 of 4 (480 views)

Permalink

Hi

Jose Gonzalez Gomez wrote:

>> Easy fix (run this under 'root' user):
>>
>> (killall sshd ; killall -9 sshd ; /etc/init.d/sshd zap ;
>> /etc/init.d/sshd start)
> Well, I know most of us don't do it, but whenever upgrading a
> service (or any package a service depends on) we should stop the service
> and restart it after the upgrade has finished.

Yes, thats right. But with SSH there are 2 problems:

1. /etc/init.d/sshd restart - Does not work after this particular update
(old process hangs, it occupied the port, so ssh server does not exit, as
result new one did not get started)

2. and of course, if you are logged via SSH most of people will be afraid
to restart it.

I know, that for 2 servers this helped me:
(killall sshd ; killall -9 sshd ; /etc/init.d/sshd zap ; /etc/init.d/sshd
start) &

In 1 case I forget, as result I had to call my colocation provider.

Alex

--
gentoo-user@gentoo.org mailing list

Re: be warned: emerge -u world: SSH login fails:hard restart after system update - keep your console open to fix [ In reply to ]

kalin at thinrope

Oct 20, 2004, 12:55 PM

Post #4 of 4 (476 views)

Permalink

Alex V. Koval wrote:
> Hi
>
> I would like to notify gentoo system administrators. I do not know
> what cause this, but I have 3 examples from my remote servers,
> when latest update to system (emerge -up world) results in
> SSH problems.
>
> Easy fix (run this under 'root' user):
>
> (killall sshd ; killall -9 sshd ; /etc/init.d/sshd zap ;
> /etc/init.d/sshd start)
>
> Simple restart of sshd /etc/init.d/sshd restart - does not help because
> old SSH
> server works in background.
>
> Please be warned, especially administrators of remote serves - if you skip
> this step, SSH server will be kept in background, but refusing any users (
> after all keys or passwords it just fails with Connection Closed error).
>
> This happend on all 3(!) servers where I made updates, so I belive this is
> common. Now, before upgrading system I am always keeping one console
> to remote server open, in case of such problems.
>
> Here is list of packages which definitely result in SSH error (if somebody
> points out which exactly package update causes this problem?):
>
> 1098263971: >>> emerge (1 of 13) sys-libs/cracklib-2.7-r9 to /
> 1098263988: >>> emerge (2 of 13) sys-devel/autoconf-2.59-r5 to /
> 1098264010: >>> emerge (3 of 13) net-www/apache-2.0.52 to /
> 1098264374: >>> emerge (4 of 13) sys-apps/baselayout-1.9.4-r5 to /
> 1098264395: >>> emerge (5 of 13) sys-libs/glibc-2.3.4.20040808-r1 to /
Here we go :-)

Isn't this the glibc upgrade bug??

See here:
http://bugs.gentoo.org/show_bug.cgi?id=67188#c10

> 1098266195: >>> emerge (6 of 13) sys-devel/make-3.80-r1 to /
> 1098266227: >>> emerge (7 of 13) sys-fs/reiserfsprogs-3.6.18 to /
> 1098266264: >>> emerge (8 of 13) sys-apps/shadow-4.0.4.1-r4 to /
> 1098266341: >>> emerge (9 of 13) dev-lang/python-2.3.4 to /
> 1098266539: >>> emerge (10 of 13) dev-util/subversion-1.0.9 to /
> 1098266814: >>> emerge (11 of 13) sys-apps/net-tools-1.60-r9 to /
> 1098266840: >>> emerge (12 of 13) net-misc/whois-4.6.13-r1 to /
> 1098266852: >>> emerge (13 of 13)
> sys-kernel/gentoo-dev-sources-2.6.8-r10 to /
>
> P.S. After this update I had to call my colocation provider asking them
> to reboot server.
>

Kalin.

--
|| ~~~~~~~~~~~~~~~~~~~~~~ ||
( ) http://ThinRope.net/ ( )
|| ______________________ ||

--
gentoo-user@gentoo.org mailing list