Mailing List Archive

On Wed, 24 Feb 2021 20:43:39 -0500,
Grant Taylor wrote:
>
> I need to update a system that hasn't been updated in 337 days
> (March 24th 2020. -- Life has been ... trying.
>
> What is the best way forward?
>
> It seems as if there have been a lot of changes in the interim;
> glibc, Python 2.7 being deprecated, default Python going to
> 3.7(?), other breaking changes....
>
> Is there a way that I can sync portage to something from April,
> May, or even June of 2020, do a full update (including "-DUNe
> @world")? Iterating through multiple rounds to get current?
>
> Any help would be appreciated.

What you could try to do, if you are syncing using git, is to roll it
back to those dates by checking out a commit each time and doing an
update. I don't guarantee it would work, but its worth a shot,
otherwise reinstall time.

--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici wb2una
covici@ccs.covici.com

On 2/24/21 6:48 PM, John Covici wrote:
> What you could try to do, if you are syncing using git, is to roll
> it back to those dates by checking out a commit each time and doing
> an update. I don't guarantee it would work, but its worth a shot,
> otherwise reinstall time.

And what if I was still using rsync?

I'm currently doing a git clone of the gentoo-mirror/gentoo.git
repository in another directory.

Once that finishes, I'll see if I can list the commits in it from March
and see if I can work my way forward.

Does it /actually/ matter how I get the portage repository as long as
it's one from a time close enough that "emerge -DUN @world" will succeed
in small increments? Even if I have to automate stepping through
hundreds of them.

My understanding is that emerge works against the contents of the
PORTDIR, usually /usr/portage. So as long as I get ... let's call it
... a compatible version.... That's my hope anyway.



--
Grant. . . .
unix || die

On 2/24/21 6:48 PM, John Covici wrote:
> What you could try to do, if you are syncing using git, is to roll
> it back to those dates by checking out a commit each time and doing
> an update. I don't guarantee it would work, but its worth a shot,
> otherwise reinstall time.

I hit send too soon.

Thank you for the reply John.



--
Grant. . . .
unix || die

On Wed, 24 Feb 2021 21:20:05 -0500,
Grant Taylor wrote:
>
> On 2/24/21 6:48 PM, John Covici wrote:
> > What you could try to do, if you are syncing using git, is to
> > roll it back to those dates by checking out a commit each time
> > and doing an update. I don't guarantee it would work, but its
> > worth a shot, otherwise reinstall time.
>
> I hit send too soon.
>
> Thank you for the reply John.
>

The portdir has to be the one gotten from git, not rsync, and remember
I think there was a major profile change during that time period
along with changes in the C compiler. Unless you have a lot of
customizations, reinstall would be much better.

--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici wb2una
covici@ccs.covici.com

On 2/24/21 9:16 PM, John Covici wrote:
> The portdir has to be the one gotten from git, not rsync,

ACK

I'm currently doing an "emerge -DUNe @system" on the restore of
/usr/portage (typical PORTDIR) from prior to messing with things today.

I've got multiple GB of git data. It looks like there are ~568 thousand
commits between March 24th last year and now. Once that's good, and I'm
back at a stable place, I'll try changing PORTDIR to be the git repo and
telling git to switch to the commit that's from March 25th. Then I'll
see if anything needs to be updated, doing so as necessary. Then I'll
leap frog a week at a time seeing what needs to be updated, doing so as
necessary. /Hopefully/ I can slowly walk forward. Time will tell.

> and remember I think there was a major profile change during that
> time period along with changes in the C compiler.

If I can slowly make my way forward in time via git commit points, I
/think/ that I /should/ be able to deal with profile and / or compiler
and / or glibc changes just like I would have X number of months ago. I
/think/!

> Unless you have a lot of customizations, reinstall would be much
> better.

I'd really rather not do that. I'm more likely to leave this system as
it is and plan on upgrading it some time in '21. There's considerably
more to it than I want to wholesale replace.

Besides, wouldn't each of the incremental processes over the last year
have been possible? ;-)



--
Grant. . . .
unix || die

Am 25.02.2021 um 02:43 schrieb Grant Taylor:
> I need to update a system that hasn't been updated in 337 days (March
> 24th 2020.  --  Life has been ... trying.
>
> What is the best way forward?
>
> It seems as if there have been a lot of changes in the interim; glibc,
> Python 2.7 being deprecated, default Python going to 3.7(?), other
> breaking changes....
>
> Is there a way that I can sync portage to something from April, May, or
> even June of 2020, do a full update (including "-DUNe @world")?
> Iterating through multiple rounds to get current?
>
> Any help would be appreciated.
>



I found it to be helpful to de-install as many programs as possible
before starting the update and the first emerge --sync. This reduces the
amount of conflicts by a considerable amount. Stuff like libreoffice or
thunderbird and so on and all of their dependencies. Everything your
system does not need to run but what you need to be productive when you
use your system. I use -av --depclean for this.

Also there is something in the gentoo wiki about upgrading old or too
old systems.

https://wiki.gentoo.org/wiki/Upgrading_Gentoo#Upgrading_from_older_systems

On Thursday, 25 February 2021 04:29:25 GMT Grant Taylor wrote:
> On 2/24/21 9:16 PM, John Covici wrote:

[snip ...]

> > Unless you have a lot of customizations, reinstall would be much
> > better.

It would probably be better even with a lot of customizations. ;-)

At least it /should/ be better in terms of time and effort spent.


> I'd really rather not do that. I'm more likely to leave this system as
> it is and plan on upgrading it some time in '21. There's considerably
> more to it than I want to wholesale replace.

A reinstall in this context is not a wholesale replace. It implies obtaining
the latest Stage 3 archive from a mirror, but retaining part of your current
installation. Your /home, /etc, /var/lib/portage/world, plus any databases
e.g. in /var/lib/mysql/ and your kernel config will be retained from your
existing system and will not be replaced. Back these up first along with any
particular customizations you have made, before you untar Stage 3, so you can
restore them.

Then rsync portage, update all your @world packages and build a new kernel
(make oldconfig). Spend some time merging existing application config files
with etc-update to make them compatible with the latest versions of these
packages, reboot and hopefully that should be all there is to it.


> Besides, wouldn't each of the incremental processes over the last year
> have been possible? ;-)

Yes, it would have been, but what is the benefit of updating multiple packages
many times over, instead of doing it just once?

On 25/02/21 08:45, hitachi303 wrote:
> Am 25.02.2021 um 02:43 schrieb Grant Taylor:
>> I need to update a system that hasn't been updated in 337 days (March
>> 24th 2020. -- Life has been ... trying.
>>
>> What is the best way forward?
>>
>> It seems as if there have been a lot of changes in the interim; glibc,
>> Python 2.7 being deprecated, default Python going to 3.7(?), other
>> breaking changes....
>>
>> Is there a way that I can sync portage to something from April, May,
>> or even June of 2020, do a full update (including "-DUNe @world")?
>> Iterating through multiple rounds to get current?
>>
>> Any help would be appreciated.
>>
>
>
>
> I found it to be helpful to de-install as many programs as possible
> before starting the update and the first emerge --sync. This reduces the
> amount of conflicts by a considerable amount. Stuff like libreoffice or
> thunderbird and so on and all of their dependencies. Everything your
> system does not need to run but what you need to be productive when you
> use your system. I use -av --depclean for this.
>
> Also there is something in the gentoo wiki about upgrading old or too
> old systems.
>
> https://wiki.gentoo.org/wiki/Upgrading_Gentoo#Upgrading_from_older_systems
>
I'd do something a bit like that. The trick is to do the ABSOLUTE
MINIMUM each step.

Can you "emerge portage"? Given the elephant in the room that is all the
python changes, if that works you're probably 90% of the way there.

Do you need to update the system profile? Update it to a very basic
profile and "emerge @system". NO --deep, NO --new-use, nothing like that.

If portage says it can update some things but not others, then tell it
to update what it can. Normally I'd "emerge -C" any blockers, but with a
minimal system that's probably not wise.

Once that's done, select your real profile that you want, and again do a
basic "emerge @system". Followed by "emerge @world".

Now do your full-blown "emerge --deep --new-use", depclean, new kernel,
all that sort of stuff.

Cheers,
Wol

On Thu, 25 Feb 2021 09:51:25 +0000, Michael wrote:

> > Besides, wouldn't each of the incremental processes over the last year
> > have been possible? ;-)
>
> Yes, it would have been, but what is the benefit of updating multiple
> packages many times over, instead of doing it just once?

One benefit, and it may be an important one depending on the usage of
this system, is that you keep the system running without significant
interruptions.

My immediate thought when I saw this was the same as John's. Now that
portage is available over git, pull in a commit from a month to a week
after the last update, update the system, rinse and repeat.

Another benefit is that it can be a good way of heating the house at this
time of year ;-)


--
Neil Bothwick

PCMCIA: People Can't Memorize Computer Industry Acronyms

On 2021-02-25, Grant Taylor <gtaylor@gentoo.tnetconsulting.net> wrote:

> I need to update a system that hasn't been updated in 337 days (March
> 24th 2020. -- Life has been ... trying.
>
> What is the best way forward?

If it were me, I'd probably back up /etc and /home and just reinstall.

--
other Grant

On 2021-02-25, Grant Taylor <gtaylor@gentoo.tnetconsulting.net> wrote:

> Besides, wouldn't each of the incremental processes over the last year
> have been possible? ;-)

Yes -- and fairly easy. Somehow, the amount of update effort required
doesn't just add up over time, it multiplies.

On 2021-02-25, hitachi303 <gentoo-user@konstantinhansen.de> wrote:

> I found it to be helpful to de-install as many programs as possible
> before starting the update and the first emerge --sync. This reduces the
> amount of conflicts by a considerable amount.

Yes, Definitely. If you can, uninstall anything "big" that you can
live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
Cups, etc.

Leave portage, sshd, Python, your init system, and GCC.

> Stuff like libreoffice or thunderbird and so on and all of their
> dependencies. Everything your system does not need to run but what
> you need to be productive when you use your system. I use -av
> --depclean for this.

On 2/25/21 2:51 AM, Michael wrote:
> It would probably be better even with a lot of customizations. ;-)

Please elaborate on what "better" means in this case. I'm thinking that
you might be meaning "faster" and / or "easier" (as in less effort).

> At least it /should/ be better in terms of time and effort spent.

Maybe.

> A reinstall in this context is not a wholesale replace.

~blink~

> It implies obtaining the latest Stage 3 archive from a mirror,
> but retaining part of your current installation. Your /home, /etc,
> /var/lib/portage/world, plus any databases e.g. in /var/lib/mysql/
> and your kernel config will be retained from your existing system and
> will not be replaced. Back these up first along with any particular
> customizations you have made, before you untar Stage 3, so you can
> restore them.

Ah. You seem to be talking about what I would call an "in place
upgrade" for Windows. As in stalling n over top of n-1 or n-2. That's
definitely less disruptive than I was thinking. I was thinking that
fdisk and / or mkfs would be involved.

> Then rsync portage, update all your @world packages and build a new
> kernel (make oldconfig). Spend some time merging existing application
> config files with etc-update to make them compatible with the latest
> versions of these packages, reboot and hopefully that should be all
> there is to it.

I may end up /needing/ to go that route. For the moment, I'm going to
try the incremental updates.

> Yes, it would have been, but what is the benefit of updating multiple
> packages many times over, instead of doing it just once?

In some ways, this is a learning experience. As in it's a proof of concept.

The computer in question spends 2/3 of it's life doing nothing but
idling a few programs. So, it spending time compiling and producing
heat is not a bad thing in this case. Especially when there's 10" of
snow on the ground. ;-)



--
Grant. . . .
unix || die

On Thu, 25 Feb 2021 15:15:55 -0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> On 2021-02-25, hitachi303 <gentoo-user@konstantinhansen.de> wrote:
>
> > I found it to be helpful to de-install as many programs as possible
> > before starting the update and the first emerge --sync. This reduces the
> > amount of conflicts by a considerable amount.
>
> Yes, Definitely. If you can, uninstall anything "big" that you can
> live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
> Cups, etc.
>
> Leave portage, sshd, Python, your init system, and GCC.
>
> > Stuff like libreoffice or thunderbird and so on and all of their
> > dependencies. Everything your system does not need to run but what
> > you need to be productive when you use your system. I use -av
> > --depclean for this.
>
>

Can't you, instead of uninstalling anything big, just start with the
system set for the upgrade?
emerge -auvDN @system

Once the system set is updated you can assume of being in a rather sane
state, where a world upgrade should just work(TM).

On Thursday, 25 February 2021 15:42:38 GMT Grant Taylor wrote:
> On 2/25/21 2:51 AM, Michael wrote:

> > A reinstall in this context is not a wholesale replace.
>
> ~blink~
>
> > It implies obtaining the latest Stage 3 archive from a mirror,
> > but retaining part of your current installation. Your /home, /etc,
> > /var/lib/portage/world, plus any databases e.g. in /var/lib/mysql/
> > and your kernel config will be retained from your existing system and
> > will not be replaced. Back these up first along with any particular
> > customizations you have made, before you untar Stage 3, so you can
> > restore them.
>
> Ah. You seem to be talking about what I would call an "in place
> upgrade" for Windows. As in stalling n over top of n-1 or n-2. That's
> definitely less disruptive than I was thinking. I was thinking that
> fdisk and / or mkfs would be involved.

Yes, it is an "in place upgrade", but bottom up. You upgrade the filesystem
to the latest OS image, then drop in your own existing settings/configuration
and finally run a system/kernel update.

Unless you want to change your partitions you won't need to use fdisk.

mkfs is advisable, it will clear out any old cruft and address any changes to
the default system directories - e.g. where $PORTDIR, $DISTDIR may be in the
latest Stage 3.

A quick diff between your backup of make.conf and repos.conf against the new
Stage 3 archive contents will inform you of changes to default settings.


> > Then rsync portage, update all your @world packages and build a new
> > kernel (make oldconfig). Spend some time merging existing application
> > config files with etc-update to make them compatible with the latest
> > versions of these packages, reboot and hopefully that should be all
> > there is to it.
>
> I may end up /needing/ to go that route. For the moment, I'm going to
> try the incremental updates.
>
> > Yes, it would have been, but what is the benefit of updating multiple
> > packages many times over, instead of doing it just once?
>
> In some ways, this is a learning experience. As in it's a proof of concept.

Yes, nowt wrong with that and a sound reason to try it out. In this case,
time spent and any problem solving would be an investment in learning.


> The computer in question spends 2/3 of it's life doing nothing but
> idling a few programs. So, it spending time compiling and producing
> heat is not a bad thing in this case. Especially when there's 10" of
> snow on the ground. ;-)

Neil highlighted the use case of a server/system which can't afford
interruptions. I'd still reinstall as explained above, but do it offline,
test, backup, and then untar over the live filesystem to keep downtime to a
minimum.

On Thu, 25 Feb 2021 15:15:55 -0000 (UTC), Grant Edwards wrote:

> > I found it to be helpful to de-install as many programs as possible
> > before starting the update and the first emerge --sync. This reduces
> > the amount of conflicts by a considerable amount.
>
> Yes, Definitely. If you can, uninstall anything "big" that you can
> live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
> Cups, etc.

You may get away with removing them from @world rather than actually
uninstalling. They may well continue to work until something they depend
on has an ABI update. I'd try to keep anything depending on boost or icu
out of the update process.


--
Neil Bothwick

WinErr 002: No Error - Yet

On Thu, Feb 25, 2021 at 9:33 AM Neil Bothwick <neil@digimed.co.uk> wrote:
>
> On Thu, 25 Feb 2021 15:15:55 -0000 (UTC), Grant Edwards wrote:
>
> > > I found it to be helpful to de-install as many programs as possible
> > > before starting the update and the first emerge --sync. This reduces
> > > the amount of conflicts by a considerable amount.
> >
> > Yes, Definitely. If you can, uninstall anything "big" that you can
> > live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
> > Cups, etc.
>
> You may get away with removing them from @world rather than actually
> uninstalling. They may well continue to work until something they depend
> on has an ABI update. I'd try to keep anything depending on boost or icu
> out of the update process.
>
>
> --
> Neil Bothwick
>
> WinErr 002: No Error - Yet

In the couple of times I've been through this

1) If I was, in the old environment, running unstable app versions then the
first thing I'd do is just set them to stable. This presumably gives a good
chance of getting apps to just build without pulling in unstable libraries
and the like.

2) Commenting out or removing at least unstable apps from world at least
gives ideas about where application problems might exist if they aren't
solved in step 1. I personally would uninstall unstable apps, keeping a
list on paper, and then reinstalling later when the basics are upgraded and
proved functional.

3) Based on other folk's issues with Gentoo and Python versions I'm not
overly confident any of this is easier than moving /home out-of-the-way,
rebuilding the machine with a new install and then seeing how the old home
directories survived.

Good luck,
Mark

HTH,
Mark

Am 25.02.2021 um 17:13 schrieb Andreas Fink:
> On Thu, 25 Feb 2021 15:15:55 -0000 (UTC)
> Grant Edwards <grant.b.edwards@gmail.com> wrote:
>
>> On 2021-02-25, hitachi303 <gentoo-user@konstantinhansen.de> wrote:
>>
>>> I found it to be helpful to de-install as many programs as possible
>>> before starting the update and the first emerge --sync. This reduces the
>>> amount of conflicts by a considerable amount.
>>
>> Yes, Definitely. If you can, uninstall anything "big" that you can
>> live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
>> Cups, etc.
>>
>> Leave portage, sshd, Python, your init system, and GCC.
>>
>>> Stuff like libreoffice or thunderbird and so on and all of their
>>> dependencies. Everything your system does not need to run but what
>>> you need to be productive when you use your system. I use -av
>>> --depclean for this.
>>
>>
>
> Can't you, instead of uninstalling anything big, just start with the
> system set for the upgrade?
> emerge -auvDN @system
>
> Once the system set is updated you can assume of being in a rather sane
> state, where a world upgrade should just work(TM).
>

In theory this should work. My experience is that it does not. I have
had a couple of time when I couldn't do an emerge -auvDN @system because
of blocks but when trying to do a emerge -auvDN @world portage could
solve the issue with the blocks.

hitachi303 wrote:
> Am 25.02.2021 um 17:13 schrieb Andreas Fink:
>> On Thu, 25 Feb 2021 15:15:55 -0000 (UTC)
>> Grant Edwards <grant.b.edwards@gmail.com> wrote:
>>
>>> On 2021-02-25, hitachi303 <gentoo-user@konstantinhansen.de> wrote:
>>>
>>>> I found it to be helpful to de-install as many programs as possible
>>>> before starting the update and the first emerge --sync. This
>>>> reduces the
>>>> amount of conflicts by a considerable amount.
>>>
>>> Yes, Definitely. If you can, uninstall anything "big" that you can
>>> live without temporarily: LibreOffice, Chromium, Qt, KDE, X11, Gnome,
>>> Cups, etc.
>>>
>>> Leave portage, sshd, Python, your init system, and GCC.
>>>
>>>> Stuff like libreoffice or thunderbird and so on and all of their
>>>> dependencies. Everything your system does not need to run but what
>>>> you need to be productive when you use your system. I use -av
>>>> --depclean for this.
>>>
>>>
>>
>> Can't you, instead of uninstalling anything big, just start with the
>> system set for the upgrade?
>> emerge -auvDN @system
>>
>> Once the system set is updated you can assume of being in a rather sane
>> state, where a world upgrade should just work(TM).
>>
>
> In theory this should work. My experience is that it does not. I have
> had a couple of time when I couldn't do an emerge -auvDN @system
> because of blocks but when trying to do a emerge -auvDN @world portage
> could solve the issue with the blocks.
>
>


I've ran into that before too.  If it were me, I'd remove/rename the
world file, update the rest in steps and start reinstalling packages
from the old world file.  Once updated, the run depclean to see what you
might want to keep or can be removed as no longer needed. 

One issue with doing @system alone, it pulls in even KDE packages if you
have KDE installed and have certain USE flags enabled.  Then those
trigger other packages that @system normally wouldn't touch or even
consider.  That can lead to a catch 22 or a circular problem that emerge
can't work through.  I don't use Gnome but wouldn't be surprised if the
same happens with it either.  It may be to a lesser degree or could even
be worse. 

This is one of those 6 of one and half dozen of the other.  It's a
learning experience but can be time and effort consuming.  It will annoy
a lot of electrons too.  lol

Either way, I look forward to seeing how this turns out.  Just the major
changes in python alone should make this really fun.

Dale

:-)  :-) 

On 2/24/21 9:29 PM, Grant Taylor wrote:
> I'm currently doing an "emerge -DUNe @system" on the restore of
> /usr/portage (typical PORTDIR) from prior to messing with things today.

The system is now stable with a full -DUNe @system.

emerge -DUNe @system
reboot
emerge -DUNe @world && emerge --depclean --verbose n && revdep-rebuild

> I've got multiple GB of git data.  It looks like there are ~568 thousand
> commits between March 24th last year and now.  Once that's good, and I'm
> back at a stable place, I'll try changing PORTDIR to be the git repo and
> telling git to switch to the commit that's from March 25th.  Then I'll
> see if anything needs to be updated, doing so as necessary.  Then I'll
> leap frog a week at a time seeing what needs to be updated, doing so as
> necessary.  /Hopefully/ I can slowly walk forward.  Time will tell.

I was able to extract the last commit for every day between now and
2020-03-24 and make a branch for it.

10 have git switch to the next day
20 emerge -aDUN @world
30 assess / deal with masked packages
40 goto 10

It /looks/ like things are working.

Yes, emerge is spending a LOT of time mulling over things. Many days
have been "Nothing to merge; quitting."

> If I can slowly make my way forward in time via git commit points, I
> /think/ that I /should/ be able to deal with profile and / or compiler
> and / or glibc changes just like I would have X number of months ago.  I
> /think/!

One added advantage of doing this day by day is that when I do get to
the big changes, things should be fairly clean. Thus hopefully
simplifying the big changes.



--
Grant. . . .
unix || die

On Thu, 25 Feb 2021 17:31:15 -0700, Grant Taylor wrote:

> One added advantage of doing this day by day is that when I do get to
> the big changes, things should be fairly clean. Thus hopefully
> simplifying the big changes.

You are also going to hit the annoying bugs that were fixed a few hours
later. The main issue I had with the python 3.7 switch could have been
dealt with by waiting until the next day. This is less of an issue with a
purely stable system but it can still happen.


--
Neil Bothwick

"An investment in knowledge always pays the best interest." - Benjamin
Franklin