Mailing List Archive

rsync speed and space taken
while waiting for a rsync on one of my computers i noticed
there are alot of patchs, gzip, and bzip2 files in portage
alot of which i wouldnt think needs to be there
x11-base/xfree/files/ for one example
sys-devel/gcc/files/ for another
im sure there are other places that are just as bad

find /mdhd/gentoo/gentoo-portage/ -name "*.bz2"|wc -l
159

find /mdhd/gentoo/gentoo-portage/ -name "*.gz"|wc -l
62

even with my local lans rsync mirror serving i think 6 computers currently
i dont even use half of those bz2 or gz files and im sure similar goes for
alot of the patchs in the tree

i seem to recall there is a script or somethin, or was, that checked sizes
of files in the tree perhaps it should be expanded to file types
or better yet watch file counts for files that arent needed
so whatever isnt a ebuild, digest, manifest, or changelog
considering how long it takes to scan the tree as it is
i would think this would help alot
and im sure the dialup users wouldnt mind not downloading
stuff they would never use

Bret

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
Its not only about dialup users either. For example if running gentoo
on a production system, you would normally run glsa-check on an hourly
basis. If your system is volnerable, you'll normally have to emerge
sync first, and with the number of files there, it takes far too long.

There has been a lot of discussion about switching away from rsync
and/or changing ebuild structure for portage-ng -- but thats mostly
gossip and any progress is halted.

I do agree that some files really should be removed from the tree,
look at this list:

find ./ -type f | egrep -v "packages|distfiles" |xargs du | sort -n | tail -n 20
72 ./media-video/nvidia-kernel/files/1.0.4499/NVIDIA_kernel-1.0-4499-2.6-20031014.diff
72 ./net-www/apache/files/httpd-2.0.49-ssl_engine_kernel.patch
72 ./net-www/apache/files/patches/2.0.49-r1/01_ssl_engine_kernel.patch
72 ./sys-devel/binutils/files/2.13/binutils-2.13.90.0.18-ppc64-tls1.patch
76 ./app-accessibility/speech-tools/files/1.2.3-gcc3.4.patch
76 ./sys-devel/gcc/ChangeLog
80 ./media-video/nvidia-kernel/files/1.0.5328/NVIDIA_kernel-1.0-5328-2.6-20031226.diff
80 ./sys-devel/binutils/files/2.14/binutils-2.14.90.0.4-cfi.patch
84 ./gnome-base/gnome-session/files/gentoo-splash.png
84 ./gnome-base/gnome-session/files/gnome-splash.png
84 ./media-video/nvidia-kernel/files/1.0.4363/NVIDIA_kernel-1.0-4363-2.5-20030714.diff
84 ./media-video/nvidia-kernel/files/1.0.4496/NVIDIA_kernel-1.0-4496-2.6-20030905.diff
84 ./media-video/nvidia-kernel/files/1.0.4496/NVIDIA_kernel-1.0-4496-2.6-20031026.diff
84 ./x11-base/xfree/ChangeLog
88 ./mail-filter/amavisd-new/files/amavisd.conf
92 ./app-emulation/wine/files/wine-alsa.patch
92 ./games-emulation/nwwine/files/wine-alsa.patch
100 ./media-video/nvidia-kernel/files/1.0.5328/NVIDIA_kernel-1.0-5328-2.6-20040105.diff
120 ./sys-libs/pam/files/pam-0.75-r7-gentoo.tbz2
168 ./licenses/perforce.pdf


On Sun, 10 Oct 2004 10:38:59 -0700, Bret Towe <magnade@gmail.com> wrote:
> while waiting for a rsync on one of my computers i noticed
> there are alot of patchs, gzip, and bzip2 files in portage
> alot of which i wouldnt think needs to be there
> x11-base/xfree/files/ for one example
> sys-devel/gcc/files/ for another
> im sure there are other places that are just as bad
>
> find /mdhd/gentoo/gentoo-portage/ -name "*.bz2"|wc -l
> 159
>
> find /mdhd/gentoo/gentoo-portage/ -name "*.gz"|wc -l
> 62
>
> even with my local lans rsync mirror serving i think 6 computers currently
> i dont even use half of those bz2 or gz files and im sure similar goes for
> alot of the patchs in the tree
>
> i seem to recall there is a script or somethin, or was, that checked sizes
> of files in the tree perhaps it should be expanded to file types
> or better yet watch file counts for files that arent needed
> so whatever isnt a ebuild, digest, manifest, or changelog
> considering how long it takes to scan the tree as it is
> i would think this would help alot
> and im sure the dialup users wouldnt mind not downloading
> stuff they would never use
>
> Bret
>
> --
> gentoo-dev@gentoo.org mailing list
>
>

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
On 10/10/04 Roman Gaufman wrote:

> Its not only about dialup users either. For example if running gentoo
> on a production system, you would normally run glsa-check on an hourly
> basis. If your system is volnerable, you'll normally have to emerge
> sync first, and with the number of files there, it takes far too long.

That's pointless: glsa-check gets the information from the tree, so you
only have to run it after you've run `emerge --sync`. There are plans to
extend it to use the GLSAs on the security website but that isn't
implemented yet (and, as you have pointed out, would require you to run
`emerge --sync` anyway).

Marius
Re: rsync speed and space taken [ In reply to ]
<calm rant>
why is it that developers can't spend an extra moment or 2 tar/bz2'ing
their patchsets and uploading them to the mirrors instead of including
them where they are and making the portage tree bulge from the seams?
there are patchsets that are currently in <pkgname>/files for packages
that are only useful on a graphical workstation... for people that run
servers, this seems like quite a bit of cruft. there are currently
100,712 files in /usr/portage... does this seem a little ridiculous to
anyone else?
</calm rant>

<angry rant>
why don't we put more stuff in mirror://gentoo/ ?? ${FILESDIR} has too
much SHIT in it... i appreciate that patches, etc have been placed
there... but we REALLY need to find a better place. I don't need my
emerge sync's taking 45 minutes on average on a 5Mbit line in
france... *45 minutes* ...just PLEASE keep the SHIT out of the portage
tree.

SHIT: (noun);
1. useless stuff that has no business being in my portage tree.
2. stuff that i'll never have use for... see #1
3. anything related to X11, kde, gnome, etc on a server.
</angry rant>

that's my 2/100ths of a monetary unit.

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
A. CALM DOWN
B. One of the main reasons some devs like to push the limits of what can be
in files/ is because it takes up to 8 hours for things to get to mirrors
sometimes.
C. There are some things that are going to be in the portage tree that are
large that not everyone needs. That's just the way it is. No, we don't like
it. No, there isn't much we can do about it.
D. Most devs know it's a problem. Most devs are also already overworked as
is.
E. If you really want to do some good, write a patch for repoman to kick
people in the "jimmy jammy" when they try to commit big files.
F. CALM DOWN


On 11:49:34 pm 10/10/04 Allen Parker <infowolfe@gmail.com> wrote:
>
> > <calm rant>
> > why is it that developers can't spend an extra moment or 2
> > tar/bz2'ing their patchsets and uploading them to the mirrors
> > instead of including them where they are and making the portage
> > tree bulge from the seams? there are patchsets that are currently
> > in <pkgname>/files for packages that are only useful on a
> > graphical workstation... for people that run servers, this seems
> > like quite a bit of cruft. there are currently 100,712 files in
> > /usr/portage... does this seem a little ridiculous to anyone else?
> > </calm rant>
> >
> > <angry rant>
> > why don't we put more stuff in mirror://gentoo/ ?? ${FILESDIR} has
> > too much SHIT in it... i appreciate that patches, etc have been
> > placed there... but we REALLY need to find a better place. I don't
> > need my emerge sync's taking 45 minutes on average on a 5Mbit line
> > in france... *45 minutes* ...just PLEASE keep the SHIT out of the
> > portage tree.
> >
> > SHIT: (noun);
> > 1. useless stuff that has no business being in my portage tree.
> > 2. stuff that i'll never have use for... see #1
> > 3. anything related to X11, kde, gnome, etc on a server.
> > </angry rant>
> >
> > that's my 2/100ths of a monetary unit.
> >
> > --
> > gentoo-dev@gentoo.org mailing list
> >
>


--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
On Monday 11 October 2004 14:19, Brian Jackson wrote:
> A. CALM DOWN
> B. One of the main reasons some devs like to push the limits of what can be
> in files/ is because it takes up to 8 hours for things to get to mirrors
> sometimes.
> C. There are some things that are going to be in the portage tree that are
> large that not everyone needs. That's just the way it is. No, we don't like
> it. No, there isn't much we can do about it.
> D. Most devs know it's a problem. Most devs are also already overworked as
> is.
> E. If you really want to do some good, write a patch for repoman to kick
> people in the "jimmy jammy" when they try to commit big files.
> F. CALM DOWN

G. RSYNC_EXCLUDEFROM

Regards,
Jason Stubbs

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
Allen Parker posted <9f27901604101016497d40b0a@mail.gmail.com>, excerpted
below, on Sun, 10 Oct 2004 15:49:34 -0800:

> I don't need my emerge sync's taking 45 minutes on average on a 5Mbit
> line in france... *45 minutes*

If your emerge syncs are taking *THAT* long, on a 5Mbit line, somethings
TERRIBLY WRONG! I have a cable modem here (Phoenix, AZ, US), capped @
4Mbps, and my syncs take perhaps a couple minutes, short enough time that
I usually do it interactively -- type in the command, and watch it work,
then do an emerge -auD world to see what is there to change, after that.

FWIW, I note that much of the time, it's actually not downloading
anything, only checking stuff locally (presumably doing local md5 sums
and matching against the new ones in the initially d/led file to see if
they've changed), in the background. Only the initial file d/l, and the
two-line entries with the percent complete and etc on the second line, are
actually transferred. The rest of the time is spent locally, generally
CPU bound (single thread/CPU only) That being the case, I suspect the
reason it takes you 45 minutes and me substantially less than five, may be
due to local CPU processing power. I'm running a dual AMD64 Opteron (1 gig
memory, swap entirely kernel disabled so the kernel doesn't bother with it
at all) here, altho as I mentioned, portage/python doesn't appear to be
multi-threaded, so it only tops off one CPU. If you are running something
less powerful, that'd probably explain your longer sync times even with a
fatter internet pipe.

.. About 3 and a half minutes. I just timed it.

--
Duncan - List replies preferred. No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin



--
gentoo-dev@gentoo.org mailing list
Re: Re: rsync speed and space taken [ In reply to ]
Duncan wrote:
> .. About 3 and a half minutes. I just timed it.

rm -rf /usr/portage and time it again.



--
gentoo-dev@gentoo.org mailing list
Re: Re: rsync speed and space taken [ In reply to ]
On Monday 11 October 2004 12:37, Travis Tilley wrote:
> Duncan wrote:
> > .. About 3 and a half minutes. I just timed it.
>
> rm -rf /usr/portage and time it again.
>

For people that have slow rsync times, there is the alternative to run
emerge-websync. It is maximally a day behind and works well.
Unfortunately it needs to download a lot more, but doesn't need to scan
the whole local and remote trees.

Rsync is for a big part actually IO bound. If you are not using dma you're
basically fucked. A good cache and enough memory helps too.

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net
Re: Re: rsync speed and space taken [ In reply to ]
Travis Tilley wrote:

> Duncan wrote:
>
>> .. About 3 and a half minutes. I just timed it.
>
>
> rm -rf /usr/portage and time it again.
>
wget
http://www.die.unipd.it/pub/Linux/distributions/gentoo-sources/snapshots/portage-20041010.tar.bz2
cd /usr ; tar -jxvf portage-20041010.tar.bz2
emerge sync

portage-20041010.tar.bz2 is 16 megs

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
On Sun, 2004-10-10 at 15:49 -0800, Allen Parker wrote:
> <angry rant>
> why don't we put more stuff in mirror://gentoo/ ?? ${FILESDIR} has too
> much SHIT in it... i appreciate that patches, etc have been placed
> there... but we REALLY need to find a better place. I don't need my
> emerge sync's taking 45 minutes on average on a 5Mbit line in
> france... *45 minutes* ...just PLEASE keep the SHIT out of the portage
> tree.

While I completely agree that things need to be taken out of
${FILESDIR}, I have a question for you. Could you give the specs of
your machine? I have several machines and even on my oldest machine
with an ATA33 drive and 128MB of RAM, it takes nowhere near 45 minutes
to rsync the tree. Are you using your geographic rsync mirror? Do you
have any other I/O bound processes running at the same time? I'm not
trying to do anything other than point out that your times seem awfully
long in my experience and there is a distinct possibility that something
else is causing the slowdown to such extremes on your system.

--
Chris Gianelloni
Release Engineering - Operations/QA Manager
Games - Developer
Gentoo Linux
Re: Re: rsync speed and space taken [ In reply to ]
Paul de Vrieze wrote:
> On Monday 11 October 2004 12:37, Travis Tilley wrote:
>
>>Duncan wrote:
>>
>>>.. About 3 and a half minutes. I just timed it.
>>
>>rm -rf /usr/portage and time it again.
>>
>
>
> For people that have slow rsync times, there is the alternative to run
> emerge-websync. It is maximally a day behind and works well.
> Unfortunately it needs to download a lot more, but doesn't need to scan
> the whole local and remote trees.

Actually, it will scan two trees, two *local* ones, on top of some extra I/O
FYI, webrsync does :
1) download a 16M snapshot
2) unpack its 100,000+ files
<rant>latest GWN sees it as a record, let's aim for 250,000 files</rant>
3) run rsync between temp dir and /usr/portage
4) rm 100,000 temp files
5) emerge metadata

emerge-webrsync is not meant to decrease I/O, in fact, it increases local I/O.
It is meant for people who can't use rsync because it's blocked (or because
there is no connection at all).


Wkr,
--
/ Xavier Neys
\_ Gentoo Documentation Project
/ French & Internationalisation Lead
\ http://www.gentoo.org/doc/en
/\
Re: rsync speed and space taken [ In reply to ]
Chris Gianelloni wrote:
> On Sun, 2004-10-10 at 15:49 -0800, Allen Parker wrote:
>
>><angry rant>
>>why don't we put more stuff in mirror://gentoo/ ?? ${FILESDIR} has too
>>much SHIT in it... i appreciate that patches, etc have been placed
>>there... but we REALLY need to find a better place. I don't need my
>>emerge sync's taking 45 minutes on average on a 5Mbit line in
>>france... *45 minutes* ...just PLEASE keep the SHIT out of the portage
>>tree.
>
>
> While I completely agree that things need to be taken out of
> ${FILESDIR}, I have a question for you. Could you give the specs of
> your machine? I have several machines and even on my oldest machine
> with an ATA33 drive and 128MB of RAM, it takes nowhere near 45 minutes
> to rsync the tree. Are you using your geographic rsync mirror? Do you
> have any other I/O bound processes running at the same time? I'm not
> trying to do anything other than point out that your times seem awfully
> long in my experience and there is a distinct possibility that something
> else is causing the slowdown to such extremes on your system.
>
I also have an old box (P100 w/ 128MB of RAM, 2 ATA33 disks at ~6MB/sec, udma
not available) and it takes ~45 minutes to run emerge --sync. Bandwidth is not
the bottleneck because its mirror is sitting next to it. The bottleneck is the
sheer number of files to check.

FYI: /usr/portage :
16276 - directories
83779 - files including
L_ 24820 files under /files/
L_ 19118 files under /metadata/cache/

16314 - Digest
16284 - .ebuild
7816 - Manifest
7812 - ChangeLog
6726 - metadata.xml
5100 - .patch / .diff
223 - Archive (gz/bz2/tar/...)
89 - make.defaults
78 - virtuals
67 - parent
61 - use.mask
49 - use.defaults
4042 - Uncategorized



My 2/100 of your currency,
--
/ Xavier Neys
\_ Gentoo Documentation Project
/ French & Internationalisation Lead
\ http://www.gentoo.org/doc/en
/\
Re: rsync speed and space taken [ In reply to ]
On Sun, 10 Oct 2004 15:49:34 -0800 Allen Parker <infowolfe@gmail.com>
wrote:
| why is it that developers can't spend an extra moment or 2 tar/bz2'ing
| their patchsets and uploading them to the mirrors instead of including
| them where they are and making the portage tree bulge from the seams?

a) because the mirror system really really sucks for that kind of thing,
so we tend to end up with dozens of duplicate bugs whining about patches
being missing because of the huge upload delay

b) because the mirror system really really sucks if you want to make a
few small changes to a patch that aren't worthy of a revbump

c) most of the stuff that you're complaining about has been there for
ages, and when it was added this wasn't an issue.

--
Ciaran McCreesh : Gentoo Developer (Vim, Fluxbox, Sparc, Mips)
Mail : ciaranm at gentoo.org
Web : http://dev.gentoo.org/~ciaranm
Re: rsync speed and space taken [ In reply to ]
On Mon, 2004-10-11 at 15:32 +0100, Ciaran McCreesh wrote:
> b) because the mirror system really really sucks if you want to make a
> few small changes to a patch that aren't worthy of a revbump

This is why I also have a patchball version.
--
Donnie Berkholz
Gentoo Linux
Re: rsync speed and space taken [ In reply to ]
On Mon, 11 Oct 2004 08:52:19 -0700 Donnie Berkholz
<spyderous@gentoo.org> wrote:
| On Mon, 2004-10-11 at 15:32 +0100, Ciaran McCreesh wrote:
| > b) because the mirror system really really sucks if you want to make
| > a few small changes to a patch that aren't worthy of a revbump
|
| This is why I also have a patchball version.

Yeah, then that gets messy if you end up trying to keep the same
patchset between revbumps. End up having to stick in a global variable
for it... Hardly ideal.

--
Ciaran McCreesh : Gentoo Developer (Vim, Fluxbox, Sparc, Mips)
Mail : ciaranm at gentoo.org
Web : http://dev.gentoo.org/~ciaranm
Re: rsync speed and space taken [ In reply to ]
On Mon, 2004-10-11 at 01:19, Brian Jackson wrote:
[snip]

> E. If you really want to do some good, write a patch for repoman to kick
> people in the "jimmy jammy" when they try to commit big files.

A few revisions back ferringb added support for this per a request.
repoman --pretend scan shows file sizes. (check it out in the binutils
dir) It's non fatal and will probably remain that will till >=2.0.52 but
after then no patch >=20k can go in the tree and we wont be able to
commit on a dir unless it's cleaned up.

Perhaps we should also display the size totals for the $PWD on a commit.

[snip]

> gentoo-dev@gentoo.org mailing list
--
Ned Ludd <solar@gentoo.org>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer
Re: rsync speed and space taken [ In reply to ]
On Mon, 11 Oct 2004 12:14:07 -0400 Ned Ludd <solar@gentoo.org> wrote:
| > E. If you really want to do some good, write a patch for repoman to
| > kick people in the "jimmy jammy" when they try to commit big files.
|
| A few revisions back ferringb added support for this per a request.
| repoman --pretend scan shows file sizes. (check it out in the
| binutils dir) It's non fatal and will probably remain that will till
| >=2.0.52 but
| after then no patch >=20k can go in the tree and we wont be able to
| commit on a dir unless it's cleaned up.

That's going to get *really* painful when we're trying to just keyword a
package without going around and changing other people's ebuilds. This
shouldn't be introduced as an error until after the existing tree is
entirely fixed.

--
Ciaran McCreesh : Gentoo Developer (Vim, Fluxbox, Sparc, Mips)
Mail : ciaranm at gentoo.org
Web : http://dev.gentoo.org/~ciaranm
Re: rsync speed and space taken [ In reply to ]
On Mon, 2004-10-11 at 17:17 +0100, Ciaran McCreesh wrote:
> That's going to get *really* painful when we're trying to just keyword a
> package without going around and changing other people's ebuilds. This
> shouldn't be introduced as an error until after the existing tree is
> entirely fixed.

I doubt the tree will get fixed until it is an error.
--
Donnie Berkholz
Gentoo Linux
Re: rsync speed and space taken [ In reply to ]
On Mon, 11 Oct 2004 09:25:19 -0700 Donnie Berkholz
<spyderous@gentoo.org> wrote:
| On Mon, 2004-10-11 at 17:17 +0100, Ciaran McCreesh wrote:
| > That's going to get *really* painful when we're trying to just
| > keyword a package without going around and changing other people's
| > ebuilds. This shouldn't be introduced as an error until after the
| > existing tree is entirely fixed.
|
| I doubt the tree will get fixed until it is an error.

Well, if it *doesn't*, then I guess anyone who touches other people's
ebuilds for, for example, keywording will have to just modify repoman...
I'll post a patch if this turns out to be a problem.

--
Ciaran McCreesh : Gentoo Developer (Vim, Fluxbox, Sparc, Mips)
Mail : ciaranm at gentoo.org
Web : http://dev.gentoo.org/~ciaranm
Re: rsync speed and space taken [ In reply to ]
On Mon, 11 Oct 2004 05:19:50 +0000, Brian Jackson <iggy@gentoo.org> wrote:
> A. CALM DOWN
> B. One of the main reasons some devs like to push the limits of what can be
> in files/ is because it takes up to 8 hours for things to get to mirrors
> sometimes.
the time it takes could be changed

> C. There are some things that are going to be in the portage tree that are
> large that not everyone needs. That's just the way it is. No, we don't like
> it. No, there isn't much we can do about it.
yes there is you just dont want to

> D. Most devs know it's a problem. Most devs are also already overworked as
> is.
yes i hear that excuse alot

> E. If you really want to do some good, write a patch for repoman to kick
> people in the "jimmy jammy" when they try to commit big files.
> F. CALM DOWN

well here is another idea for you 'overworked' devs
make a new dir on rsync called patchfiles (like distfiles) ignored by
emerge sync
add a mirror:// item support to have emerge grab patchs requested by ebuilds
from that rsync location
this would allow the devs to keep their 30m mirrors
would make rsync in general speed up for users
rsync mirrors wouldnt be out any more space than they already are
also this could allow a setup so patchs dont have to go into cvs
which i think is better if gzip/bzip items are going to be in there

>
> On 11:49:34 pm 10/10/04 Allen Parker <infowolfe@gmail.com> wrote:
> >
> > > <calm rant>
> > > why is it that developers can't spend an extra moment or 2
> > > tar/bz2'ing their patchsets and uploading them to the mirrors
> > > instead of including them where they are and making the portage
> > > tree bulge from the seams? there are patchsets that are currently
> > > in <pkgname>/files for packages that are only useful on a
> > > graphical workstation... for people that run servers, this seems
> > > like quite a bit of cruft. there are currently 100,712 files in
> > > /usr/portage... does this seem a little ridiculous to anyone else?
> > > </calm rant>
> > >
> > > <angry rant>
> > > why don't we put more stuff in mirror://gentoo/ ?? ${FILESDIR} has
> > > too much SHIT in it... i appreciate that patches, etc have been
> > > placed there... but we REALLY need to find a better place. I don't
> > > need my emerge sync's taking 45 minutes on average on a 5Mbit line
> > > in france... *45 minutes* ...just PLEASE keep the SHIT out of the
> > > portage tree.
> > >
> > > SHIT: (noun);
> > > 1. useless stuff that has no business being in my portage tree.
> > > 2. stuff that i'll never have use for... see #1
> > > 3. anything related to X11, kde, gnome, etc on a server.
> > > </angry rant>
> > >
> > > that's my 2/100ths of a monetary unit.
> > >
> > > --
> > > gentoo-dev@gentoo.org mailing list
> > >
> >
>
>
>
>
> --
> gentoo-dev@gentoo.org mailing list
>
>

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
On Monday 11 October 2004 18:14, Ned Ludd wrote:
> On Mon, 2004-10-11 at 01:19, Brian Jackson wrote:
> [snip]
>
> > E. If you really want to do some good, write a patch for repoman to kick
> > people in the "jimmy jammy" when they try to commit big files.
>
> A few revisions back ferringb added support for this per a request.
> repoman --pretend scan shows file sizes. (check it out in the binutils
> dir) It's non fatal and will probably remain that will till >=2.0.52 but
> after then no patch >=20k can go in the tree and we wont be able to
> commit on a dir unless it's cleaned up.

Please also add a check for bzipped gzipped files. CVS doesn't play nice with
them, and neither does rsync. Those shouldn't be in the tree (except for the
rescue portage)

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net
Re: rsync speed and space taken [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
| On Mon, 11 Oct 2004 09:25:19 -0700 Donnie Berkholz
| <spyderous@gentoo.org> wrote:
| | On Mon, 2004-10-11 at 17:17 +0100, Ciaran McCreesh wrote:
| | > That's going to get *really* painful when we're trying to just
| | > keyword a package without going around and changing other people's
| | > ebuilds. This shouldn't be introduced as an error until after the
| | > existing tree is entirely fixed.
| |
| | I doubt the tree will get fixed until it is an error.
|
| Well, if it *doesn't*, then I guess anyone who touches other people's
| ebuilds for, for example, keywording will have to just modify repoman...
| I'll post a patch if this turns out to be a problem.
|
Or we can have 1 person just go through and fix everyone's ebuilds on
this issue. There's only a hundred or so.

- --
Doug Goldstein
http://dev.gentoo.org/~cardoe

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x179106D0
Key fingerprint = 7001 5FBF BACE 9E66 3A1C 55E0 161C FF5C 1791 06D0

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBatI1Fhz/XBeRBtARAluwAJ9vpt3klyoA2guItEK5a1bei1H2GACfWXkM
dYlumwwErQcWPGKfzRtVLik=
=LGsp
-----END PGP SIGNATURE-----

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
On Mon, Oct 11, 2004 at 12:14:07PM -0400, Ned Ludd wrote:
>
> A few revisions back ferringb added support for this per a request.
> repoman --pretend scan shows file sizes. (check it out in the binutils
> dir) It's non fatal and will probably remain that will till >=2.0.52 but
> after then no patch >=20k can go in the tree and we wont be able to
> commit on a dir unless it's cleaned up.
>

That would really be a pain for arch maintainers who just want
to keyword the ebuild or something similar.

Ofcourse you can first go and find the dev and harass him, but
it's really annoying if you're in the middle of keywording.

I'd say simply refuse to add new files which are too large.


Christian

--
gentoo-dev@gentoo.org mailing list
Re: rsync speed and space taken [ In reply to ]
For the person asking, yes, the machine is I/O limited... cel 2.4 128M
ram 40G disk and LOTS of blog sites :(

~45 minutes nonetheless is a *long* time to wait for an emerge sync to
complete (or glsa-check for that matter). I second the suggestion of a
seperate portion of the rsync mirror SPECIFICALLY for patches. perhaps
with soft/hard-links from ${PN}/${FILESDIR} to ${PATCHDIR}/{$PN} ?
shouldn't be too much harder to just grab what is needed on-use...
carpaski? any ideas? perhaps a feature request for .52?

--
gentoo-dev@gentoo.org mailing list

1 2  View All