Mailing List Archive

Horrible performance
Me again.

I wrote here about my problems with mplayer stuttering during emerges.
Then I wrote that the problem went away when I installed Gentoo again,
moving from i686 to x86_64. But the problems are back, and worse than
ever. This is driving me crazy. CRAZY!

I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board Radeon HD
3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM volumes, all
encrypted, except for /usr/src and portage stuff. The system is ~amd64,
and I have -march=k8-sse3 in my CFLAGS. Current kernel is 2.6.34-tuxonice,
but I also tried others. I'm running KDE4 with desktop effects enabled, X
itself takes about 30-40% of CPU time according to top. After system
startup and login into KDE, 3.5G of RAM are occupied. This increases after
a while, and I need swap space. Nothing to worry about I think.

Performance does not feel too bad at first. But after a while, I cannot
even play videos during emerges. The playback stutters, sometimes I have
pauses for several seconds. As long as there is no swap space occpied,
it's not so bad I think. Maybe I have a probelm with disk I/O, and things
get much worse when swapping occurs. When I look at iotop, I see
various programs like chromium and various KDE applications appear. I
guess that's normal, but should not be noticeable. Hey, there were times
when I created a 2G tmpfs for /var/tmp/portage, with only 3G on
my 32bit system. BTW, I lowered my swappiness to 10. This helps a little I
think, because the swapping occurs later, the system is more responsive.

And it feels like things get worse and worse, it's not like there was a
specific point when I thought it's slow again. Like there were some
degragation going on - fragmentation, bitrot, I don't know. It's just how
it feels to me.

I am debugging this for some days now. I tried different kernels, from
2.6.29 to 2.6.35, including the kernel I had running after the switch to
64bit, when I thought all was fine. No change. But all kernels were
configured nearly identical, so I booted a GRML live-cd and used this
kernel .config as a template. Does not feel better.

When I thought the problem was gone, I had installed the system on my 2nd
1.5 TB drive. Meanwhile I copied the partitions back to the 1st drive, so
I suspected a difference in the drives. I use the 2nd drive for backups
(using rdiff-backup), with similar partitions, so I only have to exchange
the LVM volume group names of the two drives in order to run my system
from the 2nd one. I tried this, but it did not help.

And I have similar problems when copying data between some old PATA
drives. When I copy stuff and do a mkfs on another partition, mplayer
sometimes stutters and hangs for ten seconds. No joy. Working with KDE
sucks, switching dektops sometimes takes ages, and even now I am typing
faster than kmail can display the characters. That's with am emerge of
chromium running, with PORTAGE_NICENESS=10 and using ionice -c 3. Load is
around 8, but sometimes gets even higher. And then, load suddenly drops
back to lower values, as if somthing was blocking. Some applications
swapping, maybe.

Now I am out of ideas. I really hope someone here has one. I cannot work
with this system any more when emerges are going on.

I put my kernel config, make.conf, dmesg and such stuff to
http://www.wonkology.org/gentoo/ in case someone wants to have a look at
it. Any help is GREATLY appreciated.

Wonko
Re: Horrible performance [ In reply to ]
Hi Alex,

On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:
> Me again.
>
> I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board Radeon HD
> 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM volumes, all
> encrypted, except for /usr/src and portage stuff. The system is ~amd64,
> and I have -march=k8-sse3 in my CFLAGS. Current kernel is 2.6.34-tuxonice,
> but I also tried others. I'm running KDE4 with desktop effects enabled, X
> itself takes about 30-40% of CPU time according to top. After system
> startup and login into KDE, 3.5G of RAM are occupied. This increases after
> a while, and I need swap space. Nothing to worry about I think.

Encrypted filesystems can cause additional with activity, but I would expect
that to remain the same over a long period.
However, how is the write and read performance on those disks?

You're running KDE4, guess you went for the default and use mysql for "app-
office/akonadi-server".

I switched to using "sqlite" for this due to issues getting it to work with
mysql. I think this might help there?

> Performance does not feel too bad at first. But after a while, I cannot
> even play videos during emerges. The playback stutters, sometimes I have
> pauses for several seconds. As long as there is no swap space occpied,
> it's not so bad I think. Maybe I have a probelm with disk I/O, and things
> get much worse when swapping occurs. When I look at iotop, I see
> various programs like chromium and various KDE applications appear. I
> guess that's normal, but should not be noticeable. Hey, there were times
> when I created a 2G tmpfs for /var/tmp/portage, with only 3G on
> my 32bit system. BTW, I lowered my swappiness to 10. This helps a little I
> think, because the swapping occurs later, the system is more responsive.

Do you also encrypt swap?
Disk I/O is, in my experience, a very common cause for "freeze-ups".
Can you test with unencrypted disks to see if the issue occurs then as well?

> And I have similar problems when copying data between some old PATA
> drives. When I copy stuff and do a mkfs on another partition, mplayer
> sometimes stutters and hangs for ten seconds. No joy. Working with KDE
> sucks, switching dektops sometimes takes ages, and even now I am typing
> faster than kmail can display the characters. That's with am emerge of
> chromium running, with PORTAGE_NICENESS=10 and using ionice -c 3. Load is
> around 8, but sometimes gets even higher. And then, load suddenly drops
> back to lower values, as if somthing was blocking. Some applications
> swapping, maybe.

Very possibly, maybe an idea to check which applications are hogging the
memory. If it is the swapping of the system, then this will be caused by the
most memory-hungry processes.

Can you post the result of: "ps axu"?
This will give an indication which processes are running and using a lot of
memory.

> Now I am out of ideas. I really hope someone here has one. I cannot work
> with this system any more when emerges are going on.

Had similar issues with a desktop machine myself, managed to kill some
"features" that I wasn't using and this solved most of the problems.

> I put my kernel config, make.conf, dmesg and such stuff to
> http://www.wonkology.org/gentoo/ in case someone wants to have a look at
> it. Any help is GREATLY appreciated.

Lets see where checking for IO-speeds and memory-usage of your apps take us :)

--
Joost
Re: Horrible performance [ In reply to ]
On Tue, Aug 24, 2010 at 9:32 PM, Alex Schuster <wonko@wonkology.org> wrote:
> Now I am out of ideas. I really hope someone here has one. I cannot work
> with this system any more when emerges are going on.
>
> I put my kernel config, make.conf, dmesg and such stuff to
> http://www.wonkology.org/gentoo/ in case someone wants to have a look at
> it. Any help is GREATLY appreciated.

Just a thought: why -ggdb in your CFLAGS? If you have >=gcc-4.2, try:

CFLAGS="-march=native -O2 -pipe"

Then you should re-emerge gcc itself (twice?) and then world:

# emerge --oneshot binutils gcc virtual/libc
# emerge -e world

Best of luck.

--
Mansour Moufid
Re: Horrible performance [ In reply to ]
Dnia 2010-08-25, o godz. 03:32:40
Alex Schuster <wonko@wonkology.org> napisał(a):

> Me again.
>
> I wrote here about my problems with mplayer stuttering during
> emerges. Then I wrote that the problem went away when I installed
> Gentoo again, moving from i686 to x86_64. But the problems are back,
> and worse than ever. This is driving me crazy. CRAZY!
>
> I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board
> Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM
> volumes, all encrypted, except for /usr/src and portage stuff. The
> system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current
> kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4
> with desktop effects enabled, X itself takes about 30-40% of CPU time
> according to top. After system startup and login into KDE, 3.5G of
> RAM are occupied. This increases after a while, and I need swap
> space. Nothing to worry about I think.
>
> Performance does not feel too bad at first. But after a while, I
> cannot even play videos during emerges. The playback stutters,
> sometimes I have pauses for several seconds. As long as there is no
> swap space occpied, it's not so bad I think. Maybe I have a probelm
> with disk I/O, and things get much worse when swapping occurs. When I
> look at iotop, I see various programs like chromium and various KDE
> applications appear. I guess that's normal, but should not be
> noticeable. Hey, there were times when I created a 2G tmpfs
> for /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered
> my swappiness to 10. This helps a little I think, because the
> swapping occurs later, the system is more responsive.
>
> And it feels like things get worse and worse, it's not like there was
> a specific point when I thought it's slow again. Like there were some
> degragation going on - fragmentation, bitrot, I don't know. It's just
> how it feels to me.
>
> I am debugging this for some days now. I tried different kernels,
> from 2.6.29 to 2.6.35, including the kernel I had running after the
> switch to 64bit, when I thought all was fine. No change. But all
> kernels were configured nearly identical, so I booted a GRML live-cd
> and used this kernel .config as a template. Does not feel better.
>
> When I thought the problem was gone, I had installed the system on my
> 2nd 1.5 TB drive. Meanwhile I copied the partitions back to the 1st
> drive, so I suspected a difference in the drives. I use the 2nd drive
> for backups (using rdiff-backup), with similar partitions, so I only
> have to exchange the LVM volume group names of the two drives in
> order to run my system from the 2nd one. I tried this, but it did not
> help.
>
> And I have similar problems when copying data between some old PATA
> drives. When I copy stuff and do a mkfs on another partition, mplayer
> sometimes stutters and hangs for ten seconds. No joy. Working with
> KDE sucks, switching dektops sometimes takes ages, and even now I am
> typing faster than kmail can display the characters. That's with am
> emerge of chromium running, with PORTAGE_NICENESS=10 and using ionice
> -c 3. Load is around 8, but sometimes gets even higher. And then,
> load suddenly drops back to lower values, as if somthing was
> blocking. Some applications swapping, maybe.
>
> Now I am out of ideas. I really hope someone here has one. I cannot
> work with this system any more when emerges are going on.
>
> I put my kernel config, make.conf, dmesg and such stuff to
> http://www.wonkology.org/gentoo/ in case someone wants to have a look
> at it. Any help is GREATLY appreciated.
>
> Wonko
>

Checkout configuration of your video driver - X should not take more
than 5% of cpu when iddle (with some minor effects). Perhaps kde is not
using OpenGL? Do you have OpenGL enabled?

Amount of ram you have should be sufficent to compile everything
without access to swap space.

--
Kacper Kopczyński
Re: Horrible performance [ In reply to ]
Mansour Moufid writes:

> On Tue, Aug 24, 2010 at 9:32 PM, Alex Schuster <wonko@wonkology.org>
> wrote:

> Just a thought: why -ggdb in your CFLAGS? If you have >=gcc-4.2, try:
>
> CFLAGS="-march=native -O2 -pipe"

I just added this some days ago in order to give debug information for a a
bug in strigi I had reported. I forgot to take it out, but so far only few
packages were compiled with this setting. But thanks for mentioning this,
I removed the debug setting now.
But should it matter? Optimization still happens. And when these
interrupts hapen, the CPU is not at 100%.

I do not use march=native because I sometimes use another host with
distcc.

Wonko
Re: Horrible performance [ In reply to ]
Kacper Kopczyński writes:

> Checkout configuration of your video driver - X should not take more
> than 5% of cpu when iddle (with some minor effects). Perhaps kde is not
> using OpenGL? Do you have OpenGL enabled?

I'm using ati-drivers-10.7 with xorg-server-1.7.7-r1. No special settings
in xorg.conf. I never had much success with the open source radeon
drivers. First they did not run at all, I either did not get X started, or
I only got a blank screen. One month ago then I had success with xf86-
video-ati-6.13.1, but X crashed instantly when I moved the mouse cursor
onto the KDE panel. But now I have a new KDe version, so I wil give them a
try soon.

Wow, at the moment X takes 95%. When I disable desktop effects, it's down
to 5%. When enable desktop effects again, X uses 60%.

Now I went into systemsettings -> desktop effects -> all effects, and
played around with the individual settings. When I disable the blur effect
(two lines after transparency), X usage is at less than 20%. Weird, I do
not even see any difference this effect makes. Cool, much less X usage by
disabling something I did not even notice :)

Is 15-25% X usage still too much? I have several plasmoids running, so I'm
not surprised this eats some performance. BTW, at [*] there are some
screenshots of my desktop, which did not change much since I did the
screenshots.

The system already feels better now. But this cannot be the only problem,
I sometimes (especially during large emerges) turn desktop effects off,
and when I have too much load, they are turned off automatically.

> Amount of ram you have should be sufficent to compile everything
> without access to swap space.

Damn right. I used to have less memory, and ran very memory-intensive
applications, and the system was much more responsive than now.

Wonko

[*] http://www.wonkology.org/comp/desktop/2010-06-19/
Re: Horrible performance [ In reply to ]
On Wed, Aug 25, 2010 at 7:09 AM, Alex Schuster <wonko@wonkology.org> wrote:
> I just added this some days ago in order to give debug information for a a
> bug in strigi I had reported. I forgot to take it out, but so far only few
> packages were compiled with this setting. But thanks for mentioning this,
> I removed the debug setting now.
> But should it matter? Optimization still happens. And when these
> interrupts hapen, the CPU is not at 100%.
>
> I do not use march=native because I sometimes use another host with
> distcc.

Oh, I thought that might have been behind the memory issue...

But actually it sounds like you don't have direct rendering? Check
with glxinfo (from x11-apps/mesa-progs):

$ glxinfo | grep -i direct

If not then check out this thread in the forums for the correct kernel
configuration:
<http://forums.gentoo.org/viewtopic-t-840613-highlight-radeon+3200.html>

If that works you can also try building the KDE libraries with -Os to
help with memory. I don't think you should be using swap at all.

--
Mansour Moufid
Re: Horrible performance [ In reply to ]
J. Roeleveld writes:

> On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:

> > I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board
> > Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM
> > volumes, all encrypted, except for /usr/src and portage stuff. The
> > system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current
> > kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4
> > with desktop effects enabled, X itself takes about 30-40% of CPU
> > time according to top. After system startup and login into KDE, 3.5G
> > of RAM are occupied. This increases after a while, and I need swap
> > space. Nothing to worry about I think.
>
> Encrypted filesystems can cause additional with activity, but I would
> expect that to remain the same over a long period.

And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
Can LVM create noticeable overhead? I also resized my logical volumes a
couple of times, could this lead to some LVM fragmentation?

> However, how is the write and read performance on those disks?

Here's the output of hdparm -t for all drives, 4 times.

/dev/sda: (SATA system drive)
Timing buffered disk reads: 118 MB in 3.08 seconds = 38.37 MB/sec
Timing buffered disk reads: 194 MB in 3.11 seconds = 62.47 MB/sec
Timing buffered disk reads: 322 MB in 3.01 seconds = 106.82 MB/sec
Timing buffered disk reads: 244 MB in 3.00 seconds = 81.21 MB/sec

/dev/sdb: (PATA master)
Timing buffered disk reads: 114 MB in 3.02 seconds = 37.70 MB/sec
Timing buffered disk reads: 114 MB in 3.00 seconds = 37.97 MB/sec
Timing buffered disk reads: 116 MB in 3.05 seconds = 38.06 MB/sec
Timing buffered disk reads: 116 MB in 3.05 seconds = 38.07 MB/sec

/dev/sdc: (PATA slave)
Timing buffered disk reads: 164 MB in 3.03 seconds = 54.21 MB/sec
Timing buffered disk reads: 166 MB in 3.02 seconds = 55.04 MB/sec
Timing buffered disk reads: 166 MB in 3.01 seconds = 55.10 MB/sec
Timing buffered disk reads: 158 MB in 3.01 seconds = 52.41 MB/sec

/dev/sdd: (SATA backup drive)
Timing buffered disk reads: 314 MB in 3.00 seconds = 104.55 MB/sec
Timing buffered disk reads: 312 MB in 3.01 seconds = 103.67 MB/sec
Timing buffered disk reads: 308 MB in 3.01 seconds = 102.34 MB/sec
Timing buffered disk reads: 314 MB in 3.00 seconds = 104.60 MB/sec

The system drive throughput varies a lot, depending on other I/O.

I


> You're running KDE4, guess you went for the default and use mysql for
> "app- office/akonadi-server".
>
> I switched to using "sqlite" for this due to issues getting it to work
> with mysql. I think this might help there?

So I only have to set the sqlite use flag and remove the mysql use flag
for akonadi-server? I'm doing this now. And this also gives an example for
what is going on here:

I have just logged into KDE. I did not log out since yesterday, and when
started a VM with vmplayer, the system swapped like crazy, I could not use
it for minutes. After this, the panel did not react any more, and the
desktop background did not redraw, so I logged out and in again. The VM
started fine now. Well, could be faster, but maybe it was okay.
Then I started answering your mail, and tried to reemerge akonadi-server,
but I had a type, so portage took a long search for akomadi-server.
meanwhile the dektop became quite unresponsive, load went high, and I made
a screenshot [*]. If you look at the top right, gkrellm shows this above
'Proc'. The first increase at the left was after I started emerge, the 2nd
at the right was after I pressed the PrtSc key.


> > Performance does not feel too bad at first. But after a while, I
> > cannot even play videos during emerges. The playback stutters,
> > sometimes I have pauses for several seconds. As long as there is no
> > swap space occpied, it's not so bad I think. Maybe I have a probelm
> > with disk I/O, and things get much worse when swapping occurs. When
> > I look at iotop, I see various programs like chromium and various
> > KDE applications appear. I guess that's normal, but should not be
> > noticeable. Hey, there were times when I created a 2G tmpfs for
> > /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered my
> > swappiness to 10. This helps a little I think, because the swapping
> > occurs later, the system is more responsive.
>
> Do you also encrypt swap?

Yes.

> Disk I/O is, in my experience, a very common cause for "freeze-ups".
> Can you test with unencrypted disks to see if the issue occurs then as
> well?

Yes, I can do this. It's some work, but I tried so much, why not this. I
have some free space, and already have written a backup script that
automatically creates LVM snapshots, decrypts them, and backs it up, so I
can do this from the running system.


> > And I have similar problems when copying data between some old PATA
> > drives. When I copy stuff and do a mkfs on another partition, mplayer
> > sometimes stutters and hangs for ten seconds. No joy. Working with
> > KDE sucks, switching dektops sometimes takes ages, and even now I am
> > typing faster than kmail can display the characters. That's with am
> > emerge of chromium running, with PORTAGE_NICENESS=10 and using
> > ionice -c 3. Load is around 8, but sometimes gets even higher. And
> > then, load suddenly drops back to lower values, as if somthing was
> > blocking. Some applications swapping, maybe.
>
> Very possibly, maybe an idea to check which applications are hogging
> the memory. If it is the swapping of the system, then this will be
> caused by the most memory-hungry processes.
>
> Can you post the result of: "ps axu"?
> This will give an indication which processes are running and using a
> lot of memory.

First, here is free -m:
total used free shared buffers cached
Mem: 3452 3225 226 0 54 325
-/+ buffers/cache: 2844 607
Swap: 4094 935 3159

And here the output of top, sorted by memory. I think it is a similar
output to ps axu, but more condensed and better readable via mail. The
full ps aux output, sorted by memory, is in [2].

top - 21:23:03 up 1 day, 7 min, 11 users, load average: 0.04, 0.05, 0.02
Tasks: 418 total, 1 running, 417 sleeping, 0 stopped, 0 zombie
Cpu(s): 17.8%us, 12.7%sy, 9.9%ni, 45.9%id, 13.5%wa, 0.1%hi, 0.1%si, 0.0%st
Mem: 3534936k total, 3312312k used, 222624k free, 56264k buffers
Swap: 4193272k total, 957908k used, 3235364k free, 334048k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25993 root 20 0 847m 475m 47m S 21 13.8 49:01.11 X
26553 wonko 20 0 1312m 235m 10m S 0 6.8 0:28.32 java
26925 wonko 20 0 698m 112m 18m S 0 3.2 3:30.75 kontact
26961 wonko 20 0 526m 79m 13m S 0 2.3 2:16.01 chrome
26566 wonko 20 0 1128m 72m 10m S 0 2.1 1:02.46 amarok
27056 wonko 20 0 871m 57m 19m S 0 1.7 0:05.07 chrome
30324 root 30 10 195m 55m 1136 S 0 1.6 0:09.46 emerge
27051 wonko 20 0 874m 50m 8668 S 0 1.5 0:14.16 chrome
26126 wonko 20 0 1028m 44m 11m S 2 1.3 9:18.94 plasma-desktop
27046 wonko 20 0 878m 41m 7324 S 0 1.2 0:16.59 chrome
27122 wonko 20 0 871m 36m 7660 S 0 1.0 0:40.03 chrome
27036 wonko 20 0 865m 29m 7492 S 0 0.8 0:00.94 chrome
27198 wonko 20 0 860m 28m 6240 S 0 0.8 0:04.76 chrome
26101 wonko 20 0 427m 28m 8672 S 4 0.8 5:27.48 kwin
26298 wonko 20 0 396m 26m 9.9m S 0 0.8 0:02.91 knotes
27766 wonko 20 0 857m 26m 5924 S 0 0.8 0:03.03 chrome
26903 root 20 0 64316 25m 288 S 0 0.7 0:00.33 screen
27203 wonko 20 0 865m 24m 5960 S 0 0.7 0:04.06 chrome
30226 wonko 20 0 367m 23m 9232 S 0 0.7 0:03.21 gwenview
26221 wonko 20 0 609m 23m 3752 S 0 0.7 0:02.55 knotify4

X takes an awful lot, then comes java, which is running only for my
tvbrowser. And many many chrome processes.

> > Now I am out of ideas. I really hope someone here has one. I cannot
> > work with this system any more when emerges are going on.
>
> Had similar issues with a desktop machine myself, managed to kill some
> "features" that I wasn't using and this solved most of the problems.

I hope I can say this soon, too.

> Lets see where checking for IO-speeds and memory-usage of your apps
> take us :)

Thanks for your help! I appreciate this very much.

Wonko

[1] http://www.wonkology.org/comp/desktop/2010-08-25_emerge-akomadi.png
[2] http://www.wonkology.org/gentoo/
Re: Horrible performance [ In reply to ]
Mansour Moufid writes:

> But actually it sounds like you don't have direct rendering? Check
> with glxinfo (from x11-apps/mesa-progs):
>
> $ glxinfo | grep -i direct

Thanks, but opengl is running fine. Well, it took me quite a while until I
got it working, but finally there was a version of ati-drivers that
worked, and from then on it kept working fine.


> If that works you can also try building the KDE libraries with -Os to
> help with memory. I don't think you should be using swap at all.

Me too. So -Os would only be a workaround to make the probelm less bad. I
hope I find a better solution.

Wonko
Re: Horrible performance [ In reply to ]
J. Roeleveld writes:

> Can you post the result of: "ps axu"?
> This will give an indication which processes are running and using a
> lot of memory.

The fifth colums gives the memory, right? Should this add up to the total
of the 'used' column in free -m?

Because it does not:

wonko@weird ~ $ free -m
total used free shared buffers cached
Mem: 3452 3117 334 0 39 279
-/+ buffers/cache: 2798 653
Swap: 4094 1472 2622

wonko@weird ~ $ total=0; for rss in $( ps aux | grep -v USER | awk '{print
$6}' ); do (( total += rss )); done; echo $(( total / 1024 ))
1984

So, I get a sum of around 2 G with ps, while free -m shows 4.5 G. Whoops?
Am I missing something here, or does it look like lots of RAM is not being
freed?

Wonko
Re: Horrible performance [ In reply to ]
I think I just found the problem! I decided to give the open source ATI
drivers a try again, and replaced 'fglrx' by 'radeon' in my xorg.conf. And
all is fast now. Wow. I had already forgotten how fast a desktop should
be.

I'm not yet sure all is fine. On the first start, kwin crashed and all
windows were on the first desktop, this never happened before. But it's
working on the second login. I have no direct rendering, Xorg.0.log shows
(EE) RADEON(0): [dri] RADEONDRIGetVersion failed (libdri too old). I had
this before with xorg-server-1.7.7, now I'm emerging 1.8.2. Stay tuned.

Wonko
Re: Horrible performance [ In reply to ]
On Wednesday 25 August 2010 21:38:10 Alex Schuster wrote:
> J. Roeleveld writes:
> > On Wednesday 25 August 2010 03:32:40 Alex Schuster wrote:
> > > I have an AMD Athlon(tm) Dual Core Processor 4850e CPU, on-board
> > > Radeon HD 3200 graphics, 4GB of memory, an 1.5 TB drive. Lots of LVM
> > > volumes, all encrypted, except for /usr/src and portage stuff. The
> > > system is ~amd64, and I have -march=k8-sse3 in my CFLAGS. Current
> > > kernel is 2.6.34-tuxonice, but I also tried others. I'm running KDE4
> > > with desktop effects enabled, X itself takes about 30-40% of CPU
> > > time according to top. After system startup and login into KDE, 3.5G
> > > of RAM are occupied. This increases after a while, and I need swap
> > > space. Nothing to worry about I think.
> >
> > Encrypted filesystems can cause additional with activity, but I would
> > expect that to remain the same over a long period.
>
> And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
> Can LVM create noticeable overhead? I also resized my logical volumes a
> couple of times, could this lead to some LVM fragmentation?

Theoretically, LVM will create an additional overhead.
But I am extensibly using LVM on all my machines and haven't noticed any
significant performance drops.

LVM-fragmentation is a definite possibility.
To defragment it, have a look at the following:

http://bisqwit.iki.fi/source/lvm2defrag.html
http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-defrag-
or-move-based-on-logical-volumes-689335/

I played with the first one on an older machine once and it does work quite
nicely.

> > However, how is the write and read performance on those disks?
>
> Here's the output of hdparm -t for all drives, 4 times.
>
> /dev/sda: (SATA system drive)
> Timing buffered disk reads: 118 MB in 3.08 seconds = 38.37 MB/sec
> Timing buffered disk reads: 194 MB in 3.11 seconds = 62.47 MB/sec
> Timing buffered disk reads: 322 MB in 3.01 seconds = 106.82 MB/sec
> Timing buffered disk reads: 244 MB in 3.00 seconds = 81.21 MB/sec
>
> /dev/sdb: (PATA master)
> Timing buffered disk reads: 114 MB in 3.02 seconds = 37.70 MB/sec
> Timing buffered disk reads: 114 MB in 3.00 seconds = 37.97 MB/sec
> Timing buffered disk reads: 116 MB in 3.05 seconds = 38.06 MB/sec
> Timing buffered disk reads: 116 MB in 3.05 seconds = 38.07 MB/sec
>
> /dev/sdc: (PATA slave)
> Timing buffered disk reads: 164 MB in 3.03 seconds = 54.21 MB/sec
> Timing buffered disk reads: 166 MB in 3.02 seconds = 55.04 MB/sec
> Timing buffered disk reads: 166 MB in 3.01 seconds = 55.10 MB/sec
> Timing buffered disk reads: 158 MB in 3.01 seconds = 52.41 MB/sec
>
> /dev/sdd: (SATA backup drive)
> Timing buffered disk reads: 314 MB in 3.00 seconds = 104.55 MB/sec
> Timing buffered disk reads: 312 MB in 3.01 seconds = 103.67 MB/sec
> Timing buffered disk reads: 308 MB in 3.01 seconds = 102.34 MB/sec
> Timing buffered disk reads: 314 MB in 3.00 seconds = 104.60 MB/sec
>
> The system drive throughput varies a lot, depending on other I/O.

Those look ok to me, except that I would expect SATA-drives to be faster then
PATA drives.

> > You're running KDE4, guess you went for the default and use mysql for
> > "app- office/akonadi-server".
> >
> > I switched to using "sqlite" for this due to issues getting it to work
> > with mysql. I think this might help there?
>
> So I only have to set the sqlite use flag and remove the mysql use flag
> for akonadi-server? I'm doing this now. And this also gives an example for
> what is going on here:

And unset mysql.
There is one issue that needs to be resolved manually with getting it to work
with sqlite.
See:
http://forums.gentoo.org/viewtopic-t-834883-view-
next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4

Here is the bit that will solve the problem:
--
$ cd ~/.local/share/akonadi
$ sqlite3 akonadi.db
('akonadi_search_resource', 1);
--

> I have just logged into KDE. I did not log out since yesterday, and when
> started a VM with vmplayer, the system swapped like crazy, I could not use
> it for minutes. After this, the panel did not react any more, and the
> desktop background did not redraw, so I logged out and in again. The VM
> started fine now. Well, could be faster, but maybe it was okay.
> Then I started answering your mail, and tried to reemerge akonadi-server,
> but I had a type, so portage took a long search for akomadi-server.
> meanwhile the dektop became quite unresponsive, load went high, and I made
> a screenshot [*]. If you look at the top right, gkrellm shows this above
> 'Proc'. The first increase at the left was after I started emerge, the 2nd
> at the right was after I pressed the PrtSc key.

VMWare allows virtual machines to use more memory then is actually available.
Also, there are settings in VMWare (possibly enabled by default) that cause
the memory to be duplicated onto disk.
This can cause issues like you are seeing.

> > > Performance does not feel too bad at first. But after a while, I
> > > cannot even play videos during emerges. The playback stutters,
> > > sometimes I have pauses for several seconds. As long as there is no
> > > swap space occpied, it's not so bad I think. Maybe I have a probelm
> > > with disk I/O, and things get much worse when swapping occurs. When
> > > I look at iotop, I see various programs like chromium and various
> > > KDE applications appear. I guess that's normal, but should not be
> > > noticeable. Hey, there were times when I created a 2G tmpfs for
> > > /var/tmp/portage, with only 3G on my 32bit system. BTW, I lowered my
> > > swappiness to 10. This helps a little I think, because the swapping
> > > occurs later, the system is more responsive.
> >
> > Do you also encrypt swap?
>
> Yes.
>
> > Disk I/O is, in my experience, a very common cause for "freeze-ups".
> > Can you test with unencrypted disks to see if the issue occurs then as
> > well?
>
> Yes, I can do this. It's some work, but I tried so much, why not this. I
> have some free space, and already have written a backup script that
> automatically creates LVM snapshots, decrypts them, and backs it up, so I
> can do this from the running system.

Ok, am interested to see if running unencrypted actually has benefits here.

> > > And I have similar problems when copying data between some old PATA
> > > drives. When I copy stuff and do a mkfs on another partition, mplayer
> > > sometimes stutters and hangs for ten seconds. No joy. Working with
> > > KDE sucks, switching dektops sometimes takes ages, and even now I am
> > > typing faster than kmail can display the characters. That's with am
> > > emerge of chromium running, with PORTAGE_NICENESS=10 and using
> > > ionice -c 3. Load is around 8, but sometimes gets even higher. And
> > > then, load suddenly drops back to lower values, as if somthing was
> > > blocking. Some applications swapping, maybe.
> >
> > Very possibly, maybe an idea to check which applications are hogging
> > the memory. If it is the swapping of the system, then this will be
> > caused by the most memory-hungry processes.
> >
> > Can you post the result of: "ps axu"?
> > This will give an indication which processes are running and using a
> > lot of memory.
>
> First, here is free -m:
> total used free shared buffers cached
> Mem: 3452 3225 226 0 54 325
> -/+ buffers/cache: 2844 607
> Swap: 4094 935 3159
>
> And here the output of top, sorted by memory. I think it is a similar
> output to ps axu, but more condensed and better readable via mail. The
> full ps aux output, sorted by memory, is in [2].
>
<snipped top>
>
> X takes an awful lot, then comes java, which is running only for my
> tvbrowser. And many many chrome processes.

How many web browser windows do you have open? :)

Also, do you have file indexing enabled?

> > > Now I am out of ideas. I really hope someone here has one. I cannot
> > > work with this system any more when emerges are going on.
> >
> > Had similar issues with a desktop machine myself, managed to kill some
> > "features" that I wasn't using and this solved most of the problems.
>
> I hope I can say this soon, too.

In my experience, X uses more memory when a lot of windows are open.
And yours uses about 4 times as much as mine.
But then again, I don't have much running at the moment.

> > Lets see where checking for IO-speeds and memory-usage of your apps
> > take us :)
>
> Thanks for your help! I appreciate this very much.

I'll do my best :)

--
Joost
Re: Horrible performance [ In reply to ]
On Thursday 26 August 2010 01:06:59 Alex Schuster wrote:
> J. Roeleveld writes:
> > Can you post the result of: "ps axu"?
> > This will give an indication which processes are running and using a
> > lot of memory.
>
> The fifth colums gives the memory, right? Should this add up to the total
> of the 'used' column in free -m?
>
> Because it does not:
>
> wonko@weird ~ $ free -m
> total used free shared buffers cached
> Mem: 3452 3117 334 0 39 279
> -/+ buffers/cache: 2798 653
> Swap: 4094 1472 2622
>
> wonko@weird ~ $ total=0; for rss in $( ps aux | grep -v USER | awk '{print
> $6}' ); do (( total += rss )); done; echo $(( total / 1024 ))
> 1984
>
> So, I get a sum of around 2 G with ps, while free -m shows 4.5 G. Whoops?
> Am I missing something here, or does it look like lots of RAM is not being
> freed?
>
> Wonko

Don't forget the buffers/cache.
The 2.5G you're missing is what is used in the buffers/cache line.

At least, that is how I see it.
Also, I generally look at the percentages used for the memory to find the
memory-hogs.

And your tv-viewer (java) is using quite a bit as well.
Maybe someone with more experience with tv-viewer apps can take a look and
maybe give a few pointers?

--
Joost
Re: Horrible performance [ In reply to ]
On Thursday 26 August 2010 02:11:44 Alex Schuster wrote:
> I think I just found the problem! I decided to give the open source ATI
> drivers a try again, and replaced 'fglrx' by 'radeon' in my xorg.conf. And
> all is fast now. Wow. I had already forgotten how fast a desktop should
> be.
>
> I'm not yet sure all is fine. On the first start, kwin crashed and all
> windows were on the first desktop, this never happened before. But it's
> working on the second login. I have no direct rendering, Xorg.0.log shows
> (EE) RADEON(0): [dri] RADEONDRIGetVersion failed (libdri too old). I had
> this before with xorg-server-1.7.7, now I'm emerging 1.8.2. Stay tuned.
>
> Wonko

Ok, looks like your video-driver was using up too much resources.
Either with all the desktop-effects or with something else.

I can't help further with ATI-issues as I don't use them myself.

I do know that I need to change more then just the driver in my xorg.conf when
using nvidie-drivers. Am wondering if your crashes occured because of a
similar issue.
Eg. the "dri" module is only for the ATI-driver and not for the open-source
one?

--
Joost
Re: Horrible performance [ In reply to ]
Update:

> I think I just found the problem! I decided to give the open source ATI
> drivers a try again, and replaced 'fglrx' by 'radeon' in my xorg.conf.
> And all is fast now. Wow. I had already forgotten how fast a desktop
> should be.
>
> I'm not yet sure all is fine. On the first start, kwin crashed and all
> windows were on the first desktop, this never happened before. But it's
> working on the second login. I have no direct rendering, Xorg.0.log
> shows (EE) RADEON(0): [dri] RADEONDRIGetVersion failed (libdri too
> old). I had this before with xorg-server-1.7.7, now I'm emerging
> 1.8.2. Stay tuned.

I first got no dri with xorg-server 1.8, too, and then weird effects
started happening, like the shift and ctrl keys not working. When tings
crashed, I got a block screen and still could move the mouse, but that was
all, even after Alt-SysRQ-R (which usually helps in these cases) I could
not switch to a text terminal. But after a reboot all seemed perfect, but
when I was writing a followe-up mail, X crashed. And I got several more
crashes after working 5 minutes in KDE.

I downgraded back to xorg-server 1.7.7-r1, and now dri is working, too.
Looking good! KDE is running for over an hour now, I'm not using any swap
at all, amarok is emerging in the background, and mplayer is playing
smoothly.

I experience little rendering problems in chromium (images appear over
text) and quake3 feels a little sluggish, but I can live with this.

Thanks for anyone who helped!

Wonko
Re: Horrible performance [ In reply to ]
On Thursday 26 August 2010 11:34:23 Alex Schuster wrote:
> Update:

> I first got no dri with xorg-server 1.8, too, and then weird effects
> started happening, like the shift and ctrl keys not working. When tings
> crashed, I got a block screen and still could move the mouse, but that was
> all, even after Alt-SysRQ-R (which usually helps in these cases) I could
> not switch to a text terminal. But after a reboot all seemed perfect, but
> when I was writing a followe-up mail, X crashed. And I got several more
> crashes after working 5 minutes in KDE.
>
> I downgraded back to xorg-server 1.7.7-r1, and now dri is working, too.
> Looking good! KDE is running for over an hour now, I'm not using any swap
> at all, amarok is emerging in the background, and mplayer is playing
> smoothly.
>
> I experience little rendering problems in chromium (images appear over
> text) and quake3 feels a little sluggish, but I can live with this.
>
> Thanks for anyone who helped!
>
> Wonko

Glad you got it working properly now :)

Enjoy the smoother desktop now

--
Joost
Re: Horrible performance [ In reply to ]
Joost Roeleveld writes:

> On Thursday 26 August 2010 02:11:44 Alex Schuster wrote:
> > I think I just found the problem! I decided to give the open source
> > ATI drivers a try again, and replaced 'fglrx' by 'radeon' in my
> > xorg.conf. And all is fast now. Wow. I had already forgotten how
> > fast a desktop should be.
> >
> > I'm not yet sure all is fine. On the first start, kwin crashed and
> > all windows were on the first desktop, this never happened before.
> > But it's working on the second login. I have no direct rendering,
> > Xorg.0.log shows (EE) RADEON(0): [dri] RADEONDRIGetVersion failed
> > (libdri too old). I had this before with xorg-server-1.7.7, now I'm
> > emerging 1.8.2. Stay tuned.
> >
> > Wonko
>
> Ok, looks like your video-driver was using up too much resources.
> Either with all the desktop-effects or with something else.

I often had desktop effects turned off.

> I can't help further with ATI-issues as I don't use them myself.
>
> I do know that I need to change more then just the driver in my
> xorg.conf when using nvidie-drivers. Am wondering if your crashes
> occured because of a similar issue.
> Eg. the "dri" module is only for the ATI-driver and not for the
> open-source one?

I did not configure modules like dri in my xorg.conf, this seems tzo work
automatically. With ati-drivers, I got errors in Xorg.0.log about dri and
dri2 missing, apparently those come with the fglrx driver. With radeon, I
get messages that dr and dri2 are loaded.
So I only had to replace 'fglrx' by 'radeon' inthe device section.
And to reboot. Or something. At least it did not work until I did that. I
had loaded the drm kernel module by hand, so this was not the problem.
Whatever, I'm glad the radeon driver is runnign fine now, for the first
time after several tries.

Wonko
Re: Horrible performance [ In reply to ]
Joost Roeleveld writes:

> On Wednesday 25 August 2010 21:38:10 Alex Schuster wrote:

> > And I just moved my PORTAGE_TMPDIR to an unencrypted partition.
> > Can LVM create noticeable overhead? I also resized my logical volumes
> > a couple of times, could this lead to some LVM fragmentation?
>
> Theoretically, LVM will create an additional overhead.
> But I am extensibly using LVM on all my machines and haven't noticed
> any significant performance drops.

That's also what I heard.

> LVM-fragmentation is a definite possibility.
> To defragment it, have a look at the following:
>
> http://bisqwit.iki.fi/source/lvm2defrag.html

Cool!

> http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-
> defrag- or-move-based-on-logical-volumes-689335/
>
> I played with the first one on an older machine once and it does work
> quite nicely.

Any idea how to check how bad the fragmentation actually is?

BTW:
wonko@weird ~ $ mount | wc -l
48

I use LVM for about everything now, it makes things so much easier. First,
I had two volume groups on my system drive, one for the system, placed at
the front where the drive is supposed to be faster, and one for data. But
I don't do this any more, it cuts down flexibility, and is probably not
worth the effort.

> > > However, how is the write and read performance on those disks?
> >
> > Here's the output of hdparm -t for all drives, 4 times.
> >
> > /dev/sda: (SATA system drive)
> >
> > Timing buffered disk reads: 118 MB in 3.08 seconds = 38.37 MB/sec
> > Timing buffered disk reads: 194 MB in 3.11 seconds = 62.47 MB/sec
> > Timing buffered disk reads: 322 MB in 3.01 seconds = 106.82 MB/sec
> > Timing buffered disk reads: 244 MB in 3.00 seconds = 81.21 MB/sec
> >
> > /dev/sdb: (PATA master)
> >
> > Timing buffered disk reads: 114 MB in 3.02 seconds = 37.70 MB/sec
> > Timing buffered disk reads: 114 MB in 3.00 seconds = 37.97 MB/sec
> > Timing buffered disk reads: 116 MB in 3.05 seconds = 38.06 MB/sec
> > Timing buffered disk reads: 116 MB in 3.05 seconds = 38.07 MB/sec
> >
> > /dev/sdc: (PATA slave)
> >
> > Timing buffered disk reads: 164 MB in 3.03 seconds = 54.21 MB/sec
> > Timing buffered disk reads: 166 MB in 3.02 seconds = 55.04 MB/sec
> > Timing buffered disk reads: 166 MB in 3.01 seconds = 55.10 MB/sec
> > Timing buffered disk reads: 158 MB in 3.01 seconds = 52.41 MB/sec
> >
> > /dev/sdd: (SATA backup drive)
> >
> > Timing buffered disk reads: 314 MB in 3.00 seconds = 104.55 MB/sec
> > Timing buffered disk reads: 312 MB in 3.01 seconds = 103.67 MB/sec
> > Timing buffered disk reads: 308 MB in 3.01 seconds = 102.34 MB/sec
> > Timing buffered disk reads: 314 MB in 3.00 seconds = 104.60 MB/sec
> >
> > The system drive throughput varies a lot, depending on other I/O.
>
> Those look ok to me, except that I would expect SATA-drives to be
> faster then PATA drives.

Well, they are, except for the sometimes busy system drive. And by now I
get similar results as for the 2nd SATA drive, throughput is between 90
and 110 MB/sec.


> > > You're running KDE4, guess you went for the default and use mysql
> > > for "app- office/akonadi-server".
> > >
> > > I switched to using "sqlite" for this due to issues getting it to
> > > work with mysql. I think this might help there?
> >
> > So I only have to set the sqlite use flag and remove the mysql use
> > flag for akonadi-server? I'm doing this now. And this also gives an
> > example for what is going on here:
> And unset mysql.
> There is one issue that needs to be resolved manually with getting it
> to work with sqlite.
> See:
> http://forums.gentoo.org/viewtopic-t-834883-view-
> next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4
>
> Here is the bit that will solve the problem:
> --
> $ cd ~/.local/share/akonadi
> $ sqlite3 akonadi.db
> sqlite> INSERT INTO ResourceTable (name, isVirtual) VALUES
> ('akonadi_search_resource', 1);
> sqlite> .exit

I did nothing about this by now, but I probabyl will soon. Thanks for the
tip!
I also had trouble with akonadi in the past, and it still gives
warnings/errors at every startup, but at least it seems to work now.


> > > Disk I/O is, in my experience, a very common cause for
> > > "freeze-ups". Can you test with unencrypted disks to see if the
> > > issue occurs then as well?
> >
> > Yes, I can do this. It's some work, but I tried so much, why not
> > this. I have some free space, and already have written a backup
> > script that automatically creates LVM snapshots, decrypts them, and
> > backs it up, so I can do this from the running system.
>
> Ok, am interested to see if running unencrypted actually has benefits
> here.

Me too, but as things are quite faster now already, the priority for this
task is much lower than it was yesterday :)


> > > Can you post the result of: "ps axu"?
> > > This will give an indication which processes are running and using
> > > a lot of memory.
> >
> > First, here is free -m:
> > total used free shared buffers
> > cached
> >
> > Mem: 3452 3225 226 0 54
> > 325 -/+ buffers/cache: 2844 607
> > Swap: 4094 935 3159
> >
> > And here the output of top, sorted by memory. I think it is a similar
> > output to ps axu, but more condensed and better readable via mail.
> > The full ps aux output, sorted by memory, is in [2].
>
> <snipped top>
>
> > X takes an awful lot, then comes java, which is running only for my
> > tvbrowser. And many many chrome processes.
>
> How many web browser windows do you have open? :)

wonko@weird ~ $ ps ax | grep chromium-browser | wc -l
42

Well, that's tabs, not windows. Chromium uses one process for each tab.
I'm not sure if I really want to use Chromium, I started using it because
of a nasty konqueror bug that made KDE freeze occasionally, but it seems
to be fixed now.

> Also, do you have file indexing enabled?

Do you mean Strigi/Nepomuk? I just turned it on again today. It never
worked, but crashed. In the last days I investigated this further and
filed two bugs about this, and they were already fixed. I'm running the
git version now, and so far it seems to work fine, but my home directory
is not fully indexed yet. And there were also issues with my music folder,
let's see if indexing this will still make strigi crash.
I should have done this before - I am waiting over a year now for a fixed
version, but apparently few people if any have those crashes, so I had to
report them myself.
At the moment it's indexing, but so far system load is not very high.
That's much better than a year ago when I used it the last time.


> > > > Now I am out of ideas. I really hope someone here has one. I
> > > > cannot work with this system any more when emerges are going on.
> > >
> > > Had similar issues with a desktop machine myself, managed to kill
> > > some "features" that I wasn't using and this solved most of the
> > > problems.
> >
> > I hope I can say this soon, too.
>
> In my experience, X uses more memory when a lot of windows are open.
> And yours uses about 4 times as much as mine.
> But then again, I don't have much running at the moment.

I like to have many things open. With enough memory it's no problem.
Unless you are using ati-drivers, that is.

Wonko
Re: Horrible performance [ In reply to ]
On Thursday 26 August 2010 17:48:44 Alex Schuster wrote:
> Joost Roeleveld writes:
> > LVM-fragmentation is a definite possibility.
> > To defragment it, have a look at the following:
> >
> > http://bisqwit.iki.fi/source/lvm2defrag.html
>
> Cool!

Do test it first and check what it wants to do.
Basically, it moves all the blocks around untill you have them all in the
sequence you want them.
There are some limitations, but it worked when I tested it.
Btw, I provide no warranty what-so-ever, especially as I did not write any
part of it :)

> > http://www.linuxquestions.org/questions/linux-software-2/how-do-i-lvm2-
> > defrag- or-move-based-on-logical-volumes-689335/
> >
> > I played with the first one on an older machine once and it does work
> > quite nicely.
>
> Any idea how to check how bad the fragmentation actually is?

Yes:
desktop ~ # lvs -o lv_name,seg_pe_ranges
LV PE Ranges
distfiles /dev/sda4:37632-39423
home /dev/sda4:9472-12031
home /dev/sda4:39424-40703
home /dev/sda4:42240-44799
opt /dev/sda4:8192-9471
opt /dev/sda4:40704-42239
packages /dev/sda4:45312-47359
portage /dev/sda4:3072-3327
swap /dev/sda4:0-3071
tmp /dev/sda4:3328-4607
usr /dev/sda4:6400-8191
usr /dev/sda4:81664-81919
usr /dev/sda4:44800-45311
var /dev/sda4:4608-5119
vartmp /dev/sda4:5120-6399
vartmp /dev/sda4:60160-61183
virtualbox /dev/sda4:12032-32511
virtualbox /dev/sda4:76544-81663
virtualbox /dev/sda4:58112-60159
virtualbox /dev/sda4:81920-83310
virtualbox /dev/sda4:47360-58111
virtualbox /dev/sda4:32512-33680

home, vartmp and virtualbox are fragmented into different sections.

Alternatively, the first part of that script actually generates a text-file
which you then need to edit into the next text-file.
The first one actually shows you how the different parts are laid out.

> BTW:
> wonko@weird ~ $ mount | wc -l
> 48

desktop ~ # lvs | wc -l
12

server ~ # lvs | wc -l
99
(Ok, this one runs virtual machines with Xen, but online-resizing works and
xen-virtual machines get notified of the new size)

> I use LVM for about everything now, it makes things so much easier. First,
> I had two volume groups on my system drive, one for the system, placed at
> the front where the drive is supposed to be faster, and one for data. But
> I don't do this any more, it cuts down flexibility, and is probably not
> worth the effort.

I don't think it's worth the effort as well. You can still move the LVs around
physically using the "lvm-defrag" tool.
It's very verbose by nature as it doesn't do any changes untill you tell it
to. And you're the one starting the final script.

That tool basically generates a script that calls "pvmove" a couple of times.
And the script even contains comments describing what it is doing for each
step.
It does expect the user to determine which LVs end up where.

> > Those look ok to me, except that I would expect SATA-drives to be
> > faster then PATA drives.
>
> Well, they are, except for the sometimes busy system drive. And by now I
> get similar results as for the 2nd SATA drive, throughput is between 90
> and 110 MB/sec.

Btw, I tend to use "hdparm -Tt <device>" to do the testing:
desktop ~ # hdparm -Tt /dev/sda

/dev/sda:
Timing cached reads: 3456 MB in 2.00 seconds = 1728.51 MB/sec
Timing buffered disk reads: 276 MB in 3.01 seconds = 91.75 MB/sec

> > And unset mysql.
> > There is one issue that needs to be resolved manually with getting it
> > to work with sqlite.
> > See:
> > http://forums.gentoo.org/viewtopic-t-834883-view-
> > next.html?sid=3ae77f5bfba5e006e8745eedb4b6cfc4
> >
> > Here is the bit that will solve the problem:
> > --
> > $ cd ~/.local/share/akonadi
> > $ sqlite3 akonadi.db
> > sqlite> INSERT INTO ResourceTable (name, isVirtual) VALUES
> > ('akonadi_search_resource', 1);
> > sqlite> .exit
>
> I did nothing about this by now, but I probabyl will soon. Thanks for the
> tip!
> I also had trouble with akonadi in the past, and it still gives
> warnings/errors at every startup, but at least it seems to work now.

It doesn't complain for me anymore since I switched to sqlite.
Main reason for that: I don't like MySQL and prefer not to run a full database
on my desktop anyway. And configuring it to use the Database on the server goes
too far.

> > > > Disk I/O is, in my experience, a very common cause for
> > > > "freeze-ups". Can you test with unencrypted disks to see if the
> > > > issue occurs then as well?
> > >
> > > Yes, I can do this. It's some work, but I tried so much, why not
> > > this. I have some free space, and already have written a backup
> > > script that automatically creates LVM snapshots, decrypts them, and
> > > backs it up, so I can do this from the running system.
> >
> > Ok, am interested to see if running unencrypted actually has benefits
> > here.
>
> Me too, but as things are quite faster now already, the priority for this
> task is much lower than it was yesterday :)

Ofcourse, depending on the encryption-method used, the performance impact can
be a lot to not much at all.

> > > X takes an awful lot, then comes java, which is running only for my
> > > tvbrowser. And many many chrome processes.
> >
> > How many web browser windows do you have open? :)
>
> wonko@weird ~ $ ps ax | grep chromium-browser | wc -l
> 42
>
> Well, that's tabs, not windows. Chromium uses one process for each tab.
> I'm not sure if I really want to use Chromium, I started using it because
> of a nasty konqueror bug that made KDE freeze occasionally, but it seems
> to be fixed now.

I actually use Firefox myself and not had any real issues.

> > Also, do you have file indexing enabled?
>
> Do you mean Strigi/Nepomuk? I just turned it on again today. It never
> worked, but crashed. In the last days I investigated this further and
> filed two bugs about this, and they were already fixed. I'm running the
> git version now, and so far it seems to work fine, but my home directory
> is not fully indexed yet. And there were also issues with my music folder,
> let's see if indexing this will still make strigi crash.
> I should have done this before - I am waiting over a year now for a fixed
> version, but apparently few people if any have those crashes, so I had to
> report them myself.
> At the moment it's indexing, but so far system load is not very high.
> That's much better than a year ago when I used it the last time.

I haven't managed to get it to work to the point I can actually use it.
Maybe I need to read the docs to see how people expect it to work.

I actually archive my files in such a way that I can easily find them back.
Comes from not having decent indexing tools when I started with computers and
having a lot of data to keep.

> > In my experience, X uses more memory when a lot of windows are open.
> > And yours uses about 4 times as much as mine.
> > But then again, I don't have much running at the moment.
>
> I like to have many things open. With enough memory it's no problem.
> Unless you are using ati-drivers, that is.

Hehe, if it isn't ati-drivers, it will be something else causing problems.

--
Joost