Mailing List Archive

On Wed, 6 Jan 1999, Steve Bergman wrote:
> kernel Time Maj pf Min pf Swaps
> ---------- ----- ------ ------ -----
> 2.2.0-pre5 18:19 522333 493803 27984
> arcavm10 19:57 556299 494163 12035
> arcavm9 19:55 553783 494444 12077
> arcavm7 18:39 538520 493287 11526
Happy to hear that ! ;)
The changes in 2.2.0-pre5 looks really cool! I think the only missing
thing that I would like to see in is my calc_swapout_weight() thing. This
my change would avoid swap_out() to stall too much the system in presence
of huge tasks and so it would allow the VM to scale better... I'll do some
test starting from pre5 now...
Andrea Arcangeli
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

On Wed, 6 Jan 1999, Steve Bergman wrote:
>
> Here are my latest numbers. This is timing a complete kernel compile (make
> clean;make depend;make;make modules;make modules_install) in 16MB memory with
> netscape, kde, and various daemons running. I unknowningly had two more daemons
> running in the background this time than last so the numbers can't be compared
> directly with my last test (Which I think I only sent to Andrea). But all of
> these numbers are consistent with *each other*.
>
>
> kernel Time Maj pf Min pf Swaps
> ---------- ----- ------ ------ -----
> 2.2.0-pre5 18:19 522333 493803 27984
> arcavm10 19:57 556299 494163 12035
> arcavm9 19:55 553783 494444 12077
> arcavm7 18:39 538520 493287 11526
Don't look too closely at the "swaps" number - I think pre-5 just changed
accounting a bit. A lot of the "swaps" are really just dropping a virtual
mapping (that is later picked up again from the page cache or the swap
cache).
Basically, pre-5 uses the page cache and the swap cache more actively as a
"victim cache", and that inflates the "swaps" number simply due to the
accounting issues.
I guess I shouldn't count the simple "drop_pte" operation as a swap at
all, because it doesn't involve any IO.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

This is a MIME multipart message. If you are reading
this, you shouldn't.
--=-=-=
Linus Torvalds <torvalds@transmeta.com> writes:
> On Wed, 6 Jan 1999, Steve Bergman wrote:
> >
> > Here are my latest numbers. This is timing a complete kernel compile (make
> > clean;make depend;make;make modules;make modules_install) in 16MB memory with
> > netscape, kde, and various daemons running. I unknowningly had two more daemons
> > running in the background this time than last so the numbers can't be compared
> > directly with my last test (Which I think I only sent to Andrea). But all of
> > these numbers are consistent with *each other*.
> >
> >
> > kernel Time Maj pf Min pf Swaps
> > ---------- ----- ------ ------ -----
> > 2.2.0-pre5 18:19 522333 493803 27984
> > arcavm10 19:57 556299 494163 12035
> > arcavm9 19:55 553783 494444 12077
> > arcavm7 18:39 538520 493287 11526
>
> Don't look too closely at the "swaps" number - I think pre-5 just changed
> accounting a bit. A lot of the "swaps" are really just dropping a virtual
> mapping (that is later picked up again from the page cache or the swap
> cache).
>
> Basically, pre-5 uses the page cache and the swap cache more actively as a
> "victim cache", and that inflates the "swaps" number simply due to the
> accounting issues.
>
> I guess I shouldn't count the simple "drop_pte" operation as a swap at
> all, because it doesn't involve any IO.
>
2.2.0-pre5 works very good, indeed, but it still has some not
sufficiently explored nuisances:
1) Swap performance in pre-5 is much worse compared to pre-4 in
*certain* circumstances. I'm using quite stupid and unintelligent
program to check for raw swap speed (attached below). With 64 MB of
RAM I usually run it as 'hogmem 100 3' and watch for result which is
recently around 6 MB/sec. But when I lately decided to start two
instances of it like "hogmem 50 3 & hogmem 50 3 &" in pre-4 I got 2 x
2.5 MB/sec and in pre-5 it is only 2 x 1 MB/sec and disk is making
very weird and frightening sounds. My conclusion is that now (pre-5)
system behaves much poorer when we have more than one thrashing
task. *Please*, check this, it is a quite serious problem.
2) In pre-5, under heavy load, free memory is hovering around
freepages.min instead of being somewhere between freepages.low &
freepages.max. This could make trouble for bursts of atomic
allocations (networking!).
3) Nitpick #1: /proc/swapstats exist but is only filled with
zeros. Probably it should go away. I believe Stephen added it
recently, but only part of his patch got actually applied.
4) Nitpick #2": "Swap cache:" line in report of Alt-SysRq-M is not
useful as it is laid now. People have repeatedly sent patches (Rik,
Andrea...) to fix this but it is still not fixed, as of pre-5.
5) There is lots of #if 0 constructs in MM code, and also lots of
structures are not anymore used but still take precious memory in
compiled kernel and uncover itself under /proc (/proc/sys/vm/swapctl
for instance). Do you want a patch to remove this cruft?
6) Finally one suggestion of mine. In swapfile.c there is comment:
* We try to cluster swap pages by allocating them
* sequentially in swap. Once we've allocated
* SWAP_CLUSTER_MAX pages this way, however, we resort to
* first-free allocation, starting a new cluster. This
* prevents us from scattering swap pages all over the entire
* swap partition, so that we reduce overall disk seek times
This is good, but clustering of only 32 (SWAP_CLUSTER_MAX) * 4KB =
128KB is too small for today's disk and swap sizes. I tried to enlarge
this value to something like 2 MB and got much much better results.
This is very important now that we have swapin readahead to keep pages
as adjacent as possible to each other so hit rate is big. It is
trivial (one liner) and completely safe to make this constant much
bigger, so I'm not even attaching a patch. 512 works very well and
swapping is much faster than with default valuein place. Maybe this
should even be sysctl controllable. If you agree with the last idea,
I'll send you a patch, just confirm.
I promised memory hogger:
--=-=-=
Content-Type: application/octet-stream
Content-Disposition: attachment
Content-Description: Hogmem.c
Content-Transfer-Encoding: base64
I2luY2x1ZGUgPHN0ZGlvLmg+CiNpbmNsdWRlIDx1bmlzdGQuaD4KI2luY2x1ZGUgPHN0ZGxp
Yi5oPgojaW5jbHVkZSA8bGltaXRzLmg+CiNpbmNsdWRlIDxzaWduYWwuaD4KI2luY2x1ZGUg
PHRpbWUuaD4KI2luY2x1ZGUgPHN5cy90aW1lcy5oPgoKI2RlZmluZSBNQiAoMTAyNCAqIDEw
MjQpCgppbnQgbnIsIGludHNpemUsIGksIHQ7CmNsb2NrX3Qgc3Q7CnN0cnVjdCB0bXMgZHVt
bXk7Cgp2b2lkIGludHIoaW50IGludG51bSkKewogICAgY2xvY2tfdCBldCA9IHRpbWVzKCZk
dW1teSk7CgogICAgcHJpbnRmKCJcbk1lbW9yeSBzcGVlZDogJS4yZiBNQi9zZWNcbiIsICgy
ICogdCAqIENMS19UQ0sgKiBuciArIChkb3VibGUpIGkgKiBDTEtfVENLICogaW50c2l6ZSAv
IE1CKSAvIChldCAtIHN0KSk7CiAgICBleGl0KEVYSVRfU1VDQ0VTUyk7Cn0KCmludCBtYWlu
KGludCBhcmdjLCBjaGFyICoqYXJndikKewogICAgaW50IG1heCwgbnJfdGltZXMsICphcmVh
LCBjOwoKICAgIHNldGJ1ZihzdGRvdXQsIDApOwogICAgc2lnbmFsKFNJR0lOVCwgaW50cik7
CiAgICBzaWduYWwoU0lHVEVSTSwgaW50cik7CiAgICBpbnRzaXplID0gc2l6ZW9mKGludCk7
CiAgICBpZiAoYXJnYyA8IDIgfHwgYXJnYyA+IDMpIHsKCWZwcmludGYoc3RkZXJyLCAiVXNh
Z2U6IGhvZ21lbSA8TUI+IFt0aW1lc11cbiIpOwoJZXhpdChFWElUX0ZBSUxVUkUpOwogICAg
fQogICAgbnIgPSBhdG9pKGFyZ3ZbMV0pOwogICAgaWYgKGFyZ2MgPT0gMykKCW5yX3RpbWVz
ID0gYXRvaShhcmd2WzJdKTsKICAgIGVsc2UKCW5yX3RpbWVzID0gSU5UX01BWDsKICAgIGFy
ZWEgPSBtYWxsb2MobnIgKiBNQik7CiAgICBtYXggPSBuciAqIE1CIC8gaW50c2l6ZTsKICAg
IHN0ID0gdGltZXMoJmR1bW15KTsKICAgIGZvciAoYyA9IDA7IGMgPCBucl90aW1lczsgYysr
KQogICAgewoJZm9yIChpID0gMDsgaSA8IG1heDsgaSsrKQoJICAgIGFyZWFbaV0rKzsKCXQr
KzsKCXB1dGNoYXIoJy4nKTsKICAgIH0KICAgIGkgPSAwOwogICAgaW50cigwKTsKICAgIC8q
IG5vdHJlYWNoZWQgKi8KICAgIGV4aXQoRVhJVF9TVUNDRVNTKTsKfQo=
--=-=-=
OK, that's it for today. Don't bang heads too hard and enjoy!
--
Zlatko
--=-=-=--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

On 7 Jan 1999, Zlatko Calusic wrote:
>
> 1) Swap performance in pre-5 is much worse compared to pre-4 in
> *certain* circumstances. I'm using quite stupid and unintelligent
> program to check for raw swap speed (attached below). With 64 MB of
> RAM I usually run it as 'hogmem 100 3' and watch for result which is
> recently around 6 MB/sec. But when I lately decided to start two
> instances of it like "hogmem 50 3 & hogmem 50 3 &" in pre-4 I got 2 x
> 2.5 MB/sec and in pre-5 it is only 2 x 1 MB/sec and disk is making
> very weird and frightening sounds. My conclusion is that now (pre-5)
> system behaves much poorer when we have more than one thrashing
> task. *Please*, check this, it is a quite serious problem.
Ok, will investigate. One thing you can test is to try out different
"count" arguments to try_to_free_pages() (this was part of what Andrea
did, btw). So instead of (page_alloc.c, line 285):
freed = try_to_free_pages(gfp_mask, SWAP_CLUSTER_MAX);
you can try different things for the second argument: the thing Andrea did
was something like
freed = try_to_free_pages(gfp_mask, freepages.high - nr_free_pages);
which could work well (one thing I'm nervous about is that this probably
needs to be limited some way - it can be quite a large number on large
machines, and that's why I'd like to hear comments from people).
> 2) In pre-5, under heavy load, free memory is hovering around
> freepages.min instead of being somewhere between freepages.low &
> freepages.max. This could make trouble for bursts of atomic
> allocations (networking!).
The change above would change this too.
> 3) Nitpick #1: /proc/swapstats exist but is only filled with
> zeros. Probably it should go away. I believe Stephen added it
> recently, but only part of his patch got actually applied.
Maybe somebody can find a use for it.
> 4) Nitpick #2": "Swap cache:" line in report of Alt-SysRq-M is not
> useful as it is laid now. People have repeatedly sent patches (Rik,
> Andrea...) to fix this but it is still not fixed, as of pre-5.
I never use it, so it hasn't been a big issue.
> 5) There is lots of #if 0 constructs in MM code, and also lots of
> structures are not anymore used but still take precious memory in
> compiled kernel and uncover itself under /proc (/proc/sys/vm/swapctl
> for instance). Do you want a patch to remove this cruft?
Some of the #if 0 code should certainly be removed. Some of it is useful
as a kind of commentary - sometimes code is removed not because it doesn't
make sense, but because the implementation wasn't quite good enough.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Zlatko Calusic <Zlatko.Calusic@CARNet.hr> writes:
> 2) In pre-5, under heavy load, free memory is hovering around
> freepages.min instead of being somewhere between freepages.low &
> freepages.max. This could make trouble for bursts of atomic
> allocations (networking!).
>
To followup myself, don't trust me, check your logfiles:
Jan 7 20:12:03 atlas kernel: eth0: Insufficient memory; nuking packet.
Jan 7 20:12:05 atlas last message repeated 64 times
Uaaa, it's baaack... :)
--
Zlatko
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

. (54000)

Zlatko Calusic <Zlatko.Calusic@CARNet.hr> writes:
> 2) In pre-5, under heavy load, free memory is hovering around
> freepages.min instead of being somewhere between freepages.low &
> freepages.max. This could make trouble for bursts of atomic
> allocations (networking!).
>
To followup myself, don't trust me, check your logfiles:
Jan 7 20:12:03 atlas kernel: eth0: Insufficient memory; nuking packet.
Jan 7 20:12:05 atlas last message repeated 64 times
Uaaa, it's baaack... :)
--
Zlatko
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

On 7 Jan 1999, Zlatko Calusic wrote:
> 2) In pre-5, under heavy load, free memory is hovering around
> freepages.min instead of being somewhere between freepages.low &
> freepages.max. This could make trouble for bursts of atomic
> allocations (networking!).
Agreed and I just fixed that with my updates to the memory trashing
heuristic (see also the second patch in one of my last emails).
A new minimal patch against 2.2.0-pre5 is this:
Index: page_alloc.c
===================================================================
RCS file: /var/cvs/linux/mm/page_alloc.c,v
retrieving revision 1.1.1.6
diff -u -2 -r1.1.1.6 page_alloc.c
--- page_alloc.c 1999/01/07 11:21:35 1.1.1.6
+++ linux/mm/page_alloc.c 1999/01/07 19:34:58
@@ -4,4 +4,5 @@
* Copyright (C) 1991, 1992, 1993, 1994 Linus Torvalds
* Swap reorganised 29.12.95, Stephen Tweedie
+ * trashing_memory heuristic. Copyright (C) 1999 Andrea Arcangeli
*/

@@ -259,7 +260,4 @@
* to free things up until things are better.
*
- * Normally we shouldn't ever have to do this, with
- * kswapd doing this in the background.
- *
* Most notably, this puts most of the onus of
* freeing up memory on the processes that _use_
@@ -269,8 +267,9 @@
if (!current->trashing_memory)
goto ok_to_allocate;
- if (nr_free_pages > freepages.low) {
+ if (nr_free_pages > freepages.high) {
current->trashing_memory = 0;
goto ok_to_allocate;
- }
+ } else if (nr_free_pages > freepages.low)
+ goto ok_to_allocate;
}
/*
The problem is that I don't know if it's going to hurt performances... If
somebody would try it out would be helpful... I don't think it can
hurt but...
Andrea Arcangeli
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

On Thu, 7 Jan 1999, Andrea Arcangeli wrote:
>
> The changes in 2.2.0-pre5 looks really cool! I think the only missing
> thing that I would like to see in is my calc_swapout_weight() thing. This
> my change would avoid swap_out() to stall too much the system in presence
> of huge tasks and so it would allow the VM to scale better...
Note that if swap_out swaps something out, it will always return 1 (it has
to, as it sleeps), and that in turn will make us decrement our counter,
which will make us stop paging things out soon enough..
So I really don't think it's a scaling issue either.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Linus Torvalds <torvalds@transmeta.com> writes:
> On 7 Jan 1999, Zlatko Calusic wrote:
> >
> > 1) Swap performance in pre-5 is much worse compared to pre-4 in
> > *certain* circumstances. I'm using quite stupid and unintelligent
> > program to check for raw swap speed (attached below). With 64 MB of
> > RAM I usually run it as 'hogmem 100 3' and watch for result which is
> > recently around 6 MB/sec. But when I lately decided to start two
> > instances of it like "hogmem 50 3 & hogmem 50 3 &" in pre-4 I got 2 x
> > 2.5 MB/sec and in pre-5 it is only 2 x 1 MB/sec and disk is making
> > very weird and frightening sounds. My conclusion is that now (pre-5)
> > system behaves much poorer when we have more than one thrashing
> > task. *Please*, check this, it is a quite serious problem.
>
> Ok, will investigate. One thing you can test is to try out different
> "count" arguments to try_to_free_pages() (this was part of what Andrea
> did, btw). So instead of (page_alloc.c, line 285):
>
> freed = try_to_free_pages(gfp_mask, SWAP_CLUSTER_MAX);
>
> you can try different things for the second argument: the thing Andrea did
> was something like
>
> freed = try_to_free_pages(gfp_mask, freepages.high - nr_free_pages);
>
> which could work well (one thing I'm nervous about is that this probably
> needs to be limited some way - it can be quite a large number on large
> machines, and that's why I'd like to hear comments from people).
OK, I'll check what can be done with that, as time permits.
>
> > 2) In pre-5, under heavy load, free memory is hovering around
> > freepages.min instead of being somewhere between freepages.low &
> > freepages.max. This could make trouble for bursts of atomic
> > allocations (networking!).
>
> The change above would change this too.
Yes, probably.
>
> > 3) Nitpick #1: /proc/swapstats exist but is only filled with
> > zeros. Probably it should go away. I believe Stephen added it
> > recently, but only part of his patch got actually applied.
>
> Maybe somebody can find a use for it.
Of course it can, but *right now* it's useless. So if nobody gets in
(Stephen?), I'll take it and fix it. I have a similar, even more
verbose patch for MM statistics with ~15 columns of information, but
that one does not apply cleanly on the latest kernels (obvious
reasons).
>
> > 4) Nitpick #2": "Swap cache:" line in report of Alt-SysRq-M is not
> > useful as it is laid now. People have repeatedly sent patches (Rik,
> > Andrea...) to fix this but it is still not fixed, as of pre-5.
>
> I never use it, so it hasn't been a big issue.
But it is, cause if properly done, it could give valuable information
about swap cache hit, and thus differentiate among various algorithms
we use for memory management. If I understand correctly, lots of
changes happened in MM in last few kernel revisions, but nobody knows
exactly what we're doing (there were few macro benchmarks done, which
are by definition not very useful for selecting among algorithms).
>
> > 5) There is lots of #if 0 constructs in MM code, and also lots of
> > structures are not anymore used but still take precious memory in
> > compiled kernel and uncover itself under /proc (/proc/sys/vm/swapctl
> > for instance). Do you want a patch to remove this cruft?
>
> Some of the #if 0 code should certainly be removed. Some of it is useful
> as a kind of commentary - sometimes code is removed not because it doesn't
> make sense, but because the implementation wasn't quite good enough.
>
I'll see tomorrow if I can prepare one patch to fix few annoyances I
reported.
Good night!
--
Zlatko
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

On Thu, 7 Jan 1999, Linus Torvalds wrote:
> Note that if swap_out swaps something out, it will always return 1 (it has
> to, as it sleeps), and that in turn will make us decrement our counter,
Side note, when we swapout something we stop completly swapping out and we
return to try_to_free_pages() (still better for the issue we was talking
about ;).
> So I really don't think it's a scaling issue either.
Yes, I think you are right. I am rejecting the calc_swapout_weight code.
Andrea Arcangeli
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/