Hello,
when running Jeremy's 2.6.32.43 kernel I get OOM in dom0 when trying
to create a domU. This happened both on xen-4.0.2.gz hypervisor and
Debian's 4.0.1 xen tools and on xen-4.1.1.gz and compiled 4.1.1 xen
tools.
Full (serial console) output is attached.
The OOM part looks like this:
-------- 8< --------
[ Â 654.208337] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 654.215814] cron cpuset=/ mems_allowed=0
[ Â 654.220276] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 654.226477] Call Trace:
[ Â 654.229295] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 654.234914] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 654.241604] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 654.248101] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 654.255469] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 654.263033] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 654.270304] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 654.276217] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 654.282714] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 654.288822] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 654.296094] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 654.303946] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 654.310637] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 654.316939] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 654.323143] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 654.329639] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 654.335649] Mem-Info:
[ Â 654.338266] Node 0 DMA per-cpu:
[ Â 654.341854] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.347284] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.352713] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.358143] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.363572] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.369001] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.374431] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.379860] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.385290] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.390719] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.396148] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.401579] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.407007] Node 0 DMA32 per-cpu:
[ Â 654.410789] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.416219] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.421647] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.427077] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.432506] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.437936] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.443366] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.448796] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.454225] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.459654] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.465084] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 654.470513] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.475942] Node 0 Normal per-cpu:
[ Â 654.479821] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 654.485250] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.490679] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.496109] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.501539] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.506968] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.512398] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.517827] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.523256] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.528686] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.534116] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.539545] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.544976] active_anon:13004 inactive_anon:3325 isolated_anon:160
[ Â 654.544977] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 654.544978] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 654.544979] Â free:30116 slab_reclaimable:1654 slab_unreclaimable:10667
[ Â 654.544980] Â mapped:193 shmem:26 pagetables:639 bounce:0
[ Â 654.578715] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 654.619339] lowmem_reserve[]: 0 2171 25559 25559
[ Â 654.624574] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52016kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6616kB slab_unreclaimable:41608kB kernel_stack:2384kB
pagetables:2556kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 654.671016] lowmem_reserve[]: 0 0 23387 23387
[ Â 654.675960] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 654.718523] lowmem_reserve[]: 0 0 0 0
[ Â 654.722692] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 654.734618] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 654.747804] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 654.760021] 2124 total pagecache pages
[ Â 654.764286] 601 pages in swap cache
[ Â 654.768261] Swap cache stats: add 2432, delete 1831, find 0/0
[  654.774757] Free swap  = 514552kB
[ Â 654.778541] Total swap = 524280kB
[ Â 654.827020] 6660096 pages RAM
[ Â 654.830229] 6207063 pages reserved
[ Â 654.834108] 3196 pages shared
[ Â 654.837500] 420123 pages non-shared
[ Â 654.841481] Out of memory: kill process 2768 (exim4) score 11037 or a child
[ Â 654.849332] Killed process 2768 (exim4)
[ Â 654.854024] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 654.861504] cron cpuset=/ mems_allowed=0
[ Â 654.865961] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 654.872165] Call Trace:
[ Â 654.874983] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 654.880605] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 654.887290] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 654.893788] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 654.901156] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 654.908718] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 654.915988] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 654.921903] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 654.928400] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 654.934507] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 654.941779] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 654.949633] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 654.956324] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 654.962626] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 654.968832] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 654.975326] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 654.981336] Mem-Info:
[ Â 654.983952] Node 0 DMA per-cpu:
[ Â 654.987540] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.992970] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.998400] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.003828] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.009306] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.014689] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.020120] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.025546] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.030977] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.036405] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.041836] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.047265] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.052696] Node 0 DMA32 per-cpu:
[ Â 655.056476] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.061905] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.067334] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.072764] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.078193] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.083622] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.089052] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 35
[ Â 655.094481] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.099911] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.105376] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.110806] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 655.116235] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.121665] Node 0 Normal per-cpu:
[ Â 655.125546] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 655.130972] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.136402] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.141832] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.147260] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.152691] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.158120] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.163548] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.168978] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.174409] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.179837] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.185267] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.190698] active_anon:13004 inactive_anon:3325 isolated_anon:160
[ Â 655.190699] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 655.190700] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 655.190701] Â free:30116 slab_reclaimable:1654 slab_unreclaimable:10667
[ Â 655.190702] Â mapped:193 shmem:26 pagetables:639 bounce:0
[ Â 655.224437] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 655.265063] lowmem_reserve[]: 0 2171 25559 25559
[ Â 655.270297] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52016kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6616kB slab_unreclaimable:41608kB kernel_stack:2384kB
pagetables:2556kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 655.316737] lowmem_reserve[]: 0 0 23387 23387
[ Â 655.321682] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 655.364246] lowmem_reserve[]: 0 0 0 0
[ Â 655.368414] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 655.380340] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 655.393526] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 655.405743] 2124 total pagecache pages
[ Â 655.410009] 601 pages in swap cache
[ Â 655.413984] Swap cache stats: add 2432, delete 1831, find 0/0
[  655.420479] Free swap  = 514948kB
[ Â 655.424261] Total swap = 524280kB
[ Â 655.476216] 6660096 pages RAM
[ Â 655.479425] 6207063 pages reserved
[ Â 655.483303] 3175 pages shared
[ Â 655.486697] 420088 pages non-shared
[ Â 655.490673] Out of memory: kill process 2974 (xend) score 9860 or a child
[ Â 655.498333] Killed process 2975 (xend)
[ Â 655.506939] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 655.514415] cron cpuset=/ mems_allowed=0
[ Â 655.518875] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 655.525079] Call Trace:
[ Â 655.527896] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 655.533516] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 655.540206] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 655.546702] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 655.554071] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 655.561634] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 655.568905] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 655.574819] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 655.581315] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 655.587422] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 655.594694] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 655.602548] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 655.609238] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 655.615541] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 655.621745] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 655.628240] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 655.634250] Mem-Info:
[ Â 655.636868] Node 0 DMA per-cpu:
[ Â 655.640458] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.645884] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.651315] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.656743] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.662173] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.667603] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.673032] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.678461] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.683890] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.689320] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.694750] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.700180] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.705609] Node 0 DMA32 per-cpu:
[ Â 655.709389] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.714820] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.720250] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.725679] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.731109] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.736538] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.741968] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 35
[ Â 655.747397] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.752826] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.758255] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.763685] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 655.769114] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.774543] Node 0 Normal per-cpu:
[ Â 655.778422] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 655.783852] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.789281] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.794710] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.800141] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.805570] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.810999] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.816428] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.821858] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.827288] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.832716] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.838147] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.843578] active_anon:13003 inactive_anon:3325 isolated_anon:160
[ Â 655.843579] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 655.843580] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 655.843581] Â free:30116 slab_reclaimable:1653 slab_unreclaimable:10665
[ Â 655.843581] Â mapped:193 shmem:26 pagetables:616 bounce:0
[ Â 655.877316] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 655.917940] lowmem_reserve[]: 0 2171 25559 25559
[ Â 655.923175] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52012kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6612kB slab_unreclaimable:41600kB kernel_stack:2384kB
pagetables:2464kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 655.969617] lowmem_reserve[]: 0 0 23387 23387
[ Â 655.974561] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 656.017126] lowmem_reserve[]: 0 0 0 0
[ Â 656.021293] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 656.033219] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 656.046404] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 656.058623] 2124 total pagecache pages
[ Â 656.062887] 601 pages in swap cache
[ Â 656.066863] Swap cache stats: add 2432, delete 1831, find 0/0
[  656.073358] Free swap  = 514948kB
[ Â 656.077139] Total swap = 524280kB
[ Â 656.124934] 6660096 pages RAM
[ Â 656.128145] 6207063 pages reserved
[ Â 656.132023] 3176 pages shared
[ Â 656.135417] 420088 pages non-shared
[ Â 656.139393] Out of memory: kill process 2792 (ntpd) score 9615 or a child
[ Â 656.147051] Killed process 2792 (ntpd)
-------- >8 --------
Daniel
when running Jeremy's 2.6.32.43 kernel I get OOM in dom0 when trying
to create a domU. This happened both on xen-4.0.2.gz hypervisor and
Debian's 4.0.1 xen tools and on xen-4.1.1.gz and compiled 4.1.1 xen
tools.
Full (serial console) output is attached.
The OOM part looks like this:
-------- 8< --------
[ Â 654.208337] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 654.215814] cron cpuset=/ mems_allowed=0
[ Â 654.220276] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 654.226477] Call Trace:
[ Â 654.229295] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 654.234914] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 654.241604] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 654.248101] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 654.255469] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 654.263033] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 654.270304] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 654.276217] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 654.282714] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 654.288822] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 654.296094] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 654.303946] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 654.310637] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 654.316939] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 654.323143] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 654.329639] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 654.335649] Mem-Info:
[ Â 654.338266] Node 0 DMA per-cpu:
[ Â 654.341854] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.347284] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.352713] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.358143] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.363572] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.369001] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.374431] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.379860] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.385290] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.390719] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.396148] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.401579] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.407007] Node 0 DMA32 per-cpu:
[ Â 654.410789] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.416219] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.421647] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.427077] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.432506] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.437936] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.443366] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.448796] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.454225] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.459654] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.465084] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 654.470513] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.475942] Node 0 Normal per-cpu:
[ Â 654.479821] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 654.485250] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.490679] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.496109] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 654.501539] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.506968] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.512398] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.517827] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.523256] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.528686] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.534116] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.539545] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 654.544976] active_anon:13004 inactive_anon:3325 isolated_anon:160
[ Â 654.544977] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 654.544978] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 654.544979] Â free:30116 slab_reclaimable:1654 slab_unreclaimable:10667
[ Â 654.544980] Â mapped:193 shmem:26 pagetables:639 bounce:0
[ Â 654.578715] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 654.619339] lowmem_reserve[]: 0 2171 25559 25559
[ Â 654.624574] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52016kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6616kB slab_unreclaimable:41608kB kernel_stack:2384kB
pagetables:2556kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 654.671016] lowmem_reserve[]: 0 0 23387 23387
[ Â 654.675960] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 654.718523] lowmem_reserve[]: 0 0 0 0
[ Â 654.722692] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 654.734618] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 654.747804] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 654.760021] 2124 total pagecache pages
[ Â 654.764286] 601 pages in swap cache
[ Â 654.768261] Swap cache stats: add 2432, delete 1831, find 0/0
[  654.774757] Free swap  = 514552kB
[ Â 654.778541] Total swap = 524280kB
[ Â 654.827020] 6660096 pages RAM
[ Â 654.830229] 6207063 pages reserved
[ Â 654.834108] 3196 pages shared
[ Â 654.837500] 420123 pages non-shared
[ Â 654.841481] Out of memory: kill process 2768 (exim4) score 11037 or a child
[ Â 654.849332] Killed process 2768 (exim4)
[ Â 654.854024] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 654.861504] cron cpuset=/ mems_allowed=0
[ Â 654.865961] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 654.872165] Call Trace:
[ Â 654.874983] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 654.880605] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 654.887290] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 654.893788] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 654.901156] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 654.908718] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 654.915988] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 654.921903] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 654.928400] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 654.934507] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 654.941779] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 654.949633] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 654.956324] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 654.962626] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 654.968832] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 654.975326] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 654.981336] Mem-Info:
[ Â 654.983952] Node 0 DMA per-cpu:
[ Â 654.987540] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.992970] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 654.998400] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.003828] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.009306] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.014689] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.020120] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.025546] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.030977] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.036405] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.041836] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.047265] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.052696] Node 0 DMA32 per-cpu:
[ Â 655.056476] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.061905] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.067334] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.072764] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.078193] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.083622] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.089052] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 35
[ Â 655.094481] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.099911] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.105376] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.110806] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 655.116235] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.121665] Node 0 Normal per-cpu:
[ Â 655.125546] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 655.130972] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.136402] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.141832] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.147260] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.152691] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.158120] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.163548] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.168978] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.174409] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.179837] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.185267] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.190698] active_anon:13004 inactive_anon:3325 isolated_anon:160
[ Â 655.190699] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 655.190700] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 655.190701] Â free:30116 slab_reclaimable:1654 slab_unreclaimable:10667
[ Â 655.190702] Â mapped:193 shmem:26 pagetables:639 bounce:0
[ Â 655.224437] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 655.265063] lowmem_reserve[]: 0 2171 25559 25559
[ Â 655.270297] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52016kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6616kB slab_unreclaimable:41608kB kernel_stack:2384kB
pagetables:2556kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 655.316737] lowmem_reserve[]: 0 0 23387 23387
[ Â 655.321682] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 655.364246] lowmem_reserve[]: 0 0 0 0
[ Â 655.368414] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 655.380340] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 655.393526] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 655.405743] 2124 total pagecache pages
[ Â 655.410009] 601 pages in swap cache
[ Â 655.413984] Swap cache stats: add 2432, delete 1831, find 0/0
[  655.420479] Free swap  = 514948kB
[ Â 655.424261] Total swap = 524280kB
[ Â 655.476216] 6660096 pages RAM
[ Â 655.479425] 6207063 pages reserved
[ Â 655.483303] 3175 pages shared
[ Â 655.486697] 420088 pages non-shared
[ Â 655.490673] Out of memory: kill process 2974 (xend) score 9860 or a child
[ Â 655.498333] Killed process 2975 (xend)
[ Â 655.506939] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
[ Â 655.514415] cron cpuset=/ mems_allowed=0
[ Â 655.518875] Pid: 2504, comm: cron Not tainted 2.6.32.43 #3
[ Â 655.525079] Call Trace:
[ Â 655.527896] Â [<ffffffff810b5605>] ? T.463+0x7c/0x239
[ Â 655.533516] Â [<ffffffff810b58f6>] ? __out_of_memory+0x134/0x14b
[ Â 655.540206] Â [<ffffffff810b5a49>] ? out_of_memory+0x13c/0x16e
[ Â 655.546702] Â [<ffffffff810b9397>] ? __alloc_pages_nodemask+0x4b3/0x5d0
[ Â 655.554071] Â [<ffffffff810bacbb>] ? __do_page_cache_readahead+0x93/0x1b3
[ Â 655.561634] Â [<ffffffff8100edbf>] ? xen_restore_fl_direct_end+0x0/0x1
[ Â 655.568905] Â [<ffffffff810badf7>] ? ra_submit+0x1c/0x20
[ Â 655.574819] Â [<ffffffff810b4b12>] ? filemap_fault+0x17e/0x300
[ Â 655.581315] Â [<ffffffff810cc26a>] ? __do_fault+0x52/0x3b3
[ Â 655.587422] Â [<ffffffff8100e6f9>] ? xen_force_evtchn_callback+0x9/0xa
[ Â 655.594694] Â [<ffffffff8100c2e5>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[ Â 655.602548] Â [<ffffffff810ccff6>] ? handle_mm_fault+0x393/0x81b
[ Â 655.609238] Â [<ffffffff810673b0>] ? hrtimer_cancel+0xc/0x16
[ Â 655.615541] Â [<ffffffff81308b2e>] ? do_nanosleep+0x6a/0xa2
[ Â 655.621745] Â [<ffffffff8130b873>] ? do_page_fault+0x2f0/0x30c
[ Â 655.628240] Â [<ffffffff813097c5>] ? page_fault+0x25/0x30
[ Â 655.634250] Mem-Info:
[ Â 655.636868] Node 0 DMA per-cpu:
[ Â 655.640458] CPU Â Â 0: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.645884] CPU Â Â 1: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.651315] CPU Â Â 2: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.656743] CPU Â Â 3: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.662173] CPU Â Â 4: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.667603] CPU Â Â 5: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.673032] CPU Â Â 6: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.678461] CPU Â Â 7: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.683890] CPU Â Â 8: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.689320] CPU Â Â 9: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.694750] CPU Â 10: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.700180] CPU Â 11: hi: Â Â 0, btch: Â 1 usd: Â 0
[ Â 655.705609] Node 0 DMA32 per-cpu:
[ Â 655.709389] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.714820] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.720250] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.725679] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.731109] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.736538] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.741968] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 35
[ Â 655.747397] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.752826] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.758255] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.763685] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 2
[ Â 655.769114] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.774543] Node 0 Normal per-cpu:
[ Â 655.778422] CPU Â Â 0: hi: Â 186, btch: Â 31 usd: Â 29
[ Â 655.783852] CPU Â Â 1: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.789281] CPU Â Â 2: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.794710] CPU Â Â 3: hi: Â 186, btch: Â 31 usd: Â 30
[ Â 655.800141] CPU Â Â 4: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.805570] CPU Â Â 5: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.810999] CPU Â Â 6: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.816428] CPU Â Â 7: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.821858] CPU Â Â 8: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.827288] CPU Â Â 9: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.832716] CPU Â 10: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.838147] CPU Â 11: hi: Â 186, btch: Â 31 usd: Â 0
[ Â 655.843578] active_anon:13003 inactive_anon:3325 isolated_anon:160
[ Â 655.843579] Â active_file:308 inactive_file:1114 isolated_file:64
[ Â 655.843580] Â unevictable:6 dirty:577 writeback:1196 unstable:0
[ Â 655.843581] Â free:30116 slab_reclaimable:1653 slab_unreclaimable:10665
[ Â 655.843581] Â mapped:193 shmem:26 pagetables:616 bounce:0
[ Â 655.877316] Node 0 DMA free:12124kB min:8kB low:8kB high:12kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:11968kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[ Â 655.917940] lowmem_reserve[]: 0 2171 25559 25559
[ Â 655.923175] Node 0 DMA32 free:94700kB min:1736kB low:2168kB
high:2604kB active_anon:52012kB inactive_anon:13300kB
active_file:1232kB inactive_file:4456kB unevictable:24kB
isolated(anon):640kB isolated(file):256kB present:2223968kB
mlocked:24kB dirty:2308kB writeback:4784kB mapped:772kB shmem:104kB
slab_reclaimable:6612kB slab_unreclaimable:41600kB kernel_stack:2384kB
pagetables:2464kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:112320 all_unreclaimable? yes
[ Â 655.969617] lowmem_reserve[]: 0 0 23387 23387
[ Â 655.974561] Node 0 Normal free:13640kB min:18720kB low:23400kB
high:28080kB active_anon:0kB inactive_anon:0kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:23949120kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:1060kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ Â 656.017126] lowmem_reserve[]: 0 0 0 0
[ Â 656.021293] Node 0 DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB
2*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12124kB
[ Â 656.033219] Node 0 DMA32: 39*4kB 186*8kB 314*16kB 287*32kB 94*64kB
29*128kB 18*256kB 6*512kB 4*1024kB 2*2048kB 13*4096kB = 94700kB
[ Â 656.046404] Node 0 Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB
1*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB = 13640kB
[ Â 656.058623] 2124 total pagecache pages
[ Â 656.062887] 601 pages in swap cache
[ Â 656.066863] Swap cache stats: add 2432, delete 1831, find 0/0
[  656.073358] Free swap  = 514948kB
[ Â 656.077139] Total swap = 524280kB
[ Â 656.124934] 6660096 pages RAM
[ Â 656.128145] 6207063 pages reserved
[ Â 656.132023] 3176 pages shared
[ Â 656.135417] 420088 pages non-shared
[ Â 656.139393] Out of memory: kill process 2792 (ntpd) score 9615 or a child
[ Â 656.147051] Killed process 2792 (ntpd)
-------- >8 --------
Daniel