Mailing List Archive

Hello,

when I boot linux-3.0.0 on xen-4.0.2.gz with dom0_mem=4000M I only get
313M of RAM in dom0.

dom0 is Debian stable (squeeze) with its 4.0.1 xen tools from pkg.

Full (serial console) debugging output is attached.


Daniel
(vlt in #xen)

On Tue, Aug 02, 2011 at 12:24:36AM +0200, Daniel Musketa wrote:
> Hello,
>
> when I boot linux-3.0.0 on xen-4.0.2.gz with dom0_mem=4000M I only get
> 313M of RAM in dom0.

Yeah, use mem=4G in your Linux kernel too. It is a workaround until
we get a right fix for that.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

2011/8/2 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
> Did you stick 'mem=4G' on the Linux stanza and dom0_mem=4G on the Xen stanza?
>
> You should see around 4G in the dom0.

Yes, I did (well "4000M", not "4G"):

-------- 8< --------
root@2ven:~# xm info
host : 2ven
release : 3.0.0
version : #1 SMP Mon Aug 1 20:35:28 CEST 2011
machine : x86_64
nr_cpus : 12
nr_nodes : 1
cores_per_socket : 6
threads_per_core : 2
cpu_mhz : 2400
hw_caps :
bfebfbff:2c100800:00000000:00001f40:029ee3ff:00000000:00000001:00000000
virt_caps : hvm
total_memory : 24515
free_memory : 796
node_to_cpu : node0:0-11
node_to_memory : node0:796
node_to_dma32_mem : node0:796
max_node_id : 0
xen_major : 4
xen_minor : 0
xen_extra : .2
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : dom0_mem=4000M loglvl=all guest_loglvl=all
com1=115200,8n1 console=com1 sync_console console_to_ring lapic=debug
apic_verbosity=debug apic=debug iommu=off
cc_compiler : gcc version 4.4.5 (Debian 4.4.5-8)
cc_compile_by : root
cc_compile_domain : (none)
cc_compile_date : Mon Aug 1 11:16:18 CEST 2011
xend_config_format : 4
root@2ven:~#
root@2ven:~# cat /proc/cmdline
mem=4000M root=LABEL=2ven_root ro rootdelay=5
cryptopts=source=UUID=85d60a1f-208a-4b05-b6d5-0bf096413078,target=raid10_crypt,lvm=zven-2ven_root
console=hvc0 earlyprintk=xen nomodeset initcall_debug debug
loglevel=10
root@2ven:~#
root@2ven:~# xm list 0
Name ID Mem VCPUs State Time(s)
Domain-0 0 2560 12 r----- 6575.1
root@2ven:~#
root@2ven:~# free -m
total used free shared buffers cached
Mem: 2126 2099 26 0 122 1769
-/+ buffers/cache: 206 1919
Swap: 511 0 511
-------- >8 --------

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Aug 01, 2011 at 11:59:41PM +0200, Daniel Musketa wrote:
> Hello,
>
> when I boot linux-3.0.0 on xen-4.0.2.gz with dom0_mem=4000M I only get
> 313M of RAM in dom0.

I feel like I've told you this a couple of times and you don't seem
to be following what I've asked you to try.

As explained in http://wiki.xensource.com/xenwiki/Linux_30_bugs

Also add 'mem=4G' to your Linux stanze.
.. snip..
> (XEN) Command line: dom0_mem=4000M loglvl=all guest_loglvl=all com1=115200,8n1 console=com1 sync_console console_to_ring lapic=debug apic_verbosity=debug apic=debug iommu=off

Ok. Leave dom0_mem=4G, while..

> [ 0.000000] Command line: root=LABEL=2ven_root ro rootdelay=5 cryptopts=source=UUID=85d60a1f-208a-4b05-b6d5-0bf096413078,target=raid10_crypt,lvm=zven-2ven_root console=hvc0 earlyprintk=xen nomodeset initcall_debug debug loglevel=10

.. on the Linux stanze, add 'mem=4G'. Can you try that?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

2011/8/3 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
> On Mon, Aug 01, 2011 at 11:59:41PM +0200, Daniel Musketa wrote:
>> Hello,
>>
>> when I boot linux-3.0.0 on xen-4.0.2.gz with dom0_mem=4000M I only get
>> 313M of RAM in dom0.
>
[...]
> .. on the Linux stanze, add 'mem=4G'. Can you try that?

I did. dmesg output with "mem=4000M" in dom0's /proc/cmdline is attached.

Thanks,
Daniel

On Thu, Aug 04, 2011 at 12:07:41AM +0200, Daniel Musketa wrote:
> 2011/8/3 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
> > On Mon, Aug 01, 2011 at 11:59:41PM +0200, Daniel Musketa wrote:
> >> Hello,
> >>
> >> when I boot linux-3.0.0 on xen-4.0.2.gz with dom0_mem=4000M I only get
> >> 313M of RAM in dom0.
> >
> [...]
> > .. on the Linux stanze, add 'mem=4G'. Can you try that?
>
> I did. dmesg output with "mem=4000M" in dom0's /proc/cmdline is attached.

Aha. It is your E820 that is causing this funny thing.
>
> Thanks,
> Daniel

> [ 0.000000] Xen: 000000008f64f000 - 000000008f6e7000 (ACPI data)
> [ 0.000000] Xen: 000000008f6e7000 - 000000008f6f1000 (ACPI NVS)
> [ 0.000000] Xen: 000000008f6f1000 - 000000008f6f2000 (ACPI data)
> [ 0.000000] Xen: 000000008f6f2000 - 000000008f7cf000 (ACPI NVS)
> [ 0.000000] Xen: 000000008f7cf000 - 000000008f800000 (ACPI data)
> [ 0.000000] Xen: 000000008f800000 - 0000000090000000 (reserved)
> [ 0.000000] Xen: 00000000a0000000 - 00000000b0000000 (reserved)
> [ 0.000000] Xen: 00000000fc000000 - 00000000fd000000 (reserved)
> [ 0.000000] Xen: 00000000fec00000 - 00000000fec01000 (reserved)
> [ 0.000000] Xen: 00000000fec90000 - 00000000fec91000 (reserved)
> [ 0.000000] Xen: 00000000fed1c000 - 00000000fed44000 (reserved)
> [ 0.000000] Xen: 00000000fee00000 - 00000000fee01000 (reserved)
> [ 0.000000] Xen: 00000000ff800000 - 0000000100000000 (reserved)
> [ 0.000000] Xen: 0000000100000000 - 00000006ca000000 (usable)

So this is what the E820 sees (well, there is somet more stuff above the ACPI data but your buffer ran out)


> [ 0.000000] e820 remove range: 00000000fa000000 - ffffffffffffffff (usable)
> [ 0.000000] bootconsole [xenboot0] enabled
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] user-defined physical RAM map:
> [ 0.000000] user: 0000000000000000 - 0000000000099000 (usable)
> [ 0.000000] user: 0000000000099800 - 0000000000100000 (reserved)
> [ 0.000000] user: 0000000000100000 - 000000008c3a0000 (usable)

So if you look here, you can see hte usable section being from 1MB up
to 2352MB (duh!) That is what the BIOS has setup.

> [ 0.000000] user: 000000008c3a0000 - 000000008c47b000 (ACPI NVS)
> [ 0.000000] user: 000000008c47b000 - 000000008c560000 (ACPI data)
> [ 0.000000] user: 000000008c560000 - 000000008d960000 (ACPI NVS)
> [ 0.000000] user: 000000008d960000 - 000000008f602000 (ACPI data)
> [ 0.000000] user: 000000008f602000 - 000000008f64f000 (reserved)
> [ 0.000000] user: 000000008f64f000 - 000000008f6e7000 (ACPI data)
> [ 0.000000] user: 000000008f6e7000 - 000000008f6f1000 (ACPI NVS)
> [ 0.000000] user: 000000008f6f1000 - 000000008f6f2000 (ACPI data)
> [ 0.000000] user: 000000008f6f2000 - 000000008f7cf000 (ACPI NVS)
> [ 0.000000] user: 000000008f7cf000 - 000000008f800000 (ACPI data)
> [ 0.000000] user: 000000008f800000 - 0000000090000000 (reserved)
> [ 0.000000] user: 00000000a0000000 - 00000000b0000000 (reserved)
> [ 0.000000] user: 00000000fc000000 - 00000000fd000000 (reserved)
> [ 0.000000] user: 00000000fec00000 - 00000000fec01000 (reserved)
> [ 0.000000] user: 00000000fec90000 - 00000000fec91000 (reserved)
> [ 0.000000] user: 00000000fed1c000 - 00000000fed44000 (reserved)
> [ 0.000000] user: 00000000fee00000 - 00000000fee01000 (reserved)
> [ 0.000000] user: 00000000ff800000 - 0000000100000000 (reserved)

and since we asked the E820 to truncate any entry above the 4G, that is the
reason you don't see the rest. (if you look at the other E820 without
the mem=X you will see that)


Anyhow, other machines usually have the E820 with more usuable regions so
that when you do 'mem=4G' you actually get around 3.5GB or so - hence the
general 'try mem=4G'.

Anyhow, in your special case you need 'mem=6G' for you to see 4GB.
Kind of non-intuitive.

We really need to fix this ...


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel