Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Xen>
  3. Devel
[PATCH] xen-mapcache: Fix rlimit set size.
anthony.perard at citrix
Aug 1, 2011, 12:26 PM
Post #1 of 8 (313 views)
Permalink
Previously, the address space soft limit was set mcache_max_size. So,
before the mcache_max_size was reached by the mapcache, QEMU was killed
for overuse of the virtual address space.

This patch fix that by setting the soft limit to mcache_max_size + 80MB.
I observed that QEMU use 75MB more than max_mcache_size after several
empirical tests.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
xen-mapcache.c | 13 ++++++++++---
1 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/xen-mapcache.c b/xen-mapcache.c
index 007136a..40212f7 100644
--- a/xen-mapcache.c
+++ b/xen-mapcache.c
@@ -40,6 +40,13 @@
#endif
#define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)

+/* This is the size of the virtual address space reserve to QEMU that will not
+ * be use by MapCache.
+ * From empirical tests I observed that qemu use 75MB more than the
+ * max_mcache_size.
+ */
+#define NON_MCACHE_MEMORY_SIZE (80 * 1024 * 1024)
+
#define mapcache_lock() ((void)0)
#define mapcache_unlock() ((void)0)

@@ -93,14 +100,14 @@ void xen_map_cache_init(void)
mapcache->last_address_index = -1;

getrlimit(RLIMIT_AS, &rlimit_as);
- if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
+ if (rlimit_as.rlim_max < MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
rlimit_as.rlim_cur = rlimit_as.rlim_max;
} else {
- rlimit_as.rlim_cur = MCACHE_MAX_SIZE;
+ rlimit_as.rlim_cur = MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE;
}

setrlimit(RLIMIT_AS, &rlimit_as);
- mapcache->max_mcache_size = rlimit_as.rlim_cur;
+ mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;

mapcache->nr_buckets =
(((mapcache->max_mcache_size >> XC_PAGE_SHIFT) +
--
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
jbeulich at novell
Aug 1, 2011, 5:53 PM
Post #2 of 8 (310 views)
Permalink
>>> Anthony PERARD 08/01/11 9:27 PM >>>
>Previously, the address space soft limit was set mcache_max_size. So,
>before the mcache_max_size was reached by the mapcache, QEMU was killed
>for overuse of the virtual address space.
>
>This patch fix that by setting the soft limit to mcache_max_size + 80MB.
>I observed that QEMU use 75MB more than max_mcache_size after several
>empirical tests.

Rather fragile going forward I would say. I there any reason not to set
the limit to infinity?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
agraf at suse
Aug 2, 2011, 7:50 AM
Post #3 of 8 (311 views)
Permalink
On 08/01/2011 09:26 PM, Anthony PERARD wrote:
> Previously, the address space soft limit was set mcache_max_size. So,
> before the mcache_max_size was reached by the mapcache, QEMU was killed
> for overuse of the virtual address space.
>
> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
> I observed that QEMU use 75MB more than max_mcache_size after several
> empirical tests.

Not sure I understand. What are those 75MB?


Alex

> Signed-off-by: Anthony PERARD<anthony.perard@citrix.com>
> ---
> xen-mapcache.c | 13 ++++++++++---
> 1 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/xen-mapcache.c b/xen-mapcache.c
> index 007136a..40212f7 100644
> --- a/xen-mapcache.c
> +++ b/xen-mapcache.c
> @@ -40,6 +40,13 @@
> #endif
> #define MCACHE_BUCKET_SIZE (1UL<< MCACHE_BUCKET_SHIFT)
>
> +/* This is the size of the virtual address space reserve to QEMU that will not
> + * be use by MapCache.
> + * From empirical tests I observed that qemu use 75MB more than the
> + * max_mcache_size.
> + */
> +#define NON_MCACHE_MEMORY_SIZE (80 * 1024 * 1024)
> +
> #define mapcache_lock() ((void)0)
> #define mapcache_unlock() ((void)0)
>
> @@ -93,14 +100,14 @@ void xen_map_cache_init(void)
> mapcache->last_address_index = -1;
>
> getrlimit(RLIMIT_AS,&rlimit_as);
> - if (rlimit_as.rlim_max< MCACHE_MAX_SIZE) {
> + if (rlimit_as.rlim_max< MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
> rlimit_as.rlim_cur = rlimit_as.rlim_max;
> } else {
> - rlimit_as.rlim_cur = MCACHE_MAX_SIZE;
> + rlimit_as.rlim_cur = MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE;
> }
>
> setrlimit(RLIMIT_AS,&rlimit_as);
> - mapcache->max_mcache_size = rlimit_as.rlim_cur;
> + mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
>
> mapcache->nr_buckets =
> (((mapcache->max_mcache_size>> XC_PAGE_SHIFT) +


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
stefano.stabellini at eu
Aug 2, 2011, 3:01 PM
Post #4 of 8 (310 views)
Permalink
On Mon, 1 Aug 2011, Anthony PERARD wrote:
> Previously, the address space soft limit was set mcache_max_size. So,
> before the mcache_max_size was reached by the mapcache, QEMU was killed
> for overuse of the virtual address space.
>
> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
> I observed that QEMU use 75MB more than max_mcache_size after several
> empirical tests.
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> xen-mapcache.c | 13 ++++++++++---
> 1 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/xen-mapcache.c b/xen-mapcache.c
> index 007136a..40212f7 100644
> --- a/xen-mapcache.c
> +++ b/xen-mapcache.c
> @@ -40,6 +40,13 @@
> #endif
> #define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
>
> +/* This is the size of the virtual address space reserve to QEMU that will not
> + * be use by MapCache.
> + * From empirical tests I observed that qemu use 75MB more than the
> + * max_mcache_size.
> + */
> +#define NON_MCACHE_MEMORY_SIZE (80 * 1024 * 1024)
> +
> #define mapcache_lock() ((void)0)
> #define mapcache_unlock() ((void)0)
>
> @@ -93,14 +100,14 @@ void xen_map_cache_init(void)
> mapcache->last_address_index = -1;
>
> getrlimit(RLIMIT_AS, &rlimit_as);
> - if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
> + if (rlimit_as.rlim_max < MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
> rlimit_as.rlim_cur = rlimit_as.rlim_max;
> } else {
> - rlimit_as.rlim_cur = MCACHE_MAX_SIZE;
> + rlimit_as.rlim_cur = MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE;
> }

In the 'else' case we should just set rlimit_as.rlim_cur to
rlimit_as.rlim_max.
Also we should detect if we are running as a privileged process and if
that is the case just modify rlimit_as.rlim_max to RLIM_INFINITY.
If we are not running as a privileged process and if rlim_max is not
high enough we should print a warning because trying to guess how much
virtual memory qemu is going to use is obviously error prone.

So this is how I think it should work:

if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
if (qemu is priviledged) {
rlimit_as.rlim_max = RLIM_INFINITY;
rlimit_as.rlim_cur = RLIM_INFINITY;
} else {
print a warning
mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
rlimit_as.rlim_cur = rlimit_as.rlim_max;
}
} else {
rlimit_as.rlim_cur = rlimit_as.rlim_max;
}
setrlimit


> setrlimit(RLIMIT_AS, &rlimit_as);
> - mapcache->max_mcache_size = rlimit_as.rlim_cur;
> + mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
>
> mapcache->nr_buckets =
> (((mapcache->max_mcache_size >> XC_PAGE_SHIFT) +
> --
> Anthony PERARD
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
stefano.stabellini at eu
Aug 2, 2011, 3:03 PM
Post #5 of 8 (310 views)
Permalink
On Tue, 2 Aug 2011, Jan Beulich wrote:
> >>> Anthony PERARD 08/01/11 9:27 PM >>>
> >Previously, the address space soft limit was set mcache_max_size. So,
> >before the mcache_max_size was reached by the mapcache, QEMU was killed
> >for overuse of the virtual address space.
> >
> >This patch fix that by setting the soft limit to mcache_max_size + 80MB.
> >I observed that QEMU use 75MB more than max_mcache_size after several
> >empirical tests.
>
> Rather fragile going forward I would say.

Yes, I agree. At the very least we should print a warning if rlimit_max
is too low so that the admin can fix it.

> I there any reason not to set
> the limit to infinity?

We shouldn't assume that qemu is running as a privileged process, but
if it is we should definitely do it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
jbeulich at novell
Aug 3, 2011, 7:42 AM
Post #6 of 8 (307 views)
Permalink
>>> Stefano Stabellini 08/03/11 12:01 AM >>>
>if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {

This condition minimally needs to include what Anthony's original version had.

>if (qemu is priviledged) {
>rlimit_as.rlim_max = RLIM_INFINITY;
>rlimit_as.rlim_cur = RLIM_INFINITY;
>} else {
>print a warning
>mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
>rlimit_as.rlim_cur = rlimit_as.rlim_max;
>}
>} else {

Not printing a warning here means that there are still cases where the
fuzzy upper bound may not be precise enough anymore (hence causing
silent failure). I think the privileged case needs to be handled without
any other surrounding condition, and the warning ought to be prinited
in any case in the non-privileged case.

>rlimit_as.rlim_cur = rlimit_as.rlim_max;
>}

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: Re: [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
stefano.stabellini at eu
Aug 3, 2011, 1:57 PM
Post #7 of 8 (309 views)
Permalink
On Wed, 3 Aug 2011, Jan Beulich wrote:
> >>> Stefano Stabellini 08/03/11 12:01 AM >>>
> >if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
>
> This condition minimally needs to include what Anthony's original version had.

right


> >if (qemu is priviledged) {
> >rlimit_as.rlim_max = RLIM_INFINITY;
> >rlimit_as.rlim_cur = RLIM_INFINITY;
> >} else {
> >print a warning
> >mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
> >rlimit_as.rlim_cur = rlimit_as.rlim_max;
> >}
> >} else {
>
> Not printing a warning here means that there are still cases where the
> fuzzy upper bound may not be precise enough anymore (hence causing
> silent failure). I think the privileged case needs to be handled without
> any other surrounding condition, and the warning ought to be prinited
> in any case in the non-privileged case.

good idea

>
> >rlimit_as.rlim_cur = rlimit_as.rlim_max;
> >}

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [Qemu-devel] [PATCH] xen-mapcache: Fix rlimit set size. [ In reply to ]
anthony.perard at citrix
Aug 4, 2011, 6:33 AM
Post #8 of 8 (305 views)
Permalink
On Tue, Aug 2, 2011 at 15:50, Alexander Graf <agraf@suse.de> wrote:
> On 08/01/2011 09:26 PM, Anthony PERARD wrote:
>>
>> Previously, the address space soft limit was set mcache_max_size. So,
>> before the mcache_max_size was reached by the mapcache, QEMU was killed
>> for overuse of the virtual address space.
>>
>> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
>> I observed that QEMU use 75MB more than max_mcache_size after several
>> empirical tests.
>
> Not sure I understand. What are those 75MB?

I did the calculation like that:
- I make sure that the MapCache was full.
- I did the difference between the value in mapcache->max_mcache_size
and the size of the used address space by the QEMU process.

--
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal