Mailing List Archive

* Martin J. Bligh (mbligh@mbligh.org) wrote:
> arch/i386/mach-xen/mm/highmem.c is another one to remove it seems?

Sort of, it's actually got changed, but they are in
arch/i386/mm/highmem.c. See the broken out patch below:


--- linux-2.6.12-xen0-arch.orig/include/asm-i386/highmem.h
+++ linux-2.6.12-xen0-arch/include/asm-i386/highmem.h
@@ -69,9 +69,13 @@ extern void FASTCALL(kunmap_high(struct
void *kmap(struct page *page);
void kunmap(struct page *page);
void *kmap_atomic(struct page *page, enum km_type type);
+void *__kmap_atomic(struct page *page, enum km_type type, pgprot_t prot);
void kunmap_atomic(void *kvaddr, enum km_type type);
struct page *kmap_atomic_to_page(void *ptr);

+/* for kmap_atomic_pte */
+#include <mach_highmem.h>
+
#define flush_cache_kmaps() do { } while (0)

#endif /* __KERNEL__ */
--- /dev/null
+++ linux-2.6.12-xen0-arch/include/asm-i386/mach-default/mach_highmem.h
@@ -0,0 +1,6 @@
+#ifndef __ASM_MACH_HIGHEMEM_H
+#define __ASM_MACH_HIGHEMEM_H
+
+#define kmap_atomic_pte(_p,_t) kmap_atomic(_p,_t)
+
+#endif
--- linux-2.6.12-xen0-arch.orig/arch/i386/mm/highmem.c
+++ linux-2.6.12-xen0-arch/arch/i386/mm/highmem.c
@@ -25,7 +25,7 @@ void kunmap(struct page *page)
* However when holding an atomic kmap is is not legal to sleep, so atomic
* kmaps are appropriate for short, tight code paths only.
*/
-void *kmap_atomic(struct page *page, enum km_type type)
+void *__kmap_atomic(struct page *page, enum km_type type, pgprot_t prot)
{
enum fixed_addresses idx;
unsigned long vaddr;
@@ -41,12 +41,17 @@ void *kmap_atomic(struct page *page, enu
if (!pte_none(*(kmap_pte-idx)))
BUG();
#endif
- set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
+ set_pte(kmap_pte-idx, mk_pte(page, prot));
__flush_tlb_one(vaddr);

return (void*) vaddr;
}

+void *kmap_atomic(struct page *page, enum km_type type)
+{
+ return __kmap_atomic(page, type, kmap_prot);
+}
+
void kunmap_atomic(void *kvaddr, enum km_type type)
{
#ifdef CONFIG_DEBUG_HIGHMEM

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> arch/i386/mach-xen/mm/highmem.c is another one to remove it seems?

I've actually got a patch sitting in my tree that touches this file, and
is important for making kmap (>900MB) work well on Xen. I haven't
checked it in because I don't totally like the approach of introducing
"kset_pte_at", though we clearly need something checked in because it's
a huge win on some benchmarks.

The extra optimization in highmem.c is to fold the flush into the
update_va_mapping (which is quite a bit faster than set_pte).

Ian

diff -r e9b950192ba6 linux-2.6-xen-sparse/arch/xen/i386/mm/highmem.c
--- a/linux-2.6-xen-sparse/arch/xen/i386/mm/highmem.c Wed Aug 3
21:17:17 2005
+++ b/linux-2.6-xen-sparse/arch/xen/i386/mm/highmem.c Thu Aug 4
19:07:42 2005
@@ -41,8 +41,17 @@
if (!pte_none(*(kmap_pte-idx)))
BUG();
#endif
- set_pte(kmap_pte-idx, mk_pte(page, prot));
- __flush_tlb_one(vaddr);
+
+ /* Optimize by using linear mapping */
+ if (HYPERVISOR_update_va_mapping(
+ vaddr,
+ mk_pte(page, prot),
+ UVMF_TLB_FLUSH ) )
+ {
+ /* just in case this isn't mapped in to the PGD */
+ set_pte( kmap_pte-idx, mk_pte(page, prot));
+ __flush_tlb_one(vaddr);
+ }

return (void*) vaddr;
}
diff -r e9b950192ba6
linux-2.6-xen-sparse/include/asm-xen/asm-i386/pgtable-2level.h
--- a/linux-2.6-xen-sparse/include/asm-xen/asm-i386/pgtable-2level.h
Wed Aug 3 21:17:17 2005
+++ b/linux-2.6-xen-sparse/include/asm-xen/asm-i386/pgtable-2level.h
Thu Aug 4 19:07:42 2005
@@ -16,6 +16,18 @@
#define set_pte(pteptr, pteval) (*(pteptr) = pteval)
#define set_pte_at(mm,addr,ptep,pteval) set_pte(ptep,pteval)
#define set_pte_atomic(pteptr, pteval) set_pte(pteptr,pteval)
+
+#define kset_pte_at(addr,ptep,pteval) ({
\
+ if (HYPERVISOR_update_va_mapping(
\
+ (addr),
\
+ (pteval), 0 ) )
\
+ {
\
+ /* just in case this isn't mapped in to the PGD */
\
+ set_pte( (ptep), (pteval));
\
+ }
\
+})
+
+

#ifndef CONFIG_XEN_SHADOW_MODE
#define set_pmd(pmdptr, pmdval) xen_l2_entry_update((pmdptr), (pmdval))
diff -r e9b950192ba6 linux-2.6-xen-sparse/mm/highmem.c
--- a/linux-2.6-xen-sparse/mm/highmem.c Wed Aug 3 21:17:17 2005
+++ b/linux-2.6-xen-sparse/mm/highmem.c Thu Aug 4 19:07:42 2005
@@ -139,7 +139,7 @@
}
}
vaddr = PKMAP_ADDR(last_pkmap_nr);
- set_pte_at(&init_mm, vaddr,
+ kset_pte_at(vaddr,
&(pkmap_page_table[last_pkmap_nr]), mk_pte(page,
kmap_prot));

pkmap_count[last_pkmap_nr] = 1;

diff -u mm/vmalloc.c mm/vmalloc.c.orig
--- mm/vmalloc.c 2005-06-17 20:48:29.000000000 +0100
+++ mm/vmalloc.c.orig 2005-08-09 11:11:56.120999000 +0100
@@ -96,7 +96,7 @@
WARN_ON(!pte_none(*pte));
if (!page)
return -ENOMEM;
- set_pte_at(&init_mm, addr, pte, mk_pte(page, prot));
+ kset_pte_at(addr, pte, mk_pte(page, prot));
(*pages)++;
} while (pte++, addr += PAGE_SIZE, addr != end);
return 0;

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Ian Pratt (m+Ian.Pratt@cl.cam.ac.uk) wrote:
>
> > arch/i386/mach-xen/mm/highmem.c is another one to remove it seems?
>
> I've actually got a patch sitting in my tree that touches this file, and
> is important for making kmap (>900MB) work well on Xen. I haven't
> checked it in because I don't totally like the approach of introducing
> "kset_pte_at", though we clearly need something checked in because it's
> a huge win on some benchmarks.

Also, highmem aren't identical, there was a preceding patch which did
that. The subarch is functional, but does have redundant bits in it.
I've since removed highmem. I'd avoid kset_* since it'll look like kset
handling (as in kobject and sysfs). I didn't look closely yet, but what
cases would you not want this redifintion of set_pte_at? IOW, could
set_pte_at just be provided by Xen subarch?

> The extra optimization in highmem.c is to fold the flush into the
> update_va_mapping (which is quite a bit faster than set_pte).

OK, that just says we need an arch specific callout during kmap_atomic.
I've already patches that one anyway.

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> > I've actually got a patch sitting in my tree that touches
> this file,
> > and is important for making kmap (>900MB) work well on Xen.
> I haven't
> > checked it in because I don't totally like the approach of
> introducing
> > "kset_pte_at", though we clearly need something checked in because
> > it's a huge win on some benchmarks.
>
> Also, highmem aren't identical, there was a preceding patch
> which did that. The subarch is functional, but does have
> redundant bits in it.
> I've since removed highmem. I'd avoid kset_* since it'll
> look like kset handling (as in kobject and sysfs). I didn't
> look closely yet, but what cases would you not want this
> redifintion of set_pte_at? I

We have three mechanisms that we can use to update pagetables in Xen
that have different performance tradeoffs:

1. writeable pagetables (implemented by set_pte). This is best used for
bulk updates of pagetables e.g. fork, zap etc.

2. queue_l1_update. This is best used for singleton updates. The
'queue' in the name is a bit of a misnomer as we now no longer ever
queue things because: (a) working out where to put the flushes to avoid
read-after-write hazards on pending updates is hard, and (b) its
impossible to make SMP guests work with good perofrmance. [.This is one
of the bigest 'holes' in the VMI proposal]

2. update_va_mapping. This can be used when you know the virtual
address that the pte is responsible for, and you know that it is in your
current address space. Xen uses the linear pagetable to avoid having to
dynamicly map the domain memory. It's very useful for performance
critical things like the demand fault path and kmap (highmem).
[.for kmap you may need to fall-back to a queue_l1_update in case the
page is not mapped into the PGD]


Best,
Ian

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> 2. queue_l1_update. This is best used for singleton updates. The
> 'queue' in the name is a bit of a misnomer as we now no longer ever
> queue things because: (a) working out where to put the flushes to avoid
> read-after-write hazards on pending updates is hard, and (b) its
> impossible to make SMP guests work with good perofrmance. [This is one

Can you explain the SMP guest problem in more detail?

-And

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

On 9 Aug 2005, at 18:27, Andi Kleen wrote:

>> 2. queue_l1_update. This is best used for singleton updates. The
>> 'queue' in the name is a bit of a misnomer as we now no longer ever
>> queue things because: (a) working out where to put the flushes to
>> avoid
>> read-after-write hazards on pending updates is hard, and (b) its
>> impossible to make SMP guests work with good perofrmance. [This is one
>
> Can you explain the SMP guest problem in more detail?

The queued interface as it stands cannot work correctly for SMP guests
because it provides no atomic read-modify-write command. So you cannot
atomically toggle pte flags or xchg with zero, or any of the other
atomic things you need to do for safety on SMP. We could add some
atomic commands, but our solution was just to move over to writable
pagetables which have a simpler interface for guests and good batching
performance.

-- Keir


_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge