Mailing List Archive

Sorry for top posting - on mobile.

I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?

Sander Eikelenboom <linux@eikelenboom.it> wrote:

>Hi Konrad,
>
>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>
>[ 402.723915] ------------[ cut here ]------------
>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>[ 402.744207] invalid opcode: 0000 [#5] PREEMPT SMP
>[ 402.752692] Modules linked in:
>[ 402.761307] CPU 1
>[ 402.761358] Pid: 1329, comm: netback/1 Tainted: G D 3.6.0-pre-rc1-20121005a #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>[ 402.778214] RIP: e030:[<ffffffff814714f9>] [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>[ 402.786779] RSP: e02b:ffff880037955bb0 EFLAGS: 00010206
>[ 402.795183] RAX: 000000000000486a RBX: ffff88003878c9c0 RCX: ffffea0000b3d400
>[ 402.803536] RDX: ffff880037955cd0 RSI: ffff880037955d1c RDI: ffff88003878c9c0
>[ 402.811867] RBP: ffff880037955c20 R08: 0000000000000000 R09: 00000000000042c2
>[ 402.820008] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88002f58b100
>[ 402.828022] R13: ffff880037955cd0 R14: 00000000000005a8 R15: ffff880037955d1c
>[ 402.835927] FS: 00007ffe2ca5b760(0000) GS:ffff88003f840000(0000) knlGS:0000000000000000
>[ 402.843826] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>[ 402.851539] CR2: 00007fff99c3b018 CR3: 00000000377c9000 CR4: 0000000000000660
>[ 402.859251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>[ 402.866816] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>[ 402.874188] Process netback/1 (pid: 1329, threadinfo ffff880037954000, task ffff8800398c20a0)
>[ 402.881621] Stack:
>[ 402.888811] ffff880037955d1c 0000000000000000 ffff88002f58b100 ffff88003878c9c0
>[ 402.896102] ffff880000000000 ffff88002cf54000 0000000000000000 0000000000000000
>[ 402.903320] ffff880037955c20 ffff88002f58b100 0000000000000001 0000000000000010
>[ 402.910356] Call Trace:
>[ 402.917180] [<ffffffff81471853>] xen_netbk_rx_action+0x303/0x840
>[ 402.924065] [<ffffffff810acf0d>] ? trace_hardirqs_on+0xd/0x10
>[ 402.930776] [<ffffffff81472d7a>] xen_netbk_kthread+0xba/0xac0
>[ 402.937352] [<ffffffff810957b6>] ? try_to_wake_up+0x1b6/0x310
>[ 402.943856] [<ffffffff810867e0>] ? wake_up_bit+0x40/0x40
>[ 402.950173] [<ffffffff81472cc0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>[ 402.956370] [<ffffffff81086176>] kthread+0xd6/0xe0
>[ 402.962524] [<ffffffff817f1664>] kernel_thread_helper+0x4/0x10
>[ 402.968668] [<ffffffff817efb37>] ? retint_restore_args+0x13/0x13
>[ 402.974735] [<ffffffff817f1660>] ? gs_change+0x13/0x13
>[ 402.980715] Code: b8 01 00 00 00 48 69 d2 b8 b3 00 00 48 8d 84 f8 60 01 00 00 48 3b 0c 10 0f 85 de fc ff ff e9 e2 fc ff ff 0f 0b eb fe 0f 0b eb fe <0f> 0b eb fe 0f 1f 00 55 48 89 e5 48 83 ec 10 48 89 1c 24 4c 89
>[ 402.993584] RIP [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>[ 403.000075] RSP <ffff880037955bb0>
>[ 403.006603] ---[ end trace 6eada309643a3fc7 ]---
>
>--
>
>Sander
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Friday, October 5, 2012, 9:26:31 PM, you wrote:

> Sorry for top posting - on mobile.

> I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?

AMD Phenom II X6, will try the xsave=off and report later !

--
Sander


> Sander Eikelenboom <linux@eikelenboom.it> wrote:

>>Hi Konrad,
>>
>>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>>
>>[ 402.723915] ------------[ cut here ]------------
>>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>>[ 402.744207] invalid opcode: 0000 [#5] PREEMPT SMP
>>[ 402.752692] Modules linked in:
>>[ 402.761307] CPU 1
>>[ 402.761358] Pid: 1329, comm: netback/1 Tainted: G D 3.6.0-pre-rc1-20121005a #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>>[ 402.778214] RIP: e030:[<ffffffff814714f9>] [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>>[ 402.786779] RSP: e02b:ffff880037955bb0 EFLAGS: 00010206
>>[ 402.795183] RAX: 000000000000486a RBX: ffff88003878c9c0 RCX: ffffea0000b3d400
>>[ 402.803536] RDX: ffff880037955cd0 RSI: ffff880037955d1c RDI: ffff88003878c9c0
>>[ 402.811867] RBP: ffff880037955c20 R08: 0000000000000000 R09: 00000000000042c2
>>[ 402.820008] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88002f58b100
>>[ 402.828022] R13: ffff880037955cd0 R14: 00000000000005a8 R15: ffff880037955d1c
>>[ 402.835927] FS: 00007ffe2ca5b760(0000) GS:ffff88003f840000(0000) knlGS:0000000000000000
>>[ 402.843826] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>>[ 402.851539] CR2: 00007fff99c3b018 CR3: 00000000377c9000 CR4: 0000000000000660
>>[ 402.859251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>[ 402.866816] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>[ 402.874188] Process netback/1 (pid: 1329, threadinfo ffff880037954000, task ffff8800398c20a0)
>>[ 402.881621] Stack:
>>[ 402.888811] ffff880037955d1c 0000000000000000 ffff88002f58b100 ffff88003878c9c0
>>[ 402.896102] ffff880000000000 ffff88002cf54000 0000000000000000 0000000000000000
>>[ 402.903320] ffff880037955c20 ffff88002f58b100 0000000000000001 0000000000000010
>>[ 402.910356] Call Trace:
>>[ 402.917180] [<ffffffff81471853>] xen_netbk_rx_action+0x303/0x840
>>[ 402.924065] [<ffffffff810acf0d>] ? trace_hardirqs_on+0xd/0x10
>>[ 402.930776] [<ffffffff81472d7a>] xen_netbk_kthread+0xba/0xac0
>>[ 402.937352] [<ffffffff810957b6>] ? try_to_wake_up+0x1b6/0x310
>>[ 402.943856] [<ffffffff810867e0>] ? wake_up_bit+0x40/0x40
>>[ 402.950173] [<ffffffff81472cc0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>>[ 402.956370] [<ffffffff81086176>] kthread+0xd6/0xe0
>>[ 402.962524] [<ffffffff817f1664>] kernel_thread_helper+0x4/0x10
>>[ 402.968668] [<ffffffff817efb37>] ? retint_restore_args+0x13/0x13
>>[ 402.974735] [<ffffffff817f1660>] ? gs_change+0x13/0x13
>>[ 402.980715] Code: b8 01 00 00 00 48 69 d2 b8 b3 00 00 48 8d 84 f8 60 01 00 00 48 3b 0c 10 0f 85 de fc ff ff e9 e2 fc ff ff 0f 0b eb fe 0f 0b eb fe <0f> 0b eb fe 0f 1f 00 55 48 89 e5 48 83 ec 10 48 89 1c 24 4c 89
>>[ 402.993584] RIP [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>>[ 403.000075] RSP <ffff880037955bb0>
>>[ 403.006603] ---[ end trace 6eada309643a3fc7 ]---
>>
>>--
>>
>>Sander
>>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Friday, October 5, 2012, 9:26:31 PM, you wrote:

> Sorry for top posting - on mobile.

> I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?

Nope the xsave=off doesn't help

> Sander Eikelenboom <linux@eikelenboom.it> wrote:

>>Hi Konrad,
>>
>>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>>
>>[ 402.723915] ------------[ cut here ]------------
>>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>>[ 402.744207] invalid opcode: 0000 [#5] PREEMPT SMP
>>[ 402.752692] Modules linked in:
>>[ 402.761307] CPU 1
>>[ 402.761358] Pid: 1329, comm: netback/1 Tainted: G D 3.6.0-pre-rc1-20121005a #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>>[ 402.778214] RIP: e030:[<ffffffff814714f9>] [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>>[ 402.786779] RSP: e02b:ffff880037955bb0 EFLAGS: 00010206
>>[ 402.795183] RAX: 000000000000486a RBX: ffff88003878c9c0 RCX: ffffea0000b3d400
>>[ 402.803536] RDX: ffff880037955cd0 RSI: ffff880037955d1c RDI: ffff88003878c9c0
>>[ 402.811867] RBP: ffff880037955c20 R08: 0000000000000000 R09: 00000000000042c2
>>[ 402.820008] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88002f58b100
>>[ 402.828022] R13: ffff880037955cd0 R14: 00000000000005a8 R15: ffff880037955d1c
>>[ 402.835927] FS: 00007ffe2ca5b760(0000) GS:ffff88003f840000(0000) knlGS:0000000000000000
>>[ 402.843826] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>>[ 402.851539] CR2: 00007fff99c3b018 CR3: 00000000377c9000 CR4: 0000000000000660
>>[ 402.859251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>[ 402.866816] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>[ 402.874188] Process netback/1 (pid: 1329, threadinfo ffff880037954000, task ffff8800398c20a0)
>>[ 402.881621] Stack:
>>[ 402.888811] ffff880037955d1c 0000000000000000 ffff88002f58b100 ffff88003878c9c0
>>[ 402.896102] ffff880000000000 ffff88002cf54000 0000000000000000 0000000000000000
>>[ 402.903320] ffff880037955c20 ffff88002f58b100 0000000000000001 0000000000000010
>>[ 402.910356] Call Trace:
>>[ 402.917180] [<ffffffff81471853>] xen_netbk_rx_action+0x303/0x840
>>[ 402.924065] [<ffffffff810acf0d>] ? trace_hardirqs_on+0xd/0x10
>>[ 402.930776] [<ffffffff81472d7a>] xen_netbk_kthread+0xba/0xac0
>>[ 402.937352] [<ffffffff810957b6>] ? try_to_wake_up+0x1b6/0x310
>>[ 402.943856] [<ffffffff810867e0>] ? wake_up_bit+0x40/0x40
>>[ 402.950173] [<ffffffff81472cc0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>>[ 402.956370] [<ffffffff81086176>] kthread+0xd6/0xe0
>>[ 402.962524] [<ffffffff817f1664>] kernel_thread_helper+0x4/0x10
>>[ 402.968668] [<ffffffff817efb37>] ? retint_restore_args+0x13/0x13
>>[ 402.974735] [<ffffffff817f1660>] ? gs_change+0x13/0x13
>>[ 402.980715] Code: b8 01 00 00 00 48 69 d2 b8 b3 00 00 48 8d 84 f8 60 01 00 00 48 3b 0c 10 0f 85 de fc ff ff e9 e2 fc ff ff 0f 0b eb fe 0f 0b eb fe <0f> 0b eb fe 0f 1f 00 55 48 89 e5 48 83 ec 10 48 89 1c 24 4c 89
>>[ 402.993584] RIP [<ffffffff814714f9>] netbk_gop_frag_copy+0x379/0x380
>>[ 403.000075] RSP <ffff880037955bb0>
>>[ 403.006603] ---[ end trace 6eada309643a3fc7 ]---
>>
>>--
>>
>>Sander
>>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Sat, Oct 06, 2012 at 12:20:54AM +0200, Sander Eikelenboom wrote:
>
> Friday, October 5, 2012, 9:26:31 PM, you wrote:
>
> > Sorry for top posting - on mobile.
>
> > I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?
>
> Nope the xsave=off doesn't help
>
> > Sander Eikelenboom <linux@eikelenboom.it> wrote:
>
> >>Hi Konrad,
> >>
> >>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
> >>
> >>[ 402.723915] ------------[ cut here ]------------
> >>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!

Looking at the code, this is what we get:

/* Data must not cross a page boundary. */
BUG_ON(size + offset > PAGE_SIZE);

Looking at the commits, the one recently added was:
commit c571898ffc24a1768e1b2dabeac0fc7dd4c14601
Author: Andres Lagar-Cavilla <andres@lagarcavilla.org>
Date: Fri Sep 14 14:26:59 2012 +0000

xen/gndev: Xen backend support for paged out grant targets V4.


But after reverting it and trying the kernel I still got the crash.

So .. the weirdness is that this seems to be only happening on
certain AMD machines - for example on my AMD A8 box I did not see this.

I fear that the next step is to do a bit off git bisection to
get an idea of which merge it might be. I am going to be AFK
on Monday so I won't get to this until Tuesday/Wednesay :-(

.. Thought to help speed this process, this looks like a
candidate:

commit 229993001282e128a49a59ec43d255614775703a
Merge: 7687b80 fd0f586
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mon Oct 1 11:13:33 2012 -0700

Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86/mm changes from Ingo Molnar:
"The biggest change is new TLB partial flushing code for AMD CPUs.
(The v3.6 kernel had the Intel CPU side code, see commits
e0ba94f14f74..effee4b9b3b.)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Mon, 2012-10-08 at 00:34 +0100, Konrad Rzeszutek Wilk wrote:
> On Sat, Oct 06, 2012 at 12:20:54AM +0200, Sander Eikelenboom wrote:
> >
> > Friday, October 5, 2012, 9:26:31 PM, you wrote:
> >
> > > Sorry for top posting - on mobile.
> >
> > > I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?
> >
> > Nope the xsave=off doesn't help
> >
> > > Sander Eikelenboom <linux@eikelenboom.it> wrote:
> >
> > >>Hi Konrad,
> > >>
> > >>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
> > >>
> > >>[ 402.723915] ------------[ cut here ]------------
> > >>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>
> Looking at the code, this is what we get:
>
> /* Data must not cross a page boundary. */
> BUG_ON(size + offset > PAGE_SIZE);
>
> Looking at the commits, the one recently added was:
> commit c571898ffc24a1768e1b2dabeac0fc7dd4c14601
> Author: Andres Lagar-Cavilla <andres@lagarcavilla.org>
> Date: Fri Sep 14 14:26:59 2012 +0000
>
> xen/gndev: Xen backend support for paged out grant targets V4.
>
>
> But after reverting it and trying the kernel I still got the crash.
>
> So .. the weirdness is that this seems to be only happening on
> certain AMD machines - for example on my AMD A8 box I did not see this.

I took a look at this last week and can't repro.

The code which calls this function is supposed to ensure that the buffer
doesn't cross a page boundary.

There are two places which call it, one is looping over the skb's frags,
which just can't cross page boundaries and if they did it would be
breaking left and right for everyone AFAICT (although I'm very behind on
my LKML and netdev reading, so maybe it is ;-)).

The other case is processing the SKB's linear data area, which can cross
a page boundary but the code loops over it and processes it in chunks
which fit in single pages. I was suspicious of this code so I pulled it
out into a little userspace test harness and fed it some corner cases
but it looked like it was doing the right thing.

I speculated that this might be NIC rather than processor related
(perhaps there's some weak correlation between certain NICs and certain
processor manufacturers).

Konrad seems to have an r8169 but the module list wasn't in Sander's
output -- do you know what you have?

> I fear that the next step is to do a bit off git bisection to
> get an idea of which merge it might be. I am going to be AFK
> on Monday so I won't get to this until Tuesday/Wednesay :-(
>
> .. Thought to help speed this process, this looks like a
> candidate:
>
> commit 229993001282e128a49a59ec43d255614775703a
> Merge: 7687b80 fd0f586
> Author: Linus Torvalds <torvalds@linux-foundation.org>
> Date: Mon Oct 1 11:13:33 2012 -0700
>
> Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>
> Pull x86/mm changes from Ingo Molnar:
> "The biggest change is new TLB partial flushing code for AMD CPUs.
> (The v3.6 kernel had the Intel CPU side code, see commits
> e0ba94f14f74..effee4b9b3b.)

Would be interesting to try although I don't think anything in this area
is actively messing with page table mappings (that happens later, and
doesn't effect the non-data bits of the skb like the sizes and offsets).

Perhaps this debug patch might shed some light? PG_compound or THP might
be an interesting case?

Ian.

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 05593d8..ca4c47d 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
* Set up the grant operations for this fragment. If it's a flipping
* interface, we also set up the unmap request from here.
*/
-static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
+static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
struct netrx_pending_operations *npo,
struct page *page, unsigned long size,
unsigned long offset, int *head)
@@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
unsigned long bytes;

/* Data must not cross a page boundary. */
- BUG_ON(size + offset > PAGE_SIZE);
+ if (size + offset > PAGE_SIZE)
+ return -1;

meta = npo->meta + npo->meta_prod - 1;

@@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
*head = 0; /* There must be something in this buffer now. */

}
+ return 0;
}

/*
@@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
if (data + len > skb_tail_pointer(skb))
len = skb_tail_pointer(skb) - data;

- netbk_gop_frag_copy(vif, skb, npo,
- virt_to_page(data), len, offset, &head);
+ if (netbk_gop_frag_copy(vif, skb, npo,
+ virt_to_page(data), len, offset, &head) < 0) {
+printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
+ skb->data, skb_tail_pointer);
+printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
+ data, data+len, offset, len);
+dump_page(virt_to_page(data));
+BUG();
+ }
data += len;
}

for (i = 0; i < nr_frags; i++) {
- netbk_gop_frag_copy(vif, skb, npo,
+ if (netbk_gop_frag_copy(vif, skb, npo,
skb_frag_page(&skb_shinfo(skb)->frags[i]),
skb_frag_size(&skb_shinfo(skb)->frags[i]),
skb_shinfo(skb)->frags[i].page_offset,
- &head);
+ &head) < 0) {
+printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
+printk(KERN_CRIT "copying from offset %x, len %x\n",
+ skb_shinfo(skb)->frags[i].page_offset,
+ skb_frag_size(&skb_shinfo(skb)->frags[i]));
+dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
+BUG();
+ }
}

return npo->meta_prod - old_meta_prod;



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Monday, October 8, 2012, 10:54:21 AM, you wrote:

> On Mon, 2012-10-08 at 00:34 +0100, Konrad Rzeszutek Wilk wrote:
>> On Sat, Oct 06, 2012 at 12:20:54AM +0200, Sander Eikelenboom wrote:
>> >
>> > Friday, October 5, 2012, 9:26:31 PM, you wrote:
>> >
>> > > Sorry for top posting - on mobile.
>> >
>> > > I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?
>> >
>> > Nope the xsave=off doesn't help
>> >
>> > > Sander Eikelenboom <linux@eikelenboom.it> wrote:
>> >
>> > >>Hi Konrad,
>> > >>
>> > >>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>> > >>
>> > >>[ 402.723915] ------------[ cut here ]------------
>> > >>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>>
>> Looking at the code, this is what we get:
>>
>> /* Data must not cross a page boundary. */
>> BUG_ON(size + offset > PAGE_SIZE);
>>
>> Looking at the commits, the one recently added was:
>> commit c571898ffc24a1768e1b2dabeac0fc7dd4c14601
>> Author: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>> Date: Fri Sep 14 14:26:59 2012 +0000
>>
>> xen/gndev: Xen backend support for paged out grant targets V4.
>>
>>
>> But after reverting it and trying the kernel I still got the crash.
>>
>> So .. the weirdness is that this seems to be only happening on
>> certain AMD machines - for example on my AMD A8 box I did not see this.

> I took a look at this last week and can't repro.

> The code which calls this function is supposed to ensure that the buffer
> doesn't cross a page boundary.

> There are two places which call it, one is looping over the skb's frags,
> which just can't cross page boundaries and if they did it would be
> breaking left and right for everyone AFAICT (although I'm very behind on
> my LKML and netdev reading, so maybe it is ;-)).

> The other case is processing the SKB's linear data area, which can cross
> a page boundary but the code loops over it and processes it in chunks
> which fit in single pages. I was suspicious of this code so I pulled it
> out into a little userspace test harness and fed it some corner cases
> but it looked like it was doing the right thing.

> I speculated that this might be NIC rather than processor related
> (perhaps there's some weak correlation between certain NICs and certain
> processor manufacturers).

> Konrad seems to have an r8169 but the module list wasn't in Sander's
> output -- do you know what you have?

Surprise surprise .. a r8169 as well ..

>> I fear that the next step is to do a bit off git bisection to
>> get an idea of which merge it might be. I am going to be AFK
>> on Monday so I won't get to this until Tuesday/Wednesay :-(
>>
>> .. Thought to help speed this process, this looks like a
>> candidate:
>>
>> commit 229993001282e128a49a59ec43d255614775703a
>> Merge: 7687b80 fd0f586
>> Author: Linus Torvalds <torvalds@linux-foundation.org>
>> Date: Mon Oct 1 11:13:33 2012 -0700
>>
>> Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>>
>> Pull x86/mm changes from Ingo Molnar:
>> "The biggest change is new TLB partial flushing code for AMD CPUs.
>> (The v3.6 kernel had the Intel CPU side code, see commits
>> e0ba94f14f74..effee4b9b3b.)

> Would be interesting to try although I don't think anything in this area
> is actively messing with page table mappings (that happens later, and
> doesn't effect the non-data bits of the skb like the sizes and offsets).

> Perhaps this debug patch might shed some light? PG_compound or THP might
> be an interesting case?

> Ian.

> diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> index 05593d8..ca4c47d 100644
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
> * Set up the grant operations for this fragment. If it's a flipping
> * interface, we also set up the unmap request from here.
> */
> -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> struct netrx_pending_operations *npo,
> struct page *page, unsigned long size,
> unsigned long offset, int *head)
> @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> unsigned long bytes;
>
> /* Data must not cross a page boundary. */
> - BUG_ON(size + offset > PAGE_SIZE);
> + if (size + offset > PAGE_SIZE)
> + return -1;
>
> meta = npo->meta + npo->meta_prod - 1;
>
> @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> *head = 0; /* There must be something in this buffer now. */
>
> }
> + return 0;
> }
>
> /*
> @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
> if (data + len > skb_tail_pointer(skb))
> len = skb_tail_pointer(skb) - data;
>
> - netbk_gop_frag_copy(vif, skb, npo,
> - virt_to_page(data), len, offset, &head);
> + if (netbk_gop_frag_copy(vif, skb, npo,
> + virt_to_page(data), len, offset, &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
+ skb->>data, skb_tail_pointer);
> +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
> + data, data+len, offset, len);
> +dump_page(virt_to_page(data));
> +BUG();
> + }
> data += len;
> }
>
> for (i = 0; i < nr_frags; i++) {
> - netbk_gop_frag_copy(vif, skb, npo,
> + if (netbk_gop_frag_copy(vif, skb, npo,
> skb_frag_page(&skb_shinfo(skb)->frags[i]),
> skb_frag_size(&skb_shinfo(skb)->frags[i]),
> skb_shinfo(skb)->frags[i].page_offset,
> - &head);
> + &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
> +printk(KERN_CRIT "copying from offset %x, len %x\n",
> + skb_shinfo(skb)->frags[i].page_offset,
> + skb_frag_size(&skb_shinfo(skb)->frags[i]));
> +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
> +BUG();
> + }
> }
>
> return npo->meta_prod - old_meta_prod;





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Monday, October 8, 2012, 10:54:21 AM, you wrote:

> On Mon, 2012-10-08 at 00:34 +0100, Konrad Rzeszutek Wilk wrote:
>> On Sat, Oct 06, 2012 at 12:20:54AM +0200, Sander Eikelenboom wrote:
>> >
>> > Friday, October 5, 2012, 9:26:31 PM, you wrote:
>> >
>> > > Sorry for top posting - on mobile.
>> >
>> > > I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?
>> >
>> > Nope the xsave=off doesn't help
>> >
>> > > Sander Eikelenboom <linux@eikelenboom.it> wrote:
>> >
>> > >>Hi Konrad,
>> > >>
>> > >>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>> > >>
>> > >>[ 402.723915] ------------[ cut here ]------------
>> > >>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>>
>> Looking at the code, this is what we get:
>>
>> /* Data must not cross a page boundary. */
>> BUG_ON(size + offset > PAGE_SIZE);
>>
>> Looking at the commits, the one recently added was:
>> commit c571898ffc24a1768e1b2dabeac0fc7dd4c14601
>> Author: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>> Date: Fri Sep 14 14:26:59 2012 +0000
>>
>> xen/gndev: Xen backend support for paged out grant targets V4.
>>
>>
>> But after reverting it and trying the kernel I still got the crash.
>>
>> So .. the weirdness is that this seems to be only happening on
>> certain AMD machines - for example on my AMD A8 box I did not see this.

> I took a look at this last week and can't repro.

> The code which calls this function is supposed to ensure that the buffer
> doesn't cross a page boundary.

> There are two places which call it, one is looping over the skb's frags,
> which just can't cross page boundaries and if they did it would be
> breaking left and right for everyone AFAICT (although I'm very behind on
> my LKML and netdev reading, so maybe it is ;-)).

> The other case is processing the SKB's linear data area, which can cross
> a page boundary but the code loops over it and processes it in chunks
> which fit in single pages. I was suspicious of this code so I pulled it
> out into a little userspace test harness and fed it some corner cases
> but it looked like it was doing the right thing.

> I speculated that this might be NIC rather than processor related
> (perhaps there's some weak correlation between certain NICs and certain
> processor manufacturers).

> Konrad seems to have an r8169 but the module list wasn't in Sander's
> output -- do you know what you have?

>> I fear that the next step is to do a bit off git bisection to
>> get an idea of which merge it might be. I am going to be AFK
>> on Monday so I won't get to this until Tuesday/Wednesay :-(
>>
>> .. Thought to help speed this process, this looks like a
>> candidate:
>>

It doesn't seem to be this commit, tested before and after, both seem to work.
I don't see a r8169 related commit to test, will see for a net related one.

--
Sander

>> commit 229993001282e128a49a59ec43d255614775703a
>> Merge: 7687b80 fd0f586
>> Author: Linus Torvalds <torvalds@linux-foundation.org>
>> Date: Mon Oct 1 11:13:33 2012 -0700
>>
>> Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>>
>> Pull x86/mm changes from Ingo Molnar:
>> "The biggest change is new TLB partial flushing code for AMD CPUs.
>> (The v3.6 kernel had the Intel CPU side code, see commits
>> e0ba94f14f74..effee4b9b3b.)

> Would be interesting to try although I don't think anything in this area
> is actively messing with page table mappings (that happens later, and
> doesn't effect the non-data bits of the skb like the sizes and offsets).

> Perhaps this debug patch might shed some light? PG_compound or THP might
> be an interesting case?

> Ian.

> diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> index 05593d8..ca4c47d 100644
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
> * Set up the grant operations for this fragment. If it's a flipping
> * interface, we also set up the unmap request from here.
> */
> -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> struct netrx_pending_operations *npo,
> struct page *page, unsigned long size,
> unsigned long offset, int *head)
> @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> unsigned long bytes;
>
> /* Data must not cross a page boundary. */
> - BUG_ON(size + offset > PAGE_SIZE);
> + if (size + offset > PAGE_SIZE)
> + return -1;
>
> meta = npo->meta + npo->meta_prod - 1;
>
> @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> *head = 0; /* There must be something in this buffer now. */
>
> }
> + return 0;
> }
>
> /*
> @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
> if (data + len > skb_tail_pointer(skb))
> len = skb_tail_pointer(skb) - data;
>
> - netbk_gop_frag_copy(vif, skb, npo,
> - virt_to_page(data), len, offset, &head);
> + if (netbk_gop_frag_copy(vif, skb, npo,
> + virt_to_page(data), len, offset, &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
+ skb->>data, skb_tail_pointer);
> +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
> + data, data+len, offset, len);
> +dump_page(virt_to_page(data));
> +BUG();
> + }
> data += len;
> }
>
> for (i = 0; i < nr_frags; i++) {
> - netbk_gop_frag_copy(vif, skb, npo,
> + if (netbk_gop_frag_copy(vif, skb, npo,
> skb_frag_page(&skb_shinfo(skb)->frags[i]),
> skb_frag_size(&skb_shinfo(skb)->frags[i]),
> skb_shinfo(skb)->frags[i].page_offset,
> - &head);
> + &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
> +printk(KERN_CRIT "copying from offset %x, len %x\n",
> + skb_shinfo(skb)->frags[i].page_offset,
> + skb_frag_size(&skb_shinfo(skb)->frags[i]));
> +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
> +BUG();
> + }
> }
>
> return npo->meta_prod - old_meta_prod;





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Monday, October 8, 2012, 10:54:21 AM, you wrote:

> On Mon, 2012-10-08 at 00:34 +0100, Konrad Rzeszutek Wilk wrote:
>> On Sat, Oct 06, 2012 at 12:20:54AM +0200, Sander Eikelenboom wrote:
>> >
>> > Friday, October 5, 2012, 9:26:31 PM, you wrote:
>> >
>> > > Sorry for top posting - on mobile.
>> >
>> > > I saw it too yesterday but only on a specific hardware - AMD FX8. What type of CPU do you have? Does xsave=off on Xen line help?
>> >
>> > Nope the xsave=off doesn't help
>> >
>> > > Sander Eikelenboom <linux@eikelenboom.it> wrote:
>> >
>> > >>Hi Konrad,
>> > >>
>> > >>Just tested kernel 3.7.0-pre-rc1 but ran into a oops in netback on boot after starting some guests:
>> > >>
>> > >>[ 402.723915] ------------[ cut here ]------------
>> > >>[ 402.734629] kernel BUG at drivers/net/xen-netback/netback.c:405!
>>
>> Looking at the code, this is what we get:
>>
>> /* Data must not cross a page boundary. */
>> BUG_ON(size + offset > PAGE_SIZE);
>>
>> Looking at the commits, the one recently added was:
>> commit c571898ffc24a1768e1b2dabeac0fc7dd4c14601
>> Author: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>> Date: Fri Sep 14 14:26:59 2012 +0000
>>
>> xen/gndev: Xen backend support for paged out grant targets V4.
>>
>>
>> But after reverting it and trying the kernel I still got the crash.
>>
>> So .. the weirdness is that this seems to be only happening on
>> certain AMD machines - for example on my AMD A8 box I did not see this.

> I took a look at this last week and can't repro.

> The code which calls this function is supposed to ensure that the buffer
> doesn't cross a page boundary.

> There are two places which call it, one is looping over the skb's frags,
> which just can't cross page boundaries and if they did it would be
> breaking left and right for everyone AFAICT (although I'm very behind on
> my LKML and netdev reading, so maybe it is ;-)).

> The other case is processing the SKB's linear data area, which can cross
> a page boundary but the code loops over it and processes it in chunks
> which fit in single pages. I was suspicious of this code so I pulled it
> out into a little userspace test harness and fed it some corner cases
> but it looked like it was doing the right thing.

> I speculated that this might be NIC rather than processor related
> (perhaps there's some weak correlation between certain NICs and certain
> processor manufacturers).

> Konrad seems to have an r8169 but the module list wasn't in Sander's
> output -- do you know what you have?

>> I fear that the next step is to do a bit off git bisection to
>> get an idea of which merge it might be. I am going to be AFK
>> on Monday so I won't get to this until Tuesday/Wednesay :-(
>>
>> .. Thought to help speed this process, this looks like a
>> candidate:
>>
>> commit 229993001282e128a49a59ec43d255614775703a
>> Merge: 7687b80 fd0f586
>> Author: Linus Torvalds <torvalds@linux-foundation.org>
>> Date: Mon Oct 1 11:13:33 2012 -0700
>>
>> Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>>
>> Pull x86/mm changes from Ingo Molnar:
>> "The biggest change is new TLB partial flushing code for AMD CPUs.
>> (The v3.6 kernel had the Intel CPU side code, see commits
>> e0ba94f14f74..effee4b9b3b.)

> Would be interesting to try although I don't think anything in this area
> is actively messing with page table mappings (that happens later, and
> doesn't effect the non-data bits of the skb like the sizes and offsets).

> Perhaps this debug patch might shed some light? PG_compound or THP might
> be an interesting case?

After applying the debug patch:

[ 197.876304] netbk_gop_frag_copy failed: skb frag 0 page
[ 197.884299] copying from offset 0, len 1628
[ 197.892781] page:ffffea0000b18400 count:3 mapcount:0 mapping: (null) index:0x0
[ 197.900778] page flags: 0x40000000004000(head)
[ 197.907074] ------------[ cut here ]------------
[ 197.913345] kernel BUG at drivers/net/xen-netback/netback.c:546!
[ 197.919626] invalid opcode: 0000 [#1] PREEMPT SMP
[ 197.921573] xen_bridge: port 10(vif10.0) entered forwarding state
[ 197.932106] Modules linked in:
[ 197.938370] CPU 0
[ 197.938420] Pid: 1180, comm: netback/0 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 197.951203] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 197.957775] RSP: e02b:ffff880037911c20 EFLAGS: 00010282
[ 197.964290] RAX: 0000000000000001 RBX: ffff880036862ee0 RCX: 0000000000000000
[ 197.970956] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379102b0
[ 197.977679] RBP: ffff880037911d50 R08: 0000000000000002 R09: 0000000000000000
[ 197.984361] R10: 0000000000000001 R11: ffff880039925e40 R12: 0000000000000030
[ 197.990958] R13: 0000000000000000 R14: ffff880031e71800 R15: 0000000000000001
[ 197.997459] FS: 00007fb5dfcf7700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
[ 198.004123] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 198.010827] CR2: 00007fb5d802d000 CR3: 0000000031fd3000 CR4: 0000000000000660
[ 198.017534] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 198.024168] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 198.030717] Process netback/0 (pid: 1180, threadinfo ffff880037910000, task ffff88003997d190)
[ 198.037326] Stack:
[ 198.043817] ffff880037911d1c ffff88003997d840 ffff880037911d00 ffff880037911c80
[ 198.050573] ffffffff00000001 0000000000000662 ffffc90010824bb8 ffffc90010820050
[ 198.057413] 0000000001080083 ffffc90010820000 0000000000000000 ffff880031cf09c0
[ 198.064228] Call Trace:
[ 198.070887] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 198.077604] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
[ 198.084394] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 198.091109] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 198.097726] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 198.104343] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 198.111001] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 198.117737] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 198.124425] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 198.131008] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 198.145094] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 198.152192] RSP <ffff880037911c20>
[ 198.159344] ---[ end trace cbdd0e4e80268fa8 ]---
[ 199.703539] tty_init_dev: 2 callbacks suppressed
[ 200.712098] device vif12.0 entered promiscuous mode
[ 201.010433] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 201.020644] xen_bridge: port 12(vif12.0) entered forwarding state
[ 201.027833] xen_bridge: port 12(vif12.0) entered forwarding state
[ 206.774576] netbk_gop_frag_copy failed: skb frag 0 page
[ 206.777945] device vif13.0 entered promiscuous mode
[ 206.788845] copying from offset 1ba4, len 2c1
[ 206.795791] page:ffffea0000b18400 count:6 mapcount:0 mapping: (null) index:0x0
[ 206.802771] page flags: 0x40000000004000(head)
[ 206.809619] ------------[ cut here ]------------
[ 206.816498] kernel BUG at drivers/net/xen-netback/netback.c:546!
[ 206.823465] invalid opcode: 0000 [#2] PREEMPT SMP
[ 206.830354] Modules linked in:
[ 206.837176] CPU 3
[ 206.837234] Pid: 1183, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 206.850881] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 206.857935] RSP: e02b:ffff880037917c20 EFLAGS: 00010282
[ 206.864972] RAX: 0000000000000001 RBX: ffff880003313ae0 RCX: 0000000000000000
[ 206.872049] RDX: ffff88003997b0f0 RSI: 0000000000000001 RDI: ffff8800379102b0
[ 206.879147] RBP: ffff880037917d50 R08: 0000000000000002 R09: 0000000000000000
[ 206.886242] R10: 0000000000000001 R11: ffff880039925640 R12: 0000000000000030
[ 206.893163] R13: 0000000000000000 R14: ffff88002c7c4400 R15: 0000000000000001
[ 206.900041] FS: 00007f800341a700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
[ 206.907145] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 206.914126] CR2: 00007f8002b31fb0 CR3: 0000000001c0b000 CR4: 0000000000000660
[ 206.921181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 206.927996] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 206.934711] Process netback/3 (pid: 1183, threadinfo ffff880037916000, task ffff88003997b0f0)
[ 206.941494] Stack:
[ 206.948105] ffff880037917d1c ffff880037916010 ffff880037917d00 ffff880037917c80
[ 206.955062] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
[ 206.962007] 0000000101080083 ffffc90010841b28 0000000100000000 ffff88002c5bb9c0
[ 206.968967] Call Trace:
[ 206.975830] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 206.982789] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 206.989662] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
[ 206.996570] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 207.003523] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 207.010333] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 207.017171] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 207.023890] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 207.030540] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 207.037275] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 207.043890] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 207.057976] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 207.065064] RSP <ffff880037917c20>
[ 207.072056] ---[ end trace cbdd0e4e80268fa9 ]---
[ 207.079366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 207.090256] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 207.097403] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 208.636257] xen_bridge: port 11(vif11.0) entered forwarding state
[ 211.515779] netbk_gop_frag_copy failed: skb frag 0 page
[ 211.522711] copying from offset 2126, len 2c1
[ 211.529403] page:ffffea0000b18400 count:8 mapcount:0 mapping: (null) index:0x0
[ 211.536142] page flags: 0x40000000004000(head)
[ 211.542942] ------------[ cut here ]------------
[ 211.549664] kernel BUG at drivers/net/xen-netback/netback.c:546!
[ 211.556408] invalid opcode: 0000 [#3] PREEMPT SMP
[ 211.563168] Modules linked in:
[ 211.569739] CPU 4
[ 211.569789] Pid: 1184, comm: netback/4 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 211.583126] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 211.590041] RSP: e02b:ffff880037921c20 EFLAGS: 00010282
[ 211.596868] RAX: 0000000000000001 RBX: ffff8800375bc4e0 RCX: 0000000000000000
[ 211.603890] RDX: ffff88003997a0a0 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 211.610792] RBP: ffff880037921d50 R08: 0000000000000002 R09: 0000000000000000
[ 211.617608] R10: 0000000000000001 R11: ffff8800399249e0 R12: 0000000000000030
[ 211.624537] R13: 0000000000000000 R14: ffff88002b98d400 R15: 0000000000000001
[ 211.631302] FS: 00007f332d735740(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
[ 211.638090] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 211.644965] CR2: 00007f1023d22000 CR3: 0000000031fba000 CR4: 0000000000000660
[ 211.651894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 211.658652] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 211.665288] Process netback/4 (pid: 1184, threadinfo ffff880037920000, task ffff88003997a0a0)
[ 211.671884] Stack:
[ 211.678376] ffff880037921d1c ffff880037920010 ffff880037921d00 ffff880037921c80
[ 211.685145] ffffffff810800b5 00000000000000ba ffffc90010851a98 ffffc9001084cf30
[ 211.691837] 0000000101080083 ffffc9001084cee0 0000000100000000 ffff88002c5bd9c0
[ 211.698581] Call Trace:
[ 211.705349] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 211.712156] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 211.718907] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
[ 211.725654] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 211.732369] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 211.739111] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 211.745858] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 211.752449] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 211.758975] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 211.765575] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 211.772016] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 211.785816] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 211.792586] RSP <ffff880037921c20>
[ 211.799394] ---[ end trace cbdd0e4e80268faa ]---
[ 212.852714] device vif14.0 entered promiscuous mode
[ 213.234995] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 213.245054] xen_bridge: port 13(vif14.0) entered forwarding state
[ 213.252087] xen_bridge: port 13(vif14.0) entered forwarding state
[ 214.691532] netbk_gop_frag_copy failed: skb frag 0 page
[ 214.698515] copying from offset 26a8, len 2c1
[ 214.705472] page:ffffea0000b18400 count:10 mapcount:0 mapping: (null) index:0x0
[ 214.712415] page flags: 0x40000000004000(head)
[ 214.719170] ------------[ cut here ]------------
[ 214.725887] kernel BUG at drivers/net/xen-netback/netback.c:546!
[ 214.732563] invalid opcode: 0000 [#4] PREEMPT SMP
[ 214.739221] Modules linked in:
[ 214.745808] CPU 5
[ 214.745859] Pid: 1185, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 214.759156] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 214.766127] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
[ 214.773012] RAX: 0000000000000001 RBX: ffff8800379172e0 RCX: 0000000000000000
[ 214.780010] RDX: ffff880039ac8000 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 214.786988] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
[ 214.793870] R10: 0000000000000001 R11: ffff880039924460 R12: 0000000000000030
[ 214.800812] R13: 0000000000000000 R14: ffff88002b8b4800 R15: 0000000000000001
[ 214.807668] FS: 00007f236d331700(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
[ 214.814545] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 214.821415] CR2: 00007f236c42b6b0 CR3: 0000000039275000 CR4: 0000000000000660
[ 214.828435] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 214.835337] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 214.841963] Process netback/5 (pid: 1185, threadinfo ffff880037922000, task ffff880039ac8000)
[ 214.848655] Stack:
[ 214.855220] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
[ 214.861945] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
[ 214.868699] 0000000101080083 ffffc90010858298 0000000100000000 ffff880031e939c0
[ 214.875477] Call Trace:
[ 214.882247] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 214.889083] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 214.895851] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
[ 214.902612] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 214.909343] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 214.916115] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 214.922856] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 214.929527] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 214.936178] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 214.942781] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 214.949279] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 214.963107] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
[ 214.969952] RSP <ffff880037923c20>
[ 214.976802] ---[ end trace cbdd0e4e80268fab ]---
[ 216.045946] xen_bridge: port 12(vif12.0) entered forwarding state
[ 220.405869] device vif15.0 entered promiscuous mode
[ 220.607946] device vif15.0-emu entered promiscuous mode
[ 220.625075] xen_bridge: port 15(vif15.0-emu) entered forwarding state
[ 220.633333] xen_bridge: port 15(vif15.0-emu) entered forwarding state
[ 220.890237] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
[ 220.898814] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
[ 220.907406] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
[ 222.122750] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 225.943971] tty_init_dev: 14 callbacks suppressed
[ 226.654618] device vif16.0 entered promiscuous mode
[ 226.775073] device vif16.0-emu entered promiscuous mode
[ 226.784025] xen_bridge: port 17(vif16.0-emu) entered forwarding state
[ 226.790188] xen_bridge: port 17(vif16.0-emu) entered forwarding state
[ 228.253024] xen_bridge: port 13(vif14.0) entered forwarding state
[ 229.788197] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 229.796826] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 229.805243] device vif15.0-emu left promiscuous mode
[ 229.813385] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 231.558329] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
[ 231.569080] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
[ 231.609663] xen_bridge: port 14(vif15.0) entered forwarding state
[ 231.617943] xen_bridge: port 14(vif15.0) entered forwarding state
[ 231.934347] tty_init_dev: 25 callbacks suppressed






> Ian.

> diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> index 05593d8..ca4c47d 100644
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
> * Set up the grant operations for this fragment. If it's a flipping
> * interface, we also set up the unmap request from here.
> */
> -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> struct netrx_pending_operations *npo,
> struct page *page, unsigned long size,
> unsigned long offset, int *head)
> @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> unsigned long bytes;
>
> /* Data must not cross a page boundary. */
> - BUG_ON(size + offset > PAGE_SIZE);
> + if (size + offset > PAGE_SIZE)
> + return -1;
>
> meta = npo->meta + npo->meta_prod - 1;
>
> @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> *head = 0; /* There must be something in this buffer now. */
>
> }
> + return 0;
> }
>
> /*
> @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
> if (data + len > skb_tail_pointer(skb))
> len = skb_tail_pointer(skb) - data;
>
> - netbk_gop_frag_copy(vif, skb, npo,
> - virt_to_page(data), len, offset, &head);
> + if (netbk_gop_frag_copy(vif, skb, npo,
> + virt_to_page(data), len, offset, &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
+ skb->>data, skb_tail_pointer);
> +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
> + data, data+len, offset, len);
> +dump_page(virt_to_page(data));
> +BUG();
> + }
> data += len;
> }
>
> for (i = 0; i < nr_frags; i++) {
> - netbk_gop_frag_copy(vif, skb, npo,
> + if (netbk_gop_frag_copy(vif, skb, npo,
> skb_frag_page(&skb_shinfo(skb)->frags[i]),
> skb_frag_size(&skb_shinfo(skb)->frags[i]),
> skb_shinfo(skb)->frags[i].page_offset,
> - &head);
> + &head) < 0) {
> +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
> +printk(KERN_CRIT "copying from offset %x, len %x\n",
> + skb_shinfo(skb)->frags[i].page_offset,
> + skb_frag_size(&skb_shinfo(skb)->frags[i]));
> +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
> +BUG();
> + }
> }
>
> return npo->meta_prod - old_meta_prod;





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, 2012-10-09 at 03:24 +0100, Sander Eikelenboom wrote:

> >> Looking at the code, this is what we get:
> >>
> >> /* Data must not cross a page boundary. */
> >> BUG_ON(size + offset > PAGE_SIZE);
> >>[...]
> After applying the debug patch:
>
> [ 197.876304] netbk_gop_frag_copy failed: skb frag 0 page
> [ 197.884299] copying from offset 0, len 1628

WTF! This turns into BUG_ON(0 + 1628 > PAGE_SIZE) (where PAGE_SIZE is
4096) which simply should not be triggering.

Perhaps I screwed up the debugging patch... investigates... no I don't
think so, but someone should definitely check my working.

For belt and braces can you change, in netbk_gop_frag_copy:
/* Data must not cross a page boundary. */
if (size + offset > PAGE_SIZE)
return -1;
into:
/* Data must not cross a page boundary. */
if (size + offset > PAGE_SIZE) {
printk(KERN_CRIT "netbk_gop_frag_copy: size %lx offset %lx\n => %lx > %lx\n",
size, offset, size + offset, PAGE_SIZE);
return -1;
}

This made me notice that offset and len in the caller are variously
unsigned int, u16 or u32 while gop_frag_copy takes them as unsigned
longs. None of the numbers involved here are anywhere big enough to
cause any sort of overflow related error though.

> [ 197.892781] page:ffffea0000b18400 count:3 mapcount:0 mapping: (null) index:0x0
> [ 197.900778] page flags: 0x40000000004000(head)
> [ 197.907074] ------------[ cut here ]------------
> [ 197.913345] kernel BUG at drivers/net/xen-netback/netback.c:546!
> [ 197.919626] invalid opcode: 0000 [#1] PREEMPT SMP
> [ 197.921573] xen_bridge: port 10(vif10.0) entered forwarding state
> [ 197.932106] Modules linked in:
> [ 197.938370] CPU 0
> [ 197.938420] Pid: 1180, comm: netback/0 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 197.951203] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 197.957775] RSP: e02b:ffff880037911c20 EFLAGS: 00010282
> [ 197.964290] RAX: 0000000000000001 RBX: ffff880036862ee0 RCX: 0000000000000000
> [ 197.970956] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379102b0
> [ 197.977679] RBP: ffff880037911d50 R08: 0000000000000002 R09: 0000000000000000
> [ 197.984361] R10: 0000000000000001 R11: ffff880039925e40 R12: 0000000000000030
> [ 197.990958] R13: 0000000000000000 R14: ffff880031e71800 R15: 0000000000000001
> [ 197.997459] FS: 00007fb5dfcf7700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
> [ 198.004123] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 198.010827] CR2: 00007fb5d802d000 CR3: 0000000031fd3000 CR4: 0000000000000660
> [ 198.017534] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 198.024168] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 198.030717] Process netback/0 (pid: 1180, threadinfo ffff880037910000, task ffff88003997d190)
> [ 198.037326] Stack:
> [ 198.043817] ffff880037911d1c ffff88003997d840 ffff880037911d00 ffff880037911c80
> [ 198.050573] ffffffff00000001 0000000000000662 ffffc90010824bb8 ffffc90010820050
> [ 198.057413] 0000000001080083 ffffc90010820000 0000000000000000 ffff880031cf09c0
> [ 198.064228] Call Trace:
> [ 198.070887] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 198.077604] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> [ 198.084394] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 198.091109] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 198.097726] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 198.104343] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 198.111001] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 198.117737] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 198.124425] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 198.131008] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 198.145094] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 198.152192] RSP <ffff880037911c20>
> [ 198.159344] ---[ end trace cbdd0e4e80268fa8 ]---
> [ 199.703539] tty_init_dev: 2 callbacks suppressed
> [ 200.712098] device vif12.0 entered promiscuous mode
> [ 201.010433] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 201.020644] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 201.027833] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 206.774576] netbk_gop_frag_copy failed: skb frag 0 page
> [ 206.777945] device vif13.0 entered promiscuous mode
> [ 206.788845] copying from offset 1ba4, len 2c1
> [ 206.795791] page:ffffea0000b18400 count:6 mapcount:0 mapping: (null) index:0x0
> [ 206.802771] page flags: 0x40000000004000(head)
> [ 206.809619] ------------[ cut here ]------------
> [ 206.816498] kernel BUG at drivers/net/xen-netback/netback.c:546!
> [ 206.823465] invalid opcode: 0000 [#2] PREEMPT SMP
> [ 206.830354] Modules linked in:
> [ 206.837176] CPU 3
> [ 206.837234] Pid: 1183, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 206.850881] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 206.857935] RSP: e02b:ffff880037917c20 EFLAGS: 00010282
> [ 206.864972] RAX: 0000000000000001 RBX: ffff880003313ae0 RCX: 0000000000000000
> [ 206.872049] RDX: ffff88003997b0f0 RSI: 0000000000000001 RDI: ffff8800379102b0
> [ 206.879147] RBP: ffff880037917d50 R08: 0000000000000002 R09: 0000000000000000
> [ 206.886242] R10: 0000000000000001 R11: ffff880039925640 R12: 0000000000000030
> [ 206.893163] R13: 0000000000000000 R14: ffff88002c7c4400 R15: 0000000000000001
> [ 206.900041] FS: 00007f800341a700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
> [ 206.907145] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 206.914126] CR2: 00007f8002b31fb0 CR3: 0000000001c0b000 CR4: 0000000000000660
> [ 206.921181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 206.927996] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 206.934711] Process netback/3 (pid: 1183, threadinfo ffff880037916000, task ffff88003997b0f0)
> [ 206.941494] Stack:
> [ 206.948105] ffff880037917d1c ffff880037916010 ffff880037917d00 ffff880037917c80
> [ 206.955062] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
> [ 206.962007] 0000000101080083 ffffc90010841b28 0000000100000000 ffff88002c5bb9c0
> [ 206.968967] Call Trace:
> [ 206.975830] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 206.982789] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 206.989662] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> [ 206.996570] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 207.003523] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 207.010333] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 207.017171] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 207.023890] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 207.030540] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 207.037275] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 207.043890] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 207.057976] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 207.065064] RSP <ffff880037917c20>
> [ 207.072056] ---[ end trace cbdd0e4e80268fa9 ]---
> [ 207.079366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 207.090256] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 207.097403] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 208.636257] xen_bridge: port 11(vif11.0) entered forwarding state
> [ 211.515779] netbk_gop_frag_copy failed: skb frag 0 page
> [ 211.522711] copying from offset 2126, len 2c1
> [ 211.529403] page:ffffea0000b18400 count:8 mapcount:0 mapping: (null) index:0x0
> [ 211.536142] page flags: 0x40000000004000(head)
> [ 211.542942] ------------[ cut here ]------------
> [ 211.549664] kernel BUG at drivers/net/xen-netback/netback.c:546!
> [ 211.556408] invalid opcode: 0000 [#3] PREEMPT SMP
> [ 211.563168] Modules linked in:
> [ 211.569739] CPU 4
> [ 211.569789] Pid: 1184, comm: netback/4 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 211.583126] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 211.590041] RSP: e02b:ffff880037921c20 EFLAGS: 00010282
> [ 211.596868] RAX: 0000000000000001 RBX: ffff8800375bc4e0 RCX: 0000000000000000
> [ 211.603890] RDX: ffff88003997a0a0 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 211.610792] RBP: ffff880037921d50 R08: 0000000000000002 R09: 0000000000000000
> [ 211.617608] R10: 0000000000000001 R11: ffff8800399249e0 R12: 0000000000000030
> [ 211.624537] R13: 0000000000000000 R14: ffff88002b98d400 R15: 0000000000000001
> [ 211.631302] FS: 00007f332d735740(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
> [ 211.638090] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 211.644965] CR2: 00007f1023d22000 CR3: 0000000031fba000 CR4: 0000000000000660
> [ 211.651894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 211.658652] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 211.665288] Process netback/4 (pid: 1184, threadinfo ffff880037920000, task ffff88003997a0a0)
> [ 211.671884] Stack:
> [ 211.678376] ffff880037921d1c ffff880037920010 ffff880037921d00 ffff880037921c80
> [ 211.685145] ffffffff810800b5 00000000000000ba ffffc90010851a98 ffffc9001084cf30
> [ 211.691837] 0000000101080083 ffffc9001084cee0 0000000100000000 ffff88002c5bd9c0
> [ 211.698581] Call Trace:
> [ 211.705349] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 211.712156] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 211.718907] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> [ 211.725654] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 211.732369] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 211.739111] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 211.745858] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 211.752449] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 211.758975] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 211.765575] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 211.772016] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 211.785816] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 211.792586] RSP <ffff880037921c20>
> [ 211.799394] ---[ end trace cbdd0e4e80268faa ]---
> [ 212.852714] device vif14.0 entered promiscuous mode
> [ 213.234995] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 213.245054] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 213.252087] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 214.691532] netbk_gop_frag_copy failed: skb frag 0 page
> [ 214.698515] copying from offset 26a8, len 2c1
> [ 214.705472] page:ffffea0000b18400 count:10 mapcount:0 mapping: (null) index:0x0
> [ 214.712415] page flags: 0x40000000004000(head)
> [ 214.719170] ------------[ cut here ]------------
> [ 214.725887] kernel BUG at drivers/net/xen-netback/netback.c:546!
> [ 214.732563] invalid opcode: 0000 [#4] PREEMPT SMP
> [ 214.739221] Modules linked in:
> [ 214.745808] CPU 5
> [ 214.745859] Pid: 1185, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 214.759156] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 214.766127] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
> [ 214.773012] RAX: 0000000000000001 RBX: ffff8800379172e0 RCX: 0000000000000000
> [ 214.780010] RDX: ffff880039ac8000 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 214.786988] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
> [ 214.793870] R10: 0000000000000001 R11: ffff880039924460 R12: 0000000000000030
> [ 214.800812] R13: 0000000000000000 R14: ffff88002b8b4800 R15: 0000000000000001
> [ 214.807668] FS: 00007f236d331700(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
> [ 214.814545] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 214.821415] CR2: 00007f236c42b6b0 CR3: 0000000039275000 CR4: 0000000000000660
> [ 214.828435] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 214.835337] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 214.841963] Process netback/5 (pid: 1185, threadinfo ffff880037922000, task ffff880039ac8000)
> [ 214.848655] Stack:
> [ 214.855220] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
> [ 214.861945] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
> [ 214.868699] 0000000101080083 ffffc90010858298 0000000100000000 ffff880031e939c0
> [ 214.875477] Call Trace:
> [ 214.882247] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 214.889083] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 214.895851] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> [ 214.902612] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 214.909343] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 214.916115] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 214.922856] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 214.929527] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 214.936178] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 214.942781] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 214.949279] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 214.963107] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> [ 214.969952] RSP <ffff880037923c20>
> [ 214.976802] ---[ end trace cbdd0e4e80268fab ]---
> [ 216.045946] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 220.405869] device vif15.0 entered promiscuous mode
> [ 220.607946] device vif15.0-emu entered promiscuous mode
> [ 220.625075] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> [ 220.633333] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> [ 220.890237] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
> [ 220.898814] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
> [ 220.907406] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
> [ 222.122750] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 225.943971] tty_init_dev: 14 callbacks suppressed
> [ 226.654618] device vif16.0 entered promiscuous mode
> [ 226.775073] device vif16.0-emu entered promiscuous mode
> [ 226.784025] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> [ 226.790188] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> [ 228.253024] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 229.788197] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 229.796826] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 229.805243] device vif15.0-emu left promiscuous mode
> [ 229.813385] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 231.558329] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
> [ 231.569080] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
> [ 231.609663] xen_bridge: port 14(vif15.0) entered forwarding state
> [ 231.617943] xen_bridge: port 14(vif15.0) entered forwarding state
> [ 231.934347] tty_init_dev: 25 callbacks suppressed
>
>
>
>
>
>
> > Ian.
>
> > diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> > index 05593d8..ca4c47d 100644
> > --- a/drivers/net/xen-netback/netback.c
> > +++ b/drivers/net/xen-netback/netback.c
> > @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
> > * Set up the grant operations for this fragment. If it's a flipping
> > * interface, we also set up the unmap request from here.
> > */
> > -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> > +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> > struct netrx_pending_operations *npo,
> > struct page *page, unsigned long size,
> > unsigned long offset, int *head)
> > @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> > unsigned long bytes;
> >
> > /* Data must not cross a page boundary. */
> > - BUG_ON(size + offset > PAGE_SIZE);
> > + if (size + offset > PAGE_SIZE)
> > + return -1;
> >
> > meta = npo->meta + npo->meta_prod - 1;
> >
> > @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> > *head = 0; /* There must be something in this buffer now. */
> >
> > }
> > + return 0;
> > }
> >
> > /*
> > @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
> > if (data + len > skb_tail_pointer(skb))
> > len = skb_tail_pointer(skb) - data;
> >
> > - netbk_gop_frag_copy(vif, skb, npo,
> > - virt_to_page(data), len, offset, &head);
> > + if (netbk_gop_frag_copy(vif, skb, npo,
> > + virt_to_page(data), len, offset, &head) < 0) {
> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
> + skb->>data, skb_tail_pointer);
> > +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
> > + data, data+len, offset, len);
> > +dump_page(virt_to_page(data));
> > +BUG();
> > + }
> > data += len;
> > }
> >
> > for (i = 0; i < nr_frags; i++) {
> > - netbk_gop_frag_copy(vif, skb, npo,
> > + if (netbk_gop_frag_copy(vif, skb, npo,
> > skb_frag_page(&skb_shinfo(skb)->frags[i]),
> > skb_frag_size(&skb_shinfo(skb)->frags[i]),
> > skb_shinfo(skb)->frags[i].page_offset,
> > - &head);
> > + &head) < 0) {
> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
> > +printk(KERN_CRIT "copying from offset %x, len %x\n",
> > + skb_shinfo(skb)->frags[i].page_offset,
> > + skb_frag_size(&skb_shinfo(skb)->frags[i]));
> > +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
> > +BUG();
> > + }
> > }
> >
> > return npo->meta_prod - old_meta_prod;
>
>
>
>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Tuesday, October 9, 2012, 11:23:08 AM, you wrote:

> On Tue, 2012-10-09 at 03:24 +0100, Sander Eikelenboom wrote:

>> >> Looking at the code, this is what we get:
>> >>
>> >> /* Data must not cross a page boundary. */
>> >> BUG_ON(size + offset > PAGE_SIZE);
>> >>[...]
>> After applying the debug patch:
>>
>> [ 197.876304] netbk_gop_frag_copy failed: skb frag 0 page
>> [ 197.884299] copying from offset 0, len 1628

> WTF! This turns into BUG_ON(0 + 1628 > PAGE_SIZE) (where PAGE_SIZE is
> 4096) which simply should not be triggering.

> Perhaps I screwed up the debugging patch... investigates... no I don't
> think so, but someone should definitely check my working.

> For belt and braces can you change, in netbk_gop_frag_copy:
> /* Data must not cross a page boundary. */
> if (size + offset > PAGE_SIZE)
> return -1;
> into:
> /* Data must not cross a page boundary. */
> if (size + offset > PAGE_SIZE) {
> printk(KERN_CRIT "netbk_gop_frag_copy: size %lx offset %lx\n => %lx > %lx\n",
> size, offset, size + offset, PAGE_SIZE);
> return -1;
> }

Done:

[ 199.342570] netbk_gop_frag_copy: size 5a8 offset 7102
[ 199.342570] => 76aa > 1000
[ 199.354626] netbk_gop_frag_copy failed: skb frag 0 page
[ 199.360930] copying from offset 7102, len 5a8
[ 199.366887] page:ffffea0000b0aa00 count:3 mapcount:0 mapping: (null) index:0x7f40fec00
[ 199.373008] page flags: 0x40000000004000(head)
[ 199.379252] ------------[ cut here ]------------
[ 199.385247] kernel BUG at drivers/net/xen-netback/netback.c:548!
[ 199.391334] invalid opcode: 0000 [#1] PREEMPT SMP
[ 199.397446] Modules linked in:
[ 199.403450] CPU 4
[ 199.403500] Pid: 1183, comm: netback/4 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 199.415401] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 199.421690] RSP: e02b:ffff88003792bc20 EFLAGS: 00010282
[ 199.428048] RAX: 0000000000000001 RBX: ffff88003197c600 RCX: 0000000000000000
[ 199.434358] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 199.440582] RBP: ffff88003792bd50 R08: 0000000000000002 R09: 0000000000000000
[ 199.446740] R10: 0000000000000001 R11: ffff88003a26c000 R12: 0000000000000030
[ 199.452965] R13: 0000000000000000 R14: ffff88002c2ae900 R15: 0000000000000001
[ 199.459203] FS: 00007fcec7740700(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
[ 199.465527] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 199.471735] CR2: 00007fff5f59c000 CR3: 0000000001c0b000 CR4: 0000000000000660
[ 199.477961] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 199.484102] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 199.490274] Process netback/4 (pid: 1183, threadinfo ffff88003792a000, task ffff880037cec140)
[ 199.496631] Stack:
[ 199.502834] ffff88003792bd1c ffff880037cec7f0 ffff88003792bd00 ffff88003792bc80
[ 199.509198] ffffffff00000001 00000000000005ea ffffc90010851a98 ffffc9001084cf30
[ 199.515579] 0000000001080083 ffffc9001084cee0 0000000000000000 ffff880032b449c0
[ 199.521944] Call Trace:
[ 199.528243] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 199.534566] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
[ 199.540826] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 199.547193] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 199.553450] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 199.559683] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 199.565827] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 199.572086] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 199.578268] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 199.584344] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 199.597406] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 199.604013] RSP <ffff88003792bc20>
[ 199.610610] ---[ end trace 03f82ac72747fb5a ]---
[ 199.990340] device vif11.0 entered promiscuous mode
[ 200.466710] xen-blkback:ring-ref 9, event-channel 10, protocol 1 (x86_64-abi)
[ 200.476634] xen_bridge: port 11(vif11.0) entered forwarding state
[ 200.483621] xen_bridge: port 11(vif11.0) entered forwarding state
[ 200.653782] pciback 0000:03:06.0: enabling device (0000 -> 0001)
[ 200.661499] xen: registering gsi 22 triggering 0 polarity 1
[ 200.669003] Already setup the GSI :22
[ 200.677345] pciback 0000:03:06.0: enabling bus mastering
[ 201.267297] xen_bridge: port 9(vif9.0) entered forwarding state
[ 205.151290] tty_init_dev: 2 callbacks suppressed
[ 206.534137] device vif12.0 entered promiscuous mode
[ 206.867366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 206.877552] xen_bridge: port 12(vif12.0) entered forwarding state
[ 206.884869] xen_bridge: port 12(vif12.0) entered forwarding state
[ 208.574036] xen_bridge: port 10(vif10.0) entered forwarding state
[ 209.979799] netbk_gop_frag_copy: size 1080 offset 0
[ 209.979799] => 1080 > 1000
[ 209.994252] netbk_gop_frag_copy failed: skb frag 0 page
[ 210.001191] copying from offset 0, len 1080
[ 210.008121] page:ffffea0000b0a800 count:3 mapcount:0 mapping: (null) index:0x7f40fec00
[ 210.015124] page flags: 0x40000000004000(head)
[ 210.022122] ------------[ cut here ]------------
[ 210.029035] kernel BUG at drivers/net/xen-netback/netback.c:548!
[ 210.035973] invalid opcode: 0000 [#2] PREEMPT SMP
[ 210.042819] Modules linked in:
[ 210.049467] CPU 0
[ 210.049518] Pid: 1179, comm: netback/0 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 210.062788] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 210.069740] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
[ 210.076711] RAX: 0000000000000001 RBX: ffff880031993ae0 RCX: 0000000000000000
[ 210.083744] RDX: ffff8800398a61e0 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 210.090801] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
[ 210.097787] R10: 0000000000000001 R11: ffff88003a26b330 R12: 0000000000000030
[ 210.104759] R13: 0000000000000000 R14: ffff88002b4d8800 R15: 0000000000000001
[ 210.111611] FS: 00007f695df80700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
[ 210.118570] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 210.125586] CR2: 00007f695402e000 CR3: 0000000032a8f000 CR4: 0000000000000660
[ 210.132677] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 210.139560] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 210.146350] Process netback/0 (pid: 1179, threadinfo ffff880037922000, task ffff8800398a61e0)
[ 210.153213] Stack:
[ 210.159974] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
[ 210.166905] ffffffff810800b5 0000000000000662 ffffc90010824bb8 ffffc90010820050
[ 210.173802] 0000000001080083 ffffc90010820000 0000000000000000 ffff8800375849c0
[ 210.180780] Call Trace:
[ 210.187656] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 210.194674] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 210.201690] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
[ 210.208659] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 210.215688] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 210.222665] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 210.229651] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 210.236455] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 210.243111] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 210.249687] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 210.256195] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 210.270166] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 210.276925] RSP <ffff880037923c20>
[ 210.284112] ---[ end trace 03f82ac72747fb5b ]---
[ 213.634083] device vif13.0 entered promiscuous mode
[ 213.911267] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 213.920749] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 213.927480] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 215.509632] xen_bridge: port 11(vif11.0) entered forwarding state
[ 215.825483] netbk_gop_frag_copy: size 2c1 offset 12d6
[ 215.825483] => 1597 > 1000
[ 215.838666] netbk_gop_frag_copy failed: skb frag 0 page
[ 215.845265] copying from offset 12d6, len 2c1
[ 215.851790] page:ffffea0000b0a800 count:6 mapcount:0 mapping: (null) index:0x7f40fec00
[ 215.858389] page flags: 0x40000000004000(head)
[ 215.864925] ------------[ cut here ]------------
[ 215.871426] kernel BUG at drivers/net/xen-netback/netback.c:548!
[ 215.878069] invalid opcode: 0000 [#3] PREEMPT SMP
[ 215.884696] Modules linked in:
[ 215.891258] CPU 3
[ 215.891308] Pid: 1182, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 215.904613] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 215.911538] RSP: e02b:ffff880037929c20 EFLAGS: 00010282
[ 215.918336] RAX: 0000000000000001 RBX: ffff88002c361ee0 RCX: 0000000000000000
[ 215.925236] RDX: ffff880037ced190 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 215.932144] RBP: ffff880037929d50 R08: 0000000000000002 R09: 0000000000000000
[ 215.938988] R10: 0000000000000001 R11: ffff88003a26aca0 R12: 0000000000000030
[ 215.945835] R13: 0000000000000000 R14: ffff88002b49b400 R15: 0000000000000001
[ 215.952652] FS: 00007f695c355700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
[ 215.959476] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 215.966165] CR2: 00007faa79583000 CR3: 0000000032a8f000 CR4: 0000000000000660
[ 215.972789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 215.979339] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 215.985844] Process netback/3 (pid: 1182, threadinfo ffff880037928000, task ffff880037ced190)
[ 215.992486] Stack:
[ 215.999085] ffff880037929d1c ffff880037928010 ffff880037929d00 ffff880037929c80
[ 216.005896] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
[ 216.012651] 0000000101080083 ffffc90010841b28 0000000100000000 ffff880031a869c0
[ 216.019386] Call Trace:
[ 216.026026] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 216.032830] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 216.039668] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
[ 216.046435] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 216.053094] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 216.059670] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 216.066279] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 216.072817] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 216.079308] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 216.085783] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 216.092234] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 216.106108] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 216.113118] RSP <ffff880037929c20>
[ 216.120011] ---[ end trace 03f82ac72747fb5c ]---
[ 219.765094] device vif14.0 entered promiscuous mode
[ 220.062152] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
[ 220.072238] xen_bridge: port 13(vif14.0) entered forwarding state
[ 220.079416] xen_bridge: port 13(vif14.0) entered forwarding state
[ 221.912781] xen_bridge: port 12(vif12.0) entered forwarding state
[ 222.876167] netbk_gop_frag_copy: size 2c1 offset 1858
[ 222.876167] => 1b19 > 1000
[ 222.889279] netbk_gop_frag_copy failed: skb frag 0 page
[ 222.895959] copying from offset 1858, len 2c1
[ 222.902484] page:ffffea0000b0a800 count:8 mapcount:0 mapping: (null) index:0x7f40fec00
[ 222.909119] page flags: 0x40000000004000(head)
[ 222.915711] ------------[ cut here ]------------
[ 222.922307] kernel BUG at drivers/net/xen-netback/netback.c:548!
[ 222.928950] invalid opcode: 0000 [#4] PREEMPT SMP
[ 222.935546] Modules linked in:
[ 222.942110] CPU 5
[ 222.942161] Pid: 1184, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 222.955415] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 222.962350] RSP: e02b:ffff88003792dc20 EFLAGS: 00010282
[ 222.969198] RAX: 0000000000000001 RBX: ffff88002b4f4ce0 RCX: 0000000000000000
[ 222.976119] RDX: ffff880037ceb0f0 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 222.982987] RBP: ffff88003792dd50 R08: 0000000000000002 R09: 0000000000000000
[ 222.989869] R10: 0000000000000001 R11: ffff88003a26b380 R12: 0000000000000030
[ 222.996658] R13: 0000000000000000 R14: ffff88002b5a7800 R15: 0000000000000001
[ 223.003490] FS: 00007f71c6ce2740(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
[ 223.010257] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 223.016868] CR2: 00007f71c66b4d15 CR3: 0000000031f46000 CR4: 0000000000000660
[ 223.023470] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 223.029999] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 223.036478] Process netback/5 (pid: 1184, threadinfo ffff88003792c000, task ffff880037ceb0f0)
[ 223.043095] Stack:
[ 223.049616] ffff88003792dd1c ffff88003792c010 ffff88003792dd00 ffff88003792dc80
[ 223.056404] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
[ 223.063150] 0000000101080083 ffffc90010858298 0000000100000000 ffff88002c38d9c0
[ 223.069955] Call Trace:
[ 223.076591] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 223.083426] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 223.090261] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
[ 223.096990] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 223.103620] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 223.110195] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 223.116768] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 223.123312] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 223.129794] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 223.136217] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 223.142658] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 223.156486] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 223.163337] RSP <ffff88003792dc20>
[ 223.170212] ---[ end trace 03f82ac72747fb5d ]---
[ 228.705439] device vif15.0 entered promiscuous mode
[ 228.880399] device vif15.0-emu entered promiscuous mode
[ 228.889286] xen_bridge: port 15(vif15.0-emu) entered forwarding state
[ 228.895546] xen_bridge: port 15(vif15.0-emu) entered forwarding state
[ 228.956267] vpn_bridge: port 1(vif13.0) entered forwarding state
[ 229.119709] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
[ 229.126644] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
[ 229.133434] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
[ 234.170536] tty_init_dev: 15 callbacks suppressed
[ 235.092664] xen_bridge: port 13(vif14.0) entered forwarding state
[ 235.684229] device vif16.0 entered promiscuous mode
[ 235.805155] device vif16.0-emu entered promiscuous mode
[ 235.813948] xen_bridge: port 17(vif16.0-emu) entered forwarding state
[ 235.820242] xen_bridge: port 17(vif16.0-emu) entered forwarding state
[ 239.632852] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 239.641629] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 239.650288] device vif15.0-emu left promiscuous mode
[ 239.658618] xen_bridge: port 15(vif15.0-emu) entered disabled state
[ 240.982436] tty_init_dev: 15 callbacks suppressed
[ 241.386562] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
[ 241.400247] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
[ 241.454701] xen_bridge: port 14(vif15.0) entered forwarding state
[ 241.463330] xen_bridge: port 14(vif15.0) entered forwarding state
[ 246.690393] xen_bridge: port 17(vif16.0-emu) entered disabled state
[ 246.699042] xen_bridge: port 17(vif16.0-emu) entered disabled state
[ 246.708731] device vif16.0-emu left promiscuous mode
[ 246.717465] xen_bridge: port 17(vif16.0-emu) entered disabled state
[ 249.449321] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
[ 249.619531] xen_bridge: port 16(vif16.0) entered forwarding state
[ 249.628307] xen_bridge: port 16(vif16.0) entered forwarding state
[ 256.489967] xen_bridge: port 14(vif15.0) entered forwarding state
[ 264.654183] xen_bridge: port 16(vif16.0) entered forwarding state
[ 414.296535] tty_init_dev: 16 callbacks suppressed
[ 458.898093] netbk_gop_frag_copy: size 5a8 offset 3602
[ 458.898093] => 3baa > 1000
[ 458.920252] netbk_gop_frag_copy failed: skb frag 0 page
[ 458.928746] copying from offset 3602, len 5a8
[ 458.937114] page:ffffea0000ada800 count:32749 mapcount:0 mapping: (null) index:0xffff88002b6a6100
[ 458.945813] page flags: 0x40000000004000(head)
[ 458.954314] ------------[ cut here ]------------
[ 458.962655] kernel BUG at drivers/net/xen-netback/netback.c:548!
[ 458.970929] invalid opcode: 0000 [#5] PREEMPT SMP
[ 458.979113] Modules linked in:
[ 458.987128] CPU 1
[ 458.987178] Pid: 1180, comm: netback/1 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
[ 459.003052] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 459.011121] RSP: e02b:ffff880037925c20 EFLAGS: 00010282
[ 459.019135] RAX: 0000000000000001 RBX: ffff88002ab0bf00 RCX: 0000000000000000
[ 459.027199] RDX: ffff8800398a30f0 RSI: 0000000000000001 RDI: ffff8800379202b0
[ 459.035081] RBP: ffff880037925d50 R08: 0000000000000002 R09: 0000000000000000
[ 459.042816] R10: 0000000000000001 R11: ffff88003a26bdb0 R12: 0000000000000030
[ 459.050308] R13: 0000000000000000 R14: ffff88002b6a2e00 R15: 0000000000000001
[ 459.057725] FS: 00007f8e25af5760(0000) GS:ffff88003f840000(0000) knlGS:0000000000000000
[ 459.065052] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 459.072248] CR2: 00007fe6b4d12fb0 CR3: 000000002c2f6000 CR4: 0000000000000660
[ 459.079480] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 459.086512] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 459.093386] Process netback/1 (pid: 1180, threadinfo ffff880037924000, task ffff8800398a30f0)
[ 459.100357] Stack:
[ 459.107071] ffff880037925d1c ffff880037924010 ffff880037925d00 ffff880037925c80
[ 459.113808] ffffffff810800b5 000000000000042a ffffc9001082ff70 ffffc9001082b408
[ 459.120494] 0000000001080083 ffffc9001082b3b8 0000000000000000 ffff8800329249c0
[ 459.127129] Call Trace:
[ 459.133509] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
[ 459.140118] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
[ 459.146604] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
[ 459.153504] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
[ 459.159949] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
[ 459.166431] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
[ 459.172778] [<ffffffff810861a6>] kthread+0xd6/0xe0
[ 459.179018] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
[ 459.185291] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
[ 459.191523] [<ffffffff8174e660>] ? gs_change+0x13/0x13
[ 459.197862] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
[ 459.211184] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
[ 459.217785] RSP <ffff880037925c20>
[ 459.224501] ---[ end trace 03f82ac72747fb5e ]---




> This made me notice that offset and len in the caller are variously
> unsigned int, u16 or u32 while gop_frag_copy takes them as unsigned
> longs. None of the numbers involved here are anywhere big enough to
> cause any sort of overflow related error though.

>> [ 197.892781] page:ffffea0000b18400 count:3 mapcount:0 mapping: (null) index:0x0
>> [ 197.900778] page flags: 0x40000000004000(head)
>> [ 197.907074] ------------[ cut here ]------------
>> [ 197.913345] kernel BUG at drivers/net/xen-netback/netback.c:546!
>> [ 197.919626] invalid opcode: 0000 [#1] PREEMPT SMP
>> [ 197.921573] xen_bridge: port 10(vif10.0) entered forwarding state
>> [ 197.932106] Modules linked in:
>> [ 197.938370] CPU 0
>> [ 197.938420] Pid: 1180, comm: netback/0 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>> [ 197.951203] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 197.957775] RSP: e02b:ffff880037911c20 EFLAGS: 00010282
>> [ 197.964290] RAX: 0000000000000001 RBX: ffff880036862ee0 RCX: 0000000000000000
>> [ 197.970956] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379102b0
>> [ 197.977679] RBP: ffff880037911d50 R08: 0000000000000002 R09: 0000000000000000
>> [ 197.984361] R10: 0000000000000001 R11: ffff880039925e40 R12: 0000000000000030
>> [ 197.990958] R13: 0000000000000000 R14: ffff880031e71800 R15: 0000000000000001
>> [ 197.997459] FS: 00007fb5dfcf7700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
>> [ 198.004123] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 198.010827] CR2: 00007fb5d802d000 CR3: 0000000031fd3000 CR4: 0000000000000660
>> [ 198.017534] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 198.024168] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 198.030717] Process netback/0 (pid: 1180, threadinfo ffff880037910000, task ffff88003997d190)
>> [ 198.037326] Stack:
>> [ 198.043817] ffff880037911d1c ffff88003997d840 ffff880037911d00 ffff880037911c80
>> [ 198.050573] ffffffff00000001 0000000000000662 ffffc90010824bb8 ffffc90010820050
>> [ 198.057413] 0000000001080083 ffffc90010820000 0000000000000000 ffff880031cf09c0
>> [ 198.064228] Call Trace:
>> [ 198.070887] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
>> [ 198.077604] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
>> [ 198.084394] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
>> [ 198.091109] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
>> [ 198.097726] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>> [ 198.104343] [<ffffffff810861a6>] kthread+0xd6/0xe0
>> [ 198.111001] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
>> [ 198.117737] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
>> [ 198.124425] [<ffffffff8174e660>] ? gs_change+0x13/0x13
>> [ 198.131008] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
>> [ 198.145094] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 198.152192] RSP <ffff880037911c20>
>> [ 198.159344] ---[ end trace cbdd0e4e80268fa8 ]---
>> [ 199.703539] tty_init_dev: 2 callbacks suppressed
>> [ 200.712098] device vif12.0 entered promiscuous mode
>> [ 201.010433] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
>> [ 201.020644] xen_bridge: port 12(vif12.0) entered forwarding state
>> [ 201.027833] xen_bridge: port 12(vif12.0) entered forwarding state
>> [ 206.774576] netbk_gop_frag_copy failed: skb frag 0 page
>> [ 206.777945] device vif13.0 entered promiscuous mode
>> [ 206.788845] copying from offset 1ba4, len 2c1
>> [ 206.795791] page:ffffea0000b18400 count:6 mapcount:0 mapping: (null) index:0x0
>> [ 206.802771] page flags: 0x40000000004000(head)
>> [ 206.809619] ------------[ cut here ]------------
>> [ 206.816498] kernel BUG at drivers/net/xen-netback/netback.c:546!
>> [ 206.823465] invalid opcode: 0000 [#2] PREEMPT SMP
>> [ 206.830354] Modules linked in:
>> [ 206.837176] CPU 3
>> [ 206.837234] Pid: 1183, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>> [ 206.850881] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 206.857935] RSP: e02b:ffff880037917c20 EFLAGS: 00010282
>> [ 206.864972] RAX: 0000000000000001 RBX: ffff880003313ae0 RCX: 0000000000000000
>> [ 206.872049] RDX: ffff88003997b0f0 RSI: 0000000000000001 RDI: ffff8800379102b0
>> [ 206.879147] RBP: ffff880037917d50 R08: 0000000000000002 R09: 0000000000000000
>> [ 206.886242] R10: 0000000000000001 R11: ffff880039925640 R12: 0000000000000030
>> [ 206.893163] R13: 0000000000000000 R14: ffff88002c7c4400 R15: 0000000000000001
>> [ 206.900041] FS: 00007f800341a700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
>> [ 206.907145] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 206.914126] CR2: 00007f8002b31fb0 CR3: 0000000001c0b000 CR4: 0000000000000660
>> [ 206.921181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 206.927996] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 206.934711] Process netback/3 (pid: 1183, threadinfo ffff880037916000, task ffff88003997b0f0)
>> [ 206.941494] Stack:
>> [ 206.948105] ffff880037917d1c ffff880037916010 ffff880037917d00 ffff880037917c80
>> [ 206.955062] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
>> [ 206.962007] 0000000101080083 ffffc90010841b28 0000000100000000 ffff88002c5bb9c0
>> [ 206.968967] Call Trace:
>> [ 206.975830] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
>> [ 206.982789] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
>> [ 206.989662] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
>> [ 206.996570] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
>> [ 207.003523] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
>> [ 207.010333] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>> [ 207.017171] [<ffffffff810861a6>] kthread+0xd6/0xe0
>> [ 207.023890] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
>> [ 207.030540] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
>> [ 207.037275] [<ffffffff8174e660>] ? gs_change+0x13/0x13
>> [ 207.043890] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
>> [ 207.057976] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 207.065064] RSP <ffff880037917c20>
>> [ 207.072056] ---[ end trace cbdd0e4e80268fa9 ]---
>> [ 207.079366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
>> [ 207.090256] vpn_bridge: port 1(vif13.0) entered forwarding state
>> [ 207.097403] vpn_bridge: port 1(vif13.0) entered forwarding state
>> [ 208.636257] xen_bridge: port 11(vif11.0) entered forwarding state
>> [ 211.515779] netbk_gop_frag_copy failed: skb frag 0 page
>> [ 211.522711] copying from offset 2126, len 2c1
>> [ 211.529403] page:ffffea0000b18400 count:8 mapcount:0 mapping: (null) index:0x0
>> [ 211.536142] page flags: 0x40000000004000(head)
>> [ 211.542942] ------------[ cut here ]------------
>> [ 211.549664] kernel BUG at drivers/net/xen-netback/netback.c:546!
>> [ 211.556408] invalid opcode: 0000 [#3] PREEMPT SMP
>> [ 211.563168] Modules linked in:
>> [ 211.569739] CPU 4
>> [ 211.569789] Pid: 1184, comm: netback/4 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>> [ 211.583126] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 211.590041] RSP: e02b:ffff880037921c20 EFLAGS: 00010282
>> [ 211.596868] RAX: 0000000000000001 RBX: ffff8800375bc4e0 RCX: 0000000000000000
>> [ 211.603890] RDX: ffff88003997a0a0 RSI: 0000000000000001 RDI: ffff8800379202b0
>> [ 211.610792] RBP: ffff880037921d50 R08: 0000000000000002 R09: 0000000000000000
>> [ 211.617608] R10: 0000000000000001 R11: ffff8800399249e0 R12: 0000000000000030
>> [ 211.624537] R13: 0000000000000000 R14: ffff88002b98d400 R15: 0000000000000001
>> [ 211.631302] FS: 00007f332d735740(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
>> [ 211.638090] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 211.644965] CR2: 00007f1023d22000 CR3: 0000000031fba000 CR4: 0000000000000660
>> [ 211.651894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 211.658652] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 211.665288] Process netback/4 (pid: 1184, threadinfo ffff880037920000, task ffff88003997a0a0)
>> [ 211.671884] Stack:
>> [ 211.678376] ffff880037921d1c ffff880037920010 ffff880037921d00 ffff880037921c80
>> [ 211.685145] ffffffff810800b5 00000000000000ba ffffc90010851a98 ffffc9001084cf30
>> [ 211.691837] 0000000101080083 ffffc9001084cee0 0000000100000000 ffff88002c5bd9c0
>> [ 211.698581] Call Trace:
>> [ 211.705349] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
>> [ 211.712156] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
>> [ 211.718907] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
>> [ 211.725654] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
>> [ 211.732369] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
>> [ 211.739111] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>> [ 211.745858] [<ffffffff810861a6>] kthread+0xd6/0xe0
>> [ 211.752449] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
>> [ 211.758975] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
>> [ 211.765575] [<ffffffff8174e660>] ? gs_change+0x13/0x13
>> [ 211.772016] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
>> [ 211.785816] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 211.792586] RSP <ffff880037921c20>
>> [ 211.799394] ---[ end trace cbdd0e4e80268faa ]---
>> [ 212.852714] device vif14.0 entered promiscuous mode
>> [ 213.234995] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
>> [ 213.245054] xen_bridge: port 13(vif14.0) entered forwarding state
>> [ 213.252087] xen_bridge: port 13(vif14.0) entered forwarding state
>> [ 214.691532] netbk_gop_frag_copy failed: skb frag 0 page
>> [ 214.698515] copying from offset 26a8, len 2c1
>> [ 214.705472] page:ffffea0000b18400 count:10 mapcount:0 mapping: (null) index:0x0
>> [ 214.712415] page flags: 0x40000000004000(head)
>> [ 214.719170] ------------[ cut here ]------------
>> [ 214.725887] kernel BUG at drivers/net/xen-netback/netback.c:546!
>> [ 214.732563] invalid opcode: 0000 [#4] PREEMPT SMP
>> [ 214.739221] Modules linked in:
>> [ 214.745808] CPU 5
>> [ 214.745859] Pid: 1185, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
>> [ 214.759156] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 214.766127] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
>> [ 214.773012] RAX: 0000000000000001 RBX: ffff8800379172e0 RCX: 0000000000000000
>> [ 214.780010] RDX: ffff880039ac8000 RSI: 0000000000000001 RDI: ffff8800379202b0
>> [ 214.786988] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
>> [ 214.793870] R10: 0000000000000001 R11: ffff880039924460 R12: 0000000000000030
>> [ 214.800812] R13: 0000000000000000 R14: ffff88002b8b4800 R15: 0000000000000001
>> [ 214.807668] FS: 00007f236d331700(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
>> [ 214.814545] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 214.821415] CR2: 00007f236c42b6b0 CR3: 0000000039275000 CR4: 0000000000000660
>> [ 214.828435] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 214.835337] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 214.841963] Process netback/5 (pid: 1185, threadinfo ffff880037922000, task ffff880039ac8000)
>> [ 214.848655] Stack:
>> [ 214.855220] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
>> [ 214.861945] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
>> [ 214.868699] 0000000101080083 ffffc90010858298 0000000100000000 ffff880031e939c0
>> [ 214.875477] Call Trace:
>> [ 214.882247] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
>> [ 214.889083] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
>> [ 214.895851] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
>> [ 214.902612] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
>> [ 214.909343] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
>> [ 214.916115] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
>> [ 214.922856] [<ffffffff810861a6>] kthread+0xd6/0xe0
>> [ 214.929527] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
>> [ 214.936178] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
>> [ 214.942781] [<ffffffff8174e660>] ? gs_change+0x13/0x13
>> [ 214.949279] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
>> [ 214.963107] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
>> [ 214.969952] RSP <ffff880037923c20>
>> [ 214.976802] ---[ end trace cbdd0e4e80268fab ]---
>> [ 216.045946] xen_bridge: port 12(vif12.0) entered forwarding state
>> [ 220.405869] device vif15.0 entered promiscuous mode
>> [ 220.607946] device vif15.0-emu entered promiscuous mode
>> [ 220.625075] xen_bridge: port 15(vif15.0-emu) entered forwarding state
>> [ 220.633333] xen_bridge: port 15(vif15.0-emu) entered forwarding state
>> [ 220.890237] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
>> [ 220.898814] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
>> [ 220.907406] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
>> [ 222.122750] vpn_bridge: port 1(vif13.0) entered forwarding state
>> [ 225.943971] tty_init_dev: 14 callbacks suppressed
>> [ 226.654618] device vif16.0 entered promiscuous mode
>> [ 226.775073] device vif16.0-emu entered promiscuous mode
>> [ 226.784025] xen_bridge: port 17(vif16.0-emu) entered forwarding state
>> [ 226.790188] xen_bridge: port 17(vif16.0-emu) entered forwarding state
>> [ 228.253024] xen_bridge: port 13(vif14.0) entered forwarding state
>> [ 229.788197] xen_bridge: port 15(vif15.0-emu) entered disabled state
>> [ 229.796826] xen_bridge: port 15(vif15.0-emu) entered disabled state
>> [ 229.805243] device vif15.0-emu left promiscuous mode
>> [ 229.813385] xen_bridge: port 15(vif15.0-emu) entered disabled state
>> [ 231.558329] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
>> [ 231.569080] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
>> [ 231.609663] xen_bridge: port 14(vif15.0) entered forwarding state
>> [ 231.617943] xen_bridge: port 14(vif15.0) entered forwarding state
>> [ 231.934347] tty_init_dev: 25 callbacks suppressed
>>
>>
>>
>>
>>
>>
>> > Ian.
>>
>> > diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
>> > index 05593d8..ca4c47d 100644
>> > --- a/drivers/net/xen-netback/netback.c
>> > +++ b/drivers/net/xen-netback/netback.c
>> > @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
>> > * Set up the grant operations for this fragment. If it's a flipping
>> > * interface, we also set up the unmap request from here.
>> > */
>> > -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
>> > +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
>> > struct netrx_pending_operations *npo,
>> > struct page *page, unsigned long size,
>> > unsigned long offset, int *head)
>> > @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
>> > unsigned long bytes;
>> >
>> > /* Data must not cross a page boundary. */
>> > - BUG_ON(size + offset > PAGE_SIZE);
>> > + if (size + offset > PAGE_SIZE)
>> > + return -1;
>> >
>> > meta = npo->meta + npo->meta_prod - 1;
>> >
>> > @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
>> > *head = 0; /* There must be something in this buffer now. */
>> >
>> > }
>> > + return 0;
>> > }
>> >
>> > /*
>> > @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
>> > if (data + len > skb_tail_pointer(skb))
>> > len = skb_tail_pointer(skb) - data;
>> >
>> > - netbk_gop_frag_copy(vif, skb, npo,
>> > - virt_to_page(data), len, offset, &head);
>> > + if (netbk_gop_frag_copy(vif, skb, npo,
>> > + virt_to_page(data), len, offset, &head) < 0) {
>> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
>> + skb->>data, skb_tail_pointer);
>> > +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
>> > + data, data+len, offset, len);
>> > +dump_page(virt_to_page(data));
>> > +BUG();
>> > + }
>> > data += len;
>> > }
>> >
>> > for (i = 0; i < nr_frags; i++) {
>> > - netbk_gop_frag_copy(vif, skb, npo,
>> > + if (netbk_gop_frag_copy(vif, skb, npo,
>> > skb_frag_page(&skb_shinfo(skb)->frags[i]),
>> > skb_frag_size(&skb_shinfo(skb)->frags[i]),
>> > skb_shinfo(skb)->frags[i].page_offset,
>> > - &head);
>> > + &head) < 0) {
>> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
>> > +printk(KERN_CRIT "copying from offset %x, len %x\n",
>> > + skb_shinfo(skb)->frags[i].page_offset,
>> > + skb_frag_size(&skb_shinfo(skb)->frags[i]));
>> > +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
>> > +BUG();
>> > + }
>> > }
>> >
>> > return npo->meta_prod - old_meta_prod;
>>
>>
>>
>>





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, 2012-10-09 at 12:07 +0100, Sander Eikelenboom wrote:
> [ 199.342570] netbk_gop_frag_copy: size 5a8 offset 7102
> [ 199.342570] => 76aa > 1000
> [ 199.354626] netbk_gop_frag_copy failed: skb frag 0 page
> [ 199.360930] copying from offset 7102, len 5a8

OK, this is now at least a real error. Making that last change
(belt&braces) you made shouldn't really have changed anything though :-(

> [ 199.366887] page:ffffea0000b0aa00 count:3 mapcount:0 mapping: (null) index:0x7f40fec00
> [ 199.373008] page flags: 0x40000000004000(head)

The final 0x4000 is indeed PG_head as described, which makes this is a
compound page. This could arise either from the use of transparent huge
pages or via explicit __GFP_comp. It seems that the core networking
stuff can generate these after
69b08f62e174 "net: use bigger pages in __netdev_alloc_frag".

It's not clear to me that the r8169 driver uses those interfaces though,
seems like only tg3 does currently.

In any case it's not obvious how this interacts with bridging and
forwarding, since even if a receiving driver can handle this sort of
thing there's no guarantee that the resending driver can do so (e.g.
netback can't!).

This is one for netdev@ I think. I'll post there and CC you guys.

> [ 199.379252] ------------[ cut here ]------------
> [ 199.385247] kernel BUG at drivers/net/xen-netback/netback.c:548!
> [ 199.391334] invalid opcode: 0000 [#1] PREEMPT SMP
> [ 199.397446] Modules linked in:
> [ 199.403450] CPU 4
> [ 199.403500] Pid: 1183, comm: netback/4 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 199.415401] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 199.421690] RSP: e02b:ffff88003792bc20 EFLAGS: 00010282
> [ 199.428048] RAX: 0000000000000001 RBX: ffff88003197c600 RCX: 0000000000000000
> [ 199.434358] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 199.440582] RBP: ffff88003792bd50 R08: 0000000000000002 R09: 0000000000000000
> [ 199.446740] R10: 0000000000000001 R11: ffff88003a26c000 R12: 0000000000000030
> [ 199.452965] R13: 0000000000000000 R14: ffff88002c2ae900 R15: 0000000000000001
> [ 199.459203] FS: 00007fcec7740700(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
> [ 199.465527] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 199.471735] CR2: 00007fff5f59c000 CR3: 0000000001c0b000 CR4: 0000000000000660
> [ 199.477961] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 199.484102] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 199.490274] Process netback/4 (pid: 1183, threadinfo ffff88003792a000, task ffff880037cec140)
> [ 199.496631] Stack:
> [ 199.502834] ffff88003792bd1c ffff880037cec7f0 ffff88003792bd00 ffff88003792bc80
> [ 199.509198] ffffffff00000001 00000000000005ea ffffc90010851a98 ffffc9001084cf30
> [ 199.515579] 0000000001080083 ffffc9001084cee0 0000000000000000 ffff880032b449c0
> [ 199.521944] Call Trace:
> [ 199.528243] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 199.534566] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
> [ 199.540826] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 199.547193] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 199.553450] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 199.559683] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 199.565827] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 199.572086] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 199.578268] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 199.584344] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 199.597406] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 199.604013] RSP <ffff88003792bc20>
> [ 199.610610] ---[ end trace 03f82ac72747fb5a ]---
> [ 199.990340] device vif11.0 entered promiscuous mode
> [ 200.466710] xen-blkback:ring-ref 9, event-channel 10, protocol 1 (x86_64-abi)
> [ 200.476634] xen_bridge: port 11(vif11.0) entered forwarding state
> [ 200.483621] xen_bridge: port 11(vif11.0) entered forwarding state
> [ 200.653782] pciback 0000:03:06.0: enabling device (0000 -> 0001)
> [ 200.661499] xen: registering gsi 22 triggering 0 polarity 1
> [ 200.669003] Already setup the GSI :22
> [ 200.677345] pciback 0000:03:06.0: enabling bus mastering
> [ 201.267297] xen_bridge: port 9(vif9.0) entered forwarding state
> [ 205.151290] tty_init_dev: 2 callbacks suppressed
> [ 206.534137] device vif12.0 entered promiscuous mode
> [ 206.867366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 206.877552] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 206.884869] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 208.574036] xen_bridge: port 10(vif10.0) entered forwarding state
> [ 209.979799] netbk_gop_frag_copy: size 1080 offset 0
> [ 209.979799] => 1080 > 1000
> [ 209.994252] netbk_gop_frag_copy failed: skb frag 0 page
> [ 210.001191] copying from offset 0, len 1080
> [ 210.008121] page:ffffea0000b0a800 count:3 mapcount:0 mapping: (null) index:0x7f40fec00
> [ 210.015124] page flags: 0x40000000004000(head)
> [ 210.022122] ------------[ cut here ]------------
> [ 210.029035] kernel BUG at drivers/net/xen-netback/netback.c:548!
> [ 210.035973] invalid opcode: 0000 [#2] PREEMPT SMP
> [ 210.042819] Modules linked in:
> [ 210.049467] CPU 0
> [ 210.049518] Pid: 1179, comm: netback/0 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 210.062788] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 210.069740] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
> [ 210.076711] RAX: 0000000000000001 RBX: ffff880031993ae0 RCX: 0000000000000000
> [ 210.083744] RDX: ffff8800398a61e0 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 210.090801] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
> [ 210.097787] R10: 0000000000000001 R11: ffff88003a26b330 R12: 0000000000000030
> [ 210.104759] R13: 0000000000000000 R14: ffff88002b4d8800 R15: 0000000000000001
> [ 210.111611] FS: 00007f695df80700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
> [ 210.118570] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 210.125586] CR2: 00007f695402e000 CR3: 0000000032a8f000 CR4: 0000000000000660
> [ 210.132677] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 210.139560] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 210.146350] Process netback/0 (pid: 1179, threadinfo ffff880037922000, task ffff8800398a61e0)
> [ 210.153213] Stack:
> [ 210.159974] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
> [ 210.166905] ffffffff810800b5 0000000000000662 ffffc90010824bb8 ffffc90010820050
> [ 210.173802] 0000000001080083 ffffc90010820000 0000000000000000 ffff8800375849c0
> [ 210.180780] Call Trace:
> [ 210.187656] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 210.194674] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 210.201690] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
> [ 210.208659] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 210.215688] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 210.222665] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 210.229651] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 210.236455] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 210.243111] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 210.249687] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 210.256195] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 210.270166] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 210.276925] RSP <ffff880037923c20>
> [ 210.284112] ---[ end trace 03f82ac72747fb5b ]---
> [ 213.634083] device vif13.0 entered promiscuous mode
> [ 213.911267] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 213.920749] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 213.927480] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 215.509632] xen_bridge: port 11(vif11.0) entered forwarding state
> [ 215.825483] netbk_gop_frag_copy: size 2c1 offset 12d6
> [ 215.825483] => 1597 > 1000
> [ 215.838666] netbk_gop_frag_copy failed: skb frag 0 page
> [ 215.845265] copying from offset 12d6, len 2c1
> [ 215.851790] page:ffffea0000b0a800 count:6 mapcount:0 mapping: (null) index:0x7f40fec00
> [ 215.858389] page flags: 0x40000000004000(head)
> [ 215.864925] ------------[ cut here ]------------
> [ 215.871426] kernel BUG at drivers/net/xen-netback/netback.c:548!
> [ 215.878069] invalid opcode: 0000 [#3] PREEMPT SMP
> [ 215.884696] Modules linked in:
> [ 215.891258] CPU 3
> [ 215.891308] Pid: 1182, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 215.904613] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 215.911538] RSP: e02b:ffff880037929c20 EFLAGS: 00010282
> [ 215.918336] RAX: 0000000000000001 RBX: ffff88002c361ee0 RCX: 0000000000000000
> [ 215.925236] RDX: ffff880037ced190 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 215.932144] RBP: ffff880037929d50 R08: 0000000000000002 R09: 0000000000000000
> [ 215.938988] R10: 0000000000000001 R11: ffff88003a26aca0 R12: 0000000000000030
> [ 215.945835] R13: 0000000000000000 R14: ffff88002b49b400 R15: 0000000000000001
> [ 215.952652] FS: 00007f695c355700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
> [ 215.959476] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 215.966165] CR2: 00007faa79583000 CR3: 0000000032a8f000 CR4: 0000000000000660
> [ 215.972789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 215.979339] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 215.985844] Process netback/3 (pid: 1182, threadinfo ffff880037928000, task ffff880037ced190)
> [ 215.992486] Stack:
> [ 215.999085] ffff880037929d1c ffff880037928010 ffff880037929d00 ffff880037929c80
> [ 216.005896] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
> [ 216.012651] 0000000101080083 ffffc90010841b28 0000000100000000 ffff880031a869c0
> [ 216.019386] Call Trace:
> [ 216.026026] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 216.032830] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 216.039668] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
> [ 216.046435] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 216.053094] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 216.059670] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 216.066279] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 216.072817] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 216.079308] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 216.085783] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 216.092234] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 216.106108] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 216.113118] RSP <ffff880037929c20>
> [ 216.120011] ---[ end trace 03f82ac72747fb5c ]---
> [ 219.765094] device vif14.0 entered promiscuous mode
> [ 220.062152] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [ 220.072238] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 220.079416] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 221.912781] xen_bridge: port 12(vif12.0) entered forwarding state
> [ 222.876167] netbk_gop_frag_copy: size 2c1 offset 1858
> [ 222.876167] => 1b19 > 1000
> [ 222.889279] netbk_gop_frag_copy failed: skb frag 0 page
> [ 222.895959] copying from offset 1858, len 2c1
> [ 222.902484] page:ffffea0000b0a800 count:8 mapcount:0 mapping: (null) index:0x7f40fec00
> [ 222.909119] page flags: 0x40000000004000(head)
> [ 222.915711] ------------[ cut here ]------------
> [ 222.922307] kernel BUG at drivers/net/xen-netback/netback.c:548!
> [ 222.928950] invalid opcode: 0000 [#4] PREEMPT SMP
> [ 222.935546] Modules linked in:
> [ 222.942110] CPU 5
> [ 222.942161] Pid: 1184, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 222.955415] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 222.962350] RSP: e02b:ffff88003792dc20 EFLAGS: 00010282
> [ 222.969198] RAX: 0000000000000001 RBX: ffff88002b4f4ce0 RCX: 0000000000000000
> [ 222.976119] RDX: ffff880037ceb0f0 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 222.982987] RBP: ffff88003792dd50 R08: 0000000000000002 R09: 0000000000000000
> [ 222.989869] R10: 0000000000000001 R11: ffff88003a26b380 R12: 0000000000000030
> [ 222.996658] R13: 0000000000000000 R14: ffff88002b5a7800 R15: 0000000000000001
> [ 223.003490] FS: 00007f71c6ce2740(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
> [ 223.010257] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 223.016868] CR2: 00007f71c66b4d15 CR3: 0000000031f46000 CR4: 0000000000000660
> [ 223.023470] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 223.029999] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 223.036478] Process netback/5 (pid: 1184, threadinfo ffff88003792c000, task ffff880037ceb0f0)
> [ 223.043095] Stack:
> [ 223.049616] ffff88003792dd1c ffff88003792c010 ffff88003792dd00 ffff88003792dc80
> [ 223.056404] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
> [ 223.063150] 0000000101080083 ffffc90010858298 0000000100000000 ffff88002c38d9c0
> [ 223.069955] Call Trace:
> [ 223.076591] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 223.083426] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 223.090261] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
> [ 223.096990] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 223.103620] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 223.110195] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 223.116768] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 223.123312] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 223.129794] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 223.136217] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 223.142658] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 223.156486] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 223.163337] RSP <ffff88003792dc20>
> [ 223.170212] ---[ end trace 03f82ac72747fb5d ]---
> [ 228.705439] device vif15.0 entered promiscuous mode
> [ 228.880399] device vif15.0-emu entered promiscuous mode
> [ 228.889286] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> [ 228.895546] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> [ 228.956267] vpn_bridge: port 1(vif13.0) entered forwarding state
> [ 229.119709] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
> [ 229.126644] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
> [ 229.133434] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
> [ 234.170536] tty_init_dev: 15 callbacks suppressed
> [ 235.092664] xen_bridge: port 13(vif14.0) entered forwarding state
> [ 235.684229] device vif16.0 entered promiscuous mode
> [ 235.805155] device vif16.0-emu entered promiscuous mode
> [ 235.813948] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> [ 235.820242] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> [ 239.632852] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 239.641629] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 239.650288] device vif15.0-emu left promiscuous mode
> [ 239.658618] xen_bridge: port 15(vif15.0-emu) entered disabled state
> [ 240.982436] tty_init_dev: 15 callbacks suppressed
> [ 241.386562] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
> [ 241.400247] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
> [ 241.454701] xen_bridge: port 14(vif15.0) entered forwarding state
> [ 241.463330] xen_bridge: port 14(vif15.0) entered forwarding state
> [ 246.690393] xen_bridge: port 17(vif16.0-emu) entered disabled state
> [ 246.699042] xen_bridge: port 17(vif16.0-emu) entered disabled state
> [ 246.708731] device vif16.0-emu left promiscuous mode
> [ 246.717465] xen_bridge: port 17(vif16.0-emu) entered disabled state
> [ 249.449321] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
> [ 249.619531] xen_bridge: port 16(vif16.0) entered forwarding state
> [ 249.628307] xen_bridge: port 16(vif16.0) entered forwarding state
> [ 256.489967] xen_bridge: port 14(vif15.0) entered forwarding state
> [ 264.654183] xen_bridge: port 16(vif16.0) entered forwarding state
> [ 414.296535] tty_init_dev: 16 callbacks suppressed
> [ 458.898093] netbk_gop_frag_copy: size 5a8 offset 3602
> [ 458.898093] => 3baa > 1000
> [ 458.920252] netbk_gop_frag_copy failed: skb frag 0 page
> [ 458.928746] copying from offset 3602, len 5a8
> [ 458.937114] page:ffffea0000ada800 count:32749 mapcount:0 mapping: (null) index:0xffff88002b6a6100
> [ 458.945813] page flags: 0x40000000004000(head)
> [ 458.954314] ------------[ cut here ]------------
> [ 458.962655] kernel BUG at drivers/net/xen-netback/netback.c:548!
> [ 458.970929] invalid opcode: 0000 [#5] PREEMPT SMP
> [ 458.979113] Modules linked in:
> [ 458.987128] CPU 1
> [ 458.987178] Pid: 1180, comm: netback/1 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> [ 459.003052] RIP: e030:[<ffffffff8147463a>] [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 459.011121] RSP: e02b:ffff880037925c20 EFLAGS: 00010282
> [ 459.019135] RAX: 0000000000000001 RBX: ffff88002ab0bf00 RCX: 0000000000000000
> [ 459.027199] RDX: ffff8800398a30f0 RSI: 0000000000000001 RDI: ffff8800379202b0
> [ 459.035081] RBP: ffff880037925d50 R08: 0000000000000002 R09: 0000000000000000
> [ 459.042816] R10: 0000000000000001 R11: ffff88003a26bdb0 R12: 0000000000000030
> [ 459.050308] R13: 0000000000000000 R14: ffff88002b6a2e00 R15: 0000000000000001
> [ 459.057725] FS: 00007f8e25af5760(0000) GS:ffff88003f840000(0000) knlGS:0000000000000000
> [ 459.065052] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 459.072248] CR2: 00007fe6b4d12fb0 CR3: 000000002c2f6000 CR4: 0000000000000660
> [ 459.079480] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 459.086512] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 459.093386] Process netback/1 (pid: 1180, threadinfo ffff880037924000, task ffff8800398a30f0)
> [ 459.100357] Stack:
> [ 459.107071] ffff880037925d1c ffff880037924010 ffff880037925d00 ffff880037925c80
> [ 459.113808] ffffffff810800b5 000000000000042a ffffc9001082ff70 ffffc9001082b408
> [ 459.120494] 0000000001080083 ffffc9001082b3b8 0000000000000000 ffff8800329249c0
> [ 459.127129] Call Trace:
> [ 459.133509] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> [ 459.140118] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> [ 459.146604] [<ffffffff8147569a>] xen_netbk_kthread+0xba/0xa90
> [ 459.153504] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> [ 459.159949] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> [ 459.166431] [<ffffffff814755e0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> [ 459.172778] [<ffffffff810861a6>] kthread+0xd6/0xe0
> [ 459.179018] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> [ 459.185291] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> [ 459.191523] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> [ 459.197862] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 36 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> [ 459.211184] RIP [<ffffffff8147463a>] xen_netbk_rx_action+0x89a/0x910
> [ 459.217785] RSP <ffff880037925c20>
> [ 459.224501] ---[ end trace 03f82ac72747fb5e ]---
>
>
>
>
> > This made me notice that offset and len in the caller are variously
> > unsigned int, u16 or u32 while gop_frag_copy takes them as unsigned
> > longs. None of the numbers involved here are anywhere big enough to
> > cause any sort of overflow related error though.
>
> >> [ 197.892781] page:ffffea0000b18400 count:3 mapcount:0 mapping: (null) index:0x0
> >> [ 197.900778] page flags: 0x40000000004000(head)
> >> [ 197.907074] ------------[ cut here ]------------
> >> [ 197.913345] kernel BUG at drivers/net/xen-netback/netback.c:546!
> >> [ 197.919626] invalid opcode: 0000 [#1] PREEMPT SMP
> >> [ 197.921573] xen_bridge: port 10(vif10.0) entered forwarding state
> >> [ 197.932106] Modules linked in:
> >> [ 197.938370] CPU 0
> >> [ 197.938420] Pid: 1180, comm: netback/0 Not tainted 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> >> [ 197.951203] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 197.957775] RSP: e02b:ffff880037911c20 EFLAGS: 00010282
> >> [ 197.964290] RAX: 0000000000000001 RBX: ffff880036862ee0 RCX: 0000000000000000
> >> [ 197.970956] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8800379102b0
> >> [ 197.977679] RBP: ffff880037911d50 R08: 0000000000000002 R09: 0000000000000000
> >> [ 197.984361] R10: 0000000000000001 R11: ffff880039925e40 R12: 0000000000000030
> >> [ 197.990958] R13: 0000000000000000 R14: ffff880031e71800 R15: 0000000000000001
> >> [ 197.997459] FS: 00007fb5dfcf7700(0000) GS:ffff88003f800000(0000) knlGS:0000000000000000
> >> [ 198.004123] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> >> [ 198.010827] CR2: 00007fb5d802d000 CR3: 0000000031fd3000 CR4: 0000000000000660
> >> [ 198.017534] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> [ 198.024168] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >> [ 198.030717] Process netback/0 (pid: 1180, threadinfo ffff880037910000, task ffff88003997d190)
> >> [ 198.037326] Stack:
> >> [ 198.043817] ffff880037911d1c ffff88003997d840 ffff880037911d00 ffff880037911c80
> >> [ 198.050573] ffffffff00000001 0000000000000662 ffffc90010824bb8 ffffc90010820050
> >> [ 198.057413] 0000000001080083 ffffc90010820000 0000000000000000 ffff880031cf09c0
> >> [ 198.064228] Call Trace:
> >> [ 198.070887] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> >> [ 198.077604] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> >> [ 198.084394] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> >> [ 198.091109] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> >> [ 198.097726] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> >> [ 198.104343] [<ffffffff810861a6>] kthread+0xd6/0xe0
> >> [ 198.111001] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> >> [ 198.117737] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> >> [ 198.124425] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> >> [ 198.131008] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> >> [ 198.145094] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 198.152192] RSP <ffff880037911c20>
> >> [ 198.159344] ---[ end trace cbdd0e4e80268fa8 ]---
> >> [ 199.703539] tty_init_dev: 2 callbacks suppressed
> >> [ 200.712098] device vif12.0 entered promiscuous mode
> >> [ 201.010433] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> >> [ 201.020644] xen_bridge: port 12(vif12.0) entered forwarding state
> >> [ 201.027833] xen_bridge: port 12(vif12.0) entered forwarding state
> >> [ 206.774576] netbk_gop_frag_copy failed: skb frag 0 page
> >> [ 206.777945] device vif13.0 entered promiscuous mode
> >> [ 206.788845] copying from offset 1ba4, len 2c1
> >> [ 206.795791] page:ffffea0000b18400 count:6 mapcount:0 mapping: (null) index:0x0
> >> [ 206.802771] page flags: 0x40000000004000(head)
> >> [ 206.809619] ------------[ cut here ]------------
> >> [ 206.816498] kernel BUG at drivers/net/xen-netback/netback.c:546!
> >> [ 206.823465] invalid opcode: 0000 [#2] PREEMPT SMP
> >> [ 206.830354] Modules linked in:
> >> [ 206.837176] CPU 3
> >> [ 206.837234] Pid: 1183, comm: netback/3 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> >> [ 206.850881] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 206.857935] RSP: e02b:ffff880037917c20 EFLAGS: 00010282
> >> [ 206.864972] RAX: 0000000000000001 RBX: ffff880003313ae0 RCX: 0000000000000000
> >> [ 206.872049] RDX: ffff88003997b0f0 RSI: 0000000000000001 RDI: ffff8800379102b0
> >> [ 206.879147] RBP: ffff880037917d50 R08: 0000000000000002 R09: 0000000000000000
> >> [ 206.886242] R10: 0000000000000001 R11: ffff880039925640 R12: 0000000000000030
> >> [ 206.893163] R13: 0000000000000000 R14: ffff88002c7c4400 R15: 0000000000000001
> >> [ 206.900041] FS: 00007f800341a700(0000) GS:ffff88003f8c0000(0000) knlGS:0000000000000000
> >> [ 206.907145] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> >> [ 206.914126] CR2: 00007f8002b31fb0 CR3: 0000000001c0b000 CR4: 0000000000000660
> >> [ 206.921181] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> [ 206.927996] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >> [ 206.934711] Process netback/3 (pid: 1183, threadinfo ffff880037916000, task ffff88003997b0f0)
> >> [ 206.941494] Stack:
> >> [ 206.948105] ffff880037917d1c ffff880037916010 ffff880037917d00 ffff880037917c80
> >> [ 206.955062] ffffffff810800b5 00000000000000ba ffffc900108466e0 ffffc90010841b78
> >> [ 206.962007] 0000000101080083 ffffc90010841b28 0000000100000000 ffff88002c5bb9c0
> >> [ 206.968967] Call Trace:
> >> [ 206.975830] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> >> [ 206.982789] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> >> [ 206.989662] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> >> [ 206.996570] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> >> [ 207.003523] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> >> [ 207.010333] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> >> [ 207.017171] [<ffffffff810861a6>] kthread+0xd6/0xe0
> >> [ 207.023890] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> >> [ 207.030540] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> >> [ 207.037275] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> >> [ 207.043890] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> >> [ 207.057976] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 207.065064] RSP <ffff880037917c20>
> >> [ 207.072056] ---[ end trace cbdd0e4e80268fa9 ]---
> >> [ 207.079366] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> >> [ 207.090256] vpn_bridge: port 1(vif13.0) entered forwarding state
> >> [ 207.097403] vpn_bridge: port 1(vif13.0) entered forwarding state
> >> [ 208.636257] xen_bridge: port 11(vif11.0) entered forwarding state
> >> [ 211.515779] netbk_gop_frag_copy failed: skb frag 0 page
> >> [ 211.522711] copying from offset 2126, len 2c1
> >> [ 211.529403] page:ffffea0000b18400 count:8 mapcount:0 mapping: (null) index:0x0
> >> [ 211.536142] page flags: 0x40000000004000(head)
> >> [ 211.542942] ------------[ cut here ]------------
> >> [ 211.549664] kernel BUG at drivers/net/xen-netback/netback.c:546!
> >> [ 211.556408] invalid opcode: 0000 [#3] PREEMPT SMP
> >> [ 211.563168] Modules linked in:
> >> [ 211.569739] CPU 4
> >> [ 211.569789] Pid: 1184, comm: netback/4 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> >> [ 211.583126] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 211.590041] RSP: e02b:ffff880037921c20 EFLAGS: 00010282
> >> [ 211.596868] RAX: 0000000000000001 RBX: ffff8800375bc4e0 RCX: 0000000000000000
> >> [ 211.603890] RDX: ffff88003997a0a0 RSI: 0000000000000001 RDI: ffff8800379202b0
> >> [ 211.610792] RBP: ffff880037921d50 R08: 0000000000000002 R09: 0000000000000000
> >> [ 211.617608] R10: 0000000000000001 R11: ffff8800399249e0 R12: 0000000000000030
> >> [ 211.624537] R13: 0000000000000000 R14: ffff88002b98d400 R15: 0000000000000001
> >> [ 211.631302] FS: 00007f332d735740(0000) GS:ffff88003f900000(0000) knlGS:0000000000000000
> >> [ 211.638090] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> >> [ 211.644965] CR2: 00007f1023d22000 CR3: 0000000031fba000 CR4: 0000000000000660
> >> [ 211.651894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> [ 211.658652] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >> [ 211.665288] Process netback/4 (pid: 1184, threadinfo ffff880037920000, task ffff88003997a0a0)
> >> [ 211.671884] Stack:
> >> [ 211.678376] ffff880037921d1c ffff880037920010 ffff880037921d00 ffff880037921c80
> >> [ 211.685145] ffffffff810800b5 00000000000000ba ffffc90010851a98 ffffc9001084cf30
> >> [ 211.691837] 0000000101080083 ffffc9001084cee0 0000000100000000 ffff88002c5bd9c0
> >> [ 211.698581] Call Trace:
> >> [ 211.705349] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> >> [ 211.712156] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> >> [ 211.718907] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> >> [ 211.725654] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> >> [ 211.732369] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> >> [ 211.739111] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> >> [ 211.745858] [<ffffffff810861a6>] kthread+0xd6/0xe0
> >> [ 211.752449] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> >> [ 211.758975] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> >> [ 211.765575] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> >> [ 211.772016] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> >> [ 211.785816] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 211.792586] RSP <ffff880037921c20>
> >> [ 211.799394] ---[ end trace cbdd0e4e80268faa ]---
> >> [ 212.852714] device vif14.0 entered promiscuous mode
> >> [ 213.234995] xen-blkback:ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> >> [ 213.245054] xen_bridge: port 13(vif14.0) entered forwarding state
> >> [ 213.252087] xen_bridge: port 13(vif14.0) entered forwarding state
> >> [ 214.691532] netbk_gop_frag_copy failed: skb frag 0 page
> >> [ 214.698515] copying from offset 26a8, len 2c1
> >> [ 214.705472] page:ffffea0000b18400 count:10 mapcount:0 mapping: (null) index:0x0
> >> [ 214.712415] page flags: 0x40000000004000(head)
> >> [ 214.719170] ------------[ cut here ]------------
> >> [ 214.725887] kernel BUG at drivers/net/xen-netback/netback.c:546!
> >> [ 214.732563] invalid opcode: 0000 [#4] PREEMPT SMP
> >> [ 214.739221] Modules linked in:
> >> [ 214.745808] CPU 5
> >> [ 214.745859] Pid: 1185, comm: netback/5 Tainted: G D 3.6.0pre-rc1-20121008bisect #1 MSI MS-7640/890FXA-GD70 (MS-7640)
> >> [ 214.759156] RIP: e030:[<ffffffff8147462a>] [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 214.766127] RSP: e02b:ffff880037923c20 EFLAGS: 00010282
> >> [ 214.773012] RAX: 0000000000000001 RBX: ffff8800379172e0 RCX: 0000000000000000
> >> [ 214.780010] RDX: ffff880039ac8000 RSI: 0000000000000001 RDI: ffff8800379202b0
> >> [ 214.786988] RBP: ffff880037923d50 R08: 0000000000000002 R09: 0000000000000000
> >> [ 214.793870] R10: 0000000000000001 R11: ffff880039924460 R12: 0000000000000030
> >> [ 214.800812] R13: 0000000000000000 R14: ffff88002b8b4800 R15: 0000000000000001
> >> [ 214.807668] FS: 00007f236d331700(0000) GS:ffff88003f940000(0000) knlGS:0000000000000000
> >> [ 214.814545] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> >> [ 214.821415] CR2: 00007f236c42b6b0 CR3: 0000000039275000 CR4: 0000000000000660
> >> [ 214.828435] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> [ 214.835337] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >> [ 214.841963] Process netback/5 (pid: 1185, threadinfo ffff880037922000, task ffff880039ac8000)
> >> [ 214.848655] Stack:
> >> [ 214.855220] ffff880037923d1c ffff880037922010 ffff880037923d00 ffff880037923c80
> >> [ 214.861945] ffffffff810800b5 00000000000000ba ffffc9001085ce50 ffffc900108582e8
> >> [ 214.868699] 0000000101080083 ffffc90010858298 0000000100000000 ffff880031e939c0
> >> [ 214.875477] Call Trace:
> >> [ 214.882247] [<ffffffff810800b5>] ? __alloc_workqueue_key+0x265/0x5d0
> >> [ 214.889083] [<ffffffff810acf3d>] ? trace_hardirqs_on+0xd/0x10
> >> [ 214.895851] [<ffffffff8147568a>] xen_netbk_kthread+0xba/0xa90
> >> [ 214.902612] [<ffffffff810957e6>] ? try_to_wake_up+0x1b6/0x310
> >> [ 214.909343] [<ffffffff81086810>] ? wake_up_bit+0x40/0x40
> >> [ 214.916115] [<ffffffff814755d0>] ? xen_netbk_tx_build_gops+0xa70/0xa70
> >> [ 214.922856] [<ffffffff810861a6>] kthread+0xd6/0xe0
> >> [ 214.929527] [<ffffffff8174e664>] kernel_thread_helper+0x4/0x10
> >> [ 214.936178] [<ffffffff8174cb37>] ? retint_restore_args+0x13/0x13
> >> [ 214.942781] [<ffffffff8174e660>] ? gs_change+0x13/0x13
> >> [ 214.949279] Code: 00 00 00 42 8b 54 30 3c 41 8b 74 04 08 31 c0 e8 e5 37 2d 00 8b 83 c4 00 00 00 4c 03 b3 c8 00 00 00 4a 8b 7c 30 30 e8 46 24 c8 ff <0f> 0b eb fe 48 8b b3 d0 00 00 00 48 c7 c2 c0 36 47 81 48 c7 c7
> >> [ 214.963107] RIP [<ffffffff8147462a>] xen_netbk_rx_action+0x89a/0x910
> >> [ 214.969952] RSP <ffff880037923c20>
> >> [ 214.976802] ---[ end trace cbdd0e4e80268fab ]---
> >> [ 216.045946] xen_bridge: port 12(vif12.0) entered forwarding state
> >> [ 220.405869] device vif15.0 entered promiscuous mode
> >> [ 220.607946] device vif15.0-emu entered promiscuous mode
> >> [ 220.625075] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> >> [ 220.633333] xen_bridge: port 15(vif15.0-emu) entered forwarding state
> >> [ 220.890237] pciback 0000:06:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x10a)
> >> [ 220.898814] pciback 0000:06:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf9a00004)
> >> [ 220.907406] pciback 0000:06:00.0: restoring config space at offset 0xc (was 0x0, writing 0x10)
> >> [ 222.122750] vpn_bridge: port 1(vif13.0) entered forwarding state
> >> [ 225.943971] tty_init_dev: 14 callbacks suppressed
> >> [ 226.654618] device vif16.0 entered promiscuous mode
> >> [ 226.775073] device vif16.0-emu entered promiscuous mode
> >> [ 226.784025] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> >> [ 226.790188] xen_bridge: port 17(vif16.0-emu) entered forwarding state
> >> [ 228.253024] xen_bridge: port 13(vif14.0) entered forwarding state
> >> [ 229.788197] xen_bridge: port 15(vif15.0-emu) entered disabled state
> >> [ 229.796826] xen_bridge: port 15(vif15.0-emu) entered disabled state
> >> [ 229.805243] device vif15.0-emu left promiscuous mode
> >> [ 229.813385] xen_bridge: port 15(vif15.0-emu) entered disabled state
> >> [ 231.558329] xen-blkback:ring-ref 8, event-channel 25, protocol 1 (x86_64-abi)
> >> [ 231.569080] xen-blkback:ring-ref 9, event-channel 26, protocol 1 (x86_64-abi)
> >> [ 231.609663] xen_bridge: port 14(vif15.0) entered forwarding state
> >> [ 231.617943] xen_bridge: port 14(vif15.0) entered forwarding state
> >> [ 231.934347] tty_init_dev: 25 callbacks suppressed
> >>
> >>
> >>
> >>
> >>
> >>
> >> > Ian.
> >>
> >> > diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> >> > index 05593d8..ca4c47d 100644
> >> > --- a/drivers/net/xen-netback/netback.c
> >> > +++ b/drivers/net/xen-netback/netback.c
> >> > @@ -386,7 +386,7 @@ static struct netbk_rx_meta *get_next_rx_buffer(struct xenvif *vif,
> >> > * Set up the grant operations for this fragment. If it's a flipping
> >> > * interface, we also set up the unmap request from here.
> >> > */
> >> > -static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> >> > +static int netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> >> > struct netrx_pending_operations *npo,
> >> > struct page *page, unsigned long size,
> >> > unsigned long offset, int *head)
> >> > @@ -402,7 +402,8 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> >> > unsigned long bytes;
> >> >
> >> > /* Data must not cross a page boundary. */
> >> > - BUG_ON(size + offset > PAGE_SIZE);
> >> > + if (size + offset > PAGE_SIZE)
> >> > + return -1;
> >> >
> >> > meta = npo->meta + npo->meta_prod - 1;
> >> >
> >> > @@ -459,6 +460,7 @@ static void netbk_gop_frag_copy(struct xenvif *vif, struct sk_buff *skb,
> >> > *head = 0; /* There must be something in this buffer now. */
> >> >
> >> > }
> >> > + return 0;
> >> > }
> >> >
> >> > /*
> >> > @@ -517,17 +519,31 @@ static int netbk_gop_skb(struct sk_buff *skb,
> >> > if (data + len > skb_tail_pointer(skb))
> >> > len = skb_tail_pointer(skb) - data;
> >> >
> >> > - netbk_gop_frag_copy(vif, skb, npo,
> >> > - virt_to_page(data), len, offset, &head);
> >> > + if (netbk_gop_frag_copy(vif, skb, npo,
> >> > + virt_to_page(data), len, offset, &head) < 0) {
> >> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb head %p-%p\n",
> >> + skb->>data, skb_tail_pointer);
> >> > +printk(KERN_CRIT "copying from %p-%p, offset %x, len %x\n",
> >> > + data, data+len, offset, len);
> >> > +dump_page(virt_to_page(data));
> >> > +BUG();
> >> > + }
> >> > data += len;
> >> > }
> >> >
> >> > for (i = 0; i < nr_frags; i++) {
> >> > - netbk_gop_frag_copy(vif, skb, npo,
> >> > + if (netbk_gop_frag_copy(vif, skb, npo,
> >> > skb_frag_page(&skb_shinfo(skb)->frags[i]),
> >> > skb_frag_size(&skb_shinfo(skb)->frags[i]),
> >> > skb_shinfo(skb)->frags[i].page_offset,
> >> > - &head);
> >> > + &head) < 0) {
> >> > +printk(KERN_CRIT "netbk_gop_frag_copy failed: skb frag %d page\n", i);
> >> > +printk(KERN_CRIT "copying from offset %x, len %x\n",
> >> > + skb_shinfo(skb)->frags[i].page_offset,
> >> > + skb_frag_size(&skb_shinfo(skb)->frags[i]));
> >> > +dump_page(skb_frag_page(&skb_shinfo(skb)->frags[i]));
> >> > +BUG();
> >> > + }
> >> > }
> >> >
> >> > return npo->meta_prod - old_meta_prod;
> >>
> >>
> >>
> >>
>
>
>
>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel