Mailing List Archive

2011/9/23 Daniel De Graaf <dgdegra@tycho.nsa.gov>

> Changes since v5:
> - Unify gntdev osdep interface
> - Eliminate notify_result and revert mapping if notify ioctl fails
> - Rename functions and structures to libxenvchan
> - Use application-specified xenstore path for ring/event data
> - Enforce maximum ring size of 2^20 bytes on client
> - Change to LGPL 2.1
>
> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
> [PATCH 2/3] libxc: add xc_gntshr_* functions
> [PATCH 3/3] libvchan: interdomain communications library
>
>
>
Hello. Sorry for bumping..
What version of xen kernel i need to use this library? Now i have 2.6.32.26
in many domUs.
And what i need in dom0 for that, if i want to communicate via libxenvchan
from domU to dom0?


--
Vasiliy Tolstov,
Clodo.ru
e-mail: v.tolstov@selfip.ru
jabber: vase@selfip.ru

2011/9/30 Vasiliy Tolstov <v.tolstov@selfip.ru>

>
>
> 2011/9/23 Daniel De Graaf <dgdegra@tycho.nsa.gov>
>
>> Changes since v5:
>> - Unify gntdev osdep interface
>> - Eliminate notify_result and revert mapping if notify ioctl fails
>> - Rename functions and structures to libxenvchan
>> - Use application-specified xenstore path for ring/event data
>> - Enforce maximum ring size of 2^20 bytes on client
>> - Change to LGPL 2.1
>>
>> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
>> [PATCH 2/3] libxc: add xc_gntshr_* functions
>> [PATCH 3/3] libvchan: interdomain communications library
>>
>>
>>
> Hello. Sorry for bumping..
> What version of xen kernel i need to use this library? Now i have 2.6.32.26
> in many domUs.
> And what i need in dom0 for that, if i want to communicate via libxenvchan
> from domU to dom0?
>
>
>
And where i can find latest version of the patch?

--
Vasiliy Tolstov,
Clodo.ru
e-mail: v.tolstov@selfip.ru
jabber: vase@selfip.ru

On Thu, 2011-09-22 at 23:14 +0100, Daniel De Graaf wrote:
> Changes since v5:
> - Unify gntdev osdep interface
> - Eliminate notify_result and revert mapping if notify ioctl fails
> - Rename functions and structures to libxenvchan
> - Use application-specified xenstore path for ring/event data
> - Enforce maximum ring size of 2^20 bytes on client
> - Change to LGPL 2.1
>
> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
> [PATCH 2/3] libxc: add xc_gntshr_* functions
> [PATCH 3/3] libvchan: interdomain communications library

I meant to say this before but, modulo the spurious changes to
tools/libxl/libxlu_cfg_l.[ch] in the first patch, the whole lot are:

Acked-by: Ian Campbell <ian.campbell@citrix.com>

The error reporting for missing XC_OSDEP_GNTSDHR is missing in
xc_netbsd.c but I think that should be pulled out into common code --
I'll send out a patch.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 09/30/2011 04:28 AM, Vasiliy Tolstov wrote:
> 2011/9/30 Vasiliy Tolstov <v.tolstov@selfip.ru>
>
>>
>>
>> 2011/9/23 Daniel De Graaf <dgdegra@tycho.nsa.gov>
>>
>>> Changes since v5:
>>> - Unify gntdev osdep interface
>>> - Eliminate notify_result and revert mapping if notify ioctl fails
>>> - Rename functions and structures to libxenvchan
>>> - Use application-specified xenstore path for ring/event data
>>> - Enforce maximum ring size of 2^20 bytes on client
>>> - Change to LGPL 2.1
>>>
>>> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
>>> [PATCH 2/3] libxc: add xc_gntshr_* functions
>>> [PATCH 3/3] libvchan: interdomain communications library
>>>
>>>
>>>
>> Hello. Sorry for bumping..
>> What version of xen kernel i need to use this library? Now i have 2.6.32.26
>> in many domUs.
>> And what i need in dom0 for that, if i want to communicate via libxenvchan
>> from domU to dom0?
>>
>>
>>
> And where i can find latest version of the patch?
>

This library depends on gntdev for the client and gntalloc for the server;
these were merged in 2.6.39. The client can be used with 2.6.32.x kernels
however it is not possible to detect when a peer crashes or exits without
calling libxenvchan_close() on that kernel.

This library does not require that either peer is dom0; it can be used for
both domU-domU and domU-dom0 communication.

The latest version of the patch can be found in the xen-devel archives.

--
Daniel De Graaf
National Security Agency

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Campbell writes ("[Xen-devel] Re: [PATCH v6 0/3] libxenvchan: interdomain communications library"):
> I meant to say this before but, modulo the spurious changes to
> tools/libxl/libxlu_cfg_l.[ch] in the first patch, the whole lot are:
>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>

Thanks, I have applied all three minus those changes to
libxlu_cfg_l.[ch], which I will post separately.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Why are the binary files tools/libvchan/vchan-node1 and
tools/libvchan/vchan-node2 added to the repository?

Now after compilation I get:

M tools/libvchan/vchan-node1
M tools/libvchan/vchan-node2

Because the compilation process changes them, and I cannot navigate
trough my patch list because of this local changes.

Regards, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 07/10/2011 09:41, "Roger Pau Monné" <roger.pau@entel.upc.edu> wrote:

> Why are the binary files tools/libvchan/vchan-node1 and
> tools/libvchan/vchan-node2 added to the repository?
>
> Now after compilation I get:
>
> M tools/libvchan/vchan-node1
> M tools/libvchan/vchan-node2
>
> Because the compilation process changes them, and I cannot navigate
> trough my patch list because of this local changes.

Probably added in error after a test build. The problem is that they should
be listed in .hgignore.

-- Keir

> Regards, Roger.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser writes ("Re: [Xen-devel] Re: [PATCH v6 0/3] libxenvchan: interdomain communications library"):
> On 07/10/2011 09:41, "Roger Pau Monné" <roger.pau@entel.upc.edu> wrote:
> > Why are the binary files tools/libvchan/vchan-node1 and
> > tools/libvchan/vchan-node2 added to the repository?
> >
> > Now after compilation I get:
> >
> > M tools/libvchan/vchan-node1
> > M tools/libvchan/vchan-node2
> >
> > Because the compilation process changes them, and I cannot navigate
> > trough my patch list because of this local changes.

Sorry about this.

> Probably added in error after a test build. The problem is that they should
> be listed in .hgignore.

Yes, now fixed in staging and will be in the tested tree later today.

# HG changeset patch
# User Ian Jackson <Ian.Jackson@eu.citrix.com>
# Date 1317980824 -3600
# Node ID eb8637351535e9c76779d6dd78a1d5b4a434c082
# Parent 5b5394a55813dfa778a8c944bc0e6a0956629f64
libvchan: remove erroneously-committed libvchan-node[12]; add them to .hgignore

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>

diff -r 5b5394a55813 -r eb8637351535 .hgignore
--- a/.hgignore Fri Oct 07 09:35:31 2011 +0100
+++ b/.hgignore Fri Oct 07 10:47:04 2011 +0100
@@ -189,6 +189,7 @@
^tools/libxl/xl$
^tools/libxl/testidl$
^tools/libxl/testidl\.c$
+^tools/libvchan/vchan-node[12]$
^tools/libaio/src/.*\.ol$
^tools/libaio/src/.*\.os$
^tools/misc/cpuperf/cpuperf-perfcntr$
diff -r 5b5394a55813 -r eb8637351535 tools/libvchan/vchan-node1
Binary file tools/libvchan/vchan-node1 has changed
diff -r 5b5394a55813 -r eb8637351535 tools/libvchan/vchan-node2
Binary file tools/libvchan/vchan-node2 has changed

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

2011/10/7 Ian Jackson <Ian.Jackson@eu.citrix.com>:
> Keir Fraser writes ("Re: [Xen-devel] Re: [PATCH v6 0/3] libxenvchan: interdomain communications library"):
>> On 07/10/2011 09:41, "Roger Pau Monné" <roger.pau@entel.upc.edu> wrote:
>> > Why are the binary files tools/libvchan/vchan-node1 and
>> > tools/libvchan/vchan-node2 added to the repository?
>> >
>> > Now after compilation I get:
>> >
>> > M tools/libvchan/vchan-node1
>> > M tools/libvchan/vchan-node2
>> >
>> > Because the compilation process changes them, and I cannot navigate
>> > trough my patch list because of this local changes.
>
> Sorry about this.

NP, it's just that I didn't know if it had anything to do with
building Xen on NetBSD. Thanks!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Sep 30, 2011 at 10:40:11AM -0400, Daniel De Graaf wrote:
> >>
> >> 2011/9/23 Daniel De Graaf <dgdegra@tycho.nsa.gov>
> >>
> >>> Changes since v5:
> >>> - Unify gntdev osdep interface
> >>> - Eliminate notify_result and revert mapping if notify ioctl fails
> >>> - Rename functions and structures to libxenvchan
> >>> - Use application-specified xenstore path for ring/event data
> >>> - Enforce maximum ring size of 2^20 bytes on client
> >>> - Change to LGPL 2.1
> >>>
> >>> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
> >>> [PATCH 2/3] libxc: add xc_gntshr_* functions
> >>> [PATCH 3/3] libvchan: interdomain communications library
> >>>
<snip>
> This library depends on gntdev for the client and gntalloc for the server;
> these were merged in 2.6.39. The client can be used with 2.6.32.x kernels
> however it is not possible to detect when a peer crashes or exits without
> calling libxenvchan_close() on that kernel.

I'm trying this with your most recent dom0 patches all included, and
xen-unstable. In order to get decent performance, I tried the vchan-node
examples with a cranked up buffer size, which fails immediately on 64-bit
VMs. This below patch fixes the xc_gntshr_share_pages call (which sets
notify_offset to -1 and without this bounds check will send a big offset
to the unmap notify ioctl).

With this, communication does work with the big buffers, but it never
frees them, and so I can get an ENOSPACE error by calling vchan-node a few
times. What's the intended cleanup path for the grants in normal use of
the vchan library? Something's holding onto them, but I don't have a
serial console on my current laptop in order to drop into Xen's serial and
find out more.

diff -r 614e9f371209 tools/libxc/xc_linux_osdep.c
--- a/tools/libxc/xc_linux_osdep.c Thu Nov 24 18:53:09 2011 +0000
+++ b/tools/libxc/xc_linux_osdep.c Thu Nov 24 20:00:21 2011 +0000
@@ -709,7 +709,7 @@

notify.index = gref_info->index;
notify.action = 0;
- if (notify_offset >= 0) {
+ if (notify_offset >= 0 && notify_offset < XC_PAGE_SIZE * count) {
notify.index += notify_offset;
notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
}


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 11/24/2011 03:02 PM, Anil Madhavapeddy wrote:
> On Fri, Sep 30, 2011 at 10:40:11AM -0400, Daniel De Graaf wrote:
>>>>
>>>> 2011/9/23 Daniel De Graaf <dgdegra@tycho.nsa.gov>
>>>>
>>>>> Changes since v5:
>>>>> - Unify gntdev osdep interface
>>>>> - Eliminate notify_result and revert mapping if notify ioctl fails
>>>>> - Rename functions and structures to libxenvchan
>>>>> - Use application-specified xenstore path for ring/event data
>>>>> - Enforce maximum ring size of 2^20 bytes on client
>>>>> - Change to LGPL 2.1
>>>>>
>>>>> [PATCH 1/3] libxc: add xc_gnttab_map_grant_ref_notify
>>>>> [PATCH 2/3] libxc: add xc_gntshr_* functions
>>>>> [PATCH 3/3] libvchan: interdomain communications library
>>>>>
> <snip>
>> This library depends on gntdev for the client and gntalloc for the server;
>> these were merged in 2.6.39. The client can be used with 2.6.32.x kernels
>> however it is not possible to detect when a peer crashes or exits without
>> calling libxenvchan_close() on that kernel.
>
> I'm trying this with your most recent dom0 patches all included, and
> xen-unstable. In order to get decent performance, I tried the vchan-node
> examples with a cranked up buffer size, which fails immediately on 64-bit
> VMs. This below patch fixes the xc_gntshr_share_pages call (which sets
> notify_offset to -1 and without this bounds check will send a big offset
> to the unmap notify ioctl).
>
> With this, communication does work with the big buffers, but it never
> frees them, and so I can get an ENOSPACE error by calling vchan-node a few
> times. What's the intended cleanup path for the grants in normal use of
> the vchan library? Something's holding onto them, but I don't have a
> serial console on my current laptop in order to drop into Xen's serial and
> find out more.
>

As long as things aren't crashing, you should be able to use "xl debug-key g"
and "xl dmesg" together to view the grant table debug output.

The ENOSPC bug is due to a missing gnttab_free_grant_reference call, the patch
to fix should be sent as a reply. The libxc patch fixing notify_offset is
incomplete; a more complete version will also be sent as a reply.

It is possible to observe this ENOSPC error even with proper cleanup if the
client refuses to release the grants it has mapped. Xen does not allow domains
to force unmapping of pages they have granted, so the best we can do is try to
remove any hanging grants on later gntalloc calls.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel