Mailing List Archive

Starting Points for the Xen Introspection Project
Here are some of my thoughts on the Xen Introspection Project. We've agreed
that creating a list of requirements is the first thing the project should
do. We listed some general requirements during the first teleconference.
From ATC-NY's perspective, our highest priority feature is the ability to
trap on guest memory accesses (rwx) to specific addresses.

There are no publicly available tools that allow us to do this with recent
versions of Xen or support both Windows and Linux. So it is an open
question whether we should start with a new library or adapt an existing
introspection tool. I believe to get something running quickly we should
adapt an existing project and tweak the API over time. It is better to have
something running than to fully design a project before implementation.
What should be the basis for the introspection tools? There are three
introspection projects described in recent literature that are Xen specific:

XenAccess [1] - started by Bryan Payne from Georgia Tech. It builds on
top of XenCtrl. Its API concentrates on mmapping domU memory into
domO. It has some advantages in that it supports both Linux and
Windows. It can also mmap domU process memory into dom0, which is
a potential benefit to reverse engineering tools. It is GPL'd.
Source is available from xenaccess.googlecode.com. (I am most
familiar with this project because ATC-NY uses it as a base for
our own tools and I am a minor contributer.)

VIX (Virtual Introspection for Xen) [2]- Hay and Nance's VM
introspection tools for forensics and a library utility for
building them. Linux 2.6 specific, it consists of specific
forensic tools along with a utility library that assists in
mmapping domU kernel memory. Is source available?

Ether [3] - Another effort from Georgia Tech that concentrates more
on hypervisor modifications and is specific to Intel VT HVM domUs.
This was recently described at ACM CCS 2008. The group has
apparently abandoned Xen in favor of KVM. However, source for the
Xen modifications are available. Their tools are able to log system
calls as well as break after every guest instruction execution. Ether
is Win XP SP2 specific.

EXAMIN-C - This is our set of tools and libraries that are built on top of
XenAccess. By using libraries that parse Windows PDB and GCC
intermediate
representation files, we can write C-like scripts that are very close to
code that run as they would in the guest kernel. We support both
Windows and Linux (we've tested on Windows XP, Vista, Fedora, and
Debian).
Our goal is to make writing introspection tools less brittle and
independent
of minor OS versions. Unfortunately, at this moment it is unclear how
much
we can publicly release.

The introspection project requires two pieces: a hypervisor component and a
user-space component. XenAccess and VIX offer a user-space library for
implementing introspection tools. It would be nice to be able to compare
these libraries and use one as a basis for future feature inclusions. The
most advanced hypervisor component currently available, that I'm aware of,
is Ether. Ether is still immature but offers the features that ATC-NY
desires most.

Licensing is also an issue. We are quite happy to contribute to an open
source introspection library, but we would like the option to keep some of
the tools built using it proprietary. We would therefore like the
introspection library to be LGPL'd or less restrictive. Other commercial
vendors would likely have similar concerns. The current XenAccess version
is problematic because of its dependence on libxenctrl. We would want to
avoid any gpl dependency in a Xen introspection library.

References:

[1] Bryane Payne et al. Secure and Flexible Monitoring of Virtual
Machines. ACSAC 2007.

[2] Brian Hay and Kara Nance. Forensic Examination of Volatile System
Data Using Virtual Introspection. SIGOPS Operating Systems Review.
April 2008.

[3] Artem Dinaburg et al. Ether: Malware Analysis via Hardware
Virtualization Extensions. CCS '08.

_______________________________________________
Xen-introspect mailing list
Xen-introspect@lists.xensource.com
http://lists.xensource.com/mailman/listinfo/xen-introspect
Re: Starting Points for the Xen Introspection Project [ In reply to ]
I have a few thoughts / comments available in-line below...

> do. We listed some general requirements during the first teleconference.
> >From ATC-NY's perspective, our highest priority feature is the ability to
> trap on guest memory accesses (rwx) to specific addresses.

There are basically two ways to do this. Ether, as you described in
your message is one of them. The lead author of the Ether project is
a colleague here at GA Tech, so I can easily ask him question as
needed. Another option is Lares [4]. Lares probably is *not* what we
want here as it requires insertion of hooks at runtime into the guest
VM. As such, it can be a little tricker to make this a general
purpose solution. There are certainly pros and cons to each option,
so it's worth being aware of both.

> introspection tool. I believe to get something running quickly we should
> adapt an existing project and tweak the API over time. It is better to have
> something running than to fully design a project before implementation.

I certainly like this approach. Basically, get something in there
that works and then build on it.

> The introspection project requires two pieces: a hypervisor component and a
> user-space component. XenAccess and VIX offer a user-space library for
> implementing introspection tools. It would be nice to be able to compare
> these libraries and use one as a basis for future feature inclusions.

Agreed. If the VIX people can make their source available and / or
provide more technical details, that would be very useful at this
stage.

> most advanced hypervisor component currently available, that I'm aware of,
> is Ether. Ether is still immature but offers the features that ATC-NY
> desires most.

Another thought here is that I believe someone from Xen (Ian? or
Keir?) mentioned on the conference call the idea of adding typed
p-to-m tables. This would basically allow for similar functionality,
if I understand their thinking correctly.

> Licensing is also an issue. We are quite happy to contribute to an open
> source introspection library, but we would like the option to keep some of
> the tools built using it proprietary. We would therefore like the
> introspection library to be LGPL'd or less restrictive. Other commercial
> vendors would likely have similar concerns. The current XenAccess version
> is problematic because of its dependence on libxenctrl. We would want to
> avoid any gpl dependency in a Xen introspection library.

I have designed a new architecture that would help to alleviate this
problem. However, it is not yet implemented. If this is an issue
that needs to be resolved, perhaps we could find someone to implement
that architecture (basically just a kernel module that creates a /dev
interface allowing for physical memory access to each of the xen
domains from dom0). I guess the take home point here is that the
licensing issue is resolvable without requiring a major rewrite of
XenAccess.


References:

[4] Bryan Payne et al. Lares: An Architecture for Secure Active
Monitoring Using Virtualization. IEEE Symposium on Security and
Privacy. 2008.



--
Bryan D. Payne
Research Scientist & Graduate Student
Georgia Tech Information Security Center
http://www.bryanpayne.org

_______________________________________________
Xen-introspect mailing list
Xen-introspect@lists.xensource.com
http://lists.xensource.com/mailman/listinfo/xen-introspect
Re: Starting Points for the Xen Introspection Project [ In reply to ]
I'm on the road right now at a NASA panel review (with my head full of
science rather than computer science!), but here are some quick
thoughts on this:

On Tue, Nov 18, 2008 at 11:36 AM, Hajime Inoue <hinoue@atc-nycorp.com> wrote:
> Here are some of my thoughts on the Xen Introspection Project. We've agreed
> that creating a list of requirements is the first thing the project should
> do. We listed some general requirements during the first teleconference.
> >From ATC-NY's perspective, our highest priority feature is the ability to
> trap on guest memory accesses (rwx) to specific addresses.

Matt and I had a chat about this a while ago and it seems fairly
reasonable to get going quickly - the functionality already exists to
mark pages as -w or -x, for example, then it just seems to be an issue
of a lookup to determine if that is really the permission, or a false
permission to provide the VMI trap. I suppose that the devil is in
the details, but a coarse version of this should be fairly easy.

>
> There are no publicly available tools that allow us to do this with recent
> versions of Xen or support both Windows and Linux. So it is an open
> question whether we should start with a new library or adapt an existing
> introspection tool. I believe to get something running quickly we should
> adapt an existing project and tweak the API over time. It is better to have
> something running than to fully design a project before implementation.

I'm certainly interested in getting something running in the near
future, but I think we would benefit from at least an initial strategy
for how this functionality may evolve (e.g., I think we want to place
as little as possible in the hypervisor, if for no other reason than I
want to see as simple a hypervisor as possible in order to make it
more likely that we can write it in a high assurance manner). We
should also carefully consider how little we could add at this point
to get the project going, in order to reduce the need to deprecate
functionality in the future.

> What should be the basis for the introspection tools? There are three
> introspection projects described in recent literature that are Xen specific:
>
> XenAccess [1] - started by Bryan Payne from Georgia Tech. It builds on
> top of XenCtrl. Its API concentrates on mmapping domU memory into
> domO. It has some advantages in that it supports both Linux and
> Windows. It can also mmap domU process memory into dom0, which is
> a potential benefit to reverse engineering tools. It is GPL'd.
> Source is available from xenaccess.googlecode.com. (I am most
> familiar with this project because ATC-NY uses it as a base for
> our own tools and I am a minor contributer.)
>
> VIX (Virtual Introspection for Xen) [2]- Hay and Nance's VM
> introspection tools for forensics and a library utility for
> building them. Linux 2.6 specific, it consists of specific
> forensic tools along with a utility library that assists in
> mmapping domU kernel memory. Is source available?

I have not made the source available at this point, but don't really
have an objection to doing so.

>
> Ether [3] - Another effort from Georgia Tech that concentrates more
> on hypervisor modifications and is specific to Intel VT HVM domUs.
> This was recently described at ACM CCS 2008. The group has
> apparently abandoned Xen in favor of KVM. However, source for the
> Xen modifications are available. Their tools are able to log system
> calls as well as break after every guest instruction execution. Ether
> is Win XP SP2 specific.
>
> EXAMIN-C - This is our set of tools and libraries that are built on top of
> XenAccess. By using libraries that parse Windows PDB and GCC
> intermediate
> representation files, we can write C-like scripts that are very close to
> code that run as they would in the guest kernel. We support both
> Windows and Linux (we've tested on Windows XP, Vista, Fedora, and
> Debian).
> Our goal is to make writing introspection tools less brittle and
> independent
> of minor OS versions. Unfortunately, at this moment it is unclear how
> much
> we can publicly release.
>
> The introspection project requires two pieces: a hypervisor component and a
> user-space component. XenAccess and VIX offer a user-space library for
> implementing introspection tools. It would be nice to be able to compare
> these libraries and use one as a basis for future feature inclusions

I would be happy to have that discussion.

The
> most advanced hypervisor component currently available, that I'm aware of,
> is Ether. Ether is still immature but offers the features that ATC-NY
> desires most.
>
> Licensing is also an issue. We are quite happy to contribute to an open
> source introspection library, but we would like the option to keep some of
> the tools built using it proprietary. We would therefore like the
> introspection library to be LGPL'd or less restrictive. Other commercial
> vendors would likely have similar concerns. The current XenAccess version
> is problematic because of its dependence on libxenctrl. We would want to
> avoid any gpl dependency in a Xen introspection library.

I think this is an issue we can resolve.

>
> References:
>
> [1] Bryane Payne et al. Secure and Flexible Monitoring of Virtual
> Machines. ACSAC 2007.
>
> [2] Brian Hay and Kara Nance. Forensic Examination of Volatile System
> Data Using Virtual Introspection. SIGOPS Operating Systems Review.
> April 2008.
>
> [3] Artem Dinaburg et al. Ether: Malware Analysis via Hardware
> Virtualization Extensions. CCS '08.
>
> _______________________________________________
> Xen-introspect mailing list
> Xen-introspect@lists.xensource.com
> http://lists.xensource.com/mailman/listinfo/xen-introspect
>



--
Brian Hay

_______________________________________________
Xen-introspect mailing list
Xen-introspect@lists.xensource.com
http://lists.xensource.com/mailman/listinfo/xen-introspect