Mailing List Archive

Linux guest kernel threat model for Confidential Computing
Hi Greg,

You mentioned couple of times (last time in this recent thread:
https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
discussing the updated threat model for kernel, so this email is a start in this direction.

(Note: I tried to include relevant people from different companies, as well as linux-coco
mailing list, but I hope everyone can help by including additional people as needed).

As we have shared before in various lkml threads/conference presentations
([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
This is a big change in the threat model and requires both careful assessment of the
new (hypervisor <-> guest kernel) attack surface, as well as careful design of mitigations
and security validation techniques. This is the activity that we have started back at Intel
and the current status can be found in

1) Threat model and potential mitigations:
https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html

2) One of the described in the above doc mitigations is "hardening of the enabled
code". What we mean by this, as well as techniques that are being used are
described in this document:
https://intel.github.io/ccc-linux-guest-hardening-docs/tdx-guest-hardening.html

3) All the tools are open-source and everyone can start using them right away even
without any special HW (readme has description of what is needed).
Tools and documentation is here:
https://github.com/intel/ccc-linux-guest-hardening

4) all not yet upstreamed linux patches (that we are slowly submitting) can be found
here: https://github.com/intel/tdx/commits/guest-next

So, my main question before we start to argue about the threat model, mitigations, etc,
is what is the good way to get this reviewed to make sure everyone is aligned?
There are a lot of angles and details, so what is the most efficient method?
Should I split the threat model from https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html
into logical pieces and start submitting it to mailing list for discussion one by one?
Any other methods?

The original plan we had in mind is to start discussing the relevant pieces when submitting the code,
i.e. when submitting the device filter patches, we will include problem statement, threat model link,
data, alternatives considered, etc.

Best Regards,
Elena.

[1] https://lore.kernel.org/all/20210804174322.2898409-1-sathyanarayanan.kuppuswamy@linux.intel.com/
[2] https://lpc.events/event/16/contributions/1328/
[3] https://events.linuxfoundation.org/archive/2022/linux-security-summit-north-america/program/schedule/
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> Hi Greg,
>
> You mentioned couple of times (last time in this recent thread:
> https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> discussing the updated threat model for kernel, so this email is a start in this direction.

Any specific reason you didn't cc: the linux-hardening mailing list?
This seems to be in their area as well, right?

> As we have shared before in various lkml threads/conference presentations
> ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> change in the threat model where guest kernel doesn’t anymore trust the hypervisor.

That is, frankly, a very funny threat model. How realistic is it really
given all of the other ways that a hypervisor can mess with a guest?

So what do you actually trust here? The CPU? A device? Nothing?

> This is a big change in the threat model and requires both careful assessment of the
> new (hypervisor <-> guest kernel) attack surface, as well as careful design of mitigations
> and security validation techniques. This is the activity that we have started back at Intel
> and the current status can be found in
>
> 1) Threat model and potential mitigations:
> https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html

So you trust all of qemu but not Linux? Or am I misreading that
diagram?

> 2) One of the described in the above doc mitigations is "hardening of the enabled
> code". What we mean by this, as well as techniques that are being used are
> described in this document:
> https://intel.github.io/ccc-linux-guest-hardening-docs/tdx-guest-hardening.html

I hate the term "hardening". Please just say it for what it really is,
"fixing bugs to handle broken hardware". We've done that for years when
dealing with PCI and USB and even CPUs doing things that they shouldn't
be doing. How is this any different in the end?

So what you also are saying here now is "we do not trust any PCI
devices", so please just say that (why do you trust USB devices?) If
that is something that you all think that Linux should support, then
let's go from there.

> 3) All the tools are open-source and everyone can start using them right away even
> without any special HW (readme has description of what is needed).
> Tools and documentation is here:
> https://github.com/intel/ccc-linux-guest-hardening

Again, as our documentation states, when you submit patches based on
these tools, you HAVE TO document that. Otherwise we think you all are
crazy and will get your patches rejected. You all know this, why ignore
it?

> 4) all not yet upstreamed linux patches (that we are slowly submitting) can be found
> here: https://github.com/intel/tdx/commits/guest-next

Random github trees of kernel patches are just that, sorry.

> So, my main question before we start to argue about the threat model, mitigations, etc,
> is what is the good way to get this reviewed to make sure everyone is aligned?
> There are a lot of angles and details, so what is the most efficient method?
> Should I split the threat model from https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html
> into logical pieces and start submitting it to mailing list for discussion one by one?

Yes, start out by laying out what you feel the actual problem is, what
you feel should be done for it, and the patches you have proposed to
implement this, for each and every logical piece.

Again, nothing new here, that's how Linux is developed, again, you all
know this, it's not anything I should have to say.

> Any other methods?
>
> The original plan we had in mind is to start discussing the relevant pieces when submitting the code,
> i.e. when submitting the device filter patches, we will include problem statement, threat model link,
> data, alternatives considered, etc.

As always, we can't do anything without actual working changes to the
code, otherwise it's just a pipe dream and we can't waste our time on it
(neither would you want us to).

thanks, and good luck!

greg k-h
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > Hi Greg,
> >
> > You mentioned couple of times (last time in this recent thread:
> > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > discussing the updated threat model for kernel, so this email is a start in this direction.
>
> Any specific reason you didn't cc: the linux-hardening mailing list?
> This seems to be in their area as well, right?
>
> > As we have shared before in various lkml threads/conference presentations
> > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
>
> That is, frankly, a very funny threat model. How realistic is it really
> given all of the other ways that a hypervisor can mess with a guest?

It's what a lot of people would like; in the early attempts it was easy
to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
can mess with - remember that not just the memory is encrypted, so is
the register state, and the guest gets to see changes to mapping and a
lot of control over interrupt injection etc.

> So what do you actually trust here? The CPU? A device? Nothing?

We trust the actual physical CPU, provided that it can prove that it's a
real CPU with the CoCo hardware enabled. Both the SNP and TDX hardware
can perform an attestation signed by the CPU to prove to someone
external that the guest is running on a real trusted CPU.

Note that the trust is limited:
a) We don't trust that we can make forward progress - if something
does something bad it's OK for the guest to stop.
b) We don't trust devices, and we don't trust them by having the guest
do normal encryption; e.g. just LUKS on the disk and normal encrypted
networking. [There's a lot of schemes people are working on about how
the guest gets the keys etc for that)

> > This is a big change in the threat model and requires both careful assessment of the
> > new (hypervisor <-> guest kernel) attack surface, as well as careful design of mitigations
> > and security validation techniques. This is the activity that we have started back at Intel
> > and the current status can be found in
> >
> > 1) Threat model and potential mitigations:
> > https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html
>
> So you trust all of qemu but not Linux? Or am I misreading that
> diagram?

You're misreading it; This is about the grey part (i.e. the guest) not
trusting the host (the white part including qemu and the host kernel).

> > 2) One of the described in the above doc mitigations is "hardening of the enabled
> > code". What we mean by this, as well as techniques that are being used are
> > described in this document:
> > https://intel.github.io/ccc-linux-guest-hardening-docs/tdx-guest-hardening.html
>
> I hate the term "hardening". Please just say it for what it really is,
> "fixing bugs to handle broken hardware". We've done that for years when
> dealing with PCI and USB and even CPUs doing things that they shouldn't
> be doing. How is this any different in the end?
>
> So what you also are saying here now is "we do not trust any PCI
> devices", so please just say that (why do you trust USB devices?) If
> that is something that you all think that Linux should support, then
> let's go from there.

I don't think generally all PCI device drivers guard against all the
nasty things that a broken implementation of their hardware can do.
The USB devices are probably a bit better, because they actually worry
about people walking up with a nasty HID device; I'm skeptical that
a kernel would survive a purposely broken USB controller.

I'm not sure the request here isn't really to make sure *all* PCI devices
are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
and potentially ones that people will want to pass-through (which
generally needs a lot more work to make safe).
(I've not looked at these Intel tools to see what they cover)

Having said that, how happy are you with Thunderbolt PCI devices being
plugged into your laptop or into the hotplug NVMe slot on a server?
We're now in the position we were with random USB devices years ago.

Also we would want to make sure that any config data that the hypervisor
can pass to the guest is validated.

> > 3) All the tools are open-source and everyone can start using them right away even
> > without any special HW (readme has description of what is needed).
> > Tools and documentation is here:
> > https://github.com/intel/ccc-linux-guest-hardening
>
> Again, as our documentation states, when you submit patches based on
> these tools, you HAVE TO document that. Otherwise we think you all are
> crazy and will get your patches rejected. You all know this, why ignore
> it?
>
> > 4) all not yet upstreamed linux patches (that we are slowly submitting) can be found
> > here: https://github.com/intel/tdx/commits/guest-next
>
> Random github trees of kernel patches are just that, sorry.
>
> > So, my main question before we start to argue about the threat model, mitigations, etc,
> > is what is the good way to get this reviewed to make sure everyone is aligned?
> > There are a lot of angles and details, so what is the most efficient method?
> > Should I split the threat model from https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html
> > into logical pieces and start submitting it to mailing list for discussion one by one?
>
> Yes, start out by laying out what you feel the actual problem is, what
> you feel should be done for it, and the patches you have proposed to
> implement this, for each and every logical piece.
>
> Again, nothing new here, that's how Linux is developed, again, you all
> know this, it's not anything I should have to say.

That seems harsh.
The problem seems reasonably well understood within the CoCo world - how
far people want to push it probably varies; but it's good to make the
problem more widely understood.

> > Any other methods?
> >
> > The original plan we had in mind is to start discussing the relevant pieces when submitting the code,
> > i.e. when submitting the device filter patches, we will include problem statement, threat model link,
> > data, alternatives considered, etc.
>
> As always, we can't do anything without actual working changes to the
> code, otherwise it's just a pipe dream and we can't waste our time on it
> (neither would you want us to).
>
> thanks, and good luck!
>
> greg k-h

Dave

>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > Hi Greg,
> > >
> > > You mentioned couple of times (last time in this recent thread:
> > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > discussing the updated threat model for kernel, so this email is a start in this direction.
> >
> > Any specific reason you didn't cc: the linux-hardening mailing list?
> > This seems to be in their area as well, right?
> >
> > > As we have shared before in various lkml threads/conference presentations
> > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> >
> > That is, frankly, a very funny threat model. How realistic is it really
> > given all of the other ways that a hypervisor can mess with a guest?
>
> It's what a lot of people would like; in the early attempts it was easy
> to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> can mess with - remember that not just the memory is encrypted, so is
> the register state, and the guest gets to see changes to mapping and a
> lot of control over interrupt injection etc.
>
> > So what do you actually trust here? The CPU? A device? Nothing?
>
> We trust the actual physical CPU, provided that it can prove that it's a
> real CPU with the CoCo hardware enabled. Both the SNP and TDX hardware
> can perform an attestation signed by the CPU to prove to someone
> external that the guest is running on a real trusted CPU.
>
> Note that the trust is limited:
> a) We don't trust that we can make forward progress - if something
> does something bad it's OK for the guest to stop.
> b) We don't trust devices, and we don't trust them by having the guest
> do normal encryption; e.g. just LUKS on the disk and normal encrypted
> networking. [There's a lot of schemes people are working on about how
> the guest gets the keys etc for that)

I think we need to more precisely say what we mean by 'trust' as it
can have quite a broad interpretation.

As a baseline requirement, in the context of confidential computing the
guest would not trust the hypervisor with data that needs to remain
confidential, but would generally still expect it to provide a faithful
implementation of a given device.

IOW, the guest would expect the implementation of virtio-blk devices to
be functionally correct per the virtio-blk specification, but would not
trust the host to protect confidentiality any stored data in the disk.

Any virtual device exposed to the guest that can transfer potentially
sensitive data needs to have some form of guest controlled encryption
applied. For disks this is easy with FDE like LUKS, for NICs this is
already best practice for services by using TLS. Other devices may not
have good existing options for applying encryption.

If the guest has a virtual keyboard, mouse and graphical display, which
is backed by a VNC/RDP server in the host, then all that is visible to the
host. There's no pre-existing solutions I know can could offer easy
confidentiality for basic console I/O from the start of guest firmware
onwards. The best is to spawn a VNC/RDP server in the guest at some
point during boot. Means you can't login to the guest in single user
mode with your root password though, without compromising it.

The problem also applies for common solutions today where the host passes
in config data to the guest, for consumption by tools like cloud-init.
This is used in the past to inject an SSH key for example, or set the
guest root password. Such data received from the host can no longer be
trusted, as the host can see the data, or subsitute its own SSH key(s)
in order to gain access. Cloud-init needs to get its config data from
a trusted source, likely an external attestation server


A further challenge surrounds handling of undesirable devices. A goal
of OS development has been to ensure that both coldplugged and hotplugged
devices "just work" out of the box with zero guest admin config required.
To some extent this is contrary to what a confidential guest will want.
It doesn't want a getty spawned on any console exposed, it doesn't want
to use a virtio-rng exposed by the host which could be feeding non-random.


Protecting against malicious implementations of devices is conceivably
interesting, as a hardening task. A malicious host may try to take
advantage of the guest OS device driver impl to exploit the guest OS
kernel with an end goal of getting into a state where it can be made
to reveal confidential data that was otherwise protected.

With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > Hi Greg,
> > >
> > > You mentioned couple of times (last time in this recent thread:
> > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > discussing the updated threat model for kernel, so this email is a start in this direction.
> >
> > Any specific reason you didn't cc: the linux-hardening mailing list?
> > This seems to be in their area as well, right?
> >
> > > As we have shared before in various lkml threads/conference presentations
> > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> >
> > That is, frankly, a very funny threat model. How realistic is it really
> > given all of the other ways that a hypervisor can mess with a guest?
>
> It's what a lot of people would like; in the early attempts it was easy
> to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> can mess with - remember that not just the memory is encrypted, so is
> the register state, and the guest gets to see changes to mapping and a
> lot of control over interrupt injection etc.

And due to the fact that SEV and TDX really do not work, how is anyone
expecting any of this to work? As one heckler on IRC recently put it,
if you squint hard enough, you can kind of ignore the real-world issues
here, so perhaps this should all be called "squint-puting" in order to
feel like you have a "confidential" system? :)

> > So what do you actually trust here? The CPU? A device? Nothing?
>
> We trust the actual physical CPU, provided that it can prove that it's a
> real CPU with the CoCo hardware enabled.

Great, so why not have hardware attestation also for your devices you
wish to talk to? Why not use that as well? Then you don't have to
worry about anything in the guest.

> Both the SNP and TDX hardware
> can perform an attestation signed by the CPU to prove to someone
> external that the guest is running on a real trusted CPU.

And again, do the same thing for the other hardware devices and all is
good. To not do that is to just guess and wave hands. You know this :)

> Note that the trust is limited:
> a) We don't trust that we can make forward progress - if something
> does something bad it's OK for the guest to stop.

So the guest can stop itself?

> b) We don't trust devices, and we don't trust them by having the guest
> do normal encryption; e.g. just LUKS on the disk and normal encrypted
> networking. [There's a lot of schemes people are working on about how
> the guest gets the keys etc for that)

How do you trust you got real data on the disk? On the network? Those
are coming from the host, how is any of that data to be trusted? Where
does the trust stop and why?

> > I hate the term "hardening". Please just say it for what it really is,
> > "fixing bugs to handle broken hardware". We've done that for years when
> > dealing with PCI and USB and even CPUs doing things that they shouldn't
> > be doing. How is this any different in the end?
> >
> > So what you also are saying here now is "we do not trust any PCI
> > devices", so please just say that (why do you trust USB devices?) If
> > that is something that you all think that Linux should support, then
> > let's go from there.
>
> I don't think generally all PCI device drivers guard against all the
> nasty things that a broken implementation of their hardware can do.

I know that all PCI drivers can NOT do that today as that was never
anything that Linux was designed for.

> The USB devices are probably a bit better, because they actually worry
> about people walking up with a nasty HID device; I'm skeptical that
> a kernel would survive a purposely broken USB controller.

I agree with you there, USB drivers are only starting to be fuzzed at
the descriptor level, that's all. Which is why they too can be put into
the "untrusted" area until you trust them.

> I'm not sure the request here isn't really to make sure *all* PCI devices
> are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
> and potentially ones that people will want to pass-through (which
> generally needs a lot more work to make safe).
> (I've not looked at these Intel tools to see what they cover)

Why not just create a whole new bus path for these "trusted" devices to
attach to and do that instead of tyring to emulate a protocol that was
explicitly designed NOT to this model at all? Why are you trying to
shoehorn something here and not just designing it properly from the
beginning?

> Having said that, how happy are you with Thunderbolt PCI devices being
> plugged into your laptop or into the hotplug NVMe slot on a server?

We have protection for that, and have had it for many years. Same for
USB devices. This isn't new, perhaps you all have not noticed those
features be added and taken advantage of already by many Linux distros
and system images (i.e. ChromeOS and embedded systems?)

> We're now in the position we were with random USB devices years ago.

Nope, we are not, again, we already handle random PCI devices being
plugged in. It's up to userspace to make the policy decision if it
should be trusted or not before the kernel has access to it.

So a meta-comment, why not just use that today? If your guest OS can
not authenticate the PCI device passed to it, don't allow the kernel to
bind to it. If it can be authenticated, wonderful, bind away! You can
do this today with no kernel changes needed.

> Also we would want to make sure that any config data that the hypervisor
> can pass to the guest is validated.

Define "validated" please.

> The problem seems reasonably well understood within the CoCo world - how
> far people want to push it probably varies; but it's good to make the
> problem more widely understood.

The "CoCo" world seems distant and separate from the real-world of Linux
kernel development if you all do not even know about the authentication
methods that we have for years for enabling access to PCI and USB
devices as described above. If the impementations that we currently
have are lacking in some way, wonderful, please submit changes for them
and we will be glad to review them as needed.

Remember, it's up to you all to convince us that your changes make
actual sense and are backed up with working implementations. Not us :)

good luck!

greg k-h
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, 2023-01-25 at 15:22 +0100, Greg Kroah-Hartman wrote:
> On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert
> wrote:
> > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > Hi Greg,
> > > >
> > > > You mentioned couple of times (last time in this recent thread:
> > > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that
> > > > we ought to start
> > > > discussing the updated threat model for kernel, so this email
> > > > is a start in this direction.
> > >
> > > Any specific reason you didn't cc: the linux-hardening mailing
> > > list? This seems to be in their area as well, right?
> > >
> > > > As we have shared before in various lkml threads/conference
> > > > presentations ([1], [2], [3] and many others), for the
> > > > Confidential Computing guest kernel, we have a change in the
> > > > threat model where guest kernel doesn’t anymore trust the
> > > > hypervisor.
> > >
> > > That is, frankly, a very funny threat model.  How realistic is it
> > > really given all of the other ways that a hypervisor can mess
> > > with a guest?
> >
> > It's what a lot of people would like; in the early attempts it was
> > easy to defeat, but in TDX and SEV-SNP the hypervisor has a lot
> > less that it can mess with - remember that not just the memory is
> > encrypted, so is the register state, and the guest gets to see
> > changes to mapping and a lot of control over interrupt injection
> > etc.
>
> And due to the fact that SEV and TDX really do not work, how is
> anyone expecting any of this to work?  As one heckler on IRC recently
> put it, if you squint hard enough, you can kind of ignore the real-
> world issues here, so perhaps this should all be called "squint-
> puting" in order to feel like you have a "confidential" system?  :)

There's a difference between no trust, which requires defeating all
attacks as they occur and limited trust, which merely means you want to
detect an attack from the limited trust entity to show that trust was
violated. Trying to achieve the former with CC is a good academic
exercise, but not required for the technology to be useful. Most cloud
providers are working towards the latter ... we know there are holes,
but as long as the guest can always detect interference they can be
confident in their trust in the CSP not to attack them via various
hypervisor mechanisms.

James
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > Hi Greg,
> > > >
> > > > You mentioned couple of times (last time in this recent thread:
> > > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > > discussing the updated threat model for kernel, so this email is a start in this direction.
> > >
> > > Any specific reason you didn't cc: the linux-hardening mailing list?
> > > This seems to be in their area as well, right?
> > >
> > > > As we have shared before in various lkml threads/conference presentations
> > > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> > >
> > > That is, frankly, a very funny threat model. How realistic is it really
> > > given all of the other ways that a hypervisor can mess with a guest?
> >
> > It's what a lot of people would like; in the early attempts it was easy
> > to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> > can mess with - remember that not just the memory is encrypted, so is
> > the register state, and the guest gets to see changes to mapping and a
> > lot of control over interrupt injection etc.
>
> And due to the fact that SEV and TDX really do not work, how is anyone
> expecting any of this to work? As one heckler on IRC recently put it,
> if you squint hard enough, you can kind of ignore the real-world issues
> here, so perhaps this should all be called "squint-puting" in order to
> feel like you have a "confidential" system? :)

I agree the original SEV was that weak; I've not seen anyone give a good
argument against SNP or TDX.

> > > So what do you actually trust here? The CPU? A device? Nothing?
> >
> > We trust the actual physical CPU, provided that it can prove that it's a
> > real CPU with the CoCo hardware enabled.
>
> Great, so why not have hardware attestation also for your devices you
> wish to talk to? Why not use that as well? Then you don't have to
> worry about anything in the guest.

There were some talks at Plumbers where PCIe is working on adding that;
it's not there yet though. I think that's PCIe 'Integrity and Data
Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
SPDM. I don't know much of the detail of those, just that they're far
enough off that people aren't depending on them yet.

> > Both the SNP and TDX hardware
> > can perform an attestation signed by the CPU to prove to someone
> > external that the guest is running on a real trusted CPU.
>
> And again, do the same thing for the other hardware devices and all is
> good. To not do that is to just guess and wave hands. You know this :)

That wouldn't help you necessarily for virtual devices - where the
hypervisor implements the device (like a virtual NIC).

> > Note that the trust is limited:
> > a) We don't trust that we can make forward progress - if something
> > does something bad it's OK for the guest to stop.
>
> So the guest can stop itself?

Sure.

> > b) We don't trust devices, and we don't trust them by having the guest
> > do normal encryption; e.g. just LUKS on the disk and normal encrypted
> > networking. [There's a lot of schemes people are working on about how
> > the guest gets the keys etc for that)
>
> How do you trust you got real data on the disk? On the network? Those
> are coming from the host, how is any of that data to be trusted? Where
> does the trust stop and why?

We don't; you use LUKS2 on the disk and/or dm-verity; so there's no
trust in the disk.
You use whatever your favorite network encryption already is that
you're using to send data across the untrusted net.
So no trust in the data from the NIC.

> > > I hate the term "hardening". Please just say it for what it really is,
> > > "fixing bugs to handle broken hardware". We've done that for years when
> > > dealing with PCI and USB and even CPUs doing things that they shouldn't
> > > be doing. How is this any different in the end?
> > >
> > > So what you also are saying here now is "we do not trust any PCI
> > > devices", so please just say that (why do you trust USB devices?) If
> > > that is something that you all think that Linux should support, then
> > > let's go from there.
> >
> > I don't think generally all PCI device drivers guard against all the
> > nasty things that a broken implementation of their hardware can do.
>
> I know that all PCI drivers can NOT do that today as that was never
> anything that Linux was designed for.

Agreed; which again is why I only really worry about the subset of
devices I'd want in a CoCo VM.

> > The USB devices are probably a bit better, because they actually worry
> > about people walking up with a nasty HID device; I'm skeptical that
> > a kernel would survive a purposely broken USB controller.
>
> I agree with you there, USB drivers are only starting to be fuzzed at
> the descriptor level, that's all. Which is why they too can be put into
> the "untrusted" area until you trust them.
>
> > I'm not sure the request here isn't really to make sure *all* PCI devices
> > are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
> > and potentially ones that people will want to pass-through (which
> > generally needs a lot more work to make safe).
> > (I've not looked at these Intel tools to see what they cover)
>
> Why not just create a whole new bus path for these "trusted" devices to
> attach to and do that instead of tyring to emulate a protocol that was
> explicitly designed NOT to this model at all? Why are you trying to
> shoehorn something here and not just designing it properly from the
> beginning?

I'd be kind of OK with that for the virtual devices; but:

a) I think you'd start reinventing PCIe with enumeration etc

b) We do want those pass through NICs etc that are PCIe
- as long as you use normal guest crypto stuff then the host
can be just as nasty as it likes with the data they present.

c) The world has enough bus protocols, and people understand the
basics of PCI(e) - we really don't need another one.

> > Having said that, how happy are you with Thunderbolt PCI devices being
> > plugged into your laptop or into the hotplug NVMe slot on a server?
>
> We have protection for that, and have had it for many years. Same for
> USB devices. This isn't new, perhaps you all have not noticed those
> features be added and taken advantage of already by many Linux distros
> and system images (i.e. ChromeOS and embedded systems?)

What protection? I know we have an IOMMU, and that stops the device
stamping all over RAM by itself - but I think Intel's worries are more
subtle, things where the device starts playing with what PCI devices
are expected to do to try and trigger untested kernel paths. I don't
think there's protection against that.
I know we can lock by PCI/USB vendor/device ID - but those can be made
up trivially; protection like that is meaningless.

> > We're now in the position we were with random USB devices years ago.
>
> Nope, we are not, again, we already handle random PCI devices being
> plugged in. It's up to userspace to make the policy decision if it
> should be trusted or not before the kernel has access to it.
>
> So a meta-comment, why not just use that today? If your guest OS can
> not authenticate the PCI device passed to it, don't allow the kernel to
> bind to it. If it can be authenticated, wonderful, bind away! You can
> do this today with no kernel changes needed.

Because:
a) there's no good way to authenticate a PCI device yet
- any nasty device can claim to have a given PCI ID.
b) Even if you could, there's no man-in-the-middle protection yet.

> > Also we would want to make sure that any config data that the hypervisor
> > can pass to the guest is validated.
>
> Define "validated" please.

Lets say you get something like a ACPI table or qemu fw.cfg table
giving details of your devices; if the hypervisor builds those in a
nasty way what happens?

> > The problem seems reasonably well understood within the CoCo world - how
> > far people want to push it probably varies; but it's good to make the
> > problem more widely understood.
>
> The "CoCo" world seems distant and separate from the real-world of Linux
> kernel development if you all do not even know about the authentication
> methods that we have for years for enabling access to PCI and USB
> devices as described above. If the impementations that we currently
> have are lacking in some way, wonderful, please submit changes for them
> and we will be glad to review them as needed.

That's probably fair to some degree - the people looking at this are VM
people, not desktop people; I'm not sure what the overlap is; but as I
say above, I don't think the protection currently available really help
here. Please show us where we're wrong.

> Remember, it's up to you all to convince us that your changes make
> actual sense and are backed up with working implementations. Not us :)

Sure; I'm seeing existing implementations being used in vendors clouds
at the moment, and they're slowly getting the security people want.
I'd like to see that being done with upstream kernels and firmware.

Dave


> good luck!
>
> greg k-h
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> > > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > Hi Greg,
> > > > >
> > > > > You mentioned couple of times (last time in this recent thread:
> > > > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > > > discussing the updated threat model for kernel, so this email is a start in this direction.
> > > >
> > > > Any specific reason you didn't cc: the linux-hardening mailing list?
> > > > This seems to be in their area as well, right?
> > > >
> > > > > As we have shared before in various lkml threads/conference presentations
> > > > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> > > >
> > > > That is, frankly, a very funny threat model. How realistic is it really
> > > > given all of the other ways that a hypervisor can mess with a guest?
> > >
> > > It's what a lot of people would like; in the early attempts it was easy
> > > to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> > > can mess with - remember that not just the memory is encrypted, so is
> > > the register state, and the guest gets to see changes to mapping and a
> > > lot of control over interrupt injection etc.
> >
> > And due to the fact that SEV and TDX really do not work, how is anyone
> > expecting any of this to work? As one heckler on IRC recently put it,
> > if you squint hard enough, you can kind of ignore the real-world issues
> > here, so perhaps this should all be called "squint-puting" in order to
> > feel like you have a "confidential" system? :)
>
> I agree the original SEV was that weak; I've not seen anyone give a good
> argument against SNP or TDX.

Argument that it doesn't work? I thought that ship sailed a long time
ago but I could be wrong as I don't really pay attention to that stuff
as it's just vaporware :)

> > > > So what do you actually trust here? The CPU? A device? Nothing?
> > >
> > > We trust the actual physical CPU, provided that it can prove that it's a
> > > real CPU with the CoCo hardware enabled.
> >
> > Great, so why not have hardware attestation also for your devices you
> > wish to talk to? Why not use that as well? Then you don't have to
> > worry about anything in the guest.
>
> There were some talks at Plumbers where PCIe is working on adding that;
> it's not there yet though. I think that's PCIe 'Integrity and Data
> Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> SPDM. I don't know much of the detail of those, just that they're far
> enough off that people aren't depending on them yet.

Then work with those groups to implement that in an industry-wide way
and then take advantage of it by adding support for it to Linux! Don't
try to reinvent the same thing in a totally different way please.

> > > Both the SNP and TDX hardware
> > > can perform an attestation signed by the CPU to prove to someone
> > > external that the guest is running on a real trusted CPU.
> >
> > And again, do the same thing for the other hardware devices and all is
> > good. To not do that is to just guess and wave hands. You know this :)
>
> That wouldn't help you necessarily for virtual devices - where the
> hypervisor implements the device (like a virtual NIC).

Then create a new bus for that if you don't trust the virtio bus today.

> > > > I hate the term "hardening". Please just say it for what it really is,
> > > > "fixing bugs to handle broken hardware". We've done that for years when
> > > > dealing with PCI and USB and even CPUs doing things that they shouldn't
> > > > be doing. How is this any different in the end?
> > > >
> > > > So what you also are saying here now is "we do not trust any PCI
> > > > devices", so please just say that (why do you trust USB devices?) If
> > > > that is something that you all think that Linux should support, then
> > > > let's go from there.
> > >
> > > I don't think generally all PCI device drivers guard against all the
> > > nasty things that a broken implementation of their hardware can do.
> >
> > I know that all PCI drivers can NOT do that today as that was never
> > anything that Linux was designed for.
>
> Agreed; which again is why I only really worry about the subset of
> devices I'd want in a CoCo VM.

Everyone wants a subset, different from other's subset, which means you
need them all. Sorry.

> > > The USB devices are probably a bit better, because they actually worry
> > > about people walking up with a nasty HID device; I'm skeptical that
> > > a kernel would survive a purposely broken USB controller.
> >
> > I agree with you there, USB drivers are only starting to be fuzzed at
> > the descriptor level, that's all. Which is why they too can be put into
> > the "untrusted" area until you trust them.
> >
> > > I'm not sure the request here isn't really to make sure *all* PCI devices
> > > are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
> > > and potentially ones that people will want to pass-through (which
> > > generally needs a lot more work to make safe).
> > > (I've not looked at these Intel tools to see what they cover)
> >
> > Why not just create a whole new bus path for these "trusted" devices to
> > attach to and do that instead of tyring to emulate a protocol that was
> > explicitly designed NOT to this model at all? Why are you trying to
> > shoehorn something here and not just designing it properly from the
> > beginning?
>
> I'd be kind of OK with that for the virtual devices; but:
>
> a) I think you'd start reinventing PCIe with enumeration etc

Great, then work with the PCI group as talked about above to solve it
properly and not do whack-a-mole like seems to be happening so far.

> b) We do want those pass through NICs etc that are PCIe
> - as long as you use normal guest crypto stuff then the host
> can be just as nasty as it likes with the data they present.

Great, work with the PCI spec for verified devices.

> c) The world has enough bus protocols, and people understand the
> basics of PCI(e) - we really don't need another one.

Great, work with the PCI spec people please.

> > > Having said that, how happy are you with Thunderbolt PCI devices being
> > > plugged into your laptop or into the hotplug NVMe slot on a server?
> >
> > We have protection for that, and have had it for many years. Same for
> > USB devices. This isn't new, perhaps you all have not noticed those
> > features be added and taken advantage of already by many Linux distros
> > and system images (i.e. ChromeOS and embedded systems?)
>
> What protection? I know we have an IOMMU, and that stops the device
> stamping all over RAM by itself - but I think Intel's worries are more
> subtle, things where the device starts playing with what PCI devices
> are expected to do to try and trigger untested kernel paths. I don't
> think there's protection against that.
> I know we can lock by PCI/USB vendor/device ID - but those can be made
> up trivially; protection like that is meaningless.

Then combine it with device attestation and you have a solved solution,
don't ignore others working on this please.

> > > We're now in the position we were with random USB devices years ago.
> >
> > Nope, we are not, again, we already handle random PCI devices being
> > plugged in. It's up to userspace to make the policy decision if it
> > should be trusted or not before the kernel has access to it.
> >
> > So a meta-comment, why not just use that today? If your guest OS can
> > not authenticate the PCI device passed to it, don't allow the kernel to
> > bind to it. If it can be authenticated, wonderful, bind away! You can
> > do this today with no kernel changes needed.
>
> Because:
> a) there's no good way to authenticate a PCI device yet
> - any nasty device can claim to have a given PCI ID.
> b) Even if you could, there's no man-in-the-middle protection yet.

Where is the "man" here in the middle of?

And any PCI attestation should handle that, if not, work with them to
solve that please.

Thunderbolt has authenticated device support today, and so does PCI, and
USB has had it for a decade or so. Use the in-kernel implementation
that we already have or again, show us where it is lacking and we will
be glad to take patches to cover the holes (as we did last year when
ChromeOS implemented support for it in their userspace.)

> > > Also we would want to make sure that any config data that the hypervisor
> > > can pass to the guest is validated.
> >
> > Define "validated" please.
>
> Lets say you get something like a ACPI table or qemu fw.cfg table
> giving details of your devices; if the hypervisor builds those in a
> nasty way what happens?

You tell me, as we trust ACPI tables today, and if we can not, again
then you need to change the model of what Linux does. Why isn't the
BIOS authentication path working properly for ACPI tables already today?
I thought that was a long-solved problem with UEFI (if not, I'm sure the
UEFI people would be interested.)

Anyway, I'll wait until I see real patches as this thread seems to be
totally vague and ignores our current best-practices for pluggable
devices for some odd reason.

thanks,

greg k-h
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
Replying only to the not-so-far addressed points.

> On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > Hi Greg,
> >
> > You mentioned couple of times (last time in this recent thread:
> > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to
> start
> > discussing the updated threat model for kernel, so this email is a start in this
> direction.
>
> Any specific reason you didn't cc: the linux-hardening mailing list?
> This seems to be in their area as well, right?

Added now, I am just not sure how many mailing lists I want to cross spam this.
And this is a very special aspect of 'hardening' since it is about hardening a kernel
under different threat model/assumptions.


> I hate the term "hardening". Please just say it for what it really is,
> "fixing bugs to handle broken hardware". We've done that for years when
> dealing with PCI and USB and even CPUs doing things that they shouldn't
> be doing. How is this any different in the end?

Well, that would not be fully correct in this case. You can really see it from two
angles:

1. fixing bugs to handle broken hardware
2. fixing bugs that are result of correctly operating HW, but incorrectly or maliciously
operating hypervisor (acting as a man in the middle)

We focus on 2 but it happens to address 1 also to some level.

>
> So what you also are saying here now is "we do not trust any PCI
> devices", so please just say that (why do you trust USB devices?) If
> that is something that you all think that Linux should support, then
> let's go from there.
>
> > 3) All the tools are open-source and everyone can start using them right away
> even
> > without any special HW (readme has description of what is needed).
> > Tools and documentation is here:
> > https://github.com/intel/ccc-linux-guest-hardening
>
> Again, as our documentation states, when you submit patches based on
> these tools, you HAVE TO document that. Otherwise we think you all are
> crazy and will get your patches rejected. You all know this, why ignore
> it?

Sorry, I didn’t know that for every bug that is found in linux kernel when
we are submitting a fix that we have to list the way how it has been found.
We will fix this in the future submissions, but some bugs we have are found by
plain code audit, so 'human' is the tool.

>
> > 4) all not yet upstreamed linux patches (that we are slowly submitting) can be
> found
> > here: https://github.com/intel/tdx/commits/guest-next
>
> Random github trees of kernel patches are just that, sorry.

This was just for a completeness or for anyone who is curious to see the actual
code already now. Of course they will be submitted for review
using normal process.

>
> > So, my main question before we start to argue about the threat model,
> mitigations, etc,
> > is what is the good way to get this reviewed to make sure everyone is aligned?
> > There are a lot of angles and details, so what is the most efficient method?
> > Should I split the threat model from https://intel.github.io/ccc-linux-guest-
> hardening-docs/security-spec.html
> > into logical pieces and start submitting it to mailing list for discussion one by
> one?
>
> Yes, start out by laying out what you feel the actual problem is, what
> you feel should be done for it, and the patches you have proposed to
> implement this, for each and every logical piece.

OK, so this thread is about the actual threat model and overall problem.
We can re-write the current bug fixe patches (virtio and MSI) to refer to this threat model
properly and explain that they fix the actual bugs under this threat model.
Rest of pieces will come when other patches will be submitted for the review
in logical groups.

Does this work?

>
> Again, nothing new here, that's how Linux is developed, again, you all
> know this, it's not anything I should have to say.
>
> > Any other methods?
> >
> > The original plan we had in mind is to start discussing the relevant pieces when
> submitting the code,
> > i.e. when submitting the device filter patches, we will include problem
> statement, threat model link,
> > data, alternatives considered, etc.
>
> As always, we can't do anything without actual working changes to the
> code, otherwise it's just a pipe dream and we can't waste our time on it
> (neither would you want us to).

Of course, code exists, we just only starting submitting it. We started from
easy bug fixes because they are small trivial fixes that are easy to review.
Bigger pieces will follow (for example Satya has been addressing your comments about the
device filter in his new implementation).

Best Regards,
Elena.
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Daniel P. Berrangé (berrange@redhat.com) wrote:
> On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > Hi Greg,
> > > >
> > > > You mentioned couple of times (last time in this recent thread:
> > > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > > discussing the updated threat model for kernel, so this email is a start in this direction.
> > >
> > > Any specific reason you didn't cc: the linux-hardening mailing list?
> > > This seems to be in their area as well, right?
> > >
> > > > As we have shared before in various lkml threads/conference presentations
> > > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> > >
> > > That is, frankly, a very funny threat model. How realistic is it really
> > > given all of the other ways that a hypervisor can mess with a guest?
> >
> > It's what a lot of people would like; in the early attempts it was easy
> > to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> > can mess with - remember that not just the memory is encrypted, so is
> > the register state, and the guest gets to see changes to mapping and a
> > lot of control over interrupt injection etc.
> >
> > > So what do you actually trust here? The CPU? A device? Nothing?
> >
> > We trust the actual physical CPU, provided that it can prove that it's a
> > real CPU with the CoCo hardware enabled. Both the SNP and TDX hardware
> > can perform an attestation signed by the CPU to prove to someone
> > external that the guest is running on a real trusted CPU.
> >
> > Note that the trust is limited:
> > a) We don't trust that we can make forward progress - if something
> > does something bad it's OK for the guest to stop.
> > b) We don't trust devices, and we don't trust them by having the guest
> > do normal encryption; e.g. just LUKS on the disk and normal encrypted
> > networking. [There's a lot of schemes people are working on about how
> > the guest gets the keys etc for that)
>
> I think we need to more precisely say what we mean by 'trust' as it
> can have quite a broad interpretation.
>
> As a baseline requirement, in the context of confidential computing the
> guest would not trust the hypervisor with data that needs to remain
> confidential, but would generally still expect it to provide a faithful
> implementation of a given device.
>
> IOW, the guest would expect the implementation of virtio-blk devices to
> be functionally correct per the virtio-blk specification, but would not
> trust the host to protect confidentiality any stored data in the disk.
>
> Any virtual device exposed to the guest that can transfer potentially
> sensitive data needs to have some form of guest controlled encryption
> applied. For disks this is easy with FDE like LUKS, for NICs this is
> already best practice for services by using TLS. Other devices may not
> have good existing options for applying encryption.
>
> If the guest has a virtual keyboard, mouse and graphical display, which
> is backed by a VNC/RDP server in the host, then all that is visible to the
> host. There's no pre-existing solutions I know can could offer easy
> confidentiality for basic console I/O from the start of guest firmware
> onwards. The best is to spawn a VNC/RDP server in the guest at some
> point during boot. Means you can't login to the guest in single user
> mode with your root password though, without compromising it.
>
> The problem also applies for common solutions today where the host passes
> in config data to the guest, for consumption by tools like cloud-init.
> This is used in the past to inject an SSH key for example, or set the
> guest root password. Such data received from the host can no longer be
> trusted, as the host can see the data, or subsitute its own SSH key(s)
> in order to gain access. Cloud-init needs to get its config data from
> a trusted source, likely an external attestation server
>
>
> A further challenge surrounds handling of undesirable devices. A goal
> of OS development has been to ensure that both coldplugged and hotplugged
> devices "just work" out of the box with zero guest admin config required.
> To some extent this is contrary to what a confidential guest will want.
> It doesn't want a getty spawned on any console exposed, it doesn't want
> to use a virtio-rng exposed by the host which could be feeding non-random.
>
>
> Protecting against malicious implementations of devices is conceivably
> interesting, as a hardening task. A malicious host may try to take
> advantage of the guest OS device driver impl to exploit the guest OS
> kernel with an end goal of getting into a state where it can be made
> to reveal confidential data that was otherwise protected.

I think this is really what the Intel stuff is trying to protect
against.

Dave

> With regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 04:16:02PM +0100, Greg Kroah-Hartman wrote:
> Everyone wants a subset, different from other's subset, which means you
> need them all. Sorry.

Well if there's a very popular system (virtual in this case) that needs
a specific config to work well, then I guess
arch/x86/configs/ccguest.config or whatever might be acceptable, no?
Lots of precedent here.

--
MST
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > On Wed, Jan 25, 2023 at 01:42:53PM +0000, Dr. David Alan Gilbert wrote:
> > > > * Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > Hi Greg,
> > > > > >
> > > > > > You mentioned couple of times (last time in this recent thread:
> > > > > > https://lore.kernel.org/all/Y80WtujnO7kfduAZ@kroah.com/) that we ought to start
> > > > > > discussing the updated threat model for kernel, so this email is a start in this direction.
> > > > >
> > > > > Any specific reason you didn't cc: the linux-hardening mailing list?
> > > > > This seems to be in their area as well, right?
> > > > >
> > > > > > As we have shared before in various lkml threads/conference presentations
> > > > > > ([1], [2], [3] and many others), for the Confidential Computing guest kernel, we have a
> > > > > > change in the threat model where guest kernel doesn’t anymore trust the hypervisor.
> > > > >
> > > > > That is, frankly, a very funny threat model. How realistic is it really
> > > > > given all of the other ways that a hypervisor can mess with a guest?
> > > >
> > > > It's what a lot of people would like; in the early attempts it was easy
> > > > to defeat, but in TDX and SEV-SNP the hypervisor has a lot less that it
> > > > can mess with - remember that not just the memory is encrypted, so is
> > > > the register state, and the guest gets to see changes to mapping and a
> > > > lot of control over interrupt injection etc.
> > >
> > > And due to the fact that SEV and TDX really do not work, how is anyone
> > > expecting any of this to work? As one heckler on IRC recently put it,
> > > if you squint hard enough, you can kind of ignore the real-world issues
> > > here, so perhaps this should all be called "squint-puting" in order to
> > > feel like you have a "confidential" system? :)
> >
> > I agree the original SEV was that weak; I've not seen anyone give a good
> > argument against SNP or TDX.
>
> Argument that it doesn't work? I thought that ship sailed a long time
> ago but I could be wrong as I don't really pay attention to that stuff
> as it's just vaporware :)

You're being unfair claiming it's vaporware. You can go out and buy SNP
hardware now (for over a year), the patches are on list and under review
(and have been for quite a while).
If you're claiming it doesn't, please justify it.

> > > > > So what do you actually trust here? The CPU? A device? Nothing?
> > > >
> > > > We trust the actual physical CPU, provided that it can prove that it's a
> > > > real CPU with the CoCo hardware enabled.
> > >
> > > Great, so why not have hardware attestation also for your devices you
> > > wish to talk to? Why not use that as well? Then you don't have to
> > > worry about anything in the guest.
> >
> > There were some talks at Plumbers where PCIe is working on adding that;
> > it's not there yet though. I think that's PCIe 'Integrity and Data
> > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > SPDM. I don't know much of the detail of those, just that they're far
> > enough off that people aren't depending on them yet.
>
> Then work with those groups to implement that in an industry-wide way
> and then take advantage of it by adding support for it to Linux! Don't
> try to reinvent the same thing in a totally different way please.

Sure, people are working with them; but those are going to take time
and people want to use existing PCIe devices; and given that the hosts
are available that seems reasonable.

> > > > Both the SNP and TDX hardware
> > > > can perform an attestation signed by the CPU to prove to someone
> > > > external that the guest is running on a real trusted CPU.
> > >
> > > And again, do the same thing for the other hardware devices and all is
> > > good. To not do that is to just guess and wave hands. You know this :)
> >
> > That wouldn't help you necessarily for virtual devices - where the
> > hypervisor implements the device (like a virtual NIC).
>
> Then create a new bus for that if you don't trust the virtio bus today.

It's not that I distrust the virtio bus - just that we need to make sure
it's implementation is pessimistic enough for CoCo.

> > > > > I hate the term "hardening". Please just say it for what it really is,
> > > > > "fixing bugs to handle broken hardware". We've done that for years when
> > > > > dealing with PCI and USB and even CPUs doing things that they shouldn't
> > > > > be doing. How is this any different in the end?
> > > > >
> > > > > So what you also are saying here now is "we do not trust any PCI
> > > > > devices", so please just say that (why do you trust USB devices?) If
> > > > > that is something that you all think that Linux should support, then
> > > > > let's go from there.
> > > >
> > > > I don't think generally all PCI device drivers guard against all the
> > > > nasty things that a broken implementation of their hardware can do.
> > >
> > > I know that all PCI drivers can NOT do that today as that was never
> > > anything that Linux was designed for.
> >
> > Agreed; which again is why I only really worry about the subset of
> > devices I'd want in a CoCo VM.
>
> Everyone wants a subset, different from other's subset, which means you
> need them all. Sorry.

I think for CoCo the subset is fairly small, even including all the
people discussing it. It's the virtual devices, and a few of their
favourite physical devices, but a fairly small subset.

> > > > The USB devices are probably a bit better, because they actually worry
> > > > about people walking up with a nasty HID device; I'm skeptical that
> > > > a kernel would survive a purposely broken USB controller.
> > >
> > > I agree with you there, USB drivers are only starting to be fuzzed at
> > > the descriptor level, that's all. Which is why they too can be put into
> > > the "untrusted" area until you trust them.
> > >
> > > > I'm not sure the request here isn't really to make sure *all* PCI devices
> > > > are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
> > > > and potentially ones that people will want to pass-through (which
> > > > generally needs a lot more work to make safe).
> > > > (I've not looked at these Intel tools to see what they cover)
> > >
> > > Why not just create a whole new bus path for these "trusted" devices to
> > > attach to and do that instead of tyring to emulate a protocol that was
> > > explicitly designed NOT to this model at all? Why are you trying to
> > > shoehorn something here and not just designing it properly from the
> > > beginning?
> >
> > I'd be kind of OK with that for the virtual devices; but:
> >
> > a) I think you'd start reinventing PCIe with enumeration etc
>
> Great, then work with the PCI group as talked about above to solve it
> properly and not do whack-a-mole like seems to be happening so far.
>
> > b) We do want those pass through NICs etc that are PCIe
> > - as long as you use normal guest crypto stuff then the host
> > can be just as nasty as it likes with the data they present.
>
> Great, work with the PCI spec for verified devices.
>
> > c) The world has enough bus protocols, and people understand the
> > basics of PCI(e) - we really don't need another one.
>
> Great, work with the PCI spec people please.

As I say above; all happening - but it's going to take years.
It's wrong to leave users with less secure solutions if there are simple
fixes available. I agree that if it involves major pain all over then
I can see your dislike - but if it's small fixes then what's the
problem?

> > > > Having said that, how happy are you with Thunderbolt PCI devices being
> > > > plugged into your laptop or into the hotplug NVMe slot on a server?
> > >
> > > We have protection for that, and have had it for many years. Same for
> > > USB devices. This isn't new, perhaps you all have not noticed those
> > > features be added and taken advantage of already by many Linux distros
> > > and system images (i.e. ChromeOS and embedded systems?)
> >
> > What protection? I know we have an IOMMU, and that stops the device
> > stamping all over RAM by itself - but I think Intel's worries are more
> > subtle, things where the device starts playing with what PCI devices
> > are expected to do to try and trigger untested kernel paths. I don't
> > think there's protection against that.
> > I know we can lock by PCI/USB vendor/device ID - but those can be made
> > up trivially; protection like that is meaningless.
>
> Then combine it with device attestation and you have a solved solution,
> don't ignore others working on this please.
>
> > > > We're now in the position we were with random USB devices years ago.
> > >
> > > Nope, we are not, again, we already handle random PCI devices being
> > > plugged in. It's up to userspace to make the policy decision if it
> > > should be trusted or not before the kernel has access to it.
> > >
> > > So a meta-comment, why not just use that today? If your guest OS can
> > > not authenticate the PCI device passed to it, don't allow the kernel to
> > > bind to it. If it can be authenticated, wonderful, bind away! You can
> > > do this today with no kernel changes needed.
> >
> > Because:
> > a) there's no good way to authenticate a PCI device yet
> > - any nasty device can claim to have a given PCI ID.
> > b) Even if you could, there's no man-in-the-middle protection yet.
>
> Where is the "man" here in the middle of?

I'm worried what a malicious hypervisor could do.

> And any PCI attestation should handle that, if not, work with them to
> solve that please.

I believe the two mechanisms I mentioned above would handle that; when
it eventually gets there.

> Thunderbolt has authenticated device support today, and so does PCI, and
> USB has had it for a decade or so. Use the in-kernel implementation
> that we already have or again, show us where it is lacking and we will
> be glad to take patches to cover the holes (as we did last year when
> ChromeOS implemented support for it in their userspace.)

I'd appreciate pointers to the implementations you're referring to.

> > > > Also we would want to make sure that any config data that the hypervisor
> > > > can pass to the guest is validated.
> > >
> > > Define "validated" please.
> >
> > Lets say you get something like a ACPI table or qemu fw.cfg table
> > giving details of your devices; if the hypervisor builds those in a
> > nasty way what happens?
>
> You tell me, as we trust ACPI tables today, and if we can not, again
> then you need to change the model of what Linux does. Why isn't the
> BIOS authentication path working properly for ACPI tables already today?
> I thought that was a long-solved problem with UEFI (if not, I'm sure the
> UEFI people would be interested.)

If it's part of the BIOS image that's measured/loaded during startup
then we're fine; if it's a table dynamically generated by the hypervisor
I'm more worried.

> Anyway, I'll wait until I see real patches as this thread seems to be
> totally vague and ignores our current best-practices for pluggable
> devices for some odd reason.

Please point people at those best practices rather than just ranting
about how pointless you feel all this is!

The patches here from Intel are a TOOL to find problems; I can't see the
objections to having a tool like this.

(I suspect some of these fixes might make the kernel a bit more robust
against unexpected hot-remove of PCIe devices as well; but that's more
of a guess)

Dave

> thanks,
>
> greg k-h
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 10:45:48AM -0500, Michael S. Tsirkin wrote:
> On Wed, Jan 25, 2023 at 04:16:02PM +0100, Greg Kroah-Hartman wrote:
> > Everyone wants a subset, different from other's subset, which means you
> > need them all. Sorry.
>
> Well if there's a very popular system (virtual in this case) that needs
> a specific config to work well, then I guess
> arch/x86/configs/ccguest.config or whatever might be acceptable, no?
> Lots of precedent here.

OS vendors want the single kernel that fits all sizes: it should be
possible (and secure) to run a generic disto kernel within TDX/SEV guest.

--
Kiryl Shutsemau / Kirill A. Shutemov
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > Again, as our documentation states, when you submit patches based on
> > these tools, you HAVE TO document that. Otherwise we think you all are
> > crazy and will get your patches rejected. You all know this, why ignore
> > it?
>
> Sorry, I didn’t know that for every bug that is found in linux kernel when
> we are submitting a fix that we have to list the way how it has been found.
> We will fix this in the future submissions, but some bugs we have are found by
> plain code audit, so 'human' is the tool.

So the concern is that *you* may think it is a bug, but other people
may not agree. Perhaps what is needed is a full description of the
goals of Confidential Computing, and what is in scope, and what is
deliberately *not* in scope. I predict that when you do this, that
people will come out of the wood work and say, no wait, "CoCo ala
S/390 means FOO", and "CoCo ala AMD means BAR", and "CoCo ala RISC V
means QUUX".

Others may end up objecting, "no wait, doing this is going to mean
***insane*** changes to the entire kernel, and this will be a
performance / maintenance nightmare and unless you fix your hardware
in future chips, we wlil consider this a hardware bug and reject all
of your patches".

But it's better to figure this out now, then after you get hundreds of
patches into the upstream kernel, we discover that this is only 5% of
the necessary changes, and then the rest of your patches are rejected,
and you have to end up fixing the hardware anyway, with the patches
upstreamed so far being wasted effort. :-)

If we get consensus on that document, then that can get checked into
Documentation, and that can represent general consensus on the problem
early on.

- Ted
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 07:02:03PM +0300, Kirill A. Shutemov wrote:
> On Wed, Jan 25, 2023 at 10:45:48AM -0500, Michael S. Tsirkin wrote:
> > On Wed, Jan 25, 2023 at 04:16:02PM +0100, Greg Kroah-Hartman wrote:
> > > Everyone wants a subset, different from other's subset, which means you
> > > need them all. Sorry.
> >
> > Well if there's a very popular system (virtual in this case) that needs
> > a specific config to work well, then I guess
> > arch/x86/configs/ccguest.config or whatever might be acceptable, no?
> > Lots of precedent here.
>
> OS vendors want the single kernel that fits all sizes: it should be
> possible (and secure) to run a generic disto kernel within TDX/SEV guest.

If they want that, sure. But it then becomes this distro's
responsibility to configure things in a sane way. At least if
there's a known good config that's a place to document what
is known to work well. No?

--
MST
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, 25 Jan 2023, Greg Kroah-Hartman wrote:

> Argument that it doesn't work? I thought that ship sailed a long time
> ago but I could be wrong as I don't really pay attention to that stuff
> as it's just vaporware :)

Greg, are you sure you are talking about *SEV-SNP* here? (*)

That ship hasn't sailed as far as I can tell, it's being actively worked
on.

With SEV-SNP launch attestation, FDE, and runtime remote attestation (**)
one thing that you get is a way how to ensure that the guest image that
you have booted in a (public) cloud hasn't been tampered with, even if you
have zero trust in the cloud provider and their hypervisor.

And that without the issues and side-channels previous SEV and SEV-ES had.

Which to me is a rather valid usecase in today's world, rather than
vaporware.

(*) and corresponding Intel-TDX support counterpart, once it exists

(**) which is not necessarily a kernel work of course, but rather
userspace integration work, e.g. based on Keylime

--
Jiri Kosina
SUSE Labs
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, 25 Jan 2023, Greg Kroah-Hartman wrote:

> How do you trust you got real data on the disk? On the network? Those
> are coming from the host, how is any of that data to be trusted? Where
> does the trust stop and why?

This is all well described in AMD SEV-SNP documentation, see page 5 of
[1]. All the external devices are treated as untrusted in that model.

[1] https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf

--
Jiri Kosina
SUSE Labs
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
[cc += Jonathan Cameron, linux-pci]

On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > Great, so why not have hardware attestation also for your devices you
> > wish to talk to? Why not use that as well? Then you don't have to
> > worry about anything in the guest.
>
> There were some talks at Plumbers where PCIe is working on adding that;
> it's not there yet though. I think that's PCIe 'Integrity and Data
> Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> SPDM. I don't know much of the detail of those, just that they're far
> enough off that people aren't depending on them yet.

CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:

https://github.com/l1k/linux/commits/doe

It will allow for authentication of PCIe devices. Goal is to submit
this quarter (Q1). Afterwards we'll look into retrieving measurements
via CMA/SPDM and bringing up IDE encryption.

It's a kernel-native implementation which uses the existing crypto and
keys infrastructure and is wired into the appropriate places in the
PCI core to authenticate devices on enumeration and reauthenticate
when CMA/SPDM state is lost (after resume from D3cold, after a
Secondary Bus Reset and after a DPC-induced Hot Reset).

The device authentication service afforded here is generic.
It is up to users and vendors to decide how to employ it,
be it for "confidential computing" or something else.

Trusted root certificates to validate device certificates can be
installed into a kernel keyring using the familiar keyctl(1) utility,
but platform-specific roots of trust (such as a HSM) could be
supported as well.

I would like to stress that this particular effort is a collaboration
of multiple vendors. It is decidedly not a single vendor trying to
shoehorn something into upstream, so the criticism that has been
leveled upthread against other things does not apply here.

The Plumbers BoF you're referring to was co-authored by Jonathan Cameron
and me and its purpose was precisely to have an open discussion and
align on an approach that works for everyone:

https://lpc.events/event/16/contributions/1304/


> a) there's no good way to authenticate a PCI device yet
> - any nasty device can claim to have a given PCI ID.

CMA/SPDM prescribes that the Subject Alternative Name of the device
certificate contains the Vendor ID, Device ID, Subsystem Vendor ID,
Subsystem ID, Class Code, Revision and Serial Number (PCIe r6.0
sec 6.31.3).

Thus a forged Device ID in the Configuration Space Header will result
in authentication failure.

Thanks,

Lukas
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > Again, as our documentation states, when you submit patches based on
> > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > crazy and will get your patches rejected. You all know this, why ignore
> > > it?
> >
> > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > we are submitting a fix that we have to list the way how it has been found.
> > We will fix this in the future submissions, but some bugs we have are found by
> > plain code audit, so 'human' is the tool.
>
> So the concern is that *you* may think it is a bug, but other people
> may not agree. Perhaps what is needed is a full description of the
> goals of Confidential Computing, and what is in scope, and what is
> deliberately *not* in scope. I predict that when you do this, that
> people will come out of the wood work and say, no wait, "CoCo ala
> S/390 means FOO", and "CoCo ala AMD means BAR", and "CoCo ala RISC V
> means QUUX".

Agree, and this is the reason behind starting this thread: to make sure people
agree on the threat model. The only reason why we submitted some trivial bugs
fixes separately is the fact that they *also* can be considered bugs under existing
threat model, if one thinks that kernel should be as robust as possible against
potential erroneous devices.

As described right in the beginning of the doc I shared [1] (adjusted now to remove
'TDX' and put generic 'CC guest kernel'), we want to make sure that an untrusted
host (and hypervisor) is not able to

1. archive privileged escalation into a CC guest kernel
2. compromise the confidentiality or integrity of CC guest private memory

The above security objectives give us two primary assets we want to protect:
CC guest execution context and CC guest private memory confidentiality and
integrity.

The DoS from the host towards CC guest is explicitly out of scope and a non-security
objective.

The attack surface in question is any interface exposed from a CC guest kernel
towards untrusted host that is not covered by the CC HW protections. Here the
exact list can differ somewhat depending on what technology is being used, but as
David already pointed out before: both CC guest memory and register state is
protected from host attacks, so we are focusing on other communication channels
and on generic interfaces used by Linux today.

Examples of such interfaces for TDX (and I think SEV shares most of them, but please
correct me if I am wrong here) are access to some MSRs and CPUIDs, port IO, MMIO
and DMA, access to PCI config space, KVM hypercalls (if hypervisor is KVM), TDX specific
hypercalls (this is technology specific), data consumed from untrusted host during the
CC guest initialization (including kernel itself, kernel command line, provided ACPI tables,
etc) and others described in [1].
An important note here is that these interfaces are not limited just to device drivers
(albeit device drivers are the biggest users for some of them), they are present through the whole
kernel in different subsystems and need careful examination and development of
mitigations.

The possible range of mitigations that we can apply is also wide, but you can roughly split it into
two groups:

1. mitigations that use various attestation mechanisms (we can attest the kernel code,
cmline, ACPI tables being provided and other potential configurations, and one day we
will hopefully also be able to attest devices we connect to CC guest and their configuration)

2. other mitigations for threats that attestation cannot cover, i.e. mainly runtime
interactions with the host.

Above sounds conceptually simple but the devil is as usual in details, but it doesn’t look
very impossible or smth that would need the ***insane*** changes to the entire kernel.

>
> Others may end up objecting, "no wait, doing this is going to mean
> ***insane*** changes to the entire kernel, and this will be a
> performance / maintenance nightmare and unless you fix your hardware
> in future chips, we wlil consider this a hardware bug and reject all
> of your patches".
>
> But it's better to figure this out now, then after you get hundreds of
> patches into the upstream kernel, we discover that this is only 5% of
> the necessary changes, and then the rest of your patches are rejected,
> and you have to end up fixing the hardware anyway, with the patches
> upstreamed so far being wasted effort. :-)
>
> If we get consensus on that document, then that can get checked into
> Documentation, and that can represent general consensus on the problem
> early on.

Sure, I am willing to work on this since we already spent quite a lot of effort
looking into this problem. My only question is how to organize a review of such
document in a sane and productive way and to make sure all relevant people
are included into discussion. As I said, this spawns across many areas in kernel,
and ideally you would want different people review their area in detail.
For example, one of many aspects we need to worry is security of CC guest LRNG (
especially in cases when we don’t have a trusted security HW source of entropy)
[2] and here a feedback from LRNG experts would be important.

I guess the first clear step I can do is to re-write the relevant part of [1] into a CC-technology
neutral language and then would need feedback and input from AMD guys to make
sure it correctly reflects their case also. We can probably do this preparation work
on linux-coco mailing list and then post for a wider review?

Best Regards,
Elena.

[1] https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html#threat-model
[2] https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html#randomness-inside-tdx-guest

>
> - Ted
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 04:16:02PM +0100, Greg Kroah-Hartman wrote:
> Argument that it doesn't work? I thought that ship sailed a long time
> ago but I could be wrong as I don't really pay attention to that stuff
> as it's just vaporware :)

Well, "vaporware" is a bold word, especially given the fact that one can
get a confidential VM using AMD SEV[1] or SEV-SNP[2] the cloud today.
Hardware for SEV-SNP is also widely available since at least October
2021.

But okay, there seems to be some misunderstanding what Confidential
Computing (CoCo) implicates, so let me state my view here.

The vision for CoCo is to remove trust from the hypervisor (HV), so that
a guest owner only needs to trust the hardware and the os vendor for the
VM to be trusted and the data in it to be secure.

The implication is that the guest-HV interface becomes an attack surface
for the guest, and there are two basic strategies to mitigate the risk:

1) Move HV functionality into the guest or the hardware and
reduce the guest-HV interface. This already happened to some
degree with the SEV-ES enablement, where instruction decoding
and handling of most intercepts moved into the guest kernel.

2) Harden the guest-HV interface against malicious input.

Where possible we are going with option 1, up to the point where
scheduling our VCPUs is the only point we need to trust the HV on.

For example, the whole interrupt injection logic will also move either
into guest context or the hardware (depends on the HW vendor). That
covers most of the CPU emulation that the HV was doing, but an equally
important part is device emulation.

For device emulation it is harder to move that into the trusted guest
context, first of all because there is limited hardware support for
that, secondly because it will not perform well.

So device emulation will have to stay in the HV for the forseeable
future (except for devices carrying secrets, like the TPM). What Elena
and others are trying in this thread is to make the wider kernel
community aware that malicious input to a device driver is a real
problem in some environments and driver hardening is actually worthwile.

Regards,

Joerg


[1] https://cloud.google.com/confidential-computing
[2] https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-vm-overview
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Lukas Wunner (lukas@wunner.de) wrote:
> [cc += Jonathan Cameron, linux-pci]
>
> On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > Great, so why not have hardware attestation also for your devices you
> > > wish to talk to? Why not use that as well? Then you don't have to
> > > worry about anything in the guest.
> >
> > There were some talks at Plumbers where PCIe is working on adding that;
> > it's not there yet though. I think that's PCIe 'Integrity and Data
> > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > SPDM. I don't know much of the detail of those, just that they're far
> > enough off that people aren't depending on them yet.
>
> CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
>
> https://github.com/l1k/linux/commits/doe

Thanks for the pointer - I'll go and hunt down that spec.

> It will allow for authentication of PCIe devices. Goal is to submit
> this quarter (Q1). Afterwards we'll look into retrieving measurements
> via CMA/SPDM and bringing up IDE encryption.
>
> It's a kernel-native implementation which uses the existing crypto and
> keys infrastructure and is wired into the appropriate places in the
> PCI core to authenticate devices on enumeration and reauthenticate
> when CMA/SPDM state is lost (after resume from D3cold, after a
> Secondary Bus Reset and after a DPC-induced Hot Reset).
>
> The device authentication service afforded here is generic.
> It is up to users and vendors to decide how to employ it,
> be it for "confidential computing" or something else.

As Samuel asks about who is doing the challenge; but I guess there are
also things like what happens when the host controls intermediate
switches and BAR access and when only VFs are passed to guests.

> Trusted root certificates to validate device certificates can be
> installed into a kernel keyring using the familiar keyctl(1) utility,
> but platform-specific roots of trust (such as a HSM) could be
> supported as well.
>
> I would like to stress that this particular effort is a collaboration
> of multiple vendors. It is decidedly not a single vendor trying to
> shoehorn something into upstream, so the criticism that has been
> leveled upthread against other things does not apply here.
>
> The Plumbers BoF you're referring to was co-authored by Jonathan Cameron
> and me and its purpose was precisely to have an open discussion and
> align on an approach that works for everyone:
>
> https://lpc.events/event/16/contributions/1304/
>
>
> > a) there's no good way to authenticate a PCI device yet
> > - any nasty device can claim to have a given PCI ID.
>
> CMA/SPDM prescribes that the Subject Alternative Name of the device
> certificate contains the Vendor ID, Device ID, Subsystem Vendor ID,
> Subsystem ID, Class Code, Revision and Serial Number (PCIe r6.0
> sec 6.31.3).
>
> Thus a forged Device ID in the Configuration Space Header will result
> in authentication failure.

Good! It'll be nice when people figure out the CoCo integration for
that; I'm still guessing it's a little way off until we get hardware
for that.

Dave

> Thanks,
>
> Lukas
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, 26 Jan 2023 10:24:32 +0100
Samuel Ortiz <sameo@rivosinc.com> wrote:

> Hi Lukas,
>
> On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
>
> > [cc += Jonathan Cameron, linux-pci]
> >
> > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > Great, so why not have hardware attestation also for your devices you
> > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > worry about anything in the guest.
> > >
> > > There were some talks at Plumbers where PCIe is working on adding that;
> > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > SPDM. I don't know much of the detail of those, just that they're far
> > > enough off that people aren't depending on them yet.
> >
> > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> >
> > https://github.com/l1k/linux/commits/doe
>
> Nice, thanks a lot for that.
>
>
>
> > The device authentication service afforded here is generic.
> > It is up to users and vendors to decide how to employ it,
> > be it for "confidential computing" or something else.
> >
> > Trusted root certificates to validate device certificates can be
> > installed into a kernel keyring using the familiar keyctl(1) utility,
> > but platform-specific roots of trust (such as a HSM) could be
> > supported as well.
> >
>
> This may have been discussed at LPC, but are there any plans to also
> support confidential computing flows where the host kernel is not part
> of the TCB and would not be trusted for validating the device cert chain
> nor for running the SPDM challenge?

There are lots of possible models for this. One simple option if the assigned
VF supports it is a CMA instance per VF. That will let the guest
do full attestation including measurement of whether the device is
appropriately locked down so the hypervisor can't mess with
configuration that affects the guest (without a reset anyway and that
is guest visible). Whether anyone builds that option isn't yet clear
though. If they do, Lukas' work should work there as well as for the
host OS. (Note I'm not a security expert so may be missing something!)

For extra fun, why should the device trust the host? Mutual authentication
fun (there are usecases where that matters)

There are way more complex options supported in PCIe TDISP (Tee Device
security interface protocols). Anyone have an visibility of open solutions
that make use of that? May be too new.

Jonathan


>
> Cheers,
> Samuel.
>
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> Replying only to the not-so-far addressed points.
>
> > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > Hi Greg,

<...>

> > > 3) All the tools are open-source and everyone can start using them right away
> > even
> > > without any special HW (readme has description of what is needed).
> > > Tools and documentation is here:
> > > https://github.com/intel/ccc-linux-guest-hardening
> >
> > Again, as our documentation states, when you submit patches based on
> > these tools, you HAVE TO document that. Otherwise we think you all are
> > crazy and will get your patches rejected. You all know this, why ignore
> > it?
>
> Sorry, I didn’t know that for every bug that is found in linux kernel when
> we are submitting a fix that we have to list the way how it has been found.
> We will fix this in the future submissions, but some bugs we have are found by
> plain code audit, so 'human' is the tool.

My problem with that statement is that by applying different threat
model you "invent" bugs which didn't exist in a first place.

For example, in this [1] latest submission, authors labeled correct
behaviour as "bug".

[1] https://lore.kernel.org/all/20230119170633.40944-1-alexander.shishkin@linux.intel.com/

Thanks
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, 26 Jan 2023 10:48:50 +0000
"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:

> * Lukas Wunner (lukas@wunner.de) wrote:
> > [cc += Jonathan Cameron, linux-pci]
> >
> > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > Great, so why not have hardware attestation also for your devices you
> > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > worry about anything in the guest.
> > >
> > > There were some talks at Plumbers where PCIe is working on adding that;
> > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > SPDM. I don't know much of the detail of those, just that they're far
> > > enough off that people aren't depending on them yet.
> >
> > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> >
> > https://github.com/l1k/linux/commits/doe
>
> Thanks for the pointer - I'll go and hunt down that spec.
>
> > It will allow for authentication of PCIe devices. Goal is to submit
> > this quarter (Q1). Afterwards we'll look into retrieving measurements
> > via CMA/SPDM and bringing up IDE encryption.
> >
> > It's a kernel-native implementation which uses the existing crypto and
> > keys infrastructure and is wired into the appropriate places in the
> > PCI core to authenticate devices on enumeration and reauthenticate
> > when CMA/SPDM state is lost (after resume from D3cold, after a
> > Secondary Bus Reset and after a DPC-induced Hot Reset).
> >
> > The device authentication service afforded here is generic.
> > It is up to users and vendors to decide how to employ it,
> > be it for "confidential computing" or something else.
>
> As Samuel asks about who is doing the challenge; but I guess there are
> also things like what happens when the host controls intermediate
> switches and BAR access and when only VFs are passed to guests.

Hmm. Bringing switches into the TCB came up at Plumbers.
You can get partly around that using selective IDE (end to end encryption)
but it has some disadvantages.

You can attest the switches if you don't mind bringing them into TCB
(one particularly cloud vendor person was very strongly against doing so!)
but they don't have nice VF type abstractions so the switch attestation
needs to go through someone who isn't the guest.

>
> > Trusted root certificates to validate device certificates can be
> > installed into a kernel keyring using the familiar keyctl(1) utility,
> > but platform-specific roots of trust (such as a HSM) could be
> > supported as well.
> >
> > I would like to stress that this particular effort is a collaboration
> > of multiple vendors. It is decidedly not a single vendor trying to
> > shoehorn something into upstream, so the criticism that has been
> > leveled upthread against other things does not apply here.
> >
> > The Plumbers BoF you're referring to was co-authored by Jonathan Cameron
> > and me and its purpose was precisely to have an open discussion and
> > align on an approach that works for everyone:
> >
> > https://lpc.events/event/16/contributions/1304/
> >
> >
> > > a) there's no good way to authenticate a PCI device yet
> > > - any nasty device can claim to have a given PCI ID.
> >
> > CMA/SPDM prescribes that the Subject Alternative Name of the device
> > certificate contains the Vendor ID, Device ID, Subsystem Vendor ID,
> > Subsystem ID, Class Code, Revision and Serial Number (PCIe r6.0
> > sec 6.31.3).
> >
> > Thus a forged Device ID in the Configuration Space Header will result
> > in authentication failure.
>
> Good! It'll be nice when people figure out the CoCo integration for
> that; I'm still guessing it's a little way off until we get hardware
> for that.

FYI: We have QEMU using the DMTF reference implementation (libspdm/spdm-emu)
if anyone wants to play with it. Avery Design folk did the qemu bridging to that
a while back. Not upstream yet*, but I'm carrying it on my staging CXL qemu tree.

https://gitlab.com/jic23/qemu/-/commit/8d0ad6bc84a5d96039aaf8f929c60b9f7ba02832

In combination with Lucas' tree mentioned earlier you can get all the handshaking
to happen to attest against certs. Don't think we are yet actually checking the
IDs but trivial to add (mainly a case of generating right certs with the
Subject Alternative Name set).

Jonathan

* It's a hack using the socket interface of spdm-emu tools - at some point I need
to start a discussion on QEMU list / with dmtf tools group on whether to fix
libspdm to actually work as a shared library, or cope with the current approach
(crossing fingers the socket interface remains stable in spdm-emu).

>
> Dave
>
> > Thanks,
> >
> > Lukas
> >
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
> On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > Replying only to the not-so-far addressed points.
> >
> > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > Hi Greg,
>
> <...>
>
> > > > 3) All the tools are open-source and everyone can start using them right
> away
> > > even
> > > > without any special HW (readme has description of what is needed).
> > > > Tools and documentation is here:
> > > > https://github.com/intel/ccc-linux-guest-hardening
> > >
> > > Again, as our documentation states, when you submit patches based on
> > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > crazy and will get your patches rejected. You all know this, why ignore
> > > it?
> >
> > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > we are submitting a fix that we have to list the way how it has been found.
> > We will fix this in the future submissions, but some bugs we have are found by
> > plain code audit, so 'human' is the tool.
>
> My problem with that statement is that by applying different threat
> model you "invent" bugs which didn't exist in a first place.
>
> For example, in this [1] latest submission, authors labeled correct
> behaviour as "bug".
>
> [1] https://lore.kernel.org/all/20230119170633.40944-1-
> alexander.shishkin@linux.intel.com/

Hm.. Does everyone think that when kernel dies with unhandled page fault
(such as in that case) or detection of a KASAN out of bounds violation (as it is in some
other cases we already have fixes or investigating) it represents a correct behavior even if
you expect that all your pci HW devices are trusted? What about an error in two
consequent pci reads? What about just some failure that results in erroneous input?

Best Regards,
Elena.

1 2 3 4 5  View All