Mailing List Archive

1 2 3 4 5  View All
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 11:29:20AM +0000, Reshetova, Elena wrote:
> > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > Replying only to the not-so-far addressed points.
> > >
> > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > Hi Greg,
> >
> > <...>
> >
> > > > > 3) All the tools are open-source and everyone can start using them right
> > away
> > > > even
> > > > > without any special HW (readme has description of what is needed).
> > > > > Tools and documentation is here:
> > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > >
> > > > Again, as our documentation states, when you submit patches based on
> > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > it?
> > >
> > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > we are submitting a fix that we have to list the way how it has been found.
> > > We will fix this in the future submissions, but some bugs we have are found by
> > > plain code audit, so 'human' is the tool.
> >
> > My problem with that statement is that by applying different threat
> > model you "invent" bugs which didn't exist in a first place.
> >
> > For example, in this [1] latest submission, authors labeled correct
> > behaviour as "bug".
> >
> > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > alexander.shishkin@linux.intel.com/
>
> Hm.. Does everyone think that when kernel dies with unhandled page fault
> (such as in that case) or detection of a KASAN out of bounds violation (as it is in some
> other cases we already have fixes or investigating) it represents a correct behavior even if
> you expect that all your pci HW devices are trusted?

This is exactly what I said. You presented me the cases which exist in
your invented world. Mentioned unhandled page fault doesn't exist in real
world. If PCI device doesn't work, it needs to be replaced/blocked and not
left to be operable and accessible from the kernel/user.

> What about an error in two consequent pci reads? What about just some
> failure that results in erroneous input?

Yes, some bugs need to be fixed, but they are not related to trust/not-trust
discussion and PCI spec violations.

Thanks

>
> Best Regards,
> Elena.
>
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
> > > I hate the term "hardening". Please just say it for what it really is,
> > > "fixing bugs to handle broken hardware". We've done that for years when
> > > dealing with PCI and USB and even CPUs doing things that they shouldn't
> > > be doing. How is this any different in the end?
> > >
> > > So what you also are saying here now is "we do not trust any PCI
> > > devices", so please just say that (why do you trust USB devices?) If
> > > that is something that you all think that Linux should support, then
> > > let's go from there.
> >
> > I don't think generally all PCI device drivers guard against all the
> > nasty things that a broken implementation of their hardware can do.
>
> I know that all PCI drivers can NOT do that today as that was never
> anything that Linux was designed for.
>
> > The USB devices are probably a bit better, because they actually worry
> > about people walking up with a nasty HID device; I'm skeptical that
> > a kernel would survive a purposely broken USB controller.
>
> I agree with you there, USB drivers are only starting to be fuzzed at
> the descriptor level, that's all. Which is why they too can be put into
> the "untrusted" area until you trust them.
>
> > I'm not sure the request here isn't really to make sure *all* PCI devices
> > are safe; just the ones we care about in a CoCo guest (e.g. the virtual devices) -
> > and potentially ones that people will want to pass-through (which
> > generally needs a lot more work to make safe).
> > (I've not looked at these Intel tools to see what they cover)
>
> Why not just create a whole new bus path for these "trusted" devices to
> attach to and do that instead of tyring to emulate a protocol that was
> explicitly designed NOT to this model at all? Why are you trying to
> shoehorn something here and not just designing it properly from the
> beginning?
>
> > Having said that, how happy are you with Thunderbolt PCI devices being
> > plugged into your laptop or into the hotplug NVMe slot on a server?
>
> We have protection for that, and have had it for many years. Same for
> USB devices. This isn't new, perhaps you all have not noticed those
> features be added and taken advantage of already by many Linux distros
> and system images (i.e. ChromeOS and embedded systems?)
>
> > We're now in the position we were with random USB devices years ago.
>
> Nope, we are not, again, we already handle random PCI devices being
> plugged in. It's up to userspace to make the policy decision if it
> should be trusted or not before the kernel has access to it.
>
> So a meta-comment, why not just use that today? If your guest OS can
> not authenticate the PCI device passed to it, don't allow the kernel to
> bind to it. If it can be authenticated, wonderful, bind away! You can
> do this today with no kernel changes needed.
>
> > Also we would want to make sure that any config data that the hypervisor
> > can pass to the guest is validated.
>
> Define "validated" please.
>
> > The problem seems reasonably well understood within the CoCo world - how
> > far people want to push it probably varies; but it's good to make the
> > problem more widely understood.
>
> The "CoCo" world seems distant and separate from the real-world of Linux
> kernel development if you all do not even know about the authentication
> methods that we have for years for enabling access to PCI and USB
> devices as described above. If the impementations that we currently
> have are lacking in some way, wonderful, please submit changes for them
> and we will be glad to review them as needed.

We are aware of USB/Thunderbolt authorization framework and this is what we have
been extending now for the our CC usage in order to apply this to all devices.
The patches are currently under testing/polishing, but we will be submitting
them in the near future.

That's said even with the above in place we don’t get a protection from a man-in-
the-middle attacks that are possible by untrusted hypervisor or host. In order
to get a full protection here, we need an attestation and end-to-end secure channel
between devices and CC guest. However, since it is going to take a long time before
we have all the infrastructure in place in Linux, as well as devices that are capable of
supporting all required functionality (and some devices will never have this support such
as virtual devices), we need to have a reasonable security model now, vs waiting
until researchers are starting to post the proof-of-concept privilege escalation exploits
on smth that is even (thanks to the tools we created in in [1]) not so hard to find:
you run our fuzzing tools on the guest kernel tree of your liking and it gives you a nice set
of KASAN issues to play with. What we are trying to do is to address these findings (among
other things) for a more robust guest kernel.

Best Regards,
Elena

[1] https://github.com/intel/ccc-linux-guest-hardening
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 10:58:47AM +0000, Jonathan Cameron wrote:
> On Thu, 26 Jan 2023 10:24:32 +0100
> Samuel Ortiz <sameo@rivosinc.com> wrote:
>
> > Hi Lukas,
> >
> > On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
> >
> > > [cc += Jonathan Cameron, linux-pci]
> > >
> > > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > > Great, so why not have hardware attestation also for your devices you
> > > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > > worry about anything in the guest.
> > > >
> > > > There were some talks at Plumbers where PCIe is working on adding that;
> > > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > > SPDM. I don't know much of the detail of those, just that they're far
> > > > enough off that people aren't depending on them yet.
> > >
> > > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> > >
> > > https://github.com/l1k/linux/commits/doe
> >
> > Nice, thanks a lot for that.
> >
> >
> >
> > > The device authentication service afforded here is generic.
> > > It is up to users and vendors to decide how to employ it,
> > > be it for "confidential computing" or something else.
> > >
> > > Trusted root certificates to validate device certificates can be
> > > installed into a kernel keyring using the familiar keyctl(1) utility,
> > > but platform-specific roots of trust (such as a HSM) could be
> > > supported as well.
> > >
> >
> > This may have been discussed at LPC, but are there any plans to also
> > support confidential computing flows where the host kernel is not part
> > of the TCB and would not be trusted for validating the device cert chain
> > nor for running the SPDM challenge?
>
> There are lots of possible models for this. One simple option if the assigned
> VF supports it is a CMA instance per VF. That will let the guest
> do full attestation including measurement of whether the device is
> appropriately locked down so the hypervisor can't mess with
> configuration that affects the guest (without a reset anyway and that
> is guest visible).

So the VF would be directly assigned to the guest, and the guest kernel
would create a CMA instance for the VF, and do the SPDM authentication
(based on a guest provided trusted root certificate). I think one
security concern with that approach is assigning the VF to the
(potentially confidential) guest address space without the guest being
able to attest of the device trustworthiness first. That's what TDISP is
aiming at fixing (establish a secure SPDM between the confidential guest
and the device, lock the device from the guest, attest and then enable
DMA).

> Whether anyone builds that option isn't yet clear
> though. If they do, Lukas' work should work there as well as for the
> host OS. (Note I'm not a security expert so may be missing something!)
>
> For extra fun, why should the device trust the host? Mutual authentication
> fun (there are usecases where that matters)
>
> There are way more complex options supported in PCIe TDISP (Tee Device
> security interface protocols). Anyone have an visibility of open solutions
> that make use of that? May be too new.

It's still a PCI ECN, so quite new indeed.
FWIW the rust spdm crate [1] implements the TDISP state machine.

Cheers,
Samuel.

[1] https://github.com/jyao1/rust-spdm
>
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
> On Thu, Jan 26, 2023 at 11:29:20AM +0000, Reshetova, Elena wrote:
> > > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > > Replying only to the not-so-far addressed points.
> > > >
> > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > Hi Greg,
> > >
> > > <...>
> > >
> > > > > > 3) All the tools are open-source and everyone can start using them right
> > > away
> > > > > even
> > > > > > without any special HW (readme has description of what is needed).
> > > > > > Tools and documentation is here:
> > > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > > >
> > > > > Again, as our documentation states, when you submit patches based on
> > > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > > it?
> > > >
> > > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > > we are submitting a fix that we have to list the way how it has been found.
> > > > We will fix this in the future submissions, but some bugs we have are found
> by
> > > > plain code audit, so 'human' is the tool.
> > >
> > > My problem with that statement is that by applying different threat
> > > model you "invent" bugs which didn't exist in a first place.
> > >
> > > For example, in this [1] latest submission, authors labeled correct
> > > behaviour as "bug".
> > >
> > > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > > alexander.shishkin@linux.intel.com/
> >
> > Hm.. Does everyone think that when kernel dies with unhandled page fault
> > (such as in that case) or detection of a KASAN out of bounds violation (as it is in
> some
> > other cases we already have fixes or investigating) it represents a correct
> behavior even if
> > you expect that all your pci HW devices are trusted?
>
> This is exactly what I said. You presented me the cases which exist in
> your invented world. Mentioned unhandled page fault doesn't exist in real
> world. If PCI device doesn't work, it needs to be replaced/blocked and not
> left to be operable and accessible from the kernel/user.

Can we really assure correct operation of *all* pci devices out there?
How would such an audit be performed given a huge set of them available?
Isnt it better instead to make a small fix in the kernel behavior that would guard
us from such potentially not correctly operating devices?


>
> > What about an error in two consequent pci reads? What about just some
> > failure that results in erroneous input?
>
> Yes, some bugs need to be fixed, but they are not related to trust/not-trust
> discussion and PCI spec violations.

Let's forget the trust angle here (it only applies to the Confidential Computing
threat model and you clearly implying the existing threat model instead) and stick just to
the not-correctly operating device. What you are proposing is to fix *unknown* bugs
in multitude of pci devices that (in case of this particular MSI bug) can
lead to two different values being read from the config space and kernel incorrectly
handing this situation. Isn't it better to do the clear fix in one place to ensure such
situation (two subsequent reads with different values) cannot even happen in theory?
In security we have a saying that fixing a root cause of the problem is the most efficient
way to mitigate the problem. The root cause here is a double-read with different values,
so if it can be substituted with an easy and clear patch that probably even improves
performance as we do one less pci read and use cached value instead, where is the
problem in this particular case? If there are technical issues with the patch, of course we
need to discuss it/fix it, but it seems we are arguing here about whenever or not we want
to be fixing kernel code when we notice such cases...

Best Regards,
Elena
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 10:48:50AM +0000, Dr. David Alan Gilbert wrote:
> * Lukas Wunner (lukas@wunner.de) wrote:
> > [cc += Jonathan Cameron, linux-pci]
> >
> > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > Great, so why not have hardware attestation also for your devices you
> > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > worry about anything in the guest.
> > >
> > > There were some talks at Plumbers where PCIe is working on adding that;
> > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > SPDM. I don't know much of the detail of those, just that they're far
> > > enough off that people aren't depending on them yet.
> >
> > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> >
> > https://github.com/l1k/linux/commits/doe
>
> Thanks for the pointer - I'll go and hunt down that spec.
>
> > It will allow for authentication of PCIe devices. Goal is to submit
> > this quarter (Q1). Afterwards we'll look into retrieving measurements
> > via CMA/SPDM and bringing up IDE encryption.
> >
> > It's a kernel-native implementation which uses the existing crypto and
> > keys infrastructure and is wired into the appropriate places in the
> > PCI core to authenticate devices on enumeration and reauthenticate
> > when CMA/SPDM state is lost (after resume from D3cold, after a
> > Secondary Bus Reset and after a DPC-induced Hot Reset).
> >
> > The device authentication service afforded here is generic.
> > It is up to users and vendors to decide how to employ it,
> > be it for "confidential computing" or something else.
>
> As Samuel asks about who is doing the challenge; but I guess there are
> also things like what happens when the host controls intermediate
> switches

You'd want to protect that through IDE selective streams.

> and BAR access and when only VFs are passed to guests.

TDISP aims at addressing that afaiu. Once the VF (aka TDI) is locked,
any changes to its BAR(s) or any PF MMIO that would affect the VF would
get the VF back to unlocked (and let the guest reject it).

Cheers,
Samuel.
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 01:28:15PM +0000, Reshetova, Elena wrote:
> > On Thu, Jan 26, 2023 at 11:29:20AM +0000, Reshetova, Elena wrote:
> > > > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > > > Replying only to the not-so-far addressed points.
> > > > >
> > > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > > Hi Greg,
> > > >
> > > > <...>
> > > >
> > > > > > > 3) All the tools are open-source and everyone can start using them right
> > > > away
> > > > > > even
> > > > > > > without any special HW (readme has description of what is needed).
> > > > > > > Tools and documentation is here:
> > > > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > > > >
> > > > > > Again, as our documentation states, when you submit patches based on
> > > > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > > > it?
> > > > >
> > > > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > > > we are submitting a fix that we have to list the way how it has been found.
> > > > > We will fix this in the future submissions, but some bugs we have are found
> > by
> > > > > plain code audit, so 'human' is the tool.
> > > >
> > > > My problem with that statement is that by applying different threat
> > > > model you "invent" bugs which didn't exist in a first place.
> > > >
> > > > For example, in this [1] latest submission, authors labeled correct
> > > > behaviour as "bug".
> > > >
> > > > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > > > alexander.shishkin@linux.intel.com/
> > >
> > > Hm.. Does everyone think that when kernel dies with unhandled page fault
> > > (such as in that case) or detection of a KASAN out of bounds violation (as it is in
> > some
> > > other cases we already have fixes or investigating) it represents a correct
> > behavior even if
> > > you expect that all your pci HW devices are trusted?
> >
> > This is exactly what I said. You presented me the cases which exist in
> > your invented world. Mentioned unhandled page fault doesn't exist in real
> > world. If PCI device doesn't work, it needs to be replaced/blocked and not
> > left to be operable and accessible from the kernel/user.
>
> Can we really assure correct operation of *all* pci devices out there?

Why do we need to do it in 2022? These *all* pci devices work.

> How would such an audit be performed given a huge set of them available?

Compliance tests?
https://pcisig.com/developers/compliance-program

> Isnt it better instead to make a small fix in the kernel behavior that would guard
> us from such potentially not correctly operating devices?

Like Greg already said, this is a small drop in a ocean which needs to be changed.

However even in mentioned by me case, you are not fixing but hiding real
problem of having broken device in my machine. It is worst possible solution
for the users.

>
>
> >
> > > What about an error in two consequent pci reads? What about just some
> > > failure that results in erroneous input?
> >
> > Yes, some bugs need to be fixed, but they are not related to trust/not-trust
> > discussion and PCI spec violations.
>
> Let's forget the trust angle here (it only applies to the Confidential Computing
> threat model and you clearly implying the existing threat model instead) and stick just to
> the not-correctly operating device. What you are proposing is to fix *unknown* bugs
> in multitude of pci devices that (in case of this particular MSI bug) can
> lead to two different values being read from the config space and kernel incorrectly
> handing this situation.

Let's don't call bug for something which is not.

Random crashes are much more tolerable then "working" device which sends
random results.

> Isn't it better to do the clear fix in one place to ensure such
> situation (two subsequent reads with different values) cannot even happen in theory?
> In security we have a saying that fixing a root cause of the problem is the most efficient
> way to mitigate the problem. The root cause here is a double-read with different values,
> so if it can be substituted with an easy and clear patch that probably even improves
> performance as we do one less pci read and use cached value instead, where is the
> problem in this particular case? If there are technical issues with the patch, of course we
> need to discuss it/fix it, but it seems we are arguing here about whenever or not we want
> to be fixing kernel code when we notice such cases...

Not really, we are arguing what is the right thing to do:
1. Fix a root cause - device
2. Hide the failure and pretend what everything is perfect despite
having problematic device.

Thanks

>
> Best Regards,
> Elena
>
>
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Reshetova, Elena (elena.reshetova@intel.com) wrote:
> > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > Replying only to the not-so-far addressed points.
> > >
> > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > Hi Greg,
> >
> > <...>
> >
> > > > > 3) All the tools are open-source and everyone can start using them right
> > away
> > > > even
> > > > > without any special HW (readme has description of what is needed).
> > > > > Tools and documentation is here:
> > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > >
> > > > Again, as our documentation states, when you submit patches based on
> > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > it?
> > >
> > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > we are submitting a fix that we have to list the way how it has been found.
> > > We will fix this in the future submissions, but some bugs we have are found by
> > > plain code audit, so 'human' is the tool.
> >
> > My problem with that statement is that by applying different threat
> > model you "invent" bugs which didn't exist in a first place.
> >
> > For example, in this [1] latest submission, authors labeled correct
> > behaviour as "bug".
> >
> > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > alexander.shishkin@linux.intel.com/
>
> Hm.. Does everyone think that when kernel dies with unhandled page fault
> (such as in that case) or detection of a KASAN out of bounds violation (as it is in some
> other cases we already have fixes or investigating) it represents a correct behavior even if
> you expect that all your pci HW devices are trusted? What about an error in two
> consequent pci reads? What about just some failure that results in erroneous input?

I'm not sure you'll get general agreement on those answers for all
devices and situations; I think for most devices for non-CoCo
situations, then people are generally OK with a misbehaving PCI device
causing a kernel crash, since most people are running without IOMMU
anyway, a misbehaving device can cause otherwise undetectable chaos.

I'd say:
a) For CoCo, a guest (guaranteed) crash isn't a problem - CoCo doesn't
guarantee forward progress or stop the hypervisor doing something
truly stupid.

b) For CoCo, information disclosure, or corruption IS a problem

c) For non-CoCo some people might care about robustness of the kernel
against a failing PCI device, but generally I think they worry about
a fairly clean failure, even in the unexpected-hot unplug case.

d) It's not clear to me what 'trust' means in terms of CoCo for a PCIe
device; if it's a device that attests OK and we trust it is the device
it says it is, do we give it freedom or are we still wary?

Dave


> Best Regards,
> Elena.
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> Any virtual device exposed to the guest that can transfer potentially
> sensitive data needs to have some form of guest controlled encryption
> applied. For disks this is easy with FDE like LUKS, for NICs this is
> already best practice for services by using TLS. Other devices may not
> have good existing options for applying encryption.

I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
but not transport. If an attacker can observe all IO you better
consult a cryptographer.
LUKS has no concept of session keys or such, so the same disk sector will
always get encrypted with the very same key/iv.

--
Thanks,
//richard
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Richard Weinberger (richard.weinberger@gmail.com) wrote:
> On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrang? <berrange@redhat.com> wrote:
> > Any virtual device exposed to the guest that can transfer potentially
> > sensitive data needs to have some form of guest controlled encryption
> > applied. For disks this is easy with FDE like LUKS, for NICs this is
> > already best practice for services by using TLS. Other devices may not
> > have good existing options for applying encryption.
>
> I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
> but not transport. If an attacker can observe all IO you better
> consult a cryptographer.
> LUKS has no concept of session keys or such, so the same disk sector will
> always get encrypted with the very same key/iv.

Are you aware of anything that you'd use instead?

Are you happy with dm-verity for protection against modification?

Dave

> --
> Thanks,
> //richard
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 3:58 PM Dr. David Alan Gilbert
<dgilbert@redhat.com> wrote:
>
> * Richard Weinberger (richard.weinberger@gmail.com) wrote:
> > On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > Any virtual device exposed to the guest that can transfer potentially
> > > sensitive data needs to have some form of guest controlled encryption
> > > applied. For disks this is easy with FDE like LUKS, for NICs this is
> > > already best practice for services by using TLS. Other devices may not
> > > have good existing options for applying encryption.
> >
> > I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
> > but not transport. If an attacker can observe all IO you better
> > consult a cryptographer.
> > LUKS has no concept of session keys or such, so the same disk sector will
> > always get encrypted with the very same key/iv.
>
> Are you aware of anything that you'd use instead?

Well, I'd think towards iSCSI over TLS to protect the IO transport.

> Are you happy with dm-verity for protection against modification?

Like LUKS (actually dm-crypt) the crypto behind is designed to protect
persistent data not transport.
My fear is that an attacker who is able to observe IOs can do bad things.
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Richard Weinberger (richard.weinberger@gmail.com) wrote:
> On Thu, Jan 26, 2023 at 3:58 PM Dr. David Alan Gilbert
> <dgilbert@redhat.com> wrote:
> >
> > * Richard Weinberger (richard.weinberger@gmail.com) wrote:
> > > On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrang? <berrange@redhat.com> wrote:
> > > > Any virtual device exposed to the guest that can transfer potentially
> > > > sensitive data needs to have some form of guest controlled encryption
> > > > applied. For disks this is easy with FDE like LUKS, for NICs this is
> > > > already best practice for services by using TLS. Other devices may not
> > > > have good existing options for applying encryption.
> > >
> > > I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
> > > but not transport. If an attacker can observe all IO you better
> > > consult a cryptographer.
> > > LUKS has no concept of session keys or such, so the same disk sector will
> > > always get encrypted with the very same key/iv.
> >
> > Are you aware of anything that you'd use instead?
>
> Well, I'd think towards iSCSI over TLS to protect the IO transport.

Yeh, that's not entirely crazy for VMs which tend to come off some
remote storage system.

> > Are you happy with dm-verity for protection against modification?
>
> Like LUKS (actually dm-crypt) the crypto behind is designed to protect
> persistent data not transport.
> My fear is that an attacker who is able to observe IOs can do bad things.

Hmm, OK, I'd assumed dm-crypt was OK since it's more hashlike and
unchanging.

Dave

--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 10:24:32AM +0100, Samuel Ortiz wrote:
> On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
> > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> >
> > https://github.com/l1k/linux/commits/doe
> >
> > The device authentication service afforded here is generic.
> > It is up to users and vendors to decide how to employ it,
> > be it for "confidential computing" or something else.
> >
> > Trusted root certificates to validate device certificates can be
> > installed into a kernel keyring using the familiar keyctl(1) utility,
> > but platform-specific roots of trust (such as a HSM) could be
> > supported as well.
>
> This may have been discussed at LPC, but are there any plans to also
> support confidential computing flows where the host kernel is not part
> of the TCB and would not be trusted for validating the device cert chain
> nor for running the SPDM challenge?

As long as a device is passed through to a guest, the guest owns
that device. It is the guest's prerogative and duty to perform
CMA/SPDM authentication on its own behalf. If the guest uses
memory encryption via TDX or SEV, key material established through
a Diffie-Hellman exchange between guest and device is invisible
to the host. Consequently using that key material for IDE encryption
protects device accesses from the guest against snooping by the host.

SPDM authentication consists of a sequence of exchanges, the first
being GET_VERSION. When a responder (=device) receives a GET_VERSION
request, it resets the connection and all internal state related to
that connection. (SPDM 1.2.1 margin no 185: "a Requester can issue
a GET_VERSION to a Responder to reset a connection at any time";
see also SPDM 1.1.0 margin no 161 for details.)

Thus, even though the host may have authenticated the device,
once it's passed through to a guest and the guest performs
authentication again, SPDM state on the device is reset.

I'll amend the patches so that the host refrains from performing
reauthentication as long as a device is passed through. The host
has no business mutating SPDM state on the device once ownership
has passed to the guest.

The first few SPDM exchanges are transmitted in the clear,
so the host can eavesdrop on the negotiated algorithms,
exchanged certificates and nonces. However the host cannot
successfully modify the exchanged data due to the man in the middle
protection afforded by SPDM: The challenge/response hash is
computed over the concatenation of the exchanged messages,
so modification of the messages by a man in the middle leads
to authentication failure.

Obviously the host can DoS guest access to the device by modifying
exchanged messages, but there are much simpler ways for it to
do that, say, by clearing Bus Master Enable or Memory Space Enable
bits in the Command Register. DoS attacks from the host against
the guest cannot be part of the threat model at this point.

Thanks,

Lukas
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 03:23:34PM +0100, Richard Weinberger wrote:
> On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > Any virtual device exposed to the guest that can transfer potentially
> > sensitive data needs to have some form of guest controlled encryption
> > applied. For disks this is easy with FDE like LUKS, for NICs this is
> > already best practice for services by using TLS. Other devices may not
> > have good existing options for applying encryption.
>
> I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
> but not transport. If an attacker can observe all IO you better
> consult a cryptographer.
> LUKS has no concept of session keys or such, so the same disk sector will
> always get encrypted with the very same key/iv.

Yes, you're right, all the FDE cipher modes are susceptible to
time based analysis of I/O, so very far from ideal. You'll get
protection for your historically written confidential data at the
time a VM host is first compromised, but if (as) they retain long
term access to the host, confidentiality is increasingly undermined
the longer they can observe the ongoing I/O.

With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 04:13:11PM +0100, Richard Weinberger wrote:
> On Thu, Jan 26, 2023 at 3:58 PM Dr. David Alan Gilbert
> <dgilbert@redhat.com> wrote:
> >
> > * Richard Weinberger (richard.weinberger@gmail.com) wrote:
> > > On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > Any virtual device exposed to the guest that can transfer potentially
> > > > sensitive data needs to have some form of guest controlled encryption
> > > > applied. For disks this is easy with FDE like LUKS, for NICs this is
> > > > already best practice for services by using TLS. Other devices may not
> > > > have good existing options for applying encryption.
> > >
> > > I disagree wrt. LUKS. The cryptography behind LUKS protects persistent data
> > > but not transport. If an attacker can observe all IO you better
> > > consult a cryptographer.
> > > LUKS has no concept of session keys or such, so the same disk sector will
> > > always get encrypted with the very same key/iv.
> >
> > Are you aware of anything that you'd use instead?
>
> Well, I'd think towards iSCSI over TLS to protect the IO transport.

That just moves the problem elsewhere though surely. The remote iSCSI
server still has to persist the VMs' data, and the cloud sevice provider
can observe any I/O before it hits the final hardware storage. So the
remote iSCSI server needs to apply a FDE like encryption scheme for
the exported iSCSI block device, and using a key only accessible to the
tenant that owns the VM. It still needs to solve the same problem of
having some kind of "generation ID" that can tweak the IV for each virtual
disk sector, to protect against time based analysis.

With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, 26 Jan 2023 14:15:05 +0100
Samuel Ortiz <sameo@rivosinc.com> wrote:

> On Thu, Jan 26, 2023 at 10:58:47AM +0000, Jonathan Cameron wrote:
> > On Thu, 26 Jan 2023 10:24:32 +0100
> > Samuel Ortiz <sameo@rivosinc.com> wrote:
> >
> > > Hi Lukas,
> > >
> > > On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
> > >
> > > > [cc += Jonathan Cameron, linux-pci]
> > > >
> > > > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > > > Great, so why not have hardware attestation also for your devices you
> > > > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > > > worry about anything in the guest.
> > > > >
> > > > > There were some talks at Plumbers where PCIe is working on adding that;
> > > > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > > > SPDM. I don't know much of the detail of those, just that they're far
> > > > > enough off that people aren't depending on them yet.
> > > >
> > > > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> > > >
> > > > https://github.com/l1k/linux/commits/doe
> > >
> > > Nice, thanks a lot for that.
> > >
> > >
> > >
> > > > The device authentication service afforded here is generic.
> > > > It is up to users and vendors to decide how to employ it,
> > > > be it for "confidential computing" or something else.
> > > >
> > > > Trusted root certificates to validate device certificates can be
> > > > installed into a kernel keyring using the familiar keyctl(1) utility,
> > > > but platform-specific roots of trust (such as a HSM) could be
> > > > supported as well.
> > > >
> > >
> > > This may have been discussed at LPC, but are there any plans to also
> > > support confidential computing flows where the host kernel is not part
> > > of the TCB and would not be trusted for validating the device cert chain
> > > nor for running the SPDM challenge?
> >
> > There are lots of possible models for this. One simple option if the assigned
> > VF supports it is a CMA instance per VF. That will let the guest
> > do full attestation including measurement of whether the device is
> > appropriately locked down so the hypervisor can't mess with
> > configuration that affects the guest (without a reset anyway and that
> > is guest visible).
>
> So the VF would be directly assigned to the guest, and the guest kernel
> would create a CMA instance for the VF, and do the SPDM authentication
> (based on a guest provided trusted root certificate). I think one
> security concern with that approach is assigning the VF to the
> (potentially confidential) guest address space without the guest being
> able to attest of the device trustworthiness first. That's what TDISP is
> aiming at fixing (establish a secure SPDM between the confidential guest
> and the device, lock the device from the guest, attest and then enable
> DMA).

Agreed, TDISP is more comprehensive, but also much more complex with
more moving parts that we don't really have yet.

Depending on your IOMMU design (+ related stuff) and interaction with
the secure guest, you might be able to block any rogue DMA until
after attestation / lock down checks even if the Hypervisor was letting
it through.

>
> > Whether anyone builds that option isn't yet clear
> > though. If they do, Lukas' work should work there as well as for the
> > host OS. (Note I'm not a security expert so may be missing something!)
> >
> > For extra fun, why should the device trust the host? Mutual authentication
> > fun (there are usecases where that matters)
> >
> > There are way more complex options supported in PCIe TDISP (Tee Device
> > security interface protocols). Anyone have an visibility of open solutions
> > that make use of that? May be too new.
>
> It's still a PCI ECN, so quite new indeed.
> FWIW the rust spdm crate [1] implements the TDISP state machine.

Cool. thanks for the reference.
>
> Cheers,
> Samuel.
>
> [1] https://github.com/jyao1/rust-spdm
> >
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> And this is a very special aspect of 'hardening' since it is about hardening a kernel
> under different threat model/assumptions.

I am not sure it's that special in that hardening IMHO is not a specific
threat model or a set of assumptions. IIUC it's just something that
helps reduce severity of vulnerabilities. Similarly, one can use the CC
hardware in a variety of ways I guess. And one way is just that -
hardening linux such that ability to corrupt guest memory does not
automatically escalate into guest code execution.

If you put it this way, you get to participate in a well understood
problem space instead of constantly saying "yes but CC is special". And
further, you will now talk about features as opposed to fixing bugs.
Which will stop annoying people who currently seem annoyed by the
implication that their code is buggy simply because it does not cache in
memory all data read from hardware. Finally, you then don't really need
to explain why e.g. DoS is not a problem but info leak is a problem - when
for many users it's actually the reverse - the reason is not that it's
not part of a threat model - which then makes you work hard to define
the threat model - but simply that CC hardware does not support this
kind of hardening.

--
MST
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 04:44:49PM +0100, Lukas Wunner wrote:
> Obviously the host can DoS guest access to the device by modifying
> exchanged messages, but there are much simpler ways for it to
> do that, say, by clearing Bus Master Enable or Memory Space Enable
> bits in the Command Register.

There's a single key per guest though, isn't it? Also used
for regular memory?


--
MST
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
> * Reshetova, Elena (elena.reshetova@intel.com) wrote:
> > > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > > Replying only to the not-so-far addressed points.
> > > >
> > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > Hi Greg,
> > >
> > > <...>
> > >
> > > > > > 3) All the tools are open-source and everyone can start using them right
> > > away
> > > > > even
> > > > > > without any special HW (readme has description of what is needed).
> > > > > > Tools and documentation is here:
> > > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > > >
> > > > > Again, as our documentation states, when you submit patches based on
> > > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > > it?
> > > >
> > > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > > we are submitting a fix that we have to list the way how it has been found.
> > > > We will fix this in the future submissions, but some bugs we have are found
> by
> > > > plain code audit, so 'human' is the tool.
> > >
> > > My problem with that statement is that by applying different threat
> > > model you "invent" bugs which didn't exist in a first place.
> > >
> > > For example, in this [1] latest submission, authors labeled correct
> > > behaviour as "bug".
> > >
> > > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > > alexander.shishkin@linux.intel.com/
> >
> > Hm.. Does everyone think that when kernel dies with unhandled page fault
> > (such as in that case) or detection of a KASAN out of bounds violation (as it is in
> some
> > other cases we already have fixes or investigating) it represents a correct
> behavior even if
> > you expect that all your pci HW devices are trusted? What about an error in
> two
> > consequent pci reads? What about just some failure that results in erroneous
> input?
>
> I'm not sure you'll get general agreement on those answers for all
> devices and situations; I think for most devices for non-CoCo
> situations, then people are generally OK with a misbehaving PCI device
> causing a kernel crash, since most people are running without IOMMU
> anyway, a misbehaving device can cause otherwise undetectable chaos.

Ok, if this is a consensus within the kernel community, then we can consider
the fixes strictly from the CoCo threat model point of view.

>
> I'd say:
> a) For CoCo, a guest (guaranteed) crash isn't a problem - CoCo doesn't
> guarantee forward progress or stop the hypervisor doing something
> truly stupid.

Yes, denial of service is out of scope but I would not pile all crashes as
'safe' automatically. Depending on the crash, it can be used as a
primitive to launch further attacks: privilege escalation, information
disclosure and corruption. It is especially true for memory corruption
issues.

> b) For CoCo, information disclosure, or corruption IS a problem

Agreed, but the path to this can incorporate a number of attack
primitives, as well as bug chaining. So, if the bug is detected, and
fix is easy, instead of thinking about possible implications and its
potential usage in exploit writing, safer to fix it.

>
> c) For non-CoCo some people might care about robustness of the kernel
> against a failing PCI device, but generally I think they worry about
> a fairly clean failure, even in the unexpected-hot unplug case.

Ok.

>
> d) It's not clear to me what 'trust' means in terms of CoCo for a PCIe
> device; if it's a device that attests OK and we trust it is the device
> it says it is, do we give it freedom or are we still wary?

I would say that attestation and established secure channel to an end
device means that we don’t have to employ additional measures to
secure data transfer, as well as we 'trust' a device at least to some degree to
keep our data protected (both from untrusted host and from other
CC guests). I don’t think there is anything else behind this concept.

Best Regards,
Elena
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 05:48:33PM +0000, Reshetova, Elena wrote:
>
> > * Reshetova, Elena (elena.reshetova@intel.com) wrote:
> > > > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > > > Replying only to the not-so-far addressed points.
> > > > >
> > > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > > Hi Greg,
> > > >
> > > > <...>
> > > >
> > > > > > > 3) All the tools are open-source and everyone can start using them right
> > > > away
> > > > > > even
> > > > > > > without any special HW (readme has description of what is needed).
> > > > > > > Tools and documentation is here:
> > > > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > > > >
> > > > > > Again, as our documentation states, when you submit patches based on
> > > > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > > > it?
> > > > >
> > > > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > > > we are submitting a fix that we have to list the way how it has been found.
> > > > > We will fix this in the future submissions, but some bugs we have are found
> > by
> > > > > plain code audit, so 'human' is the tool.
> > > >
> > > > My problem with that statement is that by applying different threat
> > > > model you "invent" bugs which didn't exist in a first place.
> > > >
> > > > For example, in this [1] latest submission, authors labeled correct
> > > > behaviour as "bug".
> > > >
> > > > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > > > alexander.shishkin@linux.intel.com/
> > >
> > > Hm.. Does everyone think that when kernel dies with unhandled page fault
> > > (such as in that case) or detection of a KASAN out of bounds violation (as it is in
> > some
> > > other cases we already have fixes or investigating) it represents a correct
> > behavior even if
> > > you expect that all your pci HW devices are trusted? What about an error in
> > two
> > > consequent pci reads? What about just some failure that results in erroneous
> > input?
> >
> > I'm not sure you'll get general agreement on those answers for all
> > devices and situations; I think for most devices for non-CoCo
> > situations, then people are generally OK with a misbehaving PCI device
> > causing a kernel crash, since most people are running without IOMMU
> > anyway, a misbehaving device can cause otherwise undetectable chaos.
>
> Ok, if this is a consensus within the kernel community, then we can consider
> the fixes strictly from the CoCo threat model point of view.
>
> >
> > I'd say:
> > a) For CoCo, a guest (guaranteed) crash isn't a problem - CoCo doesn't
> > guarantee forward progress or stop the hypervisor doing something
> > truly stupid.
>
> Yes, denial of service is out of scope but I would not pile all crashes as
> 'safe' automatically. Depending on the crash, it can be used as a
> primitive to launch further attacks: privilege escalation, information
> disclosure and corruption. It is especially true for memory corruption
> issues.
>
> > b) For CoCo, information disclosure, or corruption IS a problem
>
> Agreed, but the path to this can incorporate a number of attack
> primitives, as well as bug chaining. So, if the bug is detected, and
> fix is easy, instead of thinking about possible implications and its
> potential usage in exploit writing, safer to fix it.
>
> >
> > c) For non-CoCo some people might care about robustness of the kernel
> > against a failing PCI device, but generally I think they worry about
> > a fairly clean failure, even in the unexpected-hot unplug case.
>
> Ok.

With my other hat as a representative of hardware vendor (at least for
NIC part), who cares about quality of our devices, we don't want to hide
ANY crash related to our devices, especially if it is related to misbehaving
PCI HW logic. Any uncontrolled "robustness" hides real issues and makes
QA/customer support much harder.

Thanks
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
* Leon Romanovsky (leon@kernel.org) wrote:
> On Thu, Jan 26, 2023 at 05:48:33PM +0000, Reshetova, Elena wrote:
> >
> > > * Reshetova, Elena (elena.reshetova@intel.com) wrote:
> > > > > On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > > > > > Replying only to the not-so-far addressed points.
> > > > > >
> > > > > > > On Wed, Jan 25, 2023 at 12:28:13PM +0000, Reshetova, Elena wrote:
> > > > > > > > Hi Greg,
> > > > >
> > > > > <...>
> > > > >
> > > > > > > > 3) All the tools are open-source and everyone can start using them right
> > > > > away
> > > > > > > even
> > > > > > > > without any special HW (readme has description of what is needed).
> > > > > > > > Tools and documentation is here:
> > > > > > > > https://github.com/intel/ccc-linux-guest-hardening
> > > > > > >
> > > > > > > Again, as our documentation states, when you submit patches based on
> > > > > > > these tools, you HAVE TO document that. Otherwise we think you all are
> > > > > > > crazy and will get your patches rejected. You all know this, why ignore
> > > > > > > it?
> > > > > >
> > > > > > Sorry, I didn’t know that for every bug that is found in linux kernel when
> > > > > > we are submitting a fix that we have to list the way how it has been found.
> > > > > > We will fix this in the future submissions, but some bugs we have are found
> > > by
> > > > > > plain code audit, so 'human' is the tool.
> > > > >
> > > > > My problem with that statement is that by applying different threat
> > > > > model you "invent" bugs which didn't exist in a first place.
> > > > >
> > > > > For example, in this [1] latest submission, authors labeled correct
> > > > > behaviour as "bug".
> > > > >
> > > > > [1] https://lore.kernel.org/all/20230119170633.40944-1-
> > > > > alexander.shishkin@linux.intel.com/
> > > >
> > > > Hm.. Does everyone think that when kernel dies with unhandled page fault
> > > > (such as in that case) or detection of a KASAN out of bounds violation (as it is in
> > > some
> > > > other cases we already have fixes or investigating) it represents a correct
> > > behavior even if
> > > > you expect that all your pci HW devices are trusted? What about an error in
> > > two
> > > > consequent pci reads? What about just some failure that results in erroneous
> > > input?
> > >
> > > I'm not sure you'll get general agreement on those answers for all
> > > devices and situations; I think for most devices for non-CoCo
> > > situations, then people are generally OK with a misbehaving PCI device
> > > causing a kernel crash, since most people are running without IOMMU
> > > anyway, a misbehaving device can cause otherwise undetectable chaos.
> >
> > Ok, if this is a consensus within the kernel community, then we can consider
> > the fixes strictly from the CoCo threat model point of view.
> >
> > >
> > > I'd say:
> > > a) For CoCo, a guest (guaranteed) crash isn't a problem - CoCo doesn't
> > > guarantee forward progress or stop the hypervisor doing something
> > > truly stupid.
> >
> > Yes, denial of service is out of scope but I would not pile all crashes as
> > 'safe' automatically. Depending on the crash, it can be used as a
> > primitive to launch further attacks: privilege escalation, information
> > disclosure and corruption. It is especially true for memory corruption
> > issues.
> >
> > > b) For CoCo, information disclosure, or corruption IS a problem
> >
> > Agreed, but the path to this can incorporate a number of attack
> > primitives, as well as bug chaining. So, if the bug is detected, and
> > fix is easy, instead of thinking about possible implications and its
> > potential usage in exploit writing, safer to fix it.
> >
> > >
> > > c) For non-CoCo some people might care about robustness of the kernel
> > > against a failing PCI device, but generally I think they worry about
> > > a fairly clean failure, even in the unexpected-hot unplug case.
> >
> > Ok.
>
> With my other hat as a representative of hardware vendor (at least for
> NIC part), who cares about quality of our devices, we don't want to hide
> ANY crash related to our devices, especially if it is related to misbehaving
> PCI HW logic. Any uncontrolled "robustness" hides real issues and makes
> QA/customer support much harder.

Yeh if you're adding new code to be more careful, you want the code to
fail/log the problem, not hide it.
(Although heck, I suspect there are a million apparently working PCI
cards out there that break some spec somewhere).

Dave

> Thanks
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 11:25:21AM -0500, Michael S. Tsirkin wrote:
> On Thu, Jan 26, 2023 at 04:44:49PM +0100, Lukas Wunner wrote:
> > Obviously the host can DoS guest access to the device by modifying
> > exchanged messages, but there are much simpler ways for it to
> > do that, say, by clearing Bus Master Enable or Memory Space Enable
> > bits in the Command Register.
>
> There's a single key per guest though, isn't it? Also used
> for regular memory?

The current design is to have a global keyring (per kernel, i.e. per
guest). A device presents a certificate chain and the first certificate
in that chain needs to be signed by one of the certificates on the keyring.

This is completely independent from the key used for memory encryption.

A device can have up to 8 certificate chains (called "slots" in the
SPDM spec) and I've implemented it such that all slots are iterated
and validation is considered to be successful as soon as a slot with
a valid signature is found.

We can discuss having a per-device keyring if anyone thinks it makes
sense.

The PCISIG's idea seems to be that each vendor of PCIe cards publishes
a trusted root certificate and users would then have to keep all those
vendor certificates in their global keyring. This follows from the
last paragraph of PCIe r6.0.1 sec 6.31.3, which says "it is strongly
recommended that authentication requesters [i.e. the kernel] confirm
that the information provided in the Subject Alternative Name entry
[of the device's leaf certificate] is signed by the vendor indicated
by the Vendor ID."

The astute reader will notice that for this to work, the Vendor ID
must be included in the trusted root certificate in a machine-readable
way. Unfortunately the PCIe Base Spec fails to specify that.
So I don't know how to associate a trusted root certificate with a
Vendor ID.

I'll report this and several other gaps I've found in the spec to the
editor at the PCISIG so that they can be filled in a future revision.

Thanks,

Lukas
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 04:07:29PM +0000, Jonathan Cameron wrote:
> On Thu, 26 Jan 2023 14:15:05 +0100
> Samuel Ortiz <sameo@rivosinc.com> wrote:
>
> > On Thu, Jan 26, 2023 at 10:58:47AM +0000, Jonathan Cameron wrote:
> > > On Thu, 26 Jan 2023 10:24:32 +0100
> > > Samuel Ortiz <sameo@rivosinc.com> wrote:
> > >
> > > > Hi Lukas,
> > > >
> > > > On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
> > > >
> > > > > [cc += Jonathan Cameron, linux-pci]
> > > > >
> > > > > On Wed, Jan 25, 2023 at 02:57:40PM +0000, Dr. David Alan Gilbert wrote:
> > > > > > Greg Kroah-Hartman (gregkh@linuxfoundation.org) wrote:
> > > > > > > Great, so why not have hardware attestation also for your devices you
> > > > > > > wish to talk to? Why not use that as well? Then you don't have to
> > > > > > > worry about anything in the guest.
> > > > > >
> > > > > > There were some talks at Plumbers where PCIe is working on adding that;
> > > > > > it's not there yet though. I think that's PCIe 'Integrity and Data
> > > > > > Encryption' (IDE - sigh), and PCIe 'Security Prtocol and Data Model' -
> > > > > > SPDM. I don't know much of the detail of those, just that they're far
> > > > > > enough off that people aren't depending on them yet.
> > > > >
> > > > > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> > > > >
> > > > > https://github.com/l1k/linux/commits/doe
> > > >
> > > > Nice, thanks a lot for that.
> > > >
> > > >
> > > >
> > > > > The device authentication service afforded here is generic.
> > > > > It is up to users and vendors to decide how to employ it,
> > > > > be it for "confidential computing" or something else.
> > > > >
> > > > > Trusted root certificates to validate device certificates can be
> > > > > installed into a kernel keyring using the familiar keyctl(1) utility,
> > > > > but platform-specific roots of trust (such as a HSM) could be
> > > > > supported as well.
> > > > >
> > > >
> > > > This may have been discussed at LPC, but are there any plans to also
> > > > support confidential computing flows where the host kernel is not part
> > > > of the TCB and would not be trusted for validating the device cert chain
> > > > nor for running the SPDM challenge?
> > >
> > > There are lots of possible models for this. One simple option if the assigned
> > > VF supports it is a CMA instance per VF. That will let the guest
> > > do full attestation including measurement of whether the device is
> > > appropriately locked down so the hypervisor can't mess with
> > > configuration that affects the guest (without a reset anyway and that
> > > is guest visible).
> >
> > So the VF would be directly assigned to the guest, and the guest kernel
> > would create a CMA instance for the VF, and do the SPDM authentication
> > (based on a guest provided trusted root certificate). I think one
> > security concern with that approach is assigning the VF to the
> > (potentially confidential) guest address space without the guest being
> > able to attest of the device trustworthiness first. That's what TDISP is
> > aiming at fixing (establish a secure SPDM between the confidential guest
> > and the device, lock the device from the guest, attest and then enable
> > DMA).
>
> Agreed, TDISP is more comprehensive, but also much more complex with
> more moving parts that we don't really have yet.
>
> Depending on your IOMMU design (+ related stuff) and interaction with
> the secure guest, you might be able to block any rogue DMA until
> after attestation / lock down checks even if the Hypervisor was letting
> it through.

Provided that the guest or, in the TDX and AP-TEE cases, the TSM have
protected access to the IOMMU, yes. But then the implementation becomes
platform specific.

Cheers,
Samuel.
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 04:44:49PM +0100, Lukas Wunner wrote:
> On Thu, Jan 26, 2023 at 10:24:32AM +0100, Samuel Ortiz wrote:
> > On Wed, Jan 25, 2023 at 11:03 PM Lukas Wunner <lukas@wunner.de> wrote:
> > > CMA/SPDM (PCIe r6.0 sec 6.31) is in active development on this branch:
> > >
> > > https://github.com/l1k/linux/commits/doe
> > >
> > > The device authentication service afforded here is generic.
> > > It is up to users and vendors to decide how to employ it,
> > > be it for "confidential computing" or something else.
> > >
> > > Trusted root certificates to validate device certificates can be
> > > installed into a kernel keyring using the familiar keyctl(1) utility,
> > > but platform-specific roots of trust (such as a HSM) could be
> > > supported as well.
> >
> > This may have been discussed at LPC, but are there any plans to also
> > support confidential computing flows where the host kernel is not part
> > of the TCB and would not be trusted for validating the device cert chain
> > nor for running the SPDM challenge?
>
> As long as a device is passed through to a guest, the guest owns
> that device.

I agree. On a SRIOV setup, the host typically owns the PF and assigns
VFs to the guests. Devices must be enlightened to guarantee that once
one of their VFs/interfaces is passed to a trusted VM, it can no longer
be modified by anything untrusted (e.g. the hypervisor).

> It is the guest's prerogative and duty to perform
> CMA/SPDM authentication on its own behalf. If the guest uses
> memory encryption via TDX or SEV, key material established through
> a Diffie-Hellman exchange between guest and device is invisible
> to the host. Consequently using that key material for IDE encryption
> protects device accesses from the guest against snooping by the host.

On confidential computing platforms where a security manager (e.g.
Intel TDX module) manages the confidential guests, the IDE key
management and stream settings would be handled by this manager. In
other words, the SPDM requester would not be a Linux kernel.
FWIW, Intel recently published an interesting description of TEE-IO
enabling with TDX [1].

> SPDM authentication consists of a sequence of exchanges, the first
> being GET_VERSION. When a responder (=device) receives a GET_VERSION
> request, it resets the connection and all internal state related to
> that connection. (SPDM 1.2.1 margin no 185: "a Requester can issue
> a GET_VERSION to a Responder to reset a connection at any time";
> see also SPDM 1.1.0 margin no 161 for details.)
>
> Thus, even though the host may have authenticated the device,
> once it's passed through to a guest and the guest performs
> authentication again, SPDM state on the device is reset.
>
> I'll amend the patches so that the host refrains from performing
> reauthentication as long as a device is passed through. The host
> has no business mutating SPDM state on the device once ownership
> has passed to the guest.
>
> The first few SPDM exchanges are transmitted in the clear,
> so the host can eavesdrop on the negotiated algorithms,
> exchanged certificates and nonces. However the host cannot
> successfully modify the exchanged data due to the man in the middle
> protection afforded by SPDM: The challenge/response hash is
> computed over the concatenation of the exchanged messages,
> so modification of the messages by a man in the middle leads
> to authentication failure.

Right, I was not concerned by the challenge messages integrity but by
trusting the host with verifying the response and validating the device
cert chains.

> Obviously the host can DoS guest access to the device by modifying
> exchanged messages, but there are much simpler ways for it to
> do that, say, by clearing Bus Master Enable or Memory Space Enable
> bits in the Command Register. DoS attacks from the host against
> the guest cannot be part of the threat model at this point.

Yes, the host can DoS the guest at anytime it wants and in multiple
ways. It's definitely out of the confidential computing thread model at
least.

Cheers,
Samuel.

[1] https://cdrdv2-public.intel.com/742542/software-enabling-for-tdx-tee-io-fixed.pdf
RE: Linux guest kernel threat model for Confidential Computing [ In reply to ]
> On Wed, Jan 25, 2023 at 03:29:07PM +0000, Reshetova, Elena wrote:
> > And this is a very special aspect of 'hardening' since it is about hardening a
> kernel
> > under different threat model/assumptions.
>
> I am not sure it's that special in that hardening IMHO is not a specific
> threat model or a set of assumptions. IIUC it's just something that
> helps reduce severity of vulnerabilities. Similarly, one can use the CC
> hardware in a variety of ways I guess. And one way is just that -
> hardening linux such that ability to corrupt guest memory does not
> automatically escalate into guest code execution.

I am not sure if I fully follow you on this. I do agree that it is in principle
the same 'hardening' that we have been doing in Linux for decades just
applied to a new attack surface, host <-> guest, vs userspace <->kernel.
Interfaces have changed, but the types of vulnerabilities, etc are the same.
The attacker model is somewhat different because we have
different expectations on what host/hypervisor should be able to do
to the guest (following business reasons and use-cases), versus what we
expect normal userspace being able to "do" towards kernel. The host and
hypervisor still has a lot of control over the guest (ability to start/stop it,
manage its resources, etc). But the reasons behind this doesn’t come
from the fact that security CoCo HW not being able to support this stricter
security model (it cannot now indeed, but this is a design decision), but
from the fact that it is important for Cloud service providers to retain that
level of control over their infrastructure.

>
> If you put it this way, you get to participate in a well understood
> problem space instead of constantly saying "yes but CC is special". And
> further, you will now talk about features as opposed to fixing bugs.
> Which will stop annoying people who currently seem annoyed by the
> implication that their code is buggy simply because it does not cache in
> memory all data read from hardware. Finally, you then don't really need
> to explain why e.g. DoS is not a problem but info leak is a problem - when
> for many users it's actually the reverse - the reason is not that it's
> not part of a threat model - which then makes you work hard to define
> the threat model - but simply that CC hardware does not support this
> kind of hardening.

But this won't be correct statement, because it is not limitation of HW, but the
threat and business model that Confidential Computing exists in. I am not
aware of a single cloud provider who would be willing to use the HW that
takes the full control of their infrastructure and running confidential guests,
leaving them with no mechanisms to control the load balancing, enforce
resource usage, etc. So, given that nobody needs/willing to use such HW,
such HW simply doesn’t exist.

So, I would still say that the model we operate in CoCo usecases is somewhat
special, but I do agree that given that we list a couple of these special assumptions
(over which ones we have no control or ability to influence, none of us are business
people), then the rest becomes just careful enumeration of attack surface interfaces
and break up of potential mitigations.

Best Regards,
Elena.
Re: Linux guest kernel threat model for Confidential Computing [ In reply to ]
On Thu, Jan 26, 2023 at 04:13:11PM +0100, Richard Weinberger wrote:
> On Thu, Jan 26, 2023 at 3:58 PM Dr. David Alan Gilbert
> <dgilbert@redhat.com> wrote:
> >
> > * Richard Weinberger (richard.weinberger@gmail.com) wrote:
> > > On Wed, Jan 25, 2023 at 3:22 PM Daniel P. Berrang? <berrange@redhat.com> wrote:
> > Are you aware of anything that you'd use instead?
>
> Well, I'd think towards iSCSI over TLS to protect the IO transport.

In the context of confidential computing this makes only sense if the
scsi target is part of the trusted base, which means it needs to be
attested and protected against outside attacks. Currently all CoCo
implementations I know of treat disk storage as untrusted.

Besides that the same problems exist with a VMs encrypted memory. The
hardware does not guarantee that the HV can not fiddle with your private
memory, it only guarantees that you can detect such fiddling and that
the private data is encrypted. The HV can also still trace memory access
patterns of confidential guests by setting the right permissions in the
nested page table.

So storage and memory of a CoCo VM have in common that the transport is
not secure, but there are measures to detect if someone fiddles with
your data on the transport or at rest, for memory implemented in
hardware, and for storage in software by using dm-crypt together with
dm-verity or dm-integrity.

Regards,

Joerg

1 2 3 4 5  View All