Mailing List Archive

On Wed, Jan 5, 2011 at 4:01 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
>
> * Crypto API interface for user-space (hash + skcipher)

Is there really any point to this? And can we get more explanation of
what the interface is, and who would use it?

If you need crypto in user space, it's almost invariably better done
in user space. If the CPU can do crypto on its own, and doesn't expose
those instructions to user space, it's just a stupid CPU - and the
user/kernel transfer is likely going to make it slower than a pure
software approach for any but the biggest transfers.

And if the crypto engine is off-chip, the sw version is going to be
faster anyway except for possible async versions that are hard to
interface to user space.

So I really need more convincing about the whole user-space interface.
Adding new interfaces willy-nilly isn't a good idea. They need damn
good reasons.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 10:05:46AM -0800, Linus Torvalds wrote:
>
> Is there really any point to this? And can we get more explanation of
> what the interface is, and who would use it?

I think you've answered it yourself in the third paragraph :)

> If you need crypto in user space, it's almost invariably better done
> in user space. If the CPU can do crypto on its own, and doesn't expose
> those instructions to user space, it's just a stupid CPU - and the
> user/kernel transfer is likely going to make it slower than a pure
> software approach for any but the biggest transfers.

I agree completely.

> And if the crypto engine is off-chip, the sw version is going to be
> faster anyway except for possible async versions that are hard to
> interface to user space.
>
> So I really need more convincing about the whole user-space interface.
> Adding new interfaces willy-nilly isn't a good idea. They need damn
> good reasons.

Right. This purpose of this interface is to access the async
hardware crypto drivers that we have added over the past years.

For a modern x86-64 CPU it isn't interesting at all. It's mainly
for other architectures where the CPU may not be able to keep up
with say 10Gb/s IPsec traffic and the encryption and/or hashing
must be offloaded.

This is also why only hash and skcipher are supported as they
are the main algorithm types supported by teh current async
drivers in the kernel.

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 6, 2011 at 1:16 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
> On Thu, Jan 06, 2011 at 10:05:46AM -0800, Linus Torvalds wrote:
>>
>> Is there really any point to this? And can we get more explanation of
>> what the interface is, and who would use it?
>
> I think you've answered it yourself in the third paragraph :)

No I didn't.

What part of "can we get more explanation of what the interface is" is unclear?

Explanations of interface. Code. Who uses it? What are the actual
performance benefits on real code?

Quite frankly, asynchronous external devices using DMA or similar are
seldom real performance improvements. The bus and cache traffic tends
to overwhelm any other advantage, and commonly the result is (a) lower
performance with (b) better-looking profiles.

But "better-looking profiles" isn't actually a real advantage.

And I really do want to hear about new kernel interfaces. What _are_
the interfaces, and what are the advantages to them.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 01:23:19PM -0800, Linus Torvalds wrote:
>
> Explanations of interface. Code. Who uses it? What are the actual
> performance benefits on real code?

You snipped out the bit in my reply where I expanded on it:

: Right. This purpose of this interface is to access the async
: hardware crypto drivers that we have added over the past years.
:
: For a modern x86-64 CPU it isn't interesting at all. It's mainly
: for other architectures where the CPU may not be able to keep up
: with say 10Gb/s IPsec traffic and the encryption and/or hashing
: must be offloaded.
:
: This is also why only hash and skcipher are supported as they
: are the main algorithm types supported by teh current async
: drivers in the kernel.

The interface is meant to be used by those whose CPU either cannot
provide adequate performance for encryption/hashing, or where they
need to preverse the CPU power for other tasks. This is most
likely to be used on a non-x86 architecture, as most of our async
crypto drivers are for non-x86 architectures.

Users can be anything performing bulk encryptiong/hashing in user-
space, e.g., SSL (although there are plans to implement SSL in
the kernel as well), SSH, etc.

Here is the original cover email for the patches:

: On Tue, Sep 07, 2010 at 04:42:13PM +0800, Herbert Xu wrote:
: >
: > This is what I am proposing for the Crypto API user-interface.
: >
: > Note that this is the interface for operations. There will be
: > a separate interface (most likely netlink) for configuring crypto
: > algorithms, e.g., picking a specific AES implementation as the
: > system default.
:
: OK I've gone ahead and implemented the user-space API for hashes
: and ciphers.
:
: To recap this interface is designed to allow user-space programs
: to access hardware cryptographic accelerators that we have added
: to the kernel.
:
: The intended usage scenario is where a large amount of data needs
: to be processed where the benefits offered by hardware acceleration
: that is normally unavailable in user-space (as opposed to ones
: such as the Intel AES instruction which may be used directly from
: user-space) outweigh the overhead of going through the kernel.
:
: In order to further minimise the overhead in these cases, this
: interface offers the option of avoiding copying data between
: user-space and the kernel where possible and appropriate. For
: ciphers this means the use of the splice(2) interface instead of
: sendmsg(2)
:
: Here is a sample hash program (note that these only illustrate
: what the interface looks like and are not meant to be good examples
: of coding :)
:
: int main(void)
: {
: int opfd;
: int tfmfd;
: struct sockaddr_alg sa = {
: .salg_family = AF_ALG,
: .salg_type = "hash",
: .salg_name = "sha1"
: };
: char buf[20];
: int i;
:
: tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
:
: bind(tfmfd, (struct sockaddr *)&sa, sizeof(sa));
:
: opfd = accept(tfmfd, NULL, 0);
:
: write(opfd, "abc", 3);
: read(opfd, buf, 20);
:
: for (i = 0; i < 20; i++) {
: printf("%02x", (unsigned char)buf[i]);
: }
: printf("\n");
:
: close(opfd);
: close(tfmfd);
:
: return 0;
: }
:
: And here is one for ciphers:
:
: int main(void)
: {
: int opfd;
: int tfmfd;
: struct sockaddr_alg sa = {
: .salg_family = AF_ALG,
: .salg_type = "skcipher",
: .salg_name = "cbc(aes)"
: };
: struct msghdr msg = {};
: struct cmsghdr *cmsg;
: char cbuf[CMSG_SPACE(4) + CMSG_SPACE(20)];
: char buf[16];
: struct af_alg_iv *iv;
: struct iovec iov;
: int i;
:
: tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
:
: bind(tfmfd, (struct sockaddr *)&sa, sizeof(sa));
:
: setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY,
: "\x06\xa9\x21\x40\x36\xb8\xa1\x5b"
: "\x51\x2e\x03\xd5\x34\x12\x00\x06", 16);
:
: opfd = accept(tfmfd, NULL, 0);
:
: msg.msg_control = cbuf;
: msg.msg_controllen = sizeof(cbuf);
:
: cmsg = CMSG_FIRSTHDR(&msg);
: cmsg->cmsg_level = SOL_ALG;
: cmsg->cmsg_type = ALG_SET_OP;
: cmsg->cmsg_len = CMSG_LEN(4);
: *(__u32 *)CMSG_DATA(cmsg) = ALG_OP_ENCRYPT;
:
: cmsg = CMSG_NXTHDR(&msg, cmsg);
: cmsg->cmsg_level = SOL_ALG;
: cmsg->cmsg_type = ALG_SET_IV;
: cmsg->cmsg_len = CMSG_LEN(20);
: iv = (void *)CMSG_DATA(cmsg);
: iv->ivlen = 16;
: memcpy(iv->iv, "\x3d\xaf\xba\x42\x9d\x9e\xb4\x30"
: "\xb4\x22\xda\x80\x2c\x9f\xac\x41", 16);
:
: iov.iov_base = "Single block msg";
: iov.iov_len = 16;
:
: msg.msg_iov = &iov;
: msg.msg_iovlen = 1;
:
: sendmsg(opfd, &msg, 0);
: read(opfd, buf, 16);
:
: for (i = 0; i < 16; i++) {
: printf("%02x", (unsigned char)buf[i]);
: }
: printf("\n");
:
: close(opfd);
: close(tfmfd);
:
: return 0;
: }

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On 01/06/2011 04:16 PM, Herbert Xu wrote:

> This is also why only hash and skcipher are supported as they
> are the main algorithm types supported by teh current async
> drivers in the kernel.

Are there any chances AEAD will be supported? Is the API extendable to
allow that?

If I remember correctly, the original patch was simply a port of FreeBSD
/dev/crypto, which doesn't support AEAD.

--
Regards,
Pavel Roskin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 04:46:02PM -0500, Pavel Roskin wrote:
>
> Are there any chances AEAD will be supported? Is the API extendable to
> allow that?

Yes we can support AEAD. However, as there weren't any existing
user-space users that could benefit immediately, I have not added
it at this point in time.

When combined-mode support in async hardware becomes more common
this could change of course.

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 6, 2011 at 1:39 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
> On Thu, Jan 06, 2011 at 01:23:19PM -0800, Linus Torvalds wrote:
>>
>> Explanations of interface. Code. Who uses it? What are the actual
>> performance benefits on real code?
>
> You snipped out the bit in my reply where I expanded on it:

You didn't expand on it AT ALL.

You just mentioned "the interface". I haven't seen WHAT THAT INTERFACE IS!

How hard is that to understand?

> Here is the original cover email for the patches:

Ok, this is more like it. This is roughly what I wanted to see:

> : Here is a sample hash program (note that these only illustrate
> : what the interface looks like and are not meant to be good examples
> : of coding :)

But I'm still missing the part where you show that there is any actual
use case that makes sense, and that actually improves performance.
Maybe it's been posted somewhere else, but the thing is, you're asking
_me_ to pull, and as a result you need to convince _me_ that this is a
good idea. So if it's been posted/discussed extensively elsewhere,
please point to those discussions.

I really don't like adding interfaces that don't have hard uses
associated with them. We've done it in the past, and it tends to be a
morass and a bad idea. That's been true even when the idea has been my
own, and thus obviously genius-level and clearly the RightThing(tm),
like "splice()". And it's why I push back on new interfaces when I see
them.

Btw, it doesn't have to be about performance per se. Does this allow
people to use keys without actually _seeing_ those keys? Your example
implies that that is not the case, but that's actually one of the few
reasons to actually support a kernel crypto interface - the ability to
have private personal keys around, but not having to actually let
possibly untrusted programs see them.

For example of why something like that matters, I can well see myself
using some program to encrypt things. But maybe I don't trust that
program enough to give it my actual private keys. In that case, kernel
support is a real feature.

But in your example, it looks like you just give it the key. Which to
me means that you're totally missing one of the major reasons for
having a separate protection domain.

And that makes me think that the interface is bad. And that's why it's
a big change to go from "internal kernel crypto interface" to "actual
user-space interface to the kernel crypto engine". The first one can
be fixed. The second one cannot.

So I'm not necessarily hung up on "performance", but I am hung up on
"there needs to be a point", and "the interface needs to be
-correct-". Performance would be one such point. Not just 'the
hardware is there'. I know the hardware exists, but I'm not at all
convinced that DMA with all the cacheflushing will ever actully be
faster the the CPU. And if it can, I want to hear about the real-world
situation where it actually is used.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 02:13:17PM -0800, Linus Torvalds wrote:
>
> But I'm still missing the part where you show that there is any actual
> use case that makes sense, and that actually improves performance.
> Maybe it's been posted somewhere else, but the thing is, you're asking
> _me_ to pull, and as a result you need to convince _me_ that this is a
> good idea. So if it's been posted/discussed extensively elsewhere,
> please point to those discussions.

The main use-case is bulk encryption/hashing in user-space. For
example, on Sparc Niagara2 you need to use SPU (Stream Processing
Unit) in order to do crypto at 10Gb/s over the network. Because
of the hardware design, it is difficult to make use of the SPU
directly in user-space, unless you dedicate the SPU to one single
thread/user.

That is why we need a crypto driver in the kernel as well as a
user-space interface exporting it, to make available a single
piece of hardware resource so that it may be used by multiple
users in user-space.

The same applies to most of the drivers in drivers/crypto, e.g.,
omap provides ARM crypto acceleration providing crypto throughput
that simply cannot be achieved by the CPU itself.

The actual encryption in user-space would be anything that is
done in bulk, such as SSH or SSL. In fact, the intention is
to implement it just once in a library such as libssl where it
could then be used automatically by all appliations.

> But in your example, it looks like you just give it the key. Which to
> me means that you're totally missing one of the major reasons for
> having a separate protection domain.

Providing such separation is not the primary objective of this API,
which is providing user-space access to off-chip crypto hardware.

However, this is something that has been considered and can be
easily added. The idea is to first add symmetric key storage
capability to the existing in-kernel keyring API. Once that is
done we can easily add a new setsockopt to the crypto user-interface
that retrieves the key from the kernel keyring instead of getting
it directly from user-space.

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 7 Jan 2011 09:30:42 +1100

> On Thu, Jan 06, 2011 at 02:13:17PM -0800, Linus Torvalds wrote:
>>
>> But I'm still missing the part where you show that there is any actual
>> use case that makes sense, and that actually improves performance.
>> Maybe it's been posted somewhere else, but the thing is, you're asking
>> _me_ to pull, and as a result you need to convince _me_ that this is a
>> good idea. So if it's been posted/discussed extensively elsewhere,
>> please point to those discussions.
>
> The main use-case is bulk encryption/hashing in user-space. For
> example, on Sparc Niagara2 you need to use SPU (Stream Processing
> Unit) in order to do crypto at 10Gb/s over the network. Because
> of the hardware design, it is difficult to make use of the SPU
> directly in user-space, unless you dedicate the SPU to one single
> thread/user.

The N2 SPU is not accessible at all from userspace.

> However, this is something that has been considered and can be
> easily added. The idea is to first add symmetric key storage
> capability to the existing in-kernel keyring API. Once that is
> done we can easily add a new setsockopt to the crypto user-interface
> that retrieves the key from the kernel keyring instead of getting
> it directly from user-space.

That indeed would be real interesting.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 6, 2011 at 2:30 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
>
> The main use-case is bulk encryption/hashing in user-space.  For
> example, on Sparc Niagara2 you need to use SPU (Stream Processing
> Unit) in order to do crypto at 10Gb/s over the network.

Umm. But doesn't that require that the data then be sent to the network?

Why would a user-space -> crypto engine -> user space -> network chip
thing ever be good enough? Niagara is so slow that the whole bounce
thing will totally negate all the SPU advantages.

Your interface doesn't seem to support the use case that you actually
want, which is to avoid the bouncing back and forth between user space
buffers.

And if you bounce back and forth, I bet you can't get that 10Gb/s anyway.

Can you do the "bypass directly to the TCP stream" with the interface
you added? It isn't at all obvious how it would work.

So let me repeat ONE MORE TIME:

- I understand that your interface can use the hw that exists

- but I still want real-world use cases to show that it actually
works and makes sense in practice.

Don't give me "we could use the SPU" crap. Give me "this program
actually uses the SPU and gets better performance thanks to it, and
here are the numbers".

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 02:43:35PM -0800, Linus Torvalds wrote:
>
> Can you do the "bypass directly to the TCP stream" with the interface
> you added? It isn't at all obvious how it would work.

Yes it can. The interface allows zero-copy in both directions
using the splice interface. Here is a sample program demonstrating
zero-copy in-place encryption. It doesn't send the result over TCP
but I'm sure you can imagine what that would look like.

Note that the final read(2) looks like it copies, but it doesn't.
The read(2) will setup SG lists using the user-space address and
place the encryption result in there directly. In this case as
the source/destination addresses are identical, it performs in-place
encryption.

#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <linux/splice.h>
#include <linux/types.h>
#include <linux/af_alg.h>

static char buf[4096] __attribute__((__aligned__(4096)));

int main(void)
{
int opfd;
int tfmfd;
struct sockaddr_alg sa = {
.salg_family = AF_ALG,
.salg_type = "skcipher",
.salg_name = "cbc(aes)"
};
struct msghdr msg = {};
struct cmsghdr *cmsg;
char cbuf[CMSG_SPACE(4) + CMSG_SPACE(20)];
struct aes_iv {
__u32 len;
__u8 iv[16];
} *iv;
struct iovec iov;
int i;
int pipes[2];

pipe(pipes);

tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);

bind(tfmfd, (struct sockaddr *)&sa, sizeof(sa));

setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY,
"\x06\xa9\x21\x40\x36\xb8\xa1\x5b"
"\x51\x2e\x03\xd5\x34\x12\x00\x06", 16);

opfd = accept(tfmfd, NULL, 0);

msg.msg_control = cbuf;
msg.msg_controllen = sizeof(cbuf);

cmsg = CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_OP;
cmsg->cmsg_len = CMSG_LEN(4);
*(__u32 *)CMSG_DATA(cmsg) = ALG_OP_ENCRYPT;

cmsg = CMSG_NXTHDR(&msg, cmsg);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_IV;
cmsg->cmsg_len = CMSG_LEN(20);
iv = (void *)CMSG_DATA(cmsg);
iv->len = 16;
memcpy(iv->iv, "\x3d\xaf\xba\x42\x9d\x9e\xb4\x30"
"\xb4\x22\xda\x80\x2c\x9f\xac\x41", 16);

memcpy(buf, "Single block msg", 16);

iov.iov_base = buf;
iov.iov_len = 4096;

msg.msg_iovlen = 0;
msg.msg_flags = MSG_MORE;

sendmsg(opfd, &msg, 0);
vmsplice(pipes[1], &iov, 1, SPLICE_F_GIFT);
splice(pipes[0], NULL, opfd, NULL, 16, 0);
read(opfd, buf, 16);

for (i = 0; i < 16; i++) {
printf("%02x", (unsigned char)buf[i]);
}
printf("\n");

close(opfd);
close(tfmfd);

return 0;

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thursday 06 January 2011 23:46:02 Pavel Roskin wrote:
> On 01/06/2011 04:16 PM, Herbert Xu wrote:
> > This is also why only hash and skcipher are supported as they
> > are the main algorithm types supported by teh current async
> > drivers in the kernel.
>
> Are there any chances AEAD will be supported? Is the API extendable to
> allow that?
>
> If I remember correctly, the original patch was simply a port of FreeBSD
> /dev/crypto, which doesn't support AEAD.

If it's meant to work like the FreeBSD /dev/crypto, then maybe this[1] paper
is worth mentionning, which includes some performance numbers. There was a
single thing that drew my attention, from "Drawbacks and problems [of
userspace crypto]": "Lots of scary code-duplication".

[1] http://www.paeps.cx/pubs/crypto_acceleration/slides.pdf

--
Mihai Donțu
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 6, 2011 at 2:53 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
> On Thu, Jan 06, 2011 at 02:43:35PM -0800, Linus Torvalds wrote:
>>
>> Can you do the "bypass directly to the TCP stream" with the interface
>> you added? It isn't at all obvious how it would work.
>
> Yes it can.  The interface allows zero-copy in both directions
> using the splice interface.  Here is a sample program demonstrating
> zero-copy in-place encryption.  It doesn't send the result over TCP
> but I'm sure you can imagine what that would look like.

Ok. So can we actually get numbers for this?

Put another way: I really really REALLY don't want to merge new
user-space interfaces that don't actually work in reality. But if this
allows direct encryption to a network interface, and it actually is
able to saturate 10Gb on niagara (unlike a user-mode encryption thing,
I assume, since those things are dog slow), then that would certainly
be a good real-life test.

But I really don't want to merge it unless it has had at least
real-life testing of actually doing better than regular sw user-space
encryption.

I realize that on PC's, it's unlikely to ever help. So I'm not asking
for "show me how this helps on my hardware". But I do want to get some
case on _some_ actual hardware where it works on a real load.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 03:25:32PM -0800, Linus Torvalds wrote:
>
> Ok. So can we actually get numbers for this?

As you alluded to, we need real non-x86 hardware to get some
proper numbers. Unfortunately I'm currently a continent away
from my async hardware so all I can give you are software
numbers.

Anyway, here is a test program that does a set number (1024 * 1024)
of 4096-byte (total 4GB) encryptions with libssl and kernel crypto.
The result on my Core 2 (in a KVM VM FWIW) is:

lenny0:~# time ./test_crypto > /dev/null

real 0m30.340s
user 0m0.172s
sys 0m30.166s
lenny0:~# time ./test_crypto software > /dev/null

real 0m34.555s
user 0m34.550s
sys 0m0.004s
lenny0:~#

I'm not claiming these numbers make any sense but this is just
a ballpark figure.

Dave, can you run this program on your N2 and see what it shows?

#include <fcntl.h>
#include <openssl/aes.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <linux/types.h>

#define AF_ALG 38
#define SOL_ALG 279

#define SPLICE_F_GIFT (0x08) /* pages passed in are a gift */

struct sockaddr_alg {
__u16 salg_family;
__u8 salg_type[14];
__u32 salg_feat;
__u32 salg_mask;
__u8 salg_name[64];
};

struct af_alg_iv {
__u32 ivlen;
__u8 iv[0];
};

/* Socket options */
#define ALG_SET_KEY 1
#define ALG_SET_IV 2
#define ALG_SET_OP 3

/* Operations */
#define ALG_OP_DECRYPT 0
#define ALG_OP_ENCRYPT 1

static char buf[4096] __attribute__((__aligned__(4096)));

static void crypt_ssl(const char *key, char *iv, int i)
{
AES_KEY akey;

AES_set_encrypt_key(key, 128, &akey);

while (i--)
AES_cbc_encrypt(buf, buf, 4096, &akey, iv, 1);
}

static void crypt_kernel(const char *key, char *oiv, int i)
{
int opfd;
int tfmfd;
struct sockaddr_alg sa = {
.salg_family = AF_ALG,
.salg_type = "skcipher",
.salg_name = "cbc(aes)"
};
struct msghdr msg = {};
struct cmsghdr *cmsg;
char cbuf[CMSG_SPACE(4) + CMSG_SPACE(20)] = {};
struct aes_iv {
__u32 len;
__u8 iv[16];
} *iv;
struct iovec iov;
int pipes[2];

pipe(pipes);

tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);

bind(tfmfd, (struct sockaddr *)&sa, sizeof(sa));

setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY, key, 16);

opfd = accept(tfmfd, NULL, 0);

msg.msg_control = cbuf;
msg.msg_controllen = sizeof(cbuf);

cmsg = CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_OP;
cmsg->cmsg_len = CMSG_LEN(4);
*(__u32 *)CMSG_DATA(cmsg) = ALG_OP_ENCRYPT;

cmsg = CMSG_NXTHDR(&msg, cmsg);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_IV;
cmsg->cmsg_len = CMSG_LEN(20);
iv = (void *)CMSG_DATA(cmsg);
iv->len = 16;
memcpy(iv->iv, oiv, 16);

iov.iov_base = buf;
iov.iov_len = 4096;

msg.msg_iovlen = 0;
msg.msg_flags = MSG_MORE;

while (i--) {
sendmsg(opfd, &msg, 0);
vmsplice(pipes[1], &iov, 1, SPLICE_F_GIFT);
splice(pipes[0], NULL, opfd, NULL, 4096, 0);
read(opfd, buf, 4096);
}

close(opfd);
close(tfmfd);
close(pipes[0]);
close(pipes[1]);
}

int main(int argc, char **argv)
{
int i;

const char key[16] =
"\x06\xa9\x21\x40\x36\xb8\xa1\x5b"
"\x51\x2e\x03\xd5\x34\x12\x00\x06";
char iv[16] =
"\x3d\xaf\xba\x42\x9d\x9e\xb4\x30"
"\xb4\x22\xda\x80\x2c\x9f\xac\x41";

memcpy(buf, "Single block msg", 16);

if (argc > 1)
crypt_ssl(key, iv, 1024 * 1024);
else
crypt_kernel(key, iv, 1024 * 1024);

for (i = 0; i < 4096; i++) {
printf("%02x", (unsigned char)buf[i]);
}
printf("\n");

return 0;
}

Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoting Herbert Xu <herbert@gondor.apana.org.au>:

> setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY,
> "\x06\xa9\x21\x40\x36\xb8\xa1\x5b"
> "\x51\x2e\x03\xd5\x34\x12\x00\x06", 16);

By the way, is it a good idea to use setsockopt() this way? Some keys
may be large. And if AEAD is supported, will it use setsockopt() for
the unencrypted authenticated data?

Maybe some variation of write() or sendto() would be more appropriate?

--
Regards,
Pavel Roskin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 7 Jan 2011 11:14:59 +1100

> Anyway, here is a test program that does a set number (1024 * 1024)
> of 4096-byte (total 4GB) encryptions with libssl and kernel crypto.
> The result on my Core 2 (in a KVM VM FWIW) is:
>
> lenny0:~# time ./test_crypto > /dev/null
>
> real 0m30.340s
> user 0m0.172s
> sys 0m30.166s
> lenny0:~# time ./test_crypto software > /dev/null
>
> real 0m34.555s
> user 0m34.550s
> sys 0m0.004s
> lenny0:~#
>
> I'm not claiming these numbers make any sense but this is just
> a ballpark figure.
>
> Dave, can you run this program on your N2 and see what it shows?

Sure, here goes, BTW I had to change the buffer size to 8192 since
vmsplice requires page aligned operation (I was getting -EINVAL
otherwise):

davem@maramba:~$ time ./crypto_user_test >/dev/null

real 0m46.586s
user 0m3.280s
sys 0m43.230s
davem@maramba:~$ time ./crypto_user_test software >/dev/null

real 10m40.336s
user 10m40.540s
sys 0m0.000s
davem@maramba:~$

What's that, something like a factor of 13 faster? :-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 09:39:28PM -0500, Pavel Roskin wrote:
>
> By the way, is it a good idea to use setsockopt() this way? Some keys
> may be large. And if AEAD is supported, will it use setsockopt() for
> the unencrypted authenticated data?

For large keys (i.e., asymmetric key crypto) the kernel keyring API
would be the obvious way of passing them around.

For AEAD the AD would be passed around through send/splice.

Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 06:43:02PM -0800, David Miller wrote:
>
> davem@maramba:~$ time ./crypto_user_test >/dev/null
>
> real 0m46.586s
> user 0m3.280s
> sys 0m43.230s
> davem@maramba:~$ time ./crypto_user_test software >/dev/null
>
> real 10m40.336s
> user 10m40.540s
> sys 0m0.000s
> davem@maramba:~$
>
> What's that, something like a factor of 13 faster? :-)

Heh, I missed the leading 1 in your second result and was about
to spill my drink :)

Thanks!
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Thu, Jan 06, 2011 at 02:13:17PM -0800, Linus Torvalds wrote:
> On Thu, Jan 6, 2011 at 1:39 PM, Herbert Xu <herbert@gondor.hengli.com.au> wrote:
> > On Thu, Jan 06, 2011 at 01:23:19PM -0800, Linus Torvalds wrote:
> >>
> >> Explanations of interface. Code. Who uses it? What are the actual
> >> performance benefits on real code?
> >
> > You snipped out the bit in my reply where I expanded on it:
>
> You didn't expand on it AT ALL.
>
> You just mentioned "the interface". I haven't seen WHAT THAT INTERFACE IS!
>
> How hard is that to understand?
>
> > Here is the original cover email for the patches:
>
> Ok, this is more like it. This is roughly what I wanted to see:
>
> > : Here is a sample hash program (note that these only illustrate
> > : what the interface looks like and are not meant to be good examples
> > : of coding :)
>
> But I'm still missing the part where you show that there is any actual
> use case that makes sense, and that actually improves performance.
> Maybe it's been posted somewhere else, but the thing is, you're asking
> _me_ to pull, and as a result you need to convince _me_ that this is a
> good idea. So if it's been posted/discussed extensively elsewhere,
> please point to those discussions.
>
> I really don't like adding interfaces that don't have hard uses
> associated with them. We've done it in the past, and it tends to be a
> morass and a bad idea. That's been true even when the idea has been my
> own, and thus obviously genius-level and clearly the RightThing(tm),
> like "splice()". And it's why I push back on new interfaces when I see
> them.
>
> Btw, it doesn't have to be about performance per se. Does this allow
> people to use keys without actually _seeing_ those keys? Your example
> implies that that is not the case, but that's actually one of the few
> reasons to actually support a kernel crypto interface - the ability to
> have private personal keys around, but not having to actually let
> possibly untrusted programs see them.
>
This actually is an indirect feature of this interface. Using it, you can open
a algorithm socket, select a specific alg, assign a key, and then pass that
socket descriptor over a unix socket to an another process using an
SCM_RIGHTS ancilliary message. The receiving process can then use children
acceppted from that passed socket to preform the configured crypto operation
without any knoweldge of the keys used in it. I can write a demo app if you
like.

Regards
Neil

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, Jan 7, 2011 at 2:04 PM, Neil Horman <nhorman@tuxdriver.com> wrote:

>> Btw, it doesn't have to be about performance per se. Does this allow
>> people to use keys without actually _seeing_ those keys? Your example
>> implies that that is not the case, but that's actually one of the few
>> reasons to actually support a kernel crypto interface - the ability to
>> have private personal keys around, but not having to actually let
>> possibly untrusted programs see them.
> This actually is an indirect feature of this interface.  Using it, you can open
> a algorithm socket, select a specific alg, assign a key, and then pass that
> socket descriptor over a unix socket to an another process using an
> SCM_RIGHTS ancilliary message.  The receiving process can then use children
> acceppted from that passed socket to preform the configured crypto operation
> without any knoweldge of the keys used in it.  I can write a demo app if you
> like.

Several things have to be considered when extending an interface like
that. For example, do the algorithm implementations protect against
timing attacks, or keys can be recovered, using them? What is the
purpose of cryptographic key separation? If long term keys are to be
protected, then it makes sense to support RSA, DSA keys in addition to
symmetric keys (e.g. microsoft only supports assymetric keys for
separation, since symmetric ones are rarely used as long term keys -
[in TLS-PSK]).

Some of these issues are discussed in my report on a user-space crypto
interface that specifically targeted separation and speed at [0].

regards,
Nikos

[0]. https://www.cosic.esat.kuleuven.be/publications/article-1490.pdf
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Sat, Jan 08, 2011 at 03:23:04PM +0200, Nikos Mavrogiannopoulos wrote:
> On Fri, Jan 7, 2011 at 2:04 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>
> >> Btw, it doesn't have to be about performance per se. Does this allow
> >> people to use keys without actually _seeing_ those keys? Your example
> >> implies that that is not the case, but that's actually one of the few
> >> reasons to actually support a kernel crypto interface - the ability to
> >> have private personal keys around, but not having to actually let
> >> possibly untrusted programs see them.
> > This actually is an indirect feature of this interface.  Using it, you can open
> > a algorithm socket, select a specific alg, assign a key, and then pass that
> > socket descriptor over a unix socket to an another process using an
> > SCM_RIGHTS ancilliary message.  The receiving process can then use children
> > acceppted from that passed socket to preform the configured crypto operation
> > without any knoweldge of the keys used in it.  I can write a demo app if you
> > like.
>
> Several things have to be considered when extending an interface like
> that. For example, do the algorithm implementations protect against
> timing attacks, or keys can be recovered, using them? What is the
No, the kernel does not implement any protection against timing attacks in the
algorithms per-se, but preforming a timing attack against a kernel crypto
operation is going to be near impossible anyway, as precise timing measurements
are going to get obscured by interupts, scheduling jitter, lock contention, and
various other factors that will make measuring syscall time fairly useless.

> purpose of cryptographic key separation? If long term keys are to be
My only purpose was to answer Linus' question. He wondered if other user space
programs could use instances of cyrpto algs over this interface without needing
to hold key data. I was illustrating how that could be done.

Neil

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

> On Sat, Jan 08, 2011 at 03:23:04PM +0200, Nikos Mavrogiannopoulos wrote:
> > On Fri, Jan 7, 2011 at 2:04 PM, Neil Horman <nhorman@tuxdriver.com>
> > wrote:
> >
> > > > Btw, it doesn't have to be about performance per se. Does this
> > > > allow people to use keys without actually _seeing_ those keys?
> > > > Your example implies that that is not the case, but that's
> > > > actually one of the few reasons to actually support a kernel
> > > > crypto interface - the ability to have private personal keys
> > > > around, but not having to actually let possibly untrusted programs
> > > > see them.
> > > This actually is an indirect feature of this interface.  Using it,
> > > you can open a algorithm socket, select a specific alg, assign a
> > > key, and then pass that socket descriptor over a unix socket to an
> > > another process using an SCM_RIGHTS ancilliary message.  The
> > > receiving process can then use children acceppted from that passed
> > > socket to preform the configured crypto operation without any
> > > knoweldge of the keys used in it.  I can write a demo app if you
> > > like.
> >
> > Several things have to be considered when extending an interface like
> > that. For example, do the algorithm implementations protect against
> > timing attacks, or keys can be recovered, using them? What is the
> No, the kernel does not implement any protection against timing attacks
> in the algorithms per-se, but preforming a timing attack against a
> kernel crypto operation is going to be near impossible anyway, as
> precise timing measurements are going to get obscured by interupts,
> scheduling jitter, lock contention, and various other factors that will
> make measuring syscall time fairly useless.

Let me just point out that this is not near impossible at all; instead it has already been done more than 6 years ago. And it's not only syscall time that leaks information. One practical example is recovery of a full AES key in a couple of seconds, using cache attacks against an encrypted file system. AES-NI is immune to this kind of attack, but other algorithms typically implemented using lookup tables are at risk.

Dag Arne

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hi Linus:

Here is an updated to the crypto pull request for 2.6.38:

* Crypto API interface for user-space (hash + skcipher)

This interface is intended to provide access of kernel
hardware crypto drivers to user-space applications. It
presents the kernel crypto API via a socket interface.

It is only meant to be used for bulk crypto operations,
i.e., if you're doing less than a page's worth of data,
then this is not for you.

For bulk data processing the performance is similar to
that in the kernel when zero-copy is used in both
directions.

* Memory corruption fix for via_rng.
* Fixes for new OMAP driver.
* 32-bit support in aesni-intel.
* GCM support in aesni-intel.
* Misc fixes.


Please pull from

git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6.git

or

master.kernel.org:/pub/scm/linux/kernel/git/herbert/crypto-2.6.git


Adrian Hoban (1):
crypto: rfc4106 - Extending the RC4106 AES-GCM test vectors

Adrian-Ken Rueegsegger (1):
crypto: ripemd - Set module author and update email address

Dennis Gilmore (1):
crypto: n2 - use __devexit not __exit in n2_unregister_algs

Dmitry Kasatkin (15):
crypto: omap-sham - uses digest buffer in request context
crypto: omap-sham - DMA initialization fixes for off mode
crypto: omap-sham - error handling improved
crypto: omap-sham - removed redundunt locking
crypto: omap-sham - crypto_ahash_final() now not need to be called.
crypto: omap-sham - hash-in-progress is stored in hw format
crypto: omap-sham - FLAGS_FIRST is redundant and removed
crypto: omap-sham - zero-copy scatterlist handling
crypto: omap-aes - DMA initialization fixes for OMAP off mode
crypto: omap-aes - redundant locking is removed
crypto: omap-aes - error handling implementation improved
crypto: omap-aes - unnecessary code removed
crypto: omap-aes - initialize aes module once per request
crypto: omap-aes - checkpatch --file warning fixes
crypto: omap-sham - backlog handling fix

Herbert Xu (13):
crypto: hash - Fix async import on shash algorithm
net - Add AF_ALG macros
crypto: af_alg - User-space interface for Crypto API
crypto: algif_hash - User-space interface for hash operations
crypto: algif_skcipher - User-space interface for skcipher operations
crypto: algif_skcipher - Pass on error from af_alg_make_sg
crypto: af_alg - Add dependency on NET
crypto: algif_skcipher - Fixed overflow when sndbuf is page aligned
crypto: algif_skcipher - Handle unaligned receive buffer
crypto: gf128mul - Remove experimental tag
hwrng: via_rng - Fix asm constraints
crypto: padlock - Move padlock.h into include/crypto
hwrng: via_rng - Fix memory scribbling on some CPUs

Joe Perches (1):
crypto: Use vzalloc

Mathias Krause (2):
crypto: aesni-intel - Ported implementation to x86-32
crypto: aesni-intel - Fixed build error on x86-32

Miloslav Trmač (3):
include: Install linux/if_alg.h for user-space crypto API
net: Add missing lockdep class names for af_alg
crypto: af_alg - Make sure sk_security is initialized on accept()ed sockets

Nicolas Kaiser (1):
crypto: cast5 - simplify if-statements

Randy Dunlap (1):
crypto: af_alg - fix af_alg memory_allocated data type

Roel Kluin (1):
crypto: mv_cesa - dont return PTR_ERR() of wrong pointer

Steffen Klassert (2):
crypto: scatterwalk - Add scatterwalk_crypto_chain helper
crypto: Use scatterwalk_crypto_chain

Tadeusz Struk (2):
crypto: aesni-intel - RFC4106 AES-GCM Driver Using Intel New Instructions
crypto: aesni-intel - Fixed build with binutils 2.16

Tejun Heo (1):
crypto: mark crypto workqueues CPU_INTENSIVE

Tracey Dent (1):
crypto: Makefile clean up

arch/x86/crypto/aesni-intel_asm.S | 1832 ++++++++++++++++++++++++++++++++-
arch/x86/crypto/aesni-intel_glue.c | 540 ++++++++++-
crypto/Kconfig | 34 +-
crypto/Makefile | 17 +-
crypto/af_alg.c | 483 +++++++++
crypto/algif_hash.c | 319 ++++++
crypto/algif_skcipher.c | 632 ++++++++++++
crypto/authenc.c | 22 +-
crypto/cast5.c | 74 +-
crypto/crypto_wq.c | 3 +-
crypto/deflate.c | 3 +-
crypto/eseqiv.c | 18 +-
crypto/gcm.c | 19 +-
crypto/pcrypt.c | 3 +-
crypto/rmd128.c | 3 +-
crypto/rmd160.c | 3 +-
crypto/rmd256.c | 3 +-
crypto/rmd320.c | 3 +-
crypto/shash.c | 8 +-
crypto/tcrypt.c | 11 +
crypto/testmgr.c | 24 +
crypto/testmgr.h | 361 +++++++
crypto/zlib.c | 3 +-
drivers/char/hw_random/via-rng.c | 10 +-
drivers/crypto/mv_cesa.c | 2 +-
drivers/crypto/n2_core.c | 2 +-
drivers/crypto/omap-aes.c | 260 +++---
drivers/crypto/omap-sham.c | 374 ++++---
drivers/crypto/padlock-aes.c | 2 +-
drivers/crypto/padlock-sha.c | 8 +-
include/crypto/if_alg.h | 92 ++
{drivers => include}/crypto/padlock.h | 8 +-
include/crypto/scatterwalk.h | 15 +
include/linux/Kbuild | 1 +
include/linux/if_alg.h | 40 +
include/linux/socket.h | 5 +-
net/core/sock.c | 6 +-
37 files changed, 4779 insertions(+), 464 deletions(-)

Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/