Mailing List Archive

On Mon, 27 Apr 1998 19:29:10 +0200, Werner Koch wrote:

> 1) a file with the keyids (or the complete email address) of all
> subscribers is created and each id is prefixed with "-r "
> 2) gpg is run with "--options file-with-keyids-from-step-1" and
> creates a valid message to the n receivers. This file is
> stored on disk
> 3) a tool is used to cut off one receivers public-key encrypted
> session-key and the encrypted messaage id from the file created
> in step 2 and this is passed to sendmail.
> 4) continue with step 3 but use the next user-id.


>This is quite easy but we have to do a lot of calculations.

This seems to have a "relatively" small fixed cost ([decrypt-verify]
session key, encrypt) and a considerable cost per recipient (encrypt
session key with recipient's public key). With say > 100 recipients,
the first part should be minimal.

Is there an easy way using gpg to measure the time taken for the
per-recipient step?

Is key handling good enough to not make key retrieval a bottle neck
with say 10,000 recipients (a reasonably high maximum; a mailing list
can easily be split into sublists with no more than 1000 recipients in
each, and most totally encrypted lists would probably be < 100
recipients)? If not, keys could be stored in a hash or in a gbm or
equivalent table. Adding keys will be a relatively rare event.

-Sincerely, Fred

(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)

Fred Lindberg <lindberg@id.wustl.edu> writes:

> Is there an easy way using gpg to measure the time taken for the
> per-recipient step?

I should add a profiling option - it's worth the time.

> Is key handling good enough to not make key retrieval a bottle neck
> with say 10,000 recipients (a reasonably high maximum; a mailing list

My plan is to write a fast keyserver which does this stuff. The
encryption will always take up more time than the key lookup.
You should put the email addresses in left angles into a file:

============
# This file is the list of subscribers
-r <joe@some.net>
-r <alice@net.nil>
===============

and call gpg with "gpg --options ~/.gnupg/options -e --options ./above-file"
If you would like to use the .qmail format I may add yet another special
option to use this file direct.

This file is stored in memory and should not be too large :-)


Werner

On Mon, 27 Apr 1998 21:57:24 +0200, Werner Koch wrote:

>My plan is to write a fast keyserver which does this stuff. The
>encryption will always take up more time than the key lookup.
>You should put the email addresses in left angles into a file:

>============
># This file is the list of subscribers
>-r <joe@some.net>
>-r <alice@net.nil>
>===============

>and call gpg with "gpg --options ~/.gnupg/options -e --options ./above-file"
>If you would like to use the .qmail format I may add yet another special
>option to use this file direct.

This format is fine. Better to deal with the specific format within
ezmlm. Also, the subscriber addresses are stored in a hash of up to 53
files. An alternative would be to allow the addresses to be piped in to
fd 1, as with qmail-queue. Then it would make sense to use a format
like:
Tadd1@host1\0
Tadd2@host2\0
...
\0

>This file is stored in memory and should not be too large :-)
Theoretically, a subscriber db can be quite large. Would it be possible
to just read a stream, generating the encrypted session-key blocks one
at a time as the addresses are encountered in the input stream? A nice
side-effect would be that the "client" could assume that the blocks
would appear after the message in the order they were send. ezmlm could
lock the directory to assure that the subscriber db would remain
untouched while gpg does it's job. This way, ezmlm could process the
subscriber list and message without sorting.

Which brings out another point: Would it be possible to derive the
recipient userid/address from the pgp output without asking the key
server? I assume that deriving the key id is easy, but the
keyid->address could cause some problems.

Thanks!

-Sincerely, Fred

(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)

Fred Lindberg <lindberg@id.wustl.edu> writes:

> Theoretically, a subscriber db can be quite large. Would it be possible
> to just read a stream, generating the encrypted session-key blocks one
> at a time as the addresses are encountered in the input stream? A nice

Yes, this might make some sense. But to do stream processing, the
message should be encrypted first and stored in a file - may be it
is better that gpg handles this and outputs several complete messages -
to different files or with a special delimiter to stdout.

A problem is how to handle addresses which have not enough trust
assigned; normally a user interaction is required.

> Which brings out another point: Would it be possible to derive the
> recipient userid/address from the pgp output without asking the key
> server? I assume that deriving the key id is easy, but the
> keyid->address could cause some problems.

We can use comment packets for this. The last OpenPGP draft dropped the
comment packets :-((, this is really bad. I changed my code to use the
new comment packet number because PGP said "we never used the RFC1991
specified comment packets, so we reuse this packet type for subkeys" and
assigned a new number for comment packets which are now dropped -- This
is very annoying. I see the need for comment packets and will continue
to support them. Ahhh, OpenPGP has nothing to do with open but with
"documenting" PGP 5 data structures - this is the behaviour of a Redmond
based company - I DON'T LIKE IT.



Werner

On Tue, 28 Apr 1998 08:54:30 +0200, Werner Koch wrote:

>Yes, this might make some sense. But to do stream processing, the
>message should be encrypted first and stored in a file - may be it
>is better that gpg handles this and outputs several complete messages -
>to different files or with a special delimiter to stdout.

i was thinking of:

PASSPHRASE=`cat passphrase`
gpg --skip_addresses_with_not_trusted_keys \
--decrypt --verify --sign_with_key_ID:123 --reencrypt --keyfile
addresses.txt \
--output_to storage.file --format crude_non_armored < orignal
encrypted(signed(message)) > storage.file
switch ($?)
case 0: ok
case 1: bad_encryption
case 2: bad_signature
case 3: passphrase_doesn_t_work
...
case 99: unknown fatal error

make mail messages from storage.file.
erase storage.file
---------

PASSPHRASE is the passphrase for decryption = signature

addresses.txt has user addresses = userids (ezmlm could have a database
doing address -> userid translation
this _could_ be piped in using a fd other than stdin. ezmlm could also
have the keyID in this db.

The "action" options would be in order. --verify would possibly put the
userID into the environment. An error message could go to and be
captured from stderr. This all would allow handling messages in a
pipeline without ever externalizing the clear text.

It assumes that the private key and pass phrase is secret. The list
could have a mechanism that generates a new /dev/urandom-based pass
phrase for the key at random intervals (not while gpg is running). This
way, deletion of the passphrase file would make all communication to
the list unreadable.

An option might be to keep the sender signature intact (when anonymity
of sender is not important and it is important to recipients to know
that the list has not tampered with the message contents.

The mailing list program would be responsible for rebuilding messages
from the crude gpg 1x(encrypted-message), n x (pub-key encrypted
session key block).

>A problem is how to handle addresses which have not enough trust
>assigned; normally a user interaction is required.

A moderator checks all subscriptions and signs keys. Another part of
the list assures that only subscriber keys that are signed by a
moderator are accepted. If there are userIDs in addresses.txt with keys
that haven't been signed by a moderator, those userIDs are ignored.

Moderator signatures would in turn be trusted only if signed by other
moderators, but that would be added on later. Alternatively: Anyone
with access to the list key and passphrase can read messages, knows
recipients, etc. Thus, one might require only that moderator keys are
signed by the list key. I don't know what's best here, but it seems
that one has to assume that the list-computer is secure for anything at
all to work.


-Sincerely, Fred

(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)

On Tue, 28 Apr 1998, Werner Koch wrote:

> We can use comment packets for this. The last OpenPGP draft dropped the
> comment packets :-((, this is really bad. I changed my code to use the
> new comment packet number because PGP said "we never used the RFC1991
> specified comment packets, so we reuse this packet type for subkeys" and
> assigned a new number for comment packets which are now dropped -- This
> is very annoying. I see the need for comment packets and will continue
> to support them. Ahhh, OpenPGP has nothing to do with open but with
> "documenting" PGP 5 data structures - this is the behaviour of a Redmond
> based company - I DON'T LIKE IT.

It's probably time to start thinking about an operating mode which is
clearly distinguished from OpenPGP, and is documented and available for
comment if, say, someone else wants to implement it.

It's all very well to play when someone brings out his toys, but when he
won't play fair, one should start thinking about how to give the end user a
choice of toys.


Rob Levin

On Tue, 28 Apr 1998 09:55:27 -0500, Fred Lindberg wrote:

>It assumes that the private key and pass phrase is secret. The list
>could have a mechanism that generates a new /dev/urandom-based pass
>phrase for the key at random intervals (not while gpg is running). This
>way, deletion of the passphrase file would make all communication to
>the list unreadable.

Forget the pass phrase refresh. I was thinking of guarding against
copying files/backup. However, anyone having a copy of key + passphrase
will have the key even if pass phrases are changed, so the above
complicates without any gain :-( The key itself could be changed, but
this requires notifying all subscribers of the new public key (can be
done as a regular list message) and invalidates messages en route to
the list. The list could of course have some key expiration system with
partial overlap, but again this type of complication seems better for
later.

I realize that we never introduced this discussion, although it can be
gleaned from the subject:

We (Sen Nagata and I) would like to add support for encryption and
authentication to ezmlm.

ezmlm is a fast mailing list manager based on qmail
(http://www.qmail.org/). Both were written by Dan J. Bernstein
(http://www.pobox.com/~djb). ezmlm uses "cryptographic cookies" to
authenticate subscribe/unsubscribe requests. ezmlm-idx is an add-on to
ezmlm (http://www.ezmlm.org/; written by me with Fred Ringel) that
allows "cookie-protected" message moderation, subscription moderation,
remote administration, digests, archive retrieval, MIME support and
customization (all lists set up with a single command and customized
for language, etc, with a ezmlmrc file).

ezmlm at the moment requires qmail as the MTA (sendmail-replacement).
qmail was designed for security and reliability
(http://www.pobox.com/~djb/qmail.html). ezmlm fully exploits qmail's
features. qmail is "free" but you may only distribute Dan's source
package as is (he's worried about less secure modified versions). Thus,
there are patches, but no official binary distributions. ezmlm is
publically available from Dan's ftp server and has no stated
distribution restrictions. ezmlm-idx, of couse, is GPL (if Dan grants
you other less restrictive rights to ezmlm, you automatically get them
to ezmlm-idx).


-Sincerely, Fred

(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)

lilo@dyn-max1-1.chicago.il.ameritech.net writes:

> It's all very well to play when someone brings out his toys, but when he
> won't play fair, one should start thinking about how to give the end user a
> choice of toys.

It's somewhat like the M$ Windoze situation - you can write very good
apps but M$'s apps are always better because they define a new "standard"
which is not compatible to the old one and they are always a little
bit ahead of all others (e.g. Word6 -> Word7, their JAVA API).

The KDE project does exactly that - trying to clone Windoze.

:-(


Werner

Fred Lindberg <lindberg@id.wustl.edu> writes:

> i was thinking of:
>
> PASSPHRASE=`cat passphrase`
> gpg --skip_addresses_with_not_trusted_keys \
> --decrypt --verify --sign_with_key_ID:123 --reencrypt --keyfile
> addresses.txt \
> --output_to storage.file --format crude_non_armored < orignal
> encrypted(signed(message)) > storage.file

Better use something like this:

gpg --decrypt --verify <original | gpg --sign.... --keyfile... >storage

> switch ($?)
> case 0: ok
> case 1: bad_encryption

For this I have the --status-fd option which writes textual messages
which are easy to parse; you can use two different fd for the
decryption and the encryption stage. if the program returns with an
error, a grep tells you what's the real reason.

> addresses.txt has user addresses = userids (ezmlm could have a database
> doing address -> userid translation
> this _could_ be piped in using a fd other than stdin. ezmlm could also

It's okay to pass the email address to gpg.

> An option might be to keep the sender signature intact (when anonymity
> of sender is not important and it is important to recipients to know
> that the list has not tampered with the message contents.

It is possible to add more signatures to a message.

> The mailing list program would be responsible for rebuilding messages
> from the crude gpg 1x(encrypted-message), n x (pub-key encrypted

What I have to do is to change the sequence of the packets, but that is
trivial (but the output is not valid gpg message).

> A moderator checks all subscriptions and signs keys. Another part of
> the list assures that only subscriber keys that are signed by a
> moderator are accepted. If there are userIDs in addresses.txt with keys
> that haven't been signed by a moderator, those userIDs are ignored.

So we need a way to specifiy that a key must be signed by xxxxx;
this is reasonable because it avoids lengthly trust calculations.

Hmmm, very complicated to put this all into the OpenPGP standard.



Werner

On Tue, 28 Apr 1998, Werner Koch wrote:

> It's somewhat like the M$ Windoze situation - you can write very good
> apps but M$'s apps are always better because they define a new "standard"
> which is not compatible to the old one and they are always a little
> bit ahead of all others (e.g. Word6 -> Word7, their JAVA API).

Well, let's not lose sight of the fact that PGP became well-known because it
provided worthwhile functionality, not the reverse. There's still no
freeware general encryption tool other than GPG.

> The KDE project does exactly that - trying to clone Windoze.

It'd be nice to have a program that will read PGP keys for your keyrings,
and write PGP-readable messages to users whose keys were generated with PGP,
but it'd also be nice to have a program whose features were not limited by
PGP design limitations when you're not dealing with PGP users....

I'm pleased to see a freeware program with GPG's capabilities being
developed. I don't anticipate that most of the people I'll be corresponding
with will be running PGP once that development has reached a nice solid
release point.

For what it's worth, your mileage may vary, etc.


Rob L.

On Tue, 28 Apr 1998 18:40:43 +0200, Werner Koch wrote:

Dear Werner:

>Better use something like this:
>
> gpg --decrypt --verify <original | gpg --sign.... --keyfile... >storage

Yes!

>For this I have the --status-fd option which writes textual messages
>which are easy to parse; you can use two different fd for the
>decryption and the encryption stage. if the program returns with an
>error, a grep tells you what's the real reason.

Great! I assume the relevant part to parse will stay constant over
versions.

>It's okay to pass the email address to gpg.

Good.

>It is possible to add more signatures to a message.

Yes. I was thinking of having it optional to retain the sender
signature with the message. gpg could do this as an option, or the
output could be such that it would be easy to use/discard it.

>What I have to do is to change the sequence of the packets, but that is
>trivial (but the output is not valid gpg message).

It would be ok to get the message after the keys. It will be written to
disk with the message alone kept in memory. No big advantage to put the
message first.

>So we need a way to specifiy that a key must be signed by xxxxx;
>this is reasonable because it avoids lengthly trust calculations.

Or just have a special key server for the list, accept anything and
make sure that only the "good" keys are added to the db. Since we take
assume that local security is 100% this should be ok.

>Hmmm, very complicated to put this all into the OpenPGP standard.

See above. I think the key issue is piping parsable error messages, and
a raw format that can be disassembled. Ideally, no items specific to
this application should be needed. The ML software would take care of
base64-encoding, PGP headers, etc.


-Sincerely, Fred

(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)

On Mon, 27 Apr 1998, Werner Koch wrote:

> Hi Anand,
>
> Anand Kumria <wildfire@progsoc.uts.edu.au> writes:
>
> > When you communicate there are four different modes you can communicate
> > in. Single Sender, Single Receiver (SS); Single Sender, Multiple
> > Receivers (SM); Multiple Senders, Single Receiver (MS) and Multiple
> > Senders, Multiple Receivers (MM).
>
> Examples:
> SS = private email
> SM = anouncement MLs
> MS = bug reports
> MM = MLs

Actually I wouldn't categorise Mailling lists as MM, it is normally SM. I
think most cases of MM degrade into SM.

> > I can see some immediate uses for Single Sender/Mltiple Receiver crypto;
> > one would be in the Debian group. new-maintainer@debian.org actually goes
> > to a number of people, in order to send a crypted message to them I need
> > to know who those people are, what their current correct public keys are
>
> We came up with a simple solution for a multiple receivers system:
>

[snip]

Ahh yes, you've basically degraded the SM cases into many SS transactions.

> > I can see some initial problems: key generation, secret sharing, secret
> > recombination/splitting, manipulating group membership, etc. No doubt
>
> I guess that a secret sharing scheme could heavily increase the
> performance but key distribution would be quite complex. Perhaps
> we should think about a n-party DH scheme which has some security
> advantages.

Yes a secret sharing scheme would give us some advantages (and make key
dist. more `interesting'). What we really want is the ability to generate
a public/private key pair.

Each subscriber to the mailing list would simply get a private key. We
want to have a single public key and _different_ private keys _per_
_user_.

That would solve (some) of the problems for encrypted mailling lists.
Each message is encrypted to the mailling lists's public key, and only
people with the private key share can decrypt. If you need authenticated,
encrypted mailling lists, I guess you could also sign the message
prior/post encryption.

There is a paper which discusses some of these issues: "Multi-receiver /
multi-sender network security: efficent authenticated multicast /
feedback" by Yvo Desmedt, Yair Frankel and Moti Yung. It was published in
IEEE Infocom '92, pages 2045 - 2054. They present two multi-receiver
schemes: one using polynominal interpolation and the other using geometric
interpolation.

Oh, btw, I wouldn't worry too much about OpenPGP; they are only trying to
define the format for PGP 5.x things. I think you will find that most of
the 180K+ public keys that the keyservers have on file are PGP2.x
- basically, if you have to do your own thing, do it.

Anand.

--
`When any government, or any church for that matter, undertakes to say to
its subjects, "This you may not read, this you must not see, this you are
forbidden to know," the end result is tyranny and oppression no matter how
holy the motives' -- Robert A Heinlein, "If this goes on --"

Anand Kumria <wildfire@progsoc.uts.edu.au> writes:

> feedback" by Yvo Desmedt, Yair Frankel and Moti Yung. It was published in
> IEEE Infocom '92, pages 2045 - 2054. They present two multi-receiver

Let's see whether it is in my digital IEEE subscription.

> the 180K+ public keys that the keyservers have on file are PGP2.x
> - basically, if you have to do your own thing, do it.

I'll try to stick to OpenPGP as long as it does not make too much
hassle.


Werner