Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. GnuPG>
  3. gcrypt
another question, and a small patch
vladimir at helixcode
Jun 12, 2000, 8:32 AM
Post #1 of 7 (2332 views)
Permalink
Okay, I got my issues worked out -- now, it's mainly speed. My code
basically does this.. reads 128-byte blocks from an input file, does a
gcry_mpi_scan on the blocks, then makes a SEXP from that MPI, does
a pk_encrypt(), and then writes the resulting MPI a/b pair to a
file. (I'm using ElGamal.) The problem is that it's ridiculously slow;
encoding a simple 6k file takes about 8 seconds, which is completely
unacceptable. By contrast, gpg -e takes 0.06 sec. So, I'm obviously
doing something that's rather stupid. I'm currently read()'ing blocks
one by one, an optimization would be to mmap the input file; but that
shouldn't be that big of a bottleneck. Any thoughts on this speed
problem?

I also ran into some problems with the sexp implementation.. The
problem is if you have a sexp such as that returned from pk_encrypt:
(enc-data (elg (a MPI) (b MPI))), there was no way previously to get
at any of that data using the car/cdr functions. You had to search for
the token. These patches make car 'do the right thing' (I think), by
checking to see if the first elt is a list, and if so to give you the
list itself, not just the same object back. Also, sexp_release wasn't
implemented, so I implemented that as well. Plus, the small patch that
attempts to stick the right export symbols into the .so (it still
doesn't work quite right).


Index: gcrypt/Makefile.am
===================================================================
RCS file: /home/koch/cvs/gnupg/gcrypt/Makefile.am,v
retrieving revision 1.13
diff -u -r1.13 Makefile.am
--- gcrypt/Makefile.am 2000/01/31 15:26:58 1.13
+++ gcrypt/Makefile.am 2000/06/12 15:21:51
@@ -43,8 +43,14 @@
# libtool's --export-symbols-regex does not work in 1.3.2
# so we do it ourself
libgcrypt.sym: $(libgcrypt_la_OBJECTS)
- $(NM) $(libgcrypt_la_OBJECTS) \
- | $(AWK) '$$2=="T" && $$3 ~ /^gcry_/ {print $$3}' >libgcrypt.sym
+ $(NM) $(libgcrypt_la_OBJECTS) \
+ | $(AWK) '$$2=="T" && $$3 ~ /^gcry_/ {print $$3}' >libgcrypt-local.sym
+ (cd ../mpi; $(NM) *.lo | $(AWK) '$$2=="T" && $$3 ~ /^gcry_/ {print $$3}' > libmpi.sym)
+ (cd ../cipher; $(NM) *.lo | $(AWK) '$$2=="T" && $$3 ~ /^gcry_/ {print $$3}' > libcipher.sym)
+ (cd ../util; $(NM) *.lo | $(AWK) '$$2=="T" && $$3 ~ /^gcry_/ {print $$3}' > libutil.sym)
+ cat libgcrypt-local.sym ../mpi/libmpi.sym ../cipher/libcipher.sym ../util/libutil.sym > libgcrypt.sym
+
+



Index: gcrypt/sexp.c
===================================================================
RCS file: /home/koch/cvs/gnupg/gcrypt/sexp.c,v
retrieving revision 1.9
diff -u -r1.9 sexp.c
--- gcrypt/sexp.c 2000/01/27 16:50:43 1.9
+++ gcrypt/sexp.c 2000/06/12 15:21:51
@@ -219,7 +219,35 @@
void
gcry_sexp_release( GCRY_SEXP sexp )
{
- /* FIXME! */
+ /* Note that this will do a deep release, i.e. release the mpi's
+ * and data buf's that are pointed to by this sexp. This, in theory,
+ * is the wrong thing to do -- the "Right Thing" would be to do
+ * refcounting here.
+ */
+
+ if (sexp == NULL) return;
+
+#if 0
+ fprintf (stderr, "SEXP_RELEASE: ");
+ gcry_sexp_dump (sexp);
+#endif
+
+ /* Release across */
+ gcry_sexp_release (sexp->next);
+
+ /* Then release down, if present */
+ if (sexp->type == ntLIST) {
+ gcry_sexp_release (sexp->u.list);
+ } else if (sexp->type == ntMPI) {
+ gcry_mpi_release (sexp->u.mpi);
+ } else if (sexp->type == ntDATA) {
+ /* Nothing special -- taken care of in following g10_free */
+ } else {
+ /* there should be no other types */
+ BUG();
+ }
+
+ g10_free (sexp);
}


@@ -478,7 +506,10 @@
GCRY_SEXP
gcry_sexp_car( GCRY_SEXP list )
{
- return list;
+ if( list && list->type == ntLIST )
+ list = list->u.list;
+
+ return list;
}

/****************
@@ -524,6 +555,8 @@
GCRY_SEXP
gcry_sexp_cdr( GCRY_SEXP list )
{
+ if( list && list->type == ntLIST && !list->next )
+ list = list->u.list;
if( list && (list = list->next) )
return list;
return NULL;
Re: another question, and a small patch [ In reply to ]
wk at gnupg
Jun 13, 2000, 2:07 AM
Post #2 of 7 (2291 views)
Permalink
Hi,

On Mon, 12 Jun 2000, Vladimir Vukicevic wrote:

> Okay, I got my issues worked out -- now, it's mainly speed. My code
> basically does this.. reads 128-byte blocks from an input file, does a
> gcry_mpi_scan on the blocks, then makes a SEXP from that MPI, does
> a pk_encrypt(), and then writes the resulting MPI a/b pair to a
> file. (I'm using ElGamal.) The problem is that it's ridiculously slow;
> encoding a simple 6k file takes about 8 seconds, which is completely

You are doing public-key encryption on large data blocks - this is the
problem. Nearly all encryption tools don't do this but use a hybrid
scheme: You generate a random session key, encrypt this using a
public-key algorithm and then you encrypt the bulk data with a
symmetric algorithm like Twofish. You have to take care about many
other things too - otherwise you don't get real security.

If you are looking for a data format for encryption; you should
take several things into account:

o Is it an synchronous (tcp connection) or asynchronous (store and
forward, email) communication channel.
o Is it a long term connection or is the connection establish
many times (ssh vs. http)
o What is the size of the payload
o Do you need perfect forward security
o ...

OpenPGP is a good protocol for asyncronous communication. TLS is
good for HTTP like transactions and the SSH v2 protocols are fine for
long term connections. If you are going for the SSh protocols; I have
a started with a library implementation which handles all the protocol
stuff (GSTI - GNU Secure Transport Initiative).

> I also ran into some problems with the sexp implementation.. The

No wonder. I should put the brown bag over my head. Thanks for the
patches, I'll take them into account.

BTW, I started writing some example code for libgcrypt.

Werner

--
Werner Koch OpenPGP key 621CC013
OpenIT GmbH tel +49 211 239577-0
Birkenstr. 12 email wk@OpenIT.de
D-40233 Duesseldorf http://www.OpenIT.de
Re: another question, and a small patch [ In reply to ]
vladimir at helixcode
Jun 13, 2000, 7:43 AM
Post #3 of 7 (2289 views)
Permalink
Hi.. thanks for the quick response :-)

Werner Koch wrote:
> You are doing public-key encryption on large data blocks - this is the
> problem. Nearly all encryption tools don't do this but use a hybrid
> scheme: You generate a random session key, encrypt this using a
> public-key algorithm and then you encrypt the bulk data with a
> symmetric algorithm like Twofish. You have to take care about many
> other things too - otherwise you don't get real security.

Yeah, I realized my idiocy right around 2am.. everything is MUCH faster
now,
but I'm running into a weird bug where gcry_pk_decrypt() crashes UNLESS
I do
a gcry_sexp_dump(key); right before it. If the key doesn't get printed,
decrypt crashes. If it gets printed, everything works fine. Bizzare.

> If you are looking for a data format for encryption; you should
> take several things into account:
>
> o Is it an synchronous (tcp connection) or asynchronous (store and
> forward, email) communication channel.
> o Is it a long term connection or is the connection establish
> many times (ssh vs. http)
> o What is the size of the payload
> o Do you need perfect forward security
> o ...
>
> OpenPGP is a good protocol for asyncronous communication. TLS is
> good for HTTP like transactions and the SSH v2 protocols are fine for
> long term connections. If you are going for the SSh protocols; I have
> a started with a library implementation which handles all the protocol
> stuff (GSTI - GNU Secure Transport Initiative).

Well, it's running directly over HTTP -- but we need public key support;
also, there is unfortunately no TLS/SSL library that is fully GPL'd,
something that is important
to us as our resulting application is to be released under the GPL.

> > I also ran into some problems with the sexp implementation.. The
>
> No wonder. I should put the brown bag over my head. Thanks for the
> patches, I'll take them into account.
>
> BTW, I started writing some example code for libgcrypt.

Cool.. that'll be nice.

I'm a little unclear as to the different versions of GnuPG -- what is
the planned release date for the one that includes libgcrypt?

- Vlad
Re: another question, and a small patch [ In reply to ]
vladimir at helixcode
Jun 13, 2000, 8:06 AM
Post #4 of 7 (2288 views)
Permalink
Also, small bug in gcry_pk_genkey(); it should not gcry_sexp_release()
the three sexp's at the end, since they were just put together using
vlist just above -- and vlist doesn't copy the nodes, just uses creates
a new list containing all of them. So if they're actually release()'d,
it's not a good thing :-)

I notice there's a comment at the top takling about implementing
refcounting... is this something that's needed? It might be overkill
considering that sexp's don't seem to be used in any majorly extensive
way, but OTOH it would avoid some of the double-free bugs and the like..

- Vlad
Re: another question, and a small patch [ In reply to ]
wk at gnupg
Jun 13, 2000, 9:05 AM
Post #5 of 7 (2286 views)
Permalink
On Tue, 13 Jun 2000, Vladimir Vukicevic wrote:

> Well, it's running directly over HTTP -- but we need public key support;
> also, there is unfortunately no TLS/SSL library that is fully GPL'd,
> something that is important

Some folks are working on it. See lists.gnupg.org and cvs.gnupg.org
(gnutls). I am not following cloesely but I think they can need some
help.

> I'm a little unclear as to the different versions of GnuPG -- what is
> the planned release date for the one that includes libgcrypt?

Tsss, I guess you have to live with pre-release version og libgcrupt
for a while. There are quote a lot of changes to the core GnuPg
planned.

Werner

--
Werner Koch OpenPGP key 621CC013
OpenIT GmbH tel +49 211 239577-0
Birkenstr. 12 email wk@OpenIT.de
D-40233 Duesseldorf http://www.OpenIT.de
Re: another question, and a small patch [ In reply to ]
wk at gnupg
Jun 13, 2000, 9:08 AM
Post #6 of 7 (2298 views)
Permalink
On Tue, 13 Jun 2000, Vladimir Vukicevic wrote:

> a new list containing all of them. So if they're actually release()'d,
> it's not a good thing :-)

Well that might be the reason that gcry_sexp_release was a dummy ;-)

Frankly, my plan is to first check wether this S-Exp really work and
don't impose too much parsing overhead. The nice thing with S-Exp is
that we don't need to change the API when a new PK algorithm is
introduced.

> I notice there's a comment at the top takling about implementing
> refcounting... is this something that's needed? It might be overkill

I know about the shallow copy issues, so ref-counting seems to be so
only easy way to cope with it.

werner

--
Werner Koch OpenPGP key 621CC013
OpenIT GmbH tel +49 211 239577-0
Birkenstr. 12 email wk@OpenIT.de
D-40233 Duesseldorf http://www.OpenIT.de
Re: another question, and a small patch [ In reply to ]
nisse at lysator
Jun 26, 2000, 11:22 AM
Post #7 of 7 (2287 views)
Permalink
Werner Koch <wk@gnupg.org> writes:

> I know about the shallow copy issues, so ref-counting seems to be so
> only easy way to cope with it.

In LSH I use a simple mark&sweep gc for all my objects, including
the sexps. But I guess that's no alternative for a library.

/Niels

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal