Mailing List Archive

Hello Dirk,

> now again a month went by and I still had no real answer if the code
> I sent will be applied or not and in case it will not: Why?

first of all, I would like to apologize for not taking part in this
discussion and/or giving appropriate answers. I will try to do so now.

In general I think what your patch tries to implement is a good thing:
it tries to give the user the option to release resources, which would
be lost otherwise. But this is hard to implement in Libgcrypt.

One should also keep in mind that what you need to do in respect to the
library resources is probably not something that a majority of users of
Libgcrypt need to do. Therefore it is a special-need and one COULD
argue that users with such special-needs should rip the library apart
and modify it to their needs. I think, in this context one should take
the amount of work necessary for modifying the library in such a way
into account.

Now lets become concrete.

The main problem I see here is that this concept of "bringing the
library back to initial state" is not something that has been thought of
so far during the development of Libgcrypt; I am quite sure that it is
actually more complicated than your patch tries to make it look like. :)

Examples:
* ath: your patch does not bring the ath code back to initial state
(the callbacks would not be reset, etc)
* secmem: the pool of secure memory is not touched by your patch either.
* global: debug flags and handlers are not reset

These things could be fixed, yes. But it would be more work then just
releasing the cipher/md/pubkey modules and the PRNG pool.

Let me think about the integration of your patch in the official
Libgcrypt tree again; it is kind of hairy.

I would like to point your attention to my personal, bleeding-edge
development branch of Libgcrypt; you can check it out through:

svn co svn://cvs.gnupg.org/libgcrypt/branches/LIBGCRYPT-2.0-MO libgcrypt

The tree contains a file named `DESIGN', which explains the main
differences between the official version and this highly-experimental
(!) and entirely unofficial version.

I think what you need is something that my hacked Libgcrypt branch is
supposed to provide. Of course, this is branch is rather uninteresting
to you in case you simply want this functionality in the official,
soon-to-release branch. But it might be interesting to you, in case you
want to benefit from the functionality provided by this experimental
branch without seeing the lack of `official support' and/or a stable API
as a fundamental disadvantage.

In case something related to that code is unclear to you, do not
hesitate to contact me by private mail..

Thanks & happy hacking,
Moritz.

Hello,

> In general I think what your patch tries to implement is a good thing:
> it tries to give the user the option to release resources, which would
> be lost otherwise. But this is hard to implement in Libgcrypt.

Not as hard as you think. Ok, it would be much easier to do so, when done
from the first piece of code. Nevertheless, usually every part of
initialization stuff only depends one nearly singular variables. So it is
not necessary to care for everything, but only relase everything and reset
these singular pieces (usually a "bool initialized").

The stuff I implemented works for the methods I used the library for (and
some more, as I tried to care for other possible fields as well). If
it is in the main tree and there are problems in other areas they will
soon be bug-reported and fixed. Also a short code review from the
developers could reduce the bug-reporting stage a lot.

I would also help with this, but not as long as I see no step is done in
this direction.

I'm doing programming for more than 15 years including more than 4 totally
different operating systems and memory models, some processor types and
so on. So I have a bit of experience in these fields. I sometimes heard
the "This does not work" or "This is hard to do" (and also said this
sometimes myself), but usually it is not that complicated as one may
think.

> Examples:
> * ath: your patch does not bring the ath code back to initial state
> (the callbacks would not be reset, etc)
> * secmem: the pool of secure memory is not touched by your patch either.
> * global: debug flags and handlers are not reset
>
> These things could be fixed, yes. But it would be more work then just
> releasing the cipher/md/pubkey modules and the PRNG pool.

Usually it is something like:
- Release everything
- Reset import global variables.

I think this is easy and straightforward (Thought it would be easier
without any globals :-).

BTW: It is also a necessary in case someone wants the software to be
ported to different operating systems. Not every operating system has an
memory garbage collection.

> I think what you need is something that my hacked Libgcrypt branch is
> supposed to provide. Of course, this is branch is rather uninteresting
> to you in case you simply want this functionality in the official,

That's the problem. I need a certain functionality and libgcrypt provides
it. But I also need a clean system and libgcrypt does not provide it. For
me there are two possibilities. Either I get the cleanup implemented in
libgcrypt or I need to switch to another solution (possibly including a
private reduced link-library based on libgcrypt). A experimental
development branch does not help. I can spend some work on the project,
but not too much, as it is not a major part of the things I do.

Ciao
--
____ _ _ ____ _ _ _ _ ____
| | | | | | \ / | | | the cool Gremlin from Bischofswerda
| __ | ____| | \/ | | | WWW: http://www.dstoecker.de/
| | | | | | | | PGP key available on www page.
|____| _|_ |____| _|_ _|_ |____| I hope AMIGA never stops making fun!

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Sun, 9 Oct 2005 18:45:15 +0200 (CEST), Dirk Stoecker said:

> so on. So I have a bit of experience in these fields. I sometimes heard
> the "This does not work" or "This is hard to do" (and also said this
> sometimes myself), but usually it is not that complicated as one may
> think.

It is not a matter whether it is complicated or not. We care about the
code complexity - each and every line adds to it. If you provide ways
to make the code less complex by keeping the API, it is more likely
that that this will make it into Libgcrypt.

Sorry, I see no way to apply a patch for the sake of Valgrind.


Salam-Shalom,

Werner




_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Mon, 10 Oct 2005, Werner Koch wrote:

> > so on. So I have a bit of experience in these fields. I sometimes heard
> > the "This does not work" or "This is hard to do" (and also said this
> > sometimes myself), but usually it is not that complicated as one may
> > think.
>
> It is not a matter whether it is complicated or not. We care about the
> code complexity - each and every line adds to it. If you provide ways
> to make the code less complex by keeping the API, it is more likely
> that that this will make it into Libgcrypt.

Caring for totally abstract coding styles more then for actual
implementation errors is a strange way to build security relevant
software. Securely freeing and cleaning resources is usually an important
part of cryptographic software solutions.

> Sorry, I see no way to apply a patch for the sake of Valgrind.

Well then, that's a word.
I will extract the parts of libgcrypt I need and build a reduced
LGPL-link-library from this. I will not bother this group any more with
such strange ideas like clean software design.

Ciao
--
____ _ _ ____ _ _ _ _ ____
| | | | | | \ / | | | the cool Gremlin from Bischofswerda
| __ | ____| | \/ | | | WWW: http://www.dstoecker.de/
| | | | | | | | PGP key available on www page.
|____| _|_ |____| _|_ _|_ |____| I hope AMIGA never stops making fun!

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tue, 11 Oct 2005 10:26:37 +0200 (CEST), Dirk Stoecker said:

> implementation errors is a strange way to build security relevant
> software. Securely freeing and cleaning resources is usually an important
> part of cryptographic software solutions.

Please understand that these are one time allocations. Consider them
part of the DSO loading process.


Salam-Shalom,

Werner




_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tuesday 11 October 2005 12:06, Werner Koch wrote:

> > implementation errors is a strange way to build security relevant
> > software. Securely freeing and cleaning resources is usually an important
> > part of cryptographic software solutions.
> Please understand that these are one time allocations. Consider them
> part of the DSO loading process.

I tend to agree with Dirk on that matter. If a program uses 10 shared
libraries that do not deallocate their resources, checking for memory leaks
in the program is a disaster.


--
Nikos Mavrogiannopoulos

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tuesday 11 October 2005 03:25, Nikos Mavrogiannopoulos wrote:
> On Tuesday 11 October 2005 12:06, Werner Koch wrote:
> > > implementation errors is a strange way to build security relevant
> > > software. Securely freeing and cleaning resources is usually an
> > > important part of cryptographic software solutions.
> >
> > Please understand that these are one time allocations. Consider them
> > part of the DSO loading process.
>
> I tend to agree with Dirk on that matter. If a program uses 10 shared
> libraries that do not deallocate their resources, checking for memory leaks
> in the program is a disaster.

I also agree with Dirk. In particular I an thinking of problems that would
occur if the library is loaded and unloaded repeatedly (dlopen/dlclose),
causing the malloc'ed memory to live-on. Now, Dirk's explicit shutdown-call
might not be the best choice in this context, maybe a destructor / _fini_ /
whatever that is automatically triggered by the linker (ideally paired with a
matching constructor that does the allocation) would be the perfect solution
in my opinion.

my 2 cents

Christian

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tue, 11 Oct 2005 12:25:26 +0200, Nikos Mavrogiannopoulos said:

> I tend to agree with Dirk on that matter. If a program uses 10 shared
> libraries that do not deallocate their resources, checking for memory leaks
> in the program is a disaster.

That depends on your definition of a memory leak. A one time
allocation can't be a memory leak.


Shalom-Salam,

Werner


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tue, 11 Oct 2005 07:00:24 -0700, Christian Grothoff said:

> I also agree with Dirk. In particular I an thinking of problems that would
> occur if the library is loaded and unloaded repeatedly (dlopen/dlclose),
> causing the malloc'ed memory to live-on. Now, Dirk's explicit shutdown-call

You simply can't do that with libgcrypt; it has not been designed to
be runtime loaded. It is impossible for liibgcrypt to shut itself
properly down. There is no reference counting or similar to detect
resources in use by other objects.

With the same justification one could ask for libc to shut itself down
on demand. Well, of course, there is such a feature: exit(3).

Valgrind is a great tool, but it does not exempt the developer to
analyze the results. Look only on the suppress files distributed
along with vg.


Salam-Shalom,

Werner










_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel