Mailing List Archive

Switching md5 source
I was thinking about the Opteron / MD5 problem, and went Googling to see what
I could see. I found a note in the changelog of Curl about some MD5 and 64-bit
something that was fixed in a late December version of Curl.

Curl uses the RSA implementation of MD5, which carries a notice that derived
works must identify themselves as being derived from RSA MD5 Message-Digest
Algorithm. Since we're not deriving from it, but rather building with it, this
probably does not apply to us. Curl's license is basically MIT -- use it in
any way as long as the notice is kept and no warranty is assumed.

So IC&S guys, you might need to ask your lawyer about using the RSA
implementation, but for the moment I'd suggest that we try it and see if it works!

Aaron
Re: Switching md5 source [ In reply to ]
Hi,

Wouldn't it be easier to just use the md5 stuff from libcrypto/OpenSSL?
If we eventually want to built an IMAPS/POP3S server, without using
stunnel, we'll need to depend on OpenSSL anyway.

I've done an implementation using the MD5 stuff from OpenSSL, and it's
really easy. (about the same number of lines as in the current
implementation, but without the magic number of 33 for buffer length :).

There are some licensing issues with OpenSSL with regard to linking with
GPL programs, but I don't think that they are important for us. There is
no problem linking to OpenSSL on platforms where SSL is part of the
distribution. Since (almost) every Linux distro and *BSD uses OpenSSH
(which is linked to OpenSSL :) ), it is part of all those distributions,
which makes linking to it no issue.

My idea is that it is a bit of waste to implement our own code (even if
we 'borrow' it), if there's a perfectly good implementation that we can
use, especially if it uses a library that almost everybody out there
already uses in some way.

Any other ideas about this? Please tell me if you completely (or partly)
disagree with me! ;)

Ilja



Aaron Stone wrote:

> I was thinking about the Opteron / MD5 problem, and went Googling to see what
> I could see. I found a note in the changelog of Curl about some MD5 and 64-bit
> something that was fixed in a late December version of Curl.
>
> Curl uses the RSA implementation of MD5, which carries a notice that derived
> works must identify themselves as being derived from RSA MD5 Message-Digest
> Algorithm. Since we're not deriving from it, but rather building with it, this
> probably does not apply to us. Curl's license is basically MIT -- use it in
> any way as long as the notice is kept and no warranty is assumed.
>
> So IC&S guys, you might need to ask your lawyer about using the RSA
> implementation, but for the moment I'd suggest that we try it and see if it works!
>
> Aaron
>
>
>
>
> ------------------------------------------------------------------------
>
> #ifndef __MD5_H
> #define __MD5_H
> /***************************************************************************
> * _ _ ____ _
> * Project ___| | | | _ \| |
> * / __| | | | |_) | |
> * | (__| |_| | _ <| |___
> * \___|\___/|_| \_\_____|
> *
> * Copyright (C) 1998 - 2004, Daniel Stenberg, <daniel@haxx.se>, et al.
> *
> * This software is licensed as described in the file COPYING, which
> * you should have received as part of this distribution. The terms
> * are also available at http://curl.haxx.se/docs/copyright.html.
> *
> * You may opt to use, copy, modify, merge, publish, distribute and/or sell
> * copies of the Software, and permit persons to whom the Software is
> * furnished to do so, under the terms of the COPYING file.
> *
> * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
> * KIND, either express or implied.
> *
> * $Id: md5.h,v 1.3 2004/01/07 09:19:35 bagder Exp $
> ***************************************************************************/
>
> void Curl_md5it(unsigned char *output,
> unsigned char *input);
>
> #endif
>
>
> ------------------------------------------------------------------------
>
> /***************************************************************************
> * _ _ ____ _
> * Project ___| | | | _ \| |
> * / __| | | | |_) | |
> * | (__| |_| | _ <| |___
> * \___|\___/|_| \_\_____|
> *
> * Copyright (C) 1998 - 2004, Daniel Stenberg, <daniel@haxx.se>, et al.
> *
> * This software is licensed as described in the file COPYING, which
> * you should have received as part of this distribution. The terms
> * are also available at http://curl.haxx.se/docs/copyright.html.
> *
> * You may opt to use, copy, modify, merge, publish, distribute and/or sell
> * copies of the Software, and permit persons to whom the Software is
> * furnished to do so, under the terms of the COPYING file.
> *
> * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
> * KIND, either express or implied.
> *
> * $Id: md5.c,v 1.4 2004/01/07 09:19:35 bagder Exp $
> ***************************************************************************/
>
> #include "setup.h"
>
> #ifndef USE_SSLEAY
> /* This code segment is only used if OpenSSL is not provided, as if it is
> we use the MD5-function provided there instead. No good duplicating
> code! */
>
> /* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
> rights reserved.
>
> License to copy and use this software is granted provided that it
> is identified as the "RSA Data Security, Inc. MD5 Message-Digest
> Algorithm" in all material mentioning or referencing this software
> or this function.
>
> License is also granted to make and use derivative works provided
> that such works are identified as "derived from the RSA Data
> Security, Inc. MD5 Message-Digest Algorithm" in all material
> mentioning or referencing the derived work.
>
> RSA Data Security, Inc. makes no representations concerning either
> the merchantability of this software or the suitability of this
> software for any particular purpose. It is provided "as is"
> without express or implied warranty of any kind.
>
> These notices must be retained in any copies of any part of this
> documentation and/or software.
> */
>
> #include <string.h>
>
> /* UINT4 defines a four byte word */
> typedef unsigned int UINT4;
>
> /* MD5 context. */
> struct md5_ctx {
> UINT4 state[4]; /* state (ABCD) */
> UINT4 count[2]; /* number of bits, modulo 2^64 (lsb first) */
> unsigned char buffer[64]; /* input buffer */
> };
>
> typedef struct md5_ctx MD5_CTX;
>
> static void MD5_Init(struct md5_ctx *);
> static void MD5_Update(struct md5_ctx *, unsigned char *, unsigned int);
> static void MD5_Final(unsigned char [16], struct md5_ctx *);
>
> /* Constants for MD5Transform routine.
> */
>
> #define S11 7
> #define S12 12
> #define S13 17
> #define S14 22
> #define S21 5
> #define S22 9
> #define S23 14
> #define S24 20
> #define S31 4
> #define S32 11
> #define S33 16
> #define S34 23
> #define S41 6
> #define S42 10
> #define S43 15
> #define S44 21
>
> static void MD5Transform(UINT4 [4], unsigned char [64]);
> static void Encode(unsigned char *, UINT4 *, unsigned int);
> static void Decode(UINT4 *, unsigned char *, unsigned int);
>
> #define MD5_memcpy(dst,src,len) memcpy(dst,src,len)
> #define MD5_memset(dst,val,len) memset(dst,val,len)
>
> static unsigned char PADDING[64] = {
> 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
> };
>
> /* F, G, H and I are basic MD5 functions.
> */
> #define F(x, y, z) (((x) & (y)) | ((~x) & (z)))
> #define G(x, y, z) (((x) & (z)) | ((y) & (~z)))
> #define H(x, y, z) ((x) ^ (y) ^ (z))
> #define I(x, y, z) ((y) ^ ((x) | (~z)))
>
> /* ROTATE_LEFT rotates x left n bits.
> */
> #define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32-(n))))
>
> /* FF, GG, HH, and II transformations for rounds 1, 2, 3, and 4.
> Rotation is separate from addition to prevent recomputation.
> */
> #define FF(a, b, c, d, x, s, ac) { \
> (a) += F ((b), (c), (d)) + (x) + (UINT4)(ac); \
> (a) = ROTATE_LEFT ((a), (s)); \
> (a) += (b); \
> }
> #define GG(a, b, c, d, x, s, ac) { \
> (a) += G ((b), (c), (d)) + (x) + (UINT4)(ac); \
> (a) = ROTATE_LEFT ((a), (s)); \
> (a) += (b); \
> }
> #define HH(a, b, c, d, x, s, ac) { \
> (a) += H ((b), (c), (d)) + (x) + (UINT4)(ac); \
> (a) = ROTATE_LEFT ((a), (s)); \
> (a) += (b); \
> }
> #define II(a, b, c, d, x, s, ac) { \
> (a) += I ((b), (c), (d)) + (x) + (UINT4)(ac); \
> (a) = ROTATE_LEFT ((a), (s)); \
> (a) += (b); \
> }
>
> /* MD5 initialization. Begins an MD5 operation, writing a new context.
> */
> static void MD5_Init (context)
> struct md5_ctx *context; /* context */
> {
> context->count[0] = context->count[1] = 0;
> /* Load magic initialization constants.
> */
> context->state[0] = 0x67452301;
> context->state[1] = 0xefcdab89;
> context->state[2] = 0x98badcfe;
> context->state[3] = 0x10325476;
> }
>
> /* MD5 block update operation. Continues an MD5 message-digest
> operation, processing another message block, and updating the
> context.
> */
> static void MD5_Update (context, input, inputLen)
> struct md5_ctx *context; /* context */
> unsigned char *input; /* input block */
> unsigned int inputLen; /* length of input block */
> {
> unsigned int i, index, partLen;
>
> /* Compute number of bytes mod 64 */
> index = (unsigned int)((context->count[0] >> 3) & 0x3F);
>
> /* Update number of bits */
> if ((context->count[0] += ((UINT4)inputLen << 3))
> < ((UINT4)inputLen << 3))
> context->count[1]++;
> context->count[1] += ((UINT4)inputLen >> 29);
>
> partLen = 64 - index;
>
> /* Transform as many times as possible. */
> if (inputLen >= partLen) {
> MD5_memcpy((void *)&context->buffer[index], (void *)input, partLen);
> MD5Transform(context->state, context->buffer);
>
> for (i = partLen; i + 63 < inputLen; i += 64)
> MD5Transform(context->state, &input[i]);
>
> index = 0;
> }
> else
> i = 0;
>
> /* Buffer remaining input */
> MD5_memcpy((void *)&context->buffer[index], (void *)&input[i],
> inputLen-i);
> }
>
> /* MD5 finalization. Ends an MD5 message-digest operation, writing the
> the message digest and zeroizing the context.
> */
> static void MD5_Final (digest, context)
> unsigned char digest[16]; /* message digest */
> struct md5_ctx *context; /* context */
> {
> unsigned char bits[8];
> unsigned int index, padLen;
>
> /* Save number of bits */
> Encode (bits, context->count, 8);
>
> /* Pad out to 56 mod 64. */
> index = (unsigned int)((context->count[0] >> 3) & 0x3f);
> padLen = (index < 56) ? (56 - index) : (120 - index);
> MD5_Update (context, PADDING, padLen);
>
> /* Append length (before padding) */
> MD5_Update (context, bits, 8);
>
> /* Store state in digest */
> Encode (digest, context->state, 16);
>
> /* Zeroize sensitive information. */
> MD5_memset ((void *)context, 0, sizeof (*context));
> }
>
> /* MD5 basic transformation. Transforms state based on block. */
> static void MD5Transform (state, block)
> UINT4 state[4];
> unsigned char block[64];
> {
> UINT4 a = state[0], b = state[1], c = state[2], d = state[3], x[16];
>
> Decode (x, block, 64);
>
> /* Round 1 */
> FF (a, b, c, d, x[ 0], S11, 0xd76aa478); /* 1 */
> FF (d, a, b, c, x[ 1], S12, 0xe8c7b756); /* 2 */
> FF (c, d, a, b, x[ 2], S13, 0x242070db); /* 3 */
> FF (b, c, d, a, x[ 3], S14, 0xc1bdceee); /* 4 */
> FF (a, b, c, d, x[ 4], S11, 0xf57c0faf); /* 5 */
> FF (d, a, b, c, x[ 5], S12, 0x4787c62a); /* 6 */
> FF (c, d, a, b, x[ 6], S13, 0xa8304613); /* 7 */
> FF (b, c, d, a, x[ 7], S14, 0xfd469501); /* 8 */
> FF (a, b, c, d, x[ 8], S11, 0x698098d8); /* 9 */
> FF (d, a, b, c, x[ 9], S12, 0x8b44f7af); /* 10 */
> FF (c, d, a, b, x[10], S13, 0xffff5bb1); /* 11 */
> FF (b, c, d, a, x[11], S14, 0x895cd7be); /* 12 */
> FF (a, b, c, d, x[12], S11, 0x6b901122); /* 13 */
> FF (d, a, b, c, x[13], S12, 0xfd987193); /* 14 */
> FF (c, d, a, b, x[14], S13, 0xa679438e); /* 15 */
> FF (b, c, d, a, x[15], S14, 0x49b40821); /* 16 */
>
> /* Round 2 */
> GG (a, b, c, d, x[ 1], S21, 0xf61e2562); /* 17 */
> GG (d, a, b, c, x[ 6], S22, 0xc040b340); /* 18 */
> GG (c, d, a, b, x[11], S23, 0x265e5a51); /* 19 */
> GG (b, c, d, a, x[ 0], S24, 0xe9b6c7aa); /* 20 */
> GG (a, b, c, d, x[ 5], S21, 0xd62f105d); /* 21 */
> GG (d, a, b, c, x[10], S22, 0x2441453); /* 22 */
> GG (c, d, a, b, x[15], S23, 0xd8a1e681); /* 23 */
> GG (b, c, d, a, x[ 4], S24, 0xe7d3fbc8); /* 24 */
> GG (a, b, c, d, x[ 9], S21, 0x21e1cde6); /* 25 */
> GG (d, a, b, c, x[14], S22, 0xc33707d6); /* 26 */
> GG (c, d, a, b, x[ 3], S23, 0xf4d50d87); /* 27 */
> GG (b, c, d, a, x[ 8], S24, 0x455a14ed); /* 28 */
> GG (a, b, c, d, x[13], S21, 0xa9e3e905); /* 29 */
> GG (d, a, b, c, x[ 2], S22, 0xfcefa3f8); /* 30 */
> GG (c, d, a, b, x[ 7], S23, 0x676f02d9); /* 31 */
> GG (b, c, d, a, x[12], S24, 0x8d2a4c8a); /* 32 */
>
> /* Round 3 */
> HH (a, b, c, d, x[ 5], S31, 0xfffa3942); /* 33 */
> HH (d, a, b, c, x[ 8], S32, 0x8771f681); /* 34 */
> HH (c, d, a, b, x[11], S33, 0x6d9d6122); /* 35 */
> HH (b, c, d, a, x[14], S34, 0xfde5380c); /* 36 */
> HH (a, b, c, d, x[ 1], S31, 0xa4beea44); /* 37 */
> HH (d, a, b, c, x[ 4], S32, 0x4bdecfa9); /* 38 */
> HH (c, d, a, b, x[ 7], S33, 0xf6bb4b60); /* 39 */
> HH (b, c, d, a, x[10], S34, 0xbebfbc70); /* 40 */
> HH (a, b, c, d, x[13], S31, 0x289b7ec6); /* 41 */
> HH (d, a, b, c, x[ 0], S32, 0xeaa127fa); /* 42 */
> HH (c, d, a, b, x[ 3], S33, 0xd4ef3085); /* 43 */
> HH (b, c, d, a, x[ 6], S34, 0x4881d05); /* 44 */
> HH (a, b, c, d, x[ 9], S31, 0xd9d4d039); /* 45 */
> HH (d, a, b, c, x[12], S32, 0xe6db99e5); /* 46 */
> HH (c, d, a, b, x[15], S33, 0x1fa27cf8); /* 47 */
> HH (b, c, d, a, x[ 2], S34, 0xc4ac5665); /* 48 */
>
> /* Round 4 */
> II (a, b, c, d, x[ 0], S41, 0xf4292244); /* 49 */
> II (d, a, b, c, x[ 7], S42, 0x432aff97); /* 50 */
> II (c, d, a, b, x[14], S43, 0xab9423a7); /* 51 */
> II (b, c, d, a, x[ 5], S44, 0xfc93a039); /* 52 */
> II (a, b, c, d, x[12], S41, 0x655b59c3); /* 53 */
> II (d, a, b, c, x[ 3], S42, 0x8f0ccc92); /* 54 */
> II (c, d, a, b, x[10], S43, 0xffeff47d); /* 55 */
> II (b, c, d, a, x[ 1], S44, 0x85845dd1); /* 56 */
> II (a, b, c, d, x[ 8], S41, 0x6fa87e4f); /* 57 */
> II (d, a, b, c, x[15], S42, 0xfe2ce6e0); /* 58 */
> II (c, d, a, b, x[ 6], S43, 0xa3014314); /* 59 */
> II (b, c, d, a, x[13], S44, 0x4e0811a1); /* 60 */
> II (a, b, c, d, x[ 4], S41, 0xf7537e82); /* 61 */
> II (d, a, b, c, x[11], S42, 0xbd3af235); /* 62 */
> II (c, d, a, b, x[ 2], S43, 0x2ad7d2bb); /* 63 */
> II (b, c, d, a, x[ 9], S44, 0xeb86d391); /* 64 */
>
> state[0] += a;
> state[1] += b;
> state[2] += c;
> state[3] += d;
>
> /* Zeroize sensitive information. */
> MD5_memset ((void *)x, 0, sizeof (x));
> }
>
> /* Encodes input (UINT4) into output (unsigned char). Assumes len is
> a multiple of 4.
> */
> static void Encode (unsigned char *output,
> UINT4 *input,
> unsigned int len)
> {
> unsigned int i, j;
>
> for (i = 0, j = 0; j < len; i++, j += 4) {
> output[j] = (unsigned char)(input[i] & 0xff);
> output[j+1] = (unsigned char)((input[i] >> 8) & 0xff);
> output[j+2] = (unsigned char)((input[i] >> 16) & 0xff);
> output[j+3] = (unsigned char)((input[i] >> 24) & 0xff);
> }
> }
>
> /* Decodes input (unsigned char) into output (UINT4). Assumes len is
> a multiple of 4.
> */
> static void Decode (UINT4 *output,
> unsigned char *input,
> unsigned int len)
> {
> unsigned int i, j;
>
> for (i = 0, j = 0; j < len; i++, j += 4)
> output[i] = ((UINT4)input[j]) | (((UINT4)input[j+1]) << 8) |
> (((UINT4)input[j+2]) << 16) | (((UINT4)input[j+3]) << 24);
> }
>
> #else
> /* If OpenSSL is present */
> #include <openssl/md5.h>
> #include <string.h>
> #endif
>
>
> void Curl_md5it(unsigned char *outbuffer, /* 16 bytes */
> unsigned char *input)
> {
> MD5_CTX ctx;
> MD5_Init(&ctx);
> MD5_Update(&ctx, input, strlen((char *)input));
> MD5_Final(outbuffer, &ctx);
> }
Re: Switching md5 source [ In reply to ]
On Mon, 2004-02-16 at 07:32, Ilja Booij wrote:
> There are some licensing issues with OpenSSL with regard to linking with
> GPL programs, but I don't think that they are important for us. There is
> no problem linking to OpenSSL on platforms where SSL is part of the
> distribution. Since (almost) every Linux distro and *BSD uses OpenSSH
> (which is linked to OpenSSL :) ), it is part of all those distributions,
> which makes linking to it no issue.
>

gnutls is another option, it would allow implementation of pop3s/imaps.
Not sure if it has a md5 implementation with it, just saying there are
alternatives to openssl if the licensing is a concern.

Ryan Butler
ADI Internet Solutions
rbutler@adiis.net
Re: Switching md5 source [ In reply to ]
Ryan Butler wrote:

>On Mon, 2004-02-16 at 07:32, Ilja Booij wrote:
>
>
>>There are some licensing issues with OpenSSL with regard to linking with
>>GPL programs, but I don't think that they are important for us. There is
>>no problem linking to OpenSSL on platforms where SSL is part of the
>>distribution. Since (almost) every Linux distro and *BSD uses OpenSSH
>>(which is linked to OpenSSL :) ), it is part of all those distributions,
>>which makes linking to it no issue.
>>
>>
>>
>
>gnutls is another option, it would allow implementation of pop3s/imaps.
>Not sure if it has a md5 implementation with it, just saying there are
>alternatives to openssl if the licensing is a concern.
>
>Ryan Butler
>ADI Internet Solutions
>rbutler@adiis.net
>
>
>_______________________________________________
>Dbmail-dev mailing list
>Dbmail-dev@dbmail.org
>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
>
Does anyone have anything concrete about the MD5 algorithm? I'd be happy
to write up a C function to calculate the MD5 hash of an arbitrary
amount of data given that information.

Regards,

Chris
Re: Switching md5 source [ In reply to ]
Chris Nolan wrote:
> Ryan Butler wrote:
>>
>> gnutls is another option, it would allow implementation of
>> pop3s/imaps. Not sure if it has a md5 implementation with it, just
>> saying there are
>> alternatives to openssl if the licensing is a concern.
> Does anyone have anything concrete about the MD5 algorithm? I'd be happy
> to write up a C function to calculate the MD5 hash of an arbitrary
> amount of data given that information.

There are enough implementations of MD5 floating around, so we do not
need another implementation. Besides, look at the current code for MD5
in DBMail, or any other code for it, and then decide if you still want
to code this :)

Using gnutls is also an option. I believe it also supports MD5. I don't
know if it's widely supported though. It is present in debian stable
(which is what we use here), and it's also in OpenBSD ports. I've never
used GNUTLS, but I guess programming it will be as easy as programming
for OpenSSL. I'll have a try later on :) .

Ilja
Re: Switching md5 source [ In reply to ]
The code from Curl, incidentally, is self-conditional (though I'm sure it may
require some autoconf or Makefile hinting). If OpenSSL is present it uses it,
and if not it has a RSA as a backup.

I'm not terribly comfortable with making OpenSSL or GNUTLS mandatory; some
people may simply not have a need for secure IMAP. I know that I don't because
I only bind DBMail to localhost and then use TWIG as a webmail interface to it
(I thought that I wanted to write a direct access layer for TWIG, but then
realized that by using IMAP I was shielded from the database changes in
2.0pre). For someone building a webmail system like this, SSL/TLS makes no
sense because for a cracker to see the unencrypted traffic they'd have to own
the machine in the first place.

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Chris Nolan wrote:
> > Ryan Butler wrote:
> >>
> >> gnutls is another option, it would allow implementation of
> >> pop3s/imaps. Not sure if it has a md5 implementation with it, just
> >> saying there are
> >> alternatives to openssl if the licensing is a concern.
> > Does anyone have anything concrete about the MD5 algorithm? I'd be happy
> > to write up a C function to calculate the MD5 hash of an arbitrary
> > amount of data given that information.
>
> There are enough implementations of MD5 floating around, so we do not
> need another implementation. Besides, look at the current code for MD5
> in DBMail, or any other code for it, and then decide if you still want
> to code this :)
>
> Using gnutls is also an option. I believe it also supports MD5. I don't
> know if it's widely supported though. It is present in debian stable
> (which is what we use here), and it's also in OpenBSD ports. I've never
> used GNUTLS, but I guess programming it will be as easy as programming
> for OpenSSL. I'll have a try later on :) .
>
> Ilja
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--