Mailing List Archive

gpg: fatal: zlib inflate problem
I am on a Linux system (RedHat 5.1) with a Pentium 133 MHz and used
gnupg-0.4.2. I subscribed yesterday, so it is possible that these
problems are already know.

With some playing around with gpg I had some strange errors.
I was testing with some cpio backups and encrypted these with gpg.

With large files and decrypt I got some errors. I noticed if I used
the 'no commpression option' (-z0), there were no errors. Only with
z1 .. z9 were there errors. Also random data and gziped data (almost
random) gave no problems, only large text files. These errors only
came if gpg is used on files. There are no errors if gpg is used as
filter.

So I wrote a 'little' test-script:
===========================================================
#!/bin/bash

if [ ! -f pge0112.txt ] ; then
unzip pge0112.zip
fi

SIZE="5000 5100 5200 5300 5400 5500 5600 5700 5800 5900 \
6000 6100 6200 6300 6400 6500 6600 6700 6800 6900 \
7000"

for S in $SIZE ; do
dd if=pge0112.txt of=text.$S bs=1000 count=$S
done

for S in $SIZE ; do
gpg --output text.$S.gpg --remote-user cees --encrypt text.$S
gpg --passphrase-fd 3 --output text-out.$S \
--decrypt text.$S.gpg 3< passphrase

cat text.$S \
| gpg --output - --remote-user cees --encrypt \
| gpg --passphrase-fd 3 --output - --decrypt 3< passphrase \
> stdin-out.$S
done

for S in $SIZE ; do
md5sum text.$S text-out.$S stdin-out.$S
done
===========================================================
N.B.: pge0112.zip is a ziped large text-file from
http://www.gutenberg.org/.

ls -l pgw0112.*
-rw-r--r-- 1 cees users 8441343 Dec 11 1994 pge0112.txt
-rw-r--r-- 1 cees users 3483430 Nov 19 1996 pge0112.zip
===========================================================

I got mostly no errors from the size 5000 to 7000 (step 100), only
at the sizes 5100, 5200, 5700, 5900, 6000, 6200, 6400, 6800.
(8 in 21 is 26.25% [grin]). But if gpg is used as a filter, I get NO
errors.

I got no only an error at 6000 from the size 1000 to 8000 (step 1000)
with text files. The same range with random data gave no errors.

Size 5100:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

Size 5200:
gpg: fatal: zlib inflate problem: too many length or distance symbols
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

Size 5700:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

Size 5900:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

Size 6000:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

Size 6200:
gpg: fatal: zlib inflate problem: incomplete dynamic bit lengths tree
secmem usage: 5984/7648 bytes in 6/10 blocks of pool 7872/16384

Size 6400:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/7648 bytes in 6/10 blocks of pool 7872/16384

Size 6800:
gpg: fatal: zlib inflate problem: oversubscribed dynamic bit lengths tree
secmem usage: 5984/8224 bytes in 6/12 blocks of pool 8448/16384

My system uses zlib 1.1.2.
I downloaded the last version of gnupg (gnupg-0.4.2.tar.gz) and did a
'configure --with-included-zlib', and a quick test showed that the
problems were the same.

I still have the gpg-ed text files, so the errors can be reproduced.
(You only need my keys [grin].)

Cees van de Griend
--
Send a mail to <cees-help@griend.xs4all.nl> for more information.

Dajonlu'pa' bIHeghjaj.
May you die before you are captured.
Mark Okrand, "Star Trek: The Klingon Way - A Warrier's Guide"
Re: gpg: fatal: zlib inflate problem [ In reply to ]
Cees van de Griend <cees-list@griend.xs4all.nl> writes:

>
> With large files and decrypt I got some errors. I noticed if I used
> the 'no commpression option' (-z0), there were no errors. Only with
> z1 .. z9 were there errors. Also random data and gziped data (almost
[...]
> My system uses zlib 1.1.2.

There is a bug in zlib which is fixed in zlib 1.1.3 - I look forward
to add some configure checks for a good zlib and put 1.13. into the
distribution.

Thanks for telling this the folks here,


Werner
Re: gpg: fatal: zlib inflate problem [ In reply to ]
[.N.B. It is not my habbit to post (private) email on a mailing list,
but Werner Koch asked me to do it.]

On Wed, Nov 04, 1998 at 09:29:28PM +0100, Werner Koch wrote:
> Cees van de Griend <cees-list@griend.xs4all.nl> writes:
>
>>
>> With large files and decrypt I got some errors. I noticed if I used
>> the 'no commpression option' (-z0), there were no errors. Only with
>> z1 .. z9 were there errors. Also random data and gziped data (almost
> [...]
>> My system uses zlib 1.1.2.
>
> There is a bug in zlib which is fixed in zlib 1.1.3 - I look forward
> to add some configure checks for a good zlib and put 1.13. into the
> distribution.
>
> Thanks for telling this the folks here,

Maybe the bug-report was too large, but it was a difficult bug to
reproduce.

I just installed zlib 1.1.3 (and checked that it is loaded and not
1.1.2) but the problems are the same.

The errors are with the file made with zlib 1.1.2 and read by
zlib 1.1.3, also made with 1.1.3 and read by 1.1.3.

So, the bug is still there.

-----------------------------------------------------------
#!/bin/bash

if [ ! -f pge0112.txt ] ; then
unzip pge0112.zip
fi

SIZE="5500 5600 5700 5800 5900 6000 6100 6200 6300 6400 6500"

for S in $SIZE ; do
dd if=pge0112.txt of=text.$S.in bs=1000 count=$S

gpg --output text.$S.gpg --remote-user cees \
--encrypt text.$S.in
gpg --passphrase-fd 3 --output text.$S.out \
--decrypt text.$S.gpg 3< passphrase

rm -f text.$S.gpg
done
-----------------------------------------------------------
$ls -l text.*
-rw-r--r-- 1 cees users 5500000 Nov 5 00:04 text.5500.in
-rw-r--r-- 1 cees users 5500000 Nov 5 00:05 text.5500.out
-rw-r--r-- 1 cees users 5600000 Nov 5 00:05 text.5600.in
-rw-r--r-- 1 cees users 3588073 Nov 5 00:07 text.5600.out
-rw-r--r-- 1 cees users 5700000 Nov 5 00:07 text.5700.in
-rw-r--r-- 1 cees users 5700000 Nov 5 00:08 text.5700.out
-rw-r--r-- 1 cees users 5800000 Nov 5 00:08 text.5800.in
-rw-r--r-- 1 cees users 5800000 Nov 5 00:10 text.5800.out
-rw-r--r-- 1 cees users 5900000 Nov 5 00:10 text.5900.in
-rw-r--r-- 1 cees users 3588073 Nov 5 00:11 text.5900.out
-rw-r--r-- 1 cees users 6000000 Nov 5 00:11 text.6000.in
-rw-r--r-- 1 cees users 6000000 Nov 5 00:13 text.6000.out
-rw-r--r-- 1 cees users 6100000 Nov 5 00:13 text.6100.in
-rw-r--r-- 1 cees users 6100000 Nov 5 00:14 text.6100.out
-rw-r--r-- 1 cees users 6200000 Nov 5 00:15 text.6200.in
-rw-r--r-- 1 cees users 3588073 Nov 5 00:16 text.6200.out
-rw-r--r-- 1 cees users 6300000 Nov 5 00:16 text.6300.in
-rw-r--r-- 1 cees users 6300000 Nov 5 00:18 text.6300.out
-rw-r--r-- 1 cees users 6400000 Nov 5 00:18 text.6400.in
-rw-r--r-- 1 cees users 6400000 Nov 5 00:19 text.6400.out
-rw-r--r-- 1 cees users 6500000 Nov 5 00:19 text.6500.in
-rw-r--r-- 1 cees users 6500000 Nov 5 00:21 text.6500.out
-----------------------------------------------------------

>
> Werner
>

Greetings,
Cees.
--
Send a mail to <cees-help@griend.xs4all.nl> for more information.

DabuQlu'DI' yISuv.
When threatened, fight.
Mark Okrand, "Star Trek: The Klingon Way - A Warrier's Guide"
Re: gpg: fatal: zlib inflate problem [ In reply to ]
I have done some more 'research'.

There were no errors with 'minigzip' from zlib-1.0.3, so I presume the
errors are not in libz.so but in gpg.

Also there were no errors with a 1024 bits key, only with a 2048 bits
key and large text files.

Cees.
--
Send a mail to <cees-help@griend.xs4all.nl> for more information.

not lay'Ha' thlhIngan.
No Klingon ever breaks his word.
Mark Okrand, "Star Trek: The Klingon Way - A Warrier's Guide"
Re: gpg: fatal: zlib inflate problem [ In reply to ]
Cees van de Griend <cees-list@griend.xs4all.nl> writes:

> There were no errors with 'minigzip' from zlib-1.0.3, so I presume the
> errors are not in libz.so but in gpg.
>
> Also there were no errors with a 1024 bits key, only with a 2048 bits
> key and large text files.

The key is not compressed, and the session key used is 128 or 168
bits. So it is a matter of the message size. The problem may be
that gpg does not gve constant chunks of data to zlib but may provide
some hundered bytes and then some K. I assume there is a problem in
the buffering code (either in GnuPG or zlib).