Mailing List Archive

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #4 from Jeremy Harris <jgh146exb@wizmail.org> ---
If it's repeatable, run it with debug (*) enabled. If the trace shows that
it's doing some form of regular-expression operatin at the time
a) raise a bug against PCRE2
b) work out how to change your config to avoid that specific RE


*) Commandline -d+all option, or a ACL control=debug. Both are in
the standard docs.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #5 from Niels Kobschaetzki <niels@kobschaetzki.net> ---
I think I found my problem - exim does a check_dir_size for every mail and in
mailboxes with loads of mails in the inbox the problem appears:

check_dir_size: size from
1655541855.M648217P82718.server1.tld,S=42289,W=42895:2,S is 42289
check_dir_size: size from
1670744725.H546295P50932.longisland.snafu.de,S=3020:2,S is 3020
check_dir_size: size from
1655556669.M540692P82718.server1.tld,S=600481,W=608709:2,S is 600481
check_dir_size: size from
1655551355.M261543P82718.server1.tld,S=20693,W=21014:2,RS is 20693
check_dir_size: size from
1655549485.M157286P82718.server2.tld,S=19999,W=20333:2,S is 19999
check_dir_size: size from
1655186998.M685199P49010.server1.tld,S=105516,W=107312:2,S is 105516
LOG: MAIN PANIC DIE
bad internal_store_malloc request (2147483632 bytes) from function_store_get
66
search_tidyup called

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #6 from Jeremy Harris <jgh146exb@wizmail.org> ---
Interesting:

sum += size;
DEBUG(D_transport)
debug_printf("check_dir_size: size from %s is " OFF_T_FMT "\n", name,
size);
/* pcre2_match_data_free(md); gen ctx needs no free */

I wonder if that free really is needed, for PCRE2's internal health
as opposed to the allocator it calls?

Can you build from source? Also, how many mails were there in that dir?

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #7 from Niels Kobschaetzki <niels@kobschaetzki.net> ---
I am building from source (FreeBSD - ports). I need to create patch-files
though
In that mailbox are currently 95k mails with around 15GB in size

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On Wed, 3 Jan 2024, Exim Bugzilla via Exim-dev wrote:

> https://bugs.exim.org/show_bug.cgi?id=3047
>
> --- Comment #6 from Jeremy Harris <jgh146exb@wizmail.org> ---
> Interesting:
>
> sum += size;
> DEBUG(D_transport)
> debug_printf("check_dir_size: size from %s is " OFF_T_FMT "\n", name,
> size);
> /* pcre2_match_data_free(md); gen ctx needs no free */
>
> I wonder if that free really is needed, for PCRE2's internal health
> as opposed to the allocator it calls?
>
> Can you build from source? Also, how many mails were there in that dir?

Do we know whether PCRE2 is 64-bit clean ?

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #8 from Niels Kobschaetzki <niels@kobschaetzki.net> ---
The problem is this transport in my case:

local_virtual_delivery:
driver = appendfile
maildir_format
delivery_date_add
envelope_to_add
return_path_add
directory = LDAP_TMAILBOX
quota = ${if exists {${home}/.exim_quota} {${lookup {quotasize} lsearch
{${home}/.exim_quota} {$value} {1000M}}} {${lookup {$domain:quotasize} lsearch
{VIRTUAL_DOMAINS_FILE} {$value} {1000M}}}}
quota_filecount = 400000
quota_is_inclusive = false
maildir_tag = ,S=$message_size
quota_size_regex = ,S=(\d+)
maildir_use_size_file = true
user = pop
group = exim
mode = 0660
directory_mode = 0770
#headers_add = ${if $h_}
headers_remove = "x-uidl:x-status:status"
headers_add = X-VISP-UniqueID:
${lc:${sha1:$local_part@$domain|$message_id|$sender_address|$tod_full}}


When I remove

quota_size_regex = ,S=(\d+)

the delivery works

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #9 from Niels Kobschaetzki <niels@kobschaetzki.net> ---
(In reply to Jeremy Harris from comment #6)
> Interesting:
>
> sum += size;
> DEBUG(D_transport)
> debug_printf("check_dir_size: size from %s is " OFF_T_FMT "\n",
> name,
> size);
> /* pcre2_match_data_free(md); gen ctx needs no free */
>
> I wonder if that free really is needed, for PCRE2's internal health
> as opposed to the allocator it calls?
>
> Can you build from source? Also, how many mails were there in that dir?

Any suggestions what I could do to try to fix the problem?

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #10 from Jeremy Harris <jgh146exb@wizmail.org> ---
(In reply to Niels Kobschaetzki from comment #9)
> Any suggestions what I could do to try to fix the problem?

Uncomment that line.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #11 from Niels Kobschaetzki <niels@kobschaetzki.net> ---
That doesn't help. It dies at the same place

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #12 from Jeremy Harris <jgh146exb@wizmail.org> ---
Then I think you should raise a bug against the pcre2 library, for requesting
an allocation of a negative number of bytes from an allocator call registered
with it.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

Andrew Aitchison <exim@aitchison.me.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |exim@aitchison.me.uk

--- Comment #13 from Andrew Aitchison <exim@aitchison.me.uk> ---
# git show c3d7f28680 exim.c
commit c3d7f28680dda05ca6d8415cffbff8a233c0ae74
Author: Jeremy Harris <jgh146exb@wizmail.org>
Date: Fri Jun 10 11:54:17 2022 +0100

Use pool storage for regex operations

diff --git a/src/src/exim.c b/src/src/exim.c
index eac0cb2b9..052c6bf5c 100644
--- a/src/src/exim.c
+++ b/src/src/exim.c
@@ -59,6 +59,18 @@ if (block) store_free(block);
}


+static void *
+function_store_get(PCRE2_SIZE size, void * tag)
+{
+return store_get((int)size, GET_UNTAINTED); /* loses track of taint */
+}
...


... so function_store_get truncates the size if asked to allocate more than
2GB.

This change happened after Exim 4.96.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #14 from Jeremy Harris <jgh146exb@wizmail.org> ---
(In reply to Andrew Aitchison from comment #13)
> ... so function_store_get truncates the size if asked to allocate more than
> 2GB.

I'd also think that a 2GB+ allocation is questionable!

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

Resellerdesktop Admins <customercare@resellerdesktop.de> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |customercare@resellerdeskto
| |p.de

--- Comment #15 from Resellerdesktop Admins <customercare@resellerdesktop.de> ---
Issue happens now on Fedora too, being first introduced with 4.97.1

judging from strace: it happens in mid mailtransfer.

As pcre2 had no update in that time, issue is caused by exim itself.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #16 from Jeremy Harris <jgh146exb@wizmail.org> ---
Well, it's the pcre2 library doing something unfortunate when being used
as documented. A callback from it is requesting a memory allocation of
either -16 bytes or (2^32 - 16) bytes. Exim's memory allocator is refusing
that, I think quite reasonably. So at this time I'm regarding it as something
the pcre2 maintainer needs to explain (and not a bug in Exim, until shown
otherwise).

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #17 from Andrew Aitchison <exim@aitchison.me.uk> ---
(In reply to Jeremy Harris from comment #14)
> (In reply to Andrew Aitchison from comment #13)
> > ... so function_store_get truncates the size if asked to allocate more than
> > 2GB.
>
> I'd also think that a 2GB+ allocation is questionable!

Fair enough, but fail nicely, or at least noisily.

PCRE2 asked for a >2GB allocation, and the API clearly defines size as 64bit.
You can't silently allocate a smaller amount and say "here".
Even if the allocation works, you know things will go wrong when PCRE2 tries to
access the memory it thinks it has been given.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #18 from Resellerdesktop Admins <customercare@resellerdesktop.de> ---
you know 4.97.x is now used on production servers, which now all fail to
receive big emails, which worked with 4.96.x . As prce2 did not change,
something else must have.

This is by far no medium bug, this is a critical one from my pov, as it breaks
production systems, reliable running ones with 4.96 .

yes: a 2GB malloc is false by itself
yes: it may be caused by pcre2
no: we can't wait for pcre2 to fix this and hope the entire distro base relying
on pcre2 gets rebased in a few days. You need to prevent it from happening now
and sort it our with pcre2 devs later.

even if 4.96. is the most used version atm, 4.97 gets more and more used by the
minute, so the criticalness is constantly increasing. I don't need to remind
you, how important exim has become. If people do not get reasonable sized mails
any more, it will hurt the eco system a lot. I really think about downgrading
our cluster to 4.96 , no matter what 4.97.1 fixed.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #19 from Simon Arlott <bugzilla.exim.simon@arlott.org> ---
(In reply to Niels Kobschaetzki from comment #5)
> I think I found my problem - exim does a check_dir_size for every mail and
> in mailboxes with loads of mails in the inbox the problem appears:
>
> check_dir_size: size from
> 1655541855.M648217P82718.server1.tld,S=42289,W=42895:2,S is 42289
> check_dir_size: size from
> 1670744725.H546295P50932.longisland.snafu.de,S=3020:2,S is 3020
> check_dir_size: size from
> 1655556669.M540692P82718.server1.tld,S=600481,W=608709:2,S is 600481
> check_dir_size: size from
> 1655551355.M261543P82718.server1.tld,S=20693,W=21014:2,RS is 20693
> check_dir_size: size from
> 1655549485.M157286P82718.server2.tld,S=19999,W=20333:2,S is 19999
> check_dir_size: size from
> 1655186998.M685199P49010.server1.tld,S=105516,W=107312:2,S is 105516
> LOG: MAIN PANIC DIE
> bad internal_store_malloc request (2147483632 bytes) from
> function_store_get 66
> search_tidyup called

I made a directory with 191k random filenames but I can't reproduce this. Could
you provide a list of all the filenames in the directory that reproduces this?

What version of PCRE2 are you using?

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #20 from Andrew Aitchison <exim@aitchison.me.uk> ---
Created attachment 1464
--> https://bugs.exim.org/attachment.cgi?id=1464&action=edit
patch to aid debugging 3047 - too big or too small ?

This patch (against 4.97.1) should tell us whether PCRE wants too much store,
or a negative amount.

Please report any answers.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #21 from Simon Arlott <bugzilla.exim.simon@arlott.org> ---
(In reply to Andrew Aitchison from comment #20)
> This patch (against 4.97.1) should tell us whether PCRE wants too much
> store, or a negative amount.

PCRE2_SIZE is size_t, so the value of "size" cannot be negative.

What's needed is a reliable way to reproduce it.

All I get is a request for 112 bytes for each and every check_dir_size().

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #22 from Resellerdesktop Admins <customercare@resellerdesktop.de> ---
In my case:

Name : pcre2
Version : 10.42
Release : 1.fc38.1
Architecture: x86_64
Size : 647250
Source RPM : pcre2-10.42-1.fc38.1.src.rpm
Build Date : Fr 20 Jan 2023 02:52:29 CET
Packager : Fedora Project
Summary : Perl-compatible regular expression library

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On Sun, 21 Jan 2024, Exim Bugzilla via Exim-dev wrote:

> https://bugs.exim.org/show_bug.cgi?id=3047
>
> --- Comment #21 from Simon Arlott <bugzilla.exim.simon@arlott.org> ---
> (In reply to Andrew Aitchison from comment #20)
>> This patch (against 4.97.1) should tell us whether PCRE wants too much
>> store, or a negative amount.
>
> PCRE2_SIZE is size_t, so the value of "size" cannot be negative.

Oops, yes. Bad wording.

What bothers me is that function_store_malloc is given an unsigned 64bit
value and passes it on to store_malloc as a signed 32bit value.


> What's needed is a reliable way to reproduce it.
> All I get is a request for 112 bytes for each and every check_dir_size().

Yes, though as Jeremy says, that *may* be a PCRE problem.

For now should we revert 4.97 to what we had before
https://git.exim.org/exim.git/commitdiff/c3d7f28680dda05ca6d8415cffbff8a233c0ae74
introduced function_store_get ?

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #23 from Simon Arlott <bugzilla.exim.simon@arlott.org> ---
I can now reproduce it. The difference between pcre2 v10.34 and v10.42 is that
the later version is also making allocations of 20480 bytes and that's causing
Exim to increase pp->store_block_order to 31 resulting in a 2GB allocation:

14:56:51 303075 check_dir_size: size from
18933.M685199P49010.server1.tld,S=94030,W=9068:2,S is 94030
function_store_get: 128
function_store_get: 20480
Thread 3.1 "exim" hit Breakpoint 1, internal_store_malloc (size=2147483632,
func=0x555555629ab0 <__FUNCTION__.20006> "function_store_get", line=70) at
store.c:1156
1156 log_write(0, LOG_MAIN|LOG_PANIC_DIE,
(gdb) bt
#0 internal_store_malloc (size=2147483632, func=0x555555629ab0
<__FUNCTION__.20006> "function_store_get", line=70) at store.c:1156
#1 0x00005555555f5d19 in pool_get (pp=0x5555556c0160 <paired_pools>,
size=20480, align_mem=0, func=0x555555629ab0 <__FUNCTION__.20006>
"function_store_get", linenumber=70) at store.c:449
#2 0x00005555555f6528 in store_get_3 (size=size@entry=20480,
proto_mem=proto_mem@entry=0x0, func=func@entry=0x555555629ab0
<__FUNCTION__.20006> "function_store_get", linenumber=linenumber@entry=70)
at store.c:519
#3 0x0000555555598acb in function_store_get (size=20480, tag=<optimised out>)
at exim.c:70
#4 0x00007ffff7c09fc5 in pcre2_match_8 (code=code@entry=0x5555556d1a28,
subject=subject@entry=0x5555556fd19b
"11765.M685199P49010.server1.tld,S=258620,W=1215:2,S", length=51,
length@entry=18446744073709551615, start_offset=start_offset@entry=0,
options=options@entry=0, match_data=match_data@entry=0x7fffb79a1328,
mcontext=<optimised out>) at src/pcre2_match.c:6848
#5 0x000055555560b9de in check_dir_size (dirname=dirname@entry=0x5555556e89b0
"/dev/shm/exim/Maildir/cur", countptr=countptr@entry=0x7ffffffbcec4,
re=re@entry=0x5555556d1a28) at appendfile.c:684
#6 0x000055555560ba7c in check_dir_size (dirname=dirname@entry=0x5555556ce158
"/dev/shm/exim/Maildir", countptr=countptr@entry=0x7ffffffbd050,
re=re@entry=0x5555556d1a28) at appendfile.c:721
#7 0x000055555560ddc0 in appendfile_transport_entry (tblock=0x5555556d1528,
addr=0x5555556e34c8) at appendfile.c:2388
#8 0x000055555558f756 in deliver_local (addr=addr@entry=0x5555556e34c8,
shadowing=shadowing@entry=0) at deliver.c:2387
#9 0x00005555555909ad in do_local_deliveries () at deliver.c:3004
#10 0x0000555555593e14 in deliver_message (id=<optimised out>,
forced=forced@entry=0, give_up=give_up@entry=0) at deliver.c:8003
#11 0x00005555555a23b0 in main (argc=5, cargv=0x7fffffffd9b8) at exim.c:6071
(gdb) down
#1 0x00005555555f5d19 in pool_get (pp=0x5555556c0160 <paired_pools>,
size=20480, align_mem=0, func=0x555555629ab0 <__FUNCTION__.20006>
"function_store_get", linenumber=70) at store.c:449
449 newblock = internal_store_malloc(mlength, func, linenumber);
(gdb) p mlength
$1 = 2147483632
(gdb) p size
$2 = 20480
(gdb) p length
$3 = 2147483616
(gdb) p pp->store_block_order
$4 = 31
(gdb) p *pp
$5 = {chainbase = 0x5555556c36e8, current_block = 0x7fff779a2018, next_yield =
0x7fffb79a13a8, yield_length = 3168, store_block_order = 31, store_last_get =
0x7fffb79a1328, nbytes = 36640,
maxbytes = 1073778496, nblocks = 20, maxblocks = 20, maxorder = 31}

My config just uses an "accept" router to invoke the local_virtual_delivery
transport.

It looks like none of the memory ever gets free'd.
After enough allocations of 128+20480 bytes (104207+ files) it exits.

This works ok on v10.34 because that only allocates 112 bytes per file.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

Arkadiusz Miskiewicz <arekm@maven.pl> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |arekm@maven.pl

--- Comment #24 from Arkadiusz Miskiewicz <arekm@maven.pl> ---
Could you test how 10.43rc1 behaves, too?

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

https://bugs.exim.org/show_bug.cgi?id=3047

--- Comment #25 from Jeremy Harris <jgh146exb@wizmail.org> ---
Thanks to everyone working on this.

> After enough allocations of 128+20480 bytes (104207+ files) it exits.

Hmm, yes, that is approaching 2GB already used.
We could do a bulk-release on every subdir:

rmark reset_point = store_mark(); // at the top of check_dir_size()
and
store_reset(reset_point); // just before return at the end

Though that wouldn't help a single directory with that many files.
For that, we could count file files and (reset, take a new mark)
every (eg.) thousand files.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/