Mailing List Archive

[Bug 3024] string expansion error in function reduce combined with inlisti function
https://bugs.exim.org/show_bug.cgi?id=3024

Jeremy Harris <jgh146exb@wizmail.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.exim.org/show_
| |bug.cgi?id=452

--- Comment #1 from Jeremy Harris <jgh146exb@wizmail.org> ---
What the expansion is trying to do is a de-deplication of a list.

The problem is that list lookups, of which inlisti is one, modify
$value for more sorts of matching elements than they used to - specifically
here static text list elements.

Previously it was only things like matches in file or database.

The expansion as written is expecting $value, at the time of expanding
the result for the ${if...} to be the intermediate being used by
the ${reduce...}.

I don't see a simple way to fix this.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

Jeremy Harris <jgh146exb@wizmail.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.exim.org/show_
| |bug.cgi?id=2363

--- Comment #2 from Jeremy Harris <jgh146exb@wizmail.org> ---
Possibilities:
- a 'ret=no' option, here applied to the inlisti, meaning that $value
should not be overwritten
- variables assignable within an expansion, maybe (bug 2363)



In the meantime, a workable replacement is

(ACL)

# arg1: item arg2: list-so-far
if_inlisti:
accept message = ${acl_arg2}${if inlisti{$acl_arg1}{<\n
$acl_arg2} {}{${acl_arg1}\n}}



(expansion)

${reduce {<\n 3\n4\n4\n}{}{${acl {if_inlisti}{$item}{$value}}}

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #3 from Jeremy Harris <jgh146exb@wizmail.org> ---
The option on the condition turns out to be not too invasive to code,
assuming it's done as a comma-sep option after the condition name.
This would be supported for
inlist
inlisti
match_address
match_domain
match_local_part
match_ip
and optional in all cases. Default behaviour as per 4.96 ($value gets set
to the matched item)

Current coding has "no_value" as the option name.

How do people feel? Is it worth adding this syntax? There are no other
options on expansion-conditions at this time.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #4 from Andreas Metzler <eximusers@bebt.de> ---
(In reply to Jeremy Harris from comment #3)
> The option on the condition turns out to be not too invasive to code,
> assuming it's done as a comma-sep option after the condition name.
[...]
> How do people feel? Is it worth adding this syntax? There are no other
> options on expansion-conditions at this time.

It seems to be a little bit adhoc to add options to /some/ operators to not
touch $value since reduce also touches $value. I think reduce's $value should
somehow "trump" whatever the inner expansions do. However then a further
question pops up:
"At the end of a reduce expansion, the values of $item and $value are restored
to what they were before."
What is "before" in that context? $value as set by the inner expansion?

How about instead changing reduce to use a different temporary variable instead
of $value?

cu Andreas

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #5 from Jeremy Harris <jgh146exb@wizmail.org> ---
I'm not clear that "trumping" to subvert the normal nesting would be good,
at least as default behaviour. The problem really is the use of this one
variable for side-effect results (here, from inlisti) and for the working-store
(here, in reduce).

${reduce...} is documented to use $value; we could only change it by
adding optional behaviour, which also means syntax. At least until
someone designs Exim version 5, we're stuck with it.

"Before" would be the caller of ${reduce...}, which could be a textually
enclosing expansion, an acl condition or modifier, or an option.
It is not (either of) the inner expansions.
This seemed obvious to me, but since you query it perhaps it is not.

On the point of "some operators" - I didn't see any other expansion-conditions
where it made sense. If I missed some, please say.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #6 from Andreas Metzler <eximusers@bebt.de> ---
(In reply to Jeremy Harris from comment #5)
[...]
> ${reduce...} is documented to use $value; we could only change it by
> adding optional behaviour, which also means syntax. At least until
> someone designs Exim version 5, we're stuck with it.

It is not that clear cut. - Some uses of reduce broke, because some other parts
of exim were enhanced/extended. So we already broke reduce for some users.
Still I think you right, breaking $value for all users would be worse than the
status-quo.

How about marking reduce deprecated and adding e.g. reduce2 which differs from
reduce only in using e.g. $red_value instead of $value as temporary variable?
This seems to be simpler and more robust to future changes than looking at all
expansion operators and extending all of them that change $value (either now at
some unknown point in the future) to have an option to not do this.

cu Andreas

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #7 from Jeremy Harris <jgh146exb@wizmail.org> ---
I've mulled that suggestion over for a while - and finally got to a killer:
it doesn't fix every case. Eventually someone will find a need to wrap one
reduce inside another, and use the "outer" $value (or equiv) within the "inner"
operation.

No, I've not thought of why it would be wanted! However...

That gets me started thinking of access to a stack of $value -
perhaps $value1 for the immediately-wrapping layer, $value2 for the next
one out, etc. Some amount of programming effort to implement, but feels
feasible.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #8 from Jeremy Harris <jgh146exb@wizmail.org> ---
Handling a stack of $value is this much code-churn:

src/src/expand.c | 459
++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------
src/src/functions.h | 82 ++++++++++++++++-
src/src/globals.c | 2 +-
src/src/globals.h | 2 +-
src/src/routers/manualroute.c | 8 +-
src/src/routers/queryprogram.c | 2 +-
src/src/structs.h | 7 ++
test/scripts/0000-Basic/0002 | 2 +-
test/stdout/0002 | 2 +-
9 files changed, 374 insertions(+), 192 deletions(-)

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
[Bug 3024] string expansion error in function reduce combined with inlisti function [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=3024

--- Comment #9 from Jeremy Harris <jgh146exb@wizmail.org> ---
The original expansion can be more neatly expressed as:

${reduce {<\n 1.1.1.1\n2.2.2.2\n3.3.3.3\n4.4.4.4\n4.4.4.4\n}{}{$value${if
inlisti{$item}{<\n $value}{}{$item\n}}}}

which avoids the issue.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/