Mailing List Archive

List Partition Comprehension (Feature Suggestion)
Python list comprehension is substantially faster than plan iteration, to the point where
```
l0, l1 = [],[]
for x in l:
if cond(x):
l0.append(x)
else:
l1.append(x)
```
runs at about the same speed as
```
l0 = [x for x in l if cond(x)]
l1 = [x for x in l if not cond(x)]
```
assuming `cond` is a computationally light conditional.
While this isn't an extremely common use, I suggest a "partition comprehension" syntax which extends normal filtered-generator syntax. Such as:
```
l0, l1 = ([x for x in l if cond(x) else x])
```
(parenthesis there to indicate that the output is a tuple)
It could allow extension to n-way partitions as well. Because elif doesn't fit nicely in-line, it would probably be nicer to do it like:
```
l0, l1, l2 = ([.x for x in l if cond0(x) else x**2 if cond1(x) else x**3])
```
So l0 would be all elements that match cond0, l1 would be squares of all that match cond1, and l2 would be cubes of everything that matches neither.
--
https://mail.python.org/mailman/listinfo/python-list
Re: List Partition Comprehension (Feature Suggestion) [ In reply to ]
A possible alternative would be a partition library function in the same vein as `map` and `filter`
```
def partition(n, key, iter):
"""
Partitions a list.
Args: (n: the number of partitions), (key: function which takes elements and returns the index of the partition to place them in), (iter: the iterable to be partitioned)
Returns: A tuple of lists partitioned per the key function. If this were a library function, this would likely return a tuple of generators instead in the same vein as `map` and `filter`.
"""
lists = tuple([] for _ in range(n))
for x in iter:
lists[key(x)].append(x)
return lists
```
Except it would be optimized like a library function (which means written in C, I believe)

On Tuesday, September 22, 2020 at 12:16:39 PM UTC-7, Yakov Shalunov wrote:
> Python list comprehension is substantially faster than plan iteration, to the point where
> ```
> l0, l1 = [],[]
> for x in l:
> if cond(x):
> l0.append(x)
> else:
> l1.append(x)
> ```
> runs at about the same speed as
> ```
> l0 = [x for x in l if cond(x)]
> l1 = [x for x in l if not cond(x)]
> ```
> assuming `cond` is a computationally light conditional.
> While this isn't an extremely common use, I suggest a "partition comprehension" syntax which extends normal filtered-generator syntax. Such as:
> ```
> l0, l1 = ([x for x in l if cond(x) else x])
> ```
> (parenthesis there to indicate that the output is a tuple)
> It could allow extension to n-way partitions as well. Because elif doesn't fit nicely in-line, it would probably be nicer to do it like:
> ```
> l0, l1, l2 = ([.x for x in l if cond0(x) else x**2 if cond1(x) else x**3])
> ```
> So l0 would be all elements that match cond0, l1 would be squares of all that match cond1, and l2 would be cubes of everything that matches neither.
--
https://mail.python.org/mailman/listinfo/python-list
Re: List Partition Comprehension (Feature Suggestion) [ In reply to ]
On 23/09/20 7:16 am, Yakov Shalunov wrote:
> l0, l1 = ([x for x in l if cond(x) else x])
> l0, l1, l2 = ([.x for x in l if cond0(x) else x**2 if cond1(x) else x**3])

This syntax seems a bit convoluted. One of the result expressions
is at the beginning as usual, but the rest are tacked on the end.
And it's far from obvious that it returns multiple lists instead
of a single list with alternative values, as you would get from

[.(x if cond0(x) else x**2 if cond1(x) else x**3) for x in l]

which at first glance seems very similar.

Also I'm not sure how this would interact with the existing
comprehension syntax in its full generality, where you can
have multiple "for" and "if" clauses mixed in any order.

> A possible alternative would be a partition library function in the
> same vein as `map` and `filter`
That sounds like a much better idea for something that is so
rarely needed. Although it probably wouldn't be as fast as an
LC could potentially be, due to the need to call a user-supplied
function for every item.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list
Re: List Partition Comprehension (Feature Suggestion) [ In reply to ]
On 9/22/2020 3:16 PM, Yakov Shalunov wrote:
> Python list comprehension is substantially faster than plan iteration, to the point where
> ```
> l0, l1 = [],[]
> for x in l:
> if cond(x):
> l0.append(x)
> else:
> l1.append(x)
> ```
> runs at about the same speed as
> ```
> l0 = [x for x in l if cond(x)]
> l1 = [x for x in l if not cond(x)]
> ```
> assuming `cond` is a computationally light conditional.
> While this isn't an extremely common use, I suggest a "partition comprehension" syntax which extends normal filtered-generator syntax. Such as:
> ```
> l0, l1 = ([x for x in l if cond(x) else x])
> ```
> (parenthesis there to indicate that the output is a tuple)

Parentheses only make a tuple when empty. They otherwise group items as
function arguments or to indicate precedence.

'else' by itself would be enough to indicate that one wants two streams,
even if it is a bit baroque.

'x for x in iterable if condition' is a generator comprehension.
Putting {} or [] around it says to run the iteration to make a set or
list. Passing the generator comprehension to set() or list() has the
same effect, but the set or list comprehensions can be optimized to run
faster.

'x:f(x) for x in iterable' is not quite legal as a standalone generator,
but '(x,f(x)} for x in iterable' is, and {x:f(x) for x in iterable} has
the effect of
dict((x,f(x)) for x in iterable).

By analogy,
'ftrue(x) for x in iterable if condition else ffalse(x)'
should be a double generator comprehension producing a pair of generators
'(f(x) for x in iterable if condition),
(f(x) for x in iterable if not condition)'

Then enclosing in {} or [] applies set() or list() to each separately to
produce a pair of sets or lists. But the latter could be optimized.

The double generator comprehension is similar to a conditional
expression except that both true and false values are produced -- and
kept separate.
'(ft(x) if condition else ff(x)) for x in iterable'
is legal today, but does not separate the true and false results.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list