Mailing List Archive

On Fri, May 14, 2021 at 12:45 PM Steve Dower <steve.dower@python.org> wrote:
>
> On 14May2021 0622, micro codery wrote:
> >
> > There was a discussion a while back ( a year or so?? ) on
> > Python-ideas that introduced the idea of having more "sentinel-like"
> > singletons in Python -- right now, we only have None.
> >
> > Not quite true, we also have Ellipsis, which already has a nice repr
> > that both reads easily and still follows the convention of eval(repr(x))
> > == x. It also is already safe from instantiation, survives pickle
> > round-trip and is multi-thread safe.
> > So long as you are not dealing with scientific projects, it seems a
> > quick (if dirty) solution to having a sentinel that is not None.
>
> I don't think using "..." to indicate "some currently unknown or
> unspecified value" is dirty at all, it seems perfectly consistent with
> how we use it in English (and indexing in scientific projects, for that
> matter, where it tends to imply "figure out the rest for me").
>
> All that's really missing is some kind of endorsement (from python-dev,
> presumably in the docs) that it's okay to use it as a default parameter
> value. I can't think of any reason you'd need to accept Ellipsis as a
> *specified* value that wouldn't also apply to any other kind of shared
> sentinel.

I'll try to organize my thoughts a bit here. This is a bit long,
welcome to skip to the final sentence for the "tl;dr".

Features one may want for a sentinel:
1. Unique from other objects
2. Globally unique, i.e. unique from other such sentinels (no consensus here)
3. Clear repr (the original subject of this thread) - significant
since these often appear in function signatures
4. Survives pickling round-trip (I brought this up since I was bitten
by this once, but others have mentioned that this is usually
irrelevant)
5. Can be given a clear type signature (this was brought up on
twitter[1]) - significant since without this nobody can add full type
signatures even if they want to

The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
what I've been using for years, and I now think that it isn't good
enough. This not having a nice repr is what started this thread.

I'd also personally prefer something simple, ideally without a new
class or module.

There are several simple idioms using existing features that seem like
good options, so let's review those:

1. Ellipsis, a.k.a. `...`. This has all of the features outlined above
except #2. My main issue with using Ellipsis for this is that it could
be surprising and confusing for devs first encountering such use, and
could be relatively awkward to figure out.

2. An instance of a one-off class. dataclasses.MISSING is an example
of this. It is defined thus:

class _MISSING:
pass
MISSING = _MISSING()

Besides failing #4 (surviving pickle round-trips), its repr isn't
great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily
overcome by implementing __repr__.

3. A one-off class:

class MISSING: pass

This has all of the above features except #5: having a clear type
signature (since its type is type). Using a class as a value this way
could be surprising, though. It's repr also isn't great: <class
'dataclasses._MISSING'>.

4. A value of an single-valued enum, for example (from [1]):

class _UNSET(enum.Enum):
token = enum.auto()

This has all of the features above and is simple, just requiring a
comment to explain what it is. It's repr is a bit awkward though:

>>> repr(_UNSET.token)
'<_UNSET.token: 1>'


All of these are in use by some developers, though not necessarily in
the stdlib. None is perfect, though all are probably good enough.
Since pickling is likely not relevant in most cases, I'm currently in
favor of #2 making sure to implement a nice __repr__.

- Tal

[1] https://twitter.com/nolar/status/1392962447166877697
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BYLTDD72RZUUKVRJL6TTSWF35JD3FL47/
Code of Conduct: http://python.org/psf/codeofconduct/

If we drop the requirement for pickle round-tripping then I would add a
requirement that sentinel is unpicklable, to prevent accidents.

Irit

On Fri, May 14, 2021 at 8:38 PM Tal Einat <taleinat@gmail.com> wrote:

>
> I'll try to organize my thoughts a bit here. This is a bit long,
> welcome to skip to the final sentence for the "tl;dr".
>
> Features one may want for a sentinel:
> 1. Unique from other objects
> 2. Globally unique, i.e. unique from other such sentinels (no consensus
> here)
> 3. Clear repr (the original subject of this thread) - significant
> since these often appear in function signatures
> 4. Survives pickling round-trip (I brought this up since I was bitten
> by this once, but others have mentioned that this is usually
> irrelevant)
> 5. Can be given a clear type signature (this was brought up on
> twitter[1]) - significant since without this nobody can add full type
> signatures even if they want to
>
> The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
> what I've been using for years, and I now think that it isn't good
> enough. This not having a nice repr is what started this thread.
>
> I'd also personally prefer something simple, ideally without a new
> class or module.
>
> There are several simple idioms using existing features that seem like
> good options, so let's review those:
>
> 1. Ellipsis, a.k.a. `...`. This has all of the features outlined above
> except #2. My main issue with using Ellipsis for this is that it could
> be surprising and confusing for devs first encountering such use, and
> could be relatively awkward to figure out.
>
> 2. An instance of a one-off class. dataclasses.MISSING is an example
> of this. It is defined thus:
>
> class _MISSING:
> pass
> MISSING = _MISSING()
>
> Besides failing #4 (surviving pickle round-trips), its repr isn't
> great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily
> overcome by implementing __repr__.
>
> 3. A one-off class:
>
> class MISSING: pass
>
> This has all of the above features except #5: having a clear type
> signature (since its type is type). Using a class as a value this way
> could be surprising, though. It's repr also isn't great: <class
> 'dataclasses._MISSING'>.
>
> 4. A value of an single-valued enum, for example (from [1]):
>
> class _UNSET(enum.Enum):
> token = enum.auto()
>
> This has all of the features above and is simple, just requiring a
> comment to explain what it is. It's repr is a bit awkward though:
>
> >>> repr(_UNSET.token)
> '<_UNSET.token: 1>'
>
>
> All of these are in use by some developers, though not necessarily in
> the stdlib. None is perfect, though all are probably good enough.
> Since pickling is likely not relevant in most cases, I'm currently in
> favor of #2 making sure to implement a nice __repr__.
>
> - Tal
>
> [1] https://twitter.com/nolar/status/1392962447166877697
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BYLTDD72RZUUKVRJL6TTSWF35JD3FL47/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

I think it's more future-looking to allow pickle round-tripping. Just add a
._uuid attribute and have object equality follow equality of that
attribute. There's no reason to expose that in the .__repr__, but it would
be inspectable in concept.

On Fri, May 14, 2021, 7:01 PM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

>
> If we drop the requirement for pickle round-tripping then I would add a
> requirement that sentinel is unpicklable, to prevent accidents.
>
> Irit
>
> On Fri, May 14, 2021 at 8:38 PM Tal Einat <taleinat@gmail.com> wrote:
>
>>
>> I'll try to organize my thoughts a bit here. This is a bit long,
>> welcome to skip to the final sentence for the "tl;dr".
>>
>> Features one may want for a sentinel:
>> 1. Unique from other objects
>> 2. Globally unique, i.e. unique from other such sentinels (no consensus
>> here)
>> 3. Clear repr (the original subject of this thread) - significant
>> since these often appear in function signatures
>> 4. Survives pickling round-trip (I brought this up since I was bitten
>> by this once, but others have mentioned that this is usually
>> irrelevant)
>> 5. Can be given a clear type signature (this was brought up on
>> twitter[1]) - significant since without this nobody can add full type
>> signatures even if they want to
>>
>> The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
>> what I've been using for years, and I now think that it isn't good
>> enough. This not having a nice repr is what started this thread.
>>
>> I'd also personally prefer something simple, ideally without a new
>> class or module.
>>
>> There are several simple idioms using existing features that seem like
>> good options, so let's review those:
>>
>> 1. Ellipsis, a.k.a. `...`. This has all of the features outlined above
>> except #2. My main issue with using Ellipsis for this is that it could
>> be surprising and confusing for devs first encountering such use, and
>> could be relatively awkward to figure out.
>>
>> 2. An instance of a one-off class. dataclasses.MISSING is an example
>> of this. It is defined thus:
>>
>> class _MISSING:
>> pass
>> MISSING = _MISSING()
>>
>> Besides failing #4 (surviving pickle round-trips), its repr isn't
>> great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily
>> overcome by implementing __repr__.
>>
>> 3. A one-off class:
>>
>> class MISSING: pass
>>
>> This has all of the above features except #5: having a clear type
>> signature (since its type is type). Using a class as a value this way
>> could be surprising, though. It's repr also isn't great: <class
>> 'dataclasses._MISSING'>.
>>
>> 4. A value of an single-valued enum, for example (from [1]):
>>
>> class _UNSET(enum.Enum):
>> token = enum.auto()
>>
>> This has all of the features above and is simple, just requiring a
>> comment to explain what it is. It's repr is a bit awkward though:
>>
>> >>> repr(_UNSET.token)
>> '<_UNSET.token: 1>'
>>
>>
>> All of these are in use by some developers, though not necessarily in
>> the stdlib. None is perfect, though all are probably good enough.
>> Since pickling is likely not relevant in most cases, I'm currently in
>> favor of #2 making sure to implement a nice __repr__.
>>
>> - Tal
>>
>> [1] https://twitter.com/nolar/status/1392962447166877697
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/BYLTDD72RZUUKVRJL6TTSWF35JD3FL47/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7I7S22KEOZY2MDDIYCUPQWG6WXLFZZR5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Sat, 15 May 2021, 5:39 am Tal Einat, <taleinat@gmail.com> wrote:

(snip useful feature summary)

The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
> what I've been using for years, and I now think that it isn't good
> enough. This not having a nice repr is what started this thread.
>
> I'd also personally prefer something simple, ideally without a new
> class or module.
>

The key advantage of a new base class is that it offers a way to
communicate a shift in the recommended development idiom. "sentinel =
object()" is entrenched enough that I would expect that only a
"types.Sentinel" base class would stand a good chance of displacing it.

Why the "types" module? I don't think this is important enough to be a
builtin type, and the types module is cheap to import, doesn't bring in
many transitive dependencies, and the purpose of the new base class would
be defining custom sentinel types with various desirable properties.

Cheers,
Nick.

>


>

Since the subject is this,
I will note that past week, I resorted twice to create an Enum with a
single element,
and then alias the element on the module namespace, so that it would work
as a "polished" sentinel.

So:

import enum

class Whatever(enum.Enum):
EMPTY = "EMPTY"

EMPTY = Whatever.EMPTY

A "repeat yourself three times" - certainly not that nice.
But its use was a bit off the ones listed here in a sense these
are meant to be explicitly passed to some methods (rather,
placed inside data structures) to indicate a desired behavior.



On Fri, 14 May 2021 at 21:03, Nick Coghlan <ncoghlan@gmail.com> wrote:

> On Sat, 15 May 2021, 5:39 am Tal Einat, <taleinat@gmail.com> wrote:
>
> (snip useful feature summary)
>
> The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
>> what I've been using for years, and I now think that it isn't good
>> enough. This not having a nice repr is what started this thread.
>>
>> I'd also personally prefer something simple, ideally without a new
>> class or module.
>>
>
> The key advantage of a new base class is that it offers a way to
> communicate a shift in the recommended development idiom. "sentinel =
> object()" is entrenched enough that I would expect that only a
> "types.Sentinel" base class would stand a good chance of displacing it.
>
> Why the "types" module? I don't think this is important enough to be a
> builtin type, and the types module is cheap to import, doesn't bring in
> many transitive dependencies, and the purpose of the new base class would
> be defining custom sentinel types with various desirable properties.
>
> Cheers,
> Nick.
>
>>
>
>
>> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/V67I4A6WX6INEAGGSJWYFX4WKQHA73NW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

I think that would be the primary motivating factor behind recommending
Ellipsis, it’s already a builtin and we are not likely it to get another
builtin singleton. Ever? But besides that “...” in a function signature,
although maybe looking magical, does immediately call out to the reader
that something special is happening here. And for beginners who might not
know that three dots are an Ellipsis or what an Ellipsis even is, it would
be fairly easy to internet search “three dots in a function signature” or
even “python def ...” would probably be enough to surface an explanation if
this became somewhat common.

To add to the suggestions already given in this thread I dug into code I wrote some time ago.
Offered as an inspiration.


=== missing.py ===

from typing import Any


def MISSING(klass: Any) -> Any:
"""
create a sentinel to indicate a missing instance of a class
:param klass: the class of which an instance is missing
:return: missing class object
"""
g = globals()
missing_klass_name = f"_MISSING_{klass.__module__}_{klass.__name__}_MISSING_"
if missing_klass_name not in g:
g[missing_klass_name] = type(
missing_klass_name,
(klass,),
{
"__repr__": lambda x: f"MISSING({klass.__name__})",
}
)
return g[missing_klass_name]()

===


and as a demo:

=== demo_missing.py ===

import pickle


from missing import MISSING


x = MISSING(str)
y = "bar"
print(f"{x!r} == {y!r}: {x == y}")
print(f"{x!r} is {y!r}: {x is y}")
# MISSING(str) == 'bar': False
# MISSING(str) is 'bar': False

with open("object.pickled", "wb") as f:
pickle.dump(x, f)
with open("object.pickled", "rb") as f:
y = pickle.load(f)
print(f"{x!r} == {y!r}: {x == y}")
print(f"{x!r} is {y!r}: {x is y}")
# MISSING(str) == MISSING(str): True
# MISSING(str) is MISSING(str): False


def foo(a: int = MISSING(int), b: int = MISSING(int)):
print(f"{a=} {isinstance(a, int)}")
print(f"{b=} {isinstance(b, int)}")
print(f"{a!r} == {b!r}: {a == b}")
print(f"{a!r} is {b!r}: {a is b}")


foo()
# a=MISSING(int) True
# b=MISSING(int) True
# MISSING(int) == MISSING(int): True
# MISSING(int) is MISSING(int): False

foo(1)
# a=1 True
# b=MISSING(int) True
# 1 == MISSING(int): False
# 1 is MISSING(int): False


class Test:
...


t = MISSING(Test)
print(f"{t=}")
# t=MISSING(Test)

===
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/42O33BFRJLDFVKX4XSKMZ6VLR7H7GXKP/
Code of Conduct: http://python.org/psf/codeofconduct/

I've created a poll on this subject:
https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810

On Fri, May 14, 2021 at 10:33 PM Tal Einat <taleinat@gmail.com> wrote:
>
> On Fri, May 14, 2021 at 12:45 PM Steve Dower <steve.dower@python.org> wrote:
> >
> > On 14May2021 0622, micro codery wrote:
> > >
> > > There was a discussion a while back ( a year or so?? ) on
> > > Python-ideas that introduced the idea of having more "sentinel-like"
> > > singletons in Python -- right now, we only have None.
> > >
> > > Not quite true, we also have Ellipsis, which already has a nice repr
> > > that both reads easily and still follows the convention of eval(repr(x))
> > > == x. It also is already safe from instantiation, survives pickle
> > > round-trip and is multi-thread safe.
> > > So long as you are not dealing with scientific projects, it seems a
> > > quick (if dirty) solution to having a sentinel that is not None.
> >
> > I don't think using "..." to indicate "some currently unknown or
> > unspecified value" is dirty at all, it seems perfectly consistent with
> > how we use it in English (and indexing in scientific projects, for that
> > matter, where it tends to imply "figure out the rest for me").
> >
> > All that's really missing is some kind of endorsement (from python-dev,
> > presumably in the docs) that it's okay to use it as a default parameter
> > value. I can't think of any reason you'd need to accept Ellipsis as a
> > *specified* value that wouldn't also apply to any other kind of shared
> > sentinel.
>
> I'll try to organize my thoughts a bit here. This is a bit long,
> welcome to skip to the final sentence for the "tl;dr".
>
> Features one may want for a sentinel:
> 1. Unique from other objects
> 2. Globally unique, i.e. unique from other such sentinels (no consensus here)
> 3. Clear repr (the original subject of this thread) - significant
> since these often appear in function signatures
> 4. Survives pickling round-trip (I brought this up since I was bitten
> by this once, but others have mentioned that this is usually
> irrelevant)
> 5. Can be given a clear type signature (this was brought up on
> twitter[1]) - significant since without this nobody can add full type
> signatures even if they want to
>
> The common `SENTINEL = object()` idiom fails #3, #4 and #5. This is
> what I've been using for years, and I now think that it isn't good
> enough. This not having a nice repr is what started this thread.
>
> I'd also personally prefer something simple, ideally without a new
> class or module.
>
> There are several simple idioms using existing features that seem like
> good options, so let's review those:
>
> 1. Ellipsis, a.k.a. `...`. This has all of the features outlined above
> except #2. My main issue with using Ellipsis for this is that it could
> be surprising and confusing for devs first encountering such use, and
> could be relatively awkward to figure out.
>
> 2. An instance of a one-off class. dataclasses.MISSING is an example
> of this. It is defined thus:
>
> class _MISSING:
> pass
> MISSING = _MISSING()
>
> Besides failing #4 (surviving pickle round-trips), its repr isn't
> great: <dataclasses._MISSING object at 0x7fe14b1e2e80>. That is easily
> overcome by implementing __repr__.
>
> 3. A one-off class:
>
> class MISSING: pass
>
> This has all of the above features except #5: having a clear type
> signature (since its type is type). Using a class as a value this way
> could be surprising, though. It's repr also isn't great: <class
> 'dataclasses._MISSING'>.
>
> 4. A value of an single-valued enum, for example (from [1]):
>
> class _UNSET(enum.Enum):
> token = enum.auto()
>
> This has all of the features above and is simple, just requiring a
> comment to explain what it is. It's repr is a bit awkward though:
>
> >>> repr(_UNSET.token)
> '<_UNSET.token: 1>'
>
>
> All of these are in use by some developers, though not necessarily in
> the stdlib. None is perfect, though all are probably good enough.
> Since pickling is likely not relevant in most cases, I'm currently in
> favor of #2 making sure to implement a nice __repr__.
>
> - Tal
>
> [1] https://twitter.com/nolar/status/1392962447166877697
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/77ZBKCJQTG2OFY2WUL33OSJ6H3J57AEP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote:
>
> I think it's more future-looking to allow pickle round-tripping.

I tend to agree.

> Just add a ._uuid attribute and have object equality follow equality of that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept.

I think it's worth preserving the idiom of comparing sentinels using
`is`, as we do for `None` and other existing sentinel values. It's
relatively easy to do, such as by using a single-value Enum or by
using a class with a custom __new__.

- Tal
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XNYLOHY46BWLRXVLPL5RNTLBE6JG5OEX/
Code of Conduct: http://python.org/psf/codeofconduct/

IMO you should consider writing a PEP to enhance sentinels in Python,
and maybe even provide a public API for sentinels in general.

Victor
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EY6B6PRQ2B54FVG5JK42GR6ZM2VQ7VL2/
Code of Conduct: http://python.org/psf/codeofconduct/

I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is
not a good sentinel. It's a pickable singleton testable with `is`,
readily available, and extremely unlikely to appear in a data stream.
Its repr is "Ellipsis".

If you don't like the name for this purpose, you can always define a
constant (that won't fix the `repr`, obviously, but helps with source
code readability).

SENTINEL = ...

I can't think of any case where I'd rather have my own custom
sentinel, or need a special API for sentinels. Probably my fault, of
course. Please enlighten me!

Cheers,

Luciano

On Thu, May 20, 2021 at 8:35 AM Victor Stinner <vstinner@python.org> wrote:
>
> IMO you should consider writing a PEP to enhance sentinels in Python,
> and maybe even provide a public API for sentinels in general.
>
> Victor
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EY6B6PRQ2B54FVG5JK42GR6ZM2VQ7VL2/
> Code of Conduct: http://python.org/psf/codeofconduct/



--
Luciano Ramalho
| Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
| Technical Principal at ThoughtWorks
| Twitter: @ramalhoorg
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WNCFFIJHLS5NCA5QZG5JA45U2DKEETGI/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, May 20, 2021 at 6:21 AM Tal Einat <taleinat@gmail.com> wrote:

> On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote:
> > Just add a ._uuid attribute and have object equality follow equality of
> that attribute. There's no reason to expose that in the .__repr__, but it
> would be inspectable in concept.
>
> I think it's worth preserving the idiom of comparing sentinels using
> `is`, as we do for `None` and other existing sentinel values. It's
> relatively easy to do, such as by using a single-value Enum or by
> using a class with a custom __new__.
>

This only works if:

a) Unpickling is within a single interpreter session
b) Sentinels are explicitly created in imported modules, not as a runtime,
user-level creation

Maybe there's a way to do it, but how would you handle this situation:

if some_runtime_condition:
my_sentinal = Sentinel(desc="Gosh, I need a sentinel")
# ... code ...
pickle.dump(thing_using_sentinel, fh)

Equality is certainly a lot easier to get than identity here.

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Thu, 20 May 2021 at 18:13, Luciano Ramalho <luciano@ramalho.org> wrote:
>
> I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is
> not a good sentinel. It's a pickable singleton testable with `is`,
> readily available, and extremely unlikely to appear in a data stream.
> Its repr is "Ellipsis".

Personally, I'm quite tempted by the idea of using ellipsis. It just
sort of feels reasonable (and in the context `def f(x,
optional_arg=...)` it even looks pretty natural).

But it nevertheless feels like a bit of an abuse - the original point
of ellipsis was for indexing, and in particular complex slices like
a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
it, even if it's relatively rare in everyday Python. So while I like
the idea in principle, I'm mildly worried that it's not "the right
thing to do".

I can't put my ambivalence about the idea any more precisely than
this, unfortunately.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZGMZRGHFXQQZZLKBBZKXXAO65TRB6VYX/
Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, May 21, 2021 at 3:51 AM David Mertz <mertz@gnosis.cx> wrote:
>
> On Thu, May 20, 2021 at 6:21 AM Tal Einat <taleinat@gmail.com> wrote:
>>
>> On Sat, May 15, 2021 at 2:09 AM David Mertz <mertz@gnosis.cx> wrote:
>> > Just add a ._uuid attribute and have object equality follow equality of that attribute. There's no reason to expose that in the .__repr__, but it would be inspectable in concept.
>>
>> I think it's worth preserving the idiom of comparing sentinels using
>> `is`, as we do for `None` and other existing sentinel values. It's
>> relatively easy to do, such as by using a single-value Enum or by
>> using a class with a custom __new__.
>
>
> This only works if:
>
> a) Unpickling is within a single interpreter session
> b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation
>
> Maybe there's a way to do it, but how would you handle this situation:
>
> if some_runtime_condition:
> my_sentinal = Sentinel(desc="Gosh, I need a sentinel")
> # ... code ...
> pickle.dump(thing_using_sentinel, fh)
>
> Equality is certainly a lot easier to get than identity here.
>

Probably the easiest way would be to have some kind of unique
identifier (a fully-qualified name) that can be pickled, and then any
time you attempt to construct a Sentinel with that identifier, it's
guaranteed to return the same object. That way, in your example,
unpickling it in another session where that runtime condition is false
would simply create a brand new sentinel, which would be the same as
any others that also get unpickled; and if, subsequently, you
reconstruct my_sentinal, it would be the same one as got unpickled.

The hardest part would be figuring out a reliable way to define the
identifier, without massively duplicating or requiring that something
be registered somewhere.

ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3WKMD7Q3JMLDWI3L2WOGIJB767PYJO2W/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, May 20, 2021, 2:11 PM Chris Angelico

> Probably the easiest way would be to have some kind of unique
> identifier (a fully-qualified name) that can be pickled, and then any
> time you attempt to construct a Sentinel with that identifier, it's
> guaranteed to return the same object.


Gosh, almost like a UUID :-).

Actually, there's no reason my ._uuid attribute couldn't simply be the
object id(). That's probably an improvement.

On 5/20/21 11:00 AM, Paul Moore wrote:

> But it nevertheless feels like a bit of an abuse - the original point
> of ellipsis was for indexing, and in particular complex slices like
> a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
> it,

Interesting -- do you know what ... means in that context?

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EKKZP27QJZJD7UA7BXV6TB5EG7NOI2H3/
Code of Conduct: http://python.org/psf/codeofconduct/

On 5/20/21 10:47 AM, David Mertz wrote:
> On Thu, May 20, 2021 at 6:21 AM Tal Einat wrote:

>> I think it's worth preserving the idiom of comparing sentinels using
>> `is`, as we do for `None` and other existing sentinel values. It's
>> relatively easy to do, such as by using a single-value Enum or by
>> using a class with a custom __new__.
>
>
> This only works if:
>
> a) Unpickling is within a single interpreter session

I don't understand. If I pickle a sentinel in session A, then unpickle it in session B, why wouldn't I get the "same"
sentinel?

> b) Sentinels are explicitly created in imported modules, not as a runtime, user-level creation

Why would you ever have a sentinel not always created?

Hoping-to-learn-something-new'ly yrs,

--
~Ethan~
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OLTV44EVHJL3C4AIFPGUINZAZNI3YNNK/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, May 20, 2021 at 3:03 PM Ethan Furman <ethan@stoneleaf.us> wrote:

> > But it nevertheless feels like a bit of an abuse - the original point
> > of ellipsis was for indexing, and in particular complex slices like
> > a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
> > it,
> Interesting -- do you know what ... means in that context?
>

In NumPy, the ellipsis means "fill in as many dimensions as needed (with
full range)".

So e.g., if I have a 5-D array, and I want just a portion from the first
and last dimension (but everything from the middle ones), I can type:

a[1:20:2, :, :, :, 3:5]

But as a simplification, I can use the example given:

a[1:20:2, ..., 3:5]

This is particularly useful since in NumPy it is not uncommon to expand or
contract the number of dimensions (often with some dimensions having only a
span of 1). If you don't want to think about which version of the
high-dimensional array you are working with (that might have been
.flatten()'d, .squeeze()'d, or .expand_dims()'d), this is sometimes more
expressive.


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Thu, 20 May 2021 at 20:06, Ethan Furman <ethan@stoneleaf.us> wrote:
>
> On 5/20/21 11:00 AM, Paul Moore wrote:
>
> > But it nevertheless feels like a bit of an abuse - the original point
> > of ellipsis was for indexing, and in particular complex slices like
> > a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
> > it,
>
> Interesting -- do you know what ... means in that context?

In general, it just means a.getitem((slice(1,20,2), Ellipsis,
slice(3,5))), which has no specifically-defined meaning. In numpy, it
means something along the lines of "broadcast along this axis" (I
don't know the numpy terminology very well).

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FFKSIHKW2CAQUMBWA6YUDKTGDKNSEQBB/
Code of Conduct: http://python.org/psf/codeofconduct/

> On 20 May 2021, at 19:10, Luciano Ramalho <luciano@ramalho.org> wrote:
>
> I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is
> not a good sentinel. It's a pickable singleton testable with `is`,
> readily available, and extremely unlikely to appear in a data stream.
> Its repr is "Ellipsis".
>
> If you don't like the name for this purpose, you can always define a
> constant (that won't fix the `repr`, obviously, but helps with source
> code readability).
>
> SENTINEL = ...
>
> I can't think of any case where I'd rather have my own custom
> sentinel, or need a special API for sentinels. Probably my fault, of
> course. Please enlighten me!

One use case for a sentinel that is not a predefined (builtin) singleton is APIs where an arbitrary user specified value can be used.

One example of this is the definition of dataclasses.field:

dataclasses.field(*, default=MISSING, default_factory=MISSING, repr=True, hash=None, init=True, compare=True, metadata=None)

Here the “default” and “default_factory” can be an arbitrary value, and any builtin singleton could be used. Hence the use of a custom module-private sentinel that cannot clash with values used by users of the module (unless those users poke at private details of the module, but then all bets are off anyway).

That’s why I don’t particularly like the proposal of using Ellipsis as the sanctioned sentinel value. It would be weird at best that the default for a dataclass field can be any value, except for the builtin Ellipsis value.

Ronald

>
> Cheers,
>
> Luciano
>
> On Thu, May 20, 2021 at 8:35 AM Victor Stinner <vstinner@python.org> wrote:
>>
>> IMO you should consider writing a PEP to enhance sentinels in Python,
>> and maybe even provide a public API for sentinels in general.
>>
>> Victor
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EY6B6PRQ2B54FVG5JK42GR6ZM2VQ7VL2/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Luciano Ramalho
> | Author of Fluent Python (O'Reilly, 2015)
> | http://shop.oreilly.com/product/0636920032519.do
> | Technical Principal at ThoughtWorks
> | Twitter: @ramalhoorg
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WNCFFIJHLS5NCA5QZG5JA45U2DKEETGI/
> Code of Conduct: http://python.org/psf/codeofconduct/

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

On Thu, 2021-05-20 at 19:00 +0100, Paul Moore wrote:
> On Thu, 20 May 2021 at 18:13, Luciano Ramalho <luciano@ramalho.org>
> wrote:
> >
> > I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`)
> > is
> > not a good sentinel. It's a pickable singleton testable with `is`,
> > readily available, and extremely unlikely to appear in a data
> > stream.
> > Its repr is "Ellipsis".
>
> Personally, I'm quite tempted by the idea of using ellipsis. It just
> sort of feels reasonable (and in the context `def f(x,
> optional_arg=...)` it even looks pretty natural).
>
> But it nevertheless feels like a bit of an abuse - the original point
> of ellipsis was for indexing, and in particular complex slices like
> a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
> it, even if it's relatively rare in everyday Python. So while I like
> the idea in principle, I'm mildly worried that it's not "the right
> thing to do".
>
> I can't put my ambivalence about the idea any more precisely than
> this, unfortunately.


In NumPy we use a "missing argument" sentinel currently. Mainly for
things roughly like:


def mean(arr, *, axis=np._NoValue):
if not hasattr(arr, "mean"):
# Not a duck that defines `mean`, coerce to ndarray:
arr = np.asarray(arr)

if axis is np._NoValue:
return arr.mean()
return arr.mean(axis=axis)


This allows us to add new keyword arguments without breaking backward
compatibility. I do not remember if we had particularly important
reasons for not wanting to drop the default `None`, or it was just
erring on the safe side.


In any case, I tend to agree that `Ellipsis` should be considered
"user-facing" value. And in the above code, we do not expect anyone to
ever call `np.mean(something, axis=np._NoValue)` – its not even
accessible – but if the value was `...` then I would expect users to be
encouraged to write `np.mean(arr, axis=...)` in normal code.

More importantly, I can think of a reasonable "meaning" for `axis=...`!
In NumPy `axis=None` (default) returns a scalar, `axis=...` could
return a 0-D array.
This would borrow meanings that `Ellipsis` carries in indexing. [1]

Cheers,

Sebastian



[1] In such a mental model, it would mean the same as
`axis=range(arr.ndim)`. To be clear, NumPy doesn't do this, its just a
plausible meaning if it has to continue to juggle scalars and 0-D
arrays and wants to be "clearer" about it.



>
> Paul
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZGMZRGHFXQQZZLKBBZKXXAO65TRB6VYX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/M2X6VMEDFNZ4GA3CLXKXMA56SNCEPX4O/
Code of Conduct: http://python.org/psf/codeofconduct/

The scenario I'm thinking about is like this:

(base) 84-tmp % python
Python 3.9.1 (default, Dec 11 2020, 14:32:07)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> cheap_sentinel = object()
>>> id(cheap_sentinel)
140469343429632
>>> pickle.dump(cheap_sentinel, open('sentinel.pkl', 'wb'))

(base) 85-tmp % python
Python 3.9.1 (default, Dec 11 2020, 14:32:07)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> cheap_sentinel = object()
>>> id(cheap_sentinel)
139852505814016
>>> id(pickle.load(open('sentinel.pkl', 'rb')))
139852505813968

It would be pleasant if there were a way to make "cheap_sentinel" be the
same thing—either by equality or by identity—betweeen those two runs of the
interpreter.

None or Ellipsis have that property, of course. So do, for example,
integers, at least by equality if not identity (yes, of course, we might
get identity by interning, but it's not guaranteed).

On Thu, May 20, 2021 at 3:10 PM Ethan Furman <ethan@stoneleaf.us> wrote:

> On 5/20/21 10:47 AM, David Mertz wrote:
> > On Thu, May 20, 2021 at 6:21 AM Tal Einat wrote:
>
> >> I think it's worth preserving the idiom of comparing sentinels using
> >> `is`, as we do for `None` and other existing sentinel values. It's
> >> relatively easy to do, such as by using a single-value Enum or by
> >> using a class with a custom __new__.
> >
> >
> > This only works if:
> >
> > a) Unpickling is within a single interpreter session
>
> I don't understand. If I pickle a sentinel in session A, then unpickle it
> in session B, why wouldn't I get the "same"
> sentinel?
>
> > b) Sentinels are explicitly created in imported modules, not as a
> runtime, user-level creation
>
> Why would you ever have a sentinel not always created?
>
> Hoping-to-learn-something-new'ly yrs,
>
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OLTV44EVHJL3C4AIFPGUINZAZNI3YNNK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Thu, May 20, 2021 at 10:37 PM David Mertz <mertz@gnosis.cx> wrote:
>
> The scenario I'm thinking about is like this:
>
> (base) 84-tmp % python
> Python 3.9.1 (default, Dec 11 2020, 14:32:07)
> [GCC 7.3.0] :: Anaconda, Inc. on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pickle
> >>> cheap_sentinel = object()
> >>> id(cheap_sentinel)
> 140469343429632
> >>> pickle.dump(cheap_sentinel, open('sentinel.pkl', 'wb'))
>
> (base) 85-tmp % python
> Python 3.9.1 (default, Dec 11 2020, 14:32:07)
> [GCC 7.3.0] :: Anaconda, Inc. on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pickle
> >>> cheap_sentinel = object()
> >>> id(cheap_sentinel)
> 139852505814016
> >>> id(pickle.load(open('sentinel.pkl', 'rb')))
> 139852505813968
>
> It would be pleasant if there were a way to make "cheap_sentinel" be the same thing—either by equality or by identity—betweeen those two runs of the interpreter.
>
> None or Ellipsis have that property, of course. So do, for example, integers, at least by equality if not identity (yes, of course, we might get identity by interning, but it's not guaranteed).

There are several ways of achieving this for other sentinel values,
some of which have been mentioned earlier in this thread. Some
examples are:

1. a value from a single-valued enum
2. a class object (not an instance)
3. a singleton class with a carefully implemented __new__ which always
returns the same instance

- Tal Einat
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RJXU7F6LYVJ4C4N4EPVA3A5F3QRGK253/
Code of Conduct: http://python.org/psf/codeofconduct/

> > But it nevertheless feels like a bit of an abuse - the original point
>> > of ellipsis was for indexing, and in particular complex slices like
>> > a[1:20:2, ..., 3:5]. That usage is common in numpy, as I understand
>> > it,
>> Interesting -- do you know what ... means in that context?
>>
>
> In NumPy, the ellipsis means "fill in as many dimensions as needed (with
> full range)".
>

I am in the same boat here in that until now I saw Ellipsis as a thing used
by numpy and adjacent libraries in slicing. I don't think I have ever
written a slice with "..." in it so please forgive any ignorance but wasn't
the reason for having Ellipsis originally that None already had a special
meaning to the slice object and a different sentinel was needed? In other
words the normal case of a missing argument being None caused specific side
effects that needed to be skipped, so a sentinel was created to distinguish
these cases, just like we are discussing for other objects? It seems to me
the current situation has more in common than not. Numpy could have created
its own Sentinel object for its own use in these sliceses, just as it now
creates its own Sentinel for NoValue, but somewhere along the way it was
determined to be in the better interest of the community to have this
object defined in the standard library so it could be shared by multiple
scientific libraries without additional dependency burden, even though it
had no use to CPython.

I think there was perhaps a time when Ellipsis was so closely tied to the
scientific libraries that recommending its use anywhere else would have
been a poor idea, probably mostly because of the body of existing teaching
around its use; but I also think that association is no longer as strong.
"..." is being adopted across the typing landscape with several PEPs, such
as 483, outlining their special meaning in annotations. In addition, stub
files have since the beginning normalized on putting "..." as the
function body despite already having the working conventions of `pass`
`NotImplemented` or just a good old empty docstring """""" filling the same
purpose in standard python files.

On 5/20/2021 3:24 PM, Ronald Oussoren via Python-Dev wrote:
>
>
>> On 20 May 2021, at 19:10, Luciano Ramalho <luciano@ramalho.org
>> <mailto:luciano@ramalho.org>> wrote:
>>
>> I'd like to learn about use cases where `...` (a.k.a. `Ellipsis`) is
>> not a good sentinel. It's a pickable singleton testable with `is`,
>> readily available, and extremely unlikely to appear in a data stream.
>> Its repr is "Ellipsis".
>>
>> If you don't like the name for this purpose, you can always define a
>> constant (that won't fix the `repr`, obviously, but helps with source
>> code readability).
>>
>> SENTINEL = ...
>>
>> I can't think of any case where I'd rather have my own custom
>> sentinel, or need a special API for sentinels. Probably my fault, of
>> course. Please enlighten me!
>
> One use case for a sentinel that is not a predefined (builtin)
> singleton is APIs where an arbitrary user specified value can be used.
>
> One example of this is the definition of dataclasses.field:
>
> |dataclasses.||field|(/*/, /default=MISSING/,
> /default_factory=MISSING/, /repr=True/, /hash=None/, /init=True/,
> /compare=True/, /metadata=None/)
>
> Here the “default” and “default_factory” can be an arbitrary value,
> and any builtin singleton could be used. Hence the use of a custom
> module-private sentinel that cannot clash with values used by users of
> the module (unless those users poke at private details of the module,
> but then all bets are off anyway).
>
> That’s why I don’t particularly like the proposal of using Ellipsis as
> the sanctioned sentinel value. It would be weird at best that the
> default for a dataclass field can be any value, except for the builtin
> Ellipsis value.

Completely agree. I'm opposed to Ellipsis as a sentinel for this reason,
at least for dataclasses. I can easily see wanting to store an Ellipsis
in a field of a dataclass that's describing a function's parameters. And
I can even see it being the default= value. Not so much
default_factory=, but they may as well be the same.

Eric