Mailing List Archive

Look free ID genertion (was: Is there a more efficient threading lock?)
Chris Angelico wrote at 2023-3-1 12:58 +1100:
> ...
> The
>atomicity would be more useful in that context as it would give
>lock-free ID generation, which doesn't work in Python.

I have seen `itertools.count` for that.
This works because its `__next__` is implemented in "C" and
therefore will not be interrupted by a thread switch.
--
https://mail.python.org/mailman/listinfo/python-list
RE: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
If a workaround like itertools.count.__next__() is used because it will not
be interrupted as it is implemented in C, then I have to ask if it would
make sense for Python to supply something similar in the standard library
for the sole purpose of a use in locks.

But realistically, this is one place the concept of an abstract python
language intersects aspects of what is bundled into a sort of core at or
soon after startup, as well as the reality that python can be implemented in
many ways including some ways on some hardware that may not make guarantees
to behave this way.

Realistically, the history of computing is full of choices made that now
look less useful or obvious.

What would have happened if all processors had been required to have some
low level instruction that effectively did something in an atomic way that
allowed a way for anyone using any language running on that machine a way to
do a simple thing like set a lock or check it?

Of course life has also turned out to be more complex. Some architectures
can now support a small number of operations and implement others as sort of
streams of those operations liked together. You would need to be sure your
program is very directly using the atomic operation directly.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On
Behalf Of Dieter Maurer
Sent: Wednesday, March 1, 2023 1:43 PM
To: Chris Angelico <rosuav@gmail.com>
Cc: python-list@python.org
Subject: Look free ID genertion (was: Is there a more efficient threading
lock?)

Chris Angelico wrote at 2023-3-1 12:58 +1100:
> ...
> The
>atomicity would be more useful in that context as it would give
>lock-free ID generation, which doesn't work in Python.

I have seen `itertools.count` for that.
This works because its `__next__` is implemented in "C" and therefore will
not be interrupted by a thread switch.
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On Thu, 2 Mar 2023 at 06:37, <avi.e.gross@gmail.com> wrote:
>
> If a workaround like itertools.count.__next__() is used because it will not
> be interrupted as it is implemented in C, then I have to ask if it would
> make sense for Python to supply something similar in the standard library
> for the sole purpose of a use in locks.

That's not lock-free :) The only way that it works is because it's
locked against other threads doing the same job. Lock-free ID
generation means that:

1) Two threads can request IDs simultaneously and will not block each other
2) No two "request an ID" calls will ever return the same value
3) Preferably (but not required), IDs are not wasted.

PostgreSQL has ways of doing this, and there are a few other ways, but
simply using a count object and relying on the GIL isn't going to
achieve the first (though it'll happily achieve the other two).

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On 2023-03-01 at 14:35:35 -0500,
avi.e.gross@gmail.com wrote:

> What would have happened if all processors had been required to have
> some low level instruction that effectively did something in an atomic
> way that allowed a way for anyone using any language running on that
> machine a way to do a simple thing like set a lock or check it?

Have happened? I don't know about "required," but processors have
indeed had such instructions for decades; e.g., the MC68000 from the
early to mid 1980s (and used in the original Apple Macintosh, but I
digress) has/had a Test and Set instruction.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>
> On 2023-03-01 at 14:35:35 -0500,
> avi.e.gross@gmail.com wrote:
>
> > What would have happened if all processors had been required to have
> > some low level instruction that effectively did something in an atomic
> > way that allowed a way for anyone using any language running on that
> > machine a way to do a simple thing like set a lock or check it?
>
> Have happened? I don't know about "required," but processors have
> indeed had such instructions for decades; e.g., the MC68000 from the
> early to mid 1980s (and used in the original Apple Macintosh, but I
> digress) has/had a Test and Set instruction.

As have all CPUs since; it's the only way to implement locks (push the
locking all the way down to the CPU level).

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
So I guess we know what would have happened.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Python-list <python-list-bounces+gweatherby=uchc.edu@python.org> on behalf of Chris Angelico <rosuav@gmail.com>
Sent: Wednesday, March 1, 2023 8:45:50 PM
To: python-list@python.org <python-list@python.org>
Subject: Re: Look free ID genertion (was: Is there a more efficient threading lock?)

*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>
> On 2023-03-01 at 14:35:35 -0500,
> avi.e.gross@gmail.com wrote:
>
> > What would have happened if all processors had been required to have
> > some low level instruction that effectively did something in an atomic
> > way that allowed a way for anyone using any language running on that
> > machine a way to do a simple thing like set a lock or check it?
>
> Have happened? I don't know about "required," but processors have
> indeed had such instructions for decades; e.g., the MC68000 from the
> early to mid 1980s (and used in the original Apple Macintosh, but I
> digress) has/had a Test and Set instruction.

As have all CPUs since; it's the only way to implement locks (push the
locking all the way down to the CPU level).

ChrisA
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!g70S067RzF2oPCFUYpFFzUvPHRfS0AHIGEvVyww1Tlj7BCCrsU3DWIqCE9UBO_ex0ZVanquFLHGe1d2b$
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On 2023-03-02, Chris Angelico <rosuav@gmail.com> wrote:
> On Thu, 2 Mar 2023 at 08:01, <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>> On 2023-03-01 at 14:35:35 -0500,
>> avi.e.gross@gmail.com wrote:
>> > What would have happened if all processors had been required to have
>> > some low level instruction that effectively did something in an atomic
>> > way that allowed a way for anyone using any language running on that
>> > machine a way to do a simple thing like set a lock or check it?
>>
>> Have happened? I don't know about "required," but processors have
>> indeed had such instructions for decades; e.g., the MC68000 from the
>> early to mid 1980s (and used in the original Apple Macintosh, but I
>> digress) has/had a Test and Set instruction.
>
> As have all CPUs since; it's the only way to implement locks (push the
> locking all the way down to the CPU level).

Indeed, I remember thinking it was very fancy when they added the SWP
instruction to the ARM processor.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On Thu, 2 Mar 2023 at 13:02, Weatherby,Gerard <gweatherby@uchc.edu> wrote:
>
> So I guess we know what would have happened.
>

Yep. It's not what I was talking about, but it's also a very important
concurrency management feature.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Look free ID genertion (was: Is there a more efficient threading lock?) [ In reply to ]
On Thu, 2 Mar 2023 12:45:50 +1100, Chris Angelico <rosuav@gmail.com>
declaimed the following:

>
>As have all CPUs since; it's the only way to implement locks (push the
>locking all the way down to the CPU level).
>

Xerox Sigma (circa 1970): Modify and Test (byte/halfword/word)

Granted, that was a "mainframe" system, not a microprocessor.

Looks like Intel didn't catch the boat until 1985 and the i386.



--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/
--
https://mail.python.org/mailman/listinfo/python-list