Mailing List Archive

Woa! The JavaScript JIT compiler is quite impressive. I now
ported Dogelog runtime to Python as well, so that I can compare
JavaScript and Python, and tested without clause indexing:

between(L,H,L) :- L =< H.
between(L,H,X) :- L < H, Y is L+1, between(Y,H,X).

setup :- between(1,255,N), M is N//2, assertz(edge(M,N)), fail.
setup :- edge(M,N), assertz(edge2(N,M)), fail.
setup.

anc(X,Y) :- edge(X, Y).
anc(X,Y) :- edge(X, Z), anc(Z, Y).

anc2(X,Y) :- edge2(Y, X).
anc2(X,Y) :- edge2(Y, Z), anc2(X, Z).

:- setup.
:- time((between(1,10,_), anc2(0,255), fail; true)).
:- time((between(1,10,_), anc(0,255), fail; true)).

The results are:

/* Python 3.10.0rc1 */
% Wall 188 ms, trim 0 ms
% Wall 5537 ms, trim 0 ms

/* JavaScript Chrome 92.0.4515.159 */
% Wall 5 ms, trim 0 ms
% Wall 147 ms, trim 0 ms
--
https://mail.python.org/mailman/listinfo/python-list

Thats a factor 37.8 faster! I tested the a variant of
the Albufeira instructions Prolog VM aka ZIP, which
was also the inspiration for SWI-Prolog.

Open Source:

The Python Version of the Dogelog Runtime
https://github.com/jburse/dogelog-moon/tree/main/devel/runtimepy

The Python Test Harness
https://gist.github.com/jburse/bf6c01c7524f2611d606cb88983da9d6#file-test-py


Mostowski Collapse schrieb:
> Woa! The JavaScript JIT compiler is quite impressive. I now
> ported Dogelog runtime to Python as well, so that I can compare
> JavaScript and Python, and tested without clause indexing:
>
> between(L,H,L) :- L =< H.
> between(L,H,X) :- L < H, Y is L+1, between(Y,H,X).
>
> setup :- between(1,255,N), M is N//2, assertz(edge(M,N)), fail.
> setup :- edge(M,N), assertz(edge2(N,M)), fail.
> setup.
>
> anc(X,Y) :- edge(X, Y).
> anc(X,Y) :- edge(X, Z), anc(Z, Y).
>
> anc2(X,Y) :- edge2(Y, X).
> anc2(X,Y) :- edge2(Y, Z), anc2(X, Z).
>
> :- setup.
> :- time((between(1,10,_), anc2(0,255), fail; true)).
> :- time((between(1,10,_), anc(0,255), fail; true)).
>
> The results are:
>
> /* Python 3.10.0rc1 */
> % Wall 188 ms, trim 0 ms
> % Wall 5537 ms, trim 0 ms
>
> /* JavaScript Chrome 92.0.4515.159 */
> % Wall 5 ms, trim 0 ms
> % Wall 147 ms, trim 0 ms

--
https://mail.python.org/mailman/listinfo/python-list

We = Me and my cat named socrates
The cat is a very good programmer:

def meow():
print("meow meow, Prolog is not only SWI-Prolog")

Julio Di Egidio schrieb:
> On Sunday, 15 August 2021 at 14:43:42 UTC+2, Mostowski Collapse wrote:
>
>> Yesterday we went into a little programming binge
>
> Who is this "we"?
>
>> See also:
>>
>> Python Version of Dogelog Runtime special
>> https://twitter.com/dogelogch/status/1426884473988292617
>>
>> Python Version of Dogelog Runtime special
>> https://www.facebook.com/groups/dogelog
>
> I haven't tried either but, assuming the code works ;), great stuff man.
>
> You might be the one(s) who save Prolog from oblivion...
>
> Julio
>

--
https://mail.python.org/mailman/listinfo/python-list

The world is getting ridiculous. termux seems to
not anymore be supported by Google Play because
of some Android 10 issues?

On the other hand there is this gem:

TI 84 Plus CE Python Edition Unboxing
https://www.youtube.com/watch?v=LVxP_Fki8Fc

LoL

Mostowski Collapse schrieb:
> Yesterday we went into a little programming binge, despite there
> was a free parade in Zurich. We could now already implement a transpiler
> that targets Python. We simply took the transpiler main.p that targets
>
> JavaScript and moded it into a new transpiler mainpy.p that targets
> Python. The code is already on GitHub and we present it also here
> as the Python code mainpy.p. We were also busy
>
> on machine.py and special.py. The progress is now:
>
> +------------+   cross      +-------------+
> | loader.p   |   compile    | loader.py   | 100%
> | compiler.p | -----------> | compiler.py | 100%
> +------------+              +-------------+
>                             | machine.py  |  66%
>                             | special.py  |  33%
>                             +-------------+
>
> See also:
>
> Python Version of Dogelog Runtime special
> https://twitter.com/dogelogch/status/1426884473988292617
>
> Python Version of Dogelog Runtime special
> https://www.facebook.com/groups/dogelog

--
https://mail.python.org/mailman/listinfo/python-list

Having fun with a new attended query answerer for
the Dogelog runtime. It can be bottled into a single Python
file, no server roundtrip, just ISO core Prolog in one

Python file, requires Python 3.10:

>python.exe toplevel.py
Dogelog Runtime, Prolog to the Moon, 0.9.3
(c) 1985-2021, XLOG Technologies AG, Switzerland
?- X=1; X=2.
X = 1;
X = 2.
?- X= ... ; X = ... .
X = ...;
X = ... .
?-

So we adopted displaying a dot when there are no more
choice points. This is seen in SWI-Prolog for example, but
was also adopted by Scryer Prolog recently.

Note the space between '...' and '.' in the last answer of
the last query. This is needed so that what is shown
is copyable. The query answerer is described here:

Dogelog Runtime attended Prolog query answers. (Jekejeke)
https://twitter.com/dogelogch/status/1430647215928877065

Dogelog Runtime attended Prolog query answers. (Jekejeke)
https://www.facebook.com/groups/dogelog
--
https://mail.python.org/mailman/listinfo/python-list

More best kept secrets of Prolog: Pattern Matching

Everybody loves pattern matching. Languages
like Python, release 3.10, even provide it
now. There is now a match/case statement

in Python. But Prolog users will scratch their
head. Will my if-then-else be as fast as a
imperative switch jump table lookup?

Dogelog runtime has stepped up its game
concerning pattern matching. It now provides
ECLiPSe Prolog disjunction and if-then-else

indexing. Take this example:

?- [user].
foo(X,Y) :- X=baz, Y=2; X=bar -> Y=1.

SWI-Prolog leaves a choice point, so no
clause indexing used:

/* SWI-Prolog 8.3.26 */
?- foo(baz,Z).
Z = 2 ; %%% Spurious Choice Point
false.

Dogelog doesn't leave a choice point, since
it can index the disjunction and if-then-else:

/* Dogelog Runtime 0.9.3 */
?- foo(baz,Z).
Z = 2. %%% No Choice Point

See also:

Preview: Dogelog disjunction and if-then-else indexing. (Jekejeke)
https://twitter.com/dogelogch/status/1433446729974796293

Preview: Dogelog disjunction and if-then-else indexing. (Jekejeke)
https://www.facebook.com/groups/dogelog
--
https://mail.python.org/mailman/listinfo/python-list

The Standard Python version of Dogelog runtime
is annoyingly slow. So we gave it a try with
andother Python, and it was 6x times faster.

We could test GraalVM. We worked around the missing
match in Python 3.8 by replacing it with if-then-else.
Performance is a little better, we find:

/* Standard Python Version, Warm Run */
?- time(fibo(23,X)).
% Wall 3865 ms, gc 94 ms, 71991 lips
X = 46368.

/* GraalVM Python Version, Warm Warm Run */
?- time(fibo(23,X)).
% Wall 695 ms, gc 14 ms, 400356 lips
X = 46368.

See also:

JDK 1.8 GraalVM Python is 6x faster than Standard Python
https://twitter.com/dogelogch/status/1437395917167112193

JDK 1.8 GraalVM Python is 6x faster than Standard Python
https://www.facebook.com/groups/dogelog

Mostowski Collapse schrieb:
> Yesterday we went into a little programming binge, despite there
> was a free parade in Zurich. We could now already implement a transpiler
> that targets Python. We simply took the transpiler main.p that targets
>
> JavaScript and moded it into a new transpiler mainpy.p that targets
> Python. The code is already on GitHub and we present it also here
> as the Python code mainpy.p. We were also busy
>
> on machine.py and special.py. The progress is now:
>
> +------------+   cross      +-------------+
> | loader.p   |   compile    | loader.py   | 100%
> | compiler.p | -----------> | compiler.py | 100%
> +------------+              +-------------+
>                             | machine.py  |  66%
>                             | special.py  |  33%
>                             +-------------+
>
> See also:
>
> Python Version of Dogelog Runtime special
> https://twitter.com/dogelogch/status/1426884473988292617
>
> Python Version of Dogelog Runtime special
> https://www.facebook.com/groups/dogelog

--
https://mail.python.org/mailman/listinfo/python-list

On 9/13/2021 8:46 AM, Mostowski Collapse wrote:
> The Standard Python version of Dogelog runtime
> is annoyingly slow. So we gave it a try with
> andother Python, and it was 6x times faster.
>
> We could test GraalVM. We worked around the missing
> match in Python 3.8 by replacing it with if-then-else.
> Performance is a little better, we find:
>
> /* Standard Python Version, Warm Run */
> ?- time(fibo(23,X)).
> % Wall 3865 ms, gc 94 ms, 71991 lips
> X = 46368.
>
> /* GraalVM Python Version, Warm Warm Run */
> ?- time(fibo(23,X)).
> % Wall 695 ms, gc 14 ms, 400356 lips
> X = 46368.
>
> See also:
>
> JDK 1.8 GraalVM Python is 6x faster than Standard Python
> https://twitter.com/dogelogch/status/1437395917167112193
>
> JDK 1.8 GraalVM Python is 6x faster than Standard Python
> https://www.facebook.com/groups/dogelog

You need to test more than fibonacci to make that claim. There is a
benchmark test that times around 40 different similarly small benchmarks.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

I am testing a Prolog interpreter written
in Python. So fibonacci number routine is
written in Prolog and I am running the

fibonnaci number routine inside the
Prolog interpreter that is written in
Python. The factor 6x times faster of

GraalVM can be reproduced also for other
Prolog programs running inside the Prolog
interpreter that is written in Python.

I have a benchmark suite, where I get,
the figures are milliseconds:

Test Standard GraalVM
Total 170'996 28'523

This means the factor is:

170'996 / 28'523 = 5.9950

The test harness, test cases and individual
results for all test cases are found here:

And we could test GraalVM Python, results are from 14.09.2021,
tested with Dogelog Runtime 0.9.5, Python Version:
https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587

Terry Reedy wrote:
> On 9/13/2021 8:46 AM, Mostowski Collapse wrote:
>> The Standard Python version of Dogelog runtime
>> is annoyingly slow. So we gave it a try with
>> andother Python, and it was 6x times faster.
>>
>> We could test GraalVM. We worked around the missing
>> match in Python 3.8 by replacing it with if-then-else.
>> Performance is a little better, we find:
>>
>> /* Standard Python Version, Warm Run */
>> ?- time(fibo(23,X)).
>> % Wall 3865 ms, gc 94 ms, 71991 lips
>> X = 46368.
>>
>> /* GraalVM Python Version, Warm Warm Run */
>> ?- time(fibo(23,X)).
>> % Wall 695 ms, gc 14 ms, 400356 lips
>> X = 46368.
>>
>> See also:
>>
>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>> https://twitter.com/dogelogch/status/1437395917167112193
>>
>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>> https://www.facebook.com/groups/dogelog
>
> You need to test more than fibonacci to make that claim.  There is a
> benchmark test that times around 40 different similarly small benchmarks.
>
>

--
https://mail.python.org/mailman/listinfo/python-list

But even when using GraalVM Python, the
execution is still slow. Much slower
then the same Prolog interpreter written

in JavaScript. For JavaScript node.exe
I get much better results. I get these
results comparing to a few other new

Prolog systems as well:

Test Dogelog Scryer Trealla
Total 4427 7325 1824

The test harness, test cases and individual
results for all test cases are found here:

Meanwhile could also test Trealla, results are from 12.09.2021,
tested with Dogelog Runtime 0.9.5, JavaScript Version:
https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3890013

But this is all only a moment in time.
The Prolog interpreter itself is evolving,
and the programming languages also,

so this is a moving target. Benchmark
results will look different tomorrow.

Mostowski Collapse wrote:
> I am testing a Prolog interpreter written
> in Python. So fibonacci number routine is
> written in Prolog and I am running the
>
> fibonnaci number routine inside the
> Prolog interpreter that is written in
> Python. The factor 6x times faster of
>
> GraalVM can be reproduced also for other
> Prolog programs running inside the Prolog
> interpreter that is written in Python.
>
> I have a benchmark suite, where I get,
> the figures are milliseconds:
>
> Test    Standard    GraalVM
> Total     170'996      28'523
>
> This means the factor is:
>
> 170'996 / 28'523 = 5.9950
>
> The test harness, test cases and individual
> results for all test cases are found here:
>
> And we could test GraalVM Python, results are from 14.09.2021,
> tested with Dogelog Runtime 0.9.5, Python Version:
> https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587
>
>
> Terry Reedy wrote:
>> On 9/13/2021 8:46 AM, Mostowski Collapse wrote:
>>> The Standard Python version of Dogelog runtime
>>> is annoyingly slow. So we gave it a try with
>>> andother Python, and it was 6x times faster.
>>>
>>> We could test GraalVM. We worked around the missing
>>> match in Python 3.8 by replacing it with if-then-else.
>>> Performance is a little better, we find:
>>>
>>> /* Standard Python Version, Warm Run */
>>> ?- time(fibo(23,X)).
>>> % Wall 3865 ms, gc 94 ms, 71991 lips
>>> X = 46368.
>>>
>>> /* GraalVM Python Version, Warm Warm Run */
>>> ?- time(fibo(23,X)).
>>> % Wall 695 ms, gc 14 ms, 400356 lips
>>> X = 46368.
>>>
>>> See also:
>>>
>>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>>> https://twitter.com/dogelogch/status/1437395917167112193
>>>
>>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>>> https://www.facebook.com/groups/dogelog
>>
>> You need to test more than fibonacci to make that claim.  There is a
>> benchmark test that times around 40 different similarly small benchmarks.
>>
>>
>

--
https://mail.python.org/mailman/listinfo/python-list

Opinion: Anyone who is counting on Python for truly fast compute speed is probably using Python for the wrong purpose.
Here, we use Python to control Test Equipment, to set up the equipment and ask for a measurement, get it, and proceed to the next measurement; and at the end produce a nice formatted report. If we wrote the test script in C or Rust or whatever it could not run substantially faster because it is communicating with the test equipment, setting it up and waiting for responses, and that is where the vast majority of the time goes. Especially if the measurement result requires averaging it can take a while. In my opinion this is an ideal use for Python, not just because the speed of Python is not important, but also because we can easily find people who know Python, who like coding in Python, and will join the company to program in Python ... and stay with us.

--- Joseph S.


Teledyne Confidential; Commercially Sensitive Business Data

-----Original Message-----
From: Mostowski Collapse <janburse@fastmail.fm>
Sent: Tuesday, September 14, 2021 8:56 AM
To: python-list@python.org
Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021)

I am testing a Prolog interpreter written in Python. So fibonacci number routine is written in Prolog and I am running the

fibonnaci number routine inside the
Prolog interpreter that is written in
Python. The factor 6x times faster of

GraalVM can be reproduced also for other Prolog programs running inside the Prolog interpreter that is written in Python.

I have a benchmark suite, where I get,
the figures are milliseconds:

Test Standard GraalVM
Total 170'996 28'523

This means the factor is:

170'996 / 28'523 = 5.9950

The test harness, test cases and individual results for all test cases are found here:

And we could test GraalVM Python, results are from 14.09.2021, tested with Dogelog Runtime 0.9.5, Python Version:
https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587

Terry Reedy wrote:
> On 9/13/2021 8:46 AM, Mostowski Collapse wrote:
>> The Standard Python version of Dogelog runtime is annoyingly slow. So
>> we gave it a try with andother Python, and it was 6x times faster.
>>
>> We could test GraalVM. We worked around the missing match in Python
>> 3.8 by replacing it with if-then-else.
>> Performance is a little better, we find:
>>
>> /* Standard Python Version, Warm Run */
>> ?- time(fibo(23,X)).
>> % Wall 3865 ms, gc 94 ms, 71991 lips
>> X = 46368.
>>
>> /* GraalVM Python Version, Warm Warm Run */
>> ?- time(fibo(23,X)).
>> % Wall 695 ms, gc 14 ms, 400356 lips
>> X = 46368.
>>
>> See also:
>>
>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>> https://twitter.com/dogelogch/status/1437395917167112193
>>
>> JDK 1.8 GraalVM Python is 6x faster than Standard Python
>> https://www.facebook.com/groups/dogelog
>
> You need to test more than fibonacci to make that claim.  There is a
> benchmark test that times around 40 different similarly small benchmarks.
>
>

--
https://mail.python.org/mailman/listinfo/python-list

I am not testing this use-case. But a related
use-case might highlight why speed did never
hurt anybody.

Lets say you program a flying drone with Python,
and the measurement is from the drone sensor
and communication systems.

Lets say you are using the idle time between
measurements for some complex planning. It
is then not true that you have anyway

to wait for the measurement.

Hope this helps!

BTW: If somebody knows another Python implementation
I am happy to test this implementation as well.
I am assuming that the standard Python python.exe

I tested amounts to CPython? Not sure. And the
GraalVM is practically the same as JPython? Not
sure either.

> Opinion: Anyone who is counting on Python
> for truly fast compute speed is probably using
> Python for the wrong purpose.
> Here, we use Python to control Test Equipment,
> to set up the equipment and ask for a measurement,
> get it, and proceed to the next measurement; and
> at the end produce a nice formatted report.
> If we wrote the test script in C or Rust or
> whatever it could not run substantially faster
> because it is communicating with the test equipment,
> setting it up and waiting for responses, and
> that is where the vast majority of the time goes.
> Especially if the measurement result requires
> averaging it can take a while. In my opinion
> this is an ideal use for Python, not just because
> the speed of Python is not important, but also
> because we can easily find people who know Python,
> who like coding in Python, and will join the
> company to program in Python ... and stay with us.
>
> --- Joseph S.

--
https://mail.python.org/mailman/listinfo/python-list

Oops "speed did never hurt anybody". Don't be
evil, I am talking about unarmed drones.

See also:

Drone Programming With Python Course
https://www.youtube.com/watch?v=LmEcyQnfpDA

Mostowski Collapse schrieb:
> I am not testing this use-case. But a related
> use-case might highlight why speed did never
> hurt anybody.
>
> Lets say you program a flying drone with Python,
> and the measurement is from the drone sensor
> and communication systems.
>
> Lets say you are using the idle time between
> measurements for some complex planning. It
> is then not true that you have anyway
>
> to wait for the measurement.
>
> Hope this helps!
>
> BTW: If somebody knows another Python implementation
> I am happy to test this implementation as well.
> I am assuming that the standard Python python.exe
>
> I tested amounts to CPython? Not sure. And the
> GraalVM is practically the same as JPython? Not
> sure either.
>
>> Opinion:   Anyone who is counting on Python for truly fast compute
>> speed is probably using Python for the wrong purpose. Here, we use
>> Python to control Test Equipment, to set up the equipment and ask for
>> a measurement, get it, and proceed to the next measurement; and at the
>> end produce a nice formatted report. If we wrote the test script in C
>> or Rust or whatever it could not run substantially faster because it
>> is communicating with the test equipment, setting it up and waiting
>> for responses, and that is where the vast majority of the time goes.
>> Especially if the measurement result requires averaging it can take a
>> while.  In my opinion this is an ideal use for Python, not just
>> because the speed of Python is not important, but also because we can
>> easily find people who know Python, who like coding in Python, and
>> will join the company to program in Python ... and stay with us.
>> --- Joseph S.
>

--
https://mail.python.org/mailman/listinfo/python-list

I really wonder why my Python implementation
is a factor 40 slower than my JavaScript implementation.
Structurally its the same code.

You can check yourself:

Python Version:
https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py

JavaScript Version:
https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js

Its the same while, if-then-else, etc.. its the same
classes Variable, Compound etc.. Maybe I could speed
it up by some details. For example to create an array
of length n, I use in Python:

temp = [NotImplemented] * code[pos]
pos += 1

Whereas in JavaScript I use, also
in exec_build2():

temp = new Array(code[pos++]);

So I hear Guido doesn't like ++. So in Python I use +=
and a separate statement as a workaround. But otherwise,
what about the creation of an array,

is the the idiom [_] * _ slow? I am assuming its
compiled away. Or does it really first create an
array of size 1 and then enlarge it?

Julio Di Egidio wrote:
> On Wednesday, 15 September 2021 at 15:37:19 UTC+2, Mostowski Collapse wrote:
>
>>> Opinion: Anyone who is counting on Python
>>> for truly fast compute speed is probably using
>>> Python for the wrong purpose.
>
> You just don't know anything about this environment: those who need fast computation rather use *libraries* where all the performance critical parts are written in native code... and that's pretty customary in Python.
>
> By that I don't mean Python is flawless, indeed (IMO) it isn't in so many ways: to the point that, for more professional solutions in the maths/statistics realms in particular, people rather use R: yet, the primary reason is not so much performance but really the solidity/structure of the language per se...
>
> Julio
>

--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>
> I really wonder why my Python implementation
> is a factor 40 slower than my JavaScript implementation.
> Structurally its the same code.
>

Very hard to know. Your code is detailed and complicated. Do they
produce identical results? Are you using the same sort of
floating-point data everywhere, or is one integer and the other float?
What's going on with all the globals, the continuations, etc? My
suspicion is that you're trying to write weird, wonky Python code, and
then are surprised that it doesn't perform well.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

If you find a "wonky" spot, I can replace it by "non-wonky"
code. I noticed some differences between Python Dicts
and JavaScript objects. Python tends to throw more exceptions.

So in Python I now do the following:

peek = kb.get(functor, NotImplemented)
if peek is not NotImplemented:

In JavaScript I can directly do:

peek = kb[functor];
if (peek !== undefined)

But if get() in Python is implemented under the hood with
exception handling. i.e. using the exception prone [] and
then in case an exception is thrown, returning the

default value, then Python get() will probably be quite slow.
Since usually exceptions are slow.

Chris Angelico schrieb am Mittwoch, 15. September 2021 um 19:27:13 UTC+2:
> On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse <janb...@fastmail.fm> wrote:
> >
> > I really wonder why my Python implementation
> > is a factor 40 slower than my JavaScript implementation.
> > Structurally its the same code.
> >
> Very hard to know. Your code is detailed and complicated. Do they
> produce identical results? Are you using the same sort of
> floating-point data everywhere, or is one integer and the other float?
> What's going on with all the globals, the continuations, etc? My
> suspicion is that you're trying to write weird, wonky Python code, and
> then are surprised that it doesn't perform well.
>
> ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:

> I really wonder why my Python implementation is a factor 40 slower than
> my JavaScript implementation.
> Structurally its the same code.
>
> You can check yourself:
>
> Python Version:
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
machine.py
>
> JavaScript Version:
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
machine.js
>
> Its the same while, if-then-else, etc.. its the same classes Variable,
> Compound etc.. Maybe I could speed it up by some details. For example to
> create an array of length n, I use in Python:
>
> temp = [NotImplemented] * code[pos]
> pos += 1
>
> Whereas in JavaScript I use, also in exec_build2():
>
> temp = new Array(code[pos++]);
>
> So I hear Guido doesn't like ++. So in Python I use +=
> and a separate statement as a workaround. But otherwise,
> what about the creation of an array,
>
> is the the idiom [_] * _ slow? I am assuming its compiled away. Or does
> it really first create an array of size 1 and then enlarge it?
>
> Julio Di Egidio wrote:
<sniped due to top posting>

this is probably a string contender

i = 0
while i < len(term.args) - 1:
mark_term(term.args[i])
i += 1
term = term.args[i]

try replacing with something more pythonic

for index,term in enumerate(term.args):
mark_term(term.args[i])


& possibly go all the way to changing it into a comprehension

there are other similar anti patterns throughout this code.

any time you are manually keeping a counter as an index into a list,tupple
other iterable YOU ARE DOING IT WRONG!

Do not write javascript in python, write python



--
Two percent of zero is almost nothing.




--
Whoever dies with the most toys wins.
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, 16 Sep 2021 03:26:39 +1000, Chris Angelico wrote:

> On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse
> <janburse@fastmail.fm> wrote:
>>
>> I really wonder why my Python implementation is a factor 40 slower than
>> my JavaScript implementation.
>> Structurally its the same code.
>>
>>
> Very hard to know. Your code is detailed and complicated. Do they
> produce identical results? Are you using the same sort of floating-point
> data everywhere, or is one integer and the other float?
> What's going on with all the globals, the continuations, etc? My
> suspicion is that you're trying to write weird, wonky Python code, and
> then are surprised that it doesn't perform well.
>
> ChrisA

And this demonstrates how an experienced Python programmer can make an
almost spot on diagnosis without even checking the source code.

@ this stage I would recommend watching some presentations on you tube

this one https://www.youtube.com/watch?v=wf-BqAjZb8M by Raymond Hettinger
is brilliant as it highlights there is more to checking code than just
making sure it looks nice & runs correctly.



--
Lemmings don't grow older, they just die.
--
https://mail.python.org/mailman/listinfo/python-list

Do you mean, replace this:

i = 0
while i < len(term.args) - 1:
____mark_term(term.args[i])
____i += 1
term = term.args[i]

By this:

for i,term in enumerate(term.args):
____mark_term(term.args[i])

This wouldn't be correct anymore. The
recursive call is only for the arguments
except for the last one one.

alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
> On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
>
> > I really wonder why my Python implementation is a factor 40 slower than
> > my JavaScript implementation.
> > Structurally its the same code.
> >
> > You can check yourself:
> >
> > Python Version:
> > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
> machine.py
> >
> > JavaScript Version:
> > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
> machine.js
> >
> > Its the same while, if-then-else, etc.. its the same classes Variable,
> > Compound etc.. Maybe I could speed it up by some details. For example to
> > create an array of length n, I use in Python:
> >
> > temp = [NotImplemented] * code[pos]
> > pos += 1
> >
> > Whereas in JavaScript I use, also in exec_build2():
> >
> > temp = new Array(code[pos++]);
> >
> > So I hear Guido doesn't like ++. So in Python I use +=
> > and a separate statement as a workaround. But otherwise,
> > what about the creation of an array,
> >
> > is the the idiom [_] * _ slow? I am assuming its compiled away. Or does
> > it really first create an array of size 1 and then enlarge it?
> >
> > Julio Di Egidio wrote:
> <sniped due to top posting>
>
> this is probably a string contender
>
> i = 0
> while i < len(term.args) - 1:
> mark_term(term.args[i])
> i += 1
> term = term.args[i]
>
> try replacing with something more pythonic
>
> for index,term in enumerate(term.args):
> mark_term(term.args[i])
>
>
> & possibly go all the way to changing it into a comprehension
>
> there are other similar anti patterns throughout this code.
>
> any time you are manually keeping a counter as an index into a list,tupple
> other iterable YOU ARE DOING IT WRONG!
>
> Do not write javascript in python, write python
>
>
>
> --
> Two percent of zero is almost nothing.
>
>
>
>
> --
> Whoever dies with the most toys wins.
--
https://mail.python.org/mailman/listinfo/python-list

Well I would be more than happy if an experienced
programmer can fine tune my code. My programming
experience in Python is only 4 weeks.

Mostowski Collapse schrieb am Dienstag, 14. September 2021 um 14:56:35 UTC+2:
> The test harness, test cases and individual
> results for all test cases are found here:
>
> And we could test GraalVM Python, results are from 14.09.2021,
> tested with Dogelog Runtime 0.9.5, Python Version:
> https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#gistcomment-3892587

If you follow the above link, you also find:

Test Harness
https://gist.github.com/jburse/f4e774ebb15cac722238b26b1a620f84#file-suite2-pl

Test Cases
https://github.com/jburse/jekejeke-samples/tree/master/jekrun_bench/core/tests

CPU / RAM: Intel(R) Core(TM) i7-6700HQ, 32 GB
Standard Python: python.exe Python 3.10.0rc1
GraalVM Python: WSL 2, Python 3.8.5 (GraalVM CE Native 21.2.0)

alister schrieb am Mittwoch, 15. September 2021 um 20:22:56 UTC+2:
> On Thu, 16 Sep 2021 03:26:39 +1000, Chris Angelico wrote:
>
> > On Thu, Sep 16, 2021 at 3:17 AM Mostowski Collapse
> > <janb...@fastmail.fm> wrote:
> >>
> >> I really wonder why my Python implementation is a factor 40 slower than
> >> my JavaScript implementation.
> >> Structurally its the same code.
> >>
> >>
> > Very hard to know. Your code is detailed and complicated. Do they
> > produce identical results? Are you using the same sort of floating-point
> > data everywhere, or is one integer and the other float?
> > What's going on with all the globals, the continuations, etc? My
> > suspicion is that you're trying to write weird, wonky Python code, and
> > then are surprised that it doesn't perform well.
> >
> > ChrisA
> And this demonstrates how an experienced Python programmer can make an
> almost spot on diagnosis without even checking the source code.
>
> @ this stage I would recommend watching some presentations on you tube
>
> this one https://www.youtube.com/watch?v=wf-BqAjZb8M by Raymond Hettinger
> is brilliant as it highlights there is more to checking code than just
> making sure it looks nice & runs correctly.
>
>
>
> --
> Lemmings don't grow older, they just die.
--
https://mail.python.org/mailman/listinfo/python-list

There is a further problem with this:

> for i,term in enumerate(term.args):
> ____mark_term(term.args[i])

It should read:

for i,help in enumerate(term.args):
____mark_term(help)

But then i isn't need.

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50 UTC+2:
> Do you mean, replace this:
> i = 0
> while i < len(term.args) - 1:
> ____mark_term(term.args[i])
> ____i += 1
> term = term.args[i]
>
> By this:
>
> for i,term in enumerate(term.args):
> ____mark_term(term.args[i])
>
> This wouldn't be correct anymore. The
> recursive call is only for the arguments
> except for the last one one.
> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
> >
> > > I really wonder why my Python implementation is a factor 40 slower than
> > > my JavaScript implementation.
> > > Structurally its the same code.
> > >
> > > You can check yourself:
> > >
> > > Python Version:
> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
> > machine.py
> > >
> > > JavaScript Version:
> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
> > machine.js
> > >
> > > Its the same while, if-then-else, etc.. its the same classes Variable,
> > > Compound etc.. Maybe I could speed it up by some details. For example to
> > > create an array of length n, I use in Python:
> > >
> > > temp = [NotImplemented] * code[pos]
> > > pos += 1
> > >
> > > Whereas in JavaScript I use, also in exec_build2():
> > >
> > > temp = new Array(code[pos++]);
> > >
> > > So I hear Guido doesn't like ++. So in Python I use +=
> > > and a separate statement as a workaround. But otherwise,
> > > what about the creation of an array,
> > >
> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or does
> > > it really first create an array of size 1 and then enlarge it?
> > >
> > > Julio Di Egidio wrote:
> > <sniped due to top posting>
> >
> > this is probably a string contender
> >
> > i = 0
> > while i < len(term.args) - 1:
> > mark_term(term.args[i])
> > i += 1
> > term = term.args[i]
> >
> > try replacing with something more pythonic
> >
> > for index,term in enumerate(term.args):
> > mark_term(term.args[i])
> >
> >
> > & possibly go all the way to changing it into a comprehension
> >
> > there are other similar anti patterns throughout this code.
> >
> > any time you are manually keeping a counter as an index into a list,tupple
> > other iterable YOU ARE DOING IT WRONG!
> >
> > Do not write javascript in python, write python
> >
> >
> >
> > --
> > Two percent of zero is almost nothing.
> >
> >
> >
> >
> > --
> > Whoever dies with the most toys wins.
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote:

> There is a further problem with this:
>
>> for i,term in enumerate(term.args):
>> ____mark_term(term.args[i])
>
> It should read:
>
> for i,help in enumerate(term.args):
> ____mark_term(help)
>
> But then i isn't need.
even Better (i had only skimmed the code as I was certain I would find
this, it is probably the No. 1 thing new python programmers get wrong
if your example is correct the it can be simplified even further to

for help in term.args:
mark_term(help)

& if help does not get used after this loop then a comprehension is even
better
_ == [mark_term(help) for help in term.args]


the underscore character is python convention for an unneeded place-
holder variable.

>
> Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50
> UTC+2:
>> Do you mean, replace this:
>> i = 0 while i < len(term.args) - 1:
>> ____mark_term(term.args[i])
>> ____i += 1 term = term.args[i]
>>
>> By this:
>>
>> for i,term in enumerate(term.args):
>> ____mark_term(term.args[i])
>>
>> This wouldn't be correct anymore. The recursive call is only for the
>> arguments except for the last one one.
>> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
>> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
>> >
>> > > I really wonder why my Python implementation is a factor 40 slower
>> > > than my JavaScript implementation.
>> > > Structurally its the same code.
>> > >
>> > > You can check yourself:
>> > >
>> > > Python Version:
>> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
>> > machine.py
>> > >
>> > > JavaScript Version:
>> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
>> > machine.js
>> > >
>> > > Its the same while, if-then-else, etc.. its the same classes
>> > > Variable, Compound etc.. Maybe I could speed it up by some details.
>> > > For example to create an array of length n, I use in Python:
>> > >
>> > > temp = [NotImplemented] * code[pos]
>> > > pos += 1
>> > >
>> > > Whereas in JavaScript I use, also in exec_build2():
>> > >
>> > > temp = new Array(code[pos++]);
>> > >
>> > > So I hear Guido doesn't like ++. So in Python I use +=
>> > > and a separate statement as a workaround. But otherwise,
>> > > what about the creation of an array,
>> > >
>> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or
>> > > does it really first create an array of size 1 and then enlarge it?
>> > >
>> > > Julio Di Egidio wrote:
>> > <sniped due to top posting>
>> >
>> > this is probably a string contender
>> >
>> > i = 0 while i < len(term.args) - 1:
>> > mark_term(term.args[i])
>> > i += 1 term = term.args[i]
>> >
>> > try replacing with something more pythonic
>> >
>> > for index,term in enumerate(term.args):
>> > mark_term(term.args[i])
>> >
>> >
>> > & possibly go all the way to changing it into a comprehension
>> >
>> > there are other similar anti patterns throughout this code.
>> >
>> > any time you are manually keeping a counter as an index into a
>> > list,tupple other iterable YOU ARE DOING IT WRONG!
>> >
>> > Do not write javascript in python, write python
>> >
>> >
>> >
>> > --
>> > Two percent of zero is almost nothing.
>> >
>> >
>> >
>> >
>> > --
>> > Whoever dies with the most toys wins.





--
Pie are not square. Pie are round. Cornbread are square.
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 15 Sep 2021 18:40:52 +0000, alister wrote:

> On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote:
>
>> There is a further problem with this:
>>
>>> for i,term in enumerate(term.args):
>>> ____mark_term(term.args[i])
>>
>> It should read:
>>
>> for i,help in enumerate(term.args):
>> ____mark_term(help)
>>
>> But then i isn't need.
> even Better (i had only skimmed the code as I was certain I would find
> this, it is probably the No. 1 thing new python programmers get wrong if
> your example is correct the it can be simplified even further to
>
> for help in term.args:
> mark_term(help)
>
> & if help does not get used after this loop then a comprehension is even
> better _ == [mark_term(help) for help in term.args]
>
>
> the underscore character is python convention for an unneeded place-
> holder variable.
>
>
I also notice you are using match case - this was only introduced in
python 10 so it is very new (i had not seen it before)
the break statements are probably not necessary as if ii understand this
feature correctly python does not fall-through & execute every subsequent
case after a successful match.

& finally the netiquette in this forum is to interleave of bottom post
rather than top post, it makes it easier to follow the thread.





--
Don't quit now, we might just as well lock the door and throw away the
key.
--
https://mail.python.org/mailman/listinfo/python-list

And how do you iterate over the first n-1 elements
of a list with n elements? This is what my code does:

i = 0
while i < len(term.args) - 1:
____mark_term(term.args[i])
____i += 1
term = term.args[i]

You can try yourself:

% python3
>>> foo = ["a", "b", "c"]
>>> i = 0
>>> while i < len(foo) - 1:
... print("mark_term", foo[i])
... i += 1
...
mark_term a
mark_term b
>>> foo = foo[i]
>>> foo
'c'

alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2:
> On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote:
>
> > There is a further problem with this:
> >
> >> for i,term in enumerate(term.args):
> >> ____mark_term(term.args[i])
> >
> > It should read:
> >
> > for i,help in enumerate(term.args):
> > ____mark_term(help)
> >
> > But then i isn't need.
> even Better (i had only skimmed the code as I was certain I would find
> this, it is probably the No. 1 thing new python programmers get wrong
> if your example is correct the it can be simplified even further to
>
> for help in term.args:
> mark_term(help)
>
> & if help does not get used after this loop then a comprehension is even
> better
> _ == [mark_term(help) for help in term.args]
>
>
> the underscore character is python convention for an unneeded place-
> holder variable.
> >
> > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50
> > UTC+2:
> >> Do you mean, replace this:
> >> i = 0 while i < len(term.args) - 1:
> >> ____mark_term(term.args[i])
> >> ____i += 1 term = term.args[i]
> >>
> >> By this:
> >>
> >> for i,term in enumerate(term.args):
> >> ____mark_term(term.args[i])
> >>
> >> This wouldn't be correct anymore. The recursive call is only for the
> >> arguments except for the last one one.
> >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
> >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
> >> >
> >> > > I really wonder why my Python implementation is a factor 40 slower
> >> > > than my JavaScript implementation.
> >> > > Structurally its the same code.
> >> > >
> >> > > You can check yourself:
> >> > >
> >> > > Python Version:
> >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
> >> > machine.py
> >> > >
> >> > > JavaScript Version:
> >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
> >> > machine.js
> >> > >
> >> > > Its the same while, if-then-else, etc.. its the same classes
> >> > > Variable, Compound etc.. Maybe I could speed it up by some details.
> >> > > For example to create an array of length n, I use in Python:
> >> > >
> >> > > temp = [NotImplemented] * code[pos]
> >> > > pos += 1
> >> > >
> >> > > Whereas in JavaScript I use, also in exec_build2():
> >> > >
> >> > > temp = new Array(code[pos++]);
> >> > >
> >> > > So I hear Guido doesn't like ++. So in Python I use +=
> >> > > and a separate statement as a workaround. But otherwise,
> >> > > what about the creation of an array,
> >> > >
> >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or
> >> > > does it really first create an array of size 1 and then enlarge it?
> >> > >
> >> > > Julio Di Egidio wrote:
> >> > <sniped due to top posting>
> >> >
> >> > this is probably a string contender
> >> >
> >> > i = 0 while i < len(term.args) - 1:
> >> > mark_term(term.args[i])
> >> > i += 1 term = term.args[i]
> >> >
> >> > try replacing with something more pythonic
> >> >
> >> > for index,term in enumerate(term.args):
> >> > mark_term(term.args[i])
> >> >
> >> >
> >> > & possibly go all the way to changing it into a comprehension
> >> >
> >> > there are other similar anti patterns throughout this code.
> >> >
> >> > any time you are manually keeping a counter as an index into a
> >> > list,tupple other iterable YOU ARE DOING IT WRONG!
> >> >
> >> > Do not write javascript in python, write python
> >> >
> >> >
> >> >
> >> > --
> >> > Two percent of zero is almost nothing.
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Whoever dies with the most toys wins.
> --
> Pie are not square. Pie are round. Cornbread are square.
--
https://mail.python.org/mailman/listinfo/python-list

There is a Python 3.8 compatible version here:

https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine2.py

I have replaced match by if-then-else. So as to
be able to test with GraalVM. GraalVM is still faster
despite using if-then-else.

But GraalVM needs some time to JIT the code.
You need to make some cold runs before you
see kicking it being fast.

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:48:31 UTC+2:
> And how do you iterate over the first n-1 elements
> of a list with n elements? This is what my code does:
> i = 0
> while i < len(term.args) - 1:
> ____mark_term(term.args[i])
> ____i += 1
> term = term.args[i]
> You can try yourself:
>
> % python3
> >>> foo = ["a", "b", "c"]
> >>> i = 0
> >>> while i < len(foo) - 1:
> ... print("mark_term", foo[i])
> ... i += 1
> ...
> mark_term a
> mark_term b
> >>> foo = foo[i]
> >>> foo
> 'c'
> alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2:
> > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote:
> >
> > > There is a further problem with this:
> > >
> > >> for i,term in enumerate(term.args):
> > >> ____mark_term(term.args[i])
> > >
> > > It should read:
> > >
> > > for i,help in enumerate(term.args):
> > > ____mark_term(help)
> > >
> > > But then i isn't need.
> > even Better (i had only skimmed the code as I was certain I would find
> > this, it is probably the No. 1 thing new python programmers get wrong
> > if your example is correct the it can be simplified even further to
> >
> > for help in term.args:
> > mark_term(help)
> >
> > & if help does not get used after this loop then a comprehension is even
> > better
> > _ == [mark_term(help) for help in term.args]
> >
> >
> > the underscore character is python convention for an unneeded place-
> > holder variable.
> > >
> > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50
> > > UTC+2:
> > >> Do you mean, replace this:
> > >> i = 0 while i < len(term.args) - 1:
> > >> ____mark_term(term.args[i])
> > >> ____i += 1 term = term.args[i]
> > >>
> > >> By this:
> > >>
> > >> for i,term in enumerate(term.args):
> > >> ____mark_term(term.args[i])
> > >>
> > >> This wouldn't be correct anymore. The recursive call is only for the
> > >> arguments except for the last one one.
> > >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
> > >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
> > >> >
> > >> > > I really wonder why my Python implementation is a factor 40 slower
> > >> > > than my JavaScript implementation.
> > >> > > Structurally its the same code.
> > >> > >
> > >> > > You can check yourself:
> > >> > >
> > >> > > Python Version:
> > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
> > >> > machine.py
> > >> > >
> > >> > > JavaScript Version:
> > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
> > >> > machine.js
> > >> > >
> > >> > > Its the same while, if-then-else, etc.. its the same classes
> > >> > > Variable, Compound etc.. Maybe I could speed it up by some details.
> > >> > > For example to create an array of length n, I use in Python:
> > >> > >
> > >> > > temp = [NotImplemented] * code[pos]
> > >> > > pos += 1
> > >> > >
> > >> > > Whereas in JavaScript I use, also in exec_build2():
> > >> > >
> > >> > > temp = new Array(code[pos++]);
> > >> > >
> > >> > > So I hear Guido doesn't like ++. So in Python I use +=
> > >> > > and a separate statement as a workaround. But otherwise,
> > >> > > what about the creation of an array,
> > >> > >
> > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or
> > >> > > does it really first create an array of size 1 and then enlarge it?
> > >> > >
> > >> > > Julio Di Egidio wrote:
> > >> > <sniped due to top posting>
> > >> >
> > >> > this is probably a string contender
> > >> >
> > >> > i = 0 while i < len(term.args) - 1:
> > >> > mark_term(term.args[i])
> > >> > i += 1 term = term.args[i]
> > >> >
> > >> > try replacing with something more pythonic
> > >> >
> > >> > for index,term in enumerate(term.args):
> > >> > mark_term(term.args[i])
> > >> >
> > >> >
> > >> > & possibly go all the way to changing it into a comprehension
> > >> >
> > >> > there are other similar anti patterns throughout this code.
> > >> >
> > >> > any time you are manually keeping a counter as an index into a
> > >> > list,tupple other iterable YOU ARE DOING IT WRONG!
> > >> >
> > >> > Do not write javascript in python, write python
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Two percent of zero is almost nothing.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Whoever dies with the most toys wins.
> > --
> > Pie are not square. Pie are round. Cornbread are square.
--
https://mail.python.org/mailman/listinfo/python-list

What could be slow, repeatedly requesting the "args"
field. Maybe I should do:

help = term.args
i = 0
while i < len(help) - 1:
____mark_term(help[i])
____i += 1
term = help[i]

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:48:31 UTC+2:
> And how do you iterate over the first n-1 elements
> of a list with n elements? This is what my code does:
> i = 0
> while i < len(term.args) - 1:
> ____mark_term(term.args[i])
> ____i += 1
> term = term.args[i]
> You can try yourself:
>
> % python3
> >>> foo = ["a", "b", "c"]
> >>> i = 0
> >>> while i < len(foo) - 1:
> ... print("mark_term", foo[i])
> ... i += 1
> ...
> mark_term a
> mark_term b
> >>> foo = foo[i]
> >>> foo
> 'c'
> alister schrieb am Mittwoch, 15. September 2021 um 20:41:12 UTC+2:
> > On Wed, 15 Sep 2021 11:31:48 -0700, Mostowski Collapse wrote:
> >
> > > There is a further problem with this:
> > >
> > >> for i,term in enumerate(term.args):
> > >> ____mark_term(term.args[i])
> > >
> > > It should read:
> > >
> > > for i,help in enumerate(term.args):
> > > ____mark_term(help)
> > >
> > > But then i isn't need.
> > even Better (i had only skimmed the code as I was certain I would find
> > this, it is probably the No. 1 thing new python programmers get wrong
> > if your example is correct the it can be simplified even further to
> >
> > for help in term.args:
> > mark_term(help)
> >
> > & if help does not get used after this loop then a comprehension is even
> > better
> > _ == [mark_term(help) for help in term.args]
> >
> >
> > the underscore character is python convention for an unneeded place-
> > holder variable.
> > >
> > > Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 20:22:50
> > > UTC+2:
> > >> Do you mean, replace this:
> > >> i = 0 while i < len(term.args) - 1:
> > >> ____mark_term(term.args[i])
> > >> ____i += 1 term = term.args[i]
> > >>
> > >> By this:
> > >>
> > >> for i,term in enumerate(term.args):
> > >> ____mark_term(term.args[i])
> > >>
> > >> This wouldn't be correct anymore. The recursive call is only for the
> > >> arguments except for the last one one.
> > >> alister schrieb am Mittwoch, 15. September 2021 um 20:17:23 UTC+2:
> > >> > On Wed, 15 Sep 2021 18:23:10 +0200, Mostowski Collapse wrote:
> > >> >
> > >> > > I really wonder why my Python implementation is a factor 40 slower
> > >> > > than my JavaScript implementation.
> > >> > > Structurally its the same code.
> > >> > >
> > >> > > You can check yourself:
> > >> > >
> > >> > > Python Version:
> > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/
> > >> > machine.py
> > >> > >
> > >> > > JavaScript Version:
> > >> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/
> > >> > machine.js
> > >> > >
> > >> > > Its the same while, if-then-else, etc.. its the same classes
> > >> > > Variable, Compound etc.. Maybe I could speed it up by some details.
> > >> > > For example to create an array of length n, I use in Python:
> > >> > >
> > >> > > temp = [NotImplemented] * code[pos]
> > >> > > pos += 1
> > >> > >
> > >> > > Whereas in JavaScript I use, also in exec_build2():
> > >> > >
> > >> > > temp = new Array(code[pos++]);
> > >> > >
> > >> > > So I hear Guido doesn't like ++. So in Python I use +=
> > >> > > and a separate statement as a workaround. But otherwise,
> > >> > > what about the creation of an array,
> > >> > >
> > >> > > is the the idiom [_] * _ slow? I am assuming its compiled away. Or
> > >> > > does it really first create an array of size 1 and then enlarge it?
> > >> > >
> > >> > > Julio Di Egidio wrote:
> > >> > <sniped due to top posting>
> > >> >
> > >> > this is probably a string contender
> > >> >
> > >> > i = 0 while i < len(term.args) - 1:
> > >> > mark_term(term.args[i])
> > >> > i += 1 term = term.args[i]
> > >> >
> > >> > try replacing with something more pythonic
> > >> >
> > >> > for index,term in enumerate(term.args):
> > >> > mark_term(term.args[i])
> > >> >
> > >> >
> > >> > & possibly go all the way to changing it into a comprehension
> > >> >
> > >> > there are other similar anti patterns throughout this code.
> > >> >
> > >> > any time you are manually keeping a counter as an index into a
> > >> > list,tupple other iterable YOU ARE DOING IT WRONG!
> > >> >
> > >> > Do not write javascript in python, write python
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Two percent of zero is almost nothing.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Whoever dies with the most toys wins.
> > --
> > Pie are not square. Pie are round. Cornbread are square.
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 15 Sep 2021 11:48:18 -0700, Mostowski Collapse wrote:

> And how do you iterate over the first n-1 elements of a list with n
> elements? This is what my code does:
>
> i = 0 while i < len(term.args) - 1:
> ____mark_term(term.args[i])
> ____i += 1 term = term.args[i]
>
> You can try yourself:

use a slice
as a generic example you can try in the interactive interpreter (aka
REPL):

>>>items = [1,2,3,4,5,6,7,8,9,0]
>>>for item in items[:-1]:
>>> print(item)

1
2
3
4
5
6
7
8
9

I should state for the record that I am no Python expert, I mainly visit
this news group to learn.
sometimes from reading solutions & somtimes by trying to work out what
the solution might be.

most productively by working out a solution & seeing if other posters
agree (& working out why they don't)



--
The universe seems neither benign nor hostile, merely indifferent.
-- Sagan
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Sep 16, 2021 at 5:15 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>
> If you find a "wonky" spot, I can replace it by "non-wonky"
> code. I noticed some differences between Python Dicts
> and JavaScript objects. Python tends to throw more exceptions.
>
> So in Python I now do the following:
>
> peek = kb.get(functor, NotImplemented)
> if peek is not NotImplemented:
>
> In JavaScript I can directly do:
>
> peek = kb[functor];
> if (peek !== undefined)
>
> But if get() in Python is implemented under the hood with
> exception handling. i.e. using the exception prone [] and
> then in case an exception is thrown, returning the
>
> default value, then Python get() will probably be quite slow.
> Since usually exceptions are slow.
>

No, you're thinking in terms of microoptimizations. Exception handling
isn't THAT slow. I'm talking more about how everything's getting
packaged up and then unpackaged (the repeated use of the "Compound"
class looks highly suboptimal), rather than reworking your algorithm
to behave more cleanly.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 15 Sep 2021 11:56:47 -0700, Mostowski Collapse wrote:

> What could be slow, repeatedly requesting the "args"
> field. Maybe I should do:
>
> help = term.args i = 0 while i < len(help) - 1:
> ____mark_term(help[i])
> ____i += 1 term = help[i]
>
No this construct is a common error in new python programmers
the next progression they make is when they discover the range function
items = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
for x in range(len(list)):
print (items[x])

but that is equally inefficient

the pythonic way is as previously suggested

for item in items:
print(item)

then the management of the index is being handled by the python
interpreter internally & is effectively machine code.
every time you increment your own pointer the interpreter has to process
reading the next line, reading the variable , incrementing it & then
using it. this is what makes your current code slow.


if you ever find your self creating a variable purely to use as a pointer
into a list then you are almost certainly taking the wrong approach.

more usefull links
https://www.youtube.com/watch?v=zdJEYhA2AZQ






--
"Show me a good loser, and I'll show you a loser."
-- Vince Lombardi, football coach
--
https://mail.python.org/mailman/listinfo/python-list

And how do you only iterate over n-1 elements?
I don't need a loop over all elements.

With array slicing?

Someting like:

for item in items[0:len(items)-2]:
___print(item)

Or with negative slicing indexes? Problem
is my length can be equal to one.

And when I have length equal to one, the
slice might not do the right thing?

LoL

alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2:
> for item in items:
> print(item)

--
https://mail.python.org/mailman/listinfo/python-list

Ok you suggested:

>>>items = [1,2,3,4,5,6,7,8,9,0]
>>>for item in items[:-1]:
>>> print(item)

1
2
3
4
5
6
7
8
9

Does this also work for length = 1? Ok let me try:

>>> foo = ["a","b","c"]
>>> for x in foo[:-1]:
... print(x)
...
a
b
>>> foo = ["a"]
>>> for x in foo[:-1]:
... print(x)
...

Oki Doki

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:10:50 UTC+2:
> And how do you only iterate over n-1 elements?
> I don't need a loop over all elements.
>
> With array slicing?
>
> Someting like:
>
> for item in items[0:len(items)-2]:
> ___print(item)
>
> Or with negative slicing indexes? Problem
> is my length can be equal to one.
>
> And when I have length equal to one, the
> slice might not do the right thing?
>
> LoL
> alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2:
> > for item in items:
> > print(item)
--
https://mail.python.org/mailman/listinfo/python-list

On 9/15/2021 12:23 PM, Mostowski Collapse wrote:
> I really wonder why my Python implementation
> is a factor 40 slower than my JavaScript implementation.
> Structurally its the same code.
>
> You can check yourself:
>
> Python Version:
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py
>
> JavaScript Version:
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js
>
> Its the same while, if-then-else, etc.. its the same
> classes Variable, Compound etc.. Maybe I could speed
> it up by some details. For example to create an array
> of length n, I use in Python:
>
>   temp = [NotImplemented] * code[pos]
>   pos += 1
>
> Whereas in JavaScript I use, also
> in exec_build2():
>
>   temp = new Array(code[pos++]);
>
> So I hear Guido doesn't like ++. So in Python I use +=
> and a separate statement as a workaround. But otherwise,
> what about the creation of an array,
>
> is the the idiom [_] * _ slow? I am assuming its
> compiled away. Or does it really first create an
> array of size 1 and then enlarge it?



I'm sure you know you can put in timing statements to find bottlenecks.

import time
startTime = time.perf_counter()
[code block]
print("%.2f" % (time.perf_counter() - startTime))



--
https://mail.python.org/mailman/listinfo/python-list

BTW: I could already make it faster, by not repeatedly
accessing .arg anymore. It went down from ca.:

171'000 ms

To this here:

140'000 ms

But only in the cold run. In the warm run it went back
to 171'000 ms. Possibly when my code is faster,
it will create objects more faster, and kill the Python GC.

Or it was because my Laptop went into screen black?
And throttled the CPU. Not sure.

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:13:12 UTC+2:
> Ok you suggested:
> >>>items = [1,2,3,4,5,6,7,8,9,0]
> >>>for item in items[:-1]:
> >>> print(item)
>
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
> Does this also work for length = 1? Ok let me try:
> >>> foo = ["a","b","c"]
> >>> for x in foo[:-1]:
> ... print(x)
> ...
> a
> b
> >>> foo = ["a"]
> >>> for x in foo[:-1]:
> ... print(x)
> ...
>
> Oki Doki
> Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:10:50 UTC+2:
> > And how do you only iterate over n-1 elements?
> > I don't need a loop over all elements.
> >
> > With array slicing?
> >
> > Someting like:
> >
> > for item in items[0:len(items)-2]:
> > ___print(item)
> >
> > Or with negative slicing indexes? Problem
> > is my length can be equal to one.
> >
> > And when I have length equal to one, the
> > slice might not do the right thing?
> >
> > LoL
> > alister schrieb am Mittwoch, 15. September 2021 um 22:00:30 UTC+2:
> > > for item in items:
> > > print(item)
--
https://mail.python.org/mailman/listinfo/python-list

On 9/15/2021 5:10 PM, Mostowski Collapse wrote:
> And how do you only iterate over n-1 elements?
> I don't need a loop over all elements.
>
> With array slicing?
>
> Someting like:
>
> for item in items[0:len(items)-2]:
> ___print(item)
>
> Or with negative slicing indexes? Problem
> is my length can be equal to one.
>
> And when I have length equal to one, the
> slice might not do the right thing?
>
> LoL


From the python command prompt:

items = [1,2,3,4]

for itm in items:
print(itm)
1
2
3
4

for itm in items[:-2]:
print(itm)
1
2


for itm in items[:-3]:
print(itm)
1


for itm in items[:-4]:
print(itm)
(no result, no error thrown)


for itm in items[:-5]:
print(itm)
(no result, no error thrown)
--
https://mail.python.org/mailman/listinfo/python-list

Thank you for the suggestion. The test harness
is invoked as follows. So it does already do time/1,
thats also how I did the comparison Standard Python

and GraalVM Python, a file dogelog.py:

import sys
# sys.path.append("<path>\jekrun_bench\core\harness2\libpy")
sys.path.append("/mnt/c/<path>/jekrun_bench/core/harness2/libpy")
from index import init, consult

init()
consult(":- ['suite2.p']. "
":- time(suite). "
":- nl. "
":- time(suite). ")

Here you see a GraalVM cold and warm run.The warm run is faster.
If you do a warm warm run, it even gets more faster, because of
JIT-ing, Just-in-Time machine compilation,

via the GraalVM Truffles framework:

$ export PATH=<path>/graalvm-ce-java8-21.2.0/bin:$PATH
$ cd /mnt/c/<path>/jekrun_bench/core/harness2
$ graalpython /mnt/c/<path>/jekrun_bench/core/harness2/dogelog.py
nrev % Wall 6175 ms, gc 212 ms, 154473 lips
crypt % Wall 9327 ms, gc 63 ms, 112838 lips
deriv % Wall 4101 ms, gc 90 ms, 321890 lips
poly % Wall 3594 ms, gc 415 ms, 216299 lips
sortq % Wall 3427 ms, gc 67 ms, 290070 lips
tictac % Wall 2770 ms, gc 51 ms, 136580 lips
queens % Wall 3287 ms, gc 64 ms, 325617 lips
query % Wall 1432 ms, gc 77 ms, 382969 lips
mtak % Wall 2532 ms, gc 95 ms, 533881 lips
perfect % Wall 3980 ms, gc 76 ms, 290382 lips
% Wall 40745 ms, gc 1212 ms, 235751 lips

nrev % Wall 4508 ms, gc 112 ms, 211595 lips
crypt % Wall 6063 ms, gc 61 ms, 173584 lips
deriv % Wall 3150 ms, gc 42 ms, 419070 lips
poly % Wall 3549 ms, gc 432 ms, 219042 lips
sortq % Wall 3196 ms, gc 63 ms, 311036 lips
tictac % Wall 2670 ms, gc 52 ms, 141695 lips
queens % Wall 3087 ms, gc 60 ms, 346713 lips
query % Wall 1434 ms, gc 25 ms, 382435 lips
mtak % Wall 2596 ms, gc 90 ms, 520719 lips
perfect % Wall 3521 ms, gc 43 ms, 328236 lips
% Wall 33810 ms, gc 980 ms, 284108 lips

DFS schrieb am Mittwoch, 15. September 2021 um 23:15:07 UTC+2:
> On 9/15/2021 12:23 PM, Mostowski Collapse wrote:
> > I really wonder why my Python implementation
> > is a factor 40 slower than my JavaScript implementation.
> > Structurally its the same code.
> >
> > You can check yourself:
> >
> > Python Version:
> > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py
> >
> > JavaScript Version:
> > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js
> >
> > Its the same while, if-then-else, etc.. its the same
> > classes Variable, Compound etc.. Maybe I could speed
> > it up by some details. For example to create an array
> > of length n, I use in Python:
> >
> > temp = [NotImplemented] * code[pos]
> > pos += 1
> >
> > Whereas in JavaScript I use, also
> > in exec_build2():
> >
> > temp = new Array(code[pos++]);
> >
> > So I hear Guido doesn't like ++. So in Python I use +=
> > and a separate statement as a workaround. But otherwise,
> > what about the creation of an array,
> >
> > is the the idiom [_] * _ slow? I am assuming its
> > compiled away. Or does it really first create an
> > array of size 1 and then enlarge it?
> I'm sure you know you can put in timing statements to find bottlenecks.
>
> import time
> startTime = time.perf_counter()
> [code block]
> print("%.2f" % (time.perf_counter() - startTime))
--
https://mail.python.org/mailman/listinfo/python-list

But the end-result is still very weak:
% Wall 33810 ms, gc 980 ms, 284108 lips

This is below 1 million LIPS.
The JavaScript version of Dogelog does currently around 2 million LIPS.
And SWI-Prolog can do around 20 million LIPS.

Mostowski Collapse schrieb am Mittwoch, 15. September 2021 um 23:29:48 UTC+2:
> Thank you for the suggestion. The test harness
> is invoked as follows. So it does already do time/1,
> thats also how I did the comparison Standard Python
>
> and GraalVM Python, a file dogelog.py:
>
> import sys
> # sys.path.append("<path>\jekrun_bench\core\harness2\libpy")
> sys.path.append("/mnt/c/<path>/jekrun_bench/core/harness2/libpy")
> from index import init, consult
>
> init()
> consult(":- ['suite2.p']. "
> ":- time(suite). "
> ":- nl. "
> ":- time(suite). ")
>
> Here you see a GraalVM cold and warm run.The warm run is faster.
> If you do a warm warm run, it even gets more faster, because of
> JIT-ing, Just-in-Time machine compilation,
>
> via the GraalVM Truffles framework:
>
> $ export PATH=<path>/graalvm-ce-java8-21.2.0/bin:$PATH
> $ cd /mnt/c/<path>/jekrun_bench/core/harness2
> $ graalpython /mnt/c/<path>/jekrun_bench/core/harness2/dogelog.py
> nrev % Wall 6175 ms, gc 212 ms, 154473 lips
> crypt % Wall 9327 ms, gc 63 ms, 112838 lips
> deriv % Wall 4101 ms, gc 90 ms, 321890 lips
> poly % Wall 3594 ms, gc 415 ms, 216299 lips
> sortq % Wall 3427 ms, gc 67 ms, 290070 lips
> tictac % Wall 2770 ms, gc 51 ms, 136580 lips
> queens % Wall 3287 ms, gc 64 ms, 325617 lips
> query % Wall 1432 ms, gc 77 ms, 382969 lips
> mtak % Wall 2532 ms, gc 95 ms, 533881 lips
> perfect % Wall 3980 ms, gc 76 ms, 290382 lips
> % Wall 40745 ms, gc 1212 ms, 235751 lips
>
> nrev % Wall 4508 ms, gc 112 ms, 211595 lips
> crypt % Wall 6063 ms, gc 61 ms, 173584 lips
> deriv % Wall 3150 ms, gc 42 ms, 419070 lips
> poly % Wall 3549 ms, gc 432 ms, 219042 lips
> sortq % Wall 3196 ms, gc 63 ms, 311036 lips
> tictac % Wall 2670 ms, gc 52 ms, 141695 lips
> queens % Wall 3087 ms, gc 60 ms, 346713 lips
> query % Wall 1434 ms, gc 25 ms, 382435 lips
> mtak % Wall 2596 ms, gc 90 ms, 520719 lips
> perfect % Wall 3521 ms, gc 43 ms, 328236 lips
> % Wall 33810 ms, gc 980 ms, 284108 lips
> DFS schrieb am Mittwoch, 15. September 2021 um 23:15:07 UTC+2:
> > On 9/15/2021 12:23 PM, Mostowski Collapse wrote:
> > > I really wonder why my Python implementation
> > > is a factor 40 slower than my JavaScript implementation.
> > > Structurally its the same code.
> > >
> > > You can check yourself:
> > >
> > > Python Version:
> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/machine.py
> > >
> > > JavaScript Version:
> > > https://github.com/jburse/dogelog-moon/blob/main/devel/runtime/machine.js
> > >
> > > Its the same while, if-then-else, etc.. its the same
> > > classes Variable, Compound etc.. Maybe I could speed
> > > it up by some details. For example to create an array
> > > of length n, I use in Python:
> > >
> > > temp = [NotImplemented] * code[pos]
> > > pos += 1
> > >
> > > Whereas in JavaScript I use, also
> > > in exec_build2():
> > >
> > > temp = new Array(code[pos++]);
> > >
> > > So I hear Guido doesn't like ++. So in Python I use +=
> > > and a separate statement as a workaround. But otherwise,
> > > what about the creation of an array,
> > >
> > > is the the idiom [_] * _ slow? I am assuming its
> > > compiled away. Or does it really first create an
> > > array of size 1 and then enlarge it?
> > I'm sure you know you can put in timing statements to find bottlenecks.
> >
> > import time
> > startTime = time.perf_counter()
> > [code block]
> > print("%.2f" % (time.perf_counter() - startTime))
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Sep 16, 2021 at 7:59 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>
> BTW: I could already make it faster, by not repeatedly
> accessing .arg anymore. It went down from ca.:
>
> 171'000 ms
>
> To this here:
>
> 140'000 ms
>
> But only in the cold run. In the warm run it went back
> to 171'000 ms. Possibly when my code is faster,
> it will create objects more faster, and kill the Python GC.
>
> Or it was because my Laptop went into screen black?
> And throttled the CPU. Not sure.
>

Instead of worrying about all these details, start by simplifying your
code. Focus on clean, simple, readable code, and don't microoptimize.
Specifically, focus on the core arithmetic that you're trying to do,
and get rid of all the bookkeeping overhead; most of that is a waste
of time. I mentioned earlier the repeated boxing and unboxing in
"Compound" objects - have you changed anything with those?

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Compound is not used for boxing. Integers and floats
are represented directly. Also integers are not mapped to
floats. But maybe compound could be a little flattened,

like using directly an array. But then you cannot assure
anymore "clean, simple, readable code". For example now
I have clean, simple and readable code, since I can

access the functor of a compound via:

obj.functor

but when I flatten Compound into arrays, it would become:

obj[0]

Should I declare a constant FUNCTOR = 0? Some of your
requirements have a trade-off, not all of them can be
sustained simultaneously so easy.

I am rather expecting languages like Python and JavaScript
to offer the same comfort as C or Java, except that
I don't need to write types for all the varianbles and fields

all the time. But this requires smart language compiler
and language runtime. V8 Chrome has interesting articles
how they optimize access like .functor.

Chris Angelico schrieb am Donnerstag, 16. September 2021 um 00:02:54 UTC+2:
> On Thu, Sep 16, 2021 at 7:59 AM Mostowski Collapse <burs...@gmail.com> wrote:
> >
> > BTW: I could already make it faster, by not repeatedly
> > accessing .arg anymore. It went down from ca.:
> >
> > 171'000 ms
> >
> > To this here:
> >
> > 140'000 ms
> >
> > But only in the cold run. In the warm run it went back
> > to 171'000 ms. Possibly when my code is faster,
> > it will create objects more faster, and kill the Python GC.
> >
> > Or it was because my Laptop went into screen black?
> > And throttled the CPU. Not sure.
> >
> Instead of worrying about all these details, start by simplifying your
> code. Focus on clean, simple, readable code, and don't microoptimize.
> Specifically, focus on the core arithmetic that you're trying to do,
> and get rid of all the bookkeeping overhead; most of that is a waste
> of time. I mentioned earlier the repeated boxing and unboxing in
> "Compound" objects - have you changed anything with those?
>
> ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Fri, Sep 17, 2021 at 3:20 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>
> Compound is not used for boxing. Integers and floats
> are represented directly. Also integers are not mapped to
> floats. But maybe compound could be a little flattened,
>

"Boxing" in this case isn't about ints and floats, since Java-like
bizarrenesses simply don't happen in Python; I'm talking about the way
that you frequently build up a Compound object for various situations
(even for throwing an error - you have a function that constructs a
generic Exception, and then buries a Compound inside it), and then
you're frequently checking if something is an instance of Compound.
All these constant back-and-forths are extremely expensive, since
they're not part of your algorithm at all.

At very least, use tuples instead of Compounds, but it would be far
better to ask less questions about your data and do more things by
tidying up your algorithm. Unfortunately, I can't really advise with
any detail, because you have code like this:

###
# Mark a term.
#
# @param term The term.
##
def mark_term(term):

What does that even mean?! I get it, you have a term, and you're
marking it. Whatever that mark means. The comments add absolutely
nothing that the function header didn't tell me. Are you implementing
your own garbage collection on top of Python's? Or something else?
It's extremely hard to give any sort of recommendations when your code
is hard to read, and nearly all of the comments are nothing more than
restating what can be seen in the next line of code. Also, with the
number of globals you're using, tracing the purpose of your functions
is not easy.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

About Exceptions: Thats just building ISO core
standard Prolog error terms.

About Garbage Collection: Thats just Prolog
garbage collection, which does shrink some
single linked lists, which ordinary
programmig language GC cannot do,

or maybe some Weak Pointer magic can do it?
The use case is very simple. A Prolog system
has a so called trail. The trail in Dogelog
Runtime is a single linked list:

-->[ A ]-->[ B ]-->[ C ]-->

Now if B becomes unused, you need to rewire
the trail, it should then look like:

-->[ A ]---------->[ C ]-->


If a programming language has a means to
communicate this to the Garbage Collector,
I happy to apply it. The challenge is many
fold, the pointer from A to B for example

needs not to be accounted to determine
whether B is reachable. So all the links
in the trail are weak pointers. But they
are weak pointers that need to

be able to adapt.

Chris Angelico wrote:
> On Fri, Sep 17, 2021 at 3:20 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>>
>> Compound is not used for boxing. Integers and floats
>> are represented directly. Also integers are not mapped to
>> floats. But maybe compound could be a little flattened,
>>
>
> "Boxing" in this case isn't about ints and floats, since Java-like
> bizarrenesses simply don't happen in Python; I'm talking about the way
> that you frequently build up a Compound object for various situations
> (even for throwing an error - you have a function that constructs a
> generic Exception, and then buries a Compound inside it), and then
> you're frequently checking if something is an instance of Compound.
> All these constant back-and-forths are extremely expensive, since
> they're not part of your algorithm at all.
>
> At very least, use tuples instead of Compounds, but it would be far
> better to ask less questions about your data and do more things by
> tidying up your algorithm. Unfortunately, I can't really advise with
> any detail, because you have code like this:
>
> ###
> # Mark a term.
> #
> # @param term The term.
> ##
> def mark_term(term):
>
> What does that even mean?! I get it, you have a term, and you're
> marking it. Whatever that mark means. The comments add absolutely
> nothing that the function header didn't tell me. Are you implementing
> your own garbage collection on top of Python's? Or something else?
> It's extremely hard to give any sort of recommendations when your code
> is hard to read, and nearly all of the comments are nothing more than
> restating what can be seen in the next line of code. Also, with the
> number of globals you're using, tracing the purpose of your functions
> is not easy.
>
> ChrisA
>

--
https://mail.python.org/mailman/listinfo/python-list

Here is a challenge for Python.
Can Python solve Sudoku?

Mostowski Collapse wrote:
> I am not testing this use-case. But a related
> use-case might highlight why speed did never
> hurt anybody.
>
> Lets say you program a flying drone with Python,
> and the measurement is from the drone sensor
> and communication systems.
>
> Lets say you are using the idle time between
> measurements for some complex planning. It
> is then not true that you have anyway
>
> to wait for the measurement.
>
> Hope this helps!
>
> BTW: If somebody knows another Python implementation
> I am happy to test this implementation as well.
> I am assuming that the standard Python python.exe
>
> I tested amounts to CPython? Not sure. And the
> GraalVM is practically the same as JPython? Not
> sure either.
>
>> Opinion:   Anyone who is counting on Python for truly fast compute
>> speed is probably using Python for the wrong purpose. Here, we use
>> Python to control Test Equipment, to set up the equipment and ask for
>> a measurement, get it, and proceed to the next measurement; and at the
>> end produce a nice formatted report. If we wrote the test script in C
>> or Rust or whatever it could not run substantially faster because it
>> is communicating with the test equipment, setting it up and waiting
>> for responses, and that is where the vast majority of the time goes.
>> Especially if the measurement result requires averaging it can take a
>> while.  In my opinion this is an ideal use for Python, not just
>> because the speed of Python is not important, but also because we can
>> easily find people who know Python, who like coding in Python, and
>> will join the company to program in Python ... and stay with us.
>> --- Joseph S.
>

--
https://mail.python.org/mailman/listinfo/python-list

A friend just sent me a Web Sudoku made with Dogelog Runtime
https://gist.github.com/jburse/c85297e97091caf22d306dd8c8be12fe#gistcomment-3895696

LoL

Mostowski Collapse schrieb am Donnerstag, 16. September 2021 um 21:59:05 UTC+2:
> Here is a challenge for Python.
> Can Python solve Sudoku?
>
> Mostowski Collapse wrote:
> > I am not testing this use-case. But a related
> > use-case might highlight why speed did never
> > hurt anybody.
> >
> > Lets say you program a flying drone with Python,
> > and the measurement is from the drone sensor
> > and communication systems.
> >
> > Lets say you are using the idle time between
> > measurements for some complex planning. It
> > is then not true that you have anyway
> >
> > to wait for the measurement.
> >
> > Hope this helps!
> >
> > BTW: If somebody knows another Python implementation
> > I am happy to test this implementation as well.
> > I am assuming that the standard Python python.exe
> >
> > I tested amounts to CPython? Not sure. And the
> > GraalVM is practically the same as JPython? Not
> > sure either.
> >
> >> Opinion: Anyone who is counting on Python for truly fast compute
> >> speed is probably using Python for the wrong purpose. Here, we use
> >> Python to control Test Equipment, to set up the equipment and ask for
> >> a measurement, get it, and proceed to the next measurement; and at the
> >> end produce a nice formatted report. If we wrote the test script in C
> >> or Rust or whatever it could not run substantially faster because it
> >> is communicating with the test equipment, setting it up and waiting
> >> for responses, and that is where the vast majority of the time goes.
> >> Especially if the measurement result requires averaging it can take a
> >> while. In my opinion this is an ideal use for Python, not just
> >> because the speed of Python is not important, but also because we can
> >> easily find people who know Python, who like coding in Python, and
> >> will join the company to program in Python ... and stay with us.
> >> --- Joseph S.
> >
--
https://mail.python.org/mailman/listinfo/python-list

Some questions make no sense to me.

Can a kind of snake solve Sudoku? Do you mean a specific puzzle, or any puzzle or even a puzzle with no solution?

Can a programming language do it? Well, in my experience, programming languages are tools to be used by humans, or sometimes by other programming languages. They are not sentient and cannot be asked to solve much of anything.

So is the question whether someone can program using only Python to solve an arbitrary sudoku problem? Short answer is you can do that in just about ANY language. I mean by brute force, if you have a 9 by 9 matrix with some of the 81 locations already filled in, then you can try every darn combination of the other spots using digits 1 to 9 and then ignore any where the rows and columns and the 9 3x3 submatrices do not follow the rules. At least one solution is guaranteed to pop out if there is one. Sure, such methods may run out of memory or take a while, but many can use little memory and some can speed things up by not going down blind alleys such as placing a number in a position where there already is the same number on the same row or column or sub-matrix.

So is the real question whether a human has already made a decent implementation in Python available? Sure, do a little searching and there are plenty of such things including some that use interesting features of python and some that are just translations from a more primitive language.



-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net@python.org> On Behalf Of Mostowski Collapse
Sent: Thursday, September 16, 2021 3:59 PM
To: python-list@python.org
Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021)

Here is a challenge for Python.
Can Python solve Sudoku?

Mostowski Collapse wrote:
> I am not testing this use-case. But a related use-case might highlight
> why speed did never hurt anybody.
>
> Lets say you program a flying drone with Python, and the measurement
> is from the drone sensor and communication systems.
>
> Lets say you are using the idle time between measurements for some
> complex planning. It is then not true that you have anyway
>
> to wait for the measurement.
>
> Hope this helps!
>
> BTW: If somebody knows another Python implementation I am happy to
> test this implementation as well.
> I am assuming that the standard Python python.exe
>
> I tested amounts to CPython? Not sure. And the GraalVM is practically
> the same as JPython? Not sure either.
>
>> Opinion: Anyone who is counting on Python for truly fast compute
>> speed is probably using Python for the wrong purpose. Here, we use
>> Python to control Test Equipment, to set up the equipment and ask for
>> a measurement, get it, and proceed to the next measurement; and at
>> the end produce a nice formatted report. If we wrote the test script
>> in C or Rust or whatever it could not run substantially faster
>> because it is communicating with the test equipment, setting it up
>> and waiting for responses, and that is where the vast majority of the time goes.
>> Especially if the measurement result requires averaging it can take a
>> while. In my opinion this is an ideal use for Python, not just
>> because the speed of Python is not important, but also because we can
>> easily find people who know Python, who like coding in Python, and
>> will join the company to program in Python ... and stay with us.
>> --- Joseph S.
>

--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

The new release 0.9.6 is quite speedy:

"Maailman vaikein"
850002400720000009004000000000107002305000900040000000000080070017000000000036040
time(solve(Puzzle))
% Wall 41354 ms, gc 520 ms, 3143029 lips
in Browser

See also:

Preview: New para/1 instruction for Dogelog runtime. (Jekejeke)
https://twitter.com/dogelogch/status/1438586282502983682

Preview: New para/1 instruction for Dogelog runtime. (Jekejeke)
https://www.facebook.com/groups/dogelog

Avi Gross schrieb am Donnerstag, 16. September 2021 um 23:43:10 UTC+2:
> Some questions make no sense to me.
>
> Can a kind of snake solve Sudoku? Do you mean a specific puzzle, or any puzzle or even a puzzle with no solution?
>
> Can a programming language do it? Well, in my experience, programming languages are tools to be used by humans, or sometimes by other programming languages. They are not sentient and cannot be asked to solve much of anything.
>
> So is the question whether someone can program using only Python to solve an arbitrary sudoku problem? Short answer is you can do that in just about ANY language. I mean by brute force, if you have a 9 by 9 matrix with some of the 81 locations already filled in, then you can try every darn combination of the other spots using digits 1 to 9 and then ignore any where the rows and columns and the 9 3x3 submatrices do not follow the rules. At least one solution is guaranteed to pop out if there is one. Sure, such methods may run out of memory or take a while, but many can use little memory and some can speed things up by not going down blind alleys such as placing a number in a position where there already is the same number on the same row or column or sub-matrix.
>
> So is the real question whether a human has already made a decent implementation in Python available? Sure, do a little searching and there are plenty of such things including some that use interesting features of python and some that are just translations from a more primitive language.
> -----Original Message-----
> From: Python-list <python-list-bounces+avigross=veriz...@python.org> On Behalf Of Mostowski Collapse
> Sent: Thursday, September 16, 2021 3:59 PM
> To: pytho...@python.org
> Subject: Re: ANN: Dogelog Runtime, Prolog to the Moon (2021)
>
> Here is a challenge for Python.
> Can Python solve Sudoku?
>
> Mostowski Collapse wrote:
> > I am not testing this use-case. But a related use-case might highlight
> > why speed did never hurt anybody.
> >
> > Lets say you program a flying drone with Python, and the measurement
> > is from the drone sensor and communication systems.
> >
> > Lets say you are using the idle time between measurements for some
> > complex planning. It is then not true that you have anyway
> >
> > to wait for the measurement.
> >
> > Hope this helps!
> >
> > BTW: If somebody knows another Python implementation I am happy to
> > test this implementation as well.
> > I am assuming that the standard Python python.exe
> >
> > I tested amounts to CPython? Not sure. And the GraalVM is practically
> > the same as JPython? Not sure either.
> >
> >> Opinion: Anyone who is counting on Python for truly fast compute
> >> speed is probably using Python for the wrong purpose. Here, we use
> >> Python to control Test Equipment, to set up the equipment and ask for
> >> a measurement, get it, and proceed to the next measurement; and at
> >> the end produce a nice formatted report. If we wrote the test script
> >> in C or Rust or whatever it could not run substantially faster
> >> because it is communicating with the test equipment, setting it up
> >> and waiting for responses, and that is where the vast majority of the time goes.
> >> Especially if the measurement result requires averaging it can take a
> >> while. In my opinion this is an ideal use for Python, not just
> >> because the speed of Python is not important, but also because we can
> >> easily find people who know Python, who like coding in Python, and
> >> will join the company to program in Python ... and stay with us.
> >> --- Joseph S.
> >
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list

On 16/09/21 4:23 am, Mostowski Collapse wrote:
> I really wonder why my Python implementation
> is a factor 40 slower than my JavaScript implementation.

There are Javascript implementations around nowadays that are
blazingly fast. Partly that's because a lot of effort has been
put into them, but it's also because Javascript is a different
language. There are many dynamic aspects to Python that make
fast implementations difficult.

> I use in Python:
>
>   temp = [NotImplemented] * code[pos]
>   pos += 1
>
> is the the idiom [_] * _ slow?

No, on the contrary it's probably the fastest way to do it
in Python. You could improve it a bit by precomputing
[NotImplemented]:

# once at the module level
NotImplementedList = [NotImplemented]

# whenever you want a new list
temp = NotImplementedList * code[pos]

That's probably at least as fast as built-in function for
creating lists would be.

> does it really first create an
> array of size 1 and then enlarge it?

It does:

>>> def f(code, pos):
... return [NotImplemented] * code[pos]
...
>>> from dis import dis
>>> dis(f)
2 0 LOAD_GLOBAL 0 (NotImplemented)
2 BUILD_LIST 1
4 LOAD_FAST 0 (code)
6 LOAD_FAST 1 (pos)
8 BINARY_SUBSCR
10 BINARY_MULTIPLY
12 RETURN_VALUE

BTW, the Python terminology is "list", not "array".
(There *is* something in the stdlib called an array, but
it's rarely used or needed.)

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>
> About Exceptions: Thats just building ISO core
> standard Prolog error terms.
>
> About Garbage Collection: Thats just Prolog
> garbage collection, which does shrink some
> single linked lists, which ordinary
> programmig language GC cannot do,
>

Okay, so.... you're building your own garbage collection on top of
Python's, and you're wondering why it's slow?

Change your code to not try to implement one language inside another,
and you'll see a massive performance improvement.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 16/09/21 2:56 pm, Mostowski Collapse wrote:
> I can access the functor of a compound via:
>
> obj.functor
>
> but when I flatten Compound into arrays, it would become:
>
> obj[0]
>
> Should I declare a constant FUNCTOR = 0?

You could, but keep in mind that access to a global in Python
is somewhat expensive, since it requires a dictionary lookup.

There's always a tradeoff between clarity and efficiency.
In this case, I think obj[0] is clear enough -- the reader
will be well aware that the first item of a term is the
functor. Especially if you use a name that makes it clear
what kind of object it is, rather than a generic name such
as "obj".

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

On 16/09/21 6:56 am, Mostowski Collapse wrote:
> What could be slow, repeatedly requesting the "args"
> field. Maybe I should do:
>
> help = term.args
> i = 0
> while i < len(help) - 1:
> ____mark_term(help[i])
> ____i += 1
> term = help[i]

Yes, that will certainly help.

But you're still evaluating len(help) - 1 every time around
the loop, so this is even better:

help = term.args
n = len(help) - 1
i = 0
while i < n:
mark_term(help[i])
i += 1
term = help[i]

Some general principles to be aware of:

* Almost nothing is free -- Python very literally does what
you tell it to do, even if it looks dumb.

* Access to attributes and global variables is expensive
(requires at least one dict lookup, often more in the case
of attributes).

* Access to *local* variables, on the other hand, is very
cheap (essentially an array lookup).

* Function calls are expensive -- both to look up the name, if
it's global, which it usually is, and the machinery of the
call itself.

* Creating objects is expensive. Creating instances of
user-defined objects is more expensive than built-in ones.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Thanks for your response, will have a look.
Ok, dis() is all that is need to disassemble.

Very cool!

A long term goal could be indeed to have
a Prolog interpreter produce 20MLips, like
SWI-Prolog, but tightly integrated into

Python. So that it directly makes use of
the Python objects and the Python garbage
collection like Dogelog Runtime.

Although Dogelog Runtime has its own
garbage collection, its only used to help
the native Python garbage collection.

The result is that you can enjoy bi-directly
calling Python. For example the Prolog
adding of two numbers is realized as:

###
# +(A, B, C): [ISO 9.1.7]
# The predicate succeeds in C with the sum of A and B.
##
def eval_add(alpha, beta):
check_number(alpha)
check_number(beta)
try:
return alpha + beta
except OverflowError:
raise make_error(Compound("evaluation_error", ["float_overflow"]))

And then register it:

add("+", 3, make_dispatch(eval_add, MASK_MACH_FUNC))

Could also map the exception to a Prolog term later.
Thats not so much an issue for speed. The sunshine
case is straight forward.

But I might try dis() on eval_add(). Are exceptions
blocks in Python cheap or expensive? Are they like
in Java, some code annotation, or like in Go

programming language pushing some panic handler?

Greg Ewing schrieb:
> On 16/09/21 4:23 am, Mostowski Collapse wrote:
>> I really wonder why my Python implementation
>> is a factor 40 slower than my JavaScript implementation.
>
> There are Javascript implementations around nowadays that are
> blazingly fast. Partly that's because a lot of effort has been
> put into them, but it's also because Javascript is a different
> language. There are many dynamic aspects to Python that make
> fast implementations difficult.
>
>> I use in Python:
>>
>>    temp = [NotImplemented] * code[pos]
>>    pos += 1
>>
>> is the the idiom [_] * _ slow?
>
> No, on the contrary it's probably the fastest way to do it
> in Python. You could improve it a bit by precomputing
> [NotImplemented]:
>
> # once at the module level
> NotImplementedList = [NotImplemented]
>
> # whenever you want a new list
> temp = NotImplementedList * code[pos]
>
> That's probably at least as fast as built-in function for
> creating lists would be.
>
>> does it really first create an
>> array of size 1 and then enlarge it?
>
> It does:
>
> >>> def f(code, pos):
> ...  return [NotImplemented] * code[pos]
> ...
> >>> from dis import dis
> >>> dis(f)
>   2           0 LOAD_GLOBAL              0 (NotImplemented)
>               2 BUILD_LIST               1
>               4 LOAD_FAST                0 (code)
>               6 LOAD_FAST                1 (pos)
>               8 BINARY_SUBSCR
>              10 BINARY_MULTIPLY
>              12 RETURN_VALUE
>
> BTW, the Python terminology is "list", not "array".
> (There *is* something in the stdlib called an array, but
> it's rarely used or needed.)
>

--
https://mail.python.org/mailman/listinfo/python-list

No its cooperative. Usually objects do get
garbage collected by the native garbage collector
of the host language in Dogelog runtime.

The Prolog garbage collection is only to help
the host language garbage collector when you have
a deep recursion in Prolog.

You can then reclaim intermediate variables.
A simple example to test the slightly idio-
syncratic Prolog garbage collection is:

fibo(0, 1) :- !.
fibo(1, 1) :- !.
fibo(N, X) :-
M is N-1, fibo(M, Y),
L is M-1, fibo(L, Z),
X is Y+Z.

When running fibo(30,X) SWI-Prolog does around
800 garbage collections to keep the environment
small. But SWI-Prolog allocates all its objects

only very seldom on the heap. It uses its own
stack. On the other hand Dogelog runtime creates
everything on the heap. And completely relies on

the host language garbage collection. It only
helps the host language garbage collection it
that it performs from time to time this movement:

Before:

-->[ A ]-->[ B ]-->[ C ]-->

After:

-->[ A ]---------->[ C ]-->

A,B,C are objects of type Variable. The above
movement only happens for objects of type Variables
from time to time. For objects of type Compound

no modifications are done during Prolog garbage
collection. The Prolog garbage collection aggressively
nulls the Variable object B, and the host language

later will garbage collect what the Variable object B
was pointing to. But the Variable object B might
nevertheless have point to something shared with

some other Variable object or a local or a global
Python variable, or a Compound. This is then all
courtesy of the host language to decide reachability.

Chris Angelico schrieb:
> On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>>
>> About Exceptions: Thats just building ISO core
>> standard Prolog error terms.
>>
>> About Garbage Collection: Thats just Prolog
>> garbage collection, which does shrink some
>> single linked lists, which ordinary
>> programmig language GC cannot do,
>>
>
> Okay, so.... you're building your own garbage collection on top of
> Python's, and you're wondering why it's slow?
>
> Change your code to not try to implement one language inside another,
> and you'll see a massive performance improvement.
>
> ChrisA
>

--
https://mail.python.org/mailman/listinfo/python-list

The Prolog garbage collection that does
the movement on the variable trail is only
a very small fraction of the runtime.

The garbage collection time is measured.
Some measurements with version 0.9.5
took the following values:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Standard Python Version, Warm Run
% ?- time(fibo(23,X)).
% % Wall 3865 ms, gc 94 ms, 71991 lips
% X = 46368.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% GraalVM Python Version, Warm Warm Run
% ?- time(fibo(23,X)).
% % Wall 695 ms, gc 14 ms, 400356 lips
% X = 46368.

The "gc" timing measures Prolog garbage
collection. So you get the following percentage
of time spent in Prolog garbage collection:

Standard Python: 94 / 3865 = 2.4%

GraalVM Python: 14 / 695 = 2.0%

I consider this a good result. The Prolog
garbage collection is not utterly expensive.
The detecting the movement and performing

the variable movement on the trail, doesn't
take so much time. Currently the garbage collection
in Dogelog runtime is configured towards

synchronization with 60FPS, it does around
60-120 garbage collections per second. This
is less than what SWI-Prolog does.

SWI-Prolog has a much higher GC rate.

But I did not yet measure new version 0.9.6.

Mostowski Collapse schrieb:
> No its cooperative. Usually objects do get
> garbage collected by the native garbage collector
> of the host language in Dogelog runtime.
>
> The Prolog garbage collection is only to help
> the host language garbage collector when you have
> a deep recursion in Prolog.
>
> You can then reclaim intermediate variables.
> A simple example to test the slightly idio-
> syncratic Prolog garbage collection is:
>
> fibo(0, 1) :- !.
> fibo(1, 1) :- !.
> fibo(N, X) :-
>    M is N-1, fibo(M, Y),
>    L is M-1, fibo(L, Z),
>    X is Y+Z.
>
> When running fibo(30,X) SWI-Prolog does around
> 800 garbage collections to keep the environment
> small. But SWI-Prolog allocates all its objects
>
> only very seldom on the heap. It uses its own
> stack. On the other hand Dogelog runtime creates
> everything on the heap. And completely relies on
>
> the host language garbage collection. It only
> helps the host language garbage collection it
> that it performs from time to time this movement:
>
> Before:
>
>     -->[ A ]-->[ B ]-->[ C ]-->
>
> After:
>
>     -->[ A ]---------->[ C ]-->
>
> A,B,C are objects of type Variable. The above
> movement only happens for objects of type Variables
> from time to time. For objects of type Compound
>
> no modifications are done during Prolog garbage
> collection. The Prolog garbage collection aggressively
> nulls the Variable object B, and the host language
>
> later will garbage collect what the Variable object B
> was pointing to. But the Variable object B might
> nevertheless have point to something shared with
>
> some other Variable object or a local or a global
> Python variable, or a Compound. This is then all
> courtesy of the host language to decide reachability.
>
> Chris Angelico schrieb:
>> On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse
>> <janburse@fastmail.fm> wrote:
>>>
>>> About Exceptions: Thats just building ISO core
>>> standard Prolog error terms.
>>>
>>> About Garbage Collection: Thats just Prolog
>>> garbage collection, which does shrink some
>>> single linked lists, which ordinary
>>> programmig language GC cannot do,
>>>
>>
>> Okay, so.... you're building your own garbage collection on top of
>> Python's, and you're wondering why it's slow?
>>
>> Change your code to not try to implement one language inside another,
>> and you'll see a massive performance improvement.
>>
>> ChrisA
>>
>

--
https://mail.python.org/mailman/listinfo/python-list

Concerning garbage collection, did a long term
measurement for the first time. I measured
LIPS for fibonacci numbers, i.e. time(fibo(23,X)).

Doing the same thing 16 times, how long does it take?
Here is a depiction how the LIPS relatively differ in each run:
https://gist.github.com/jburse/c85297e97091caf22d306dd8c8be12fe#gistcomment-3896343

I can also calculate the mean and standard deviation.
From this we see that Python has a 5% deviation, whereas
GraalVM has a 1% deviation. So the GraalVM garbage

collector works more evenly? Disclaimer, I measured
different time spans, the GraalVM is now 7x times
faster than Standard Python, so this is inconclusive.

Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> The Prolog garbage collection that does
> the movement on the variable trail is only
> a very small fraction of the runtime.
>
> The garbage collection time is measured.
> Some measurements with version 0.9.5
> took the following values:
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % Standard Python Version, Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 3865 ms, gc 94 ms, 71991 lips
> % X = 46368.
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % GraalVM Python Version, Warm Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 695 ms, gc 14 ms, 400356 lips
> % X = 46368.
>
> The "gc" timing measures Prolog garbage
> collection. So you get the following percentage
> of time spent in Prolog garbage collection:
>
> Standard Python: 94 / 3865 = 2.4%
>
> GraalVM Python: 14 / 695 = 2.0%
>
> I consider this a good result. The Prolog
> garbage collection is not utterly expensive.
> The detecting the movement and performing
>
> the variable movement on the trail, doesn't
> take so much time. Currently the garbage collection
> in Dogelog runtime is configured towards
>
> synchronization with 60FPS, it does around
> 60-120 garbage collections per second. This
> is less than what SWI-Prolog does.
>
> SWI-Prolog has a much higher GC rate.
>
> But I did not yet measure new version 0.9.6.
>
> Mostowski Collapse schrieb:
> > No its cooperative. Usually objects do get
> > garbage collected by the native garbage collector
> > of the host language in Dogelog runtime.
> >
> > The Prolog garbage collection is only to help
> > the host language garbage collector when you have
> > a deep recursion in Prolog.
> >
> > You can then reclaim intermediate variables.
> > A simple example to test the slightly idio-
> > syncratic Prolog garbage collection is:
> >
> > fibo(0, 1) :- !.
> > fibo(1, 1) :- !.
> > fibo(N, X) :-
> > M is N-1, fibo(M, Y),
> > L is M-1, fibo(L, Z),
> > X is Y+Z.
> >
> > When running fibo(30,X) SWI-Prolog does around
> > 800 garbage collections to keep the environment
> > small. But SWI-Prolog allocates all its objects
> >
> > only very seldom on the heap. It uses its own
> > stack. On the other hand Dogelog runtime creates
> > everything on the heap. And completely relies on
> >
> > the host language garbage collection. It only
> > helps the host language garbage collection it
> > that it performs from time to time this movement:
> >
> > Before:
> >
> > -->[ A ]-->[ B ]-->[ C ]-->
> >
> > After:
> >
> > -->[ A ]---------->[ C ]-->
> >
> > A,B,C are objects of type Variable. The above
> > movement only happens for objects of type Variables
> > from time to time. For objects of type Compound
> >
> > no modifications are done during Prolog garbage
> > collection. The Prolog garbage collection aggressively
> > nulls the Variable object B, and the host language
> >
> > later will garbage collect what the Variable object B
> > was pointing to. But the Variable object B might
> > nevertheless have point to something shared with
> >
> > some other Variable object or a local or a global
> > Python variable, or a Compound. This is then all
> > courtesy of the host language to decide reachability.
> >
> > Chris Angelico schrieb:
> >> On Fri, Sep 17, 2021 at 7:17 AM Mostowski Collapse
> >> <janb...@fastmail.fm> wrote:
> >>>
> >>> About Exceptions: Thats just building ISO core
> >>> standard Prolog error terms.
> >>>
> >>> About Garbage Collection: Thats just Prolog
> >>> garbage collection, which does shrink some
> >>> single linked lists, which ordinary
> >>> programmig language GC cannot do,
> >>>
> >>
> >> Okay, so.... you're building your own garbage collection on top of
> >> Python's, and you're wondering why it's slow?
> >>
> >> Change your code to not try to implement one language inside another,
> >> and you'll see a massive performance improvement.
> >>
> >> ChrisA
> >>
> >
--
https://mail.python.org/mailman/listinfo/python-list

On 17/09/21 7:56 am, Mostowski Collapse wrote:
> The trail in Dogelog
> Runtime is a single linked list:
>
>     -->[ A ]-->[ B ]-->[ C ]-->
>
> Now if B becomes unused, you need to rewire
> the trail, it should then look like:
>
>     -->[ A ]---------->[ C ]-->

Python has a way of creating weak references (see the weakref
module). You could set the trail up like this:

-->[*]-->[*]-->[*]-->
| | |
v v v
[w] [w] [w]
: : :
v v v
[A] [B] [C]

where [w] is a weak reference object. Then you could periodically
scan the trail looking for dead weakref objects and remove the
corresponding [*] node from the list.

You can also attach callbacks to weakref objects that are triggered
when the referenced object dies. You might be able to make use of
that to remove items from the trail instead of the periodic scanning.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

On 16/09/21 6:13 am, Mostowski Collapse wrote:
> So in Python I now do the following:
>
> peek = kb.get(functor, NotImplemented)
> if peek is not NotImplemented:

If you're able to use None instead of NotImplemented, you
could simplify that to

peek = kb.get(functor)
if peek:
...

> But if get() in Python is implemented under the hood with
> exception handling. ... then Python get() will probably be quite slow.

No, it doesn't use exceptions as far as I know. It should
be fairly fast.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

On 17/09/21 8:41 pm, Mostowski Collapse wrote:
> Are exceptions
> blocks in Python cheap or expensive? Are they like
> in Java, some code annotation, or like in Go
> programming language pushing some panic handler?

They're not free, but they're not hugely expensive either.
Don't be afraid to use them when it makes sense to do so.

I'm not sure what you mean by "some code annotations",
so I don't know how to answer that question. I suspect
that the way exceptions are implemented in the JVM is
similar to the way CPython does it, but I don't know
a lot about the JVM.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Yeah, it seems weak references could indeed spare
me mark_term(). But then I am stil left with sweep_trail().
I did not yet measure what takes more time mark_term()
or sweep_trail(). The displayed "gc" is the sum of both.

From what I have seen, very large trail practically reduced
to a zero trail during Prolog GC, I am assuming that
mark_term() is not the working horse. Usually mark_term()
only marks what is not-Garbage, and sweep_trail()

has to deal with Garbage and not-Garbage. And there
is usually a lot of Garbage, much more than not-Garbage.
Finding the objects that survive, is like finding the needle
in the haystack, except we do not have to scan the

haystack, the needles are on the goal list. But afterwards,
the second pass, scanning the trail is the work of Heracles
cleaning the Augeas stables. This scan is trowing away the
hay and keeping the needles.

Greg Ewing schrieb am Samstag, 18. September 2021 um 05:49:37 UTC+2:
> On 17/09/21 7:56 am, Mostowski Collapse wrote:
> > The trail in Dogelog
> > Runtime is a single linked list:
> >
> > -->[ A ]-->[ B ]-->[ C ]-->
> >
> > Now if B becomes unused, you need to rewire
> > the trail, it should then look like:
> >
> > -->[ A ]---------->[ C ]-->
> Python has a way of creating weak references (see the weakref
> module). You could set the trail up like this:
>
> -->[*]-->[*]-->[*]-->
> | | |
> v v v
> [w] [w] [w]
> : : :
> v v v
> [A] [B] [C]
>
> where [w] is a weak reference object. Then you could periodically
> scan the trail looking for dead weakref objects and remove the
> corresponding [*] node from the list.
>
> You can also attach callbacks to weakref objects that are triggered
> when the referenced object dies. You might be able to make use of
> that to remove items from the trail instead of the periodic scanning.
>
> --
> Greg
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>
> Yeah, it seems weak references could indeed spare
> me mark_term(). But then I am stil left with sweep_trail().
> I did not yet measure what takes more time mark_term()
> or sweep_trail(). The displayed "gc" is the sum of both.
>
> From what I have seen, very large trail practically reduced
> to a zero trail during Prolog GC, I am assuming that
> mark_term() is not the working horse. Usually mark_term()
> only marks what is not-Garbage, and sweep_trail()
>
> has to deal with Garbage and not-Garbage. And there
> is usually a lot of Garbage, much more than not-Garbage.
> Finding the objects that survive, is like finding the needle
> in the haystack, except we do not have to scan the

If you stop referring to something, it is garbage. Python will dispose of it.

You literally need to do nothing at all, and let the language take
care of things.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

I am refering to:

Greg Ewing schrieb:
> where [w] is a weak reference object. Then you could periodically
> scan the trail looking for dead weakref objects and remove the
> corresponding [*] node from the list.
>
> You can also attach callbacks to weakref objects that are triggered
> when the referenced object dies. You might be able to make use of
> that to remove items from the trail instead of the periodic scanning.

Question to Chris Angelico: If I stay with my
sweep_trail(), which is the periodically scanning,
I can use a single linked list.

On the other hand if I would use the trigger
from Python, I possibly would need a double linked
list, to remove an element.

Chris Angelico, is there a third option, that I have
overlooked? Single linked list uses less space
than double linked list, this why I go with scan.

Chris Angelico schrieb:
> On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>>
>> Yeah, it seems weak references could indeed spare
>> me mark_term(). But then I am stil left with sweep_trail().
>> I did not yet measure what takes more time mark_term()
>> or sweep_trail(). The displayed "gc" is the sum of both.
>>
>> From what I have seen, very large trail practically reduced
>> to a zero trail during Prolog GC, I am assuming that
>> mark_term() is not the working horse. Usually mark_term()
>> only marks what is not-Garbage, and sweep_trail()
>>
>> has to deal with Garbage and not-Garbage. And there
>> is usually a lot of Garbage, much more than not-Garbage.
>> Finding the objects that survive, is like finding the needle
>> in the haystack, except we do not have to scan the
>
> If you stop referring to something, it is garbage. Python will dispose of it.
>
> You literally need to do nothing at all, and let the language take
> care of things.
>
> ChrisA
>

--
https://mail.python.org/mailman/listinfo/python-list

The trail itself can possibly not be eliminated. Its like a
database logfile. The trail is used during backtracking to
undo variable bindings. Which is like a database rollback.

Here is an example where a tail is used:

/* X equals 1 or X equals 2 */
?- X=1; X=2.
X = 1;
X = 2.

In the first answer the trail will have recorded that X
was bound to 1. When the second answer is requested,
the trail is used to unbind X, so that it can be bound to 2.

The Prolog garbage collection is a compactification
of the trail. Maybe databases can do the same with their
logfiles, for example if a logfile contains an insert and

then a delete of the same row, then these two logentries
can be merged. The only difference here is that Prolog
garbage collection primarily compactes towards trail

entries that have become irrelevant. This is part of
intelligent bracktracking, and backjumping over multiple
goal invocations, that didn't record a choice point.

The Prolog garbage collection can compact the trail
before backjumping happens. So that Prolog search
has more space available.

Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 10:51:03 UTC+2:
> I am refering to:
>
> Greg Ewing schrieb:
> > where [w] is a weak reference object. Then you could periodically
> > scan the trail looking for dead weakref objects and remove the
> > corresponding [*] node from the list.
> >
> > You can also attach callbacks to weakref objects that are triggered
> > when the referenced object dies. You might be able to make use of
> > that to remove items from the trail instead of the periodic scanning.
> Question to Chris Angelico: If I stay with my
> sweep_trail(), which is the periodically scanning,
> I can use a single linked list.
>
> On the other hand if I would use the trigger
> from Python, I possibly would need a double linked
> list, to remove an element.
>
> Chris Angelico, is there a third option, that I have
> overlooked? Single linked list uses less space
> than double linked list, this why I go with scan.
>
> Chris Angelico schrieb:
> > On Sun, Sep 19, 2021 at 11:46 AM Mostowski Collapse <burs...@gmail.com> wrote:
> >>
> >> Yeah, it seems weak references could indeed spare
> >> me mark_term(). But then I am stil left with sweep_trail().
> >> I did not yet measure what takes more time mark_term()
> >> or sweep_trail(). The displayed "gc" is the sum of both.
> >>
> >> From what I have seen, very large trail practically reduced
> >> to a zero trail during Prolog GC, I am assuming that
> >> mark_term() is not the working horse. Usually mark_term()
> >> only marks what is not-Garbage, and sweep_trail()
> >>
> >> has to deal with Garbage and not-Garbage. And there
> >> is usually a lot of Garbage, much more than not-Garbage.
> >> Finding the objects that survive, is like finding the needle
> >> in the haystack, except we do not have to scan the
> >
> > If you stop referring to something, it is garbage. Python will dispose of it.
> >
> > You literally need to do nothing at all, and let the language take
> > care of things.
> >
> > ChrisA
> >
--
https://mail.python.org/mailman/listinfo/python-list

On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>
> I am refering to:
>
> Greg Ewing schrieb:
> > where [w] is a weak reference object. Then you could periodically
> > scan the trail looking for dead weakref objects and remove the
> > corresponding [*] node from the list.
> >
> > You can also attach callbacks to weakref objects that are triggered
> > when the referenced object dies. You might be able to make use of
> > that to remove items from the trail instead of the periodic scanning.
>
> Question to Chris Angelico: If I stay with my
> sweep_trail(), which is the periodically scanning,
> I can use a single linked list.
>
> On the other hand if I would use the trigger
> from Python, I possibly would need a double linked
> list, to remove an element.
>
> Chris Angelico, is there a third option, that I have
> overlooked? Single linked list uses less space
> than double linked list, this why I go with scan.
>

I don't know. I don't understand your code well enough to offer advice
like that, because *your code is too complicated* and not nearly clear
enough.

But however it is that you're doing things, the best way is almost
always to directly refer to objects. Don't fiddle around with creating
your own concept of a doubly-linked list and a set of objects; just
refer directly to the objects. Let Python be Python, don't try to
build your own language on top of it.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

sympy also builds a language on top of Python.
pandas also builds a language on top of Python.

Is there some pope that says this wouldn't be
allowed, I dont think so, otherwise sympy, pandas, etc..

wouldn't exist. I dont understand your argument.

Chris Angelico schrieb:
> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>>
>> I am refering to:
>>
>> Greg Ewing schrieb:
>> > where [w] is a weak reference object. Then you could periodically
>> > scan the trail looking for dead weakref objects and remove the
>> > corresponding [*] node from the list.
>> >
>> > You can also attach callbacks to weakref objects that are triggered
>> > when the referenced object dies. You might be able to make use of
>> > that to remove items from the trail instead of the periodic scanning.
>>
>> Question to Chris Angelico: If I stay with my
>> sweep_trail(), which is the periodically scanning,
>> I can use a single linked list.
>>
>> On the other hand if I would use the trigger
>> from Python, I possibly would need a double linked
>> list, to remove an element.
>>
>> Chris Angelico, is there a third option, that I have
>> overlooked? Single linked list uses less space
>> than double linked list, this why I go with scan.
>>
>
> I don't know. I don't understand your code well enough to offer advice
> like that, because *your code is too complicated* and not nearly clear
> enough.
>
> But however it is that you're doing things, the best way is almost
> always to directly refer to objects. Don't fiddle around with creating
> your own concept of a doubly-linked list and a set of objects; just
> refer directly to the objects. Let Python be Python, don't try to
> build your own language on top of it.
>
> ChrisA
>

--
https://mail.python.org/mailman/listinfo/python-list

Also I nowhere wrote that I would use doubly-linked list or
a set of objects. The trail is neither, its a single linked list. I
also refer directly to objects, I do not need the trail to refer

to objects. The Prolog trail is only a logbook. The Prolog trail
has similarity to a database log:

transaction journal, database log, binary log or audit **trail**
https://en.wikipedia.org/wiki/Transaction_log

Do you say Python should not be used to implement
such things? In my opinion, Python has a high potential
to implement Prolog, because it has also ast.parse()

and compile(). But I do not yet have a showcase that uses
these features of Python to compile Prolog. I dont work 24/7
and I cannot clone myself. Currently the Prolog code is interpreted.

I have a prototype where Prolog code is compiled into JavaScript,
but I did not yet try this approach with Python. Here you see how
JavaScript closures are generated, a first prototype:

const alt4 = make_defined([new Clause(1, [0, 0], function(
display, actual, cont) {return(new Compound(".", [new Compound(
"==", [deref(actual.args[0]), "end_of_file"]), new Compound(
".", [new Compound("$CUT", [deref(display[0])]), cont
])]))}, 0, undefined), new Clause(1, [0, 0], function(
display, actual, cont) {return(new Compound(".", [new Compound(
"expand_include", [deref(actual.args[0]), deref(actual.args[1]
), display[0] = new Variable()]), new Compound(".",
[new Compound("handle_term", [deref(display[0])]), new Compound(
".", ["fail", cont])])]))}, -1, undefined)]);

add("next_term", 1, new Clause(2, [0], function(display, actual,
cont) {return(new Compound(".", [new Compound("read_term",
[deref(actual.args[0]), display[0] = new Variable(),
new Compound(".", [new Compound("variable_names", [
display[1] = new Variable()]), "[]"])]), new Compound(
".", [new Compound(alt4, [deref(display[0]), deref(
display[1])]), cont])]))}, -1, undefined));

https://github.com/jburse/dogelog-moon/issues/184

Will do the same for Python in the next weeks. Then later this approach
will be combined with a few planned optimizations. So far got a 25%
speed increase for JavaScript with this new compilation scheme, but

there is no official release out yet, that features this approach. And
there should be much more in it, also for Python.

Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 21:46:20 UTC+2:
> sympy also builds a language on top of Python.
> pandas also builds a language on top of Python.
>
> Is there some pope that says this wouldn't be
> allowed, I dont think so, otherwise sympy, pandas, etc..
>
> wouldn't exist. I dont understand your argument.
>
> Chris Angelico schrieb:
> > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janb...@fastmail.fm> wrote:
> >>
> >> I am refering to:
> >>
> >> Greg Ewing schrieb:
> >> > where [w] is a weak reference object. Then you could periodically
> >> > scan the trail looking for dead weakref objects and remove the
> >> > corresponding [*] node from the list.
> >> >
> >> > You can also attach callbacks to weakref objects that are triggered
> >> > when the referenced object dies. You might be able to make use of
> >> > that to remove items from the trail instead of the periodic scanning.
> >>
> >> Question to Chris Angelico: If I stay with my
> >> sweep_trail(), which is the periodically scanning,
> >> I can use a single linked list.
> >>
> >> On the other hand if I would use the trigger
> >> from Python, I possibly would need a double linked
> >> list, to remove an element.
> >>
> >> Chris Angelico, is there a third option, that I have
> >> overlooked? Single linked list uses less space
> >> than double linked list, this why I go with scan.
> >>
> >
> > I don't know. I don't understand your code well enough to offer advice
> > like that, because *your code is too complicated* and not nearly clear
> > enough.
> >
> > But however it is that you're doing things, the best way is almost
> > always to directly refer to objects. Don't fiddle around with creating
> > your own concept of a doubly-linked list and a set of objects; just
> > refer directly to the objects. Let Python be Python, don't try to
> > build your own language on top of it.
> >
> > ChrisA
> >
--
https://mail.python.org/mailman/listinfo/python-list

Please be patient. A big problem with development can be
burnout. So I am trying to slow down things at the moment.
The ideas are easy to sketch, but implementation can take

weeks. Here is the idea again in a nutshell: use ast.parse()
and compile(). Or build directly an AST, not using the string
detour as in ast.parse(). How long will it take to have a

working solution? 11 years ago there was:

Pyrolog: Prolog written in Python using PyPy's RPython tool chain
https://www.reddit.com/r/prolog/comments/fbuz1/pyrolog_prolog_written_in_python_using_pypys/

RPython is a framework for implementing interpreters and virtual
machines for programming languages, especially dynamic languages.
https://rpython.readthedocs.io/en/latest/faq.html

Currently I am not planning to use RPython, want to to use
standard Python AST and compile(). Might have a look at
RPython or similar stuff later.

Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 22:02:41 UTC+2:
> Also I nowhere wrote that I would use doubly-linked list or
> a set of objects. The trail is neither, its a single linked list. I
> also refer directly to objects, I do not need the trail to refer
>
> to objects. The Prolog trail is only a logbook. The Prolog trail
> has similarity to a database log:
>
> transaction journal, database log, binary log or audit **trail**
> https://en.wikipedia.org/wiki/Transaction_log
>
> Do you say Python should not be used to implement
> such things? In my opinion, Python has a high potential
> to implement Prolog, because it has also ast.parse()
>
> and compile(). But I do not yet have a showcase that uses
> these features of Python to compile Prolog. I dont work 24/7
> and I cannot clone myself. Currently the Prolog code is interpreted.
>
> I have a prototype where Prolog code is compiled into JavaScript,
> but I did not yet try this approach with Python. Here you see how
> JavaScript closures are generated, a first prototype:
>
> const alt4 = make_defined([new Clause(1, [0, 0], function(
> display, actual, cont) {return(new Compound(".", [new Compound(
> "==", [deref(actual.args[0]), "end_of_file"]), new Compound(
> ".", [new Compound("$CUT", [deref(display[0])]), cont
> ])]))}, 0, undefined), new Clause(1, [0, 0], function(
> display, actual, cont) {return(new Compound(".", [new Compound(
> "expand_include", [deref(actual.args[0]), deref(actual.args[1]
> ), display[0] = new Variable()]), new Compound(".",
> [new Compound("handle_term", [deref(display[0])]), new Compound(
> ".", ["fail", cont])])]))}, -1, undefined)]);
>
> add("next_term", 1, new Clause(2, [0], function(display, actual,
> cont) {return(new Compound(".", [new Compound("read_term",
> [deref(actual.args[0]), display[0] = new Variable(),
> new Compound(".", [new Compound("variable_names", [
> display[1] = new Variable()]), "[]"])]), new Compound(
> ".", [new Compound(alt4, [deref(display[0]), deref(
> display[1])]), cont])]))}, -1, undefined));
>
> https://github.com/jburse/dogelog-moon/issues/184
>
> Will do the same for Python in the next weeks. Then later this approach
> will be combined with a few planned optimizations. So far got a 25%
> speed increase for JavaScript with this new compilation scheme, but
>
> there is no official release out yet, that features this approach. And
> there should be much more in it, also for Python.
> Mostowski Collapse schrieb am Sonntag, 19. September 2021 um 21:46:20 UTC+2:
> > sympy also builds a language on top of Python.
> > pandas also builds a language on top of Python.
> >
> > Is there some pope that says this wouldn't be
> > allowed, I dont think so, otherwise sympy, pandas, etc..
> >
> > wouldn't exist. I dont understand your argument.
> >
> > Chris Angelico schrieb:
> > > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janb...@fastmail.fm> wrote:
> > >>
> > >> I am refering to:
> > >>
> > >> Greg Ewing schrieb:
> > >> > where [w] is a weak reference object. Then you could periodically
> > >> > scan the trail looking for dead weakref objects and remove the
> > >> > corresponding [*] node from the list.
> > >> >
> > >> > You can also attach callbacks to weakref objects that are triggered
> > >> > when the referenced object dies. You might be able to make use of
> > >> > that to remove items from the trail instead of the periodic scanning.
> > >>
> > >> Question to Chris Angelico: If I stay with my
> > >> sweep_trail(), which is the periodically scanning,
> > >> I can use a single linked list.
> > >>
> > >> On the other hand if I would use the trigger
> > >> from Python, I possibly would need a double linked
> > >> list, to remove an element.
> > >>
> > >> Chris Angelico, is there a third option, that I have
> > >> overlooked? Single linked list uses less space
> > >> than double linked list, this why I go with scan.
> > >>
> > >
> > > I don't know. I don't understand your code well enough to offer advice
> > > like that, because *your code is too complicated* and not nearly clear
> > > enough.
> > >
> > > But however it is that you're doing things, the best way is almost
> > > always to directly refer to objects. Don't fiddle around with creating
> > > your own concept of a doubly-linked list and a set of objects; just
> > > refer directly to the objects. Let Python be Python, don't try to
> > > build your own language on top of it.
> > >
> > > ChrisA
> > >
--
https://mail.python.org/mailman/listinfo/python-list

> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>>
>> On the other hand if I would use the trigger
>> from Python, I possibly would need a double linked
>> list, to remove an element.

Here's another idea: Put weak references in the trail, but
don't bother with the scanning -- just skip over dead ones
during backtracking.

Seems to me you would be doing about the same amount of
work either way, just doing it at different times.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

This strategy works if you use failure driven loops.
It doesn't work you program recursive loops that go
on forever. Like Erlang processes.

foo(X) :-
bar(X,Y), foo(Y).

Typically Y is a fresh variable. A good Prolog system
with good Garbage Collection can run such a process
for days and months.

If you don't clean up the trail, you exhaust the
memory after a few minutes or seconds.

Greg Ewing schrieb:
>> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse
>> <janburse@fastmail.fm> wrote:
>>>
>>> On the other hand if I would use the trigger
>>> from Python, I possibly would need a double linked
>>> list, to remove an element.
>
> Here's another idea: Put weak references in the trail, but
> don't bother with the scanning -- just skip over dead ones
> during backtracking.
>
> Seems to me you would be doing about the same amount of
> work either way, just doing it at different times.
>

--
https://mail.python.org/mailman/listinfo/python-list

But I dont see any utility in investing too much time with
the Prolog garbage collection of Dogelog runtime. It is
only a small share of the execution time:

Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % Standard Python Version, Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 3865 ms, gc 94 ms, 71991 lips
> % X = 46368.
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % GraalVM Python Version, Warm Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 695 ms, gc 14 ms, 400356 lips
> % X = 46368.
>
> The "gc" timing measures Prolog garbage
> collection. So you get the following percentage
> of time spent in Prolog garbage collection:
>
> Standard Python: 94 / 3865 = 2.4%
>
> GraalVM Python: 14 / 695 = 2.0%

If you spare these 2-3% it will not speed-up Dogelog runtime.
The Prolog garbage collection is there to allow Erlang
processes. And its also there to allow more Prolog search

without exhausting the memory. But it cost you only 2-3%
judging from the Fibonnacci Numbers example. Need to
check with other examples whether its higher.

But since the ratio between Garbage and non-Garbage is
usually high, and non-Garbage is easily identified, and
the Garbage is also easily removed, the time will

possibly not exceed much more than the same 2-3% for
other examples. So in conclusion I am least worried
about the Prolog garbage collection. You guys are only

worried because its something new. But its nothing that
does any harm and costs a lot, it only does good and
is very cheap in terms of extra runtime effort.

Mostowski Collapse schrieb am Montag, 20. September 2021 um 08:44:49 UTC+2:
> This strategy works if you use failure driven loops.
> It doesn't work you program recursive loops that go
> on forever. Like Erlang processes.
>
> foo(X) :-
> bar(X,Y), foo(Y).
>
> Typically Y is a fresh variable. A good Prolog system
> with good Garbage Collection can run such a process
> for days and months.
>
> If you don't clean up the trail, you exhaust the
> memory after a few minutes or seconds.
>
> Greg Ewing schrieb:
> >> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse
> >> <janb...@fastmail.fm> wrote:
> >>>
> >>> On the other hand if I would use the trigger
> >>> from Python, I possibly would need a double linked
> >>> list, to remove an element.
> >
> > Here's another idea: Put weak references in the trail, but
> > don't bother with the scanning -- just skip over dead ones
> > during backtracking.
> >
> > Seems to me you would be doing about the same amount of
> > work either way, just doing it at different times.
> >
--
https://mail.python.org/mailman/listinfo/python-list

On 2021-09-20 04:33:39 +1000, Chris Angelico wrote:
> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
> > Question to Chris Angelico: If I stay with my
> > sweep_trail(), which is the periodically scanning,
> > I can use a single linked list.
> >
> > On the other hand if I would use the trigger
> > from Python, I possibly would need a double linked
> > list, to remove an element.
> >
> > Chris Angelico, is there a third option, that I have
> > overlooked? Single linked list uses less space
> > than double linked list, this why I go with scan.
> >
>
> I don't know. I don't understand your code well enough to offer advice
> like that, because *your code is too complicated* and not nearly clear
> enough.
>
> But however it is that you're doing things, the best way is almost
> always to directly refer to objects. Don't fiddle around with creating
> your own concept of a doubly-linked list and a set of objects; just
> refer directly to the objects.

And almost certainly: Just use the builtin list type if you need a list.
Don't build one yourself.


> Let Python be Python, don't try to build your own language on top of
> it.

Well, he's writing a Prolog interpreter, so building his own language on
top of Python is sort of the point. I think a better way to put it is
"Don't try to write Python as if it was C". A C operation may be
compiled to a single machine instruction which is executed in a fraction
of a nanosecond. A Python instruction (in CPython) always includes at
least the interpreter overhead and often several method lookups and method
calls. You want to amortize that overhead over as much work as possible.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

I read the following, and you should also know:

> Python's [] is implemented as an array, not a linked list.
> Although resizing is O(n), appending to it is amortized O(1),
> because resizes happen very rarely.
https://stackoverflow.com/a/5932364/502187

The list type doesn't have an O(1) operation to remove
an element during sweep. The list type, not like its name
would suggest, in Python is an array.

These arrays are not so expensive when you append()
an element. Because they are allocated with some excess
capacity. And they grow exponentially.

So amortisized you can append() a lot of elements to
a Python list, which is an array. But you cannot poke so
cheaply holes into it. So if you have this scenario:

Before:
- [ A1, .., An , B, C1, .., Cm ]

After:
- [ A1, .., An , C1, .., Cm ]

You have to copy C1,..,Cm one position down. On the other
hand, when scanning the single list, removing the
element is just pointer swizzling.

The code is here, the positive if-then-else branch keeps
the element, the negative if-then-else branch drops the
element. Thats quite standard algorithm for linked lists:

/* pointer swizzling */
while temp is not None:
term = temp
temp = term.tail
if (term.flags & MASK_VAR_MARK) != 0:
term.flags &= ~MASK_VAR_MARK
if back is not None:
back.tail = term
else:
trail = term
back = term
else:
term.instantiated = NotImplemented
term.tail = None
count -= 1

https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163

There is nothing wrong with implementing a single list
in Python. Only you never saw that one maybe. If you would
indeed use Python lists which are arrays, you would

maybe get a much slower sweep_trail() and this would
be seen. But currently its not seen. It happens that 1000000
of elements are sweeped, if you would do this with copy

inside a Python list which are arrays, it would get much
more expensive, and the extremly cheap Prolog garbage
collection, as it stands now, wouldn't be that cheap anymore.

You can try yourself. My sweep_trail() needs frequent resize,
which would be O(n) each, so that sweep_trail becomes O(n^2).
Which the current implementation its only O(n).

Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2:
> On 2021-09-20 04:33:39 +1000, Chris Angelico wrote:
> > On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janb...@fastmail.fm> wrote:
> > > Question to Chris Angelico: If I stay with my
> > > sweep_trail(), which is the periodically scanning,
> > > I can use a single linked list.
> > >
> > > On the other hand if I would use the trigger
> > > from Python, I possibly would need a double linked
> > > list, to remove an element.
> > >
> > > Chris Angelico, is there a third option, that I have
> > > overlooked? Single linked list uses less space
> > > than double linked list, this why I go with scan.
> > >
> >
> > I don't know. I don't understand your code well enough to offer advice
> > like that, because *your code is too complicated* and not nearly clear
> > enough.
> >
> > But however it is that you're doing things, the best way is almost
> > always to directly refer to objects. Don't fiddle around with creating
> > your own concept of a doubly-linked list and a set of objects; just
> > refer directly to the objects.
> And almost certainly: Just use the builtin list type if you need a list.
> Don't build one yourself.
> > Let Python be Python, don't try to build your own language on top of
> > it.
> Well, he's writing a Prolog interpreter, so building his own language on
> top of Python is sort of the point. I think a better way to put it is
> "Don't try to write Python as if it was C". A C operation may be
> compiled to a single machine instruction which is executed in a fraction
> of a nanosecond. A Python instruction (in CPython) always includes at
> least the interpreter overhead and often several method lookups and method
> calls. You want to amortize that overhead over as much work as possible.
>
> hp
>
> --
> _ | Peter J. Holzer | Story must make more sense than reality.
> |_|_) | |
> | | | h...@hjp.at | -- Charles Stross, "Creative writing
> __/ | http://www.hjp.at/ | challenge!"
--
https://mail.python.org/mailman/listinfo/python-list

What would be maybe possible, is to
scan the trail from the other side, and
use a first pass to determine the new

size, and use a second pass to fill a new
array with the remaining elments. This would
be two times O(n), so it would be linear

and not quadratic O(n^2) as when you scan
from the top and poke holes. But then something
else doesn't work so easily. Namely:

def adjust_mark(temp):
while temp is not None:
if (temp.flags & MASK_VAR_MARK) != 0:
return temp
else:
temp = temp.tail
return temp

https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L151

This is needed to adjust the choice points.
If you have an array instead of a linked
listed as I have now, you would need

to adjust array indexes instead pointers
into linked list elements. I havent come
up with an array solution yet for the trail,

since I dont see any utility in investing too
much time with the Prolog garbage collection of
Dogelog runtime. It is only a small share

of the execution time:

Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % Standard Python Version, Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 3865 ms, gc 94 ms, 71991 lips
> % X = 46368.
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % GraalVM Python Version, Warm Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 695 ms, gc 14 ms, 400356 lips
> % X = 46368.

Mostowski Collapse wrote:
> I read the following, and you should also know:
>
>> Python's [] is implemented as an array, not a linked list.
>> Although resizing is O(n), appending to it is amortized O(1),
>> because resizes happen very rarely.
> https://stackoverflow.com/a/5932364/502187
>
> The list type doesn't have an O(1) operation to remove
> an element during sweep. The list type, not like its name
> would suggest, in Python is an array.
>
> These arrays are not so expensive when you append()
> an element. Because they are allocated with some excess
> capacity. And they grow exponentially.
>
> So amortisized you can append() a lot of elements to
> a Python list, which is an array. But you cannot poke so
> cheaply holes into it. So if you have this scenario:
>
> Before:
> - [ A1, .., An , B, C1, .., Cm ]
>
> After:
> - [ A1, .., An , C1, .., Cm ]
>
> You have to copy C1,..,Cm one position down. On the other
> hand, when scanning the single list, removing the
> element is just pointer swizzling.
>
> The code is here, the positive if-then-else branch keeps
> the element, the negative if-then-else branch drops the
> element. Thats quite standard algorithm for linked lists:
>
> /* pointer swizzling */
> while temp is not None:
> term = temp
> temp = term.tail
> if (term.flags & MASK_VAR_MARK) != 0:
> term.flags &= ~MASK_VAR_MARK
> if back is not None:
> back.tail = term
> else:
> trail = term
> back = term
> else:
> term.instantiated = NotImplemented
> term.tail = None
> count -= 1
>
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163
>
> There is nothing wrong with implementing a single list
> in Python. Only you never saw that one maybe. If you would
> indeed use Python lists which are arrays, you would
>
> maybe get a much slower sweep_trail() and this would
> be seen. But currently its not seen. It happens that 1000000
> of elements are sweeped, if you would do this with copy
>
> inside a Python list which are arrays, it would get much
> more expensive, and the extremly cheap Prolog garbage
> collection, as it stands now, wouldn't be that cheap anymore.
>
> You can try yourself. My sweep_trail() needs frequent resize,
> which would be O(n) each, so that sweep_trail becomes O(n^2).
> Which the current implementation its only O(n).
>
> Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2:
>> On 2021-09-20 04:33:39 +1000, Chris Angelico wrote:
>>> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse <janb...@fastmail.fm> wrote:
>>>> Question to Chris Angelico: If I stay with my
>>>> sweep_trail(), which is the periodically scanning,
>>>> I can use a single linked list.
>>>>
>>>> On the other hand if I would use the trigger
>>>> from Python, I possibly would need a double linked
>>>> list, to remove an element.
>>>>
>>>> Chris Angelico, is there a third option, that I have
>>>> overlooked? Single linked list uses less space
>>>> than double linked list, this why I go with scan.
>>>>
>>>
>>> I don't know. I don't understand your code well enough to offer advice
>>> like that, because *your code is too complicated* and not nearly clear
>>> enough.
>>>
>>> But however it is that you're doing things, the best way is almost
>>> always to directly refer to objects. Don't fiddle around with creating
>>> your own concept of a doubly-linked list and a set of objects; just
>>> refer directly to the objects.
>> And almost certainly: Just use the builtin list type if you need a list.
>> Don't build one yourself.
>>> Let Python be Python, don't try to build your own language on top of
>>> it.
>> Well, he's writing a Prolog interpreter, so building his own language on
>> top of Python is sort of the point. I think a better way to put it is
>> "Don't try to write Python as if it was C". A C operation may be
>> compiled to a single machine instruction which is executed in a fraction
>> of a nanosecond. A Python instruction (in CPython) always includes at
>> least the interpreter overhead and often several method lookups and method
>> calls. You want to amortize that overhead over as much work as possible.
>>
>> hp
>>
>> --
>> _ | Peter J. Holzer | Story must make more sense than reality.
>> |_|_) | |
>> | | | h...@hjp.at | -- Charles Stross, "Creative writing
>> __/ | http://www.hjp.at/ | challenge!"

--
https://mail.python.org/mailman/listinfo/python-list

In general the algorithm I am using is from
a paper by Matts Carlson from SICStus Prolog.
Its this paper here:

Garbage Collection for Prolog Based on WAM
January 1986
Karen Appleby, Mats Carlsson, Seif Haridi, Dan Sahlin
https://www.researchgate.net/publication/279463524

But since you guys are so facinated with
the Prolog garbage collection aspect, which
is not the locus where you can do a lot

of optimizations, feel free to investigate
and come up with a solution. It will not
change the performance of Dogelog runtime,

but it could be an academic experiment
neverthless. There is nothing wrong with the
simgle linked list as it stands, since

it has O(n) sweep_trail(). It uses a litte
more storage than an array would do.

Mostowski Collapse wrote:
> What would be maybe possible, is to
> scan the trail from the other side, and
> use a first pass to determine the new
>
> size, and use a second pass to fill a new
> array with the remaining elments. This would
> be two times O(n), so it would be linear
>
> and not quadratic O(n^2) as when you scan
> from the top and poke holes. But then something
> else doesn't work so easily. Namely:
>
>    def adjust_mark(temp):
>        while temp is not None:
>         if (temp.flags & MASK_VAR_MARK) != 0:
>             return temp
>         else:
>             temp = temp.tail
>     return temp
>
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L151
>
>
> This is needed to adjust the choice points.
> If you have an array instead of a linked
> listed as I have now, you would need
>
> to adjust array indexes instead pointers
> into linked list elements. I havent come
> up with an array solution yet for the trail,
>
> since I dont see any utility in investing too
> much time with the Prolog garbage collection of
> Dogelog runtime. It is only a small share
>
> of the execution time:
>
> Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57
> UTC+2:
> > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > % Standard Python Version, Warm Run
> > % ?- time(fibo(23,X)).
> > % % Wall 3865 ms, gc 94 ms, 71991 lips
> > % X = 46368.
> >
> > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > % GraalVM Python Version, Warm Warm Run
> > % ?- time(fibo(23,X)).
> > % % Wall 695 ms, gc 14 ms, 400356 lips
> > % X = 46368.
>
> Mostowski Collapse wrote:
>> I read the following, and you should also know:
>>
>>> Python's [] is implemented as an array, not a linked list.
>>> Although resizing is O(n), appending to it is amortized O(1),
>>> because resizes happen very rarely.
>> https://stackoverflow.com/a/5932364/502187
>>
>> The list type doesn't have an O(1) operation to remove
>> an element during sweep. The list type, not like its name
>> would suggest, in Python is an array.
>>
>> These arrays are not so expensive when you append()
>> an element. Because they are allocated with some excess
>> capacity. And they grow exponentially.
>>
>> So amortisized you can append() a lot of elements to
>> a Python list, which is an array. But you cannot poke so
>> cheaply holes into it. So if you have this scenario:
>>
>> Before:
>>       - [ A1, .., An , B, C1, .., Cm ]
>>
>> After:
>>       - [ A1, .., An , C1, .., Cm ]
>> You have to copy C1,..,Cm one position down. On the other
>> hand, when scanning the single list, removing the
>> element is just pointer swizzling.
>>
>> The code is here, the positive if-then-else branch keeps
>> the element, the negative if-then-else branch drops the
>> element. Thats quite standard algorithm for linked lists:
>>
>>       /* pointer swizzling */
>>      while temp is not None:
>>          term = temp
>>          temp = term.tail
>>          if (term.flags & MASK_VAR_MARK) != 0:
>>              term.flags &= ~MASK_VAR_MARK
>>              if back is not None:
>>                  back.tail = term
>>              else:
>>                  trail = term
>>              back = term
>>          else:
>>              term.instantiated = NotImplemented
>>              term.tail = None
>>              count -= 1
>>
>> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163
>>
>>
>> There is nothing wrong with implementing a single list
>> in Python. Only you never saw that one maybe. If you would
>> indeed use Python lists which are arrays, you would
>>
>> maybe get a much slower sweep_trail() and this would
>> be seen. But currently its not seen. It happens that 1000000
>> of elements are sweeped, if you would do this with copy
>>
>> inside a Python list which are arrays, it would get much
>> more expensive, and the extremly cheap Prolog garbage
>> collection, as it stands now, wouldn't be that cheap anymore.
>>
>> You can try yourself. My sweep_trail() needs frequent resize,
>> which would be O(n) each, so that sweep_trail becomes O(n^2).
>> Which the current implementation its only O(n).
>>
>> Peter J. Holzer schrieb am Montag, 20. September 2021 um 13:49:49 UTC+2:
>>> On 2021-09-20 04:33:39 +1000, Chris Angelico wrote:
>>>> On Mon, Sep 20, 2021 at 3:19 AM Mostowski Collapse
>>>> <janb...@fastmail.fm> wrote:
>>>>> Question to Chris Angelico: If I stay with my
>>>>> sweep_trail(), which is the periodically scanning,
>>>>> I can use a single linked list.
>>>>>
>>>>> On the other hand if I would use the trigger
>>>>> from Python, I possibly would need a double linked
>>>>> list, to remove an element.
>>>>>
>>>>> Chris Angelico, is there a third option, that I have
>>>>> overlooked? Single linked list uses less space
>>>>> than double linked list, this why I go with scan.
>>>>>
>>>>
>>>> I don't know. I don't understand your code well enough to offer advice
>>>> like that, because *your code is too complicated* and not nearly clear
>>>> enough.
>>>>
>>>> But however it is that you're doing things, the best way is almost
>>>> always to directly refer to objects. Don't fiddle around with creating
>>>> your own concept of a doubly-linked list and a set of objects; just
>>>> refer directly to the objects.
>>> And almost certainly: Just use the builtin list type if you need a list.
>>> Don't build one yourself.
>>>> Let Python be Python, don't try to build your own language on top of
>>>> it.
>>> Well, he's writing a Prolog interpreter, so building his own language on
>>> top of Python is sort of the point. I think a better way to put it is
>>> "Don't try to write Python as if it was C". A C operation may be
>>> compiled to a single machine instruction which is executed in a fraction
>>> of a nanosecond. A Python instruction (in CPython) always includes at
>>> least the interpreter overhead and often several method lookups and
>>> method
>>> calls. You want to amortize that overhead over as much work as possible.
>>>
>>> hp
>>>
>>> --
>>> _ | Peter J. Holzer | Story must make more sense than reality.
>>> |_|_) | |
>>> | | | h...@hjp.at | -- Charles Stross, "Creative writing
>>> __/ | http://www.hjp.at/ | challenge!"
>

--
https://mail.python.org/mailman/listinfo/python-list

On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer <hjp-python@hjp.at> wrote:
> > Let Python be Python, don't try to build your own language on top of
> > it.
>
> Well, he's writing a Prolog interpreter, so building his own language on
> top of Python is sort of the point. I think a better way to put it is
> "Don't try to write Python as if it was C".

Fair point. Or combining them both: Writing a language interpreter in
Python as if you were writing it in C, and then complaining that it is
slow, is only going to elicit "well uhh yes?" responses.

Languages like NetRexx (and, I think, Jython, although I can't find
any definitive and current answers) are slightly different from their
"parent" languages, because they make good use of their implementation
languages' features. This Prolog interpreter might not even need to be
different in functionality, but its implementation would be different,
and it could take advantage of the underlying garbage collection.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

The sweep_trail() is not an issue. There must be a bottleneck
somewhere else in Python. The sweep_trail() respectively the
paremt call gc() only takes a very small fraction of the runtime:

Check the "gc" timing, the bottleneck is somewhere else?

Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % Standard Python Version, Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 3865 ms, gc 94 ms, 71991 lips
> % X = 46368.
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % GraalVM Python Version, Warm Warm Run
> % ?- time(fibo(23,X)).
> % % Wall 695 ms, gc 14 ms, 400356 lips
> % X = 46368.

Also my code is not C-style. If I would use C-style code, I would
use address calculations and the adress operator &. But you
don't find and according C-style in the Python or JavaScript code.

Also there is no substitute for such coding style in the for
of value holders or some such. Its all plain Python respectively
JavaScript not at all inspired by the C programming language.

The single linked list is not some indicative of C programming
language style. With C programming language sytle one would
do other tricks, you cannot read off from my Python or JavaScript

code, since I cannot apply them to Python or JavaScript. Among
the other C programming language tricks not available in Python
or JavaScript would for example be inlining the args in Compound

and so on. But I am not sure whether this is the bottleneck.

Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2:
> On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > > Let Python be Python, don't try to build your own language on top of
> > > it.
> >
> > Well, he's writing a Prolog interpreter, so building his own language on
> > top of Python is sort of the point. I think a better way to put it is
> > "Don't try to write Python as if it was C".
> Fair point. Or combining them both: Writing a language interpreter in
> Python as if you were writing it in C, and then complaining that it is
> slow, is only going to elicit "well uhh yes?" responses.
>
> Languages like NetRexx (and, I think, Jython, although I can't find
> any definitive and current answers) are slightly different from their
> "parent" languages, because they make good use of their implementation
> languages' features. This Prolog interpreter might not even need to be
> different in functionality, but its implementation would be different,
> and it could take advantage of the underlying garbage collection.
>
> ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Also I am not a C programmer. The last time I was programming C
was 30 years ago. I am mostly a Java programmer the recent years.
Dogelog Runtime adopts a lot of implementation details from

Jekejeke Prolog which is a Prolog written in Java. The difference
betweeen the two is that Jekejeke Prolog has another Prolog term
model where Prolog terms are passed around as molecs, which

are pairs of skeleton and display. On the other in Dogelog Runtime
uses a simpler representation, Prolog terms are passed around
as only one object. Programming language wise the difference

between using Java and JavaScript or Python, is that Java has
types. So variables need a type declaration. Otherwise Java is
very similar to JavaScript or Python, it also provides a runtime with

a garbage collection. The idea I would use C-style is a little absurd. It
would also require that a free() objects manually. But sweep_trail() has
nothing to do with freeing objects manually, its the anti-thesis to

freeing objects manually, its a Prolog garbage collector after all!

Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:36:01 UTC+2:
> The sweep_trail() is not an issue. There must be a bottleneck
> somewhere else in Python. The sweep_trail() respectively the
> paremt call gc() only takes a very small fraction of the runtime:
>
> Check the "gc" timing, the bottleneck is somewhere else?
> Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > % Standard Python Version, Warm Run
> > % ?- time(fibo(23,X)).
> > % % Wall 3865 ms, gc 94 ms, 71991 lips
> > % X = 46368.
> >
> > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > % GraalVM Python Version, Warm Warm Run
> > % ?- time(fibo(23,X)).
> > % % Wall 695 ms, gc 14 ms, 400356 lips
> > % X = 46368.
> Also my code is not C-style. If I would use C-style code, I would
> use address calculations and the adress operator &. But you
> don't find and according C-style in the Python or JavaScript code.
>
> Also there is no substitute for such coding style in the for
> of value holders or some such. Its all plain Python respectively
> JavaScript not at all inspired by the C programming language.
>
> The single linked list is not some indicative of C programming
> language style. With C programming language sytle one would
> do other tricks, you cannot read off from my Python or JavaScript
>
> code, since I cannot apply them to Python or JavaScript. Among
> the other C programming language tricks not available in Python
> or JavaScript would for example be inlining the args in Compound
>
> and so on. But I am not sure whether this is the bottleneck.
> Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2:
> > On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > > > Let Python be Python, don't try to build your own language on top of
> > > > it.
> > >
> > > Well, he's writing a Prolog interpreter, so building his own language on
> > > top of Python is sort of the point. I think a better way to put it is
> > > "Don't try to write Python as if it was C".
> > Fair point. Or combining them both: Writing a language interpreter in
> > Python as if you were writing it in C, and then complaining that it is
> > slow, is only going to elicit "well uhh yes?" responses.
> >
> > Languages like NetRexx (and, I think, Jython, although I can't find
> > any definitive and current answers) are slightly different from their
> > "parent" languages, because they make good use of their implementation
> > languages' features. This Prolog interpreter might not even need to be
> > different in functionality, but its implementation would be different,
> > and it could take advantage of the underlying garbage collection.
> >
> > ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Now I am expecting somebody telling me I should not
program Java style in Python. So I wonder what would have
happened if I would tell you I am FORTRAN programmer.

Then somebody would tell me I should not program
FORTRAN style in Python. Hopefully this makes it evident
that this argumentation is moot.

Better would be more facts on the table based on
the existing code of Dogelog runtime. The future Dogelog
runtime code will possibly use ast.parse() and compile().

Thats why I am conducting this experiment which has only
been half way completed. The experiment is conducted
because Python belongs to the champ of dynamic languages:

Dynamic programming language
https://en.wikipedia.org/wiki/Dynamic_programming_language

The experiment will be completed when the ast.parse()
and compile() thingy was explored as well. As it happens I
am conducting the experiment in parallel for JavaScript and

Python, both being dynamic programming languages.

Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:43:01 UTC+2:
> Also I am not a C programmer. The last time I was programming C
> was 30 years ago. I am mostly a Java programmer the recent years.
> Dogelog Runtime adopts a lot of implementation details from
>
> Jekejeke Prolog which is a Prolog written in Java. The difference
> betweeen the two is that Jekejeke Prolog has another Prolog term
> model where Prolog terms are passed around as molecs, which
>
> are pairs of skeleton and display. On the other in Dogelog Runtime
> uses a simpler representation, Prolog terms are passed around
> as only one object. Programming language wise the difference
>
> between using Java and JavaScript or Python, is that Java has
> types. So variables need a type declaration. Otherwise Java is
> very similar to JavaScript or Python, it also provides a runtime with
>
> a garbage collection. The idea I would use C-style is a little absurd. It
> would also require that a free() objects manually. But sweep_trail() has
> nothing to do with freeing objects manually, its the anti-thesis to
>
> freeing objects manually, its a Prolog garbage collector after all!
> Mostowski Collapse schrieb am Montag, 20. September 2021 um 14:36:01 UTC+2:
> > The sweep_trail() is not an issue. There must be a bottleneck
> > somewhere else in Python. The sweep_trail() respectively the
> > paremt call gc() only takes a very small fraction of the runtime:
> >
> > Check the "gc" timing, the bottleneck is somewhere else?
> > Mostowski Collapse schrieb am Freitag, 17. September 2021 um 10:58:57 UTC+2:
> > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > > % Standard Python Version, Warm Run
> > > % ?- time(fibo(23,X)).
> > > % % Wall 3865 ms, gc 94 ms, 71991 lips
> > > % X = 46368.
> > >
> > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> > > % GraalVM Python Version, Warm Warm Run
> > > % ?- time(fibo(23,X)).
> > > % % Wall 695 ms, gc 14 ms, 400356 lips
> > > % X = 46368.
> > Also my code is not C-style. If I would use C-style code, I would
> > use address calculations and the adress operator &. But you
> > don't find and according C-style in the Python or JavaScript code.
> >
> > Also there is no substitute for such coding style in the for
> > of value holders or some such. Its all plain Python respectively
> > JavaScript not at all inspired by the C programming language.
> >
> > The single linked list is not some indicative of C programming
> > language style. With C programming language sytle one would
> > do other tricks, you cannot read off from my Python or JavaScript
> >
> > code, since I cannot apply them to Python or JavaScript. Among
> > the other C programming language tricks not available in Python
> > or JavaScript would for example be inlining the args in Compound
> >
> > and so on. But I am not sure whether this is the bottleneck.
> > Chris Angelico schrieb am Montag, 20. September 2021 um 14:25:12 UTC+2:
> > > On Mon, Sep 20, 2021 at 9:50 PM Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > > > > Let Python be Python, don't try to build your own language on top of
> > > > > it.
> > > >
> > > > Well, he's writing a Prolog interpreter, so building his own language on
> > > > top of Python is sort of the point. I think a better way to put it is
> > > > "Don't try to write Python as if it was C".
> > > Fair point. Or combining them both: Writing a language interpreter in
> > > Python as if you were writing it in C, and then complaining that it is
> > > slow, is only going to elicit "well uhh yes?" responses.
> > >
> > > Languages like NetRexx (and, I think, Jython, although I can't find
> > > any definitive and current answers) are slightly different from their
> > > "parent" languages, because they make good use of their implementation
> > > languages' features. This Prolog interpreter might not even need to be
> > > different in functionality, but its implementation would be different,
> > > and it could take advantage of the underlying garbage collection.
> > >
> > > ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Sep 21, 2021 at 3:51 AM Mostowski Collapse <janburse@fastmail.fm> wrote:
>
> sympy also builds a language on top of Python.
> pandas also builds a language on top of Python.
>
> Is there some pope that says this wouldn't be
> allowed, I dont think so, otherwise sympy, pandas, etc..
>
> wouldn't exist. I dont understand your argument.
>

That's not the same thing as reimplementing your own low-level
features on top of a high level language.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Sep 21, 2021 at 3:58 AM Mostowski Collapse <bursejan@gmail.com> wrote:
>
> I read the following, and you should also know:
>
> > Python's [] is implemented as an array, not a linked list.
> > Although resizing is O(n), appending to it is amortized O(1),
> > because resizes happen very rarely.
> https://stackoverflow.com/a/5932364/502187
>
> The list type doesn't have an O(1) operation to remove
> an element during sweep. The list type, not like its name
> would suggest, in Python is an array.
>
> These arrays are not so expensive when you append()
> an element. Because they are allocated with some excess
> capacity. And they grow exponentially.
>
> So amortisized you can append() a lot of elements to
> a Python list, which is an array. But you cannot poke so
> cheaply holes into it. So if you have this scenario:
>
> Before:
> - [ A1, .., An , B, C1, .., Cm ]
>
> After:
> - [ A1, .., An , C1, .., Cm ]
>
> You have to copy C1,..,Cm one position down. On the other
> hand, when scanning the single list, removing the
> element is just pointer swizzling.
>
> The code is here, the positive if-then-else branch keeps
> the element, the negative if-then-else branch drops the
> element. Thats quite standard algorithm for linked lists:
>
> /* pointer swizzling */
> while temp is not None:
> term = temp
> temp = term.tail
> if (term.flags & MASK_VAR_MARK) != 0:
> term.flags &= ~MASK_VAR_MARK
> if back is not None:
> back.tail = term
> else:
> trail = term
> back = term
> else:
> term.instantiated = NotImplemented
> term.tail = None
> count -= 1
>
> https://github.com/jburse/dogelog-moon/blob/main/devel/runtimepy/drawer/machine.py#L163
>
> There is nothing wrong with implementing a single list
> in Python. Only you never saw that one maybe. If you would
> indeed use Python lists which are arrays, you would
> maybe get a much slower sweep_trail() and this would
> be seen. But currently its not seen. It happens that 1000000
> of elements are sweeped, if you would do this with copy
> inside a Python list which are arrays, it would get much
> more expensive, and the extremly cheap Prolog garbage
> collection, as it stands now, wouldn't be that cheap anymore.
>
> You can try yourself. My sweep_trail() needs frequent resize,
> which would be O(n) each, so that sweep_trail becomes O(n^2).
> Which the current implementation its only O(n).
>

How about, instead: Use a Python list, and instead of removing
individual items one by one, filter out the ones you don't want, using
a list comprehension? That would be O(n) to completely remove all the
ones you don't want, instead of O(n) for each individual removal.

Also, have you actually benchmarked a version that uses Python's
lists, or are you simply assuming that the removals will be slow?
Implementing your own singly-linked list was clearly suboptimal, but
have you tried writing simpler code and seeing if it is also faster?

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 2021-09-20 05:08:55 -0700, Mostowski Collapse wrote:
> You have to copy C1,..,Cm one position down. On the other
> hand, when scanning the single list, removing the
> element is just pointer swizzling.

That's the problem. You think that pointer swizzling is fast because you
are used to languages where this is compiled into a few machine
instructions. (Even there it depends a lot on locality: A random memory
access can take hundreds of clock cycles.)

In Python something like ?a.next = b.next? normally does a dict lookup
on both objects, plus it also has to check for the existence of special
methods on both objects. You can probably copy quite a few contiguous
bytes while this is happening.

Here is a simple demo:

------------------------------------------------------------------------
#!/usr/bin/python3

import time

n = 10000

def build_linked_list(n):
head = [None, None]
tail = head
for i in range(n):
tail[1] = [i, None]
tail = tail[1]
return head

def empty_list(ls):
while ls[1]:
ls[1] = ls[1][1]

t0 = time.monotonic()
ll = build_linked_list(n)
t1 = time.monotonic()
empty_list(ll)
t2 = time.monotonic()
print(n, t1 - t0, t2 - t1)
------------------------------------------------------------------------
#!/usr/bin/python3

import time

n = 10000

def build_list(n):
return list(range(n))

def empty_list(ls):
while ls:
ls.pop(0)

t0 = time.monotonic()
ll = build_list(n)
t1 = time.monotonic()
empty_list(ll)
t2 = time.monotonic()
print(n, t1 - t0, t2 - t1)
------------------------------------------------------------------------

Both scripts create a list of n integers, then repeatedly pop off the
first element until the list is empty. The first uses a linked list
(where each element is a Python list - I guess this is faster than an
object although I haven't timed it) the second a python list. Popping
off the first element is the worst case for a python list since it is
implemented as an array of pointers and n-1 pointers have to be copied
for each operation and the best case for a linked list.

Still, on my laptop with CPython 3.8, the builtin list is faster for
lists of less than 100 elements, they are about on par between 100 and
1000 elements and only for longer lists does the linked list become
faster.

With Pypy 6.0 (yes, I know it's old), the linked list seems to be always
a bit faster although the difference is very small for lists shorter
than 100 elements - GraalVM is probably closer to PyPy than to CPython
in its performance characteristics, but the point is that the dynamic
nature of Python makes some seemingly cheap operations much more costly
than one would expect - a good optimizing compiler may be able to detect
that all that is not needed in a specific case and produce
straightforward code. But unlike for JavaScript (which has a very
similar problem) no company has spent millions of dollars on an
optimizing JIT compiler for Python yet.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"