Mailing List Archive

PEP 638: Syntactic macros
Hi everyone,

I've submitted my PEP on syntactic macros as PEP 638.
https://www.python.org/dev/peps/pep-0638/

All comments and suggestions are welcome.

Cheers,
Mark
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
I like this, but IMHO adding a character at the end of the macro name
(the exclamation mark), lowers readability.
I'd prefer a character at the start of the macro, a character that is
not used as an unary operator.

On Sat, 26 Sep 2020 at 14:11, Mark Shannon <mark@hotpy.org> wrote:
>
> Hi everyone,
>
> I've submitted my PEP on syntactic macros as PEP 638.
> https://www.python.org/dev/peps/pep-0638/
>
> All comments and suggestions are welcome.
>
> Cheers,
> Mark
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WXR5RKPOIWWA32QIMYFD7ECQ4OBP4JXN/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
On Mon., 28 Sep. 2020, 1:22 am Marco Sulla, <Marco.Sulla.Python@gmail.com>
wrote:

> I like this, but IMHO adding a character at the end of the macro name
> (the exclamation mark), lowers readability.
> I'd prefer a character at the start of the macro, a character that is
> not used as an unary operator.
>

For usage, whether something is a macro or not should either be irrelevant
(when they're used as a more powerful function call or decorator), or else
entirely obvious from the way you use it (when they're defining a new
pseudo-statement), so it doesn't make sense to emphasize the syntactic
marker too much. Putting the marker gives the compiler the info and reader
the info they need without being too obtrusive (and also matches the way
Rust macros are used). Using "!" as a prefix operator is also effectively
already claimed by IPython for shell command execution, so I'd be surprised
if we ever used that spelling for anything else.

For the PEP itself, I'd like to see the bijection macro presented in the
abstract to give context for the specification section. While I've had my
doubts about macros in the past due to the inevitable evolution of
different "dialects" of Python, I think we already hit that point years ago
through sophisticated use of metaclasses and other features (Django code
looks very different from NumPy code, for example). Judicious use of macros
should offer opportunities to make domain specific dialects *easier* for
new users to pick up, rather than harder.

For the specification section, my main question/suggestion would be around
sibling macros: how about calling those "decorator macros" and making the
invocation syntax "@name!" rather than the bare "name!"?

Cheers,
Nick.






>
Re: PEP 638: Syntactic macros [ In reply to ]
On Mon, 28 Sep 2020 at 13:56, Nick Coghlan <ncoghlan@gmail.com> wrote:
> For usage, whether something is a macro or not should either be irrelevant (when they're used as a more powerful function call or
> decorator), or else entirely obvious from the way you use it (when they're defining a new pseudo-statement), so it doesn't make sense to
> emphasize the syntactic marker too much.

IMHO `import!` can be easily confused with `import`, even with a
monospace font. An alternative is that macro must have a name
different from keywords (so you will have macro_import!, for example),
even if I'd prefer my first proposal (maybe something like %import).

> Putting the marker gives the compiler the info and reader the info they need without being too obtrusive (and also matches the way Rust
> macros are used). Using "!" as a prefix operator is also effectively already claimed by IPython for shell command execution, so I'd be
> surprised if we ever used that spelling for anything else.

Python uses None instead of Null and try-except instead of try-catch :-)

PS: I agree with the rest of the post.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/H5N5YE5JYSZZTR73CJLJNUTWRDPRYN2R/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
On Mon, Sep 28, 2020 at 7:23 AM Marco Sulla <Marco.Sulla.Python@gmail.com>
wrote:

> On Mon, 28 Sep 2020 at 13:56, Nick Coghlan <ncoghlan@gmail.com> wrote:
> > For usage, whether something is a macro or not should either be
> irrelevant (when they're used as a more powerful function call or
> > decorator), or else entirely obvious from the way you use it (when
> they're defining a new pseudo-statement), so it doesn't make sense to
> > emphasize the syntactic marker too much.
>
> IMHO `import!` can be easily confused with `import`, even with a
> monospace font. An alternative is that macro must have a name
> different from keywords (so you will have macro_import!, for example),
> even if I'd prefer my first proposal (maybe something like %import).
>

I think the whole point is that import! won't be obvious unless you're
looking for it, and those macros won't be obvious unless you're looking for
them. They're meant to blend in rather than stand out.
Re: PEP 638: Syntactic macros [ In reply to ]
Hi,

On 29/09/2020 1:43 am, Emily Bowman wrote:
> On Mon, Sep 28, 2020 at 7:23 AM Marco Sulla
> <Marco.Sulla.Python@gmail.com <mailto:Marco.Sulla.Python@gmail.com>> wrote:
>
> On Mon, 28 Sep 2020 at 13:56, Nick Coghlan <ncoghlan@gmail.com
> <mailto:ncoghlan@gmail.com>> wrote:
> > For usage, whether something is a macro or not should either be
> irrelevant (when they're used as a more powerful function call or
> > decorator), or else entirely obvious from the way you use it
> (when they're defining a new pseudo-statement), so it doesn't make
> sense to
> > emphasize the syntactic marker too much.
>
> IMHO `import!` can be easily confused with `import`, even with a
> monospace font. An alternative is that macro must have a name
> different from keywords (so you will have macro_import!, for example),
> even if I'd prefer my first proposal (maybe something like %import).
>
>
> I think the whole point is that import! won't be obvious unless you're
> looking for it, and those macros won't be obvious unless you're looking
> for them. They're meant to blend in rather than stand out.


My assumption is that almost everyone uses an editor that has syntax
highlighting. Macros will stand out just as much as the users of those
editors think they should.

Personally, I think they should be fairly prominent.
But don't let me stop configuring your editor as you choose :)

Cheers,
Mark.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HRRT34FXZ52I3YUNBLL3FA4NOTO3Q2YB/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
My 2c as a Python user (mostly) and someone who dabbled in ES2020:

The shouting syntax! does not sit well with me.
The $hygenic is also cumbersome.

To contrast, babel macros:
* looks like regular code, without special syntax: existing tooling
works, less mental strain
* have access to call site environment, so not strictly hygienic(?):
allow for greater expressive power

I these the two points above really helped adopt babel macros in the
js community and should, at the very least be seriously considered by
the py community.

Cheers,
d.

On Sat, 26 Sep 2020 at 21:16, Mark Shannon <mark@hotpy.org> wrote:
>
> Hi everyone,
>
> I've submitted my PEP on syntactic macros as PEP 638.
> https://www.python.org/dev/peps/pep-0638/
>
> All comments and suggestions are welcome.
>
> Cheers,
> Mark
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VEC7VWY5TJJGBXWFQUX3XO43SQAZ7FMR/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
Dima,

Do you have a link to "babel macros"? Searching for that brought up several
different things; not being a frequent JS user I don't know how to filter
these.

--Guido

On Wed, Oct 14, 2020 at 11:55 PM Dima Tisnek <dimaqq@gmail.com> wrote:

> My 2c as a Python user (mostly) and someone who dabbled in ES2020:
>
> The shouting syntax! does not sit well with me.
> The $hygenic is also cumbersome.
>
> To contrast, babel macros:
> * looks like regular code, without special syntax: existing tooling
> works, less mental strain
> * have access to call site environment, so not strictly hygienic(?):
> allow for greater expressive power
>
> I these the two points above really helped adopt babel macros in the
> js community and should, at the very least be seriously considered by
> the py community.
>
> Cheers,
> d.
>
> On Sat, 26 Sep 2020 at 21:16, Mark Shannon <mark@hotpy.org> wrote:
> >
> > Hi everyone,
> >
> > I've submitted my PEP on syntactic macros as PEP 638.
> > https://www.python.org/dev/peps/pep-0638/
> >
> > All comments and suggestions are welcome.
> >
> > Cheers,
> > Mark
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VEC7VWY5TJJGBXWFQUX3XO43SQAZ7FMR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Re: PEP 638: Syntactic macros [ In reply to ]
On Sat, Sep 26, 2020 at 5:11 AM Mark Shannon <mark@hotpy.org> wrote:

> Hi everyone,
>
> I've submitted my PEP on syntactic macros as PEP 638.
> https://www.python.org/dev/peps/pep-0638/
>

Speaking as a former C developer, why do "We need to let the community
develop their own extensions"? What's insufficient about Python's current
extensibility?

The complexity of a language varies with the square of its feature count,
and adding macros intended for creation of domain-specific language
features balloons the feature count. Granted, they don't much increase the
complexity of CPython, the language implementation, but the de facto
language then becomes more than CPython - potentially a lot more.

That is, making it easier to extend python means a proliferation of
extensions with little thought given to their long term viability or
cross-domain compatibility.

It's kind of like zsh vs. bash. zsh has a smaller implementation, but a
larger _language_. For this reason, I'm not terribly interested in zsh,
but I like bash. On the other hand, bash has seen a proliferation of
unnecessary extensions in recent years, so I may jump ship to something
else someday - something with a smaller language, that isn't afraid of
fork+exec.

I've used m4 before as a macro system for Python+Cython, but I would never
consider suggesting that m4 should become part of Python itself.

IMO Perl is dying because of its exuberant design. One of the most
important things a language designer has to do is say "no" sometimes.
Re: PEP 638: Syntactic macros [ In reply to ]
On Fri, Oct 16, 2020, at 18:59, Dan Stromberg wrote:
> The complexity of a language varies with the square of its feature
> count,

Says who? I'd assume the orthogonality and regularity of features matters at least as much if not more than the number of features, and providing a system like this would guarantee some degree of regularity.

Is there some notion of "complexity of a language" [.other than by trivially *defining* it as the square of the number of features] for which this can be shown to be true?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JVJDRCNNSDMDIIFWJ65MBQVTWC375SJW/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
On Fri, 16 Oct 2020 at 23:22, Guido van Rossum <guido@python.org> wrote:
>
> Dima,
>
> Do you have a link to "babel macros"? Searching for that brought up several different things; not being a frequent JS user I don't know how to filter these.

These links should help:

https://babeljs.io/blog/2017/09/11/zero-config-with-babel-macros
https://github.com/kentcdodds/babel-plugin-macros
https://github.com/jgierer12/awesome-babel-macros

That's a general intro, the code repo for the macro plugin, and a repo
that lists implemented macros.

Martin
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HJR6UF2XPOC6UUY6QRLKYCYZOWK2BYDD/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
Martin sent good links, I'll just add a practical example:


Without macros, styled-components works like this:

import styled from "styled-components";

const Label = styled.div`
color: red;
`;

(that's a js template literal, next level after f-strings, here
without any arguments)
When e.g. <Label>Hi</Label> is rendered, the result is <div
class="4242">Hi</div> where class name is a hash of the inline css,
and css is injected into the document head.

This is great for many reasons, except developer often ends up with
the below on the page (dev tools, though page source is possible too)
where it's quite hard to mentally work back which styled.div was is
that generated hash 7676...
<div class="1231">
<div class="7676">
<div class="9898">
...


Here's the same with babel macros:

import styled from "styled-components/macro";

const Label = styled.div`
color: red
`;

When <Label>Hi</Label> is rendered, the result is <div
class="filename_Label_4242">Hi</div> where class name encodes hash of
the inline css, but also the file name where it was "defined" as well
as the name of the variable to which it was assigned, changing the div
tree e.g. to:
<div class="bezel_Head_1231">
<div class="bezel_Container_7676">
<div class="menu_Header_9898">
...


How it works:
* babel macros work on AST, not text
* babel macro has access to entire module AST, and can thus infer and
modify the module:
* here, module name is recorded, and if the result of macro call is
assigned, then target variable name is recorded
* I've seen automatic import injection (useful to pass glocal state
in e.g. localisation libraries)
* almost anything is possible

There are downsides:
1. the macro code is more involved, e.g. see
https://github.com/styled-components/styled-components/blob/master/packages/styled-components/src/macro/index.js
2. multiple macros can and do collide on occasion (usually when
written naively), which somewhat limits composability by end users

On Sat, 17 Oct 2020 at 07:18, Guido van Rossum <guido@python.org> wrote:
>
> Dima,
>
> Do you have a link to "babel macros"? Searching for that brought up several different things; not being a frequent JS user I don't know how to filter these.
>
> --Guido
>
> On Wed, Oct 14, 2020 at 11:55 PM Dima Tisnek <dimaqq@gmail.com> wrote:
>>
>> My 2c as a Python user (mostly) and someone who dabbled in ES2020:
>>
>> The shouting syntax! does not sit well with me.
>> The $hygenic is also cumbersome.
>>
>> To contrast, babel macros:
>> * looks like regular code, without special syntax: existing tooling
>> works, less mental strain
>> * have access to call site environment, so not strictly hygienic(?):
>> allow for greater expressive power
>>
>> I these the two points above really helped adopt babel macros in the
>> js community and should, at the very least be seriously considered by
>> the py community.
>>
>> Cheers,
>> d.
>>
>> On Sat, 26 Sep 2020 at 21:16, Mark Shannon <mark@hotpy.org> wrote:
>> >
>> > Hi everyone,
>> >
>> > I've submitted my PEP on syntactic macros as PEP 638.
>> > https://www.python.org/dev/peps/pep-0638/
>> >
>> > All comments and suggestions are welcome.
>> >
>> > Cheers,
>> > Mark
>> > _______________________________________________
>> > Python-Dev mailing list -- python-dev@python.org
>> > To unsubscribe send an email to python-dev-leave@python.org
>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>> > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNRC4SHS3TPZWCTY4SN4QU3TT6V/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VEC7VWY5TJJGBXWFQUX3XO43SQAZ7FMR/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/56CWO3OM52CM6ANOOIPFXWQVGL75C4JK/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
I'm excited about the potential introduction of Lisp-style syntactic macros into Python. ????

It's unclear to me whether PEP 638 ("Syntactic Macros") is still being actively developed, since the last activity I see on it is over a year ago (Sep & Oct 2020), but I thought I'd leave some initial comments I have anyway.

Quoting from the PEP text...

(1)

> Lexical analysis
> ~~~~~~~~~~~~~~~~
>
> Any sequence of identifier characters followed by an exclamation point
> (exclamation mark, UK English) will be tokenized as a ``MACRO_NAME``.

+1 to using the ! character to mark macros explicitly and loudly, since they are very powerful and can completely customize the syntax used inside them.

I like the postfix syntax myself (as proposed) - so `macro_name!` is cool. I wouldn't be a fan of prefix syntax; `!macro_name` looks ugly.

(2)

> Statement form
> ~~~~~~~~~~~~~~
>
> macro_stmt = MACRO_NAME testlist [ "import" NAME ] [ "as" NAME ] [ ":" NEWLINE suite ]

(2.1)

What is `testlist`? It is not defined in this PEP, nor is it defined in the Python Grammar Specification [1].

Also, the `[ "import" NAME ]` and `[ "as" NAME ]` parts appear to be special-purpose syntax only useful for supporting the `from!` and `with!` macros mentioned later in the PEP. It feels odd that this syntax isn't more general-purpose.

Perhaps the grammar could be made more general with something like:

> macro_stmt = MACRO_NAME macro_expr_parameters ( NEWLINE | ":" NEWLINE suite )
> macro_expr_parameters = ( expression | <keyword> )*

That would allow a statement macro to take some number of expressions and keywords as arguments, in addition to a suite.

(2.2) +0 to the idea of prefixing a @ to a MACRO_NAME that is intended to be used as a sibling-macro. That would look like:

> @do_nothing_marker!
> def foo(...):
> ...

(3)

> Expression form
> ~~~~~~~~~~~~~~~
>
> macro_expr = MACRO_NAME "(" testlist ")"

Again, what is `testlist`? Perhaps you're looking for something more like:

> macro_expr = MACRO_NAME "(" macro_parameters ")"
> macro_parameters = ( expression ( "," expression )* ","? )?

(4)

> Resolving ambiguity
> ~~~~~~~~~~~~~~~~~~~
>
> The statement form of a macro takes precedence, so that the code
> ``macro_name!(x)`` will be parsed as a macro statement,
> not as an expression statement containing a macro expression.

It seems to me that if you were to define an expression macro - like `cast!(T, expression)` - that users would expect to be able to call such an expression macro on a line by itself. The rule stated here suggests that the parser would get confused by such a call, incorrectly treating the call as a call of a macro *statement* rather than a macro *expression*.

(5)

> Compilation
> ~~~~~~~~~~~
>
> For macros with multiple names, [...]

Nit: "Multiple names"? I think you meant macros with "additional names", as described later in the PEP.

(6)

> Defining macro processors
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
> A macro processor is defined by a four-tuple, consisting of
> ``(func, kind, version, additional_names)``:
>
> * [...]
> * ``additional_names`` are the names of the additional parts of the macro, and must be a tuple of strings.

Seems to me that "additional_names" might make more sense to call "additional clause names" or just "clause names". That would be consistent with the following "anatomy of a statement-macro" sketch:

try_!: # begin try_ statement; is try_ clause header
... # is try_ clause body
finally_!: # is finally_ clause header
... # is finally_ clause body; end of try_ statement

(7)

> Hygiene and debugging
> ~~~~~~~~~~~~~~~~~~~~~
>
> [...] No rules for naming will be enforced, but to ensure hygiene and help debugging, the following naming scheme is recommended: [...]

This appears to imply that it is up to the macro processor author to take special care that their macro is hygenic. That is, *unhygenic* macros are the default. Wouldn't it be safer to define a system where *hygenic* macros would be the default instead? Seems like if we can head off the introduction of Command Injection by default at the design level then we should do so.

(8)

> Examples
> ''''''''

Not a single example in this section defines an actual macro processor function. Recommend implementing at least a toy macro end-to-end (including the macro processor function) for full clarity, for each type of macro (i.e. statements, sibling, and expression).

(9)

> Backwards Compatibility
> =======================
>
> This PEP is fully backwards compatible.

Nit: In the §"Implementation" section below, it is mentioned that nodes in the`_ast` module would be made immutable. That sounds like a backward-incompatible change to me.

(10)

> Currently, all AST nodes are allocated using an arena allocator.
> Changing to use the standard allocator might slow compilation down a little,
> but has advantages in terms of maintenance, as much code can be deleted.

I presume the arena allocator was introduced in the first place for a reason. Perhaps to improve performance? By removing the arena allocator are there potential downsides other than a performance regression?

(11)

> Reference Implementation
> ''''''''''''''''''''''''
>
> None as yet.

Seems like you could get a prototype off the ground by implementing an initial version as a fake Python source file text encoding.

Then you could put something like `# coding=macros` at the top of a source file to have it preprocessed by a prototype macro system.

(12)

Thanks for taking the time to read my comments.

--
David Foster | Seattle, WA, USA
Contributor to mypy, TypedDict, and Python's type system


[1]: https://docs.python.org/3/reference/grammar.html
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AW6KOQ2V3TS6TQUHS4VGFIKSR3SOLJCJ/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
It looks like this hasn't gone anywhere in the past few years, which is a shame. Syntactic macros are one of the 2 or 3 "Killer features" that pushed me out of Python and into Julia (along with JITting inferred types and multiple dispatch). Math+data science code written in Julia is a lot more readable because of this.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPCXZJYPNNT6XZ6EQ35OQE4SG2QBAZRT/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
I’ve used lisp and scheme and one reason why you wouldn’t want a syntactic macro is because there should be one and only one way to do a task.

Sure we have deviated that in the ecosystem but allowing syntactic macros can have the side effect of many programs or projects to have multiple ways to do something because they would have the feature .

I think that this would be better as a library in my opinion.

Sent from my iPhone

> On Jan 29, 2023, at 2:20 PM, cdp49@cam.ac.uk wrote:
>
> ?It looks like this hasn't gone anywhere in the past few years, which is a shame. Syntactic macros are one of the 2 or 3 "Killer features" that pushed me out of Python and into Julia (along with JITting inferred types and multiple dispatch). Math+data science code written in Julia is a lot more readable because of this.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPCXZJYPNNT6XZ6EQ35OQE4SG2QBAZRT/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TKMQKXLQYFQ5CWS76SW7UXFEDZOIXJ2E/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
Joshua Herman writes:

> I think that this would be better as a library in my opinion.

There's a third party package called MacroPy that provides macros,
although I haven't heard anything about it in a couple of years.

I seem to recall that it's a preprocessor that hooks into the import
system.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VRY55MMVRVZGHJRPPWWHKHLPPI2HLHZR/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
Unfortunately, it's no longer being maintained.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UEPTBUIK67NLNE4JBNN6JNJON3BRGSUK/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
I think that's exactly the problem with a lack of Python macros. The full quote, of course, goes: "There should be one-- and preferably only one --*obvious* way to do it."

Often, there's a mathematical notation for something, and *this* is the only obvious way to write anything out. But this doesn't work if you force every package to adopt the same syntax. For example, if you'd like a package to work with probabilities, it's very reasonable to want to write `x ~ Normal(0, 1)` to say x follows a normal distribution. This is the only syntax I consider natural for this problem; but packages can't do that, since `~` already has a meaning outside of probability.

Not to mention, DSLs are forced to adopt all kinds of weird syntax when the behavior of base Python doesn't align with what the DSL needs to do. Obviously, the only way to write out a `for` loop should be to use the `for` keyword. This doesn't work in JAX. If you want to use a `for` loop in JAX, you have to use the `lax.fori_loop` function, or else `for` will end up being unrolled, because of various requirements of the JAX compiler. Having to use `lax.fori_loop` is, to put it mildly, *incredibly* unpythonic.

This really is the biggest reason I switched to Julia: Python math is unpythonic. I don't want to be forced to learn lots of weird little functions like `np.matmul(x1, x2)` when there's already one obvious syntax I'm very familiar with: `x1 * x2`.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6QM4GMH5FDX2H5OZHCE33EOJHCV3TI2R/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
cdp49@cam.ac.uk writes:

> I think that's exactly the problem with a lack of Python
> macros. The full quote, of course, goes: "There should be one-- and
> preferably only one --*obvious* way to do it."

You understand that the Zen is humorous? Most of the Zen, if taken
universally and seriously, advocates the impossible. And as a whole,
it's definitely impossible even in the limited scope it's intended to
apply to -- it's internally inconsistent.

> Often, there's a mathematical notation for something, and *this* is
> the only obvious way to write anything out.

But that's not the way Python looks at it, you see. First, "a"
mathematical notation doesn't exclude multiple notations, and for most
mathematical concepts there are indeed multiple common notations (eg,
for multiplication, juxtaposition of the factors, *, ?, and × are all
in common use depending on the kind of multiplication). I'm pretty
sure Tim Peters was well aware of such. Frequently the most commonly
used expressions are really ugly (at least in my experience in
mathematical applications to game theory ;-). Now, you can recover
your position from that issue by appealing to other more or less Zen
desiderata (readability counts, for example), but they're not quite as
strong arguments here.

The second objection is more serious: the Zen is intended to be
restricted to Python. "There should be one-- and preferably only one
--obvious way to do it [*in Python*]." Guido (and the other OG core
devs) wanted consistent and obvious ways to do it *in Python*. The
consistent and obvious way to write "X ~ N(0,1)" (oops, not so obvious
after all!) in Python is "X = Normal(0,1)", where presumably the
Normal class provides facilities such as CDF and PDF as well as the
PRNG of random.normalvariate.

> Not to mention, DSLs are forced to adopt all kinds of weird syntax

This is more or less intentional, though, as is the restriction to a
predefined set of operator symbols (you can't define ? as an operator
symbol when you need two kinds of multiplication for example). Python
has consistently refused to be turned into a platform for DSLs for
almost 3 decades. Ruby and Lisps are better for that.

You don't have to like that (quite a few people don't, of course
that's why MacroPy was written), but it's really not un-Pythonic.
There is method to this madness: Python aims for readability and
flexibility for the community of Python programmers who might
encounter the code, rather than catering to authors and their domain
specialist community. The fact that Python adoption is still growing
should tell you something about the preferences and needs of the
general Python community.

> I don't want to be forced to learn lots of weird little functions
> like `np.matmul(x1, x2)` when there's already one obvious syntax
> I'm very familiar with: `x1 * x2`.

I don't recall ever writing matrix multiplication that way in
mathematics though. That's universally written as juxtaposition in my
experience. And the obvious way to write it in Python (and np) has
been "x1 @ x2" for some years now. In np, "*" means element-wise
multiplication, I believe.

Perhaps some BDFL will arise to merge the benefits of Python with
those of Julia, but for the near term we're all going to have to
choose one or the other.

Steve

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/D2427HD63JN5MTBFQL2SZFBJU3UEXE3L/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
On Thu, Feb 2, 2023 at 8:34 AM Stephen J. Turnbull <
stephenjturnbull@gmail.com> wrote:

> cdp49@cam.ac.uk writes:
>
> > I don't want to be forced to learn lots of weird little functions
> > like `np.matmul(x1, x2)` when there's already one obvious syntax
> > I'm very familiar with: `x1 * x2`.
>
> I don't recall ever writing matrix multiplication that way in
> mathematics though. That's universally written as juxtaposition in my
> experience. And the obvious way to write it in Python (and np) has
> been "x1 @ x2" for some years now. In np, "*" means element-wise
> multiplication, I believe.
>

My 2 cents as a former mathematician:

Mathematicians have come up with hundreds of symbols to express the variety
of structures they are dealing with.

Just to list a few:

https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode
https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols

So +, *, -, /, @ and ** are a good start to express the most common
mathematical structures (groups, rings, vector spaces), and <, >, <=, >=,
==, != for ordered sets, but that's it.

It's great that Python supports overloading these operations. That's
something that drew me to Python when I was still working in computational
number theory, 25 years ago.

But that's probably not enough for some people. A way to add new symbols
(including non-ascii Unicode characters) and operations could have some
value, but also the drawback of tremendous additional complexity, specially
if done in the most generic way.

NB: on a very basic level, I remember trying, a few years ago, to use the
Unicode "empty set" symbol as a synonym for set(), and it didn't end well,
for several reasons, including the fact that Python didn't like it as a
variable name.

S.

--
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Co-Founder & Co-Chairman, National Council for Free & Open Source Software
(CNLL) - http://cnll.fr/
Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel
Libre (APELL) - https://www.apell.info/
Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) -
https://www.euclidia.eu/

"I wish you were as accurate, & as much to be relied on, as I am myself" -
Lady Ada Lovelace
Re: PEP 638: Syntactic macros [ In reply to ]
On Thu, Feb 2, 2023 at 12:46 PM Stéfane Fermigier <sf@fermigier.com> wrote:

> https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode
> https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols
> NB: on a very basic level, I remember trying, a few years ago, to use the
> Unicode "empty set" symbol as a synonym for set(), and it didn't end well,
> for several reasons, including the fact that Python didn't like it as a
> variable name.
>

I use the `vim` conceal plugin to make some of these appear on screen while
I'm editing. So, for example, when I type `set()` I see `?`. When I type
`all(...)` I see `?(...)`.

Much to the chagrin of Moshe Zadka, when I type `None` I see `?` ...
because I argue that ?_0 really should be considered an inaccessible
cardinal :-).

However, it never causes me problems because the files on disk are just
plain old ASCII (for the most part), and the special symbols are just what
my screen shows, not overloads to underlying operators.

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
Re: PEP 638: Syntactic macros [ In reply to ]
Stéfane Fermigier writes:

> NB: on a very basic level, I remember trying, a few years ago, to use the
> Unicode "empty set" symbol as a synonym for set(), and it didn't end well,
> for several reasons, including the fact that Python didn't like it as a
> variable name.

I know about the issue that '?' is not valid in identifiers, but I'm
curious about these other "several reasons".



_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7OHL242ZWVOYTBKTZOC72PQK5V4BKX2I/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: PEP 638: Syntactic macros [ In reply to ]
"


On Fri, Feb 3, 2023 at 12:28 PM Stephen J. Turnbull <
stephenjturnbull@gmail.com> wrote:

> Stéfane Fermigier writes:
>
> > NB: on a very basic level, I remember trying, a few years ago, to use
> the
> > Unicode "empty set" symbol as a synonym for set(), and it didn't end
> well,
> > for several reasons, including the fact that Python didn't like it as a
> > variable name.
>
> I know about the issue that '?' is not valid in identifiers, but I'm
> curious about these other "several reasons".
>

IIRC (this was a few years back): '?' (aka EMPTY SET) was not accepted by
Python, so I used instead ''Ø" (aka "LATIN CAPITAL LETTER O WITH STROKE").

Python was OK with it, but some of the tools I use (one of flake8, isort,
black... or maybe one of my IDE) barfed on it.

S.

--
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Co-Founder & Co-Chairman, National Council for Free & Open Source Software
(CNLL) - http://cnll.fr/
Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel
Libre (APELL) - https://www.apell.info/
Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) -
https://www.euclidia.eu/

"I wish you were as accurate, & as much to be relied on, as I am myself" -
Lady Ada Lovelace
Re: PEP 638: Syntactic macros [ In reply to ]
El vie, 3 feb 2023 a las 8:01, Stéfane Fermigier (<sf@fermigier.com>)
escribió:

> "
>
>
> On Fri, Feb 3, 2023 at 12:28 PM Stephen J. Turnbull <
> stephenjturnbull@gmail.com> wrote:
>
>> Stéfane Fermigier writes:
>>
>> > NB: on a very basic level, I remember trying, a few years ago, to use
>> the
>> > Unicode "empty set" symbol as a synonym for set(), and it didn't end
>> well,
>> > for several reasons, including the fact that Python didn't like it as a
>> > variable name.
>>
>> I know about the issue that '?' is not valid in identifiers, but I'm
>> curious about these other "several reasons".
>>
>
> IIRC (this was a few years back): '?' (aka EMPTY SET) was not accepted by
> Python, so I used instead ''Ø" (aka "LATIN CAPITAL LETTER O WITH STROKE").
>
> Python was OK with it, but some of the tools I use (one of flake8, isort,
> black... or maybe one of my IDE) barfed on it.
>

Black is fine with it; we've had the empty set symbol in our own source
code since the beginning (
https://github.com/psf/black/blob/b0d1fba7ac3be53c71fb0d3211d911e629f8aecb/src/black/linegen.py#L455
now). I believe we've had someone ask that we remove it because it was
breaking some tool that was processing the Black source code, but ?ukasz
wasn't having it.


>
> S.
>
> --
> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier
> - http://linkedin.com/in/sfermigier
> Founder & CEO, Abilian - Enterprise Social Software -
> http://www.abilian.com/
> Co-Founder & Co-Chairman, National Council for Free & Open Source Software
> (CNLL) - http://cnll.fr/
> Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel
> Libre (APELL) - https://www.apell.info/
> Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) -
> https://www.euclidia.eu/
>
> "I wish you were as accurate, & as much to be relied on, as I am myself"
> - Lady Ada Lovelace
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/H2UALG4YAA6IBMZQMWEL2QIOPDPK4RTG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: PEP 638: Syntactic macros [ In reply to ]
> Python has consistently refused to be turned into a platform for DSLs for almost 3 decades.

I think SymPy, PyMC, Pyomo, Pyro, and many more packages would all be very surprised to hear they're no longer welcome in Python. Still, it seems like it would be quite hard to kick them out, and would probably make the scientific programming community pretty angry. If you don't like having DSLs in Python, I think you're trying to close the barn door after the horse has bolted; you'd have to go back in time to the creation of NumPy.

Syntactic macros aren't necessary for DSLs; it just makes them better. Without syntactic macros, DSLs are forced to use clunky, complicated, and error-prone string manipulation, rather than cleaner syntactic transformations. For instance, here's NumPy's einsum, effectively behaving like a string macro:
```
X = np.einsum('ij,jk->ik', A, B, optimize='optimal')
```

And now here's the same thing in Julia:
```
@einsum X[i, k] := A[i, j] * B[j, k]
```

Which is more readable? Which is more Pythonic?

It's not that Python doesn't have DSLs (NumPy is effectively a DSL for linear algebra). It's just that their syntax is sufficiently obscure that it's not at all clear that's what they're doing.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RWSSY4KZLQYXHFF34AR544C44NZ6K7XE/
Code of Conduct: http://python.org/psf/codeofconduct/

1 2  View All