Mailing List Archive

On Sun, 27 Mar 2022 at 00:52, Ethan Furman <ethan@stoneleaf.us> wrote:
>
> [.apologies for the late post, just found this in my drafts folder]
>
> On 2/7/22 12:49 AM, Stéfane Fermigier wrote:
>
> > 3. Overall, I think the days where "battery included" was a positive argument are over
>
> I strongly disagree. Being able to download something and immediately get something to work and see results is hugely
> rewarding; on the other hand, having to research, find, compare & contrast available third-party modules (especially for
> new-comers) can be extremely discouraging.

100% agreed.

I think people massively underestimate how frustrating it can be to
have to hunt out some piece of functionality that *doesn't* come with
the base download. Or worse, to find 5 different implementations, and
have no idea which one you "should" use.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2BIMFGNREXPOAFKGNVC7FZMT5LNVNVAY/
Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Mar 27, 2022 at 3:08 AM Paul Moore <p.f.moore@gmail.com> wrote:

> > > 3. Overall, I think the days where "battery included" was a positive
> argument are over
> >
> > I strongly disagree. Being able to download something and immediately
> get something to work and see results is hugely
> > rewarding; on the other hand, having to research, find, compare &
> contrast available third-party modules (especially for
> > new-comers) can be extremely discouraging.
>

exactly - let's say someone needs to write some JSON for the first time.

With the json package included, all they need to do is `import json`. If
that wasn't there, they's look in PyPi for a JSON implementation, and find
an absolutely astonishing number of options. I just did a search for "JSON"
on PYPI, and it's HUGE -- most of them are for specialized JSON-using
protocols of some sort. I was actually really surprised that couple I know
about of the top of my head (ujson, orjson) are actually hard to find.

"You can just pip install it" is really not a good solution.

In fact, this is an example, I think, of where we should put some effort
into making the included batteries better -- it's great to have a JSON lib
built in, but it's too bad that it's not best-of-bread by pretty much any
definition (speed, memory performance, flexibility) -- there are quite a
few feature requests open for it -- it would be nice to
actually implement some of those. (but yes, that's a lot of work that
someone(s) would have to do)

Back to the topic at hand, rather than remove urllib, maybe it could be
made better -- an as-easy-to-use-as-requests package in the stdlib would be
really great.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

> Back to the topic at hand, rather than remove urllib, maybe it could
> be made better -- an as-easy-to-use-as-requests package in the stdlib
> would be really great.

Agreed. I think one thing that's easy to forget is that Python is useful
not just for very large applications but also for your standard tiny
one-off scripts (i.e., cron jobs and the like). The big standard library
is indispensible for writing clean code in those somewhat inherently
hacky situations. Note that this isn't really unique to Python. If we
look across the landscape for scripting lanugages useful in the same
areas, Bash stands out as being a bit of an odd language on its own, but
one that pairs well with all your traditional UNIX tools. These tools
make for a decently capable 'standard library', and Bash a surprisingly
versatile tool. My point obviously isn't that Python is or should be
Bash, but that Python gets a tangible benefit from its batteries in much
the same way as Bash gets a benefit from sed, awk, &c. Namely, these
languages are both made so much more /versatile/ by their respective
included tools. Being able to count on some common problems *just being
solved already* is a highly appreciated language feature, and something
I'd love to see stable and supported going forward.

So, let's not remove urllib. The fact that we're having this discussion
indicates that the scope of urllib is somewhat undefined, so perhaps it
would benefit from a firm and semi-formal (re)definition of scope
(hopefully in a manner motivated by real code). After that, it would
probably benefit from patching up all the rough spots in the
implementation side of things. To keep the metaphor going, let's
recharge our dead batteries.

--
lincoln auster
they/them


As an aside, I've opened PR#30520 on the CPython repo, which makes
urllib.urlparse less dependent on the hard-coded list of schemes it uses
to determine parsing behavior. In the interest of recharging batteries,
would anyone be willing to review that? @orsenthil volunteered a while
back, but with all of my first-PR issues, it's sort of stalled.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FR4AGGHFUQ3XUHUG3Q3QF7PAO6HJ3CJI/
Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, 27 Mar 2022 at 17:11, Christopher Barker <pythonchb@gmail.com> wrote:
>
> With the json package included, all they need to do is `import json`. If that wasn't there, they's look in PyPi for a JSON implementation, and find an absolutely astonishing number of options. I just did a search for "JSON"
> on PYPI, and it's HUGE -- most of them are for specialized JSON-using protocols of some sort. I was actually really surprised that couple I know about of the top of my head (ujson, orjson) are actually hard to find.
>
> "You can just pip install it" is really not a good solution.
>
> In fact, this is an example, I think, of where we should put some effort into making the included batteries better -- it's great to have a JSON lib built in, but it's too bad that it's not best-of-bread by pretty much any definition (speed, memory performance, flexibility) -- there are quite a few feature requests open for it -- it would be nice to actually implement some of those. (but yes, that's a lot of work that someone(s) would have to do)
>
> Back to the topic at hand, rather than remove urllib, maybe it could be made better -- an as-easy-to-use-as-requests package in the stdlib would be really great.

I think that's where the mistake happens, though. Someone who needs
"best of breed" is motivated (and likely knowledgeable enough) to make
informed decisions about what's on PyPI. But someone who just wants to
get the job done probably doesn't - and that's the audience for the
stdlib. A stdlib module needs to be a good, reliable set of basic
functionality that non-experts can use successfully. There can be
better libraries on PyPI, but that doesn't mean the stdlib module is
unnecessary, nor does it mean that the stdlib has to match the PyPI
library feature for feature.

So here, specifically, I'd rather see urlllib be the best urlllib it
can be, and not demand that it turn into requests. Requests is there
if people need/want it (as is httpx, and urllib3, and aiohttp). But
urllib is for people who want to get a file from the web, and *not*
have to deal with dependencies, 3rd party libraries, etc.

The "batteries included" standard library and PyPI complement each
other. Neither is redundant, and neither implies the other is
unnecessary.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/E2FGXTYOEYLC6GSJGMWBQKHOO62LZPHQ/
Code of Conduct: http://python.org/psf/codeofconduct/

Time for a __legacy__ package?

Kind regards,
Steve


On Sun, Mar 27, 2022 at 7:06 PM Paul Moore <p.f.moore@gmail.com> wrote:

> On Sun, 27 Mar 2022 at 17:11, Christopher Barker <pythonchb@gmail.com>
> wrote:
> >
> > With the json package included, all they need to do is `import json`. If
> that wasn't there, they's look in PyPi for a JSON implementation, and find
> an absolutely astonishing number of options. I just did a search for "JSON"
> > on PYPI, and it's HUGE -- most of them are for specialized JSON-using
> protocols of some sort. I was actually really surprised that couple I know
> about of the top of my head (ujson, orjson) are actually hard to find.
> >
> > "You can just pip install it" is really not a good solution.
> >
> > In fact, this is an example, I think, of where we should put some effort
> into making the included batteries better -- it's great to have a JSON lib
> built in, but it's too bad that it's not best-of-bread by pretty much any
> definition (speed, memory performance, flexibility) -- there are quite a
> few feature requests open for it -- it would be nice to actually implement
> some of those. (but yes, that's a lot of work that someone(s) would have to
> do)
> >
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it (as is httpx, and urllib3, and aiohttp). But
> urllib is for people who want to get a file from the web, and *not*
> have to deal with dependencies, 3rd party libraries, etc.
>
> The "batteries included" standard library and PyPI complement each
> other. Neither is redundant, and neither implies the other is
> unnecessary.
>
> Paul
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/E2FGXTYOEYLC6GSJGMWBQKHOO62LZPHQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Sat, Mar 26, 2022 at 5:58 PM Ethan Furman <ethan@stoneleaf.us> wrote:

> [.apologies for the late post, just found this in my drafts folder]
>
> On 2/7/22 12:49 AM, Stéfane Fermigier wrote:
>
> > 3. Overall, I think the days where "battery included" was a positive
> argument are over
>
> I strongly disagree. Being able to download something and immediately get
> something to work and see results is hugely
> rewarding; on the other hand, having to research, find, compare & contrast
> available third-party modules (especially for
> new-comers) can be extremely discouraging.
>

It might make sense to have CPython's release cadence decoupled from the
Standard Library's release cadence. That is, maybe they should be separate
downloads.

On 2022-03-27 19:52, Dan Stromberg wrote:
>
> On Sat, Mar 26, 2022 at 5:58 PM Ethan Furman <ethan@stoneleaf.us
> <mailto:ethan@stoneleaf.us>> wrote:
>
> [.apologies for the late post, just found this in my drafts folder]
>
> On 2/7/22 12:49 AM, Stéfane Fermigier wrote:
>
> > 3. Overall, I think the days where "battery included" was a
> positive argument are over
>
> I strongly disagree.  Being able to download something and
> immediately get something to work and see results is hugely
> rewarding; on the other hand, having to research, find, compare &
> contrast available third-party modules (especially for
> new-comers) can be extremely discouraging.
>
>
> It might make sense to have CPython's release cadence decoupled from the
> Standard Library's release cadence.  That is, maybe they should be
> separate downloads.
>
On the other hand, it's nice to get it all in one go. Perhaps the
installer could gain controls to download optional parts of the standard
library or update those parts that have been installed.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/73DJBBJS7CF5CGSSVRPITQR6NU54LXJV/
Code of Conduct: http://python.org/psf/codeofconduct/

> On the other hand, it's nice to get it all in one go. Perhaps the
> installer could gain controls to download optional parts of the
> standard library or update those parts that have been installed.

I'm not sure the proposed is really a standard library if it's
distributed like that. If a given script cannot rely on the presence of
some given module --- one which may need to be installed as though it
were just another 3rd-party component --- then it's not really all that
standard, is it?

--
Lincoln Auster
they/them
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OZSDKTLIGHSNQY32ZIX3B2ALSB4CARAM/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mar 27, 2022, at 11:52, Dan Stromberg <drsalists@gmail.com> wrote:
>
> It might make sense to have CPython's release cadence decoupled from the Standard Library's release cadence. That is, maybe they should be separate downloads.

While I don’t underestimate the work and complexity, we can do both. I.e. separate the stdlib development cycle from the interpreter (for all but a handful of required packages perhaps). We could still distribute “sumo” releases which include all the batteries, but develop and maintain them outside the cpython repo, and even release them separately on PyPI. It’s *possible* but I don’t know if it’s *practical*.

-Barry

Barry Warsaw writes:

> While I don’t underestimate the work and complexity, we can do
> both. I.e. separate the stdlib development cycle from the
> interpreter (for all but a handful of required packages perhaps).
> We could still distribute “sumo” releases which include all the
> batteries, but develop and maintain them outside the cpython repo,
> and even release them separately on PyPI. It’s *possible* but I
> don’t know if it’s *practical*.

After a couple decades maintaining XEmacs, I'd have to say that one of
the things that made our sumos (you heard that word here first,
folks!) practical was that about 80% of the code was developed in GNU
Emacs. That is, developed in concert with core Emacs. This meant
that dependencies (think "hybrid of hydra, kitsune-yokai, and
ouroboros" :-) evolve together, as they do in Python. This is
probably more important in Emacs because of the big-ball-of-string
nature of big Lisp systems (at least with a relatively undisciplined
dev process), but I'd still say there would be *substantial*
coordination costs to be paid if you split the repos.

Also, the only feature in Emacs Lisp that really changed the way
people code since 1990 (and likely earlier) is lexical scope.
Everything else is merely a new callable for practical purposes, and
almost all of it can be implemented in Lisp either as a function or a
macro (although usually at great expense in performance). But in
Python, every new release makes the Mailman crew want to stop
supporting all previous releases of Python because there's some
feature that can't be emulated that we really love: genexps or async
or walrus operator or .... It was certainly quite costly in XEmacs
trying to most packages usable in 21.4, mostly in frustration about
not being able to use 21.5 features. ;-) But Python has like 4 or 5
generations of stdlib in some form of development or maintenance at
any given time. It's really not attractive to think either about
creating a separate Sumo Dev Core to maintain several generations of
stdlib corresponding to in-maintenance versions of Python, or about a
lowest-common-denominator stdlib.

Until XEmacs split off the stdlib into packages, Emacs didn't have
anything like a package repository, certainly nothing like PyPI. I
don't have experience with the Emacs package repos ELPA and MELPA for,
uh, hysterical raisins, so season the next comment to taste. My
feeling is that both splitting out the stdlib and increasing the rate
at which new packages are added are excessively expensive non-
solutions to the problem that Paul has described, that finding the
right package on PyPI is pretty hard, especially for people new to a
technology. The extra effort that would be needed to achieve either,
as well as the increased on-going maintenance burden on the core team,
would not really resolve the problem, given that there are now whole
large ecosystems like Django and SciPy outside of Python core
development.

Although it's a different kind of effort from package incorporation
and maintenance, to the extent that effort is fungible, I think it
would be better devoted to creating an annotated catalog of PyPI.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZKODGJ3I6XQNX7ISE4EVCI5RVRCTTOA4/
Code of Conduct: http://python.org/psf/codeofconduct/

Barry writes (in part):

> We could still distribute “sumo” releases which include all the
> batteries, but develop and maintain them outside the cpython repo,
> and even release them separately on PyPI. It’s *possible* but I
> don’t know if it’s *practical*.

to which Stephen responds (in part):

> [Emacs really didn't change much in 20 years.] .... [I]n
> Python, every new release makes the Mailman crew want to stop
> supporting all previous releases of Python because there's some
> feature that can't be emulated that we really love: genexps or async
> or walrus operator or ....

It seems to me that what we have is the possibility that, say, package
P migrates to PyPI in concert with the release of Python version X
where maintenance can be picked up by the broader Python community.
Whether or not it would proceed to track ongoing changes to the
language isn't clear, and it's not obvious to me that less effort
would be required other than perhaps by core devs (though some of them
would likely pitch in to maintain packages with which they are
currently involved). If you go with the "discard the batteries"
approach, I think it would at least be worthwhile distributing a
requirements.txt file ("sumo.txt" or "batteries.txt"?) which would
tell someone installing Python how to reclaim the batteries of their
Python youth, and for other Python implementations to track a set of
packages which would make them plausibly compatible with CPython. CI
could still rely on that to provide as much test coverage as current
today (I think). If nothing else, it might alert the core devs to the
potential for breakage of presumably widely used packages by changes
to CPython.

What happens to P when Python version Y grows a new syntactic feature?
Do P's maintainers fork to both continue feature growth as well as
syntactic modernization? If something like batteries.txt is created to
tie a slimmed-down CPython distribution to the batteries it once
contained, would the core group exert any control over unit testing,
documentation, package variations and such?

Just thinking out loud...

Skip
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/76ZTGO2PD6Q5NZOO5JWJRTTR2E2JY2H6/
Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Mar 27, 2022 at 3:08 AM Paul Moore p.f.moore@gmail.com wrote: https://mail.python.org/archives/list/python-dev@python.org/thread/EZ6O7MOPZ4GA75MKTDO7LAELKXUHK2QS/#"]...

exactly - let's say someone needs to write some JSON for the first time.

With the json package included, all they need to do is `import json`. If that wasn't there, they's look in PyPi for a JSON implementation, and find an absolutely astonishing number of options. I just did a search for "JSON" on PYPI, and it's HUGE -- most of them are for specialized JSON-using protocols of some sort. I was actually really surprised that couple I know about of the top of my head (ujson, orjson) are actually hard to find.

"You can just pip install it" is really not a good solution.

In fact, this is an example, I think, of where we should put some effort into making the included batteries better -- it's great to have a JSON lib built in, but it's too bad that it's not best-of-bread by pretty much any definition (speed, memory performance, flexibility) -- there are quite a few feature requests open for it -- it would be nice to actually implement some of those. (but yes, that's a lot of work that someone(s) would have to do)

Back to the topic at hand, rather than remove urllib, maybe it could be made better -- an as-easy-to-use-as-requests package in the stdlib would be really great.

-CHB

Strong plus one to this. Part of Python's huge attraction and widespread adoption is that one doesn't need a lot of formal training or engineering experience to just dive right in. Having a rich set of libraries only helps this, forcing newbies to pip/PYPI is a great way to lose them.

I also completely agree that the right answer is to make the standard libraries better, not throw them away.

And for those that don't use them, ignore them.

Best,

coyot
P.S. There are uses for urllib outside of standard web programming. I recently needed to handle strings in query parameter format (`?...&...&...`) and found urllib parse() the easiest way of doing that even though I wasn't taking them in as a webservice. Deleting the library would break this code. :(

"Coyot Linden (Glenn Glazer)" <coyot@lindenlab.com> writes:
> P.S. There are uses for urllib outside of standard web programming. I
> recently needed to handle strings in query parameter format
> (`?...&...&...`) and found urllib parse() the easiest way of doing
> that even though I wasn't taking them in as a webservice. Deleting the
> library would break this code. :(

+1. If urllib is removed, I would very much like to preserve at least
the functionality of urlparse /somewhere/.

--
Lincoln Auster
they/them
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C3JZDCWLCMED32Q4MFHMITI2QQGK2ME5/
Code of Conduct: http://python.org/psf/codeofconduct/

On 28. 03. 22 17:34, Skip Montanaro wrote:
> Barry writes (in part):
>
>> We could still distribute “sumo” releases which include all the
>> batteries, but develop and maintain them outside the cpython repo,
>> and even release them separately on PyPI. It’s *possible* but I
>> don’t know if it’s *practical*.
>
> to which Stephen responds (in part):
>
>> [Emacs really didn't change much in 20 years.] .... [I]n
>> Python, every new release makes the Mailman crew want to stop
>> supporting all previous releases of Python because there's some
>> feature that can't be emulated that we really love: genexps or async
>> or walrus operator or ....
>
> It seems to me that what we have is the possibility that, say, package
> P migrates to PyPI in concert with the release of Python version X
> where maintenance can be picked up by the broader Python community.
> Whether or not it would proceed to track ongoing changes to the
> language isn't clear, and it's not obvious to me that less effort
> would be required other than perhaps by core devs (though some of them
> would likely pitch in to maintain packages with which they are
> currently involved). If you go with the "discard the batteries"
> approach, I think it would at least be worthwhile distributing a
> requirements.txt file ("sumo.txt" or "batteries.txt"?) which would
> tell someone installing Python how to reclaim the batteries of their
> Python youth, and for other Python implementations to track a set of
> packages which would make them plausibly compatible with CPython. CI
> could still rely on that to provide as much test coverage as current
> today (I think). If nothing else, it might alert the core devs to the
> potential for breakage of presumably widely used packages by changes
> to CPython.


What happens when the new maintainer puts malware in the next release of
a package in sumo.txt?
Will core devs be blamed for listing it?
As a user, how do I determine if I can trust the packages there? (This
is easily the hardest part of finding and installing a package from
PyPI, though many people seem to skip it.)


> What happens to P when Python version Y grows a new syntactic feature?
> Do P's maintainers fork to both continue feature growth as well as
> syntactic modernization? If something like batteries.txt is created to
> tie a slimmed-down CPython distribution to the batteries it once
> contained, would the core group exert any control over unit testing,
> documentation, package variations and such?

If they do exert control, why not keep it in stdlib?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YLK5UMSWYXX6JT2PL23F26DGQ5XSOVV2/
Code of Conduct: http://python.org/psf/codeofconduct/

I think to most people "batteries included" doesn't necessarily mean
"standard library," with all that implies. It just means "it came with
the first thing that I installed" (or alternatively, "at no additional
charge").

Given there are *plenty* of existing distros out there that install more
than just the standard library for the benefit of their users, and yet
the people asking for a fatter stdlib don't seem to want them, perhaps
we just need to officially endorse one?

I've got no issue preinstalling a curated sumo.txt into the Windows
installers, provided we're clear about support (i.e. none from us) and
someone besides me is tracking everything in that curated list. That
probably requires new volunteers who are somehow already trustworthy,
but that should be easier than people to actually write and maintain new
functionality on stdlib timelines.

We would still have to have the "light" distribution, as many of our
users wouldn't be able to use a "this is full of unsupported stuff"
package, but at least most users could be pointed to the bigger install
by default and not even care that the batteries just came in the same
package and aren't actually an integral part of the core product.

Cheers,
Steve

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TC5EEDFNSZJBR22GQDJ32UWVTQNILU2P/
Code of Conduct: http://python.org/psf/codeofconduct/

> If urllib is removed, I would very much like to preserve at least the
functionality of urlparse /somewhere/.

Given every alternative library to urllib relies on urllib.parse (and some
rely on urllib.request), as well as popular libraries like pip, in this
hypothetical it would definitely need to be maintained somewhere.

Damian (he/him)

On Mon, Mar 28, 2022 at 1:38 PM lincoln auster [they/them] <
lincolnauster@gmail.com> wrote:

>
> "Coyot Linden (Glenn Glazer)" <coyot@lindenlab.com> writes:
> > P.S. There are uses for urllib outside of standard web programming. I
> > recently needed to handle strings in query parameter format
> > (`?...&...&...`) and found urllib parse() the easiest way of doing
> > that even though I wasn't taking them in as a webservice. Deleting the
> > library would break this code. :(
>
> +1. If urllib is removed, I would very much like to preserve at least
> the functionality of urlparse /somewhere/.
>
> --
> Lincoln Auster
> they/them
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/C3JZDCWLCMED32Q4MFHMITI2QQGK2ME5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Mon, 28 Mar 2022 at 17:45, Steve Dower <steve.dower@python.org> wrote:
>
> I think to most people "batteries included" doesn't necessarily mean
> "standard library," with all that implies. It just means "it came with
> the first thing that I installed" (or alternatively, "at no additional
> charge").

I think that for users it means "came as part of the official
installer" where "official" means "whoever I chose to trust" - which
in turn often means "python.org".

For package maintainers it means "the bunch of stuff I can assume
exists when someone says they have python" - which is why distros that
debundle parts of the stdlib get so much grief. Also note that "just
add a dependency" is not a simple request for a library maintainer -
if I depend on requests, and someone using my library uses httpx, then
I'm imposing a potential transitive dependency conflict on them. So
using stdlib functionality is a non-trivial advantage for library
maintainers (of course, sometimes the benefits of using a 3rd party
library outweigh this, but not always).

> Given there are *plenty* of existing distros out there that install more
> than just the standard library for the benefit of their users, and yet
> the people asking for a fatter stdlib don't seem to want them, perhaps
> we just need to officially endorse one?

But people don't use those existing options presumably for a reason.
Having the core devs say "distro X is approved" won't remove those
reasons. And removing the python.org distro because X is now approved
will create a problem for those users.

And none of this alters the fact that "stdlib" means "can be assumed
to be present in all Python installations" for some use cases.

> I've got no issue preinstalling a curated sumo.txt into the Windows
> installers, provided we're clear about support (i.e. none from us) and
> someone besides me is tracking everything in that curated list. That
> probably requires new volunteers who are somehow already trustworthy,
> but that should be easier than people to actually write and maintain new
> functionality on stdlib timelines.

"Support" is a somewhat ambiguous matter in the context of
volunteer-supported software. The point of having a stdlib is that
people know where to report issues, not that we're necessarily any
more responsive than a 3rd party maintainer. There's a value in
knowing that if you find an issue with "anything you installed as part
of the basic Python install", you report it in the same place. And
having the documentation all in one place. Etc.

Yes, people come along and make a big fuss about how they expect more
in the way of support ("this bug is urgent, my production system is
failing because of it") and yes, people with that attitude have a
negative effect on core developer morale. But we don't promise fixes,
or solutions. At best there's an implication that the core devs care
about the stdlib in some sort of vague, generalised sense. (I care
about the health of the stdlib, but I'm not likely to try to fix a bug
in ossaudiodev, for example).

If we did try to mandate something more concrete, then maybe the
conversation would be different. But I'm not aware of anyone proposing
anything like that.

> We would still have to have the "light" distribution, as many of our
> users wouldn't be able to use a "this is full of unsupported stuff"
> package, but at least most users could be pointed to the bigger install
> by default and not even care that the batteries just came in the same
> package and aren't actually an integral part of the core product.

To be honest, I feel like I'm just reiterating stuff I've said before
here, and I think the same is true of the points I'm responding to. Is
there actually any new development here, or is it just a repeat of the
same positions people have expressed in the past, based on the same
underlying realities? (I'm not *against* going over the debate again,
it helps make sure people haven't changed their minds, but it's
important to be clear that none of the practical facts have changed,
if that is the case).

I should also point out (as someone heavily involved in Python's
packaging ecosystem) that I don't believe that the existing packaging
infrastructure is sufficiently robust to support a world where the
Python stdlib was all delivered as external packages. And even if it
were, bootstrapping something like pip would be a nightmare - for a
start, we don't vendor libraries that include C extensions, so we
could only handle pure Python stdlib modules - and even vendoring all
of them (and all of our vendored tools' stdlib dependencies) would be
a nightmare.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SKIKUC4U3L74G43BNC5SDCXFYUY3G447/
Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Mar 27, 2022 at 11:04 AM Paul Moore <p.f.moore@gmail.com> wrote:

> > In fact, this is an example, I think, of where we should put some effort
> into making the included batteries better -- it's great to have a JSON lib
> built in, but it's too bad that it's not best-of-bread by pretty much any
> definition (speed, memory performance, flexibility) -- there are quite a
> few feature requests open for it -- it would be nice to actually implement
> some of those. (but yes, that's a lot of work that someone(s) would have to
> do)
> >
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it.


Actually, I think this is very hard to talk about in generalities. If. you
look at the two examples I used, json and urllib, we have two different
"problems".

I would like to see a requests-like interface in the stdlib not because
it's "best of breed", and has, e.g. more features or better performance, or
...., I would like to see it because it has a cleaner, simpler interface
(the easy things should be easy) -- so I'm not necessarily advocating that
the entirely of requests be brought in to the stdlib, or that request be
made obsolete, but that we borrow a bit of its API. Which would also make
the transition from stdlib to requests easier -- write you scripts with the
stdlib, and when you find you need some more "advanced" features, you simpy
drop requests in and move along.

json is actually the opposite- most of the third-party json libs mimic (and
maybe extend) the json API -- so you can, in most cases, start out with the
stdlib, and drop in another app when you need more performance, or an extra
feature.

The issue I have with the stdlib json lib is that I think it could have a
somewhat better API, and there's a few features I'd like to see it grow, so
that third party libs are less necessary. But it seems there's not much
motivation to do so, because "folks can always use a third party lib". Why
is this an problem? two reasons:

1) There are quite a few json libs out there, and they tend to be a fourth
party dependency (I just coined that term for a dependency of a dependency)
. I literally have some of my code requiring four different json libs to
run. It works, so maybe not a huge deal, but it would be really nice to
simplify that!

2) I don't think any of the third party libs provide a full menu -- JSON5
support, performance, etc, so sometimes there's no single best of breed for
even a particular application. Can/should that be solved by the stdlib?
maybe not, but it would be nice to see it addressed somewhere -- a grand
unification of JSON libs.

In short: I think I agree with most folks here that we should still include
the batteries, and they should be updated / maintained to some extent. What
exactly could/should be done is going to have to be worked on on a case by
case basis.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Mon, Mar 28, 2022 at 11:29 AM Paul Moore <p.f.moore@gmail.com> wrote:

> To be honest, I feel like I'm just reiterating stuff I've said before
> here, and I think the same is true of the points I'm responding to

...

> (I'm not *against* going over the debate again,
> it helps make sure people haven't changed their minds, but it's
> important to be clear that none of the practical facts have changed,
> if that is the case).
>

Maybe there's a way to make this discussion (it feels more like a
discussion than debate at the moment) more productive by writing some
things down. I'm not sure it's a PEP, but some kind of:

"policy for the stdlib" document in which we could capture the primary
points of view, places where there's consensus, etc. would be helpful to
keep us retreading this over and over again.

I suggest this without the bandwidth to actually shepherd the project, but
if someone wants to, I think it would be a great idea.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On 3/28/2022 7:26 PM, Paul Moore wrote:
> To be honest, I feel like I'm just reiterating stuff I've said before
> here, and I think the same is true of the points I'm responding to. Is
> there actually any new development here, or is it just a repeat of the
> same positions people have expressed in the past, based on the same
> underlying realities? (I'm not *against* going over the debate again,
> it helps make sure people haven't changed their minds, but it's
> important to be clear that none of the practical facts have changed,
> if that is the case).

Yep, same. Nothing much has changed, but if those of us who know that
nothing has changed fail to reiterate those points, everyone new to the
discussion will assume that everyone agrees with them.

Just deleted a long rant to the discussion in general (not directed at
Paul at all ;) ) that would have achieved nothing, but I'll summarise to
this: email doesn't fix bugs; maintainers fix bugs. Please let us know
*publicly* if you want to become the maintainer for a stdlib module and
then we can support them, but if nobody is willing/able/ready to care
for them it's irresponsible for us to keep shipping them to users.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I37FCHGNGCKACI4C4S3OA6MO4MZLRABY/
Code of Conduct: http://python.org/psf/codeofconduct/

> What happens when the new maintainer puts malware in the next release of
> a package in sumo.txt?
> Will core devs be blamed for listing it?
> As a user, how do I determine if I can trust the packages there? (This
> is easily the hardest part of finding and installing a package from
> PyPI, though many people seem to skip it.)

I will point out that you quoted my entire post except for the most
important line, the one which reads:

> Just thinking out loud...

I am still thinking out loud, so keep that in mind. (Perhaps this belongs
on one of the two ideas groups, but this is where things started, so I will
continue here for now.)

Consider a hypothetical situation.

- Suppose Python v. X is the last version delivered with batteries, so
v. Y lacks a json module (assuming that's not determined to be so
important that it must be delivered as part of the CPython installer proper)
.
- Downstream, some Linux distro realizes their users still want
batteries (as do their Python admin tools), so recreates a Python
package with them. Their package of v. Y *does* include (at least) the
json battery.
- One of the maintainers of the now externally maintained json battery
takes it upon themselves to protest an incursion into East WhatsItStan by
West WhatsItStan in the worst manner possible and inserts code to scrub
disks of all West WhatsItStan-ian homed computers (as the primary
maintainer of some widely installed (Node? JS?) package package did).
- Some poor West WhatsItStan-ian upgrades their Linux distro, only to
find when they next run their favorite Python application that their disk
drive is wiped clean.

Note that I haven't postulated the existence of a sumo.txt file. Whether
the CPython distribution contains batteries or not, someone will recreate
the batteries, if not in toto, at least piecemeal (some application or
package depends on json and will blindly pull it in). Who gets the blame
here? The core Python devs (because json "always came with Python")? The
maintainers of the Linux distro for recreating the batteries but not having
any West WhatsItStanian-based testers? The new maintainers of the now
external (and corrupted) json package? Maybe the several levels of
indirection would serve to insulate the Python devs, but maybe not.

> If they do exert control, why not keep it in stdlib?

Perhaps that is the best route. Someone here (Stéfane Fermigier, it seems)
questioned whether batteries included are such a good idea. It's quite
possible that enough core devs disagree with that sentiment that the
batteries will stay (and perhaps grow in number, though more slowly than in
the past). That said, if enough devs agree with Stéfane, then what's the
best route forward? Discard all batteries and let the chips fall where they
may? Extract the dead batteries (and presumably their docs and test bits)
into separate github.com/python repositories? Ask others to extract them
into their own non-github.com/python repositories?

One of the common reasons old platform support is dropped from CPython
(OS/2 or AmigaOS anyone?) is that the maintenance load for the core devs is
too high relative to the community benefit derived from supporting small
minority platforms. The unlucky modules named by PEP 594 are about to
suffer the same fate. (I'm not arguing that they shouldn't be deleted.)

Once PEP 594 has been implemented, all the low-hanging fruit will have been
picked. At that point, it's keep everything or keep (almost) nothing. I
think that will largely depend on the willingness of the core devs to keep
maintaining modules they may well never use in their own work (like getopt
and wave). One thing I think is obvious. If you remove all the batteries
not deemed absolutely essential to build and test CPython, I think you have
to somehow symbolically say, "these correspond to the various batteries
which used to be in the CPython distribution." Maybe it's text in a README
file, a new PEP, a sumo.txt file, or a Linux distro's packaging.

Skip

P.S. Personally, I was never fond of shutil or the logging package (as two
examples — though for different reasons). We might have something better in
both realms by now if they hadn't been added to the stdlib long ago. This
ability of presence in the stdlib to forestall further development might
well be the strongest argument to remove most batteries. On the other hand,
we seem to have had little trouble cycling through several completely
different regular expression modules. Maybe I'm just imagining a barrier
where none exists.

On Mon, 28 Mar 2022 at 20:37, Steve Dower <steve.dower@python.org> wrote:

> Please let us know
> *publicly* if you want to become the maintainer for a stdlib module and
> then we can support them, but if nobody is willing/able/ready to care
> for them it's irresponsible for us to keep shipping them to users.

I'm sorry for picking on a single point you made and continuing to
simply state my contrary view, but I think it's important.

IMO, it's irresponsible of us to *remove* functionality if there's no
suitable alternative (and remember, my definition of "suitable"
includes "appropriate for cases where a PyPI library isn't acceptable
for some reason"). Of course, it's not OK for us to promise a level of
support that we can't or won't provide, but *we don't provide such
promises*. We say that we support the stdlib, but there's no actual
statement of what that means - so it's not clear to me how we're
failing to deliver "enough". Nor is it at all clear to me that dumping
the responsibility on an external maintainer will be any better - most
likely it just lets *us* feel better about the situation while not
actually being better for the end user.

If we were to define some sort of actual support guarantees, we might
be having a different conversation. But I'm unclear how a volunteer
organisation can provide meaningful promises that we wouldn't be able
to deliver for the stdlib as it stands now.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OSYFNLIULB56EVSG57QXKIW4NRWI4CNZ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Mar 28, 2022 at 11:52 AM Christopher Barker <pythonchb@gmail.com>
wrote:

> On Mon, Mar 28, 2022 at 11:29 AM Paul Moore <p.f.moore@gmail.com> wrote:
>
>> To be honest, I feel like I'm just reiterating stuff I've said before
>> here, and I think the same is true of the points I'm responding to
>
> ...
>
>> (I'm not *against* going over the debate again,
>> it helps make sure people haven't changed their minds, but it's
>> important to be clear that none of the practical facts have changed,
>> if that is the case).
>>
>
> Maybe there's a way to make this discussion (it feels more like a
> discussion than debate at the moment) more productive by writing some
> things down. I'm not sure it's a PEP, but some kind of:
>
> "policy for the stdlib" document in which we could capture the primary
> points of view, places where there's consensus, etc. would be helpful to
> keep us retreading this over and over again.
>
> I suggest this without the bandwidth to actually shepherd the project, but
> if someone wants to, I think it would be a great idea.
>

Once
https://mail.python.org/archives/list/python-committers@python.org/thread/5EUZLT5PNA4HT42NGB5WVN5YWW5ASTT5/
is considered resolved, the next part of my "what *is* the stdlib" plan is
to finally try to suss all of this out and more-or-less write a stdlib
policy PEP so we stop talking about this. My guess it will be more of
guidance about what we want the stdlib to be and thus guide what things
belong in it. No ETA on that work since I also have four other big Python
projects on the go right now whose work I am constantly alternating between.

-Brett


>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GAZAFRFVJVQZMEIHTQUJASP7VRAKA5RR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Sun, Mar 27, 2022, 11:07 AM Paul Moore <p.f.moore@gmail.com> wrote:

> On Sun, 27 Mar 2022 at 17:11, Christopher Barker <pythonchb@gmail.com>
> wrote:
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it (as is httpx, and urllib3, and aiohttp). But
> urllib is for people who want to get a file from the web, and *not*
> have to deal with dependencies, 3rd party libraries, etc.
>


One thing about talking about "make urllib more like requests" that is
different than any of the other libs, though, is that requests aims to be
easier to use than anything else (which I note Chris Barker called out as
why he wanted urllib to be more like it). I think that's important to
think about because i think ease of use is also the number one thing that
the audience you talk about is also looking for.

Of course, figuring out whether an api like request's is actually easier to
use than urllib or merely more featureful is open to debate.

-toshio