Mailing List Archive

On Thu, 15 Apr 2021 11:37:28 +0200
Victor Stinner <vstinner@python.org> wrote:
> Paul Bryan:
> > Seems like this is something that should make its way into stdlib?
>
> In the last 10 years, the trend is more to remove anything related to
> packaging *outside* the stdlib :-) Since it's evolving way faster

What is "evolving way faster"? packaging.version implements PEP 440,
which has been approved in 2014. Recent changes have been really minor:
https://github.com/pypa/packaging/commits/main/packaging/version.py



_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NFPGSSDTD7SNZI4XJF542NS4R7OXVVMZ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org> wrote:

> Paul Bryan:
> > Seems like this is something that should make its way into stdlib?
>
> In the last 10 years, the trend is more to remove anything related to
> packaging *outside* the stdlib :-)


Well, maybe all of us don't think that's a good idea ;-)

But anyway, I would say removing anything *related* to packaging outside
the stdlib is a bad idea -- requiring an external tool to build a package
is OK, but requireing an external use packages, not so much.

Presumably that's why importlib.metadata exists in the stdlib.

Version is arguably useful from the package user side. As I believe Victor
mentioned, there are two uses for version information: display to the user
-- for which version strings are fine, or programmatic comparison -- for
which something like the Version object is very helpful. Do we only need to
use version information programmatically when we are creating (or
installing) packages? I don't think so -- I know I have code that (poorly)
does version checking programmatically.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 4:55 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> Presumably that's why importlib.metadata exists in the stdlib.
>

I was so hopeful about this, but in the end... not really. I have not used
this capability before. Here are a few different situations I know of:

>>> import re, statistics, pandas, vaex, bs4
>>> for mod in [re, statistics, pandas, vaex, bs4]:
... try:
... print(mod.__name__, end=" ")
... print(mod.__version__)
... except:
... print("__version__ not present")
...
re 2.2.1
statistics __version__ not present
pandas 1.2.4
vaex {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0',
'vaex-hdf5': '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0',
'vaex-jupyter': '0.6.0', 'vaex-ml': '0.11.1'}
bs4 4.9.3

>>> for mod in [re, statistics, pandas, vaex, bs4]:
... try:
... print(mod.__name__, version(mod.__name__))
... except Exception as err:
... print(mod.__name__, repr(err))
...
re PackageNotFoundError('re')
statistics PackageNotFoundError('statistics')
pandas 1.2.4
vaex 4.1.0
bs4 PackageNotFoundError('bs4')

It seems like (somehow or another) importlib.metadata is doing something
perhaps more useful for vaex. But it is doing distinctly worse for re,
statistics, and bs4.

Version is arguably useful from the package user side. As I believe Victor
> mentioned, there are two uses for version information: display to the user
> -- for which version strings are fine, or programmatic comparison -- for
> which something like the Version object is very helpful. Do we only need to
> use version information programmatically when we are creating (or
> installing) packages? I don't think so -- I know I have code that (poorly)
> does version checking programmatically.
>

I think the key thing that Chris and I are pointing to is that there is a
definite use for versions that is NOT for package maintainers/creators.
Interactive use is definitely one such case, and eyeballing something, even
if it is the oddball use that Vaex has, can help with figuring out an
issue. But in actual code, I relatively often do something like this. Or
rather, the below is what I would find really nice to be able to do.

ver = robust_version(module)
if ver >= (5, 2, 1):
doit_modern_style()
elif ver < (5, 2, 1):
doit_old_style
else:
doit_unversioned_style()

I'd like code like that to ALWAYS succeed. No exceptions raised. And it
would *usually* use reasonable heuristics to figure out some sort of
structured version info as well as is possible. I.e. probably look in
several places and return some custom object that represents a "best
effort." That object might be like NaN in being neither less than nor more
than other things, in the fallback case.

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>
> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>
> Are you sure? See this discussion, I don't think it's as simple as all that.

From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

PEP 396 has been dormant for a long time and I have no interest in pushing it forward any more. If someone else wants to take up the cause, I recommend creating a new PEP and referring back to 396.

-Barry

On Thu, 15 Apr 2021 09:38:41 -0700
Barry Warsaw <barry@python.org> wrote:
> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
> >
> > You wrote the original PEP, so of course you can withdraw it (or reject it), but...
> >
> > Are you sure? See this discussion, I don't think it's as simple as all that.
>
> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

This is often handled using `setuptools_scm` or `versioneer` (see PyPI
for each of these packages).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J74GLEQS4HHSUQU3WO57GAURKQWWTTJP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 9:36 AM David Mertz <mertz@gnosis.cx> wrote:

> On Thu, Apr 15, 2021 at 4:55 PM Christopher Barker <pythonchb@gmail.com>
> wrote:
>
>> Presumably that's why importlib.metadata exists in the stdlib.
>>
>
> I was so hopeful about this, but in the end... not really. I have not
> used this capability before. Here are a few different situations I know of:
>
> >>> import re, statistics, pandas, vaex, bs4
> >>> for mod in [re, statistics, pandas, vaex, bs4]:
> ... try:
> ... print(mod.__name__, end=" ")
> ... print(mod.__version__)
> ... except:
> ... print("__version__ not present")
> ...
> re 2.2.1
> statistics __version__ not present
> pandas 1.2.4
> vaex {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0',
> 'vaex-hdf5': '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0',
> 'vaex-jupyter': '0.6.0', 'vaex-ml': '0.11.1'}
> bs4 4.9.3
>
> >>> for mod in [re, statistics, pandas, vaex, bs4]:
> ... try:
> ... print(mod.__name__, version(mod.__name__))
> ... except Exception as err:
> ... print(mod.__name__, repr(err))
> ...
> re PackageNotFoundError('re')
> statistics PackageNotFoundError('statistics')
> pandas 1.2.4
> vaex 4.1.0
> bs4 PackageNotFoundError('bs4')
>
> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex. But it is doing distinctly worse for re,
> statistics, and bs4.
>

I don't see how importlib.metadata plays into your code sample since you
didn't import it to use it?


>
> Version is arguably useful from the package user side. As I believe Victor
>> mentioned, there are two uses for version information: display to the user
>> -- for which version strings are fine, or programmatic comparison -- for
>> which something like the Version object is very helpful. Do we only need to
>> use version information programmatically when we are creating (or
>> installing) packages? I don't think so -- I know I have code that (poorly)
>> does version checking programmatically.
>>
>
And is installing a dependency that much of a burden for that code?


>
> I think the key thing that Chris and I are pointing to is that there is a
> definite use for versions that is NOT for package maintainers/creators.
> Interactive use is definitely one such case, and eyeballing something, even
> if it is the oddball use that Vaex has, can help with figuring out an
> issue. But in actual code, I relatively often do something like this. Or
> rather, the below is what I would find really nice to be able to do.
>
> ver = robust_version(module)
> if ver >= (5, 2, 1):
> doit_modern_style()
> elif ver < (5, 2, 1):
> doit_old_style
> else:
> doit_unversioned_style()
>
> I'd like code like that to ALWAYS succeed. No exceptions raised. And it
> would *usually* use reasonable heuristics to figure out some sort of
> structured version info as well as is possible. I.e. probably look in
> several places and return some custom object that represents a "best
> effort." That object might be like NaN in being neither less than nor more
> than other things, in the fallback case.
>

All doable with importlib.metadata and 'packaging'.

-Brett


>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born. Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4575HZHNN45FNOTKH5DSD6NKNRUJGXFU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
wrote:

> On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> wrote:
>
>> Paul Bryan:
>> > Seems like this is something that should make its way into stdlib?
>>
>> In the last 10 years, the trend is more to remove anything related to
>> packaging *outside* the stdlib :-)
>
>
> Well, maybe all of us don't think that's a good idea ;-)
>

As an active participant in the PyPA side of the packaging ecosystem and
one of the maintainers of 'packaging', I am definitely NOT in favour of
moving any of 'packaging' into the stdlib, nor breaking it up. We are
taking distutils out of the stdlib for a reason. Backsliding on plans that
have been in place for years is not something I would want to see happen as
the motivations have not changed.


>
> But anyway, I would say removing anything *related* to packaging outside
> the stdlib is a bad idea -- requiring an external tool to build a package
> is OK, but requireing an external use packages, not so much.
>

You can totally use a package already as things exist today and will in the
future; you're just wanting a specific way to interpret a package's
metadata which I don't view as the same thing.


>
> Presumably that's why importlib.metadata exists in the stdlib.
>

It's there because of how tightly it binds to the import system; same goes
for importlib.resources.


>
> Version is arguably useful from the package user side. As I believe Victor
> mentioned, there are two uses for version information: display to the user
> -- for which version strings are fine, or programmatic comparison -- for
> which something like the Version object is very helpful. Do we only need to
> use version information programmatically when we are creating (or
> installing) packages? I don't think so -- I know I have code that (poorly)
> does version checking programmatically.
>

No one said that's the *only* use-case, but just because there's more than
that does not mean we should move any packaging code into the stdlib to
support a zero-install use-case.

A key point I want to make here is the version metadata is in every
project's .dist-info/METADATA file. That's what importlib.metadata knows
how to read and hence why people have been suggesting to use it. The
version information is fundamental to the wheel format and in fact you
can't have a wheel without it. So asking every Python project to set a
__version__ isn't going to change accessibility of version numbers when it
comes to installed projects.

On Thu, 15 Apr 2021 10:28:53 -0700
Brett Cannon <brett@python.org> wrote:
> On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
> wrote:
>
> > On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> > wrote:
> >
> >> Paul Bryan:
> >> > Seems like this is something that should make its way into stdlib?
> >>
> >> In the last 10 years, the trend is more to remove anything related to
> >> packaging *outside* the stdlib :-)
> >
> >
> > Well, maybe all of us don't think that's a good idea ;-)
> >
>
> As an active participant in the PyPA side of the packaging ecosystem and
> one of the maintainers of 'packaging', I am definitely NOT in favour of
> moving any of 'packaging' into the stdlib, nor breaking it up. We are
> taking distutils out of the stdlib for a reason. Backsliding on plans that
> have been in place for years is not something I would want to see happen as
> the motivations have not changed.

This seems gratuitously dogmatic. Version numbers are a simple feature
that is stably encoded in PEP 440. It's purely computational, does
not involve reading or saving any persistent state in the filesystem,
making any network access, etc. Why wouldn't it have its place in the
stdlib?

> > But anyway, I would say removing anything *related* to packaging outside
> > the stdlib is a bad idea -- requiring an external tool to build a package
> > is OK, but requireing an external use packages, not so much.
> >
>
> You can totally use a package already as things exist today and will in the
> future; you're just wanting a specific way to interpret a package's
> metadata which I don't view as the same thing.

That specific way is a Python standard (PEP 440). Having the
functionality in the stdlib would encourage people to use it. Not
having it in the stdlib encourages people to use adhoc version parsing,
or worse, naive string comparison.

> So asking every Python project to set a
> __version__ isn't going to change accessibility of version numbers when it
> comes to installed projects.

That doesn't have much to do with the suggestion of shipping a Version
class in the stdlib, though. Many projects already provide a
__version__, and that field is often inspected programmatically in
dependent packages (to work around behaviour changes, for example).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JAZRZL4FPJ7BW763TVKZVPY2OQBHQI5V/
Code of Conduct: http://python.org/psf/codeofconduct/

On Apr 15, 2021, at 09:54, Antoine Pitrou <antoine@python.org> wrote:
>
> On Thu, 15 Apr 2021 09:38:41 -0700
> Barry Warsaw <barry@python.org> wrote:
>> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>>>
>>> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>>>
>>> Are you sure? See this discussion, I don't think it's as simple as all that.
>>
>> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.
>
> This is often handled using `setuptools_scm` or `versioneer` (see PyPI
> for each of these packages).

Yep, exactly. (I knew about and had in mind setuptools_scm but I didn’t know about versioneer; thanks for that reference!)

-Barry

Replying to the original email because my comment is more a thought
after reading the whole thread rather than a response to any
particular message.

What I love about __version__ is that it is simple. It doesn't come
with the complexity of solutions like `importlib.metadata.version`.
The maintainers of the project arranged for it to be set**. It doesn't
depend on how the module was packaged or distributed.

I also don't think it's really a language update or requires any
changes in the stdlib -- it's just a standard for the community to
follow where none currently exists.

It's also probably far from the end of the world*** if it doesn't
happen, but I still like the idea.

** It might have been updated somewhat automatically -- my point here
is that distutils, setuptools, packaging, pip, wheels, PyPI were all
not involved.

*** Although that is likely true of most PEPs, even much more
complicated ones. :)

Schiavo,
Simon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O2LGPEIOZMJYDJ4GMZXMOQK7GDWYS6X5/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/2021 12:35 PM, David Mertz wrote:

> re 2.2.1

re.__version__ was last modified 2001-10-07 by F. Lundh in
bec95b9d8825b39cff46a8c645fa0eeb8409854e. What does '2.2.1' mean? Only
Fredrik knows. The commit message says that he rewrote the pattern.sub
and pattern.subn methods in C. That should have been user invisible
except for speed.

I believe that it is dead data left only for back compatibility and
because of an inability to give a deprecation warning.

> re PackageNotFoundError('re')

> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex.  But it is doing distinctly worse for re,

Is an effectively meaningless and misleading value better than none?

IDLE once had a version last set, I believe, in 2.5 or 2.6, by K.
Kaiser, with whatever meaning it had to him. I removed it in 3.6
(allowed by PEP 434 + RM decision). No one complained.

The real version of any stdlib module is the Python version it is
released with, plus and external app versions it is associated with
(sqlite and ssl, for instance). It has been suggested, if not decided,
that other versions should go.


--
Terry Jan Reedy


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4AA7I3RS6D3UDIBZ6ELTEEWXKXIGE5WS/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/2021 12:38 PM, Barry Warsaw wrote:
> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>>
>> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>>
>> Are you sure? See this discussion, I don't think it's as simple as all
that.
>
> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

At least some in the stdlib have not been. For re.__version__, not
since Nov 2001.


--
Terry Jan Reedy


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3UNG6KD3JIBHSMWILZGGIFHRSQV6PDKG/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 9:38 AM Barry Warsaw <barry@python.org> wrote:

> From a library maintainers point of view, I personally want to get away
> from using __version__ strings at all. They’re kind of a pain to remember
> to bump on every new release.


That’s a tooling issue, and I don’t think that Python itself should take
any official stance on the tooling. But yes, anything that Python takes an
official stance on should be able to be easily supported with
not-too-complex tools.and I think a __version__ string fits this need.

Personally, I use the __version__ string in my packages as the canonical
source when I bold my packages.

And there are fancier tools for more complex needs.

> PEP 396 has been dormant for a long time and I have no interest in
pushing it forward any more. If someone else wants to take up the cause, I
recommend creating a new PEP and referring back to 396.

Fair enough -- I'm still gathering data to see if I should do just that.

I am leaning toward at least SOME PEP about this -- having __version__
around as a semi-convention with no official recommendation as to if or how
it should be used is kind of a mess.

-CHB

On Thu, Apr 15, 2021 at 12:27 PM Terry Reedy <tjreedy@udel.edu> wrote:

> At least some in the stdlib have not been. For re.__version__, not
> since Nov 2001.
>

From Serhiy over on python-ideas, he's started an effort to clean that up.

https://mail.python.org/archives/list/python-dev@python.org/thread/KBU4EU2JULXSMUZULD5HJJWCGOMN52MK/
).

I believe there is an unfinished PR

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 9:36 AM David Mertz <mertz@gnosis.cx> wrote:

> I was so hopeful about this, but in the end... not really. I have not
> used this capability before. Here are a few different situations I know of:
>

...

...
re PackageNotFoundError('re')
statistics PackageNotFoundError('statistics')
pandas 1.2.4
vaex 4.1.0
bs4 PackageNotFoundError('bs4')

>
> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex. But it is doing distinctly worse for re,
> statistics, and bs4.
>

funny you should try bs4, which I also used as an example in another post.

But what you have found is that there is a difference between a pacakge
(which you can import) and a "distribution" which is something you install.

importlib.metadata is looking for distributions.

re and statistics are not distributions, but rather, built in pacakges
(modules).

bs4 is not a distributiuion, it is a package that is provided by the
"beautifulsoup4" distribution.

In [3]: importlib.metadata.version("beautifulsoup4")
Out[3]: '4.9.3'

Frankly, I'm still a bit confused about the distiction, but I do know for
sure that a single distribution (maybe somethign that you can install via
PyPi) can install more than one top-level package -- and certainly the
packages/modules installed can have a different name than the distribution
name.

And if a distribution installs more than one package, they may have
different version numbers.

I'm not sure what to make of all this, though I'm leaning toward better
suporting the distiction by asking for __version__ strings in top-level
packages -- and maybe making importlib.metadata.version a bit smarter about
looking for packages, and not just distributions.

If you look at the docstring of metadata.version:

"""
In [2]: importlib.metadata.version?
Signature: importlib.metadata.version(distribution_name)
Docstring:
Get the version string for the named package.

:param distribution_name: The name of the distribution package to query.
:return: The version string for the package as defined in the package's
"Version" metadata key.
File: ~/miniconda3/envs/py3/lib/python3.9/importlib/metadata.py
Type: function
"""

It's a bit inconsistent with the use of the term "distribution" vs
"package". That should get cleaned up if nothing else.

Also, the Exception raised is "PackageNotFoundError" -- which should maybe
be "DistributionNotFoundError"?


Version is arguably useful from the package user side. As I believe Victor
mentioned, there are two uses for version information: display to the user
-- for which version strings are fine, or programmatic comparison -- for
which something like the Version object is very helpful. Do we only need to
use version information programmatically when we are creating (or
installing) packages? I don't think so -- I know I have code that (poorly)
does version checking programmatically.


> Or rather, the below is what I would find really nice to be able to do.
>
> ver = robust_version(module)
> if ver >= (5, 2, 1):
> doit_modern_style()
> elif ver < (5, 2, 1):
> doit_old_style
> else:
> doit_unversioned_style()
>

Exactly -- and I htink we are close, if pacakges(modules) had compliant
__version__ strings, thenyou could do
(with the Version object from packaging -- maybe why it should be in the
stdlib)

ver = Version(module.__version__)
if ver >= Version("5.2.1"):
doit_modern_style()
elif ver < Version("5.2.1"):
doit_old_style
else:
doit_unversioned_style()

And if my PR is accepted (which looks unlikley) to allow camparison between
Version objects and strings:

ver = Version(module.__version__)
if ver >= "5.2.1":
doit_modern_style()
elif ver < "5.2.1":
doit_old_style
else:
doit_unversioned_style()

A key distiction here from the importlib.metadata approach is the level of
indirection -- I like being able to ask a module itself what version it is,
rather than asking some other part of teh system to go look it up for me.
So I could do:

import numpy as np

print("using np version:", np.__version)

And this is pretty consitent with the rest of Python, where many objects
(functions, classes, modules) have a __name__ attribute if things "know"
their name, shouldn't they "know" their version as well?

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 9:54 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> Or rather, the below is what I would find really nice to be able to do.
>
>> ver = robust_version(module)
>> if ver >= (5, 2, 1):
>> doit_modern_style()
>> elif ver < (5, 2, 1):
>> doit_old_style
>> else:
>> doit_unversioned_style()
>>
>
> Exactly -- and I htink we are close, if pacakges(modules) had compliant
> __version__ strings, then you could do
> (with the Version object from packaging -- maybe why it should be in the
> stdlib)
>
> ver = Version(module.__version__)
> if ver >= Version("5.2.1"):
> doit_modern_style()
> elif ver < Version("5.2.1"):
> doit_old_style
> else:
> doit_unversioned_style()
>

This wouldn't be bad. A little more verbose, but fine.

The thing is, I almost never care about the version of a *distribution* in
my programs. I care about a version of the *package*. Since, as Chris
mentions, a distribution could contain multiple packages, the same package
could be vendorized (in different versions) by different distributions.
The question I want to ask above is always because "version such-and-such
changed an API," and that's about a package. When I care about
distribution versions, it is because there are dependencies stated in
requirements.txt or environment.yaml or whatever.deb.

Asking for the `.__version__` attributes feels like needless redundancy in
Chris' variation. The constructor should be able to ask "Is this a module?
If so, does it have that attribute". But whatever, I can learn to do that
easily enough.

Someone else mentioned that the versions of standard library modules are
an odd collection of relics. I do not disagree, of course. But sometimes
the stdlib modules *do* change. I thought of the `re` example because I
was doing a tutorial on it, and something or another depended on a change
between 3.8 and 3.9 (I think, I forget the detail). For training material,
this can just be approached as describing the Python version... but for
running code, a version check could be relevant.

Therefore, I think this `Version` constructor (or `robust_version()`
function, or whatever) should have access to that. I.e. if not an
automated update to `.__version__` inside each stdlib module during builds,
then a fallback to "If this is a stdlib module, report sys.version_info
(within the right kind of "Version" object).


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Thu, 15 Apr 2021 at 21:03, Christopher Barker <pythonchb@gmail.com> wrote:

> I am leaning toward at least SOME PEP about this -- having __version__ around as a semi-convention with no official recommendation as to if or how it should be used is kind of a mess.

On the other hand, having a PEP that says you "may" have a version
attribute, but mandates nothing, is pretty useless. People who conform
to that convention don't need telling, people who don't have little
reason to change, and people wanting to use the information still have
to cater for it being missing, so they gain nothing.

But a PEP that makes __version__ a requirement fails instantly,
because there's too many packages in the wild that violate it.

Welcome to the world of packaging PEPs :-)

> I'm not sure what to make of all this, though I'm leaning toward better suporting the distiction by asking for __version__ strings in top-level packages -- and maybe making importlib.metadata.version a bit smarter about looking for packages, and not just distributions.

Personally, I'm leaning more and more towards the view that it's
*distributions* that are versioned, and that's an established
practice. Versioning packages as well (except in the very limited
sense that package X installed by distribution Y can expose Y's
version via its API) only adds confusion and makes the situation less,
not more, useful for users.

So I'm now -1 on any PEP to try to formalise the use of __version__.
If people want to discuss the use of a __version__ attribute, they
should consider contributing to the discussions on versions in
https://packaging.python.org/guides/ (where the necessary context is
already present).

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ECJ2HHV2EXJ7GOJU7ILLQVVLF54VVD3U/
Code of Conduct: http://python.org/psf/codeofconduct/

Replying to myself to update my own thoughts:

I think __version__ and packaging tools answer separate questions.

__version__ answers the question of "this module I imported, what
version is it".

The packaging tools answer the question "what dependency does this
distribution satisfy".

This mismatch is why importlib.metadata.version("bs4") doesn't return
a useful answer. A single distribution might even install multiple
top-level modules with different versions.

Schiavo,
Simon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O242SRB6SBLBBMPCOLCKURBEXXHUSJZN/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 10:53 AM Antoine Pitrou <antoine@python.org> wrote:

> On Thu, 15 Apr 2021 10:28:53 -0700
> Brett Cannon <brett@python.org> wrote:
> > On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
> > wrote:
> >
> > > On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> > > wrote:
> > >
> > >> Paul Bryan:
> > >> > Seems like this is something that should make its way into stdlib?
> > >>
> > >> In the last 10 years, the trend is more to remove anything related to
> > >> packaging *outside* the stdlib :-)
> > >
> > >
> > > Well, maybe all of us don't think that's a good idea ;-)
> > >
> >
> > As an active participant in the PyPA side of the packaging ecosystem and
> > one of the maintainers of 'packaging', I am definitely NOT in favour of
> > moving any of 'packaging' into the stdlib, nor breaking it up. We are
> > taking distutils out of the stdlib for a reason. Backsliding on plans
> that
> > have been in place for years is not something I would want to see happen
> as
> > the motivations have not changed.
>
> This seems gratuitously dogmatic. Version numbers are a simple feature
> that is stably encoded in PEP 440. It's purely computational, does
> not involve reading or saving any persistent state in the filesystem,
> making any network access, etc. Why wouldn't it have its place in the
> stdlib?
>

Technically PyPA specs might start as PEPs but their official home is
packaging.python.org and updates can occur there outside of the PEP
process. In the case of version specifiers (
https://packaging.python.org/specifications/version-specifiers/), they
point back to the original PEP, but that is not universally true nor
guaranteed to hold in the future. In other words I personally don't view
the fact that it's a PEP as a reason to have it in the stdlib (else pretty
much every single packaging PEP is missing).

We also have other things standardized in PEPs and not covered in the
stdlib, e.g. @ as an operator.

So I don't think version comparison occurs often enough to be in the
stdlib, and the fact that an external project exists which isn't interested
in being put in the stdlib suggests to me it isn't worth doing.

But that's a separate topic to discuss at the language summit. :)


>
> > > But anyway, I would say removing anything *related* to packaging
> outside
> > > the stdlib is a bad idea -- requiring an external tool to build a
> package
> > > is OK, but requireing an external use packages, not so much.
> > >
> >
> > You can totally use a package already as things exist today and will in
> the
> > future; you're just wanting a specific way to interpret a package's
> > metadata which I don't view as the same thing.
>
> That specific way is a Python standard (PEP 440). Having the
> functionality in the stdlib would encourage people to use it. Not
> having it in the stdlib encourages people to use adhoc version parsing,
> or worse, naive string comparison.
>

I don't know if I draw the same line since the packaging community is
relying on 'packaging' without issue for this exact thing.


>
> > So asking every Python project to set a
> > __version__ isn't going to change accessibility of version numbers when
> it
> > comes to installed projects.
>
> That doesn't have much to do with the suggestion of shipping a Version
> class in the stdlib, though. Many projects already provide a
> __version__, and that field is often inspected programmatically in
> dependent packages (to work around behaviour changes, for example).
>

But this thread is about standardizing on __version__ which then led to
wanting to bring in some Version object into the stdlib to make that more
useful. So to me they are tied together, else a separate thread should
probably be started if a new module *just *for version parsing support is
desired.

On Thu, 15 Apr 2021 16:12:57 -0700
Brett Cannon <brett@python.org> wrote:
>
> So I don't think version comparison occurs often enough to be in the
> stdlib, and the fact that an external project exists which isn't interested
> in being put in the stdlib suggests to me it isn't worth doing.
>
> But that's a separate topic to discuss at the language summit. :)

Ah, I should submit a question then.

> > That specific way is a Python standard (PEP 440). Having the
> > functionality in the stdlib would encourage people to use it. Not
> > having it in the stdlib encourages people to use adhoc version parsing,
> > or worse, naive string comparison.
>
> I don't know if I draw the same line since the packaging community is
> relying on 'packaging' without issue for this exact thing.

Right, but the functionality is useful generally, not only for the
packaging generally.

> But this thread is about standardizing on __version__ which then led to
> wanting to bring in some Version object into the stdlib to make that more
> useful. So to me they are tied together, else a separate thread should
> probably be started if a new module *just *for version parsing support is
> desired.

You're right, this thread was hijacked somewhat for the Version object
discussion.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U2KO5AZRIHNXWKRQ5YEY3CGW4FZ6SAEU/
Code of Conduct: http://python.org/psf/codeofconduct/

Hi,

sorry for being late to the party, but I may not be the only one wondering…

Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>
> I’d forgotten that this PEP was in Deferred state. I think it should be rejected and I plan on making that change. importlib.metadata is a much better approach to programmatically determining package versions.
>
> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions

This is indeed the correct approach, thanks for letting me learn this.

However, I unsuccessfully searched for the canonical way to look up the
distribution name based on either a module name or an imported module
object. Is there one?

Looks like it could be computed based on information in
"*.egg-info/installed-files.txt", but it's far from trivial. Is
"installed-files.txt" even guaranteed to exist for all distributions?

Cheers,
Baptiste
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SVQ777EGE56OIWHIKPCURGPXGMUJ7HCY/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
devel2021@baptiste-carvello.net> wrote:

> Hi,
>
> sorry for being late to the party, but I may not be the only one wondering…
>
> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
> >
> > I’d forgotten that this PEP was in Deferred state. I think it should be
> rejected and I plan on making that change. importlib.metadata is a much
> better approach to programmatically determining package versions.
> >
> >
> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>
> This is indeed the correct approach, thanks for letting me learn this.
>
> However, I unsuccessfully searched for the canonical way to look up the
> distribution name based on either a module name or an imported module
> object. Is there one?
>

If you mean how to tie a module back to its name on PyPI, you should be
able to look up the "Name" in the project's metadata:
https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
.

-Brett


>
> Looks like it could be computed based on information in
> "*.egg-info/installed-files.txt", but it's far from trivial. Is
> "installed-files.txt" even guaranteed to exist for all distributions?
>
> Cheers,
> Baptiste
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVQ777EGE56OIWHIKPCURGPXGMUJ7HCY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Mon, Apr 26, 2021 at 10:31 PM Brett Cannon <brett@python.org> wrote:

>
>
> On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
> devel2021@baptiste-carvello.net> wrote:
>
>> Hi,
>>
>> sorry for being late to the party, but I may not be the only one
>> wondering…
>>
>> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>> >
>> > I’d forgotten that this PEP was in Deferred state. I think it should
>> be rejected and I plan on making that change. importlib.metadata is a much
>> better approach to programmatically determining package versions.
>> >
>> >
>> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>>
>> This is indeed the correct approach, thanks for letting me learn this.
>>
>> However, I unsuccessfully searched for the canonical way to look up the
>> distribution name based on either a module name or an imported module
>> object. Is there one?
>>
>
> If you mean how to tie a module back to its name on PyPI, you should be
> able to look up the "Name" in the project's metadata:
> https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
> .
>
>
The missing bit here, for me, is how do you map a module back to it's
project (distribution)?

For example:

```
>>> import bs4
>>> bs4.__version__
'4.9.1'
>>> importlib.metadata.metadata('bs4')
PackageNotFoundError: bs4
```

This is because the distribution calls itself 'beautifulsoup4' instead.

The same goes for another package: `umap`, for which the distribution is
called `umap-learn`


This is the best I could come up with from reading the docs:

import bs4 #<- This is the module we want the version of

import importlib
import sys
from itertools import chain
from pathlib import Path

loaders = sys.meta_path

target_path = Path(bs4.__file__)

distros = list(chain(*(finder.find_distributions() for finder in
loaders if hasattr(finder, 'find_distributions'))))
distros_files = chain(*(f for f in (d.files for d in distros)))
distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for f
in d.files]
matching = [d for d, f in distro_files if f == target_path]

for match in matching:
print("Found Version:", match.version)

Steve

On Mon, Apr 26, 2021 at 3:25 PM Stestagg <stestagg@gmail.com> wrote:

>
>
> On Mon, Apr 26, 2021 at 10:31 PM Brett Cannon <brett@python.org> wrote:
>
>>
>>
>> On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
>> devel2021@baptiste-carvello.net> wrote:
>>
>>> Hi,
>>>
>>> sorry for being late to the party, but I may not be the only one
>>> wondering…
>>>
>>> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>>> >
>>> > I’d forgotten that this PEP was in Deferred state. I think it should
>>> be rejected and I plan on making that change. importlib.metadata is a much
>>> better approach to programmatically determining package versions.
>>> >
>>> >
>>> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>>>
>>> This is indeed the correct approach, thanks for letting me learn this.
>>>
>>> However, I unsuccessfully searched for the canonical way to look up the
>>> distribution name based on either a module name or an imported module
>>> object. Is there one?
>>>
>>
>> If you mean how to tie a module back to its name on PyPI, you should be
>> able to look up the "Name" in the project's metadata:
>> https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
>> .
>>
>>
> The missing bit here, for me, is how do you map a module back to it's
> project (distribution)?
>
> For example:
>
> ```
> >>> import bs4
> >>> bs4.__version__
> '4.9.1'
> >>> importlib.metadata.metadata('bs4')
> PackageNotFoundError: bs4
> ```
>
> This is because the distribution calls itself 'beautifulsoup4' instead.
>

Unfortunately I thought importlib.metadata would have used the module name
instead of the metadata details, but in hindsight am guessing that the
.dist-info is what it's using to do the lookup and that's based on the
package name instead of the project name.

This is a long-standing issue with projects that use project names which
differ from their module name, but there's no good way without checking
what files a project installed (which is what you're doing below).

-Brett


>
> The same goes for another package: `umap`, for which the distribution is
> called `umap-learn`
>
>
> This is the best I could come up with from reading the docs:
>
> import bs4 #<- This is the module we want the version of
>
> import importlib
> import sys
> from itertools import chain
> from pathlib import Path
>
> loaders = sys.meta_path
>
> target_path = Path(bs4.__file__)
>
> distros = list(chain(*(finder.find_distributions() for finder in
> loaders if hasattr(finder, 'find_distributions'))))
> distros_files = chain(*(f for f in (d.files for d in distros)))
> distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for
> f in d.files]
> matching = [d for d, f in distro_files if f == target_path]
>
> for match in matching:
> print("Found Version:", match.version)
>
> Steve
>

On Tue, 27 Apr 2021 at 18:01, Brett Cannon <brett@python.org> wrote:

> Unfortunately I thought importlib.metadata would have used the module name instead of the metadata details, but in hindsight am guessing that the .dist-info is what it's using to do the lookup and that's based on the package name instead of the project name.
>
> This is a long-standing issue with projects that use project names which differ from their module name, but there's no good way without checking what files a project installed (which is what you're doing below).

That's correct. There is no link from a given import name back to the
PyPI project name. And the metadata for installed packages (defined
here: https://packaging.python.org/specifications/recording-installed-packages/)
is keyed on the PyPI project name, as the OP noted ("beautifulsoup4"
rather than "bs4", "setuptools" rather than "pkg_resources", etc).

>> This is the best I could come up with from reading the docs:
>>
>> import bs4 #<- This is the module we want the version of
>>
>> import importlib
>> import sys
>> from itertools import chain
>> from pathlib import Path
>>
>> loaders = sys.meta_path
>>
>> target_path = Path(bs4.__file__)
>>
>> distros = list(chain(*(finder.find_distributions() for finder in loaders if hasattr(finder, 'find_distributions'))))
>> distros_files = chain(*(f for f in (d.files for d in distros)))
>> distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for f in d.files]
>> matching = [d for d, f in distro_files if f == target_path]
>>
>> for match in matching:
>> print("Found Version:", match.version)

The following is a bit simpler. The file.locate() method of a
distribution is undocumented - but I tried to use resolve() on the
path object I got from dist.files, and it appears not to be
implemented (at least in Python 3.9) - PackagePath objects are a
subclass of *Pure* Path objects, not concrete paths :-( TBH, I suspect
the fact that it's undocumented is an oversight, it's clearly
deliberately added.

import importlib.metadata
from pathlib import Path

import pkg_resources
target = Path(pkg_resources.__file__)

for dist in importlib.metadata.distributions():
for file in dist.files:
path = file.locate()
if path == target:
print(f"{dist.metadata['name']}: {dist.version}")
break

To be honest, if anyone were interested in making a PEP from any of
this, having Python modules contain a __distribution_name__ attribute
that links the module back to the PyPI distribution, would probably be
more useful than standardising __version__. But I'm not sufficiently
interested to do anything more than mention that as a possibility.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/66G7A5ZZRYT7H6MRJVELFMC23IDP67CL/
Code of Conduct: http://python.org/psf/codeofconduct/