Mailing List Archive

On Wed, Apr 14, 2021 at 7:48 AM Christopher Barker <pythonchb@gmail.com> wrote:
> So what do you'all think? After thirteen years, it would be nice to put this to bed.

There are two main use cases for versions:

* Display them to the user
* Compare versions to check if one is newer, older or the same

I dislike using strings for comparison. You need to use
packaging.version for that:
https://packaging.pypa.io/en/latest/version.html

Many C libraries provide the version as a number of as 3 numbers
(major, minor, micro). In its C API, Python provides all of them:

* PY_VERSION_HEX: single number
* (PY_MAJOR_VERSION, PY_MINOR_VERSION, PY_MICRO_VERSION,
PY_RELEASE_LEVEL, PY_RELEASE_SERIAL): as 5 numbers
* PY_VERSION: string

In my Python projects, I like to provide the version as a tuple which
can be used directly for comparison: version_a <= version_b. Example:

VERSION = (2, 2, 1)
__version__ = '.'.join(map(str, VERSION))

The tuple might contain strings like "beta" or "rc", as soon as
comparison makes sense ;-) Sadly, such tuple is no standardized. Which
part is the major version? How to format it as a string?

Good luck with trying to standardize that ;-)

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MBBYB5AWX76O3TOUFATRKSU2QND2TPKS/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, 14 Apr 2021 at 13:28, Victor Stinner <vstinner@python.org> wrote:
> There are two main use cases for versions:
>
> * Display them to the user
> * Compare versions to check if one is newer, older or the same
>
> I dislike using strings for comparison. You need to use
> packaging.version for that:
> https://packaging.pypa.io/en/latest/version.html

Agreed, programmatic use should really conform to PEP 440 versioning,
as implemented in packaging.version, if it's to align with existing
packaging standards (which currently focus on *project* versions
rather than package versions, but the distinction is not something we
should be making a big deal of).

> Many C libraries provide the version as a number of as 3 numbers
> (major, minor, micro). In its C API, Python provides all of them:
>
> * PY_VERSION_HEX: single number
> * (PY_MAJOR_VERSION, PY_MINOR_VERSION, PY_MICRO_VERSION,
> PY_RELEASE_LEVEL, PY_RELEASE_SERIAL): as 5 numbers
> * PY_VERSION: string
>
> In my Python projects, I like to provide the version as a tuple which
> can be used directly for comparison: version_a <= version_b. Example:
>
> VERSION = (2, 2, 1)
> __version__ = '.'.join(map(str, VERSION))
>
> The tuple might contain strings like "beta" or "rc", as soon as
> comparison makes sense ;-) Sadly, such tuple is no standardized. Which
> part is the major version? How to format it as a string?
>
> Good luck with trying to standardize that ;-)

Agreed. We should stick to PEP 440 format strings, and not try to
dictate other formats like (named) tuples.

But does the PEP want to cover programmatic use *at all*? It would be
possible to define __version__ as "a human-readable string in PEP 440
format" and note that any use other than for display to a person is
outside the scope of the PEP. That's closer in spirit to the 13-year
old version we currently have, but it does miss an opportunity as we
now do have a standard version class ("standard" in the sense of "a
packaging standard", but not "in the stdlib").

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5LFCLXIPXF3PO6PZ5ZLS7TD2OE4U35OD/
Code of Conduct: http://python.org/psf/codeofconduct/

> In my Python projects, I like to provide the version as a tuple which
> can be used directly for comparison

To add to this, comparing tuples doesn't always work well for projects
where multiple release lines are maintained simultaneously, e.g.
user-facing changes introduced in minor/point releases across several major
versions. People use version numbers in wildly different ways.

Barney

On Wed, 14 Apr 2021 at 13:26, Victor Stinner <vstinner@python.org> wrote:

> On Wed, Apr 14, 2021 at 7:48 AM Christopher Barker <pythonchb@gmail.com>
> wrote:
> > So what do you'all think? After thirteen years, it would be nice to put
> this to bed.
>
> There are two main use cases for versions:
>
> * Display them to the user
> * Compare versions to check if one is newer, older or the same
>
> I dislike using strings for comparison. You need to use
> packaging.version for that:
> https://packaging.pypa.io/en/latest/version.html
>
> Many C libraries provide the version as a number of as 3 numbers
> (major, minor, micro). In its C API, Python provides all of them:
>
> * PY_VERSION_HEX: single number
> * (PY_MAJOR_VERSION, PY_MINOR_VERSION, PY_MICRO_VERSION,
> PY_RELEASE_LEVEL, PY_RELEASE_SERIAL): as 5 numbers
> * PY_VERSION: string
>
> In my Python projects, I like to provide the version as a tuple which
> can be used directly for comparison: version_a <= version_b. Example:
>
> VERSION = (2, 2, 1)
> __version__ = '.'.join(map(str, VERSION))
>
> The tuple might contain strings like "beta" or "rc", as soon as
> comparison makes sense ;-) Sadly, such tuple is no standardized. Which
> part is the major version? How to format it as a string?
>
> Good luck with trying to standardize that ;-)
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/MBBYB5AWX76O3TOUFATRKSU2QND2TPKS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

I recently encountered this, which is very useful, but only for a
human-readable perspective.

>>> import vaex
>>> vaex.__version__
{'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0', 'vaex-hdf5':
'0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0', 'vaex-jupyter':
'0.6.0', 'vaex-ml': '0.11.1'}



On Wed, Apr 14, 2021 at 2:01 PM Barney Gale <barney.gale@gmail.com> wrote:

> > In my Python projects, I like to provide the version as a tuple which
> > can be used directly for comparison
>
> To add to this, comparing tuples doesn't always work well for projects
> where multiple release lines are maintained simultaneously, e.g.
> user-facing changes introduced in minor/point releases across several major
> versions. People use version numbers in wildly different ways.
>
> Barney
>
> On Wed, 14 Apr 2021 at 13:26, Victor Stinner <vstinner@python.org> wrote:
>
>> On Wed, Apr 14, 2021 at 7:48 AM Christopher Barker <pythonchb@gmail.com>
>> wrote:
>> > So what do you'all think? After thirteen years, it would be nice to put
>> this to bed.
>>
>> There are two main use cases for versions:
>>
>> * Display them to the user
>> * Compare versions to check if one is newer, older or the same
>>
>> I dislike using strings for comparison. You need to use
>> packaging.version for that:
>> https://packaging.pypa.io/en/latest/version.html
>>
>> Many C libraries provide the version as a number of as 3 numbers
>> (major, minor, micro). In its C API, Python provides all of them:
>>
>> * PY_VERSION_HEX: single number
>> * (PY_MAJOR_VERSION, PY_MINOR_VERSION, PY_MICRO_VERSION,
>> PY_RELEASE_LEVEL, PY_RELEASE_SERIAL): as 5 numbers
>> * PY_VERSION: string
>>
>> In my Python projects, I like to provide the version as a tuple which
>> can be used directly for comparison: version_a <= version_b. Example:
>>
>> VERSION = (2, 2, 1)
>> __version__ = '.'.join(map(str, VERSION))
>>
>> The tuple might contain strings like "beta" or "rc", as soon as
>> comparison makes sense ;-) Sadly, such tuple is no standardized. Which
>> part is the major version? How to format it as a string?
>>
>> Good luck with trying to standardize that ;-)
>>
>> Victor
>> --
>> Night gathers, and now my watch begins. It shall not end until my death.
>> _______________________________________________
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-leave@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/MBBYB5AWX76O3TOUFATRKSU2QND2TPKS/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4XWIWIKDJJRWVKNGIOENJ2ZJVQR4PP23/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Wed, Apr 14, 2021 at 7:48 AM David Mertz <mertz@gnosis.cx> wrote:

> >>> vaex.__version__
> {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0', 'vaex-hdf5':
> '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0', 'vaex-jupyter':
> '0.6.0', 'vaex-ml': '0.11.1'}
>

Well, THAT is a great argument for some official standardization!

There is sometimes a need for that sort of thing, but I think it's best
handled by either putting __version__ in each sub_package, or having a
different attribute altogether.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

Simon,

I'm not sure your note made it to the list -- bringing it back on.

On Wed, Apr 14, 2021 at 1:15 AM Simon Cross <hodgestar+pythondev@gmail.com>
wrote:

> I would be +1 on having a small PEP like this one to give __version__
> an official blessing and some well-defined expected behaviour.
>
> I don't have a deep opinion on the specifics of the specification, but
> I'm guessing they should be thought about again even if the answers
> end up being the same. Quite a lot has changed in the Python ecosystem
> in the last 13 years.
>

Yes indeed. Critically, over on the packaging side, standardization of
versioning specs.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

Agreed!

I was trying to figure out why an API from Vaex 3.x was no longer working,
and to my human eye, this quickly pointed me at the issue. However as a
way to automatically check for versions, this is a mess. I have no idea
what additional keys the next version might add or remove, for example.
I'm not even sure if this is dynamically determined based on optional
components being installed or not.

Providing this kind of information *somehow* feels like a useful thing to
do. But .__version__ is probably not the right way to do it.

On Wed, Apr 14, 2021 at 5:08 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> On Wed, Apr 14, 2021 at 7:48 AM David Mertz <mertz@gnosis.cx> wrote:
>
>> >>> vaex.__version__
>> {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0', 'vaex-hdf5':
>> '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0', 'vaex-jupyter':
>> '0.6.0', 'vaex-ml': '0.11.1'}
>>
>
> Well, THAT is a great argument for some official standardization!
>
> There is sometimes a need for that sort of thing, but I think it's best
> handled by either putting __version__ in each sub_package, or having a
> different attribute altogether.
>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
>


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Wed, Apr 14, 2021 at 5:24 AM Victor Stinner <vstinner@python.org> wrote:

> There are two main use cases for versions:
>
> * Display them to the user
> * Compare versions to check if one is newer, older or the same
>
> I dislike using strings for comparison. You need to use
> packaging.version for that:
> https://packaging.pypa.io/en/latest/version.html


Yes, though it looks a little too heavyweight to be required, nifty
features like ``is_prerelease` is going pretty far.

On the other hand, maybe no harm in all that, as long as:

- It's easy to do the easy stuff (which is looks like it is)
- It would be added to the stdlib

However, there is a lot of code out there that's already using strings for
__version__, so it would be a real shame if we had to ask everyone to
upgrade. And Version isn't going to be in older versions of the Python
stdlib, so hard to make code compatible.

But: If we make a Version object that can be compared to (confoming)
strings then there might be a path forward.


> In my Python projects, I like to provide the version as a tuple which
> can be used directly for comparison: version_a <= version_b. Example:
>

Yes, and tuples are already built in to Python, so an easier lift than
adding the Version object.

Another possible issue: using Version would require an extra import in many
module initializations -- is that a performance issue that would matter?

In any case, I think the PEP should specify a standard for what is in
__version__

And ideally it would be backward compatible with strings, which are the
most commonly used currently.

And conform to PEP 440

Sadly, such tuple is no standardized. Which
> part is the major version? How to format it as a string?
>
> Good luck with trying to standardize that ;-)
>

well, PEP 440 gets us close -- and I *think* compliant strings could be
unambiguously converted to-from tuples.

Anyway, the question now for me is whether this is worth any more of my
time.

So:
- Is there someone willing to sponsor?
- Do folks generally think there's a chance of it being accepted without a
huge debate and expansion of scope.

If the answers are not yes, I can better spend my Python time of other
things.

BTW: one source of hope:

__version__ strings are pretty common, but, as evidenced already by this
brief discussion, it's not by any means a universal standard -- so we
really can't break anything that already works in any universal way.

What i mean by that is that anything done here might break someone's
special use case of __version__ (Like the one David Mertz identified), but
that scheme only works with that one package's code anyway. There are no
general tools that expect that scheme.

And it could be preserved by making a subclass of Version (Or str) that was
also a mapping :-)

- CHB



--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Wed, Apr 14, 2021 at 5:44 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> Another possible issue: using Version would require an extra import in
> many module initializations -- is that a performance issue that would
> matter?
>

I like having a `Version` object that is easily importable in the standard
library. I think it should relatively polymorphic. I.e. it should accept
a module or package as an argument, but also should accept a string or a
tuple. Maybe other objects from which one could reasonably extract a
version.

In particular, I think initializing this object with a module object should
at least look for a .__version__ attribute, and appropriately case either a
string (that looks sufficiently version-like) or a tuple. I think that if
it doesn't succeed, it should become some sort of "cannot determine" object
that is neither less than nor greater than any other Version object. In
particular, the Vaex example with a dictionary of versions of each
component should probably just become this "cannot determine" value (but as
an instance of Version).

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On 4/14/2021 12:38 PM, Christopher Barker wrote:
>
>
> Anyway, the question now for me is whether this is worth any more of
> my time.
>
> So:
> - Is there someone willing to sponsor?
> - Do folks generally think there's a chance of it being accepted
> without a huge debate and expansion of scope.
>
I think that before these can be answered, you need to decide why this
needs to be standardized. I don't see any reason to standardize it
unless there's some programmatic use for these version strings. If it's
just the user poking around on the REPL, then I think the status quo is
fine. I've read the PEP, and the User Stories section talks about user
convenience, not programmatic access.

I also think the distribution version is more useful than any
__version__ information in each module/package. I realize this
information might not be available, depending on how the code was
installed (say, just by copying some files into the right place).

Eric

On Apr 13, 2021, at 22:40, Christopher Barker <pythonchb@gmail.com> wrote:
>
> Turns out this was suggested in PEP 396 -- and deferred almost 13 years ago!
>
> https://www.python.org/dev/peps/pep-0396/

I’d forgotten that this PEP was in Deferred state. I think it should be rejected and I plan on making that change. importlib.metadata is a much better approach to programmatically determining package versions.

https://docs.python.org/3/library/importlib.metadata.html#distribution-versions

-Barry

On Wed, 14 Apr 2021 at 18:01, David Mertz <mertz@gnosis.cx> wrote:
>
> On Wed, Apr 14, 2021 at 5:44 PM Christopher Barker <pythonchb@gmail.com> wrote:
>>
>> Another possible issue: using Version would require an extra import in many module initializations -- is that a performance issue that would matter?
>
>
> I like having a `Version` object that is easily importable in the standard library. I think it should relatively polymorphic. I.e. it should accept a module or package as an argument, but also should accept a string or a tuple. Maybe other objects from which one could reasonably extract a version.

If it's not basically equivalent to packaging.version.Version (and
based on PEP 440) then we'll be creating a nightmare of confusion,
because PEP 440 versions are fundamental to packaging.

> In particular, I think initializing this object with a module object should at least look for a .__version__ attribute, and appropriately case either a string (that looks sufficiently version-like) or a tuple. I think that if it doesn't succeed, it should become some sort of "cannot determine" object that is neither less than nor greater than any other Version object. In particular, the Vaex example with a dictionary of versions of each component should probably just become this "cannot determine" value (but as an instance of Version).

What's wrong with Version(module.__version__)? And if the __version__
attribute isn't a valid version, raise an exception? That's what
packaging.version does, and it's worked fine for the packaging
ecosystem. Is there a benefit that justifies being different here?

Can I remind people that the packaging community have done a *huge*
amount of work designing and standardising concepts like version
identifiers, version comparisons and constraints, etc. (PEPs 440 and
508). Understanding those standards should really be the basic
starting point for any discussion like this.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U2PUBZZ2AULMULLESO3LZ74MCRV7BQ3G/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, 14 Apr 2021 at 18:04, Eric V. Smith <eric@trueblade.com> wrote:
>> Anyway, the question now for me is whether this is worth any more of my time.
>>
>> So:
>> - Is there someone willing to sponsor?
>> - Do folks generally think there's a chance of it being accepted without a huge debate and expansion of scope.
>>
>> I think that before these can be answered, you need to decide why this needs to be standardized. I don't see any reason to standardize it unless there's some programmatic use for these version strings. If it's just the user poking around on the REPL, then I think the status quo is fine. I've read the PEP, and the User Stories section talks about user convenience, not programmatic access.
>
> I also think the distribution version is more useful than any __version__ information in each module/package. I realize this information might not be available, depending on how the code was installed (say, just by copying some files into the right place).

Agreed. The original PEP was very limited, just proposing a string
value for user convenience, with no programmatic interface. Doing
anything more than that is opening up a whole load of complexity and
compatibility (with packaging) that no-one has expressed any need for.

The distribution version (exposed via importlib.metadata) is the
correct thing to use for anything programmatic.

Paul

PS I see Barry plans on rejecting the PEP, which I think is probably
the right decision, given the way this thread has developed.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MQIFSJYQYCY4YBHVECJ7H2WR443EE32W/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Apr 14, 2021 at 4:19 PM Paul Moore <p.f.moore@gmail.com> wrote:

> PS I see Barry plans on rejecting the PEP, which I think is probably
> the right decision, given the way this thread has developed.
>

Barry makes good plans.

Putting the version into the sources is a bit of an anti-pattern. IMO.


-Fred

--
Fred L. Drake, Jr. <fred at fdrake.net>
"There is nothing more uncommon than common sense."
--Frank Lloyd Wright

On Wed, Apr 14, 2021 at 9:12 PM Paul Moore <p.f.moore@gmail.com> wrote:

> If it's not basically equivalent to packaging.version.Version (and
> based on PEP 440) then we'll be creating a nightmare of confusion,
> because PEP 440 versions are fundamental to packaging.
>

Are you suggesting that users should have to install an external module to
tell what version of packages they are using?!

What's wrong with Version(module.__version__)? And if the __version__
> attribute isn't a valid version, raise an exception? That's what
> packaging.version does, and it's worked fine for the packaging
> ecosystem. Is there a benefit that justifies being different here?
>

Doesn't that seem really, really painful to use in an interactive shell?
This is honestly (a simplified version of) what I tried in trying to learn
packaging.version.Version. It was not a good experience:

% conda create -n test python=3.9 pandas
[...]
% conda activate test
% python
>>> import packaging
ModuleNotFoundError: No module named 'packaging'
^D
% pip install packaging
[...]
% python
>>> import packaging
>>> import panda, numpy, re, statistics
>>> packaging.version.Version(pandas)
AttributeError: module 'packaging' has no attribute 'version'
>>> from packaging.version import Version
>>> Version(pandas)
TypeError: expected string or bytes-like object
>>> Version(pandas.__version__)
<Version('1.2.4')>
>>> Version(statistics.__version__)
AttributeError: module 'statistics' has no attribute '__version__'
>>> Version(re.__version__)
<Version('2.2.1')>
>>> import vaex
>>> Version(vaex.__version__)
TypeError: expected string or bytes-like object

I cannot find a case where the more verbose spelling is ever nicer than
just referencing `mod.__version__` directly for such exploration. Yes, I
get that formal packaging has different needs. But here it just seems like
a lot more work to get a lot less information back.

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Wed, 14 Apr 2021 at 21:59, David Mertz <mertz@gnosis.cx> wrote:
>
> On Wed, Apr 14, 2021 at 9:12 PM Paul Moore <p.f.moore@gmail.com> wrote:
>>
>> If it's not basically equivalent to packaging.version.Version (and
>> based on PEP 440) then we'll be creating a nightmare of confusion,
>> because PEP 440 versions are fundamental to packaging.
>
> Are you suggesting that users should have to install an external module to tell what version of packages they are using?!

No. To tell what version of a package they are using, a string is sufficient.

They only need a version object if they want to do additional
processing (like comparing versions, or checking whether a version
meets a constraint).

Given that the packaging ecosystem already has a canonical version
object (provided by the packaging library), which has been used and
tested extensively in many environments, inventing a different API
seems at best ill-advised. Whether the stdlib needs a version object.
rather than leaving that functionality to a 3rd party library, is the
same question that comes up for *any* functionality that's proposed
for the stdlib, and I have no particular opinion in this case.

>> What's wrong with Version(module.__version__)? And if the __version__
>> attribute isn't a valid version, raise an exception? That's what
>> packaging.version does, and it's worked fine for the packaging
>> ecosystem. Is there a benefit that justifies being different here?
>
> Doesn't that seem really, really painful to use in an interactive shell? This is honestly (a simplified version of) what I tried in trying to learn packaging.version.Version. It was not a good experience:

It's designed for programmatic use, not interactive use, yes. But
that's sort of my point, why do you want anything more than the bare
string in the REPL? What are you planning on doing with it?

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6424E3GOVU463R7SOFQQXCWKV2M3LZRV/
Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, 14 Apr 2021 22:23:44 +0100
Paul Moore <p.f.moore@gmail.com> wrote:
> On Wed, 14 Apr 2021 at 21:59, David Mertz <mertz@gnosis.cx> wrote:
> >
> > On Wed, Apr 14, 2021 at 9:12 PM Paul Moore <p.f.moore@gmail.com> wrote:
> >>
> >> If it's not basically equivalent to packaging.version.Version (and
> >> based on PEP 440) then we'll be creating a nightmare of confusion,
> >> because PEP 440 versions are fundamental to packaging.
> >
> > Are you suggesting that users should have to install an external module to tell what version of packages they are using?!
>
> No. To tell what version of a package they are using, a string is sufficient.
>
> They only need a version object if they want to do additional
> processing (like comparing versions, or checking whether a version
> meets a constraint).
>
> Given that the packaging ecosystem already has a canonical version
> object (provided by the packaging library), which has been used and
> tested extensively in many environments, inventing a different API
> seems at best ill-advised. Whether the stdlib needs a version object.
> rather than leaving that functionality to a 3rd party library, is the
> same question that comes up for *any* functionality that's proposed
> for the stdlib, and I have no particular opinion in this case.

Tangentially, until now projects could use distutils's LooseVersion if
they wanted to compare version numbers reliably. With distutils being
deprecated, they'll have to either depending on packaging (which is a
large dependency just for comparison version numbers) or vendor
packaging's Version class (which is doable but still some bothersome
additional work).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T4J2JD454XP3ZGULM777H5EG5Z3WVNMJ/
Code of Conduct: http://python.org/psf/codeofconduct/

Seems like this is something that should make its way into stdlib?

On Thu, 2021-04-15 at 00:15 +0200, Antoine Pitrou wrote:
> On Wed, 14 Apr 2021 22:23:44 +0100
> Paul Moore <p.f.moore@gmail.com> wrote:
> > On Wed, 14 Apr 2021 at 21:59, David Mertz <mertz@gnosis.cx> wrote:
> > >
> > > On Wed, Apr 14, 2021 at 9:12 PM Paul Moore <p.f.moore@gmail.com>
> > > wrote: 
> > > >
> > > > If it's not basically equivalent to packaging.version.Version
> > > > (and
> > > > based on PEP 440) then we'll be creating a nightmare of
> > > > confusion,
> > > > because PEP 440 versions are fundamental to packaging. 
> > >
> > > Are you suggesting that users should have to install an external
> > > module to tell what version of packages they are using?! 
> >
> > No. To tell what version of a package they are using, a string is
> > sufficient.
> >
> > They only need a version object if they want to do additional
> > processing (like comparing versions, or checking whether a version
> > meets a constraint).
> >
> > Given that the packaging ecosystem already has a canonical version
> > object (provided by the packaging library), which has been used and
> > tested extensively in many environments, inventing a different API
> > seems at best ill-advised. Whether the stdlib needs a version
> > object.
> > rather than leaving that functionality to a 3rd party library, is
> > the
> > same question that comes up for *any* functionality that's proposed
> > for the stdlib, and I have no particular opinion in this case.
>
> Tangentially, until now projects could use distutils's LooseVersion
> if
> they wanted to compare version numbers reliably.  With distutils
> being
> deprecated, they'll have to either depending on packaging (which is a
> large dependency just for comparison version numbers) or vendor
> packaging's Version class (which is doable but still some bothersome
> additional work).
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/T4J2JD454XP3ZGULM777H5EG5Z3WVNMJ/
> Code of Conduct: http://python.org/psf/codeofconduct/

Paul Moore wrote:
> What's wrong with Version(module.__version__)? And if the __version__
> attribute isn't a valid version, raise an exception?

I don't have a deep answer, but I do think __version__ should be specified (or at least mentioned) at https://docs.python.org/3/reference/datamodel.html

At the moment, I can't even find a listing of possible __dunder__ attributes, though I'm almost sure I've seen one in the past.

-jJ
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AG344CLSKM7BDOYP3CWGXGBNFSPWVQU4/
Code of Conduct: http://python.org/psf/codeofconduct/

https://docs.python.org/3/reference/datamodel.html

On Wed, 2021-04-14 at 22:56 +0000, Jim J. Jewett wrote:
> Paul Moore wrote:
> > What's wrong with Version(module.__version__)? And if the
> > __version__
> > attribute isn't a valid version, raise an exception?
>
> I don't have a deep answer, but I do think __version__ should be
> specified (or at least mentioned) at
> https://docs.python.org/3/reference/datamodel.html
>
> At the moment, I can't even find a listing of possible __dunder__
> attributes, though I'm almost sure I've seen one in the past.
>
> -jJ
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/AG344CLSKM7BDOYP3CWGXGBNFSPWVQU4/
> Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Apr 14, 2021 at 7:04 PM Jim J. Jewett <jimjjewett@gmail.com> wrote:

> I don't have a deep answer, but I do think __version__ should be specified
> (or at least mentioned) at
> https://docs.python.org/3/reference/datamodel.html


Given the intent to reject PEP 394, I can't imagine any good would come of
that.

Drop a __version__ in your modules if you find it valuable, but otherwise,
there's nothing to do here.


-Fred

--
Fred L. Drake, Jr. <fred at fdrake.net>
"There is nothing more uncommon than common sense."
--Frank Lloyd Wright

On Wed, Apr 14, 2021 at 12:10 PM Barry Warsaw <barry@python.org> wrote:

I’d forgotten that this PEP was in Deferred state. I think it should be
> rejected and I plan on making that change. importlib.metadata is a much
> better approach to programmatically determining package versions.
>
>
> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions


Barry,

You wrote the original PEP, so of course you can withdraw it (or reject
it), but...

Are you sure? See this discussion, I don't think it's as simple as all that.

Honestly, I'm still kind of confused about a "distribution" vs a package or
module, but in general, it seems to me that the packaging community has
done a great job of building a system that can accommodate a lot of
complexity, but as they say:

The easy things should be easy, and the hard things should be possible.

And the full process of using packaging and distributions, and a Version
object and all that is not so easy anymore --at least compared to putting:

__version__ = "1.2.3"

in a package __init__.py.

Anyway, a couple points:

1) it seems self-evident that it would be easier for the whole community if
"how to declare", and "how to find" the version of a package was
standardized: "there's only one way to do it"

1b) It seems like a really good idea that the (or one of the) official
recommended way to do it is supported in the standard library itself, and
using a string is pretty easy to support :-) -- do we really want people to
have to install a third party package to set or get the version of a simple
module?

2) __version__ strings are already pretty darn common -- maybe more common
than proper distribution metadata? -- are we saying that everyone should
stop doing that?

3) I don't know that having __version__ strings is incompatible with
packging.version.Version or importlib.metadata.version

4) Yes, there are real advantages to the more complex system for
programmatically working with versions. But there are real advantages to
something simple for human readability / interactive use -- and, frankly,
simple scripting. See David Mertz's recent post -- I went through a similar
process in iPython.

Consider this analogy: if I want to know what version of git I have
installed, I can run:

$ git --version

And that's a convention adhered to by many *nix command line tools. Would
we all really be happier if we had to go use an rpm or deb or homebrew, or
?? command to see what package i have installed? I think this is similar --
while a good distribution management system is a good thing, it's also good
if I can tell something about the version of a package/module from
the package itself.

With regard to having both __version__ and importlib.metadata.version, see
beautiful soup for example:

In [10]: import beautifulsoup4
...
ModuleNotFoundError: No module named 'beautifulsoup4'

oops, that's not what the importable module is called. So:

In [11]: import bs4

That worked -- now, what version is it?

In [12]: importlib.metadata.version('bs4')
---------------------------------------------------------------------------
PackageNotFoundError Traceback (most recent call last)
...
PackageNotFoundError: bs4

[*]

That didn't work -- I guess I need to use the distribution name:

In [13]: importlib.metadata.version('beautifulsoup4')
Out[13]: '4.9.3'

OK, that worked.

But I imported "bs4", so what if I do this?

In [14]: bs4.__version__
Out[14]: '4.9.3'

Ah yes, that worked too.

So I guess what I'm suggesting is that we officially recommend that folks
do what the BeautifulSoup folks, and many others, are already doing. In
many (most) cases, the distribution name and the importable package name
are the same, but this does accommodate both.

[*] One other annoyance to importlib.metadata.version is that in addition
to having to import importlib to use it, I also need to use
importlib.metadata.PackageNotFoundError if I want to trap the exception.

BTW: I'm not making that last point up -- I recently updated some in-house
code (that someone else had written) in a web app that presented the
version of various packages core to the system. We had just refactor and
one of these packages what now optional, but the whole system was crashing
on that darn PackageNotFoundError. I ended up jsut commenting out that
package, rather than dealing with how to catch the exception and deal with
it properly. I would have so much rathered a simple:

try:
import the_package
the_pacakge_version = the_package.__version__
except
the_package_version = None

Granted, it's not that hard to "do it right", but I still like things
simple :-)

Anyway, as the original PEP author is no longer supportive, this is dead in
the water unless another core dev is interested in sponsoring this (or a
new) PEP. If not, I can stop wasting my and everyone else's time.

-Chris

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Wed, Apr 14, 2021 at 2:23 PM Paul Moore <p.f.moore@gmail.com> wrote:

> >> because PEP 440 versions are fundamental to packaging.
> >
> > Are you suggesting that users should have to install an external module
> to tell what version of packages they are using?!
>
> No. To tell what version of a package they are using, a string is
> sufficient.
>

exactly -- if we do anything at all here, it should probably be at least.

if you provide a __version__ attribute, it should be a PEP 440 compliant
string.

That would at least make the use __version__ a touch more consistent.

The next, more controversial, step would be to suggest that people SHOULD
provide a __version__string on all top-level packages. I would like that --
the current situation where many, but not all packages have __version__ is
really annoying.

They only need a version object if they want to do additional
> processing (like comparing versions, or checking whether a version
> meets a constraint).
>

And indeed, we could add the ability for packaging.version.Version objects
to be able to compare themselves with compatible strings -- I think that
would be pretty handy.

Given that the packaging ecosystem already has a canonical version
> object (provided by the packaging library), which has been used and
> tested extensively in many environments, inventing a different API
> seems at best ill-advised.


absolutely.


> Whether the stdlib needs a version object.
> rather than leaving that functionality to a 3rd party library, is the
> same question that comes up for *any* functionality that's proposed
> for the stdlib, and I have no particular opinion in this case.
>

I don't think it's the same as any functionality -- if we do want to better
standardize package versioning in Python, and I think we should, then the
Version object may become something useful to, and maybe even needed by,
virtually every third party package. Which makes it a pretty prime
candidate for the stdlib.

Alternatively, the packaging package is pretty small, but if it grows, it
might be good to split out the run-time vs build-time pieces.

It's designed for programmatic use, not interactive use, yes. But
> that's sort of my point, why do you want anything more than the bare
> string in the REPL? What are you planning on doing with it?
>

there is something in between the REPL and full on system development --
something simple for quickly scripts is nice too. But a simple standardised
version string is fine for that.

-CHB


--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

Paul Bryan:
> Seems like this is something that should make its way into stdlib?

In the last 10 years, the trend is more to remove anything related to
packaging *outside* the stdlib :-) Since it's evolving way faster than
Python releases and the stdlib cannot be updated once installed, I
don't think that it's a good idea.

On Thu, Apr 15, 2021 at 12:22 AM Antoine Pitrou <antoine@python.org> wrote:
> Tangentially, until now projects could use distutils's LooseVersion if
> they wanted to compare version numbers reliably. With distutils being
> deprecated, they'll have to either depending on packaging (which is a
> large dependency just for comparison version numbers) or vendor
> packaging's Version class (which is doable but still some bothersome
> additional work).

If packaging is too big and if packaging maintainters like the idea,
maybe packaging.version could be moved into a dedicated package? I
didn't look if it makes sense from a technical point of view.

$ wc -l packaging/version.py packaging/_structures.py
556 packaging/version.py
86 packaging/_structures.py
642 total

version.py uses _structures.py (InfinityType, NegativeInfinityType).

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GT6NZDIKA222ADKKSJN7W4WZROMAIRDF/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, 15 Apr 2021 11:37:28 +0200
Victor Stinner <vstinner@python.org> wrote:
> Paul Bryan:
> > Seems like this is something that should make its way into stdlib?
>
> In the last 10 years, the trend is more to remove anything related to
> packaging *outside* the stdlib :-) Since it's evolving way faster

What is "evolving way faster"? packaging.version implements PEP 440,
which has been approved in 2014. Recent changes have been really minor:
https://github.com/pypa/packaging/commits/main/packaging/version.py



_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NFPGSSDTD7SNZI4XJF542NS4R7OXVVMZ/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org> wrote:

> Paul Bryan:
> > Seems like this is something that should make its way into stdlib?
>
> In the last 10 years, the trend is more to remove anything related to
> packaging *outside* the stdlib :-)


Well, maybe all of us don't think that's a good idea ;-)

But anyway, I would say removing anything *related* to packaging outside
the stdlib is a bad idea -- requiring an external tool to build a package
is OK, but requireing an external use packages, not so much.

Presumably that's why importlib.metadata exists in the stdlib.

Version is arguably useful from the package user side. As I believe Victor
mentioned, there are two uses for version information: display to the user
-- for which version strings are fine, or programmatic comparison -- for
which something like the Version object is very helpful. Do we only need to
use version information programmatically when we are creating (or
installing) packages? I don't think so -- I know I have code that (poorly)
does version checking programmatically.

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 4:55 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> Presumably that's why importlib.metadata exists in the stdlib.
>

I was so hopeful about this, but in the end... not really. I have not used
this capability before. Here are a few different situations I know of:

>>> import re, statistics, pandas, vaex, bs4
>>> for mod in [re, statistics, pandas, vaex, bs4]:
... try:
... print(mod.__name__, end=" ")
... print(mod.__version__)
... except:
... print("__version__ not present")
...
re 2.2.1
statistics __version__ not present
pandas 1.2.4
vaex {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0',
'vaex-hdf5': '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0',
'vaex-jupyter': '0.6.0', 'vaex-ml': '0.11.1'}
bs4 4.9.3

>>> for mod in [re, statistics, pandas, vaex, bs4]:
... try:
... print(mod.__name__, version(mod.__name__))
... except Exception as err:
... print(mod.__name__, repr(err))
...
re PackageNotFoundError('re')
statistics PackageNotFoundError('statistics')
pandas 1.2.4
vaex 4.1.0
bs4 PackageNotFoundError('bs4')

It seems like (somehow or another) importlib.metadata is doing something
perhaps more useful for vaex. But it is doing distinctly worse for re,
statistics, and bs4.

Version is arguably useful from the package user side. As I believe Victor
> mentioned, there are two uses for version information: display to the user
> -- for which version strings are fine, or programmatic comparison -- for
> which something like the Version object is very helpful. Do we only need to
> use version information programmatically when we are creating (or
> installing) packages? I don't think so -- I know I have code that (poorly)
> does version checking programmatically.
>

I think the key thing that Chris and I are pointing to is that there is a
definite use for versions that is NOT for package maintainers/creators.
Interactive use is definitely one such case, and eyeballing something, even
if it is the oddball use that Vaex has, can help with figuring out an
issue. But in actual code, I relatively often do something like this. Or
rather, the below is what I would find really nice to be able to do.

ver = robust_version(module)
if ver >= (5, 2, 1):
doit_modern_style()
elif ver < (5, 2, 1):
doit_old_style
else:
doit_unversioned_style()

I'd like code like that to ALWAYS succeed. No exceptions raised. And it
would *usually* use reasonable heuristics to figure out some sort of
structured version info as well as is possible. I.e. probably look in
several places and return some custom object that represents a "best
effort." That object might be like NaN in being neither less than nor more
than other things, in the fallback case.

--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>
> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>
> Are you sure? See this discussion, I don't think it's as simple as all that.

From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

PEP 396 has been dormant for a long time and I have no interest in pushing it forward any more. If someone else wants to take up the cause, I recommend creating a new PEP and referring back to 396.

-Barry

On Thu, 15 Apr 2021 09:38:41 -0700
Barry Warsaw <barry@python.org> wrote:
> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
> >
> > You wrote the original PEP, so of course you can withdraw it (or reject it), but...
> >
> > Are you sure? See this discussion, I don't think it's as simple as all that.
>
> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

This is often handled using `setuptools_scm` or `versioneer` (see PyPI
for each of these packages).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J74GLEQS4HHSUQU3WO57GAURKQWWTTJP/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 9:36 AM David Mertz <mertz@gnosis.cx> wrote:

> On Thu, Apr 15, 2021 at 4:55 PM Christopher Barker <pythonchb@gmail.com>
> wrote:
>
>> Presumably that's why importlib.metadata exists in the stdlib.
>>
>
> I was so hopeful about this, but in the end... not really. I have not
> used this capability before. Here are a few different situations I know of:
>
> >>> import re, statistics, pandas, vaex, bs4
> >>> for mod in [re, statistics, pandas, vaex, bs4]:
> ... try:
> ... print(mod.__name__, end=" ")
> ... print(mod.__version__)
> ... except:
> ... print("__version__ not present")
> ...
> re 2.2.1
> statistics __version__ not present
> pandas 1.2.4
> vaex {'vaex': '4.1.0', 'vaex-core': '4.1.0', 'vaex-viz': '0.5.0',
> 'vaex-hdf5': '0.7.0', 'vaex-server': '0.4.0', 'vaex-astro': '0.8.0',
> 'vaex-jupyter': '0.6.0', 'vaex-ml': '0.11.1'}
> bs4 4.9.3
>
> >>> for mod in [re, statistics, pandas, vaex, bs4]:
> ... try:
> ... print(mod.__name__, version(mod.__name__))
> ... except Exception as err:
> ... print(mod.__name__, repr(err))
> ...
> re PackageNotFoundError('re')
> statistics PackageNotFoundError('statistics')
> pandas 1.2.4
> vaex 4.1.0
> bs4 PackageNotFoundError('bs4')
>
> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex. But it is doing distinctly worse for re,
> statistics, and bs4.
>

I don't see how importlib.metadata plays into your code sample since you
didn't import it to use it?


>
> Version is arguably useful from the package user side. As I believe Victor
>> mentioned, there are two uses for version information: display to the user
>> -- for which version strings are fine, or programmatic comparison -- for
>> which something like the Version object is very helpful. Do we only need to
>> use version information programmatically when we are creating (or
>> installing) packages? I don't think so -- I know I have code that (poorly)
>> does version checking programmatically.
>>
>
And is installing a dependency that much of a burden for that code?


>
> I think the key thing that Chris and I are pointing to is that there is a
> definite use for versions that is NOT for package maintainers/creators.
> Interactive use is definitely one such case, and eyeballing something, even
> if it is the oddball use that Vaex has, can help with figuring out an
> issue. But in actual code, I relatively often do something like this. Or
> rather, the below is what I would find really nice to be able to do.
>
> ver = robust_version(module)
> if ver >= (5, 2, 1):
> doit_modern_style()
> elif ver < (5, 2, 1):
> doit_old_style
> else:
> doit_unversioned_style()
>
> I'd like code like that to ALWAYS succeed. No exceptions raised. And it
> would *usually* use reasonable heuristics to figure out some sort of
> structured version info as well as is possible. I.e. probably look in
> several places and return some custom object that represents a "best
> effort." That object might be like NaN in being neither less than nor more
> than other things, in the fallback case.
>

All doable with importlib.metadata and 'packaging'.

-Brett


>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born. Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4575HZHNN45FNOTKH5DSD6NKNRUJGXFU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
wrote:

> On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> wrote:
>
>> Paul Bryan:
>> > Seems like this is something that should make its way into stdlib?
>>
>> In the last 10 years, the trend is more to remove anything related to
>> packaging *outside* the stdlib :-)
>
>
> Well, maybe all of us don't think that's a good idea ;-)
>

As an active participant in the PyPA side of the packaging ecosystem and
one of the maintainers of 'packaging', I am definitely NOT in favour of
moving any of 'packaging' into the stdlib, nor breaking it up. We are
taking distutils out of the stdlib for a reason. Backsliding on plans that
have been in place for years is not something I would want to see happen as
the motivations have not changed.


>
> But anyway, I would say removing anything *related* to packaging outside
> the stdlib is a bad idea -- requiring an external tool to build a package
> is OK, but requireing an external use packages, not so much.
>

You can totally use a package already as things exist today and will in the
future; you're just wanting a specific way to interpret a package's
metadata which I don't view as the same thing.


>
> Presumably that's why importlib.metadata exists in the stdlib.
>

It's there because of how tightly it binds to the import system; same goes
for importlib.resources.


>
> Version is arguably useful from the package user side. As I believe Victor
> mentioned, there are two uses for version information: display to the user
> -- for which version strings are fine, or programmatic comparison -- for
> which something like the Version object is very helpful. Do we only need to
> use version information programmatically when we are creating (or
> installing) packages? I don't think so -- I know I have code that (poorly)
> does version checking programmatically.
>

No one said that's the *only* use-case, but just because there's more than
that does not mean we should move any packaging code into the stdlib to
support a zero-install use-case.

A key point I want to make here is the version metadata is in every
project's .dist-info/METADATA file. That's what importlib.metadata knows
how to read and hence why people have been suggesting to use it. The
version information is fundamental to the wheel format and in fact you
can't have a wheel without it. So asking every Python project to set a
__version__ isn't going to change accessibility of version numbers when it
comes to installed projects.

On Thu, 15 Apr 2021 10:28:53 -0700
Brett Cannon <brett@python.org> wrote:
> On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
> wrote:
>
> > On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> > wrote:
> >
> >> Paul Bryan:
> >> > Seems like this is something that should make its way into stdlib?
> >>
> >> In the last 10 years, the trend is more to remove anything related to
> >> packaging *outside* the stdlib :-)
> >
> >
> > Well, maybe all of us don't think that's a good idea ;-)
> >
>
> As an active participant in the PyPA side of the packaging ecosystem and
> one of the maintainers of 'packaging', I am definitely NOT in favour of
> moving any of 'packaging' into the stdlib, nor breaking it up. We are
> taking distutils out of the stdlib for a reason. Backsliding on plans that
> have been in place for years is not something I would want to see happen as
> the motivations have not changed.

This seems gratuitously dogmatic. Version numbers are a simple feature
that is stably encoded in PEP 440. It's purely computational, does
not involve reading or saving any persistent state in the filesystem,
making any network access, etc. Why wouldn't it have its place in the
stdlib?

> > But anyway, I would say removing anything *related* to packaging outside
> > the stdlib is a bad idea -- requiring an external tool to build a package
> > is OK, but requireing an external use packages, not so much.
> >
>
> You can totally use a package already as things exist today and will in the
> future; you're just wanting a specific way to interpret a package's
> metadata which I don't view as the same thing.

That specific way is a Python standard (PEP 440). Having the
functionality in the stdlib would encourage people to use it. Not
having it in the stdlib encourages people to use adhoc version parsing,
or worse, naive string comparison.

> So asking every Python project to set a
> __version__ isn't going to change accessibility of version numbers when it
> comes to installed projects.

That doesn't have much to do with the suggestion of shipping a Version
class in the stdlib, though. Many projects already provide a
__version__, and that field is often inspected programmatically in
dependent packages (to work around behaviour changes, for example).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JAZRZL4FPJ7BW763TVKZVPY2OQBHQI5V/
Code of Conduct: http://python.org/psf/codeofconduct/

On Apr 15, 2021, at 09:54, Antoine Pitrou <antoine@python.org> wrote:
>
> On Thu, 15 Apr 2021 09:38:41 -0700
> Barry Warsaw <barry@python.org> wrote:
>> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>>>
>>> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>>>
>>> Are you sure? See this discussion, I don't think it's as simple as all that.
>>
>> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.
>
> This is often handled using `setuptools_scm` or `versioneer` (see PyPI
> for each of these packages).

Yep, exactly. (I knew about and had in mind setuptools_scm but I didn’t know about versioneer; thanks for that reference!)

-Barry

Replying to the original email because my comment is more a thought
after reading the whole thread rather than a response to any
particular message.

What I love about __version__ is that it is simple. It doesn't come
with the complexity of solutions like `importlib.metadata.version`.
The maintainers of the project arranged for it to be set**. It doesn't
depend on how the module was packaged or distributed.

I also don't think it's really a language update or requires any
changes in the stdlib -- it's just a standard for the community to
follow where none currently exists.

It's also probably far from the end of the world*** if it doesn't
happen, but I still like the idea.

** It might have been updated somewhat automatically -- my point here
is that distutils, setuptools, packaging, pip, wheels, PyPI were all
not involved.

*** Although that is likely true of most PEPs, even much more
complicated ones. :)

Schiavo,
Simon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O2LGPEIOZMJYDJ4GMZXMOQK7GDWYS6X5/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/2021 12:35 PM, David Mertz wrote:

> re 2.2.1

re.__version__ was last modified 2001-10-07 by F. Lundh in
bec95b9d8825b39cff46a8c645fa0eeb8409854e. What does '2.2.1' mean? Only
Fredrik knows. The commit message says that he rewrote the pattern.sub
and pattern.subn methods in C. That should have been user invisible
except for speed.

I believe that it is dead data left only for back compatibility and
because of an inability to give a deprecation warning.

> re PackageNotFoundError('re')

> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex.  But it is doing distinctly worse for re,

Is an effectively meaningless and misleading value better than none?

IDLE once had a version last set, I believe, in 2.5 or 2.6, by K.
Kaiser, with whatever meaning it had to him. I removed it in 3.6
(allowed by PEP 434 + RM decision). No one complained.

The real version of any stdlib module is the Python version it is
released with, plus and external app versions it is associated with
(sqlite and ssl, for instance). It has been suggested, if not decided,
that other versions should go.


--
Terry Jan Reedy


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4AA7I3RS6D3UDIBZ6ELTEEWXKXIGE5WS/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/2021 12:38 PM, Barry Warsaw wrote:
> On Apr 14, 2021, at 23:11, Christopher Barker <pythonchb@gmail.com> wrote:
>>
>> You wrote the original PEP, so of course you can withdraw it (or reject it), but...
>>
>> Are you sure? See this discussion, I don't think it's as simple as all
that.
>
> From a library maintainers point of view, I personally want to get away from using __version__ strings at all. They’re kind of a pain to remember to bump on every new release.

At least some in the stdlib have not been. For re.__version__, not
since Nov 2001.


--
Terry Jan Reedy


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3UNG6KD3JIBHSMWILZGGIFHRSQV6PDKG/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 9:38 AM Barry Warsaw <barry@python.org> wrote:

> From a library maintainers point of view, I personally want to get away
> from using __version__ strings at all. They’re kind of a pain to remember
> to bump on every new release.


That’s a tooling issue, and I don’t think that Python itself should take
any official stance on the tooling. But yes, anything that Python takes an
official stance on should be able to be easily supported with
not-too-complex tools.and I think a __version__ string fits this need.

Personally, I use the __version__ string in my packages as the canonical
source when I bold my packages.

And there are fancier tools for more complex needs.

> PEP 396 has been dormant for a long time and I have no interest in
pushing it forward any more. If someone else wants to take up the cause, I
recommend creating a new PEP and referring back to 396.

Fair enough -- I'm still gathering data to see if I should do just that.

I am leaning toward at least SOME PEP about this -- having __version__
around as a semi-convention with no official recommendation as to if or how
it should be used is kind of a mess.

-CHB

On Thu, Apr 15, 2021 at 12:27 PM Terry Reedy <tjreedy@udel.edu> wrote:

> At least some in the stdlib have not been. For re.__version__, not
> since Nov 2001.
>

From Serhiy over on python-ideas, he's started an effort to clean that up.

https://mail.python.org/archives/list/python-dev@python.org/thread/KBU4EU2JULXSMUZULD5HJJWCGOMN52MK/
).

I believe there is an unfinished PR

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 9:36 AM David Mertz <mertz@gnosis.cx> wrote:

> I was so hopeful about this, but in the end... not really. I have not
> used this capability before. Here are a few different situations I know of:
>

...

...
re PackageNotFoundError('re')
statistics PackageNotFoundError('statistics')
pandas 1.2.4
vaex 4.1.0
bs4 PackageNotFoundError('bs4')

>
> It seems like (somehow or another) importlib.metadata is doing something
> perhaps more useful for vaex. But it is doing distinctly worse for re,
> statistics, and bs4.
>

funny you should try bs4, which I also used as an example in another post.

But what you have found is that there is a difference between a pacakge
(which you can import) and a "distribution" which is something you install.

importlib.metadata is looking for distributions.

re and statistics are not distributions, but rather, built in pacakges
(modules).

bs4 is not a distributiuion, it is a package that is provided by the
"beautifulsoup4" distribution.

In [3]: importlib.metadata.version("beautifulsoup4")
Out[3]: '4.9.3'

Frankly, I'm still a bit confused about the distiction, but I do know for
sure that a single distribution (maybe somethign that you can install via
PyPi) can install more than one top-level package -- and certainly the
packages/modules installed can have a different name than the distribution
name.

And if a distribution installs more than one package, they may have
different version numbers.

I'm not sure what to make of all this, though I'm leaning toward better
suporting the distiction by asking for __version__ strings in top-level
packages -- and maybe making importlib.metadata.version a bit smarter about
looking for packages, and not just distributions.

If you look at the docstring of metadata.version:

"""
In [2]: importlib.metadata.version?
Signature: importlib.metadata.version(distribution_name)
Docstring:
Get the version string for the named package.

:param distribution_name: The name of the distribution package to query.
:return: The version string for the package as defined in the package's
"Version" metadata key.
File: ~/miniconda3/envs/py3/lib/python3.9/importlib/metadata.py
Type: function
"""

It's a bit inconsistent with the use of the term "distribution" vs
"package". That should get cleaned up if nothing else.

Also, the Exception raised is "PackageNotFoundError" -- which should maybe
be "DistributionNotFoundError"?


Version is arguably useful from the package user side. As I believe Victor
mentioned, there are two uses for version information: display to the user
-- for which version strings are fine, or programmatic comparison -- for
which something like the Version object is very helpful. Do we only need to
use version information programmatically when we are creating (or
installing) packages? I don't think so -- I know I have code that (poorly)
does version checking programmatically.


> Or rather, the below is what I would find really nice to be able to do.
>
> ver = robust_version(module)
> if ver >= (5, 2, 1):
> doit_modern_style()
> elif ver < (5, 2, 1):
> doit_old_style
> else:
> doit_unversioned_style()
>

Exactly -- and I htink we are close, if pacakges(modules) had compliant
__version__ strings, thenyou could do
(with the Version object from packaging -- maybe why it should be in the
stdlib)

ver = Version(module.__version__)
if ver >= Version("5.2.1"):
doit_modern_style()
elif ver < Version("5.2.1"):
doit_old_style
else:
doit_unversioned_style()

And if my PR is accepted (which looks unlikley) to allow camparison between
Version objects and strings:

ver = Version(module.__version__)
if ver >= "5.2.1":
doit_modern_style()
elif ver < "5.2.1":
doit_old_style
else:
doit_unversioned_style()

A key distiction here from the importlib.metadata approach is the level of
indirection -- I like being able to ask a module itself what version it is,
rather than asking some other part of teh system to go look it up for me.
So I could do:

import numpy as np

print("using np version:", np.__version)

And this is pretty consitent with the rest of Python, where many objects
(functions, classes, modules) have a __name__ attribute if things "know"
their name, shouldn't they "know" their version as well?

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython

On Thu, Apr 15, 2021 at 9:54 PM Christopher Barker <pythonchb@gmail.com>
wrote:

> Or rather, the below is what I would find really nice to be able to do.
>
>> ver = robust_version(module)
>> if ver >= (5, 2, 1):
>> doit_modern_style()
>> elif ver < (5, 2, 1):
>> doit_old_style
>> else:
>> doit_unversioned_style()
>>
>
> Exactly -- and I htink we are close, if pacakges(modules) had compliant
> __version__ strings, then you could do
> (with the Version object from packaging -- maybe why it should be in the
> stdlib)
>
> ver = Version(module.__version__)
> if ver >= Version("5.2.1"):
> doit_modern_style()
> elif ver < Version("5.2.1"):
> doit_old_style
> else:
> doit_unversioned_style()
>

This wouldn't be bad. A little more verbose, but fine.

The thing is, I almost never care about the version of a *distribution* in
my programs. I care about a version of the *package*. Since, as Chris
mentions, a distribution could contain multiple packages, the same package
could be vendorized (in different versions) by different distributions.
The question I want to ask above is always because "version such-and-such
changed an API," and that's about a package. When I care about
distribution versions, it is because there are dependencies stated in
requirements.txt or environment.yaml or whatever.deb.

Asking for the `.__version__` attributes feels like needless redundancy in
Chris' variation. The constructor should be able to ask "Is this a module?
If so, does it have that attribute". But whatever, I can learn to do that
easily enough.

Someone else mentioned that the versions of standard library modules are
an odd collection of relics. I do not disagree, of course. But sometimes
the stdlib modules *do* change. I thought of the `re` example because I
was doing a tutorial on it, and something or another depended on a change
between 3.8 and 3.9 (I think, I forget the detail). For training material,
this can just be approached as describing the Python version... but for
running code, a version check could be relevant.

Therefore, I think this `Version` constructor (or `robust_version()`
function, or whatever) should have access to that. I.e. if not an
automated update to `.__version__` inside each stdlib module during builds,
then a fallback to "If this is a stdlib module, report sys.version_info
(within the right kind of "Version" object).


--
The dead increasingly dominate and strangle both the living and the
not-yet born. Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.

On Thu, 15 Apr 2021 at 21:03, Christopher Barker <pythonchb@gmail.com> wrote:

> I am leaning toward at least SOME PEP about this -- having __version__ around as a semi-convention with no official recommendation as to if or how it should be used is kind of a mess.

On the other hand, having a PEP that says you "may" have a version
attribute, but mandates nothing, is pretty useless. People who conform
to that convention don't need telling, people who don't have little
reason to change, and people wanting to use the information still have
to cater for it being missing, so they gain nothing.

But a PEP that makes __version__ a requirement fails instantly,
because there's too many packages in the wild that violate it.

Welcome to the world of packaging PEPs :-)

> I'm not sure what to make of all this, though I'm leaning toward better suporting the distiction by asking for __version__ strings in top-level packages -- and maybe making importlib.metadata.version a bit smarter about looking for packages, and not just distributions.

Personally, I'm leaning more and more towards the view that it's
*distributions* that are versioned, and that's an established
practice. Versioning packages as well (except in the very limited
sense that package X installed by distribution Y can expose Y's
version via its API) only adds confusion and makes the situation less,
not more, useful for users.

So I'm now -1 on any PEP to try to formalise the use of __version__.
If people want to discuss the use of a __version__ attribute, they
should consider contributing to the discussions on versions in
https://packaging.python.org/guides/ (where the necessary context is
already present).

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ECJ2HHV2EXJ7GOJU7ILLQVVLF54VVD3U/
Code of Conduct: http://python.org/psf/codeofconduct/

Replying to myself to update my own thoughts:

I think __version__ and packaging tools answer separate questions.

__version__ answers the question of "this module I imported, what
version is it".

The packaging tools answer the question "what dependency does this
distribution satisfy".

This mismatch is why importlib.metadata.version("bs4") doesn't return
a useful answer. A single distribution might even install multiple
top-level modules with different versions.

Schiavo,
Simon
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O242SRB6SBLBBMPCOLCKURBEXXHUSJZN/
Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Apr 15, 2021 at 10:53 AM Antoine Pitrou <antoine@python.org> wrote:

> On Thu, 15 Apr 2021 10:28:53 -0700
> Brett Cannon <brett@python.org> wrote:
> > On Thu, Apr 15, 2021 at 8:50 AM Christopher Barker <pythonchb@gmail.com>
> > wrote:
> >
> > > On Thu, Apr 15, 2021 at 2:40 AM Victor Stinner <vstinner@python.org>
> > > wrote:
> > >
> > >> Paul Bryan:
> > >> > Seems like this is something that should make its way into stdlib?
> > >>
> > >> In the last 10 years, the trend is more to remove anything related to
> > >> packaging *outside* the stdlib :-)
> > >
> > >
> > > Well, maybe all of us don't think that's a good idea ;-)
> > >
> >
> > As an active participant in the PyPA side of the packaging ecosystem and
> > one of the maintainers of 'packaging', I am definitely NOT in favour of
> > moving any of 'packaging' into the stdlib, nor breaking it up. We are
> > taking distutils out of the stdlib for a reason. Backsliding on plans
> that
> > have been in place for years is not something I would want to see happen
> as
> > the motivations have not changed.
>
> This seems gratuitously dogmatic. Version numbers are a simple feature
> that is stably encoded in PEP 440. It's purely computational, does
> not involve reading or saving any persistent state in the filesystem,
> making any network access, etc. Why wouldn't it have its place in the
> stdlib?
>

Technically PyPA specs might start as PEPs but their official home is
packaging.python.org and updates can occur there outside of the PEP
process. In the case of version specifiers (
https://packaging.python.org/specifications/version-specifiers/), they
point back to the original PEP, but that is not universally true nor
guaranteed to hold in the future. In other words I personally don't view
the fact that it's a PEP as a reason to have it in the stdlib (else pretty
much every single packaging PEP is missing).

We also have other things standardized in PEPs and not covered in the
stdlib, e.g. @ as an operator.

So I don't think version comparison occurs often enough to be in the
stdlib, and the fact that an external project exists which isn't interested
in being put in the stdlib suggests to me it isn't worth doing.

But that's a separate topic to discuss at the language summit. :)


>
> > > But anyway, I would say removing anything *related* to packaging
> outside
> > > the stdlib is a bad idea -- requiring an external tool to build a
> package
> > > is OK, but requireing an external use packages, not so much.
> > >
> >
> > You can totally use a package already as things exist today and will in
> the
> > future; you're just wanting a specific way to interpret a package's
> > metadata which I don't view as the same thing.
>
> That specific way is a Python standard (PEP 440). Having the
> functionality in the stdlib would encourage people to use it. Not
> having it in the stdlib encourages people to use adhoc version parsing,
> or worse, naive string comparison.
>

I don't know if I draw the same line since the packaging community is
relying on 'packaging' without issue for this exact thing.


>
> > So asking every Python project to set a
> > __version__ isn't going to change accessibility of version numbers when
> it
> > comes to installed projects.
>
> That doesn't have much to do with the suggestion of shipping a Version
> class in the stdlib, though. Many projects already provide a
> __version__, and that field is often inspected programmatically in
> dependent packages (to work around behaviour changes, for example).
>

But this thread is about standardizing on __version__ which then led to
wanting to bring in some Version object into the stdlib to make that more
useful. So to me they are tied together, else a separate thread should
probably be started if a new module *just *for version parsing support is
desired.

On Thu, 15 Apr 2021 16:12:57 -0700
Brett Cannon <brett@python.org> wrote:
>
> So I don't think version comparison occurs often enough to be in the
> stdlib, and the fact that an external project exists which isn't interested
> in being put in the stdlib suggests to me it isn't worth doing.
>
> But that's a separate topic to discuss at the language summit. :)

Ah, I should submit a question then.

> > That specific way is a Python standard (PEP 440). Having the
> > functionality in the stdlib would encourage people to use it. Not
> > having it in the stdlib encourages people to use adhoc version parsing,
> > or worse, naive string comparison.
>
> I don't know if I draw the same line since the packaging community is
> relying on 'packaging' without issue for this exact thing.

Right, but the functionality is useful generally, not only for the
packaging generally.

> But this thread is about standardizing on __version__ which then led to
> wanting to bring in some Version object into the stdlib to make that more
> useful. So to me they are tied together, else a separate thread should
> probably be started if a new module *just *for version parsing support is
> desired.

You're right, this thread was hijacked somewhat for the Version object
discussion.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U2KO5AZRIHNXWKRQ5YEY3CGW4FZ6SAEU/
Code of Conduct: http://python.org/psf/codeofconduct/

Hi,

sorry for being late to the party, but I may not be the only one wondering…

Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>
> I’d forgotten that this PEP was in Deferred state. I think it should be rejected and I plan on making that change. importlib.metadata is a much better approach to programmatically determining package versions.
>
> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions

This is indeed the correct approach, thanks for letting me learn this.

However, I unsuccessfully searched for the canonical way to look up the
distribution name based on either a module name or an imported module
object. Is there one?

Looks like it could be computed based on information in
"*.egg-info/installed-files.txt", but it's far from trivial. Is
"installed-files.txt" even guaranteed to exist for all distributions?

Cheers,
Baptiste
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SVQ777EGE56OIWHIKPCURGPXGMUJ7HCY/
Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
devel2021@baptiste-carvello.net> wrote:

> Hi,
>
> sorry for being late to the party, but I may not be the only one wondering…
>
> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
> >
> > I’d forgotten that this PEP was in Deferred state. I think it should be
> rejected and I plan on making that change. importlib.metadata is a much
> better approach to programmatically determining package versions.
> >
> >
> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>
> This is indeed the correct approach, thanks for letting me learn this.
>
> However, I unsuccessfully searched for the canonical way to look up the
> distribution name based on either a module name or an imported module
> object. Is there one?
>

If you mean how to tie a module back to its name on PyPI, you should be
able to look up the "Name" in the project's metadata:
https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
.

-Brett


>
> Looks like it could be computed based on information in
> "*.egg-info/installed-files.txt", but it's far from trivial. Is
> "installed-files.txt" even guaranteed to exist for all distributions?
>
> Cheers,
> Baptiste
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SVQ777EGE56OIWHIKPCURGPXGMUJ7HCY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Mon, Apr 26, 2021 at 10:31 PM Brett Cannon <brett@python.org> wrote:

>
>
> On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
> devel2021@baptiste-carvello.net> wrote:
>
>> Hi,
>>
>> sorry for being late to the party, but I may not be the only one
>> wondering…
>>
>> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>> >
>> > I’d forgotten that this PEP was in Deferred state. I think it should
>> be rejected and I plan on making that change. importlib.metadata is a much
>> better approach to programmatically determining package versions.
>> >
>> >
>> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>>
>> This is indeed the correct approach, thanks for letting me learn this.
>>
>> However, I unsuccessfully searched for the canonical way to look up the
>> distribution name based on either a module name or an imported module
>> object. Is there one?
>>
>
> If you mean how to tie a module back to its name on PyPI, you should be
> able to look up the "Name" in the project's metadata:
> https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
> .
>
>
The missing bit here, for me, is how do you map a module back to it's
project (distribution)?

For example:

```
>>> import bs4
>>> bs4.__version__
'4.9.1'
>>> importlib.metadata.metadata('bs4')
PackageNotFoundError: bs4
```

This is because the distribution calls itself 'beautifulsoup4' instead.

The same goes for another package: `umap`, for which the distribution is
called `umap-learn`


This is the best I could come up with from reading the docs:

import bs4 #<- This is the module we want the version of

import importlib
import sys
from itertools import chain
from pathlib import Path

loaders = sys.meta_path

target_path = Path(bs4.__file__)

distros = list(chain(*(finder.find_distributions() for finder in
loaders if hasattr(finder, 'find_distributions'))))
distros_files = chain(*(f for f in (d.files for d in distros)))
distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for f
in d.files]
matching = [d for d, f in distro_files if f == target_path]

for match in matching:
print("Found Version:", match.version)

Steve

On Mon, Apr 26, 2021 at 3:25 PM Stestagg <stestagg@gmail.com> wrote:

>
>
> On Mon, Apr 26, 2021 at 10:31 PM Brett Cannon <brett@python.org> wrote:
>
>>
>>
>> On Mon, Apr 26, 2021 at 9:37 AM Baptiste Carvello <
>> devel2021@baptiste-carvello.net> wrote:
>>
>>> Hi,
>>>
>>> sorry for being late to the party, but I may not be the only one
>>> wondering…
>>>
>>> Le 14/04/2021 à 20:56, Barry Warsaw a écrit :
>>> >
>>> > I’d forgotten that this PEP was in Deferred state. I think it should
>>> be rejected and I plan on making that change. importlib.metadata is a much
>>> better approach to programmatically determining package versions.
>>> >
>>> >
>>> https://docs.python.org/3/library/importlib.metadata.html#distribution-versions
>>>
>>> This is indeed the correct approach, thanks for letting me learn this.
>>>
>>> However, I unsuccessfully searched for the canonical way to look up the
>>> distribution name based on either a module name or an imported module
>>> object. Is there one?
>>>
>>
>> If you mean how to tie a module back to its name on PyPI, you should be
>> able to look up the "Name" in the project's metadata:
>> https://docs.python.org/3/library/importlib.metadata.html#distribution-metadata
>> .
>>
>>
> The missing bit here, for me, is how do you map a module back to it's
> project (distribution)?
>
> For example:
>
> ```
> >>> import bs4
> >>> bs4.__version__
> '4.9.1'
> >>> importlib.metadata.metadata('bs4')
> PackageNotFoundError: bs4
> ```
>
> This is because the distribution calls itself 'beautifulsoup4' instead.
>

Unfortunately I thought importlib.metadata would have used the module name
instead of the metadata details, but in hindsight am guessing that the
.dist-info is what it's using to do the lookup and that's based on the
package name instead of the project name.

This is a long-standing issue with projects that use project names which
differ from their module name, but there's no good way without checking
what files a project installed (which is what you're doing below).

-Brett


>
> The same goes for another package: `umap`, for which the distribution is
> called `umap-learn`
>
>
> This is the best I could come up with from reading the docs:
>
> import bs4 #<- This is the module we want the version of
>
> import importlib
> import sys
> from itertools import chain
> from pathlib import Path
>
> loaders = sys.meta_path
>
> target_path = Path(bs4.__file__)
>
> distros = list(chain(*(finder.find_distributions() for finder in
> loaders if hasattr(finder, 'find_distributions'))))
> distros_files = chain(*(f for f in (d.files for d in distros)))
> distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for
> f in d.files]
> matching = [d for d, f in distro_files if f == target_path]
>
> for match in matching:
> print("Found Version:", match.version)
>
> Steve
>

On Tue, 27 Apr 2021 at 18:01, Brett Cannon <brett@python.org> wrote:

> Unfortunately I thought importlib.metadata would have used the module name instead of the metadata details, but in hindsight am guessing that the .dist-info is what it's using to do the lookup and that's based on the package name instead of the project name.
>
> This is a long-standing issue with projects that use project names which differ from their module name, but there's no good way without checking what files a project installed (which is what you're doing below).

That's correct. There is no link from a given import name back to the
PyPI project name. And the metadata for installed packages (defined
here: https://packaging.python.org/specifications/recording-installed-packages/)
is keyed on the PyPI project name, as the OP noted ("beautifulsoup4"
rather than "bs4", "setuptools" rather than "pkg_resources", etc).

>> This is the best I could come up with from reading the docs:
>>
>> import bs4 #<- This is the module we want the version of
>>
>> import importlib
>> import sys
>> from itertools import chain
>> from pathlib import Path
>>
>> loaders = sys.meta_path
>>
>> target_path = Path(bs4.__file__)
>>
>> distros = list(chain(*(finder.find_distributions() for finder in loaders if hasattr(finder, 'find_distributions'))))
>> distros_files = chain(*(f for f in (d.files for d in distros)))
>> distro_files = [.(d, d.locate_file(f)) for d in distros if d.files for f in d.files]
>> matching = [d for d, f in distro_files if f == target_path]
>>
>> for match in matching:
>> print("Found Version:", match.version)

The following is a bit simpler. The file.locate() method of a
distribution is undocumented - but I tried to use resolve() on the
path object I got from dist.files, and it appears not to be
implemented (at least in Python 3.9) - PackagePath objects are a
subclass of *Pure* Path objects, not concrete paths :-( TBH, I suspect
the fact that it's undocumented is an oversight, it's clearly
deliberately added.

import importlib.metadata
from pathlib import Path

import pkg_resources
target = Path(pkg_resources.__file__)

for dist in importlib.metadata.distributions():
for file in dist.files:
path = file.locate()
if path == target:
print(f"{dist.metadata['name']}: {dist.version}")
break

To be honest, if anyone were interested in making a PEP from any of
this, having Python modules contain a __distribution_name__ attribute
that links the module back to the PyPI distribution, would probably be
more useful than standardising __version__. But I'm not sufficiently
interested to do anything more than mention that as a possibility.

Paul
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/66G7A5ZZRYT7H6MRJVELFMC23IDP67CL/
Code of Conduct: http://python.org/psf/codeofconduct/