Mailing List Archive

Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7)
So, there won't be any package management tool shipped with Python 2.7
and users will have to download and install `setuptools` manually as
before:

"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python
setup.py install"


Therefore I still propose shipping bootstrap package that instruct
user how to download and install an actual package management tool
when users tries to use it. So far I know only one stable tool -
`easy_install` - a part of `setuptools` package.

The required behavior for very basic user friendliness:
1. user installs Python 2.7
2. user issues `python -m easy_install something`
3. user gets message
'easy_install' tool is not installed on this system. To make it
available, download and install `setuptools` package from
http://pypi.python.org/pypi/setuptools/

4. the screen is paused before exit (for windows systems)

Other design notes:
1. if package tries to import `easy_install` module used for
bootstrap, it gets the same ImportException as if there were no
`easy_install` at all
2. bootstrap module is overwritten by actual package when users installs it


So, do we need a PEP for that? How else can I know if consensus is
reached? Anybody is willing to elaborate on implementation?


P.S. Please be careful to reply to relevant lists
--
anatoly t.



On Mon, Mar 29, 2010 at 9:37 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
> 2010/3/29 anatoly techtonik <techtonik@gmail.com>:
> [..]
>> It is really hard to follow. You should at least change subjects when
>> switching topic.
>
> I was talking about the work going on and the decisions taken lately.
>
> I never change topics of threads mails when there's less than 100 mails,
> because I find it way more confusing :)
>
>>
>> So, what is the verdict? Will there be a package management tool or
>> bootstrap package for it shipped with Python 2.7 or not?
>
> As I said in the mail you've quoted, all improvements are made in
> Distutils2. So the answer is no.
> Python 2.7b1 is due in less than a week anyways, so any new
> development on this topic will happen after.
>
> Basically Python 2.7 == Python 2.6 in term of packaging.
>
> Regards
> Tarek
>
> --
> Tarek Ziadé | http://ziade.org
>
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 9:30 AM, anatoly techtonik <techtonik@gmail.com> wrote:
> So, there won't be any package management tool shipped with Python 2.7
> and users will have to download and install `setuptools` manually as
> before:
>
>  "search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python
> setup.py install"
>
>
> Therefore I still propose shipping bootstrap package that instruct
> user how to download and install an actual package  management tool
> when users tries to use it. So far I know only one stable tool -
> `easy_install` - a part of `setuptools` package.

There are other tools to install something in a vanilla Python :

- pip (which also have an uninstall feature)
- distutils
- distribute

>
> The required behavior for very basic user friendliness:
> 1. user installs Python 2.7
> 2. user issues `python -m easy_install something`
> 3. user gets message
> 'easy_install' tool is not installed on this system. To make it
> available, download and install `setuptools` package from
> http://pypi.python.org/pypi/setuptools/

Are you thinking about something generic ?

Like, being able to call :

$ python -m ANYTHING

And get an error message saying that the ANYTHING script is not installed ?
Then have a registry somewhere to get the name of the project that
owns the ANYTHING script ?


[..]
> So, do we need a PEP for that? How else can I know if consensus is
> reached? Anybody is willing to elaborate on implementation?

I see two paths here:
1 - do you want to define a general mechanism for Python to install scripts ?
2 - are you just suggesting to have this mechanism only for Package Managers ?

In case of 2), what you would need to do is propose an extension to
PEP 376, we are currently working
on, then we can add it there once people have discussed it on
distutils-SIG, or maybe create a new PEP if the topic is too big to
fit in 376.

>
>
> P.S. Please be careful to reply to relevant lists

Yes, so if it's 2) let's keep this thread in Distutils-SIG.
Cross-posting like this makes it hard to follow.

Tarek

--
Tarek Ziadé | http://ziade.org
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 10:55 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
>>
>> Therefore I still propose shipping bootstrap package that instruct
>> user how to download and install an actual package  management tool
>> when users tries to use it. So far I know only one stable tool -
>> `easy_install` - a part of `setuptools` package.
>
> There are other tools to install something in a vanilla Python :
>
> - pip  (which also have an uninstall feature)
> - distutils
> - distribute

distutils is not a `package management` tool, because it doesn't know
anything even about installed packages, not saying anything about
dependencies.

`pip` and `distribute` are unknown for a vast majority of Python
users, so if you have a perspective replacement for `easy_install` -
it can be said in bootstrap package message. There is no problem with
packages that require `setuptools` either - they will require
`setuptools` as dependency anyway.

For now there are two questions:
1. Are they stable enough for the replacement of user command line
`easy_install` tool?
2. Which one is the recommended?

P.S. Should there be an accessible FAQ in addition to ML?

>>
>> The required behavior for very basic user friendliness:
>> 1. user installs Python 2.7
>> 2. user issues `python -m easy_install something`
>> 3. user gets message
>> 'easy_install' tool is not installed on this system. To make it
>> available, download and install `setuptools` package from
>> http://pypi.python.org/pypi/setuptools/
>
> Are you thinking about something generic ?
>
> Like, being able to call :
>
> $ python -m ANYTHING
>
> And get an error message saying that the ANYTHING script is not installed ?
> Then have a registry somewhere to get the name of the project that
> owns the ANYTHING script ?
>

No. To get something in 2.7 I would refrain from developing into this
direction for now.

> [..]
>> So, do we need a PEP for that? How else can I know if consensus is
>> reached? Anybody is willing to elaborate on implementation?
>
> I see two paths here:
> 1 - do you want to define a general mechanism for Python to install scripts ?
> 2 - are you just suggesting to have this mechanism only for Package Managers ?

3 - I want current "best practice" for installing Python modules with
dependencies from PyPI to be shipped with Python 2.7 by default

Answering your questions:
1. I want to have some user-friendly way for installing Python
scripts, but I am more concerned that I will miss `easy_install` in
Python 2.7
2. Bootstrap script is aimed at users. Package managers are assumed to
already have their package
at PyPI and provide install instructions that involve `easy_install`,
so nothing is required to be done on their part.

>
> In case of 2), what you would need to do is propose an extension to
> PEP 376, we are currently working
> on, then we can add it there once people have discussed it on
> distutils-SIG, or maybe create a new PEP if the topic is too big to
> fit in 376.

PEP 376 is completely irrelevant to user side boot package proposal.
This proposal is to recommend whatever package managing tool you think
user need and instruct how to get the tool. The boot package doesn't
know anything at all about how packages are managed.


>> P.S. Please be careful to reply to relevant lists
>
> Yes, so if it's 2) let's keep this thread in Distutils-SIG.
> Cross-posting like this makes it hard to follow.

My vision is that decision about having bootstrap package or not in
2.7 should be in python-dev and specific packaging, implementation and
pip/distutils/distribute questions in distutils-sig.

--
anatoly t.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 11:02 AM, anatoly techtonik <techtonik@gmail.com> wrote:
[..]
> distutils is not a `package management` tool, because it doesn't know
> anything even about installed packages, not saying anything about
> dependencies.

At this point, no one knows anything about installed packages at the
Python level.
Keeping track of installed projects is a feature done within each
package managment system.

And the whole purpose of PEP 376 is to define a database of what's
installed, for the sake of interoperability.

>
> `pip` and `distribute` are unknown for a vast majority of Python
> users, so if you have a perspective replacement for `easy_install` -

Depending on how you call a Python user, I disagree here. Many people
use pip and distribute.

The first one because it has an uninstall feature among other things.
The second one because it fixes some bugs of setuptools and provides
Python 3 support


>
> For now there are two questions:
> 1. Are they stable enough for the replacement of user command line
> `easy_install` tool?
> 2. Which one is the recommended?
>
> P.S. Should there be an accessible FAQ in addition to ML?

This FAQ work has been started in th "HitchHicker's guide to
Packaging" you can find here:

http://guide.python-distribute.org

[..]
> No. To get something in 2.7 I would refrain from developing into this
> direction for now.
>
>> [..]
>>> So, do we need a PEP for that? How else can I know if consensus is
>>> reached? Anybody is willing to elaborate on implementation?
>>
>> I see two paths here:
>> 1 - do you want to define a general mechanism for Python to install scripts ?
>> 2 - are you just suggesting to have this mechanism only for Package Managers ?
>
> 3 - I want current "best practice" for installing Python modules with
> dependencies from PyPI to be shipped with Python 2.7 by default

Again, any new code work will not happen because 2.7 is due in less
than a week. Things are happening in Distutils2.

Now, for the "best practice" documentation, I think the guide is the
best plce to look at.


[..]
> PEP 376 is completely irrelevant to user side boot package proposal.

It is, since it proposes an uninstall script called via -m. So having
a install script called by -m is definitely its business.


> My vision is that decision about having bootstrap package or not in
> 2.7 should be in python-dev and specific packaging, implementation and
> pip/distutils/distribute questions in distutils-sig.

If the mentioned bootstrap script is about a package managment system,
you should keep the discussion in distutils-SIG I think.

Regards
Tarek

--
Tarek Ziadé | http://ziade.org
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 11:15 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
[..]
> Depending on how you call a Python user, I disagree here. Many people
> use pip and distribute.

s/how/who :)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 12:15 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
> [..]
>> distutils is not a `package management` tool, because it doesn't know
>> anything even about installed packages, not saying anything about
>> dependencies.
>
> At this point, no one knows anything about installed packages at the
> Python level.

Users do not care about this, and `distutils` doesn't know this even
at package level.

> Keeping track of installed projects is a feature done within each
> package managment system.
>
> And the whole purpose of PEP 376 is to define a database of what's
> installed, for the sake of interoperability.

That's great. When it will be ready everybody would be happy to make
their package management tool compliant.

>>
>> `pip` and `distribute` are unknown for a vast majority of Python
>> users, so if you have a perspective replacement for `easy_install` -
>
> Depending on how you call a Python user, I disagree here. Many people
> use pip and distribute.
>
> The first one because it has an uninstall feature among other things.
> The second one because it fixes some bugs of setuptools and provides
> Python 3 support

I do not mind if we can distribute three stubs, they will also serve
as pointers for a better way of packaging when an ultimate tool is
finally born. Me personally is willing to elaborate for `easy_install`
stub in 2.7.

>>
>> For now there are two questions:
>> 1. Are they stable enough for the replacement of user command line
>> `easy_install` tool?
>> 2. Which one is the recommended?
>>
>> P.S. Should there be an accessible FAQ in addition to ML?
>
> This FAQ work has been started in th "HitchHicker's guide to
> Packaging" you can find here:
>
> http://guide.python-distribute.org

I can see any FAQ. To me the FAQ is something that could be posted to
distutils ML once a month to reflect current state of packaging. It
should also carry version number. So anybody can comment on the FAQ,
ask another question or ask to make a change.

> Again, any new code work will not happen because 2.7 is due in less
> than a week. Things are happening in Distutils2.

That doesn't solve the problem. Bootstrap script can be written in one
day. What we need is a consensus whatever this script is welcomed in
2.7 or not? Who is the person to make the decision?

> Now, for the "best practice" documentation, I think the guide is the
> best plce to look at.

Let's refer to original user story:
"I installed Python and need a quick way to install my packages on top of it."
"I execute `easy_install something` as said in installation manual,
but nothing/error happens."

> [..]
>> PEP 376 is completely irrelevant to user side boot package proposal.
>
> It is, since it proposes an uninstall script called via -m. So having
> a install script called by -m is definitely its business.

The scope of my proposal is a bootstrap script that instructs user how
to install the package management tool of user's choice if this tool
is not yet installed on user system (this is the case with new Python
installation). If you'll come up with a better way of package
management - you will update this bootstrap script with relevant
information in future Python releases.

Right now it is essential to get this _feature_ in Python 2.7 until
2.7 is frozen for new features. The script that shows message upon
invocation and is overwritten by the version of the package it
proposes to install. Is it hard to do?

>> My vision is that decision about having bootstrap package or not in
>> 2.7 should be in python-dev and specific packaging, implementation and
>> pip/distutils/distribute questions in distutils-sig.
>
> If the mentioned bootstrap script is about a package managment system,
> you should keep the discussion in distutils-SIG I think.

If there will be no bootstrap script in 2.7, I won't have any
motivation to continue discussion until 2.8 deadline is near. Of
course I will be disappointed, because I will have to explain
everything to 2.7 users again and again instead of letting them
execute one command and follow its instructions.
--
anatoly t.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
Anatoly, I am now answering only in Distutils-SIG.

On Mon, Mar 29, 2010 at 3:45 PM, anatoly techtonik <techtonik@gmail.com> wrote:
[..]
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 16:21, anatoly techtonik <techtonik@gmail.com> wrote:
> Just a thought about user story my customers would likely write if I
> shipped Python as a product:
> "As a user, I think Python is suxx, because it makes its users suffer
> for a long time from packaging disorder".

So fix it. Python is open source. It's created by it's users. If there
is something you think it lacking, you step up and help make it
better.

But the packaging solution will not be ready for 2.7. It's a really
hard problem. Your help is appreciated. Complaints are not.

--
Lennart Regebro: http://regebro.wordpress.com/
Python 3 Porting: http://python-incompatibility.googlecode.com/
+33 661 58 14 64
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
anatoly techtonik wrote:
> So, there won't be any package management tool shipped with Python 2.7
> and users will have to download and install `setuptools` manually as
> before:
>
> "search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python
> setup.py install"
>
>
> Therefore I still propose shipping bootstrap package that instruct
> user how to download and install an actual package management tool
> when users tries to use it.

For what it's worth, Guido prototyped something similar in March of
2008, but his was an actual bootstrapping tool for package management:

http://mail.python.org/pipermail/python-dev/2008-March/077837.html

His tool knew how to download a tar file, untar it, and run "python
setup.py install" on it. No version numbers, no dependency management,
simple enough that it should be easy to get right. Only appropriate for
bootstrapping into a real package management tool.

The thread ends with him saying "I don't have time to deal with this
further this week", and I dunno, maybe it just fell off the radar? I'd
been thinking about resurrecting the discussion but I didn't have time
either.

I fear it is too late for 2.7,


//larry//
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com
Re: Bootstrap script for package management tool in Python 2.7 (Was: Re: [Distutils] At least one package management tool for 2.7) [ In reply to ]
On Mon, Mar 29, 2010 at 11:26 AM, Larry Hastings <larry@hastings.org> wrote:

> anatoly techtonik wrote:
>
>> So, there won't be any package management tool shipped with Python 2.7
>> and users will have to download and install `setuptools` manually as
>> before:
>>
>> "search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python
>> setup.py install"
>>
>>
>> Therefore I still propose shipping bootstrap package that instruct
>> user how to download and install an actual package management tool
>> when users tries to use it.
>>
>
> For what it's worth, Guido prototyped something similar in March of 2008,
> but his was an actual bootstrapping tool for package management:
>
> http://mail.python.org/pipermail/python-dev/2008-March/077837.html
>
> His tool knew how to download a tar file, untar it, and run "python
> setup.py install" on it. No version numbers, no dependency management,
> simple enough that it should be easy to get right. Only appropriate for
> bootstrapping into a real package management tool.
>
> The thread ends with him saying "I don't have time to deal with this
> further this week", and I dunno, maybe it just fell off the radar? I'd been
> thinking about resurrecting the discussion but I didn't have time either.
>

I would consider this bootstrap to be quite workable, though I would add
that any extra option to the bootstrap script should be passed to setup.py
install, and the download should be cached (so you can do -h and not have to
re-download the package once you figure out the extra options -- at least a
--user option is reasonable here for people without root). Specifically
targeting this bootstrap for tools like pip and virtualenv is no problem.

I think looking around PyPI etc is kind of more than I'd bother with. Those
things change, this bootstrap code won't change, it could cause unnecessary
future pain. Maybe (*maybe*) it could look in
http://pypi.python.org/well-known-packages/PACKAGE_NAME and so we can have
it install a certain small number of things quickly that way -- if the URL
it looks to is targeted only for the bootstrap script itself then we don't
have to worry about compatibility problems as much.

Oh... then i can think of a half dozen other options it could take, and then
it becomes an installer. Blech. OK, I'd be willing to cut off the options
at --user (which I think is a minimum... maybe --prefix too), and maybe some
simple package detection so people could write "python -m boostrap
Setuptools --user" -- entirely based on some well-known URL baked into
bootstrap.py, where the URL is independent of any other service (and so is
least likely to cause future problems or ambiguities).

An advantage to this kind of bootstrapper is that as future packaging
systems are developed there's a clear way to get started with them, without
prematurely baking anything in to Python.

--
Ian Bicking | http://blog.ianbicking.org | http://twitter.com/ianbicking