Mailing List Archive: PEP 668 starting to bite me

PEP 668 starting to bite me

Jun 20, 2023, 10:04 AM

Post #1 of 8 (267 views)

I was starting to build mythtv from source on Endeavour OS which is a Arch
linux based distro. It complied fine but the sudo make install failed due
to new rules that are starting to creep into OSs. It will affect Debian,
Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today on
something that last week didn't cause a problem.

The error is related to installing pip packages. Here is a the console
error I saw at the end of sudo make install:

make[1]: Entering directory '/home/jim/build/mythtv/mythtv/bindings/python'
python3 -m pip install --no-build-isolation --no-cache-dir --no-index
--disable-pip-version-check --no-deps --find-links ./dist MythTV error:
externally-managed-environment × This environment is externally managed ??>
To install Python packages system-wide, try 'pacman -S python-xyz', where
xyz is the package you are trying to install. If you wish to install a
non-Arch-packaged Python package, create a virtual environment using 'python
-m venv path/to/venv'. Then use path/to/venv/bin/python and
path/to/venv/bin/pip.
If you wish to install a non-Arch packaged Python application, it may be
easiest to use 'pipx install xyz', which will manage a virtual environment
for you. Make sure you have python-pipx installed via pacman. note: If you
believe this is a mistake, please contact your Python installation or OS
distribution provider. You can override this, at the risk of breaking your
Python installation or OS, by passing --break-system-packages hint: See PEP
668 for the detailed specification. make[1]: *** [Makefile:35: install]
Error 1

This link gives a better explanation of what’s going on:
https://pythonspeed.com/articles/externally-managed-environment-pep-668/

So my question is, now what?

Jim A

Re: PEP 668 starting to bite me [ In reply to ]

rcrernst at gmail

Jun 20, 2023, 1:03 PM

Post #2 of 8 (266 views)

Permalink

On Tue, Jun 20, 2023 at 7:04?PM James Abernathy <jfabernathy@gmail.com>
wrote:

> I was starting to build mythtv from source on Endeavour OS which is a Arch
> linux based distro. It complied fine but the sudo make install failed due
> to new rules that are starting to creep into OSs. It will affect Debian,
> Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today on
> something that last week didn't cause a problem.
>
> The error is related to installing pip packages. Here is a the console
> error I saw at the end of sudo make install:
>
>
> make[1]: Entering directory
> '/home/jim/build/mythtv/mythtv/bindings/python' python3 -m pip install
> --no-build-isolation --no-cache-dir --no-index --disable-pip-version-check
> --no-deps --find-links ./dist MythTV error:
> externally-managed-environment × This environment is externally managed
> ??> To install Python packages system-wide, try 'pacman -S python-xyz',
> where xyz is the package you are trying to install. If you wish to
> install a non-Arch-packaged Python package, create a virtual environment
> using 'python -m venv path/to/venv'. Then use path/to/venv/bin/python and
> path/to/venv/bin/pip. If you wish to install a non-Arch packaged Python
> application, it may be easiest to use 'pipx install xyz', which will
> manage a virtual environment for you. Make sure you have python-pipx
> installed via pacman. note: If you believe this is a mistake, please
> contact your Python installation or OS distribution provider. You can
> override this, at the risk of breaking your Python installation or OS, by
> passing --break-system-packages hint: See PEP 668 for the detailed
> specification. make[1]: *** [Makefile:35: install] Error 1
>
> This link gives a better explanation of what’s going on:
> https://pythonspeed.com/articles/externally-managed-environment-pep-668/
>
> So my question is, now what?
>
> Jim A
>
>
See https://lists.archive.carbon60.com/mythtv/dev/643410
For advanced users compiling and installing MythTV from source,
do a "export PIP_BREAK_SYSTEM_PACKAGES=1" before the
installation of the local build.

Roland

Re: PEP 668 starting to bite me [ In reply to ]

jfabernathy at gmail

Jun 20, 2023, 1:25 PM

Post #3 of 8 (266 views)

Permalink

On Tue, Jun 20, 2023 at 4:06?PM Roland Ernst <rcrernst@gmail.com> wrote:

>
> On Tue, Jun 20, 2023 at 7:04?PM James Abernathy <jfabernathy@gmail.com>
> wrote:
>
>> I was starting to build mythtv from source on Endeavour OS which is a
>> Arch linux based distro. It complied fine but the sudo make install failed
>> due to new rules that are starting to creep into OSs. It will affect
>> Debian, Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today
>> on something that last week didn't cause a problem.
>>
>> The error is related to installing pip packages. Here is a the console
>> error I saw at the end of sudo make install:
>>
>>
>> make[1]: Entering directory
>> '/home/jim/build/mythtv/mythtv/bindings/python' python3 -m pip install
>> --no-build-isolation --no-cache-dir --no-index --disable-pip-version-check
>> --no-deps --find-links ./dist MythTV error:
>> externally-managed-environment × This environment is externally managed
>> ??> To install Python packages system-wide, try 'pacman -S python-xyz',
>> where xyz is the package you are trying to install. If you wish to
>> install a non-Arch-packaged Python package, create a virtual environment
>> using 'python -m venv path/to/venv'. Then use path/to/venv/bin/python and
>> path/to/venv/bin/pip. If you wish to install a non-Arch packaged Python
>> application, it may be easiest to use 'pipx install xyz', which will
>> manage a virtual environment for you. Make sure you have python-pipx
>> installed via pacman. note: If you believe this is a mistake, please
>> contact your Python installation or OS distribution provider. You can
>> override this, at the risk of breaking your Python installation or OS, by
>> passing --break-system-packages hint: See PEP 668 for the detailed
>> specification. make[1]: *** [Makefile:35: install] Error 1
>>
>> This link gives a better explanation of what’s going on:
>> https://pythonspeed.com/articles/externally-managed-environment-pep-668/
>>
>> So my question is, now what?
>>
>> Jim A
>>
>>
> See https://lists.archive.carbon60.com/mythtv/dev/643410
> For advanced users compiling and installing MythTV from source,
> do a "export PIP_BREAK_SYSTEM_PACKAGES=1" before the
> installation of the local build.
>
> Roland
>

Thanks for that information. I'll give it a shot. The odd thing is I need
it on EndeavourOS and not Archlinux, which EOS is built on.

JIm A

Re: PEP 668 starting to bite me [ In reply to ]

jfabernathy at gmail

Jun 20, 2023, 1:42 PM

Post #4 of 8 (266 views)

Permalink

On Tue, Jun 20, 2023 at 4:25?PM James Abernathy <jfabernathy@gmail.com>
wrote:

>
>
> On Tue, Jun 20, 2023 at 4:06?PM Roland Ernst <rcrernst@gmail.com> wrote:
>
>>
>> On Tue, Jun 20, 2023 at 7:04?PM James Abernathy <jfabernathy@gmail.com>
>> wrote:
>>
>>> I was starting to build mythtv from source on Endeavour OS which is a
>>> Arch linux based distro. It complied fine but the sudo make install failed
>>> due to new rules that are starting to creep into OSs. It will affect
>>> Debian, Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today
>>> on something that last week didn't cause a problem.
>>>
>>> The error is related to installing pip packages. Here is a the console
>>> error I saw at the end of sudo make install:
>>>
>>>
>>> make[1]: Entering directory
>>> '/home/jim/build/mythtv/mythtv/bindings/python' python3 -m pip install
>>> --no-build-isolation --no-cache-dir --no-index --disable-pip-version-check
>>> --no-deps --find-links ./dist MythTV error:
>>> externally-managed-environment × This environment is externally managed
>>> ??> To install Python packages system-wide, try 'pacman -S python-xyz',
>>> where xyz is the package you are trying to install. If you wish to
>>> install a non-Arch-packaged Python package, create a virtual environment
>>> using 'python -m venv path/to/venv'. Then use path/to/venv/bin/python
>>> and path/to/venv/bin/pip. If you wish to install a non-Arch packaged
>>> Python application, it may be easiest to use 'pipx install xyz', which
>>> will manage a virtual environment for you. Make sure you have
>>> python-pipx installed via pacman. note: If you believe this is a
>>> mistake, please contact your Python installation or OS distribution
>>> provider. You can override this, at the risk of breaking your Python
>>> installation or OS, by passing --break-system-packages hint: See PEP 668
>>> for the detailed specification. make[1]: *** [Makefile:35: install]
>>> Error 1
>>>
>>> This link gives a better explanation of what’s going on:
>>> https://pythonspeed.com/articles/externally-managed-environment-pep-668/
>>>
>>> So my question is, now what?
>>>
>>> Jim A
>>>
>>>
>> See https://lists.archive.carbon60.com/mythtv/dev/643410
>> For advanced users compiling and installing MythTV from source,
>> do a "export PIP_BREAK_SYSTEM_PACKAGES=1" before the
>> installation of the local build.
>>
>> Roland
>>
>
> Thanks for that information. I'll give it a shot. The odd thing is I need
> it on EndeavourOS and not Archlinux, which EOS is built on.
>
> JIm A
>

That didn't fix it for me. After make -j5
I did export PIP_BREAK_SYSTEM_PACKAGES=1 then
sudo make install.
Same basic error message as before:

ake -C bindings/python install INSTALL_ROOT=
make[1]: Entering directory '/home/jim/build/mythtv/mythtv/bindings/python'
python3 -m pip install --no-build-isolation --no-cache-dir --no-index
--disable
-pip-version-check --no-deps --find-links ./dist MythTV
error: externally-managed-environment

× This environment is externally managed
??> To install Python packages system-wide, try 'pacman -S
python-xyz', where xyz is the package you are trying to
install.

If you wish to install a non-Arch-packaged Python package,
create a virtual environment using 'python -m venv path/to/venv'.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.

If you wish to install a non-Arch packaged Python application,
it may be easiest to use 'pipx install xyz', which will manage a
virtual environment for you. Make sure you have python-pipx
installed via pacman.

note: If you believe this is a mistake, please contact your Python
installation o
r OS distribution provider. You can override this, at the risk of breaking
your P
ython installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
make[1]: *** [Makefile:35: install] Error 1

Jim A

Re: PEP 668 starting to bite me [ In reply to ]

rcrernst at gmail

Jun 20, 2023, 2:02 PM

Post #5 of 8 (266 views)

Permalink

On Tue, Jun 20, 2023 at 10:42?PM James Abernathy <jfabernathy@gmail.com>
wrote:

>
>
> On Tue, Jun 20, 2023 at 4:25?PM James Abernathy <jfabernathy@gmail.com>
> wrote:
>
>>
>>
>> On Tue, Jun 20, 2023 at 4:06?PM Roland Ernst <rcrernst@gmail.com> wrote:
>>
>>>
>>> On Tue, Jun 20, 2023 at 7:04?PM James Abernathy <jfabernathy@gmail.com>
>>> wrote:
>>>
>>>> I was starting to build mythtv from source on Endeavour OS which is a
>>>> Arch linux based distro. It complied fine but the sudo make install failed
>>>> due to new rules that are starting to creep into OSs. It will affect
>>>> Debian, Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today
>>>> on something that last week didn't cause a problem.
>>>>
>>>> The error is related to installing pip packages. Here is a the console
>>>> error I saw at the end of sudo make install:
>>>>
>>>>
>>>> make[1]: Entering directory
>>>> '/home/jim/build/mythtv/mythtv/bindings/python' python3 -m pip install
>>>> --no-build-isolation --no-cache-dir --no-index --disable-pip-version-check
>>>> --no-deps --find-links ./dist MythTV error:
>>>> externally-managed-environment × This environment is externally
>>>> managed ??> To install Python packages system-wide, try 'pacman -S
>>>> python-xyz', where xyz is the package you are trying to install. If
>>>> you wish to install a non-Arch-packaged Python package, create a
>>>> virtual environment using 'python -m venv path/to/venv'. Then use path/
>>>> to/venv/bin/python and path/to/venv/bin/pip. If you wish to install a
>>>> non-Arch packaged Python application, it may be easiest to use 'pipx
>>>> install xyz', which will manage a virtual environment for you. Make
>>>> sure you have python-pipx installed via pacman. note: If you believe
>>>> this is a mistake, please contact your Python installation or OS
>>>> distribution provider. You can override this, at the risk of breaking
>>>> your Python installation or OS, by passing --break-system-packages
>>>> hint: See PEP 668 for the detailed specification. make[1]: ***
>>>> [Makefile:35: install] Error 1
>>>>
>>>> This link gives a better explanation of what’s going on:
>>>> https://pythonspeed.com/articles/externally-managed-environment-pep-668/
>>>>
>>>> So my question is, now what?
>>>>
>>>> Jim A
>>>>
>>>>
>>> See https://lists.archive.carbon60.com/mythtv/dev/643410
>>> For advanced users compiling and installing MythTV from source,
>>> do a "export PIP_BREAK_SYSTEM_PACKAGES=1" before the
>>> installation of the local build.
>>>
>>> Roland
>>>
>>
>> Thanks for that information. I'll give it a shot. The odd thing is I need
>> it on EndeavourOS and not Archlinux, which EOS is built on.
>>
>> JIm A
>>
>
> That didn't fix it for me. After make -j5
> I did export PIP_BREAK_SYSTEM_PACKAGES=1 then
> sudo make install.
> Same basic error message as before:
>
> ake -C bindings/python install INSTALL_ROOT=
> make[1]: Entering directory
> '/home/jim/build/mythtv/mythtv/bindings/python'
> python3 -m pip install --no-build-isolation --no-cache-dir --no-index
> --disable
> -pip-version-check --no-deps --find-links ./dist MythTV
> error: externally-managed-environment
>
> × This environment is externally managed
> ??> To install Python packages system-wide, try 'pacman -S
> python-xyz', where xyz is the package you are trying to
> install.
>
> If you wish to install a non-Arch-packaged Python package,
> create a virtual environment using 'python -m venv path/to/venv'.
> Then use path/to/venv/bin/python and path/to/venv/bin/pip.
>
> If you wish to install a non-Arch packaged Python application,
> it may be easiest to use 'pipx install xyz', which will manage a
> virtual environment for you. Make sure you have python-pipx
> installed via pacman.
>
> note: If you believe this is a mistake, please contact your Python
> installation o
> r OS distribution provider. You can override this, at the risk of breaking
> your P
> ython installation or OS, by passing --break-system-packages.
> hint: See PEP 668 for the detailed specification.
> make[1]: *** [Makefile:35: install] Error 1
>
> Jim A
>
>
Try
sudo PIP_BREAK_SYSTEM_PACKAGES=1 make install

Once again:
This is only recommended for test installations of MythTV
Use the default way to build packages accepted by the official package
manager of the OS distributions you are running.
You did it already for Debian as shown in previous mails.
For production systems use a similar way for Endeavour OS.
The OS maintainers have very good reasons to restrict python pip to install
packages system-wide!

Roland

Re: PEP 668 starting to bite me [ In reply to ]

jfabernathy at gmail

Jun 20, 2023, 2:53 PM

Post #6 of 8 (266 views)

Permalink

On Tue, Jun 20, 2023 at 5:04?PM Roland Ernst <rcrernst@gmail.com> wrote:

>
>
> On Tue, Jun 20, 2023 at 10:42?PM James Abernathy <jfabernathy@gmail.com>
> wrote:
>
>>
>>
>> On Tue, Jun 20, 2023 at 4:25?PM James Abernathy <jfabernathy@gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Tue, Jun 20, 2023 at 4:06?PM Roland Ernst <rcrernst@gmail.com> wrote:
>>>
>>>>
>>>> On Tue, Jun 20, 2023 at 7:04?PM James Abernathy <jfabernathy@gmail.com>
>>>> wrote:
>>>>
>>>>> I was starting to build mythtv from source on Endeavour OS which is a
>>>>> Arch linux based distro. It complied fine but the sudo make install failed
>>>>> due to new rules that are starting to creep into OSs. It will affect
>>>>> Debian, Ubuntu, Fedora, Archlinux, etc very soon. It happened to me today
>>>>> on something that last week didn't cause a problem.
>>>>>
>>>>> The error is related to installing pip packages. Here is a the console
>>>>> error I saw at the end of sudo make install:
>>>>>
>>>>>
>>>>> make[1]: Entering directory
>>>>> '/home/jim/build/mythtv/mythtv/bindings/python' python3 -m pip
>>>>> install --no-build-isolation --no-cache-dir --no-index
>>>>> --disable-pip-version-check --no-deps --find-links ./dist MythTV
>>>>> error: externally-managed-environment × This environment is
>>>>> externally managed ??> To install Python packages system-wide, try 'pacman
>>>>> -S python-xyz', where xyz is the package you are trying to install. If
>>>>> you wish to install a non-Arch-packaged Python package, create a
>>>>> virtual environment using 'python -m venv path/to/venv'. Then use
>>>>> path/to/venv/bin/python and path/to/venv/bin/pip. If you wish to
>>>>> install a non-Arch packaged Python application, it may be easiest to
>>>>> use 'pipx install xyz', which will manage a virtual environment for
>>>>> you. Make sure you have python-pipx installed via pacman. note: If
>>>>> you believe this is a mistake, please contact your Python
>>>>> installation or OS distribution provider. You can override this, at
>>>>> the risk of breaking your Python installation or OS, by passing
>>>>> --break-system-packages hint: See PEP 668 for the detailed
>>>>> specification. make[1]: *** [Makefile:35: install] Error 1
>>>>>
>>>>> This link gives a better explanation of what’s going on:
>>>>> https://pythonspeed.com/articles/externally-managed-environment-pep-668/
>>>>>
>>>>> So my question is, now what?
>>>>>
>>>>> Jim A
>>>>>
>>>>>
>>>> See https://lists.archive.carbon60.com/mythtv/dev/643410
>>>> For advanced users compiling and installing MythTV from source,
>>>> do a "export PIP_BREAK_SYSTEM_PACKAGES=1" before the
>>>> installation of the local build.
>>>>
>>>> Roland
>>>>
>>>
>>> Thanks for that information. I'll give it a shot. The odd thing is I
>>> need it on EndeavourOS and not Archlinux, which EOS is built on.
>>>
>>> JIm A
>>>
>>
>> That didn't fix it for me. After make -j5
>> I did export PIP_BREAK_SYSTEM_PACKAGES=1 then
>> sudo make install.
>> Same basic error message as before:
>>
>> ake -C bindings/python install INSTALL_ROOT=
>> make[1]: Entering directory
>> '/home/jim/build/mythtv/mythtv/bindings/python'
>> python3 -m pip install --no-build-isolation --no-cache-dir --no-index
>> --disable
>> -pip-version-check --no-deps --find-links ./dist MythTV
>> error: externally-managed-environment
>>
>> × This environment is externally managed
>> ??> To install Python packages system-wide, try 'pacman -S
>> python-xyz', where xyz is the package you are trying to
>> install.
>>
>> If you wish to install a non-Arch-packaged Python package,
>> create a virtual environment using 'python -m venv path/to/venv'.
>> Then use path/to/venv/bin/python and path/to/venv/bin/pip.
>>
>> If you wish to install a non-Arch packaged Python application,
>> it may be easiest to use 'pipx install xyz', which will manage a
>> virtual environment for you. Make sure you have python-pipx
>> installed via pacman.
>>
>> note: If you believe this is a mistake, please contact your Python
>> installation o
>> r OS distribution provider. You can override this, at the risk of
>> breaking your P
>> ython installation or OS, by passing --break-system-packages.
>> hint: See PEP 668 for the detailed specification.
>> make[1]: *** [Makefile:35: install] Error 1
>>
>> Jim A
>>
>>
> Try
> sudo PIP_BREAK_SYSTEM_PACKAGES=1 make install
>
> Once again:
> This is only recommended for test installations of MythTV
> Use the default way to build packages accepted by the official package
> manager of the OS distributions you are running.
> You did it already for Debian as shown in previous mails.
> For production systems use a similar way for Endeavour OS.
> The OS maintainers have very good reasons to restrict python pip to
> install packages system-wide!
>
> Roland
>

Thanks for the suggestion. It worked.

I understand your warning. I've been asked to test master specifically for
the new Web App. It is officially built only on Ubuntu as v34. In fact
that is the only official package build I'm aware of. Arch has an AUR
package that sort of works for v33 only. That is what you use for EOS but
again doesn't work for master and still needs this workaround you pointed
me to.

PEP 668 has the potential to reduce Mythtv to Ubuntu only which will kill
it since Canonical wants everything to be a snap package. I've moved away
from Ubuntu to Debian 12 for now. There is no package officially supported
for that either.

Jim A

Re: PEP 668 starting to bite me [ In reply to ]

stephen_agent at jsw

Jun 21, 2023, 3:07 AM

Post #7 of 8 (263 views)

Permalink

On Tue, 20 Jun 2023 17:53:41 -0400, you wrote:

>PEP 668 has the potential to reduce Mythtv to Ubuntu only which will kill
>it since Canonical wants everything to be a snap package. I've moved away
>from Ubuntu to Debian 12 for now. There is no package officially supported
>for that either.

Actually, Ubuntu has decided to abandon snap:

https://news.itsfoss.com/ubuntu-ditch-snap/
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: PEP 668 starting to bite me [ In reply to ]

jfabernathy at gmail

Jun 21, 2023, 3:18 AM

Post #8 of 8 (263 views)

Permalink

On Wed, Jun 21, 2023 at 6:09?AM Stephen Worthington <
stephen_agent@jsw.gen.nz> wrote:

> On Tue, 20 Jun 2023 17:53:41 -0400, you wrote:
>
> >PEP 668 has the potential to reduce Mythtv to Ubuntu only which will kill
> >it since Canonical wants everything to be a snap package. I've moved away
> >from Ubuntu to Debian 12 for now. There is no package officially supported
> >for that either.
>
> Actually, Ubuntu has decided to abandon snap:
>
> https://news.itsfoss.com/ubuntu-ditch-snap/
> _______________________________________________
>
> That was a funny April Fools day gag. I saw it back on 4/1

Jim A