Mailing List Archive

Is MythTV ready for Python 3.9?
Python on Fedora Rawhide is being updated to 3.9 as we speak.

I don't have a good way to test this until it's in the stable repository
since I'm still not allowed to build MythTV in COPR due to ffmpeg, so I
thought I'd ask if anyone has done any testing yet.

Thanks,
Richard
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Thu, May 28, 2020 at 12:16 PM Richard Shaw <hobbes1069@gmail.com> wrote:
>
> Python on Fedora Rawhide is being updated to 3.9 as we speak.
>
> I don't have a good way to test this until it's in the stable repository since I'm still not allowed to build MythTV in COPR due to ffmpeg, so I thought I'd ask if anyone has done any testing yet.

Well, using something like:

koji mock-config --tag f33-python --arch=x86_64 f33

one can get the fragments needed to configure mock
using the side tag for the python 3.9 rebuild (there are
various ways to utilize that fragment) and run the
build locally. And while I rarely care about side-tags
before they land in the updates/stable repo, the
approach has come in handy from time to time.

I have done so, and MythTV builds. I have no
idea if it actually runs (that would be actual QA
work), and I would not be surprised if some 3rd
party python3 package might need an update
or three (since, as we all know, building does
not always equal working).

Anyway, once the side-tag hits updates/stable
the MythTV buildbot network should build with
it and identify any issues since there is a fedora
rawhide builder (and from time to time some
change in rawhide does require changes
elsewhere, which is why that builder is a useful
canary).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Thu, May 28, 2020 at 11:51 AM Gary Buhrmaster <gary.buhrmaster@gmail.com>
wrote:

> On Thu, May 28, 2020 at 12:16 PM Richard Shaw <hobbes1069@gmail.com>
> wrote:
> >
> > Python on Fedora Rawhide is being updated to 3.9 as we speak.
> >
> > I don't have a good way to test this until it's in the stable repository
> since I'm still not allowed to build MythTV in COPR due to ffmpeg, so I
> thought I'd ask if anyone has done any testing yet.
>
> Well, using something like:
>
> koji mock-config --tag f33-python --arch=x86_64 f33
>

Wow, I've been a packager for 12+ years and didn't know this trick. Learn
something new every day!

Yesterday I learned that you can have rpmbuild use git for applying
patches, which was needed because there was a binary patch for one of the
translation files which regular patch doesn't handle.

%autosetup -S git
or
%patch -S git

Thanks,
Richard
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On 5/28/20 11:49 AM, Gary Buhrmaster wrote:
> On Thu, May 28, 2020 at 12:16 PM Richard Shaw <hobbes1069@gmail.com> wrote:
>>
>> Python on Fedora Rawhide is being updated to 3.9 as we speak.
>>
>> I don't have a good way to test this until it's in the stable repository since I'm still not allowed to build MythTV in COPR due to ffmpeg, so I thought I'd ask if anyone has done any testing yet.
>
> Well, using something like:
>
> koji mock-config --tag f33-python --arch=x86_64 f33

Ubuntu person here. I've got Rawhide running (at least
cat /etc/fedora-release = Fedora release 33 (Rawhide)).

sudo dnf install python3.9 did the expected.

Is there a way to get things like: dnf install python3-lxml
to find a 3.9 version?

> one can get the fragments needed to configure mock
> using the side tag for the python 3.9 rebuild (there are
> various ways to utilize that fragment) and run the
> build locally. And while I rarely care about side-tags
> before they land in the updates/stable repo, the
> approach has come in handy from time to time.
>
> I have done so, and MythTV builds. I have no
> idea if it actually runs (that would be actual QA
> work), and I would not be surprised if some 3rd
> party python3 package might need an update
> or three (since, as we all know, building does
> not always equal working).
>
> Anyway, once the side-tag hits updates/stable
> the MythTV buildbot network should build with
> it and identify any issues since there is a fedora
> rawhide builder (and from time to time some
> change in rawhide does require changes
> elsewhere, which is why that builder is a useful
> canary).

Thanks,

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Fri, May 29, 2020 at 4:34 PM Bill Meek <keemllib@gmail.com> wrote:

> On 5/28/20 11:49 AM, Gary Buhrmaster wrote:
> > On Thu, May 28, 2020 at 12:16 PM Richard Shaw <hobbes1069@gmail.com>
> wrote:
> >>
> >> Python on Fedora Rawhide is being updated to 3.9 as we speak.
> >>
> >> I don't have a good way to test this until it's in the stable
> repository since I'm still not allowed to build MythTV in COPR due to
> ffmpeg, so I thought I'd ask if anyone has done any testing yet.
> >
> > Well, using something like:
> >
> > koji mock-config --tag f33-python --arch=x86_64 f33
>
> Ubuntu person here. I've got Rawhide running (at least
> cat /etc/fedora-release = Fedora release 33 (Rawhide)).
>
> sudo dnf install python3.9 did the expected.
>
> Is there a way to get things like: dnf install python3-lxml
> to find a 3.9 version?
>

The side tag was just merged, so pending some mirrors syncing the --tag
f33-python should no longer be necessary.

Thanks,
Richard
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Fri, May 29, 2020 at 10:21 PM Richard Shaw <hobbes1069@gmail.com> wrote:

>
> The side tag was just merged, so pending some mirrors syncing the --tag f33-python should no longer be necessary.
>

(as follow-on to Bill Meek in particular, but also
background to others for future expectation
setting....).

In addition to the usual mirror delays (typically a
day or so for major merges, but have been known
to be somewhat longer), there are also a handful
of random packages that failed to rebuild, mostly
due issues outside of the python 3.9 uplift (they
would likely not have successfully rebuilt in python
3.8 due to other issues)(*).

The package wranglers are working through
those random packages (although AFAIK none
are directly required by MythTV; as I said,
using the side tag I was able to build MythTV
with python 3.9).

Fedora is the best of times(**), and the worst of
times(***). Some settling will occur during
installation(****).


(*) There is always a trade-off when doing a
major uplift as to when to merge, as there
are always a few packages that will fail
to rebuild, but you don't want to wait for
100% to get fixed before the merge. The
proposal owner has to make an decision
as when the rebuild is "good enough" to
merge for Fedora-Next, and expect the
package wranglers to do the usual
cleanup for their specific issues.
(**) Often the latest of the latest with all the
newest shiny included. Yeah for leading
edge capabilities!
(***) Often the latest of the latest with all the
newest poo that needs to be cleaned up
(often by the upstream) once the rubber
hits the road (and you may need a bigger
power washer to clean up some of that
poo).
(****) While *some* package wranglers are
either upstream maintainers and/or
experienced code engineers, a lot are
not (they packaged the package because
*they* wanted it available), and they depend
on the upstream SMEs to do some of the
heavier lifting, or at least provide the correct
advice for moving forward, so some delays
for rebuilding some packages may happen.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
> …
> (often by the upstream) once the rubber
> hits the road (and you may need a bigger
> power washer to clean up some of that


Didn't know the packaging tools had a power washer option!


Back in my day, closest was rpm —erase ...


:-)


--
Nigel Pearson, 02 9792 6998, 0408 66 44 35
nigel.pearson.au@gmail.com
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On 5/29/20 6:52 PM, Gary Buhrmaster wrote:
> The package wranglers are working through
> those random packages (although AFAIK none
> are directly required by MythTV; as I said,
> using the side tag I was able to build MythTV
> with python 3.9).

I'm thinking there another step I don't {know|know how} to run.

When you do: python3 --version, does it return 3.9?

If not, then was: ./configure --python=python3.9 used?

I looked in 3.9 site-packages and the directory is empty
except for an empty __pycache__ and a README.txt.

Running ansible with ansible_python_interpreter=python3.9 fails
as there's no python3-dnf.

Thanks,
--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On 5/29/20 8:49 PM, Bill Meek wrote:
> On 5/29/20 6:52 PM, Gary Buhrmaster wrote:
>> The package wranglers are working through
>> those random packages (although AFAIK none
>> are directly required by MythTV; as I said,
>> using the side tag I was able to build MythTV
>> with python 3.9).
>
> I'm thinking there another step I don't {know|know how} to run.
>
> When you do: python3 --version, does it return 3.9?
>
> If not, then was: ./configure --python=python3.9 used?
>
> I looked in 3.9 site-packages and the directory is empty
> except for an empty __pycache__ and a README.txt.

Ignore the noise please. ./configure OK, building now.

Did an update this morning, and like Richard said, 3.9 packages now
appear in /usr/lib64/python3.9/site-packages.

ansible still relies on 3.8 and won't install, but I'll just give it
some time.

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Sat, May 30, 2020 at 2:39 PM Bill Meek <keemllib@gmail.com> wrote:

> ansible still relies on 3.8 and won't install, but I'll just give it
> some time.

Ansible is one of the (random) packages that
need further work to build in 3.9 that were
alluded to earlier(*). There was a fix applied
upstream to ansible a while ago to address
the issue, but the (initial) build for the python
3.9 side tag did not also include any new level
of ansible with the patch (that work would be
up to the ansible packagers). The backport
of the fix has been since been applied, and
a build has been successful, so it should
make it to the rawhide repo in another day
(or two).




(*) Perhaps amusingly, the failure was in the
test suite, which has explicit testing for the
python version, and it previously did not
include 3.9 as a valid python. So the test
suite proved that the test suite failed.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On 5/30/20 1:27 PM, Gary Buhrmaster wrote:
> On Sat, May 30, 2020 at 2:39 PM Bill Meek <keemllib@gmail.com> wrote:
>
>> ansible still relies on 3.8 and won't install, but I'll just give it
>> some time.
>
> Ansible is one of the (random) packages that
> need further work to build in 3.9 that were
> alluded to earlier(*). There was a fix applied
> upstream to ansible a while ago to address
> the issue, but the (initial) build for the python
> 3.9 side tag did not also include any new level
> of ansible with the patch (that work would be
> up to the ansible packagers). The backport
> of the fix has been since been applied, and
> a build has been successful, so it should
> make it to the rawhide repo in another day
> (or two).
>
>
>
>
> (*) Perhaps amusingly, the failure was in the
> test suite, which has explicit testing for the
> python version, and it previously did not
> include 3.9 as a valid python. So the test
> suite proved that the test suite failed.

Ansible works OK now. The only error I'm seeing with the
Python bindings in 3.9 is:

File "/usr/local/lib/python3.9/site-packages/MythTV/dataheap.py", line 18, in <module>
import xml.etree.cElementTree as etree
ModuleNotFoundError: No module named 'xml.etree.cElementTree'

There's a simple solution, but is cElementTree an
example of something that will show up in the
future? Any script that uses: from MythTV import MythDB
will fail.

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Sun, Jun 21, 2020 at 5:19 PM Bill Meek <keemllib@gmail.com> wrote:

> There's a simple solution, but is cElementTree an
> example of something that will show up in the
> future?

Never say never, but per the docs at
https://docs.python.org/3/library/xml.etree.elementtree.html
... cElementTree ... is deprecated (and ElementTree
will use a fast implementation when possible as of
Python 3.3 (ages ago)).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Sun, Jun 21, 2020 at 7:19 PM Bill Meek <keemllib@gmail.com> wrote:

>
>
> Ansible works OK now. The only error I'm seeing with the
> Python bindings in 3.9 is:
>
> File "/usr/local/lib/python3.9/site-packages/MythTV/dataheap.py", line
> 18, in <module>
> import xml.etree.cElementTree as etree
> ModuleNotFoundError: No module named 'xml.etree.cElementTree'
>
> There's a simple solution, but is cElementTree an
> example of something that will show up in the
> future? Any script that uses: from MythTV import MythDB
> will fail.
>
> --
> Bill
>
>
That's not the only error:
Python3.9 moved the 'Abstract Base Classes' from 'collections' to
'collections.abc'.
Affected grabber modules:
- tmdb.py
- tvdb.py
The classes 'Sequence', 'Iterator', 'Iterable' are now members of
'collections.abc'

The module 'xml' removed the methods 'getchildren()' and other methods.
I need to check if pythons 'lxml' module does the same.

I will set up a 'Python Virtual Environment' for Python3.9 with the modules
selected for fedora33:
- python3-mysqlclient-1.4.6-2.fc33.x86_64.rpm
- python3-lxml-4.5.1-1.fc33.x86_64.rpm
- python3-requests-2.23.0-4.fc33.noarch.rpm
- python3-requests-cache-0.5.1-5.fc33.noarch.rpm
- python3-future-0.18.2-6.fc33.noarch.rpm
- python3-simplejson-3.17.0-4.fc33.x86_64.rpm
- python3-urllib3-1.25.8-3.fc33.noarch.rpm
and run my tests with it.

See the bug tacker for python3 at fedora at
https://bugzilla.redhat.com/show_bug.cgi?id=1785415
for the changes the fedora maintainers raised: It's quite a lot that broke.

BTW: Though 'xml.etree.cElementTree' was deprecated for a long time, it
never raised a
'deprecation warning' in my tests. Therefore, it is hard to find it up
front.

Roland
Re: Is MythTV ready for Python 3.9? [ In reply to ]
On Mon, Jun 22, 2020 at 8:11 PM Roland Ernst <rcrernst@gmail.com> wrote:

>
> On Sun, Jun 21, 2020 at 7:19 PM Bill Meek <keemllib@gmail.com> wrote:
>
>>
>>
>> Ansible works OK now. The only error I'm seeing with the
>> Python bindings in 3.9 is:
>>
>> File "/usr/local/lib/python3.9/site-packages/MythTV/dataheap.py", line
>> 18, in <module>
>> import xml.etree.cElementTree as etree
>> ModuleNotFoundError: No module named 'xml.etree.cElementTree'
>>
>> There's a simple solution, but is cElementTree an
>> example of something that will show up in the
>> future? Any script that uses: from MythTV import MythDB
>> will fail.
>>
>> --
>> Bill
>>
>>
> That's not the only error:
> Python3.9 moved the 'Abstract Base Classes' from 'collections' to
> 'collections.abc'.
> Affected grabber modules:
> - tmdb.py
> - tvdb.py
> The classes 'Sequence', 'Iterator', 'Iterable' are now members of
> 'collections.abc'
>
> The module 'xml' removed the methods 'getchildren()' and other methods.
> I need to check if pythons 'lxml' module does the same.
>
> I will set up a 'Python Virtual Environment' for Python3.9 with the
> modules selected for fedora33:
> - python3-mysqlclient-1.4.6-2.fc33.x86_64.rpm
> - python3-lxml-4.5.1-1.fc33.x86_64.rpm
> - python3-requests-2.23.0-4.fc33.noarch.rpm
> - python3-requests-cache-0.5.1-5.fc33.noarch.rpm
> - python3-future-0.18.2-6.fc33.noarch.rpm
> - python3-simplejson-3.17.0-4.fc33.x86_64.rpm
> - python3-urllib3-1.25.8-3.fc33.noarch.rpm
> and run my tests with it.
>
> See the bug tacker for python3 at fedora at
> https://bugzilla.redhat.com/show_bug.cgi?id=1785415
> for the changes the fedora maintainers raised: It's quite a lot that broke.
>
> BTW: Though 'xml.etree.cElementTree' was deprecated for a long time, it
> never raised a
> 'deprecation warning' in my tests. Therefore, it is hard to find it up
> front.
>
> Roland
>
>
I need to revoke my last mail.
Please excuse the noise.

According to the latest documentation, python3.9 only emits a deprecation
warning
if 'Abstract Base Classes' from 'collections 'are imported instead of
'collections.abc'.
No error is reported.

See https://docs.python.org/3.9/whatsnew/3.9.html

$ python3 -W default
Python 3.9.0b3 (default, Jun 10 2020, 06:01:41)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import Sequence
<stdin>:1: DeprecationWarning: Using or importing the ABCs from
'collections'
instead of from 'collections.abc' is deprecated since Python 3.3, and in
3.10 it will stop working

And pythons 'lxml' module 4.5.1 for python3.9 still allows the method
'getchildren()'.
Tested with python3-lxml-4.5.1 and current master of MythTV.

So far, so good ( at least, for python3.9 ).

I will resolve this deprecation warning soon.

Roland