Mailing List Archive

Moving to Python 3 problems
I'm working through updating the mythtv package at RPM Fusion for python 3
since the future Fedora 30 release will be python3 only (maybe not only,
but packages are being moved over).

While building I see the following from configure:

WARNING: disabling Python bindings; missing MySQLdb
WARNING: disabling Python bindings; missing lxml

Both python3-mysql and python3-lxml are installed and there doesn't seem to
be any configure options specific to those packages. I am specifying

--python=/usr/bin/python3

Thanks,
Richard
Re: Moving to Python 3 problems [ In reply to ]
The actual plan, as I understand it, is a bit more
complicated. First, python2 will be marked as
deprecated, but will still be available (for those
that need it), but the default installs will not
include python2 as python2 will no longer be
supported. Second, the python alias will point
to python3, rather than python2 (so shebangs
that specify python will need to change if they
really require python2).

For MythTV, moving to python3 will require going
through all the bindings and fixing not only the
code, but some of the build/install processes
(Qt has made some changes that indirectly
effect these things too). There are some
automated tools that assist the process.

Going through the python bindings is on my
(long) list to look at, but is behind a number of
other things that are also on the list. If you are
are a developer that can do the work, I will delete
it from my (eventual) todo list. Thanks!
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Moving to Python 3 problems [ In reply to ]
On Fri, Nov 30, 2018 at 8:45 AM Gary Buhrmaster <gary.buhrmaster@gmail.com>
wrote:

> The actual plan, as I understand it, is a bit more
> complicated. First, python2 will be marked as
> deprecated, but will still be available (for those
> that need it), but the default installs will not
> include python2 as python2 will no longer be
> supported. Second, the python alias will point
> to python3, rather than python2 (so shebangs
> that specify python will need to change if they
> really require python2).
>

Unfortunately some packages have already moved to being python3 only...



> For MythTV, moving to python3 will require going
> through all the bindings and fixing not only the
> code, but some of the build/install processes
> (Qt has made some changes that indirectly
> effect these things too). There are some
> automated tools that assist the process.
>
> Going through the python bindings is on my
> (long) list to look at, but is behind a number of
> other things that are also on the list. If you are
> are a developer that can do the work, I will delete
> it from my (eventual) todo list. Thanks!
>

I know enough to hack at it but I wouldn't consider myself a developer...

Thanks,
Richard
Re: Moving to Python 3 problems [ In reply to ]
On Fri, 2018-11-30 at 14:45 +0000, Gary Buhrmaster wrote:
> The actual plan, as I understand it, is a bit more
> complicated. First, python2 will be marked as
> deprecated, but will still be available (for those
> that need it), but the default installs will not
> include python2 as python2 will no longer be
> supported. Second, the python alias will point
> to python3, rather than python2 (so shebangs
> that specify python will need to change if they
> really require python2).
>
> For MythTV, moving to python3 will require going
> through all the bindings and fixing not only the
> code, but some of the build/install processes
> (Qt has made some changes that indirectly
> effect these things too). There are some
> automated tools that assist the process.
>
> Going through the python bindings is on my
> (long) list to look at, but is behind a number of
> other things that are also on the list. If you are
> are a developer that can do the work, I will delete
> it from my (eventual) todo list. Thanks!

Its on my long list too. My concern is that there are no tests for the
basic functionality of the bindings, so how would I know if I'd broken
them in the conversion. Thinking about that makes me think I should
write test cases first, but to do that there would need to be a
standardized database to test against that's in the source, which leads
me to think that we need an option for an embedded mysql database for
testing, which makes me go think about other things.

I'd be happy to take a swing at it after I finish my current project,
but I'll definitely need some help on the testing side. It would be
nice to get this into the next release...

David


_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Moving to Python 3 problems [ In reply to ]
On Fri, Nov 30, 2018 at 3:38 PM David Hampton <mythtv@love2code.net> wrote:
> ... which leads
> me to think that we need an option for an embedded mysql database for
> testing, which makes me go think about other things.

I think the embedded database was on Micheal's list.
Maybe prod him to get-r-done? Yeah, not likely to
happen any time soonish.

Presuming that all the existing python bindings are
still useful (and reviewing them should be step 0),
the bindings and/or scripts that use bindings should
possibly be migrated and/or refactored to use the
services APIs where practical to avoid dealing
with direct database access that at least some of
the existing functionality uses (and where the services
API is not sufficient, change the services API?).

Of course that just means you now also need a
functional BE for unit testing of the services API.

I, too, have thought a bit about it and turned what
some might look at as simple into a bigger thing,
and decided that punting to someone with more
immediate interest was the much easier path (I
don't use the bindings, so if they disappeared
tomorrow I would not notice).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Moving to Python 3 problems [ In reply to ]
On 11/30/18 9:37 AM, David Hampton wrote:
> Its on my long list too. My concern is that there are no tests for the
> basic functionality of the bindings, so how would I know if I'd broken
> them in the conversion. Thinking about that makes me think I should
> write test cases first, but to do that there would need to be a
> standardized database to test against that's in the source, which leads
> me to think that we need an option for an embedded mysql database for
> testing, which makes me go think about other things.
>
> I'd be happy to take a swing at it after I finish my current project,
> but I'll definitely need some help on the testing side. It would be
> nice to get this into the next release...
>
> David

I've used the 2to3 tool: 2to3-2.7 --no-backups --no-diffs --write .
from the .../mythtv/mythtv/bindings/python directory.

It completes without error (tested on ubuntu 18.10.) For now, just
adding PYTHON=python3 in the Makefile after the include line. make and
sudo make install also complete.

But programs error out. So there appears to be more things that the tool
doesn't catch. I've only solved one, and it was very simple (removing all
's from a MySQLdb.version_info >= ('1','2','2') line.)

I couldn't find any unit tests in the bindings either, let alone data.

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Moving to Python 3 problems [ In reply to ]
On 11/30/18 10:15 AM, Gary Buhrmaster wrote:
> Presuming that all the existing python bindings are
> still useful (and reviewing them should be step 0),
> the bindings and/or scripts that use bindings should
> possibly be migrated and/or refactored to use the
> services APIs where practical to avoid dealing
> with direct database access that at least some of
> the existing functionality uses (and where the services
> API is not sufficient, change the services API?).

For new programs, and for those that want to convert their
existing ones,

In v30-Pre, there is now a Services API module that cares for
access to the backend/frontend. Folks will have to learn the names
of the endpoints and parameters sent/returned. But that's similar to
the existing bindings. The API Wiki needs some work (I've got four
on my TODO list.)

The MythTV.services_api module is Python version agnostic.
But currently installs under 2.x (most likely x=7.) A symbolic
link makes it work for both v2 and v3

Normal Python help() works or an alias is my choice:
alias apihelp='echo -e "from MythTV.services_api import send\nhelp(send)" | python | less'

Examples, *very* simple, no error checking etc.:

#!/usr/bin/env python2 <----------------------------------
# -*- coding: utf-8 -*-

''' Very short v2 example that prints upcoming programs '''

from __future__ import print_function
from MythTV import MythBE

BACKEND = MythBE()
UPCOMING_COUNT = 0

for recording in BACKEND.getUpcomingRecordings():
if recording.subtitle is None:
recording.subtitle = ''
print(' {0:<7.7} {1:<14.14}{2:<27.27} {3:<39.39}'
.format(recording.callsign,
recording.starttime.strftime("%b %d %H:%M"),
recording.title.encode('utf-8'),
recording.subtitle.encode('utf-8')))
UPCOMING_COUNT += 1

print('\n Upcoming Shows: {}.\n'.format(UPCOMING_COUNT))



#!/usr/bin/env python3 <----------------------------------
# -*- coding: utf-8 -*-

''' Very short v3 example that prints upcoming programs '''

import sys
from MythTV.services_api import send as api, utilities as util

BACKEND = api.Send(host=sys.argv[1])

RESPONSE = BACKEND.send(endpoint='Dvr/GetUpcomingList')

COUNT = int(RESPONSE['ProgramList']['TotalAvailable'])

PROGS = RESPONSE['ProgramList']['Programs']

util.get_utc_offset(BACKEND)

for upcoming in range(COUNT):
startts = util.utc_to_local(PROGS[upcoming]['Recording']['StartTs'])
print(' {} {:45.45} {:15.15}'.format(
startts,
PROGS[upcoming]['Title'],
PROGS[upcoming]['SubTitle']))

print('\nUpcoming Shows: {}.\n'.format(COUNT))

--
Bill
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: Moving to Python 3 problems [ In reply to ]
I haven't had much time to look closer at this but I do have one question
that hopefully has a "relatively" easy answer...

Currently the only package causing issues for building for Fedora is that
python2-requests-cache was dropped in favor of python3-requests-cache...

To my knowledge this only affect ttvdb.py which is supposedly python 3
ready.

Is it possible to eat this elephant one part at a time? Can just the
requirements for ttvdb.py be updated to python3?

Thanks,
Richard