Mailing List Archive

On 4/18/21 2:23 AM, Jan Ceuleers wrote:
> Hello,
>
> I've been importing new (old) media into mythvideo and I had misnumbered
> some of the episodes. This led me to notice that when tmdb3.py tries to
> look up metadata in respect of episodes that don't exist in
> themoviedb.org it spins consuming 100% of a CPU core.
>
> That's when the grabber is run from mythfrontend, but I can't reproduce
> that same behaviour when running the grabber from the command line.
> Instead I get this:
>
> mythtv@dracor:~$ /usr/share/mythtv/metadata/Television/tmdb3tv.py -l en
> -a GB -D 873 11 1
> Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
> 117, in open
> return _py23urlopen(self)
> File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
> return opener.open(url, data, timeout)
> File "/usr/lib/python3.6/urllib/request.py", line 532, in open
> response = meth(req, response)
> File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
> 'http', request, response, code, msg, hdrs)
> File "/usr/lib/python3.6/urllib/request.py", line 570, in error
> return self._call_chain(*args)
> File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
> result = func(*args)
> File "/usr/lib/python3.6/urllib/request.py", line 650, in
> http_error_default
> raise HTTPError(req.full_url, code, msg, hdrs, fp)
> urllib.error.HTTPError: HTTP Error 404: Not Found
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
> 133, in readJSON
> data = json.load(self.open())
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
> 119, in open
> raise TMDBHTTPError(e)
> MythTV.tmdb3.tmdb_exceptions.TMDBHTTPError: HTTP Error 404: Not Found
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 159, in
> <module>
> sys.exit(main("television",'tmdb3tv.py'))
> File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 136, in main
> xml = buildEpisode(args[0:3], opts)
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/lookup.py", line
> 323, in buildEpisode
> episode = season.episodes[episode_number]mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>
>
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 162,
> in __get__
> self.poller.__get__(inst, owner)()
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 79,
> in __call__
> if not self.apply(req.readJSON(), False):
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/cache.py", line 117,
> in __call__
> data = self.func(*args, **kwargs)
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
> 152, in readJSON
> handle_status(data, url)
> File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
> 194, in handle_status
> raise status
> MythTV.tmdb3.tmdb_exceptions.TMDBRequestInvalid: The resource you
> requested could not be found.
>
> Easily avoided by numbering the episodes the same way as themoviedb.org
> has done (which in this case is different from the imdb.org numbering,
> in case you're interested), but still worth fixing?
>
> Thanks, Jan
> _______________________________________________

This looks like a bug in the recent tmdb3 changes to support TV shows. I
will take a look at it. Please tell me the misnumbered/misnamed file
name and directory of the video so that I can reproduce the problem and
fix it.

Peter

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 18/04/2021 15:21, Peter Bennett wrote:
> This looks like a bug in the recent tmdb3 changes to support TV shows. I
> will take a look at it. Please tell me the misnumbered/misnamed file
> name and directory of the video so that I can reproduce the problem and
> fix it.

Thank you Peter. I can reliably replicate the above traceback with:

/path/to/tmdb3tv.py -l en -a GB -D 873 11 1

(This is Columbo, which according to themoviedb.org doesn't have an 11th
season)

I'm running the following MythTV version, if it makes a difference:

MythTV Version : v31.0+fixes.202104012002.563a05b7a8~ubuntu18.04.1
MythTV Branch : fixes/31
Network Protocol : 91
Library API : 31.20200101-1
QT Version : 5.9.5
Options compiled in:
linux profile use_hidesyms using_alsa using_oss using_pulse
using_pulseoutput using_backend using_bindings_perl
using_bindings_python using_bindings_php using_dvb using_firewire
using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr
using_ivtv using_joystick_menu using_libcec using_libcrypto using_gnutls
using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg
using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus
using_taglib using_v4l2 using_v4l2prime using_x11
using_libbluray_external using_xrandr using_profiletype
using_systemd_notify using_systemd_journal using_drm using_bindings_perl
using_bindings_python using_bindings_php using_freetype2
using_mythtranscode using_opengl using_egl using_drm using_vaapi
using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass
using_libxml2 using_libmp3lame

Cheers, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 4/18/21 10:30 AM, Jan Ceuleers wrote:
> On 18/04/2021 15:21, Peter Bennett wrote:
>> This looks like a bug in the recent tmdb3 changes to support TV shows. I
>> will take a look at it. Please tell me the misnumbered/misnamed file
>> name and directory of the video so that I can reproduce the problem and
>> fix it.
> Thank you Peter. I can reliably replicate the above traceback with:
>
> /path/to/tmdb3tv.py -l en -a GB -D 873 11 1
>
> (This is Columbo, which according to themoviedb.org doesn't have an 11th
> season)
>
> I'm running the following MythTV version, if it makes a difference:
>
> MythTV Version : v31.0+fixes.202104012002.563a05b7a8~ubuntu18.04.1
> MythTV Branch : fixes/31
> Network Protocol : 91
> Library API : 31.20200101-1
> QT Version : 5.9.5
> Options compiled in:
> linux profile use_hidesyms using_alsa using_oss using_pulse
> using_pulseoutput using_backend using_bindings_perl
> using_bindings_python using_bindings_php using_dvb using_firewire
> using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr
> using_ivtv using_joystick_menu using_libcec using_libcrypto using_gnutls
> using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg
> using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus
> using_taglib using_v4l2 using_v4l2prime using_x11
> using_libbluray_external using_xrandr using_profiletype
> using_systemd_notify using_systemd_journal using_drm using_bindings_perl
> using_bindings_python using_bindings_php using_freetype2
> using_mythtranscode using_opengl using_egl using_drm using_vaapi
> using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass
> using_libxml2 using_libmp3lame
>
> Cheers, Jan
> _______________________________________________

Thanks for that. However I am more concerned about the endless loop you
said occurs with a misnumbered episode.

Peter


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 18/04/2021 17:49, Peter Bennett wrote:
> Thanks for that. However I am more concerned about the endless loop you
> said occurs with a misnumbered episode.

It [1] does that on one of my frontend machines (my laptop). If I hit
Ctrl-C I get the backtrace below. On the backend machine I immediately
get the backtrace I reported earlier.

[1]: /path/to/tmdb3tv.py -l en -a GB -D 873 11 1

$ /usr/share/mythtv/metadata/Television/tmdb3tv.py -l en -a GB -D 873 11 1
^CTraceback (most recent call last):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
117, in open
return _py23urlopen(self)
File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
133, in readJSON
data = json.load(self.open())
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
119, in open
raise TMDBHTTPError(e)
MythTV.tmdb3.tmdb_exceptions.TMDBHTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 159, in
<module>
sys.exit(main("television",'tmdb3tv.py'))
File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 136, in main
xml = buildEpisode(args[0:3], opts)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/lookup.py", line
323, in buildEpisode
episode = season.episodes[episode_number]
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 162,
in __get__
self.poller.__get__(inst, owner)()
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 79,
in __call__
if not self.apply(req.readJSON(), False):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/cache.py", line 117,
in __call__
data = self.func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
152, in readJSON
handle_status(data, url)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
194, in handle_status
raise status
MythTV.tmdb3.tmdb_exceptions.TMDBRequestInvalid: The resource you
requested could not be found.
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 140,
in apport_excepthook
crash_counter = get_recent_crashes(f) + 1
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 262,
in get_recent_crashes
pr.load(report, False, key_filter=['CrashCounter', 'Date'])
File "/usr/lib/python3/dist-packages/problem_report.py", line 168, in load
value += line[1:-1]
KeyboardInterrupt

Original exception was:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
117, in open
return _py23urlopen(self)
File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
133, in readJSON
data = json.load(self.open())
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
119, in open
raise TMDBHTTPError(e)
MythTV.tmdb3.tmdb_exceptions.TMDBHTTPError: HTTP Error 404: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 159, in
<module>
sys.exit(main("television",'tmdb3tv.py'))
File "/usr/share/mythtv/metadata/Television/tmdb3tv.py", line 136, in main
xml = buildEpisode(args[0:3], opts)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/lookup.py", line
323, in buildEpisode
episode = season.episodes[episode_number]
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 162,
in __get__
self.poller.__get__(inst, owner)()
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/util.py", line 79,
in __call__
if not self.apply(req.readJSON(), False):
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/cache.py", line 117,
in __call__
data = self.func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
152, in readJSON
handle_status(data, url)
File "/usr/lib/python3/dist-packages/MythTV/tmdb3/request.py", line
194, in handle_status
raise status
MythTV.tmdb3.tmdb_exceptions.TMDBRequestInvalid: The resource you
requested could not be found.

HTH, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 4/18/21 11:49 AM, Peter Bennett wrote:
>
> On 4/18/21 10:30 AM, Jan Ceuleers wrote:
>> On 18/04/2021 15:21, Peter Bennett wrote:
>>> This looks like a bug in the recent tmdb3 changes to support TV
>>> shows. I
>>> will take a look at it. Please tell me the misnumbered/misnamed file
>>> name and directory of the video so that I can reproduce the problem and
>>> fix it.
>> Thank you Peter. I can reliably replicate the above traceback with:
>>
>> /path/to/tmdb3tv.py -l en -a GB -D 873 11 1
>>
>> (This is Columbo, which according to themoviedb.org doesn't have an 11th
>> season)
>>
>> I'm running the following MythTV version, if it makes a difference:
>>
>> MythTV Version : v31.0+fixes.202104012002.563a05b7a8~ubuntu18.04.1
>> MythTV Branch : fixes/31
>> Network Protocol : 91
>> Library API : 31.20200101-1
>> QT Version : 5.9.5
>> Options compiled in:
>>   linux profile use_hidesyms using_alsa using_oss using_pulse
>> using_pulseoutput using_backend using_bindings_perl
>> using_bindings_python using_bindings_php using_dvb using_firewire
>> using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr
>> using_ivtv using_joystick_menu using_libcec using_libcrypto using_gnutls
>> using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg
>> using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus
>> using_taglib using_v4l2 using_v4l2prime using_x11
>> using_libbluray_external using_xrandr using_profiletype
>> using_systemd_notify using_systemd_journal using_drm using_bindings_perl
>> using_bindings_python using_bindings_php using_freetype2
>> using_mythtranscode using_opengl using_egl using_drm using_vaapi
>> using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass
>> using_libxml2 using_libmp3lame
>>
>> Cheers, Jan
>> _______________________________________________
>
> Thanks for that. However I am more concerned about the endless loop
> you said occurs with a misnumbered episode.
>
> Peter
>
>
I have committed a fix in master that prevents the exception. I don't
know if it will address the runaway cpu loop.

Peter

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 18/04/2021 21:45, Peter Bennett wrote:
> I have committed a fix in master that prevents the exception. I don't
> know if it will address the runaway cpu loop.

Peter, thank you for the quick action.

Do you have instructions for me to test this fix? I found the patch [1]
but I'm not clear on how to apply it to my system without upgrading to
master.


[1]:
https://github.com/MythTV/mythtv/commit/a064e11921d19fd3ee08a09553e48b80b5eba6cf

Thanks, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 19/04/2021 13:52, Jan Ceuleers wrote:
> On 18/04/2021 21:45, Peter Bennett wrote:
>> I have committed a fix in master that prevents the exception. I don't
>> know if it will address the runaway cpu loop.
>
> Peter, thank you for the quick action.
>
> Do you have instructions for me to test this fix? I found the patch [1]
> but I'm not clear on how to apply it to my system without upgrading to
> master.
>
>
> [1]:
> https://github.com/MythTV/mythtv/commit/a064e11921d19fd3ee08a09553e48b80b5eba6cf
>
> Thanks, Jan
>

Never mind: I found where the python bindings are on my machine, was
able to manually apply the patch and can confirm that it resolves the
problem (including the endless loop).

Thanks again, Jan
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 4/19/21 8:46 AM, Jan Ceuleers wrote:
> On 19/04/2021 13:52, Jan Ceuleers wrote:
>> On 18/04/2021 21:45, Peter Bennett wrote:
>>> I have committed a fix in master that prevents the exception. I don't
>>> know if it will address the runaway cpu loop.
>> Peter, thank you for the quick action.
>>
>> Do you have instructions for me to test this fix? I found the patch [1]
>> but I'm not clear on how to apply it to my system without upgrading to
>> master.
>>
>>
>> [1]:
>> https://github.com/MythTV/mythtv/commit/a064e11921d19fd3ee08a09553e48b80b5eba6cf
>>
>> Thanks, Jan
>>
> Never mind: I found where the python bindings are on my machine, was
> able to manually apply the patch and can confirm that it resolves the
> problem (including the endless loop).
>
> Thanks again, Jan
> _______________________________________________

Thanks for confirming. I have now cherry-picked the change into fixes/31.

Peter

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org