Mailing List Archive

season and episode population from schedules direct json has been incomplete.
As the subject indicates, I am seeing empty season episode information for many shows that should have it in mythtv.

I did get a sampling of data from SD that showed S/E info in their data, that wasn’t making it into Mythtv.

Any ideas!?

What kind of data would I need to help trouble shoot?
Thanks!
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 12, 2024, at 4:15?PM, Jay Harbeston <jharbestonus@gmail.com> wrote:
>
> As the subject indicates, I am seeing empty season episode information for many shows that should have it in mythtv.
>
> I did get a sampling of data from SD that showed S/E info in their data, that wasn’t making it into Mythtv.
>
> Any ideas!?
>
> What kind of data would I need to help trouble shoot?
> Thanks!


I forgot to mention version. I am running:

Please attach all output as a file in bug reports.
MythTV Version : v34.0~master.202312062250.f6c7551148~ubuntu22.04.1
MythTV Branch : master
Network Protocol : 91
Library API : 34.20220913-1
QT Version : 5.15.3
Options compiled in:
linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_firewire using_frontend using_hdhomerun using_satip using_vbox using_ceton using_joystick_menu using_libcec using_libcrypto using_gnutls using_libdns_sd using_libxml2 using_lirc using_mheg using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_v4l2prime using_x11 using_system_libbluray using_system_libudfread using_systemd_notify using_systemd_journal using_drm using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_egl using_drm using_vaapi using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2 using_libmp3lame


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 12, 2024, at 4:22?PM, Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>
>
>> On Jan 12, 2024, at 4:15?PM, Jay Harbeston <jharbestonus@gmail.com> wrote:
>>
>> As the subject indicates, I am seeing empty season episode information for many shows that should have it in mythtv.
>>
>> I did get a sampling of data from SD that showed S/E info in their data, that wasn’t making it into Mythtv.
>>
>> Any ideas!?
>>
>> What kind of data would I need to help trouble shoot?
>> Thanks!
>
>
> I forgot to mention version. I am running:
>
> Please attach all output as a file in bug reports.
> MythTV Version : v34.0~master.202312062250.f6c7551148~ubuntu22.04.1
> MythTV Branch : master
> Network Protocol : 91
> Library API : 34.20220913-1
> QT Version : 5.15.3
> Options compiled in:
> linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_firewire using_frontend using_hdhomerun using_satip using_vbox using_ceton using_joystick_menu using_libcec using_libcrypto using_gnutls using_libdns_sd using_libxml2 using_lirc using_mheg using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_v4l2prime using_x11 using_system_libbluray using_system_libudfread using_systemd_notify using_systemd_journal using_drm using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_egl using_drm using_vaapi using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2 using_libmp3lame
>
>

So I have captured the xml file generated during the mythfilldatabase run, and searching through the file I don’t find any references to <season> or <episode> tags in it at all.

I am using tv_grab_zz_sdjson as the grabber for schedules direct, and I am not seeing any errors during process.

I am using the xmltv package for ubuntu 22.04 v 1.1.1. with the tv_grab_zz_sdjson version 1.1.1 as well.

I do notice one thing in the xml file and that is that the program tags are programme instead of program.

Strange, Help!

Regards!



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On 13/01/2024 15:34, Jay Harbeston wrote:
>
>
>> On Jan 12, 2024, at 4:22?PM, Jay Harbeston <jharbestonus@gmail.com> wrote:
>>
>>
>>
>>> On Jan 12, 2024, at 4:15?PM, Jay Harbeston <jharbestonus@gmail.com> wrote:
>>>
>>> As the subject indicates, I am seeing empty season episode information for many shows that should have it in mythtv.
>>>
>>> I did get a sampling of data from SD that showed S/E info in their data, that wasn’t making it into Mythtv.
>>>
>>> Any ideas!?
>>>
>>> What kind of data would I need to help trouble shoot?
>>> Thanks!
>>
>>
>> I forgot to mention version. I am running:
>>
>> Please attach all output as a file in bug reports.
>> MythTV Version : v34.0~master.202312062250.f6c7551148~ubuntu22.04.1
>> MythTV Branch : master
>> Network Protocol : 91
>> Library API : 34.20220913-1
>> QT Version : 5.15.3
>> Options compiled in:
>> linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_firewire using_frontend using_hdhomerun using_satip using_vbox using_ceton using_joystick_menu using_libcec using_libcrypto using_gnutls using_libdns_sd using_libxml2 using_lirc using_mheg using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_v4l2prime using_x11 using_system_libbluray using_system_libudfread using_systemd_notify using_systemd_journal using_drm using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_egl using_drm using_vaapi using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2 using_libmp3lame
>>
>>
>
> So I have captured the xml file generated during the mythfilldatabase run, and searching through the file I don’t find any references to <season> or <episode> tags in it at all.
>
> I am using tv_grab_zz_sdjson as the grabber for schedules direct, and I am not seeing any errors during process.
>
> I am using the xmltv package for ubuntu 22.04 v 1.1.1. with the tv_grab_zz_sdjson version 1.1.1 as well.
>
> I do notice one thing in the xml file and that is that the program tags are programme instead of program.
>
> Strange, Help!
>
> Regards!
>
I can't help more but I noticed the same thing beginning recently... maybe at the start of the new
year? Series which I know have had series/episode info for years are suddenly blank. This also means
that the scheduler doesn't pick anything much up and I have to eyeball them all (using mythweb) to
make sure I'm not missing anything.

I'm running v33.1 from deb-multimedia so I doubt it's something mythtv related. Maybe a change
within xmltv coding?

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sat, Jan 13, 2024 at 3:36?PM Jay Harbeston <jharbestonus@gmail.com> wrote:

> So I have captured the xml file generated during the mythfilldatabase run, and searching through the file I don’t find any references to <season> or <episode> tags in it at all.

You would likely not, as there is no <season>
or <episode> tag defined in the xmltv dtd.
Do you think you were getting them previously?

The season/episode numbers are typically
found in the <episode-num system="xmltv_ns">
tag.

> I do notice one thing in the xml file and that is that the program tags are programme instead of program.

That would be the proper English spelling (as in
the King's English).

> Strange, Help!

First thing to do is check your mythfilldatabase
logs (with the --verbose general,xmltv option)
to see if there is anything of interest there.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sat, Jan 13, 2024 at 3:36?PM Jay Harbeston <jharbestonus@gmail.com>
wrote:
>> So I have captured the xml file generated during the mythfilldatabase run, and searching through the file I don’t find any references to <season> or <episode> tags in it at all.
>
What's the Episode ID?  If it doesn't begin with EP, then Gracenote
doesn't have the episode info yet.   It would also help to provide an
example of the missing show (EP* would be best)

RobertE
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 13, 2024, at 11:27?AM, Gary Buhrmaster <gary.buhrmaster@gmail.com> wrote:
>
> On Sat, Jan 13, 2024 at 3:36?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>
>> So I have captured the xml file generated during the mythfilldatabase run, and searching through the file I don’t find any references to <season> or <episode> tags in it at all.
>
> You would likely not, as there is no <season>
> or <episode> tag defined in the xmltv dtd.
> Do you think you were getting them previously?

I don’t know. Just an observation of the data. I thought I might get lucky and that would be the way it was identified. :-)

>
> The season/episode numbers are typically
> found in the <episode-num system="xmltv_ns">
> Tag.

There are several shows that do have the <episode-num system="xmltv_ns">2.0.0/2</episode-num> type lines

And there are also several missing for multiple shows and multiple channels.

Weird.

Is there a way to look into the cache file to see things there. I do see references to season and episode referenced in it for each of the shows/episodes, but is partially a binary file and can’t tell what I’m getting.

>> I do notice one thing in the xml file and that is that the program tags are programme instead of program.
>
> That would be the proper English spelling (as in
> the King's English).

Figured that. :-)
>
>> Strange, Help!
>
> First thing to do is check your mythfilldatabase
> logs (with the --verbose general,xmltv option)
> to see if there is anything of interest there.

So I have run mythfilldatabase as follows:

mythfilldatabase --verbose general,xmltv

And there are no errors indicating any issues. There are Note lines and Info lines. There is only one warning about UTC not defined, and guessing implicit UTC.

None of the I lines or N lines indicate anything amiss regarding the data.

——————————————————

Robert,

This is the Doc Martin episode Fly Me to The Moonthat will be broadcast on 39.1 WUNC on April 20th at 6pm.

<episode-num system="dd_progid">EP006870310129</episode-num>
It does not have the xmltv_ns line Gary is referring to above.

Thank you for the assistance!

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
This is the raw JSON for that programID; the metadata element is there and has data:

{
"programID": "EP006870310129",
"resourceID": "368906",
"titles": [
{
"title120": "Doc Martin",
"titleLanguage": "en-GB"
}
],
"descriptions": {
"description1000": [
{
"descriptionLanguage": "en",
"description": "Louisa is shocked when her father, Terry, makes an unannounced visit; it becomes clear Terry is unwell; he agrees to let Martin examine him, but that backfires on the doctor, who is now bound by the rules of medical confidentiality."
}
],
"description100": [
{
"descriptionLanguage": "en",
"description": "Ruth employs an unconventional method to test Martins blood phobia; Bert tries to vacate a squatter."
}
]
},
"originalAirDate": "2022-10-05",
"showType": "Series",
"entityType": "Episode",
"country": [
"GBR"
],
"genres": [
"Comedy drama",
"Medical",
"Romance"
],
"cast": [.
{
"billingOrder": "01",
"role": "Actor",
"name": "Martin Clunes",
"characterName": "Dr Martin Ellingham",
"nameId": "80277",
"personId": "80277"
},
{
"billingOrder": "02",
"role": "Actor",
"name": "Caroline Catz",
"characterName": "Louisa Ellingham",
"nameId": "242072",
"personId": "238543"
},
{
"billingOrder": "03",
"role": "Actor",
"name": "Ian McNeice",
"characterName": "Bert Large",
"nameId": "73546",
"personId": "73546"
},
{
"billingOrder": "04",
"role": "Actor",
"name": "Jessica Ransom",
"characterName": "Morwenna Newcross",
"nameId": "780011",
"personId": "745884"
},
{
"billingOrder": "05",
"role": "Actor",
"name": "Selina Cadell",
"characterName": "Mrs Tishell",
"nameId": "151667",
"personId": "151560"
},
{
"billingOrder": "06",
"role": "Actor",
"name": "Joe Absolom",
"characterName": "Al Large",
"nameId": "253910",
"personId": "250356"
},
{
"billingOrder": "07",
"role": "Actor",
"name": "John Marquez",
"characterName": "PC Penhale",
"nameId": "186855",
"personId": "185259"
},
{
"billingOrder": "08",
"role": "Actor",
"name": "Robyn Addison",
"characterName": "Janice",
"nameId": "864527",
"personId": "824355"
},
{
"billingOrder": "09",
"role": "Actor",
"name": "Angela Curran",
"characterName": "Caitlin"
},
{
"billingOrder": "10",
"role": "Actor",
"name": "Eileen Atkins",
"characterName": "Ruth Ellingham",
"nameId": "196764",
"personId": "194955"
},
{
"billingOrder": "11",
"role": "Actor",
"name": "Michael Webber",
"characterName": "Howard",
"nameId": "209153",
"personId": "207167"
},
{
"billingOrder": "12",
"role": "Actor",
"name": "Kenneth Cranham",
"characterName": "Terry Glasson",
"nameId": "36655",
"personId": "36655"
},
{
"billingOrder": "13",
"role": "Actor",
"name": "Tom Godwin",
"characterName": "Andrew"
},
{
"billingOrder": "14",
"role": "Actor",
"name": "Joshua McCord",
"characterName": "Ross"
},
{
"billingOrder": "15",
"role": "Actor",
"name": "Alice Walker",
"characterName": "Bevy - Pizza Delivery"
},
{
"billingOrder": "16",
"role": "Actor",
"name": "Kate Lassman Long",
"characterName": "Meg Morgan"
},
{
"billingOrder": "17",
"role": "Actor",
"name": "Sam Glen",
"characterName": "Matt Morgan"
},
{
"billingOrder": "18",
"role": "Actor",
"name": "Elliott Blake",
"characterName": "James Henry"
}
],
"crew": [.
{
"billingOrder": "01",
"role": "Writer",
"name": "Chris Reddy"
},
{
"billingOrder": "02",
"role": "Creator",
"name": "Dominic Minghella"
},
{
"billingOrder": "03",
"role": "Producer",
"name": "Philippa Braithwaite"
},
{
"billingOrder": "04",
"role": "Director",
"name": "Kate Cheeseman"
},
{
"billingOrder": "05",
"role": "Composer",
"name": "Colin Towns"
},
{
"billingOrder": "06",
"role": "Executive Producer",
"name": "Mark Crowdy"
}
],
"contentRating": [.
{
"body": "USA Parental Rating",
"code": "TVPG",
"country": "USA"
},
{
"body": "Canadian Parental Rating",
"code": "PG",
"country": "CAN"
},
{
"body": "Australian Classification Board",
"code": "M",
"country": "AUS"
},
{
"body": "Freiwillige Selbstkontrolle Fernsehen",
"code": "12",
"country": "DEU"
},
{
"body": "Departamento de Justi\\u00e7a, Classifica\\u00e7\\u00e3o, T\\u00edtulos e Qualifica\\u00e7\\u00e3o",
"code": "12",
"country": "BRA"
},
{
"body": "R\\u00e9gie du cin\\u00e9ma",
"code": "G",
"country": "CAN"
},
{
"body": "Krajowa Rada Radiofonii i Telewizji",
"code": "12",
"country": "POL"
},
{
"body": "\\ubc29\\uc1a1\\ud1b5\\uc2e0\\uc2ec\\uc758\\uc704\\uc6d0\\ud68c",
"code": "15",
"country": "KOR"
},
{
"body": "Medier\\u00e5det",
"code": "7",
"country": "DNK"
},
{
"body": "Kijkwijzer",
"code": "12",
"country": "NLD",
"contentWarning": [
"Coarse Language",
"Drug\\/Alcohol Use",
"Fear"
]
},
{
"body": "Mediakasvatus- ja kuvaohjelmayksikk\\u00f6",
"code": "K7",
"country": "FIN"
},
{
"body": "Freiwillige Selbstkontrolle der Filmwirtschaft",
"code": "12",
"country": "FIN"
}
],
"episodeTitle150": "Fly Me To The Moon",
"metadata": [.
{
"TVmaze": {
"season": 10,
"episode": 5,
"url": "https:\\/\\/www.tvmaze.com\\/episodes\\/2397455\\/doc-martin-10x05-fly-me-to-the-moon"
}
},
{
"Gracenote": {
"season": 10,
"episode": 5
}
}
],
"duration": 3900,
"hasImageArtwork": true,
"hasEpisodeArtwork": true,
"hasSeasonArtwork": true,
"hasSeriesArtwork": true,
"hash": "7c3d1e4c27a985d586e74b72a6fdd5b8",
"md5": "fD0eTCephdWG50typv3VuA"
}

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sat, Jan 13, 2024 at 5:24?PM Jay Harbeston <jharbestonus@gmail.com> wrote:

> There are several shows that do have the <episode-num system="xmltv_ns">2.0.0/2</episode-num> type lines
>
> And there are also several missing for multiple shows and multiple channels.


As I understand it (I don't use that grabber, for obvious
reasons), that particular grabber caches some information
(as do many/most others). It is possible that that cached
data is somehow stale, and not being updated properly.
That would presumably be a bug(*).

You could just delete all the cached data and let
that grabber download it all over again and see
if the output file has the episode numbers you are
looking for.

> Is there a way to look into the cache file to see things there.

I would guess the cache file is in perl Storable format
(based only on the "use Storable;" line in the code).
You can (presumably) write a perl program to
decode it.




(*) And if you can create a reproducer, open a bug
with that grabber (the usual problem is creating a
reliable reproducer as anecdotal reports are
rarely entirely helpful).
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.

Since my last email I have switched back to using the sqlite version and am getting good data now.

Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!

Thanks all for the assistance!


I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)

Regards!
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sun, Jan 14, 2024 at 7:22?PM Jay Harbeston <jharbestonus@gmail.com> wrote:

> Sounds Kinda like the non-sqlite version might be fading from use anyway

That is hard to quantify (and I am not sure one
should use responses from people on a extended
holiday weekend (for US users) as statistically
significant).

The XMLTV project (like most projects without
telemetry) has no idea IRT usage(*) in this,
or other projects, that use the XMLTV feeds.

There is a part of me that celebrates that there
are competing implementations of the upstream
Schedules Direct (Gracenote) data that individuals
can compare and contrast to determine the
best fit for their use cases (both implementations
are slightly different in certain types of uses even
as they start from the same raw upstream EPG
source). I also recognize that that can cause
confusion some of the time for some of the
people.

I expect that there are many satisfied users of
both implementations who may never come
forward (if it works, don't "fix" it), even if I am
admittedly biased towards the sqlite variant as
it is what I use.

My bottom line has always been (and what I
recommend to others, not just for XMLTV
feeds, but all software) is use what works for
you, and if it stops working, look elsewhere
(or better yet, contribute patches).




(*) The lack of instrumentation is a real issue
for (especially) open source projects trying to
decide where to prioritize and invest resources.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sun, Jan 14, 2024 at 2:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>
> Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.
>
> Since my last email I have switched back to using the sqlite version and am getting good data now.
>
> Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!
>
> Thanks all for the assistance!
>
>
> I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)
>
> Regards!

I've just recently started noticing the same missing seasons and
episodes. I've never used anything except for the non-sqlite version.

I suppose I may have to change to that sqlite one. On the whole,
Schedules Direct seems to do a good job, but on occasion they do some
really questionable stuff. There was that bizarre thing they did some
time back where they apparently replaced their backend stuff with some
new python version, which totally broke a lot of stuff. They ended up
having to roll that back, and I've never heard anything about it
since. It all makes me wonder what they might have done here.

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 17, 2024, at 4:38?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
>
> On Sun, Jan 14, 2024 at 2:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>>
>> Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.
>>
>> Since my last email I have switched back to using the sqlite version and am getting good data now.
>>
>> Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!
>>
>> Thanks all for the assistance!
>>
>>
>> I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)
>>
>> Regards!
>
> I've just recently started noticing the same missing seasons and
> episodes. I've never used anything except for the non-sqlite version.
>
> I suppose I may have to change to that sqlite one. On the whole,
> Schedules Direct seems to do a good job, but on occasion they do some
> really questionable stuff. There was that bizarre thing they did some
> time back where they apparently replaced their backend stuff with some
> new python version, which totally broke a lot of stuff. They ended up
> having to roll that back, and I've never heard anything about it
> since. It all makes me wonder what they might have done here.
>
> Tom

It wasn’t hard to switch, as I didn’t have to change any of the channels except for changing the number of the source id in channels to the new number for the new sqlite grabber.

Alternatively, you will need to do as Gary suggested and do an issue request to the xmltv group for the problem.


I took the path of least resistance and switched to the sqlite version. :-)


Regards!
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On 17/01/2024 21:55, Jay Harbeston wrote:
>
>
>> On Jan 17, 2024, at 4:38?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
>>
>> On Sun, Jan 14, 2024 at 2:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>>>
>>> Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.
>>>
>>> Since my last email I have switched back to using the sqlite version and am getting good data now.
>>>
>>> Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!
>>>
>>> Thanks all for the assistance!
>>>
>>>
>>> I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)
>>>
>>> Regards!
>>
>> I've just recently started noticing the same missing seasons and
>> episodes. I've never used anything except for the non-sqlite version.
>>
>> I suppose I may have to change to that sqlite one. On the whole,
>> Schedules Direct seems to do a good job, but on occasion they do some
>> really questionable stuff. There was that bizarre thing they did some
>> time back where they apparently replaced their backend stuff with some
>> new python version, which totally broke a lot of stuff. They ended up
>> having to roll that back, and I've never heard anything about it
>> since. It all makes me wonder what they might have done here.
>>
>> Tom
>
> It wasn’t hard to switch, as I didn’t have to change any of the channels except for changing the number of the source id in channels to the new number for the new sqlite grabber.
>
> Alternatively, you will need to do as Gary suggested and do an issue request to the xmltv group for the problem.
>
>
> I took the path of least resistance and switched to the sqlite version. :-)
>
Why did you have to change the source id in channels? Surely all that was needed was to go to the
source and change the grabber?

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Wed, Jan 17, 2024 at 6:03?PM Mike Perkins
<mikep@randomtraveller.org.uk> wrote:
>
> On 17/01/2024 21:55, Jay Harbeston wrote:
> >
> >
> >> On Jan 17, 2024, at 4:38?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
> >>
> >> On Sun, Jan 14, 2024 at 2:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
> >>>
> >>> Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.
> >>>
> >>> Since my last email I have switched back to using the sqlite version and am getting good data now.
> >>>
> >>> Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!
> >>>
> >>> Thanks all for the assistance!
> >>>
> >>>
> >>> I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)
> >>>
> >>> Regards!
> >>
> >> I've just recently started noticing the same missing seasons and
> >> episodes. I've never used anything except for the non-sqlite version.
> >>
> >> I suppose I may have to change to that sqlite one. On the whole,
> >> Schedules Direct seems to do a good job, but on occasion they do some
> >> really questionable stuff. There was that bizarre thing they did some
> >> time back where they apparently replaced their backend stuff with some
> >> new python version, which totally broke a lot of stuff. They ended up
> >> having to roll that back, and I've never heard anything about it
> >> since. It all makes me wonder what they might have done here.
> >>
> >> Tom
> >
> > It wasn’t hard to switch, as I didn’t have to change any of the channels except for changing the number of the source id in channels to the new number for the new sqlite grabber.
> >
> > Alternatively, you will need to do as Gary suggested and do an issue request to the xmltv group for the problem.
> >
> >
> > I took the path of least resistance and switched to the sqlite version. :-)
> >
> Why did you have to change the source id in channels? Surely all that was needed was to go to the
> source and change the grabber?
>
> --
>
> Mike Perkins
>
I think he's referring to the xmltvids on the channel table. The
non-sqlite grabber requires just the numeric id but the sqlite version
requires them to be in the form Innnnn.json.schedulesdirect.org.

Jay: Correct me if I'm wrong on that. I was actually going to reply to
verify that that needs to be changed.

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sat, Jan 13, 2024 at 11:25?AM Mike Perkins
<mikep@randomtraveller.org.uk> wrote:
>
> I can't help more but I noticed the same thing beginning recently... maybe at the start of the new
> year? Series which I know have had series/episode info for years are suddenly blank. This also means
> that the scheduler doesn't pick anything much up and I have to eyeball them all (using mythweb) to
> make sure I'm not missing anything.
>
> I'm running v33.1 from deb-multimedia so I doubt it's something mythtv related. Maybe a change
> within xmltv coding?
>
> --
>
> Mike Perkins
>
>
Just out of curiosity, are you also using the non-sqlite
(tv_grab_zz_sdjson) grabber?

Thanks!
Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Wed, Jan 17, 2024 at 4:38?PM Tom Dexter <digitalaudiorock@gmail.com> wrote:
>
> I've just recently started noticing the same missing seasons and
> episodes. I've never used anything except for the non-sqlite version.
>
> I suppose I may have to change to that sqlite one. On the whole,
> Schedules Direct seems to do a good job, but on occasion they do some
> really questionable stuff. There was that bizarre thing they did some
> time back where they apparently replaced their backend stuff with some
> new python version, which totally broke a lot of stuff. They ended up
> having to roll that back, and I've never heard anything about it
> since. It all makes me wonder what they might have done here.
>
> Tom

Wow. Can someone clarify where those seasons and episodes are even stored?

Tonight I have an episode of Family Law recording right now. MythTV
and MythWeb bother show season 3 and episode 1 and an "Episode Number"
of S3E1, but nothing I see in the database has that data. For example:

--------------
select * from recorded where title='family law' and
starttime='2024-01-18 02:00:00'
--------------

*************************** 1. row ***************************
chanid: 1111
starttime: 2024-01-18 02:00:00
endtime: 2024-01-18 03:00:00
title: Family Law
subtitle: What Came First?
description: Still dealing with the aftermath of her own divorce,
Abby represents a woman whose ex-husband left her for a man and now
wants to use their last remaining embryo -- created when they were
married -- to start a new family together.
season: 0
episode: 0
category: Drama
hostname: mythback
bookmark: 0
editing: 0
cutlist: 0
autoexpire: 1
commflagged: 0
recgroup: Default
recordid: 11633
seriesid: EP03905844
programid: EP039058440021
inetref:
lastmodified: 2024-01-17 21:30:01
filesize: 1309370180
stars: 0
previouslyshown: 1
originalairdate: 2022-12-23
preserve: 0
findid: 0
deletepending: 0
transcoder: 0
timestretch: 1
recpriority: 0
basename: 1111_20240118020000.ts
progstart: 2024-01-18 02:00:00
progend: 2024-01-18 03:00:00
playgroup: Default
profile: Default
duplicate: 0
transcoded: 0
watched: 0
storagegroup: Default
bookmarkupdate: 0000-00-00 00:00:00
recgroupid: 1
recordedid: 20276
inputname: HDHR 2-1 03

Can't make any sense out of that.

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Wed, Jan 17, 2024 at 9:37?PM Tom Dexter <digitalaudiorock@gmail.com> wrote:
>
> On Wed, Jan 17, 2024 at 4:38?PM Tom Dexter <digitalaudiorock@gmail.com> wrote:
> >
> > I've just recently started noticing the same missing seasons and
> > episodes. I've never used anything except for the non-sqlite version.
> >
> > I suppose I may have to change to that sqlite one. On the whole,
> > Schedules Direct seems to do a good job, but on occasion they do some
> > really questionable stuff. There was that bizarre thing they did some
> > time back where they apparently replaced their backend stuff with some
> > new python version, which totally broke a lot of stuff. They ended up
> > having to roll that back, and I've never heard anything about it
> > since. It all makes me wonder what they might have done here.
> >
> > Tom
>
> Wow. Can someone clarify where those seasons and episodes are even stored?
>
> Tonight I have an episode of Family Law recording right now. MythTV
> and MythWeb bother show season 3 and episode 1 and an "Episode Number"
> of S3E1, but nothing I see in the database has that data. For example:
>
> --------------
> select * from recorded where title='family law' and
> starttime='2024-01-18 02:00:00'
> --------------
>
> *************************** 1. row ***************************
> chanid: 1111
> starttime: 2024-01-18 02:00:00
> endtime: 2024-01-18 03:00:00
> title: Family Law
> subtitle: What Came First?
> description: Still dealing with the aftermath of her own divorce,
> Abby represents a woman whose ex-husband left her for a man and now
> wants to use their last remaining embryo -- created when they were
> married -- to start a new family together.
> season: 0
> episode: 0
> category: Drama
> hostname: mythback
> bookmark: 0
> editing: 0
> cutlist: 0
> autoexpire: 1
> commflagged: 0
> recgroup: Default
> recordid: 11633
> seriesid: EP03905844
> programid: EP039058440021
> inetref:
> lastmodified: 2024-01-17 21:30:01
> filesize: 1309370180
> stars: 0
> previouslyshown: 1
> originalairdate: 2022-12-23
> preserve: 0
> findid: 0
> deletepending: 0
> transcoder: 0
> timestretch: 1
> recpriority: 0
> basename: 1111_20240118020000.ts
> progstart: 2024-01-18 02:00:00
> progend: 2024-01-18 03:00:00
> playgroup: Default
> profile: Default
> duplicate: 0
> transcoded: 0
> watched: 0
> storagegroup: Default
> bookmarkupdate: 0000-00-00 00:00:00
> recgroupid: 1
> recordedid: 20276
> inputname: HDHR 2-1 03
>
> Can't make any sense out of that.
>
> Tom

OK...Found that. That's all in the program table. It's interesting
that I have this data specifically for a case with an older original
airdate. Here's that data:

--------------
select * from program where title='family law' and
starttime='2024-01-18 02:00:00'
--------------

*************************** 1. row ***************************
chanid: 1111
starttime: 2024-01-18 02:00:00
endtime: 2024-01-18 03:00:00
title: Family Law
subtitle: What Came First?
description: Still dealing with the aftermath of her own
divorce, Abby represents a woman whose ex-husband left her for a man
and now wants to use their last remaining embryo -- created when they
were married -- to start a new family together.
category: Drama
category_type: series
airdate: 0000
stars: 0
previouslyshown: 1
title_pronounce:
stereo: 0
subtitled: 1
hdtv: 1
closecaptioned: 0
partnumber: 0
parttotal: 0
seriesid: EP03905844
originalairdate: 2022-12-23
showtype:
colorcode:
syndicatedepisodenumber: S3E1
programid: EP039058440021
manualid: 0
generic: 0
listingsource: 4
first: 1
last: 0
audioprop: DOLBY
subtitletypes: NORMAL
videoprop: HDTV
inetref:
season: 3
episode: 1
totalepisodes: 0

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On 18/01/2024 01:40, Tom Dexter wrote:
> On Sat, Jan 13, 2024 at 11:25?AM Mike Perkins
> <mikep@randomtraveller.org.uk> wrote:
>>
>> I can't help more but I noticed the same thing beginning recently... maybe at the start of the new
>> year? Series which I know have had series/episode info for years are suddenly blank. This also means
>> that the scheduler doesn't pick anything much up and I have to eyeball them all (using mythweb) to
>> make sure I'm not missing anything.
>>
>> I'm running v33.1 from deb-multimedia so I doubt it's something mythtv related. Maybe a change
>> within xmltv coding?
>>
>> --
>>
>> Mike Perkins
>>
>>
> Just out of curiosity, are you also using the non-sqlite
> (tv_grab_zz_sdjson) grabber?
>
I am. I took one look at the sqlite version and backed away. There is sufficient work getting the
non-sqlite version configured and changing would essentially double that (the first time). My
circumstances are such that I am unable to spare the time or focus at the moment.

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 17, 2024, at 6:44?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
>
> On Wed, Jan 17, 2024 at 6:03?PM Mike Perkins
> <mikep@randomtraveller.org.uk <mailto:mikep@randomtraveller.org.uk>> wrote:
>>
>> On 17/01/2024 21:55, Jay Harbeston wrote:
>>>
>>>
>>>> On Jan 17, 2024, at 4:38?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
>>>>
>>>> On Sun, Jan 14, 2024 at 2:21?PM Jay Harbeston <jharbestonus@gmail.com> wrote:
>>>>>
>>>>> Up to 2020, I used the sqlite version, and switched to the non-sqlite version … I think it maybe have been related to when HDHR was doing a ‘cable’ subsrciption thing in 2018 or so and then switched away in 2020.
>>>>>
>>>>> Since my last email I have switched back to using the sqlite version and am getting good data now.
>>>>>
>>>>> Sounds Kinda like the non-sqlite version might be fading from use anyway, so now I am converted, and I have confirmed that the S/E data is populating correctly!
>>>>>
>>>>> Thanks all for the assistance!
>>>>>
>>>>>
>>>>> I did take a look at the perl program for non-sqlite.. and decided I would stay away from the dragons in it as I haven’t touched perl in at least 15 years! :-)
>>>>>
>>>>> Regards!
>>>>
>>>> I've just recently started noticing the same missing seasons and
>>>> episodes. I've never used anything except for the non-sqlite version.
>>>>
>>>> I suppose I may have to change to that sqlite one. On the whole,
>>>> Schedules Direct seems to do a good job, but on occasion they do some
>>>> really questionable stuff. There was that bizarre thing they did some
>>>> time back where they apparently replaced their backend stuff with some
>>>> new python version, which totally broke a lot of stuff. They ended up
>>>> having to roll that back, and I've never heard anything about it
>>>> since. It all makes me wonder what they might have done here.
>>>>
>>>> Tom
>>>
>>> It wasn’t hard to switch, as I didn’t have to change any of the channels except for changing the number of the source id in channels to the new number for the new sqlite grabber.
>>>
>>> Alternatively, you will need to do as Gary suggested and do an issue request to the xmltv group for the problem.
>>>
>>>
>>> I took the path of least resistance and switched to the sqlite version. :-)
>>>
>> Why did you have to change the source id in channels? Surely all that was needed was to go to the
>> source and change the grabber?
>>
>> --
>>
>> Mike Perkins
>>
> I think he's referring to the xmltvids on the channel table. The
> non-sqlite grabber requires just the numeric id but the sqlite version
> requires them to be in the form Innnnn.json.schedulesdirect.org <http://innnnn.json.schedulesdirect.org/>.
>
> Jay: Correct me if I'm wrong on that. I was actually going to reply to
> verify that that needs to be changed.
>
> Tom


Actually the sourced field in the channel table points to the video source table with information on the data source to use(such as schedules direct). When I moved to the sqlite version, a new video source is added, and hence the source id had to change in the channel table. In my case, the old nonsqlite version had sourceid of 4 and after the sqlite version was added, the source id for it was now 6. So updated the channel table to reflect the change.
NOTE that this change could be made in the backend setup(either the original mythtv-setup, or the new web version).
For almost 40 channels, this is a pain to do, so I went ‘under the hood/ and modded the channel table using sql.

The xmltv ids did not have to be modified as they are the same for both non sqlite and sqlite versions as far as I can tell, and everything was working fine after the change.

Regards!
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Thu, Jan 18, 2024 at 4:38?AM Mike Perkins
<mikep@randomtraveller.org.uk> wrote:
>
> On 18/01/2024 01:40, Tom Dexter wrote:
> > On Sat, Jan 13, 2024 at 11:25?AM Mike Perkins
> > <mikep@randomtraveller.org.uk> wrote:
> >>
> >> I can't help more but I noticed the same thing beginning recently... maybe at the start of the new
> >> year? Series which I know have had series/episode info for years are suddenly blank. This also means
> >> that the scheduler doesn't pick anything much up and I have to eyeball them all (using mythweb) to
> >> make sure I'm not missing anything.
> >>
> >> I'm running v33.1 from deb-multimedia so I doubt it's something mythtv related. Maybe a change
> >> within xmltv coding?
> >>
> >> --
> >>
> >> Mike Perkins
> >>
> >>
> > Just out of curiosity, are you also using the non-sqlite
> > (tv_grab_zz_sdjson) grabber?
> >
> I am. I took one look at the sqlite version and backed away. There is sufficient work getting the
> non-sqlite version configured and changing would essentially double that (the first time). My
> circumstances are such that I am unable to spare the time or focus at the moment.
>
> --
>
> Mike Perkins
>
Yea, it's a little daunting for sure. I had already done the research
on that when Schedules Direct temporarily went to that new python
backend, and everyone was suggesting switching to it then. However
they abandoned that apparently so I never had to.

I actually just converted to the sqlite version now and re-ran
mythfilldatabase. Some of the existing things that had already
recorded don't show the season or episode but I can clearly see that
all the upcoming recordings do, and the number or records in the
program table with that data increased by about 10,000 or so. I
actually never mentioned that, for reasons I won't get into, I'm
actually still running MythTV 29.1 (under gentoo).

The ugly part of the sqlite version is enabling / disabling channels
in that sqlite database. Using my old xmltv config with the
"channel=NNNNN" lines I was able to generate a SQL script that I ran
against the sqlite database (initialized with all the channels as per
the wiki). That looks like this:

UPDATE channels SET selected=0;
UPDATE channels SET selected=1 WHERE lineup='USA-OTA-08880' AND station=16689;
UPDATE channels SET selected=1 WHERE lineup='USA-OTA-08880' AND station=74916;
UPDATE channels SET selected=1 WHERE lineup='USA-OTA-08880' AND station=112501;
UPDATE channels SET selected=1 WHERE lineup='USA-OTA-08880' AND station=19611;
...etc...

After backing up the database (as well as a separate backup of just
the channel table), I converted the xmltvid fields on the channels
from the 12345 format to the I12345.json.schedulesdirect.org format
using this SQL against the mythconverg database:

update channel set
xmltvid=concat('I',xmltvid,'.json.schedulesdirect.org') where xmltvid
regexp '^[0-9]+$';

Everything actually went pretty smoothly.

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
> On Jan 17, 2024, at 4:38?PM, Tom Dexter <digitalaudiorock@gmail.com> wrote:
>
> I think he's referring to the xmltvids on the channel table. The
> non-sqlite grabber requires just the numeric id but the sqlite version
> requires them to be in the form Innnnn.json.schedulesdirect.org.
>
> Jay: Correct me if I'm wrong on that. I was actually going to reply to
> verify that that needs to be changed.
>
> Tom
>
>
>
> Actually the sourced field in the channel table points to the video source table with information on the data source to use(such as schedules direct). When I moved to the sqlite version, a new video source is added, and hence the source id had to change in the channel table. In my case, the old nonsqlite version had sourceid of 4 and after the sqlite version was added, the source id for it was now 6. So updated the channel table to reflect the change.
> NOTE that this change could be made in the backend setup(either the original mythtv-setup, or the new web version).
> For almost 40 channels, this is a pain to do, so I went ‘under the hood/ and modded the channel table using sql.
>
> The xmltv ids did not have to be modified as they are the same for both non sqlite and sqlite versions as far as I can tell, and everything was working fine after the change.
>
> Regards!
>

Wow. I'm not clear as to what you did there, but I just went into
mythtv-setup and changed the grabber on the existing source, which is
still the only one I have. As noted in another reply, I used a SQL
command to change the xmltvid fields on the channel table to the
format expected by the sqlite grabber and everything worked fine.

Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sun, Jan 14, 2024 at 5:32?PM Gary Buhrmaster
<gary.buhrmaster@gmail.com> wrote:

> (*) And if you can create a reproducer, open a bug
> with that grabber (the usual problem is creating a
> reliable reproducer as anecdotal reports are
> rarely entirely helpful).

It looks like someone (perhaps someone from
this thread?) opened a bug report with what
looks like a likely cause against that grabber.

I presume the author of that grabber will
review the bug and work up a fix at some
point.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Wed, Jan 17, 2024 at 9:40?PM Tom Dexter <digitalaudiorock@gmail.com> wrote:

> I suppose I may have to change to that sqlite one. On the whole,
> Schedules Direct seems to do a good job, but on occasion they do some
> really questionable stuff. There was that bizarre thing they did some
> time back where they apparently replaced their backend stuff with some
> new python version, which totally broke a lot of stuff.

If the case you referring to is the one I am thinking of
it was moving to a python based microservices
architecture from a monolithic php implementation.

The goal was to be both API compliant, and
to (substantially) reduce technical debt and
cloud services costs.

There were some early identified issues that were
fixed quickly on the server side (some during
early private testing, although some additional
ones were immediately seen when "the public"
first experienced the new implementation).

Reportedly most specification compliant applications
(and there are a number of applications of the
EPG, most of which are not XMLTV grabbers)
worked. The non-sqlite version of the XMLTV
grabber was not fully HTTP compliant (although
that was fixed upstream), but there was also some
failure when pulling the data that was not even
understood before SD reverted (this was
compounded by an issue that the author of
the non-sqlite grabber was hard to contact,
and the test instance was shutdown as part of the
reversion so as far as I know there was never a
clear understanding of what was going on (either
in the grabber or the server side)).

> They ended up
> having to roll that back, and I've never heard anything about it
> since. It all makes me wonder what they might have done here.

As I understand it, after some initial teething
issues were addressed, the lambda implementation
both ran into the classic thundering herd issue,
and with a database performance issue that made
the thundering herd issue worse. Rather than
continue to work the issue (there were probably
various solutions, but they would have likely
come at some additional cost and complexity) a
decision was made to revert. Which, in a way,
is too bad, as it was expected to save SD money
in AWS charges (and if you know AWS, one
thing they are very good at is taking your money).

It is also my understanding that API-NEXT for
the Schedules Direct sources (which at this
point has just a vague "sometime in the future"
target date) will be using the new architecture
(although as the old applications will be using
the older API for quite some time the migration
will be over a longer time period and SD will
be able to see how the herds need to be
directed)
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: season and episode population from schedules direct json has been incomplete. [ In reply to ]
On Sat, Jan 20, 2024 at 2:08?PM Gary Buhrmaster
<gary.buhrmaster@gmail.com> wrote:
>
> On Sun, Jan 14, 2024 at 5:32?PM Gary Buhrmaster
> <gary.buhrmaster@gmail.com> wrote:
>
> > (*) And if you can create a reproducer, open a bug
> > with that grabber (the usual problem is creating a
> > reliable reproducer as anecdotal reports are
> > rarely entirely helpful).
>
> It looks like someone (perhaps someone from
> this thread?) opened a bug report with what
> looks like a likely cause against that grabber.
>
> I presume the author of that grabber will
> review the bug and work up a fix at some
> point.

Interesting. That definitely sounds like it explains what's happening.
I wonder what exactly changed given that it was working.

Now that I've switched to the sqlite version I'm fine with that. It
seems to be working great in fact. I'm pretty comfortable with that,
especially now that I know how to do things like manually enabling or
disabling channels in its sqlite database without using the rather
tedious method that the grabber itself has.

Thanks!
Tom
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

1 2  View All