Mailing List Archive

EPG script fails to update mythfilldatabase
Hi all

As I mentioned in my last post,the EPG update script I had been using for
several years failed from a week ago.
I have resolved the problem now.This is what happened.

The script worked if I executed it manually but not if run by cron.

The script is:

#! /bin/bash
wget http://epg.org.nz/freeview.xml.gz
gzip -d freeview.xml.gz
mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
rm freeview.xml
exit


I arranged to have the output mailed to a file and saw this message:

...........................................................................

saving to: `freeview.xml.gz.10'

0K .......... .......... .......... .......... .......... 29% 166K 1s
50K .......... .......... .......... .......... .......... 59% 170K 0s
100K .......... .......... .......... .......... .......... 89% 166K 0s
150K .......... ....... 100%
173K=1.0s

2016-06-06 10:32:03 (168 KB/s) - `freeview.xml.gz.10' saved [171009/171009]


gzip: freeview.xml.gz: not in gzip format

....................................................................................


I noted the ".10" added to the gzip archive and found 10 freeview.xml.gz
archives in the root directory.I deleted these and now it works fine when
run by cron.
It appears for some reason a week ago the freeview.xml.gz file wasn't
removed automatically when the extraction was done.

Cheers

-Paul

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: EPG script fails to update mythfilldatabase [ In reply to ]
On Mon, Jun 6, 2016 at 11:00 AM, Paulgir <paulgir@gmail.com> wrote:
>
>
> #! /bin/bash
> wget http://epg.org.nz/freeview.xml.gz
> gzip -d freeview.xml.gz
> mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
> rm freeview.xml
> exit
>

Hi Paul,

The coding pattern in that script is problematic, as if you ever end
up with an invalid freeview.xml.gz, the script will never attempt to
clean it up.

E.g. if 'wget' fails, then 'gzip -d' fails, then the 'rm' at the end
will be trying to remove a file that doesn't exist.

Here's some suggestions:
1. Use 'wget' with the '-O - ' option, and pipe the output straight
through 'gzip -d' to generate the xml file on one line
2. Use 'rm -f' so the script won't throw errors if the file doesn't exist
3. Use '#! /bin/bash -e', so that the script will exit at the first
failed command, rather than blindly carrying on
3. Finally, you could use a 'trap' function, which will ensure the
clean up code is run at script exit, regardless of failures


Here's a quick example:


#! /bin/bash -e

function clean_up {
rm -f freeview.xml
}

trap clean_up EXIT

wget -q http://epg.org.nz/freeview.xml.gz -O - | gzip -d > freeview.xml
mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml



Cheers,
Jonathan

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: EPG script fails to update mythfilldatabase [ In reply to ]
On Mon, 06 Jun 2016 13:09:20 +1200, Jonathan Hoskin
<jonathan.hoskin@gmail.com> wrote:

> On Mon, Jun 6, 2016 at 11:00 AM, Paulgir <paulgir@gmail.com> wrote:
>>
>>
>> #! /bin/bash
>> wget http://epg.org.nz/freeview.xml.gz
>> gzip -d freeview.xml.gz
>> mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
>> rm freeview.xml
>> exit
>>
>
> Hi Paul,
>
> The coding pattern in that script is problematic, as if you ever end
> up with an invalid freeview.xml.gz, the script will never attempt to
> clean it up.
>
> E.g. if 'wget' fails, then 'gzip -d' fails, then the 'rm' at the end
> will be trying to remove a file that doesn't exist.
>
> Here's some suggestions:
> 1. Use 'wget' with the '-O - ' option, and pipe the output straight
> through 'gzip -d' to generate the xml file on one line
> 2. Use 'rm -f' so the script won't throw errors if the file doesn't exist
> 3. Use '#! /bin/bash -e', so that the script will exit at the first
> failed command, rather than blindly carrying on
> 3. Finally, you could use a 'trap' function, which will ensure the
> clean up code is run at script exit, regardless of failures
>
>
> Here's a quick example:
>
>
> #! /bin/bash -e
>
> function clean_up {
> rm -f freeview.xml
> }
>
> trap clean_up EXIT
>
> wget -q http://epg.org.nz/freeview.xml.gz -O - | gzip -d > freeview.xml
> mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
>
>
>
> Cheers,
> Jonathan
>
> _______________________________________________

Thanks for the tip.I'll try this.
There's no need for an exit line at the end of this?

-Paul

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: EPG script fails to update mythfilldatabase [ In reply to ]
On Mon, 06 Jun 2016 13:09:20 +1200, Jonathan Hoskin
<jonathan.hoskin@gmail.com> wrote:

> On Mon, Jun 6, 2016 at 11:00 AM, Paulgir <paulgir@gmail.com> wrote:
>>
>>
>> #! /bin/bash
>> wget http://epg.org.nz/freeview.xml.gz
>> gzip -d freeview.xml.gz
>> mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
>> rm freeview.xml
>> exit
>>
>
> Hi Paul,
>
> The coding pattern in that script is problematic, as if you ever end
> up with an invalid freeview.xml.gz, the script will never attempt to
> clean it up.
>
> E.g. if 'wget' fails, then 'gzip -d' fails, then the 'rm' at the end
> will be trying to remove a file that doesn't exist.
>
> Here's some suggestions:
> 1. Use 'wget' with the '-O - ' option, and pipe the output straight
> through 'gzip -d' to generate the xml file on one line
> 2. Use 'rm -f' so the script won't throw errors if the file doesn't exist
> 3. Use '#! /bin/bash -e', so that the script will exit at the first
> failed command, rather than blindly carrying on
> 3. Finally, you could use a 'trap' function, which will ensure the
> clean up code is run at script exit, regardless of failures
>
>
> Here's a quick example:
>
>
> #! /bin/bash -e
>
> function clean_up {
> rm -f freeview.xml
> }
>
> trap clean_up EXIT
>
> wget -q http://epg.org.nz/freeview.xml.gz -O - | gzip -d > freeview.xml
> mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
>
>
>
> Cheers,
> Jonathan
>
> _______________________________________________

That script works well.
Also, I noticed the following message when the scripts run:

-- update deprecated,use --only-update-guide

so I changed the command to suit.

-Paul

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
Re: EPG script fails to update mythfilldatabase [ In reply to ]
On Tue, Jun 7, 2016 at 7:28 AM, Paulgir <paulgir@gmail.com> wrote:
> There's no need for an exit line at the end of this?

Correct, the 'exit' is redundant, as that's what the script will do
when it hits the bottom anyway.

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/