Mailing List Archive

schedule howto
G’day all
playing with the rules I can solve once, but everytime is a pain …

Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
Bprogram scheduled 1930 to 2230

Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
As suggested I put “prefer HD”

Now Aprogram records on tuner one, two, three, four
One recording of News is [red] conflict
One recording of Bprogram is [red] conflict

Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.

James

PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram




_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:

>G?day all
>playing with the rules I can solve once, but everytime is a pain ?
>
>Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>Bprogram scheduled 1930 to 2230
>
>Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I?ve set <no> virtual channels.
>As suggested I put ?prefer HD?
>
>Now Aprogram records on tuner one, two, three, four
>One recording of News is [red] conflict
>One recording of Bprogram is [red] conflict
>
>Wunce-upon-a-time I recall a rule ?record 1 showing only?. That with ?Prefer HD? would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>
>James
>
>PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram

How sure are you that the pixelation is caused by virtual tuners? Most
people do not have that problem, and there are plenty of users of
Hauppauge USB tuners. The way that virtual tuners work is just to
allow more than one set of streams to be recorded at the same time
from the same tuner. So the tuner hardware gets told to send streams
a,b, and c to the PC for one channel and streams f, g, and h for
another channel. It sends all those streams at once to the CPU. The
CPU sorts out which streams are for which channel and writes them to
separate recording files. The only problem you could get from getting
the tuner to send more streams is if the total bit rate exceeded the
maximum for USB 2.0, which is not possible for a DVB-T transmitter,
especially an Australian one that operates on 7 MHz bandwidth where
the rest of the world mostly uses 8 MHz.

When you set virtual tuners to 0, all you are doing is telling
mythbackend to record from effectively one virtual tuner, in other
words, record all the streams for one channel only from the tuner. But
the same code is used as when multiple virtual tuners is specified.

So your problem is more likely to be at the other end of things - if
you write too many recordings at once to a hard drive, you will
eventually get to the point where the heads will not be able to move
between the file locations fast enough to keep up, and mythbackend's
recording buffers will overflow before all the data can be written to
disk. Which causes gaps in the data, which you see as pixellation and
missing bits, and it also causes glitches in the sound. If this is
happening, you should be getting bad recordings for all the recordings
going to the overloaded hard disk during the time it was overloaded. I
have been there and done that.

So how many recordings are you writing to disk when you are getting
pixilated recordings? How many disks are they going to? How fast are
the disks?

If this is what your problem is, then the solution is normally just to
had another hard drive to the PC. I have 7 recording drives in order
to prevent this problem. To be completely sure not to get overloaded
hard drives, I work on the basis that there should be only 2
recordings at once going to one hard drive. That then allows for
playback of another recording from the same drive while those two
recordings are in progress. In theory, the faster more recent hard
drives should be able to cope with 3 recordings at once plus playback,
or maybe more, but I have never tested that. I still have 2 old hard
drives that probably can not do that, but I keep them as they are very
quiet compared to more modern ones, so I can use them for the
relatively few overnight recordings and not have the hard drive noises
keep me awake.

Along with recordings and playback, there can be other hard disk
activity for commercial skip processing. Modern CPUs can normally
cope with doing commercial skip processing in real-time from the
in-memory recording buffers on the basis of one core per recording.
Such processing does not cause any extra hard disk activity. If you
are making more recordings at once than you have CPU cores, you
normally set the number of commercial skip processes to the number of
cores, and the extra recordings get scheduled for later processing. In
which case, the commercial skip processes will be reading back those
recordings from hard disk at a speed significantly greater than for
playback or recording. And that can interfere with any recordings
going on at the time as the heads will be moving back and forth quite
a bit. I do not have this problem as I do not do any commercial skip
processing - it does not work usefully on any New Zealand channels
that I know of.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
On 08/12/2023 15:56, Stephen Worthington wrote:
> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>
>> G’day all
>> playing with the rules I can solve once, but everytime is a pain …
>>
>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>> Bprogram scheduled 1930 to 2230
>>
>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>> As suggested I put “prefer HD”
>>
>> Now Aprogram records on tuner one, two, three, four
>> One recording of News is [red] conflict
>> One recording of Bprogram is [red] conflict
>>
>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>
>> James
>>
>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>
> How sure are you that the pixelation is caused by virtual tuners? Most
> people do not have that problem, and there are plenty of users of
> Hauppauge USB tuners. The way that virtual tuners work is just to
> allow more than one set of streams to be recorded at the same time
> from the same tuner. So the tuner hardware gets told to send streams
> a,b, and c to the PC for one channel and streams f, g, and h for
> another channel. It sends all those streams at once to the CPU. The
> CPU sorts out which streams are for which channel and writes them to
> separate recording files. The only problem you could get from getting
> the tuner to send more streams is if the total bit rate exceeded the
> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
> especially an Australian one that operates on 7 MHz bandwidth where
> the rest of the world mostly uses 8 MHz.
>
> When you set virtual tuners to 0, all you are doing is telling
> mythbackend to record from effectively one virtual tuner, in other
> words, record all the streams for one channel only from the tuner. But
> the same code is used as when multiple virtual tuners is specified.
>
> So your problem is more likely to be at the other end of things - if
> you write too many recordings at once to a hard drive, you will
> eventually get to the point where the heads will not be able to move
> between the file locations fast enough to keep up, and mythbackend's
> recording buffers will overflow before all the data can be written to
> disk. Which causes gaps in the data, which you see as pixellation and
> missing bits, and it also causes glitches in the sound. If this is
> happening, you should be getting bad recordings for all the recordings
> going to the overloaded hard disk during the time it was overloaded. I
> have been there and done that.
>
> So how many recordings are you writing to disk when you are getting
> pixilated recordings? How many disks are they going to? How fast are
> the disks?
>
> If this is what your problem is, then the solution is normally just to
> had another hard drive to the PC. I have 7 recording drives in order
> to prevent this problem. To be completely sure not to get overloaded
> hard drives, I work on the basis that there should be only 2
> recordings at once going to one hard drive. That then allows for
> playback of another recording from the same drive while those two
> recordings are in progress. In theory, the faster more recent hard
> drives should be able to cope with 3 recordings at once plus playback,
> or maybe more, but I have never tested that. I still have 2 old hard
> drives that probably can not do that, but I keep them as they are very
> quiet compared to more modern ones, so I can use them for the
> relatively few overnight recordings and not have the hard drive noises
> keep me awake.
>
> Along with recordings and playback, there can be other hard disk
> activity for commercial skip processing. Modern CPUs can normally
> cope with doing commercial skip processing in real-time from the
> in-memory recording buffers on the basis of one core per recording.
> Such processing does not cause any extra hard disk activity. If you
> are making more recordings at once than you have CPU cores, you
> normally set the number of commercial skip processes to the number of
> cores, and the extra recordings get scheduled for later processing. In
> which case, the commercial skip processes will be reading back those
> recordings from hard disk at a speed significantly greater than for
> playback or recording. And that can interfere with any recordings
> going on at the time as the heads will be moving back and forth quite
> a bit. I do not have this problem as I do not do any commercial skip
> processing - it does not work usefully on any New Zealand channels
> that I know of.
>
There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that
whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.

However, virtual /channels/ could be a different problem, depending on specific language. In the US
I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram
more subchannels in. I don't know if the same applies in AU, but that could be the problem.

Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out.
Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That
might go a long way to solving the problem.

--

Mike Perkins


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
> On Dec 9, 2023, at 00:27, Mike Perkins <mikep@randomtraveller.org.uk> wrote:
>
> On 08/12/2023 15:56, Stephen Worthington wrote:
>> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>>> G’day all
>>> playing with the rules I can solve once, but everytime is a pain …
>>>
>>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>>> Bprogram scheduled 1930 to 2230
>>>
>>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>>> As suggested I put “prefer HD”
>>>
>>> Now Aprogram records on tuner one, two, three, four
>>> One recording of News is [red] conflict
>>> One recording of Bprogram is [red] conflict
>>>
>>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>>
>>> James
>>>
>>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>> How sure are you that the pixelation is caused by virtual tuners? Most
>> people do not have that problem, and there are plenty of users of
>> Hauppauge USB tuners. The way that virtual tuners work is just to
>> allow more than one set of streams to be recorded at the same time
>> from the same tuner. So the tuner hardware gets told to send streams
>> a,b, and c to the PC for one channel and streams f, g, and h for
>> another channel. It sends all those streams at once to the CPU. The
>> CPU sorts out which streams are for which channel and writes them to
>> separate recording files. The only problem you could get from getting
>> the tuner to send more streams is if the total bit rate exceeded the
>> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
>> especially an Australian one that operates on 7 MHz bandwidth where
>> the rest of the world mostly uses 8 MHz.
>> When you set virtual tuners to 0, all you are doing is telling
>> mythbackend to record from effectively one virtual tuner, in other
>> words, record all the streams for one channel only from the tuner. But
>> the same code is used as when multiple virtual tuners is specified.
>> So your problem is more likely to be at the other end of things - if
>> you write too many recordings at once to a hard drive, you will
>> eventually get to the point where the heads will not be able to move
>> between the file locations fast enough to keep up, and mythbackend's
>> recording buffers will overflow before all the data can be written to
>> disk. Which causes gaps in the data, which you see as pixellation and
>> missing bits, and it also causes glitches in the sound. If this is
>> happening, you should be getting bad recordings for all the recordings
>> going to the overloaded hard disk during the time it was overloaded. I
>> have been there and done that.
>> So how many recordings are you writing to disk when you are getting
>> pixilated recordings? How many disks are they going to? How fast are
>> the disks?
>> If this is what your problem is, then the solution is normally just to
>> had another hard drive to the PC. I have 7 recording drives in order
>> to prevent this problem. To be completely sure not to get overloaded
>> hard drives, I work on the basis that there should be only 2
>> recordings at once going to one hard drive. That then allows for
>> playback of another recording from the same drive while those two
>> recordings are in progress. In theory, the faster more recent hard
>> drives should be able to cope with 3 recordings at once plus playback,
>> or maybe more, but I have never tested that. I still have 2 old hard
>> drives that probably can not do that, but I keep them as they are very
>> quiet compared to more modern ones, so I can use them for the
>> relatively few overnight recordings and not have the hard drive noises
>> keep me awake.
>> Along with recordings and playback, there can be other hard disk
>> activity for commercial skip processing. Modern CPUs can normally
>> cope with doing commercial skip processing in real-time from the
>> in-memory recording buffers on the basis of one core per recording.
>> Such processing does not cause any extra hard disk activity. If you
>> are making more recordings at once than you have CPU cores, you
>> normally set the number of commercial skip processes to the number of
>> cores, and the extra recordings get scheduled for later processing. In
>> which case, the commercial skip processes will be reading back those
>> recordings from hard disk at a speed significantly greater than for
>> playback or recording. And that can interfere with any recordings
>> going on at the time as the heads will be moving back and forth quite
>> a bit. I do not have this problem as I do not do any commercial skip
>> processing - it does not work usefully on any New Zealand channels
>> that I know of.
> There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.
>
> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>
> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.
>

How I wish sane logic would prevail.

Here be what happens:

With Virtual Tuners about 1 recording a week is ‘damaged’ with bad pixelation
With my bizare setting I’ve seen NO damaged recordings in 5 months.
All the storage is SSD
OS and all the associated stuff is on a samsung 980 M2 (2T)
All myth including Videos is on a samsung QVO SATA (4T)
CPU is Ryzen 7 5700U (8 core) with 24G ram (I use this box as as workstation during the day)

The Oz TV (successfully !) channels defeat commercial skipping by changing the algorithm even for a single show so I disable all commercial skipping.

Please wax lyrical on virtual channels. How could I tell a virtual channel from a mux?

Definitely there is some fiddling:
The usual bitrate on 1080 is cica 1000 Kb/s
Last year ABC had a live documentary of coral spawning where they raised the bitrate to over 2mb/s
I did not check what they did on other channels to compensate

[diversion] coral spawning is special: on 1 moonlit night per year at the right time ALL the coral spawn. On the barrier reef that means simultainous spawning over 1000s km.

There is no logic on damaged channels. It could be the sole recording being made.
I have tested 8 simultainous recordings all without any problem.

I use an external antenna some 10 km line-of-sight to the transmitter farm. We have various ‘filler’ (low power) transmitters round Perth. I carefully pruned them from my list.

James


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
On Sat, 9 Dec 2023 05:18:34 +0800, you wrote:

>
>
>> On Dec 9, 2023, at 00:27, Mike Perkins <mikep@randomtraveller.org.uk> wrote:
>>
>> On 08/12/2023 15:56, Stephen Worthington wrote:
>>> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>>>> G?day all
>>>> playing with the rules I can solve once, but everytime is a pain ?
>>>>
>>>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>>>> Bprogram scheduled 1930 to 2230
>>>>
>>>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>>>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I?ve set <no> virtual channels.
>>>> As suggested I put ?prefer HD?
>>>>
>>>> Now Aprogram records on tuner one, two, three, four
>>>> One recording of News is [red] conflict
>>>> One recording of Bprogram is [red] conflict
>>>>
>>>> Wunce-upon-a-time I recall a rule ?record 1 showing only?. That with ?Prefer HD? would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>>>
>>>> James
>>>>
>>>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>>> How sure are you that the pixelation is caused by virtual tuners? Most
>>> people do not have that problem, and there are plenty of users of
>>> Hauppauge USB tuners. The way that virtual tuners work is just to
>>> allow more than one set of streams to be recorded at the same time
>>> from the same tuner. So the tuner hardware gets told to send streams
>>> a,b, and c to the PC for one channel and streams f, g, and h for
>>> another channel. It sends all those streams at once to the CPU. The
>>> CPU sorts out which streams are for which channel and writes them to
>>> separate recording files. The only problem you could get from getting
>>> the tuner to send more streams is if the total bit rate exceeded the
>>> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
>>> especially an Australian one that operates on 7 MHz bandwidth where
>>> the rest of the world mostly uses 8 MHz.
>>> When you set virtual tuners to 0, all you are doing is telling
>>> mythbackend to record from effectively one virtual tuner, in other
>>> words, record all the streams for one channel only from the tuner. But
>>> the same code is used as when multiple virtual tuners is specified.
>>> So your problem is more likely to be at the other end of things - if
>>> you write too many recordings at once to a hard drive, you will
>>> eventually get to the point where the heads will not be able to move
>>> between the file locations fast enough to keep up, and mythbackend's
>>> recording buffers will overflow before all the data can be written to
>>> disk. Which causes gaps in the data, which you see as pixellation and
>>> missing bits, and it also causes glitches in the sound. If this is
>>> happening, you should be getting bad recordings for all the recordings
>>> going to the overloaded hard disk during the time it was overloaded. I
>>> have been there and done that.
>>> So how many recordings are you writing to disk when you are getting
>>> pixilated recordings? How many disks are they going to? How fast are
>>> the disks?
>>> If this is what your problem is, then the solution is normally just to
>>> had another hard drive to the PC. I have 7 recording drives in order
>>> to prevent this problem. To be completely sure not to get overloaded
>>> hard drives, I work on the basis that there should be only 2
>>> recordings at once going to one hard drive. That then allows for
>>> playback of another recording from the same drive while those two
>>> recordings are in progress. In theory, the faster more recent hard
>>> drives should be able to cope with 3 recordings at once plus playback,
>>> or maybe more, but I have never tested that. I still have 2 old hard
>>> drives that probably can not do that, but I keep them as they are very
>>> quiet compared to more modern ones, so I can use them for the
>>> relatively few overnight recordings and not have the hard drive noises
>>> keep me awake.
>>> Along with recordings and playback, there can be other hard disk
>>> activity for commercial skip processing. Modern CPUs can normally
>>> cope with doing commercial skip processing in real-time from the
>>> in-memory recording buffers on the basis of one core per recording.
>>> Such processing does not cause any extra hard disk activity. If you
>>> are making more recordings at once than you have CPU cores, you
>>> normally set the number of commercial skip processes to the number of
>>> cores, and the extra recordings get scheduled for later processing. In
>>> which case, the commercial skip processes will be reading back those
>>> recordings from hard disk at a speed significantly greater than for
>>> playback or recording. And that can interfere with any recordings
>>> going on at the time as the heads will be moving back and forth quite
>>> a bit. I do not have this problem as I do not do any commercial skip
>>> processing - it does not work usefully on any New Zealand channels
>>> that I know of.
>> There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.
>>
>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>>
>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.
>>
>
>How I wish sane logic would prevail.
>
>Here be what happens:
>
>With Virtual Tuners about 1 recording a week is ?damaged? with bad pixelation
>With my bizare setting I?ve seen NO damaged recordings in 5 months.
>All the storage is SSD
>OS and all the associated stuff is on a samsung 980 M2 (2T)
>All myth including Videos is on a samsung QVO SATA (4T)
>CPU is Ryzen 7 5700U (8 core) with 24G ram (I use this box as as workstation during the day)
>
>The Oz TV (successfully !) channels defeat commercial skipping by changing the algorithm even for a single show so I disable all commercial skipping.
>
>Please wax lyrical on virtual channels. How could I tell a virtual channel from a mux?

There is no such thing as a virtual channel. All channels on digital
TV are exactly the same technically - they are simply a collections of
streams of data. The streams are normally 1 video, 1 or more audio
(eg AAC stereo and AC3 5.1, more for different languages) and 0 or
more for subtitles. Each stream consists of packets of data with the
same PID (packet ID) value at the start of the packet. There are
occasional packets that are repeated regularly that give the data
about what PIDs are part of each channel that is being broadcast, and
some other streams that are not part of channels such as for EIT
and/or MHEG5 EPG data.

In ATSC regions, there is political nomenclature that confuses this
simple technical setup, where a mux will typically be owned by a TV
station and will have one major channel in HD and high bit rate, and
then one or more other channels, typically SD and lower bit rate.
These two types of channels are differentiated by the language used to
describe them, and possibly also by legislation, but technically they
are no different to each other.

>Definitely there is some fiddling:
>The usual bitrate on 1080 is cica 1000 Kb/s
>Last year ABC had a live documentary of coral spawning where they raised the bitrate to over 2mb/s
>I did not check what they did on other channels to compensate
>
>[diversion] coral spawning is special: on 1 moonlit night per year at the right time ALL the coral spawn. On the barrier reef that means simultainous spawning over 1000s km.
>
>There is no logic on damaged channels. It could be the sole recording being made.
>I have tested 8 simultainous recordings all without any problem.
>
>I use an external antenna some 10 km line-of-sight to the transmitter farm. We have various ?filler? (low power) transmitters round Perth. I carefully pruned them from my list.
>
>James

The Samsung 870 QVO 4 Tbyte specifications say it has a 4 Gbyte DRAM
cache. What the specs carefully do not say is the actual speed of the
flash memory (especially the write and erase speeds). So whenever you
fill the DRAM cache, I would expect a very significant slow down in
the write speed. When you get bad recordings, is it always when you
have many simultaneous recordings happening? Is it possible that you
have exceeded the cache size as a result?

How full is the QVO drive normally? When a flash drive is fairly
full, it has few spare blocks available that can be pre-erased and put
on the writeable queue. If all the blocks on the writeable queue have
been used up, then a no longer in use block will need to be erased
before it can be written to and the erase times are horrendously
slower than the write times - they can be seconds, not milliseconds.
The recommendation for formatting any flash SSD is that you leave 10%
of the space as non-partitioned, so that 10% of the flash blocks are
available to be erased and put on the writeable queue, even when the
rest of the SSD is full. Did you do that?

When you tested with 8 simultaneous recordings, did you leave them
running for the typical recording time of 1 hour + pre- and post-roll
times? If not, then you do not really know if that situation could
cause problems. I have found in my testing of spinning rust that the
internal buffering by mythbackend is sufficient to cope with my
typical pre- and post-roll settings causing the number of simultaneous
recordings to double during the pre/post roll overlaps, as long as the
overlaps are not more than about 4-5 minutes. So I always do test
recordings that are longer than that, about 15 minutes. To fill a 4
Gbyte cache if the data was not being written to flash at all would
take about 1.3 hours of one of the NZ HD channels.

When you have had a sole recording that was damaged, was it possible
that it was made following a very busy set of multiple recordings that
might have overflowed the DRAM cache or the available erased blocks?
Or was it completely on its own?

One other thing to think about with USB tuners is their connections
and cables. USB plugs, cables and sockets vary in their tolerances,
and do not always fit very well. When they are not a close fit,
movement of the cable can cause disconnections of pins, which can
cause the same sort of bad recordings you are having. I used to have
this problem when I was using multiple USB tuners, and typically it
was one or two of the USB tuners out of the three I was using that had
the problem. Changing the cables on the bad tuners to better fitting
ones and switching the USB ports they were plugged into greatly
reduced the incidence of bad recordings, as did laying out the tuners
and cables carefully. My ultimate fix for this was to change to using
PCIe tuners. I now keep the old USB tuners as emergency backups and
for use on my laptop when I am away from home.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
> On Dec 9, 2023, at 11:47, Stephen Worthington <stephen_agent@jsw.gen.nz> wrote:
>
> On Sat, 9 Dec 2023 05:18:34 +0800, you wrote:
>
>>
>>
>>> On Dec 9, 2023, at 00:27, Mike Perkins <mikep@randomtraveller.org.uk> wrote:
>>>
>>> On 08/12/2023 15:56, Stephen Worthington wrote:
>>>> On Fri, 8 Dec 2023 20:55:31 +0800, you wrote:
>>>>> G’day all
>>>>> playing with the rules I can solve once, but everytime is a pain
>
>>>>>
>>>>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>>>>> Bprogram scheduled 1930 to 2230
>>>>>
>>>>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>>>>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>>>>> As suggested I put “prefer HD”
>>>>>
>>>>> Now Aprogram records on tuner one, two, three, four
>>>>> One recording of News is [red] conflict
>>>>> One recording of Bprogram is [red] conflict
>>>>>
>>>>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>>>>
>>>>> James
>>>>>
>>>>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>>>> How sure are you that the pixelation is caused by virtual tuners? Most
>>>> people do not have that problem, and there are plenty of users of
>>>> Hauppauge USB tuners. The way that virtual tuners work is just to
>>>> allow more than one set of streams to be recorded at the same time
>>>> from the same tuner. So the tuner hardware gets told to send streams
>>>> a,b, and c to the PC for one channel and streams f, g, and h for
>>>> another channel. It sends all those streams at once to the CPU. The
>>>> CPU sorts out which streams are for which channel and writes them to
>>>> separate recording files. The only problem you could get from getting
>>>> the tuner to send more streams is if the total bit rate exceeded the
>>>> maximum for USB 2.0, which is not possible for a DVB-T transmitter,
>>>> especially an Australian one that operates on 7 MHz bandwidth where
>>>> the rest of the world mostly uses 8 MHz.
>>>> When you set virtual tuners to 0, all you are doing is telling
>>>> mythbackend to record from effectively one virtual tuner, in other
>>>> words, record all the streams for one channel only from the tuner. But
>>>> the same code is used as when multiple virtual tuners is specified.
>>>> So your problem is more likely to be at the other end of things - if
>>>> you write too many recordings at once to a hard drive, you will
>>>> eventually get to the point where the heads will not be able to move
>>>> between the file locations fast enough to keep up, and mythbackend's
>>>> recording buffers will overflow before all the data can be written to
>>>> disk. Which causes gaps in the data, which you see as pixellation and
>>>> missing bits, and it also causes glitches in the sound. If this is
>>>> happening, you should be getting bad recordings for all the recordings
>>>> going to the overloaded hard disk during the time it was overloaded. I
>>>> have been there and done that.
>>>> So how many recordings are you writing to disk when you are getting
>>>> pixilated recordings? How many disks are they going to? How fast are
>>>> the disks?
>>>> If this is what your problem is, then the solution is normally just to
>>>> had another hard drive to the PC. I have 7 recording drives in order
>>>> to prevent this problem. To be completely sure not to get overloaded
>>>> hard drives, I work on the basis that there should be only 2
>>>> recordings at once going to one hard drive. That then allows for
>>>> playback of another recording from the same drive while those two
>>>> recordings are in progress. In theory, the faster more recent hard
>>>> drives should be able to cope with 3 recordings at once plus playback,
>>>> or maybe more, but I have never tested that. I still have 2 old hard
>>>> drives that probably can not do that, but I keep them as they are very
>>>> quiet compared to more modern ones, so I can use them for the
>>>> relatively few overnight recordings and not have the hard drive noises
>>>> keep me awake.
>>>> Along with recordings and playback, there can be other hard disk
>>>> activity for commercial skip processing. Modern CPUs can normally
>>>> cope with doing commercial skip processing in real-time from the
>>>> in-memory recording buffers on the basis of one core per recording.
>>>> Such processing does not cause any extra hard disk activity. If you
>>>> are making more recordings at once than you have CPU cores, you
>>>> normally set the number of commercial skip processes to the number of
>>>> cores, and the extra recordings get scheduled for later processing. In
>>>> which case, the commercial skip processes will be reading back those
>>>> recordings from hard disk at a speed significantly greater than for
>>>> playback or recording. And that can interfere with any recordings
>>>> going on at the time as the heads will be moving back and forth quite
>>>> a bit. I do not have this problem as I do not do any commercial skip
>>>> processing - it does not work usefully on any New Zealand channels
>>>> that I know of.
>>> There is a possible confusion here between virtual /tuners/ and virtual /channels/. I agree that whether a tuner is 'virtual' or not is going to make any difference, it is all the same code.
>>>
>>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>>>
>>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.
>>>
>>
>> How I wish sane logic would prevail.
>>
>> Here be what happens:
>>
>> With Virtual Tuners about 1 recording a week is ‘damaged’ with bad pixelation
>> With my bizare setting I’ve seen NO damaged recordings in 5 months.
>> All the storage is SSD
>> OS and all the associated stuff is on a samsung 980 M2 (2T)
>> All myth including Videos is on a samsung QVO SATA (4T)
>> CPU is Ryzen 7 5700U (8 core) with 24G ram (I use this box as as workstation during the day)
>>
>> The Oz TV (successfully !) channels defeat commercial skipping by changing the algorithm even for a single show so I disable all commercial skipping.
>>
>> Please wax lyrical on virtual channels. How could I tell a virtual channel from a mux?
>
> There is no such thing as a virtual channel.

This was posted and I did not understand

>>> However, virtual /channels/ could be a different problem, depending on specific language. In the US I know they call those 'subchannels' and it is likely that some or most have lower bitrates to cram more subchannels in. I don't know if the same applies in AU, but that could be the problem.
>>>
>>> Alternatively, it is simply possible that the signal is not strong enough and sometimes drops out. Perhaps a tweak to the antenna direction, check all the joints for soundness and/or water, etc? That might go a long way to solving the problem.

> All channels on digital
> TV are exactly the same technically - they are simply a collections of
> streams of data. The streams are normally 1 video, 1 or more audio
> (eg AAC stereo and AC3 5.1, more for different languages) and 0 or
> more for subtitles. Each stream consists of packets of data with the
> same PID (packet ID) value at the start of the packet. There are
> occasional packets that are repeated regularly that give the data
> about what PIDs are part of each channel that is being broadcast, and
> some other streams that are not part of channels such as for EIT
> and/or MHEG5 EPG data.
>
> In ATSC regions, there is political nomenclature that confuses this
> simple technical setup, where a mux will typically be owned by a TV
> station and will have one major channel in HD and high bit rate, and
> then one or more other channels, typically SD and lower bit rate.
> These two types of channels are differentiated by the language used to
> describe them, and possibly also by legislation, but technically they
> are no different to each other.
>
>> Definitely there is some fiddling:
>> The usual bitrate on 1080 is cica 1000 Kb/s
>> Last year ABC had a live documentary of coral spawning where they raised the bitrate to over 2mb/s
>> I did not check what they did on other channels to compensate
>>
>> [diversion] coral spawning is special: on 1 moonlit night per year at the right time ALL the coral spawn. On the barrier reef that means simultainous spawning over 1000s km.
>>
>> There is no logic on damaged channels. It could be the sole recording being made.
>> I have tested 8 simultainous recordings all without any problem.
>>
>> I use an external antenna some 10 km line-of-sight to the transmitter farm. We have various ‘filler’ (low power) transmitters round Perth. I carefully pruned them from my list.
>>
>> James
>
> The Samsung 870 QVO 4 Tbyte specifications say it has a 4 Gbyte DRAM
> cache. What the specs carefully do not say is the actual speed of the
> flash memory (especially the write and erase speeds). So whenever you
> fill the DRAM cache, I would expect a very significant slow down in
> the write speed. When you get bad recordings, is it always when you
> have many simultaneous recordings happening? Is it possible that you
> have exceeded the cache size as a result?
>
> How full is the QVO drive normally?

About 60%, 5% unallocated, trim daily cron

> When a flash drive is fairly
> full, it has few spare blocks available that can be pre-erased and put
> on the writeable queue. If all the blocks on the writeable queue have
> been used up, then a no longer in use block will need to be erased
> before it can be written to and the erase times are horrendously
> slower than the write times - they can be seconds, not milliseconds.
> The recommendation for formatting any flash SSD is that you leave 10%
> of the space as non-partitioned, so that 10% of the flash blocks are
> available to be erased and put on the writeable queue, even when the
> rest of the SSD is full. Did you do that?
>
> When you tested with 8 simultaneous recordings, did you leave them
> running for the typical recording time of 1 hour + pre- and post-roll
> times?

Almost certainly not. probably 15 minutes (6 months ago, memory is sus’)

> If not, then you do not really know if that situation could
> cause problems. I have found in my testing of spinning rust that the
> internal buffering by mythbackend is sufficient to cope with my
> typical pre- and post-roll settings causing the number of simultaneous
> recordings to double during the pre/post roll overlaps, as long as the
> overlaps are not more than about 4-5 minutes. So I always do test
> recordings that are longer than that, about 15 minutes. To fill a 4
> Gbyte cache if the data was not being written to flash at all would
> take about 1.3 hours of one of the NZ HD channels.
>
> When you have had a sole recording that was damaged, was it possible
> that it was made following a very busy set of multiple recordings that
> might have overflowed the DRAM cache or the available erased blocks?
> Or was it completely on its own?

Again 6 months ago!

>
> One other thing to think about with USB tuners is their connections
> and cables. USB plugs, cables and sockets vary in their tolerances,
> and do not always fit very well. When they are not a close fit,
> movement of the cable can cause disconnections of pins, which can
> cause the same sort of bad recordings you are having. I used to have
> this problem when I was using multiple USB tuners, and typically it
> was one or two of the USB tuners out of the three I was using that had
> the problem. Changing the cables on the bad tuners to better fitting
> ones and switching the USB ports they were plugged into greatly
> reduced the incidence of bad recordings, as did laying out the tuners
> and cables carefully. My ultimate fix for this was to change to using
> PCIe tuners. I now keep the old USB tuners as emergency backups and
> for use on my laptop when I am away from home.
>

The dog might have nested in the wires, a zillion things, so I need to carefully rinse and repeat.
Will do in the next week.
What about the original question, specially if cache size is the issue. I have one record rule that may record 1 to 5 copies, typically 2 to 4. How to impliment ‘record 1 showing’ ‘prefer HD’ ?

James
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
On Sat, 9 Dec 2023 13:44:40 +0800, you wrote:


>What about the original question, specially if cache size is the issue. I have one record rule that may record 1 to 5 copies, typically 2 to 4. How to impliment ?record 1 showing? ?prefer HD? ?

That depends on how you define HD. Does your EPG do that? Or is it
just that some channels are HD and some are SD? The power priority
rules work for EPG telling you if the particular recording is HD on
that channel. If the channels are HD and SD, then you normally just
give the HD channels higher priority in mythtv-setup. I have a number
of channels where there is a +1 channel that broadcasts the same
programming an hour later. Each main channel is HD, and each +1
channel is SD. I also have the same channels repeated on a satellite
mux, where they are all SD, and worse SD than the DVB-T SD. Here is
an example:

MariaDB [mythconverg]> select
chanid,channum,freqid,sourceid,mplexid,callsign,name,xmltvid,recpriority
from channel where visible=1 and callsign like 'TVNZ 1%' order by
callsign,channum+0;
+--------+---------+--------+----------+---------+-----------+-------------+---------------------------+-------------+
| chanid | channum | freqid | sourceid | mplexid | callsign | name | xmltvid | recpriority |
+--------+---------+--------+----------+---------+-----------+-------------+---------------------------+-------------+
| 1001 | 1 | 34 | 1 | 2 | TVNZ 1 | TVNZ 1 | tvnz1.freeviewnz.tv | 10 |
| 3001 | 1 | | 3 | 6 | TVNZ 1 | TVNZ 1 | tvnz1.freeviewnz.tv | -5 |
| 3091 | 91 | | 3 | 6 | TVNZ 1 | TVNZ 1 HAM | tvnz1.freeviewnz.tv | -5 |
| 3092 | 92 | | 3 | 6 | TVNZ 1 | TVNZ 1 WGTN | tvnz1.freeviewnz.tv | -5 |
| 3093 | 93 | | 3 | 6 | TVNZ 1 | TVNZ 1 CHCH | tvnz1.freeviewnz.tv | -5 |
| 1006 | 11 | 34 | 1 | 2 | TVNZ 1 +1 | TVNZ 1 +1 | tvnz1-plus1.freeviewnz.tv | 5 |
+--------+---------+--------+----------+---------+-----------+-------------+---------------------------+-------------+
6 rows in set (0.000 sec)

The sourceid=1 channels are terrestrial DVB-T, and sourceid=3 are
satellite DVB-S. So I have the DVB-T HD channel with priority 10, the
DVB-T +1 channel with priority 5 and the DVB-S channels (including
three regional versions with different advertising) all as priority
-5. Since I have enough DVB-T tuners to have one per mux, I always
get recordings for TVNZ 1 from the DVB-T HD channel. If I had too few
tuners, then other priorities (such as a different programme of higher
priority recording on a different channel) might override that and the
second choice of DVB-T from the +1 channel might be where it recorded
from. If the conflict for the DVB-T tuners was really bad, it might
record from one of the DVB-S channels.

I am not sure how you get a 'record 1 showing' rule to record more
than one copy of a programme. Such a rule should record only one copy
and then delete itself from the database. I think your rules that are
doing that will be "Record all showings" rules. On such rules, you
can set the "This channel only" option, but then if you then run out
of tuners it may not record the programme at all. You would get an on
screen Conflict notice in the Upcoming Recordings screen if it
happened.

You can create 'record 1 showing per week' rules also, but they are
done using Custom Edit (from the EPG listing, M=Menu > Recording
Options > Custom Edit) so a lot of people do not know about them. They
use SQL saved in the record.description field of the database, and the
scheduler has to run that SQL as part of its big scheduler query, so
having too many rules of this type will slow down the scheduler.
However, your MythTV PC is sufficiently powerful that it is unlikely
to matter for you.

Also from Custom Edit, you can use arbitrary SQL to choose what to
record. So you can make very complex decisions that way. If you run
out of room for the SQL text in the record.description field, you can
externally create stored procedures in the database and call them from
the Custom Edit SQL.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
> There is no such thing as a virtual channel.

Minor correction: here in the U.S. we tend to use the old analog channel
number for station identification.  For instance, KWQC-TV was on analog
(NTSC format) Channel 6.  With the digital conversion (ATSC) the station
is transmitting on RF Channel 17 but is still known as "Channel 6" and
is branded and advertised as "TV 6".  The television itself remaps the
station: I push the 6.1 buttons on the remote to view what is really at
17.1.

So the virtual channel is 6.1 but one is really watching the primary on
channel 17.
Re: schedule howto [ In reply to ]
On 12/8/23 07:55, James Linder wrote:
> G’day all
> playing with the rules I can solve once, but everytime is a pain …
>
> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
> Bprogram scheduled 1930 to 2230
>
> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
> As suggested I put “prefer HD”
>
> Now Aprogram records on tuner one, two, three, four
> One recording of News is [red] conflict
> One recording of Bprogram is [red] conflict
>
> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>
> James
>
> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>
>
>
Getting back to your original question.

As I understand it,  ABC news is on 5 channels so MythTV records 4
copies of it, and then fails to record another program because it ran
out of tuners (conflict).

If you use schedules direct for the guide, schedules direct assigns a
unique id to every program so that this does not happen. MythTV
recognizes the extra showings as duplicates and only records one of
them.  I don't know about Schedules direct in Australia, whether it is
available or whether it assigns unique ids to shows. Schedules direct
does offer a 7 day free trial and a monthly option that you could use to
try it out. It is a bit of a pain to set up.

I assume you are using EPG for the program schedule. Can you set a
filter of "this channel" in your recording rule? You must know which
channel carries a program. Surely they don't switch channels around on a
daily basis?

Another thing you could do is give a higher priority to the recording
rule for program B. Then when 5 channels show program A and one shows
program B, then program B will be recorded along with three copies of
program A. At least you will not miss program B.

Record One showing only or only this showing are not helpful, that means
one showing only, ever. You would have to set it up again every day.

Some of the filters may help. Perhaps "First Showing" may help. Even if
4 showings are at the same time, MythTV may decide one of them is a
first showing. I have not used this so I may be wrong here.

Power search allows you to set your own SQL, so if  there is something
unique about the subtitle or description that identifies which show you
want, you could create a power rule that restricts it to the one you want.

A lot of options to try. You can set up different ways then look at
Upcoming recordings to see what will be the result.

Note: If you are running a recent version of master (pre-34) you can use
the web app to set up and modify recording rules. Preferably don't use
mythweb, it is out of date and has not been maintained.

Peter

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: schedule howto [ In reply to ]
> On Dec 9, 2023, at 22:37, Peter Bennett <pb.mythtv@gmail.com> wrote:
>
>
> On 12/8/23 07:55, James Linder wrote:
>> G’day all
>> playing with the rules I can solve once, but everytime is a pain …
>>
>> Aprogram (news on Australian ABC) scheduled to record from 1900 to 2000 (ish)
>> Bprogram scheduled 1930 to 2230
>>
>> Aprogram (daily) can be on any combination of 2, 20, 21, 22, 23
>> My hardware causes some virtual channels to sometimes record broken recordings (badly pixelated) as a result I’ve set <no> virtual channels.
>> As suggested I put “prefer HD”
>>
>> Now Aprogram records on tuner one, two, three, four
>> One recording of News is [red] conflict
>> One recording of Bprogram is [red] conflict
>>
>> Wunce-upon-a-time I recall a rule “record 1 showing only”. That with “Prefer HD” would completely solve my problem. What is the solution? Better tuners, they are Haupauge USB dual tuners.
>>
>> James
>>
>> PS I can't give prority to any 2, 20-23, that channel may not carry Aprogram
>>
>>
>>
> Getting back to your original question.
>
> As I understand it, ABC news is on 5 channels so MythTV records 4 copies of it, and then fails to record another program because it ran out of tuners (conflict).
>
> If you use schedules direct for the guide, schedules direct assigns a unique id to every program so that this does not happen. MythTV recognizes the extra showings as duplicates and only records one of them. I don't know about Schedules direct in Australia, whether it is available or whether it assigns unique ids to shows. Schedules direct does offer a 7 day free trial and a monthly option that you could use to try it out. It is a bit of a pain to set up.
I’ll look at details. When I first investigated opinion was ‘EPG data in Australia is regulated consequently very good’.

> I assume you are using EPG for the program schedule. Can you set a filter of "this channel" in your recording rule? You must know which channel carries a program. Surely they don't switch channels around on a daily basis?

No but every few days some channels are preset to something else. IE normally CH 21 is news but today it’s tennis.
ie today 20,21 news, 23, 24 other programs

> Another thing you could do is give a higher priority to the recording rule for program B. Then when 5 channels show program A and one shows program B, then program B will be recorded along with three copies of program A. At least you will not miss program B.
>
> Record One showing only or only this showing are not helpful, that means one showing only, ever. You would have to set it up again every day.
>
> Some of the filters may help. Perhaps "First Showing" may help. Even if 4 showings are at the same time, MythTV may decide one of them is a first showing. I have not used this so I may be wrong here.

First showing (today I tried) no definite result.

> Power search allows you to set your own SQL, so if there is something unique about the subtitle or description that identifies which show you want, you could create a power rule that restricts it to the one you want.

No such luck! All copies say same words.

> A lot of options to try. You can set up different ways then look at Upcoming recordings to see what will be the result.
>
> Note: If you are running a recent version of master (pre-34) you can use the web app to set up and modify recording rules. Preferably don't use mythweb, it is out of date and has not been maintained.

Minor irritation deleting 4 copies every day. I just wondered if I was being dumb about a common problem.

Thanks for lots of input everybody
James
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org