Mailing List Archive

On 08/08/17 21:41, Mark Goldberg wrote:
> I have added some videos and do a rescan. Some videos are not seen or
> added to the list. I can rename those videos and they are still not seen
> on a rescan. I can put them in a different subdirectory and they are
> still not seen.
>
> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
>
> Any suggestions how to troubleshoot this?
>
> Thanks,
>
> Mark

Have you tried scanning an empty videos folder first? AIUI a video is
not added if its hash is already present.



_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 08/08/17 22:35, John Pilkington wrote:
> On 08/08/17 21:41, Mark Goldberg wrote:
>> I have added some videos and do a rescan. Some videos are not seen or
>> added to the list. I can rename those videos and they are still not
>> seen on a rescan. I can put them in a different subdirectory and they
>> are still not seen.
>>
>> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
>>
>> Any suggestions how to troubleshoot this?
>>
>> Thanks,
>>
>> Mark
>
> Have you tried scanning an empty videos folder first? AIUI a video is
> not added if its hash is already present.

I'm not sure that fits what you say, though. The Video path can be set
up either via Storage Groups in mythtvsetup or in the frontend via Setup
> Media Settings. A mismatch?

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 8/8/17 4:41 PM, Mark Goldberg wrote:
> I have added some videos and do a rescan. Some videos are not seen or
> added to the list. I can rename those videos and they are still not
> seen on a rescan. I can put them in a different subdirectory and they
> are still not seen.
One other thing to look at: do you use "parental levels" for videos? If
so, the videos might have already been added, but are hidden because
your default parental level is lower than that of the missing videos.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Wed, Aug 9, 2017 at 5:55 AM, Dan Wilga wrote:
>
> On 8/8/17 4:41 PM, Mark Goldberg wrote:
>>
>> I have added some videos and do a rescan. Some videos are not seen or
added to the list. I can rename those videos and they are still not seen on
a rescan. I can put them in a different subdirectory and they are still not
seen.
>
> One other thing to look at: do you use "parental levels" for videos? If
so, the videos might have already been added, but are hidden because your
default parental level is lower than that of the missing videos.


On 08/09/17, John Pilkington wrote:
>On 08/08/17 22:35, John Pilkington wrote:
>> On 08/08/17 21:41, Mark Goldberg wrote:
>>> I have added some videos and do a rescan. Some videos are not seen or
>>> added to the list. I can rename those videos and they are still not
>>> seen on a rescan. I can put them in a different subdirectory and they
>>> are still not seen.
>>>
>>> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
>>>
>>> Any suggestions how to troubleshoot this?
>>>
>>> Thanks,
>>>
>>> Mark
>>
>> Have you tried scanning an empty videos folder first? AIUI a video is
>> not added if its hash is already present.
>
>I'm not sure that fits what you say, though. The Video path can be set
>up either via Storage Groups in mythtvsetup or in the frontend via Setup
> Media Settings. A mismatch?

I have videos set up in mythsetup for years. They are in one directory,
/archives/videos.
In a subdirectory, /archives/videos/hdv, I am creating h.264 / PCM files
with names like
MVI_6007_5994_18D.MOV
MVI_6007_5994_18E.MOV
MVI_6007_5994_18F.MOV
MVI_6007_5994_18G.MOV
with ffmpeg, trying different encoding parameters.

I do a rescan, and I can see the first two files but not the second two.
All are original files, not downloaded from anywhere and have no matching
metadata.

Mediainfo for the files seen and not seen is the same except for the
filename.

I can do a "touch junk.mov" in that subdirectory, and that is seen after a
scan, but it obviously complains when trying to play it as it is empty.

I hope this explains things better.

I am not sure how to scan an empty video folder. There is just one option,
rescan the whole directory.

I am not using parental levels.

Mark

On 09/08/17 14:25, Mark Goldberg wrote:
> On Wed, Aug 9, 2017 at 5:55 AM, Dan Wilga wrote:
> >
> > On 8/8/17 4:41 PM, Mark Goldberg wrote:
> >>
> >> I have added some videos and do a rescan. Some videos are not seen
> or added to the list. I can rename those videos and they are still not
> seen on a rescan. I can put them in a different subdirectory and they
> are still not seen.
> >
> > One other thing to look at: do you use "parental levels" for videos?
> If so, the videos might have already been added, but are hidden because
> your default parental level is lower than that of the missing videos.
>
>
> On 08/09/17, John Pilkington wrote:
> >On 08/08/17 22:35, John Pilkington wrote:
> >> On 08/08/17 21:41, Mark Goldberg wrote:
> >>> I have added some videos and do a rescan. Some videos are not seen or
> >>> added to the list. I can rename those videos and they are still not
> >>> seen on a rescan. I can put them in a different subdirectory and they
> >>> are still not seen.
> >>>
> >>> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
> >>>
> >>> Any suggestions how to troubleshoot this?
> >>>
> >>> Thanks,
> >>>
> >>> Mark
> >>
> >> Have you tried scanning an empty videos folder first? AIUI a video is
> >> not added if its hash is already present.
> >
> >I'm not sure that fits what you say, though. The Video path can be set
> >up either via Storage Groups in mythtvsetup or in the frontend via Setup
> > Media Settings. A mismatch?
>
> I have videos set up in mythsetup for years. They are in one
> directory, /archives/videos.
> In a subdirectory, /archives/videos/hdv, I am creating h.264 / PCM files
> with names like
> MVI_6007_5994_18D.MOV
> MVI_6007_5994_18E.MOV
> MVI_6007_5994_18F.MOV
> MVI_6007_5994_18G.MOV
> with ffmpeg, trying different encoding parameters.
>
> I do a rescan, and I can see the first two files but not the second two.
> All are original files, not downloaded from anywhere and have no
> matching metadata.
>
> Mediainfo for the files seen and not seen is the same except for the
> filename.
>
> I can do a "touch junk.mov" in that subdirectory, and that is seen after
> a scan, but it obviously complains when trying to play it as it is empty.
>
> I hope this explains things better.
>
> I am not sure how to scan an empty video folder. There is just one
> option, rescan the whole directory.
>
> I am not using parental levels.
>
> Mark

Create an empty directory, define it as your video source in
mythtvsetup-or-media-files, and scan it. Then go back to your original
configuration and rescan. I don't think this has changed recently, but
Ticket #10500 was perhaps similar. Or search 'mythtv video hash'

Perhaps your ffmpeg processes gave identical results.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 09/08/17 15:04, John Pilkington wrote:
> On 09/08/17 14:25, Mark Goldberg wrote:
>> On Wed, Aug 9, 2017 at 5:55 AM, Dan Wilga wrote:
>> >
>> > On 8/8/17 4:41 PM, Mark Goldberg wrote:
>> >>
>> >> I have added some videos and do a rescan. Some videos are not seen or added to the list. I can
>> rename those videos and they are still not seen on a rescan. I can put them in a different
>> subdirectory and they are still not seen.
>> >
>> > One other thing to look at: do you use "parental levels" for videos? If so, the videos might
>> have already been added, but are hidden because your default parental level is lower than that of
>> the missing videos.
>>
>>
>> On 08/09/17, John Pilkington wrote:
>> >On 08/08/17 22:35, John Pilkington wrote:
>> >> On 08/08/17 21:41, Mark Goldberg wrote:
>> >>> I have added some videos and do a rescan. Some videos are not seen or
>> >>> added to the list. I can rename those videos and they are still not
>> >>> seen on a rescan. I can put them in a different subdirectory and they
>> >>> are still not seen.
>> >>>
>> >>> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
>> >>>
>> >>> Any suggestions how to troubleshoot this?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Mark
>> >>
>> >> Have you tried scanning an empty videos folder first? AIUI a video is
>> >> not added if its hash is already present.
>> >
>> >I'm not sure that fits what you say, though. The Video path can be set
>> >up either via Storage Groups in mythtvsetup or in the frontend via Setup
>> > Media Settings. A mismatch?
>>
>> I have videos set up in mythsetup for years. They are in one directory, /archives/videos.
>> In a subdirectory, /archives/videos/hdv, I am creating h.264 / PCM files with names like
>> MVI_6007_5994_18D.MOV
>> MVI_6007_5994_18E.MOV
>> MVI_6007_5994_18F.MOV
>> MVI_6007_5994_18G.MOV
>> with ffmpeg, trying different encoding parameters.
>>
>> I do a rescan, and I can see the first two files but not the second two. All are original files,
>> not downloaded from anywhere and have no matching metadata.
>>
>> Mediainfo for the files seen and not seen is the same except for the filename.
>>
>> I can do a "touch junk.mov" in that subdirectory, and that is seen after a scan, but it obviously
>> complains when trying to play it as it is empty.
>>
>> I hope this explains things better.
>>
>> I am not sure how to scan an empty video folder. There is just one option, rescan the whole
>> directory.
>>
>> I am not using parental levels.
>>
>> Mark
>
> Create an empty directory, define it as your video source in mythtvsetup-or-media-files, and scan
> it. Then go back to your original configuration and rescan. I don't think this has changed
> recently, but
> Ticket #10500 was perhaps similar. Or search 'mythtv video hash'
>
> Perhaps your ffmpeg processes gave identical results.
>
Clarify: "define it as your *only* video source".

--

Mike Perkins

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 08/09/2017 10:04 AM, John Pilkington wrote:
> Or search 'mythtv video hash'
>
> Perhaps your ffmpeg processes gave identical results.

Yes, this is most likely the issue. The hash that's created to identify
unique videos is done with only a small part of the file at the
beginning and the end (because reading multiple gigabytes per file from
disk and running the entire video through a hash algorithm would take
ages to complete). It's likely your test files are just too close to
the same, so are being considered the same and only showed once (one or
the other is shown and the one not shown is ignored).

There's no way to "fix" this as it's working as designed (and typical
use cases don't involve multiple almost identical files). So, you can
rotate them in and out to see them all. Note, too, that the title of
the video shown is not necessarily the name of the file being
played--you'd have to look at the filename in the detailed metadata
screen to know for sure which you're playing--and if you swap files the
new file may show with the same title, even though it would now play the
other file.

You may be able to work around this by creating test encodes of
differing lengths/portions of the video.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 10 August 2017 12:01:01 am "Michael T. Dean" <mtdean@thirdcontact.com>
wrote:

> On 08/09/2017 10:04 AM, John Pilkington wrote:
>> Or search 'mythtv video hash'
>>
>> Perhaps your ffmpeg processes gave identical results.
>
> Yes, this is most likely the issue. The hash that's created to identify
> unique videos is done with only a small part of the file at the
> beginning and the end (because reading multiple gigabytes per file from
> disk and running the entire video through a hash algorithm would take
> ages to complete). It's likely your test files are just too close to
> the same, so are being considered the same and only showed once (one or
> the other is shown and the one not shown is ignored).
>
> There's no way to "fix" this as it's working as designed (and typical
> use cases don't involve multiple almost identical files).

I wonder if the hash algorithm could be updated to include the file
creation date or something else similar to help mitigate this type of overlap?

> So, you can
> rotate them in and out to see them all. Note, too, that the title of
> the video shown is not necessarily the name of the file being
> played--you'd have to look at the filename in the detailed metadata
> screen to know for sure which you're playing--and if you swap files the
> new file may show with the same title, even though it would now play the
> other file.
>
> You may be able to work around this by creating test encodes of
> differing lengths/portions of the video.
>
> Mike
>


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 08/09/2017 05:05 PM, Mark Perkins wrote:
> On 10 August 2017 12:01:01 am "Michael T. Dean" wrote:
>
>> On 08/09/2017 10:04 AM, John Pilkington wrote:
>>> Or search 'mythtv video hash'
>>>
>>> Perhaps your ffmpeg processes gave identical results.
>> Yes, this is most likely the issue. The hash that's created to identify
>> unique videos is done with only a small part of the file at the
>> beginning and the end (because reading multiple gigabytes per file from
>> disk and running the entire video through a hash algorithm would take
>> ages to complete). It's likely your test files are just too close to
>> the same, so are being considered the same and only showed once (one or
>> the other is shown and the one not shown is ignored).
>>
>> There's no way to "fix" this as it's working as designed (and typical
>> use cases don't involve multiple almost identical files).
> I wonder if the hash algorithm could be updated to include the file
> creation date or something else similar to help mitigate this type of overlap?

We're using the same hash algorithm used by other projects, the
OpenSubtitles hashing algorithm, which is based on Media Player Classic.
http://trac.opensubtitles.org/projects/opensubtitles/wiki/HashSourceCodes

Any changes make ours a custom hash, with custom bugs and custom testing
and custom maintenance required. It's also quite possible that some
people are using the hash with other programs.

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

> Date: Wed, 9 Aug 2017 21:05:17 +0000
> From: Mark Perkins <perkins1724@hotmail.com>

> I wonder if the hash algorithm could be updated to include the file
> creation date or something else similar to help mitigate this type of overlap?

Probably a bad idea, since those who use "cp" instead of "cp -p" to copy
a file will reset that date.

Since this is such a specialized case (possibly-identical results from
manual runs of ffmpeg while testing something), I wonder if there might
be some other workaround that can be used in that workflow, rather than
by changing mythtv. ffmpeg and avconv both have the -metadata flag, for
example; perhaps that can be used to embed unique metadata at the start
of the file in a place that the hash will see. The easiest metadata to
use is probably seconds or nanoseconds since the epoch.


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Aug 10, 2017, "Michael T. Dean" wrote:
>
> On Wed, Aug 9, 2017 at 5:55 AM, Dan Wilga wrote:
> On 08/09/2017 05:05 PM, Mark Perkins wrote:
> > On 10 August 2017 12:01:01 am "Michael T. Dean" wrote:
> >
> >> On 08/09/2017 10:04 AM, John Pilkington wrote:
> >>> Or search 'mythtv video hash'
> >>>
> >>> Perhaps your ffmpeg processes gave identical results.
> >> Yes, this is most likely the issue. The hash that's created to identify
> >> unique videos is done with only a small part of the file at the
> >> beginning and the end (because reading multiple gigabytes per file from
> >> disk and running the entire video through a hash algorithm would take
> >> ages to complete). It's likely your test files are just too close to
> >> the same, so are being considered the same and only showed once (one or
> >> the other is shown and the one not shown is ignored).
> >>
> >> There's no way to "fix" this as it's working as designed (and typical
> >> use cases don't involve multiple almost identical files).
> > I wonder if the hash algorithm could be updated to include the file
> > creation date or something else similar to help mitigate this type of
overlap?
>
> We're using the same hash algorithm used by other projects, the
> OpenSubtitles hashing algorithm, which is based on Media Player Classic.
> http://trac.opensubtitles.org/projects/opensubtitles/wiki/HashSourceCodes
>
> Any changes make ours a custom hash, with custom bugs and custom testing
> and custom maintenance required. It's also quite possible that some
> people are using the hash with other programs.
>
>
>

I checked the files and they are, in fact, identical, despite using
different command options to ffmpeg. So, I suppose there is not really a
big issue. Thanks for pointing out the hash algorithm, at least it is not a
mystery to me any longer.

Mark

Mark,
pls make sure that this not seen movie hasn’t parental control level higher that currently selected to see videos.
When it will be higher - You will not see movie in the list…

> Wiadomo?? napisana przez Mark Goldberg <marklgoldberg@gmail.com> w dniu 08.08.2017, o godz. 22:41:
>
> I have added some videos and do a rescan. Some videos are not seen or added to the list. I can rename those videos and they are still not seen on a rescan. I can put them in a different subdirectory and they are still not seen.
>
> This is on mythtv 0.28.1-3 from rpmfusion on Fedora 26.r
>
> Any suggestions how to troubleshoot this?
>
> Thanks,
>
> Mark
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org