Mailing List Archive: Question about new build requirements for NVDEC

Question about new build requirements for NVDEC

Jan 15, 2019, 8:10 AM

Post #1 of 12 (2482 views)

David

I am ready to commit the first stage of NVDEC support (NVidia hardware
decode). In order to build FFmpeg with NVDEC support it needs something
unusual.

You need to clone the repo git@github.com:FFmpeg/nv-codec-headers.git
and run "sudo make install". All this does is install some header files
into /usr/local/include/ffnvcodec and installs a pkgconfig file in
/usr/local/lib/pkgconfig. The ffmpeg build looks for the pkginfo
information to find the header files.

peter@raza:~$ ls -l /usr/local/include/ffnvcodec
-rw-r--r-- 1 root root 14233 Dec 9 18:43 dynlink_cuda.h
-rw-r--r-- 1 root root 49467 Dec 9 18:43 dynlink_cuviddec.h
-rw-r--r-- 1 root root 17015 Dec 9 18:43 dynlink_loader.h
-rw-r--r-- 1 root root 22042 Dec 9 18:43 dynlink_nvcuvid.h
-rw-r--r-- 1 root root 198550 Dec 9 18:43 nvEncodeAPI.h

peter@raza:~$ ls -l /usr/local/lib/pkgconfig
-rw-r--r-- 1 root root 162 Dec 9 18:43 ffnvcodec.pc

peter@raza:~$ cat /usr/local/lib/pkgconfig/ffnvcodec.pc
prefix=/usr/local
includedir=${prefix}/include

Name: ffnvcodec
Description: FFmpeg version of Nvidia Codec SDK headers
Version: 8.2.15.7
Cflags: -I${includedir}

I could not find any Ubuntu package that contains these files that could
be added as a dependency.

I think the best action is to add a copy of the ffnvcodec repository to
the mythtv/external directory. Before ffmpeg is built, build ffnvcodec
with a prefix that is in some temporary location, and add the temporary
location for the pkgconfig directory to the PKG_CONFIG_PATH environment
variable. What do you think?

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

gary.buhrmaster at gmail

Jan 16, 2019, 11:21 AM

Post #2 of 12 (2469 views)

Permalink

On Tue, Jan 15, 2019 at 4:10 PM Peter Bennett <pb.mythtv@gmail.com> wrote:

> I could not find any Ubuntu package that contains these files that could
> be added as a dependency.

For a one-time add, possibly the nv-codec-headers deb file from
deb-multimedia? Since they are only headers (and config), there
is some hope they might install and be usable in stretch and
supported ubuntu variants even though deb-multimedia only has
them formally available for sid/buster.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

pb.mythtv at gmail

Jan 16, 2019, 11:41 AM

Post #3 of 12 (2469 views)

Permalink

On 1/16/19 2:21 PM, Gary Buhrmaster wrote:
> On Tue, Jan 15, 2019 at 4:10 PM Peter Bennett <pb.mythtv@gmail.com> wrote:
>
>> I could not find any Ubuntu package that contains these files that could
>> be added as a dependency.
> For a one-time add, possibly the nv-codec-headers deb file from
> deb-multimedia? Since they are only headers (and config), there
> is some hope they might install and be usable in stretch and
> supported ubuntu variants even though deb-multimedia only has
> them formally available for sid/buster.
> _______________________________________________
>
Hi Gary

Thanks for the info. I did not realize it was there.

For a one time add, it is easy. I simply ran the sudo make install.

For future builders including end users and package builders, requiring
a package from deb-multimedia may be a problem? I imagine Ubuntu would
not accept a package that required a package from deb-multimedia as a
dependency for building. Ubuntu must have a solution for when they build
FFmpeg and if I am feeling very enthusiastic I may take a look at the
Ubuntu FFmpeg package.

On the other hand, we carry our own copy of FFmpeg in the MythTV
repository, so it may make sense that we should carry a copy of this as
well, since it is really a part of FFmpeg.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

gary.buhrmaster at gmail

Jan 16, 2019, 12:15 PM

Post #4 of 12 (2469 views)

Permalink

FD: I do not use debian or any of its derivatives on
any regular basis. Anything I think I know is probably
at best stale, and possibly wrong.

On Wed, Jan 16, 2019 at 7:41 PM Peter Bennett <pb.mythtv@gmail.com> wrote:

> For future builders including end users and package builders, requiring
> a package from deb-multimedia may be a problem?

The number of people building from source is likely
smallish compared to what gets installed via someone
else's packaging. deb-multimedia, as I understand it,
has an interesting history, and can require careful
configuration to avoid certain issues and anomalies.

Installing anything locally into /usr/local has its own
issues and anomalies of course.

> I imagine Ubuntu would not accept a package that
> required a package from deb-multimedia as a
> dependency for building.

Probably not, but it is also possible no one has
started the process(es) to get nv-codec-headers
into Ubuntu along with ffmpeg. I have no standing,
but someone who did could look into starting the
process to get those headers packaged for the
good of everyone.

And one could consider creating a PPA now, until
Ubuntu steps up.

> On the other hand, we carry our own copy of FFmpeg in the MythTV
> repository, so it may make sense that we should carry a copy of this as
> well, since it is really a part of FFmpeg.

Certainly a possibility, but anything you do not need
to carry is likely better. And if the distro provides the
headers, you should probably prefer them (as
configure now does for a number of libraries)
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

david at istwok

Jan 16, 2019, 3:35 PM

Post #5 of 12 (2467 views)

Permalink

On Wed, Jan 16, 2019 at 08:15:56PM +0000, Gary Buhrmaster wrote:
> On Wed, Jan 16, 2019 at 7:41 PM Peter Bennett <pb.mythtv@gmail.com> wrote:
> > For future builders including end users and package builders, requiring
> > a package from deb-multimedia may be a problem?
>
> The number of people building from source is likely
> smallish compared to what gets installed via someone
> else's packaging. deb-multimedia, as I understand it,
> has an interesting history, and can require careful
> configuration to avoid certain issues and anomalies.
>
> Installing anything locally into /usr/local has its own
> issues and anomalies of course.
>
> > I imagine Ubuntu would not accept a package that
> > required a package from deb-multimedia as a
> > dependency for building.
>
> Probably not, but it is also possible no one has
> started the process(es) to get nv-codec-headers
> into Ubuntu along with ffmpeg. I have no standing,
> but someone who did could look into starting the
> process to get those headers packaged for the
> good of everyone.
>
> And one could consider creating a PPA now, until
> Ubuntu steps up.
>
> > On the other hand, we carry our own copy of FFmpeg in the MythTV
> > repository, so it may make sense that we should carry a copy of this as
> > well, since it is really a part of FFmpeg.
>
> Certainly a possibility, but anything you do not need
> to carry is likely better. And if the distro provides the
> headers, you should probably prefer them (as
> configure now does for a number of libraries)

As a long-time, Debian user, and one who has been bitten by problems
caused by installing d.m.o packages, I strongly recommend against
using them. We should either include the headers in our own source or
create packages for them until they are included in the distros.

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

gary.buhrmaster at gmail

Jan 16, 2019, 3:59 PM

Post #6 of 12 (2467 views)

Permalink

On Wed, Jan 16, 2019 at 11:35 PM David Engel <david@istwok.net> wrote:
> until they are included in the distros.

Good to hear you are going to push Ubuntu
to get them included.
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

ijc at hellion

Jan 17, 2019, 1:10 AM

Post #7 of 12 (2465 views)

Permalink

On Wed, 2019-01-16 at 14:41 -0500, Peter Bennett wrote:
>
> On 1/16/19 2:21 PM, Gary Buhrmaster wrote:
> > On Tue, Jan 15, 2019 at 4:10 PM Peter Bennett <pb.mythtv@gmail.com>
> wrote:
> >
> >> I could not find any Ubuntu package that contains these files that
> could
> >> be added as a dependency.
> > For a one-time add, possibly the nv-codec-headers deb file from
> > deb-multimedia? Since they are only headers (and config), there
> > is some hope they might install and be usable in stretch and
> > supported ubuntu variants even though deb-multimedia only has
> > them formally available for sid/buster.
> >

> Thanks for the info. I did not realize it was there.

In Debian there is:
$ apt-file search dynlink_cuda.h
nvidia-cuda-dev: /usr/include/dynlink_cuda.h
$ apt policy nvidia-cuda-dev
nvidia-cuda-dev:
Installed: (none)
Candidate: 9.1.85-8
Version table:
9.2.148-5 500
500 http://mirror/debian sid/non-free amd64 Packages
9.1.85-8 990
990 http://mirror/debian buster/non-free amd64 Packages
8.0.44-4 500
500 http://mirror/debian stretch/non-free amd64 Packages

However:
$ apt show nvidia-cuda-dev | grep Section
Section: non-free/libdevel

So it is in non-free[0] (so not strictly part of Debian). It's possible
that in Ubuntu it is also in whatever their non-free equivalent is?

I think mythtv already requires other deps from non-free, so possibly
using those is not a big deal.

Ian.

[0] Reasons in
https://metadata.ftp-master.debian.org/changelogs/non-free/n/nvidia-cuda-toolkit/nvidia-cuda-toolkit_9.2.148-5_copyright

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

david at istwok

Jan 17, 2019, 8:51 AM

Post #8 of 12 (2463 views)

Permalink

On Wed, Jan 16, 2019 at 11:59:03PM +0000, Gary Buhrmaster wrote:
> On Wed, Jan 16, 2019 at 11:35 PM David Engel <david@istwok.net> wrote:
> > until they are included in the distros.
>
> Good to hear you are going to push Ubuntu
> to get them included.

You looking at me? :)

All I would do is request to Debian that the files be packaged. Ian
found the right package for the Nvidia owned parts. I looked myself
but failed. I looked for dynlink when I should have looked for cuda.
That leaves some ffmpeg-specific parts still missing, I believe. My
current feeling is that ffmpeg should eventually include those and we
should add them to our copy of ffmpeg until they do.

As for Ubuntu, no hablo Ubuntu! Seriously, if they follow Debian's
packaging of the Nvidia driver, they should already have it. If they
reinvent that wheel themselves, then I don't know. Don't some of our
main packagers have relationships with Ubuntu already?

David
--
David Engel
david@istwok.net
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

ijc at hellion

Jan 17, 2019, 8:59 AM

Post #9 of 12 (2463 views)

Permalink

On Thu, 2019-01-17 at 10:51 -0600, David Engel wrote:
> On Wed, Jan 16, 2019 at 11:59:03PM +0000, Gary Buhrmaster wrote:
> > On Wed, Jan 16, 2019 at 11:35 PM David Engel <david@istwok.net>
> > wrote:
> > > until they are included in the distros.
> >
> > Good to hear you are going to push Ubuntu
> > to get them included.
>
> You looking at me? :)
>
> All I would do is request to Debian that the files be packaged. Ian
> found the right package for the Nvidia owned parts. I looked myself
> but failed. I looked for dynlink when I should have looked for cuda.
> That leaves some ffmpeg-specific parts still missing, I believe. My
> current feeling is that ffmpeg should eventually include those and we
> should add them to our copy of ffmpeg until they do.
>
> As for Ubuntu, no hablo Ubuntu! Seriously, if they follow Debian's
> packaging of the Nvidia driver, they should already have it.

Google took me to
https://packages.ubuntu.com/bionic/amd64/nvidia-cuda-dev/filelist which
includes dynlink_cuda.h at least.

https://packages.ubuntu.com/bionic/nvidia-cuda-dev might also be of
interest, I don't know which Ubuntu is relevant here, but e.g. I can
see that dynlink_cuda.h is not listed in
https://packages.ubuntu.com/trusty/amd64/nvidia-cuda-dev/filelist so at
least that one must be too old.

Ian.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

gary.buhrmaster at gmail

Jan 17, 2019, 9:25 AM

Post #10 of 12 (2463 views)

Permalink

On Thu, Jan 17, 2019 at 5:00 PM Ian Campbell <ijc@hellion.org.uk> wrote:
> I don't know which Ubuntu is relevant here

For development, only "current" and "LTS" are considered relevant
for Ubuntu, and "stable" for debian. "Current" and "Current-1"
for Fedora. ( from: https://www.mythtv.org/wiki/Coding_Standards ).

Support for other distros/OS's is apparently aspirational (and while
the build network certainly tests for other distros and variants, I
highly doubt that trying to support trusty for a future release is
going to gain any traction).

Obviously all of those OS versions are moving targets, and are
relevant only at MythTV release time (which could be around a
year or so from now unless the project moves to a "release early,
release often" approach).
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

ijc at hellion

Jan 17, 2019, 11:33 AM

Post #11 of 12 (2461 views)

Permalink

On Thu, 2019-01-17 at 17:25 +0000, Gary Buhrmaster wrote:
> On Thu, Jan 17, 2019 at 5:00 PM Ian Campbell <ijc@hellion.org.uk>
> wrote:
> > I don't know which Ubuntu is relevant here
>
> For development, only "current" and "LTS" are considered relevant
> for Ubuntu

Which according to https://wiki.ubuntu.com/Releases currently means
"Bionic Beaver" (LTS) and "Cosmic Cuttlefish" (current) so from
https://packages.ubuntu.com/bionic/amd64/nvidia-cuda-dev/filelist
https://packages.ubuntu.com/cosmic/amd64/nvidia-cuda-dev/filelist

it seems that they have what was asked for at the head of the thread.

> , and "stable" for debian

Which is
https://packages.debian.org/stretch/amd64/nvidia-cuda-dev/filelist
but that doesn't have the necessary stuff. But
https://packages.debian.org/stretch-backports/amd64/nvidia-cuda-dev/filelist
does so maybe that is workable, particularly by the time 31 happens
(since presence in stretch-backports means it is already in the dev
version of buster).

Ian.

_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

Re: Question about new build requirements for NVDEC [ In reply to ]

pb.mythtv at gmail

Jan 22, 2019, 8:49 AM

Post #12 of 12 (2429 views)

Permalink

On 1/17/19 4:10 AM, Ian Campbell wrote:
> On Wed, 2019-01-16 at 14:41 -0500, Peter Bennett wrote:
>> On 1/16/19 2:21 PM, Gary Buhrmaster wrote:
>>> On Tue, Jan 15, 2019 at 4:10 PM Peter Bennett <pb.mythtv@gmail.com>
>> wrote:
>>>> I could not find any Ubuntu package that contains these files that
>> could
>>>> be added as a dependency.
>>> For a one-time add, possibly the nv-codec-headers deb file from
>>> deb-multimedia? Since they are only headers (and config), there
>>> is some hope they might install and be usable in stretch and
>>> supported ubuntu variants even though deb-multimedia only has
>>> them formally available for sid/buster.
>>>
>> Thanks for the info. I did not realize it was there.
> In Debian there is:
> $ apt-file search dynlink_cuda.h
> nvidia-cuda-dev: /usr/include/dynlink_cuda.h
> $ apt policy nvidia-cuda-dev
> nvidia-cuda-dev:
> Installed: (none)
> Candidate: 9.1.85-8
> Version table:
> 9.2.148-5 500
> 500 http://mirror/debian sid/non-free amd64 Packages
> 9.1.85-8 990
> 990 http://mirror/debian buster/non-free amd64 Packages
> 8.0.44-4 500
> 500 http://mirror/debian stretch/non-free amd64 Packages
>
> However:
> $ apt show nvidia-cuda-dev | grep Section
> Section: non-free/libdevel
>
> So it is in non-free[0] (so not strictly part of Debian). It's possible
> that in Ubuntu it is also in whatever their non-free equivalent is?
>
> I think mythtv already requires other deps from non-free, so possibly
> using those is not a big deal.
>
> Ian.
>
> [0] Reasons in
> https://metadata.ftp-master.debian.org/changelogs/non-free/n/nvidia-cuda-toolkit/nvidia-cuda-toolkit_9.2.148-5_copyright
>
>
I checked and found that although nvidia-cuda-dev and
nvidia-cuda-toolkit include the dynlink header files, they are very
different from the ones FFmpeg uses, and FFmpeg has an extra header file.

At this stage I think it is best to add these to a directory in our
MythTV repository and work on getting it to build without needing root
access.

Peter
_______________________________________________
mythtv-dev mailing list
mythtv-dev@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-dev
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org