Mailing List Archive

>
> Thanks again for your help! Hopefully this troubleshooting exercise
>> ultimately results in the ability to build the app with all dependencies
>> included-
>>
>
Steve, thanks for the update - this confirms that the python install is the
culprit (thank you to Bill as well as I never would have stumbled on this
without you suggesting the library paths).

Still banging my head against the python deployment problem and have run
out of ideas. It may just be a reality that users who want working python
bindings may have to install the necessary support packages via macports.
This is what happens in Ubuntu (i.e. apt installs all of the appropriate
python goodies), I was just hoping to get them inserted and running in the
application somehow...

> On Jul 19, 2020, at 4:45 PM, John Hoyt <john.hoyt@gmail.com> wrote:
>
> Thanks again for your help! Hopefully this troubleshooting exercise ultimately results in the ability to build the app with all dependencies included-
>
> Steve, thanks for the update - this confirms that the python install is the culprit (thank you to Bill as well as I never would have stumbled on this without you suggesting the library paths).
>
> Still banging my head against the python deployment problem and have run out of ideas. It may just be a reality that users who want working python bindings may have to install the necessary support packages via macports. This is what happens in Ubuntu (i.e. apt installs all of the appropriate python goodies), I was just hoping to get them inserted and running in the application somehow…


I’m not really familiar with how it the python bindings should work in the bundled app*…but around 2014, one of the devs modified the official packaging script to make them functional. Perhaps you can glean something from his commit:

https://github.com/MythTV/packaging/commit/e44052360ea268d0cf927897140e6e06134391ae#diff-6d1353539e2de6c424680420280cc319

Craig
*This was one of the key reasons I only pursued installing Myth in the ‘linux’ fashion. Prior to the above commit, I didn’t think it was possible to make the bindings work.

On 7/19/20 3:45 PM, John Hoyt wrote:
> Thanks again for your help!  Hopefully this troubleshooting exercise ultimately results in the ability to build the app with all
> dependencies included-
>
>
> Steve, thanks for the update - this confirms that the python install is the culprit (thank you to Bill as well as I never would have stumbled on
> this without you suggesting the library paths).
>
> Still banging my head against the python deployment problem and have run out of ideas.  It may just be a reality that users who want working
> python bindings may have to install the necessary support packages via macports.  This is what happens in Ubuntu (i.e. apt installs all of the
> appropriate python goodies), I was just hoping to get them inserted and running in the application somehow...

Fish out of water here, but the Ansible task:

- name: install packages
become: yes
become_user: root
become_method: sudo
macports:
name:
'{{ lookup("flattened", macports_pkg_list) }}'
update_cache: yes

uses the macports command to install the packages. I don't know why they're not
going to the proper place.

You could run ansible with -vvv and look through (the very large) output
to see it it makes sense.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

>
> uses the macports command to install the packages. I don't know why
> they're not
> going to the proper place.
>
> You could run ansible with -vvv and look through (the very large) output
> to see it it makes sense.
>

Bill - they are going to the correct place and macports/ansible is working
correctly. Anyone who installs the python packages either manually or with
ansible seems to have a fully working setup. The issue is trying to get a
fully bundled application where anyone can download it and run the
application without the need to install anything (beyond maybe just
python38) from macports.

To that end, I've got everything working and bundled into the application
except for the python executable and the python support packages (e.g.
py39-mysqlclient, py38-lxml). The MyhTV python bundles builds and installs
correctly.

Craig - thanks for the pointer. I'll take another pass at that build
script (and more specifically that check in). There's some magic with
install_name_tool and the osx_bundler.pl script that I'm missing and
hopefully those will give me the proper insights. The interesting thing to
note is that when I install python38 (and the extra packages) into the
application as a framework (with all support libraries), the application
stops seeing the MythTV python package even though it is present and pathed
correctly (I tested the path by modifying one of the metadata grabbers to
dump the sys.path to a file).

Thanks for the pointers, I'll keep hacking at it.

>
> Craig - thanks for the pointer. I'll take another pass at that build
> script (and more specifically that check in). There's some magic with
> install_name_tool and the osx_bundler.pl script that I'm missing and
> hopefully those will give me the proper insights. The interesting thing to
> note is that when I install python38 (and the extra packages) into the
> application as a framework (with all support libraries), the application
> stops seeing the MythTV python package even though it is present and pathed
> correctly (I tested the path by modifying one of the metadata grabbers to
> dump the sys.path to a file).
>

Craig - it looks like the osx_bunder.pl script downloads and compiles the
python packages as static libraries - this is why they were able to be
shipped with the application without modification. MacPorts compiles
the necessary python parts (mysqlclient, lxml, curl) with dynamic
libraries (point back to /opt/local/lib), so any *.so files copied into the
application need to have their support .so or .dylib files copied into the
app's framework and re-linked to the app's internal Framework and not
/opt/local/lib.

osx_bundler.pl does a lot of the necessary magic copying the dynamically
linked libraries into the application Framework directory and even updates
the link internal to the .so files with "install_name_tool" to a new path
relative to the executable (example:
"@executable_path/../Frameworks//mariadb.framework//mariadb") - BUT the
application can't seem to follow the @executable_path links.

If I update all of the "@executable_path" links in app's
Resources/lib/python3.8/sites-available .so files with install_name_tool to
non-relative (i.e. hard code) paths pointing to the libraries copied into
the Framework by osx-bundler.pl everyhting works. Unfortunately since
these are now hard linked to my filesystem, the app will break if I move it
to any new directory.

I feel like I'm a relative path away from wrapping this up - just can't
seem to stumble on the right path for the requesting application
(mythmetadata).

>
> I feel like I'm a relative path away from wrapping this up - just can't
> seem to stumble on the right path for the requesting application
> (mythmetadata).
>

I believe the latest versions of MythFrontend.app that I just uploaded to
SourceForge (https://sourceforge.net/projects/mythtvformacosx/) fix the
python and metadata grabber issues.

Getting python portable was quite a bit more involved than I originally
thought (been hacking at this for the past week - lots of dead ends...).
The new builds now include a portable version of python3. The trick to
getting this to work was using py2app to build a portable version of one of
the metadata grabbers (ttvdb.pl seems to work best as it has the most
dependencies).

I also had to steal a play from the mythbuntu folks where the application's
executable is now an sh script that sets some local variables (mainly
PYTHONHOME and PYTHONPATH) then calls the real executable now named
"mythfrontend.real".

I've tested that the new applications work on a clean (i.e. no macports no
python3) High Sierra VM, so hopefully no one comes across any new python
related issues :)

BTW - Are there any devs out there kind enough to take on my Trac ticket
getting the compile script into the packaging baseline (
https://code.mythtv.org/trac/ticket/13642)? It could certainly use a
review and probably some verified testing by other users - BUT - it seems
to be working very well for me and I've put a substantial amount of hours
into it at this point...

>

On Sat, Jul 25, 2020 at 7:37 AM John Hoyt <john.hoyt@gmail.com> wrote:

> I feel like I'm a relative path away from wrapping this up - just can't
>> seem to stumble on the right path for the requesting application
>> (mythmetadata).
>>
>
> I believe the latest versions of MythFrontend.app that I just uploaded to
> SourceForge (https://sourceforge.net/projects/mythtvformacosx/) fix the
> python and metadata grabber issues.
>
> Getting python portable was quite a bit more involved than I originally
> thought (been hacking at this for the past week - lots of dead ends...).
> The new builds now include a portable version of python3. The trick to
> getting this to work was using py2app to build a portable version of one of
> the metadata grabbers (ttvdb.pl seems to work best as it has the most
> dependencies).
>
> I also had to steal a play from the mythbuntu folks where the
> application's executable is now an sh script that sets some local variables
> (mainly PYTHONHOME and PYTHONPATH) then calls the real executable now named
> "mythfrontend.real".
>
> I've tested that the new applications work on a clean (i.e. no macports no
> python3) High Sierra VM, so hopefully no one comes across any new python
> related issues :)
>
> BTW - Are there any devs out there kind enough to take on my Trac ticket
> getting the compile script into the packaging baseline (
> https://code.mythtv.org/trac/ticket/13642)? It could certainly use a
> review and probably some verified testing by other users - BUT - it seems
> to be working very well for me and I've put a substantial amount of hours
> into it at this point...
>
>> _______________________________________________
> mythtv-users mailing list
> mythtv-users@mythtv.org
> http://lists.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org
>

Hi John,

I tested this build today on my High Sierra frontend (no macports) and it
could successfully use the metadata grabbers. Nice work!

I'm still trying to understand why on the High Sierra frontend, regardless
of which build I use, I can't get rid of the Mac OS menu bar at the top of
the screen, either for the gui or playback. I've been through every
setting in Appearance.
Am I correct in assuming that only the files in the "Catalina" folder were
built with 10.15, and everything else was built with 10.13?

Steve
--
________________________
"destroying the fundamental underpinnings of the broadcast television
ecosystem"

>
> I'm still trying to understand why on the High Sierra frontend, regardless
> of which build I use, I can't get rid of the Mac OS menu bar at the top of
> the screen, either for the gui or playback. I've been through every
> setting in Appearance.
>

I noticed that too during testing. I did notice that if I launch the app
and wait (i.e. don't change to anything else, just double click and wait
the LONG wait) the menu bar goes away. No clue how to fix it - it would
require some deeper knowledge of how mythtv renders and tells the OS that
the app goes full screen. If it really bothers you, I'd suggest opening a
ticket in trac (https://code.mythtv.org/trac).

On Catalina, none of these issues occur.


> Am I correct in assuming that only the files in the "Catalina" folder were
> built with 10.15, and everything else was built with 10.13?
>

I tried to annotate the sourceforge readme.md to help clear this up -
anything in the top level are built with High Sierra, anything in the
Catalina folder are built on Catalina (unless I screw up the upload which
can happen :) ).

Additionally, if you download a file and want to make sure you have the
right version, the compile script automatically puts the OS version in the
.dmg filename - so anything with a "10.13" in the filename was compiled on
High Sierra, anything compiles on Catalina has a 10.15 in the filename.