Mailing List Archive

Ticket #13614: Python Bindings fails on Recorded.update() during silence detection
#13614: Python Bindings fails on Recorded.update() during silence detection
-----------------------------------+-------------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: needs_triage
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Keywords: silence detection
Ticket locked: 0 |
-----------------------------------+-------------------------------
A user on the forum reported a traceback in the python bindings (1)
when running the detection for silence according the wiki page
"Commercial detection with silences" (2).

(1) https://forum.mythtv.org/viewtopic.php?f=36&t=3763

(2) https://www.mythtv.org/wiki/Commercial_detection_with_silences

This affects Ubuntu 20.04 for
- Python 3.8
- MySQLdb 1.4.4
- MariaDB 10.3

I had not any errors on Ubuntu 19.10 for
- Python 3.7
- MySQLdb 1.3.10
- MariaDB 10.3

The following traceback occurs when running silence.py with chanid
and starttime as paramters:

{{{
Traceback (most recent call last):
File "/home/local-admin/MythTV/silence/MythTV/_conn_mysqldb.py", line
98, in executemany
return super(LoggedCursor, self).executemany(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 235, in
executemany
return self._do_execute_many(q_prefix, q_values, q_postfix, args,
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 264, in
_do_execute_many
rows += self.execute(sql + postfix)
File "/home/local-admin/MythTV/silence/MythTV/_conn_mysqldb.py", line
68, in execute
query = self._sanitize(query)
File "/home/local-admin/MythTV/silence/MythTV/_conn_mysqldb.py", line
44, in _sanitize
def _sanitize(self, query): return query.replace('?', '%s')
TypeError: a bytes-like object is required, not 'str'
}}}

The offended call to 'MySQLdb.curser.execute' was
{{{
bytearray(b"INSERT INTO recordedmarkup (chanid,starttime,mark,type,data)
VALUES(10111,\'2020-02-09 16:10:00\',74367,4,NULL),(10111,\'2020-02-09
16:10:00\',81984,5,NULL)")
}}}

Note:
This is a callback from MySQLdb as described in a similar issue #13123.
MySQLdb resolves a call to 'executemany' to multiple calls of 'execute',
but as bytearray and not as string.


To reproduce this issue, one needs to patch the silence.py from the wiki
(2)
with the attached patch for python3 ('silence_py_patch_01.patch') and
install
it according the wiki page.

Then run 'silence.py -- chanid YYYY --starttime YYY' with correct
parameters.

To solve this issue, apply the path '0001_sanitize_mysql_queries.patch'
and, of course, the patch for 'silence.py'.

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+------------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: needs_triage
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution:
Keywords: silence detection | Ticket locked: 0
-------------------------------+------------------------------
Changes (by rcrdnalor):

* Attachment "silence_py_patch_01.patch" added.

Patch for a python3 version of silence.py on the wiki

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+------------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: needs_triage
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution:
Keywords: silence detection | Ticket locked: 0
-------------------------------+------------------------------
Changes (by rcrdnalor):

* Attachment "0001_sanitize_mysql_queries.patch" added.

Patch to fix Python Bindings for MySQLDB calls of executemany

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+------------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: needs_triage
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution:
Keywords: silence detection | Ticket locked: 0
-------------------------------+------------------------------
Changes (by rcrdnalor):

* Attachment "silence_py_patch_02.patch" added.

Patch for silece.py tested with python3.6 and python3.8

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+------------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: needs_triage
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution:
Keywords: silence detection | Ticket locked: 0
-------------------------------+------------------------------

Comment (by rcrdnalor):

Patches tested on
{{{
Ubuntu 18.04 python2 master
python: 2.7.17
python-MySQLdb: 1.3.10

Ubuntu 18.04 python3 master
python3: 3.6.9
python3-MySQLdb: 1.3.10

Ubuntu 20.04 python3 fixes/31
python3: 3.8.2
python3-MySQLdb: 1.4.4
}}}

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614#comment:1>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+---------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: assigned
Priority: minor | Milestone: 31.1
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution:
Keywords: silence detection | Ticket locked: 0
-------------------------------+---------------------------
Changes (by rcrdnalor):

* milestone: needs_triage => 31.1


--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614#comment:2>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+---------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: closed
Priority: minor | Milestone: 31.1
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution: fixed
Keywords: silence detection | Ticket locked: 0
-------------------------------+---------------------------
Changes (by Roland Ernst <rcrernst@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"b2e9c6a44233570704554894bf45e01bfa8e26a7/mythtv"
b2e9c6a442/mythtv]:
{{{
#!CommitTicketReference repository="mythtv"
revision="b2e9c6a44233570704554894bf45e01bfa8e26a7"
Fix mysql cursor class to handle bytearrays

Newer Python MySQLdb modules call 'cursor.execute()' multiple times
from 'cursor.executemany()'.
With python3 and python3-MySQLdb > 1.4.0 these call-backs containing
a query are bytearrays, resulting in a traceback in the '_sanitize'
method.

Note: This '_sanitize' method is only needed when creating a query within
the Python Bindings, but not necessary when python-mysqldb itself calls
the
cursor.execute() method.

Fixes #13614
}}}

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614#comment:3>
MythTV <http://www.mythtv.org>
MythTV Media Center
Re: Ticket #13614: Python Bindings fails on Recorded.update() during silence detection [ In reply to ]
#13614: Python Bindings fails on Recorded.update() during silence detection
-------------------------------+---------------------------
Reporter: rcrdnalor | Owner: rcrdnalor
Type: Patch - Bug Fix | Status: closed
Priority: minor | Milestone: 31.1
Component: Bindings - Python | Version: v31-fixes
Severity: medium | Resolution: fixed
Keywords: silence detection | Ticket locked: 0
-------------------------------+---------------------------

Comment (by Roland Ernst <rcrernst@…>):

In [changeset:"ce23a0225fcec2afbdfe5a7e82170e28f406c830/mythtv"
ce23a0225/mythtv]:
{{{
#!CommitTicketReference repository="mythtv"
revision="ce23a0225fcec2afbdfe5a7e82170e28f406c830"
Fix mysql cursor class to handle bytearrays

Newer Python MySQLdb modules call 'cursor.execute()' multiple times
from 'cursor.executemany()'.
With python3 and python3-MySQLdb > 1.4.0 these call-backs containing
a query are bytearrays, resulting in a traceback in the '_sanitize'
method.

Note: This '_sanitize' method is only needed when creating a query within
the Python Bindings, but not necessary when python-mysqldb itself calls
the
cursor.execute() method.

Fixes #13614

(cherry picked from commit b2e9c6a44233570704554894bf45e01bfa8e26a7)
}}}

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13614#comment:4>
MythTV <http://www.mythtv.org>
MythTV Media Center