First, I want to caveat this with the declaration that I am not a
programmer however I do deal with process control as part of my day job as
an engineer.
I have been running the render branch (now master) for a while and using
AVSync2 for audio/video synchronization mostly with LiveTV. In evaluating,
I have found the sync algorithm would only provides acceptably stable video
performance if I use an AVSync2 timing setting of 1ms. Anything more than
this results in jerky video performance along with frequent frame drops on
my systems.
Investigating by spending some time studying the AVSync2 code and looking
in more detail at playback,timestamp logs, I have come to the conclusion
that there might be some potential for refinement of the AVSync2
algorithm.
The current algorithm is much like a proportional control approach
with fixed controller gain of 1 (with each adjustment, try to adjust the
time sync exactly back to equilibrium) with clamping of the maximum
fix_amount limited to the user defined setting of avsync2adjustms. I think
the problem with this approach is that a controller gain of 1 seems to be
too high and there is also occasionally some noise in the calculated sync
measurement. Both of these issues destabilize the AV sync control. With a
larger avsync2adjustms setting, the system never achieves stable control
under many conditions with the sync oscillating over / under target. In
turn, the calculated "lateness" value tends to also swing around causing
more frequent frame drops further upsetting the control.
In an attempt to improve, I manually modified the AVSync2 algorithm as
follows:
1. A control gain value is declared and included in the fix_amount
calculation to allow for a code configured gain value less than 1. In my
case, I am currently using a value of 0.4.
2. A first order exponential smoothing filter is applied to dampen any
noise in the sync measurement. A code configured filter coefficient is set
to a value between 0 and 1. In my case, I am using 0.9.
Note that the 0.4 and 0.9 are the result of some limited eyeball tuning and
are probably not optimum values.
After implementing these changes, my frontends seem to behave MUCH better
in both achieving sync quickly and achieving more stable long term control
of the av sync along with visually improved video smoothness. The modified
approach allows the applied fix_amount to be larger initially when there is
a big sync offset but avoids excessive overshoot / instability /
overcontrol as the sync approaches equilibrium.
I throw this out there to share and to see if others that have looked at
the AVSynce2 code have any thoughts on the approach at least.
In this scenario, the avsync2adjustms is no longer needed. However, if
wanting to have user configurable/tuneable settings, the gain value (
range: 0-1) and the filter coefficient (range: 0-1) could be exposed in the
frontend setup menu under AVSync2 rather than hard-coded.
Excerpted code with modifications shown in mythplayer.cpp AVSync2 function:
.....
auto playspeed1000 = static_cast<int64_t>(1000.0F / play_speed);
bool reset = false;
// controller gain
float av_control_gain = 0.4;
// variable to carry forward prior adjustment for smoothing filter
calculation
float last_fix = 0;
// filter coefficient for the smoothing filter.
float sync_fc = 0.9;
.......
int sign = audio_adjustment < 0 ? -1 : 1;
// Use proportional control with configured gain & add an
exponential smoothing filter.
float fix_amount = (last_fix * sync_fc + (1 - sync_fc) *
audio_adjustment) * sign * av_control_gain;
last_fix = fix_amount;
//disable clamping of adjustment outputs
//if (fix_amount > avsync2adjustms)
// fix_amount = avsync2adjustms;
// Faster catch-up when off by more than 200 ms
//if (audio_adjustment * sign > 200)
// fix the sync within 15 - 20 frames
//fix_amount = audio_adjustment * sign / 15;
auto speedup1000 = static_cast<int64_t>(1000 * play_speed);
rtcbase -= static_cast<int64_t>(1000000 * fix_amount * sign /
speedup1000);
......
-Tim
programmer however I do deal with process control as part of my day job as
an engineer.
I have been running the render branch (now master) for a while and using
AVSync2 for audio/video synchronization mostly with LiveTV. In evaluating,
I have found the sync algorithm would only provides acceptably stable video
performance if I use an AVSync2 timing setting of 1ms. Anything more than
this results in jerky video performance along with frequent frame drops on
my systems.
Investigating by spending some time studying the AVSync2 code and looking
in more detail at playback,timestamp logs, I have come to the conclusion
that there might be some potential for refinement of the AVSync2
algorithm.
The current algorithm is much like a proportional control approach
with fixed controller gain of 1 (with each adjustment, try to adjust the
time sync exactly back to equilibrium) with clamping of the maximum
fix_amount limited to the user defined setting of avsync2adjustms. I think
the problem with this approach is that a controller gain of 1 seems to be
too high and there is also occasionally some noise in the calculated sync
measurement. Both of these issues destabilize the AV sync control. With a
larger avsync2adjustms setting, the system never achieves stable control
under many conditions with the sync oscillating over / under target. In
turn, the calculated "lateness" value tends to also swing around causing
more frequent frame drops further upsetting the control.
In an attempt to improve, I manually modified the AVSync2 algorithm as
follows:
1. A control gain value is declared and included in the fix_amount
calculation to allow for a code configured gain value less than 1. In my
case, I am currently using a value of 0.4.
2. A first order exponential smoothing filter is applied to dampen any
noise in the sync measurement. A code configured filter coefficient is set
to a value between 0 and 1. In my case, I am using 0.9.
Note that the 0.4 and 0.9 are the result of some limited eyeball tuning and
are probably not optimum values.
After implementing these changes, my frontends seem to behave MUCH better
in both achieving sync quickly and achieving more stable long term control
of the av sync along with visually improved video smoothness. The modified
approach allows the applied fix_amount to be larger initially when there is
a big sync offset but avoids excessive overshoot / instability /
overcontrol as the sync approaches equilibrium.
I throw this out there to share and to see if others that have looked at
the AVSynce2 code have any thoughts on the approach at least.
In this scenario, the avsync2adjustms is no longer needed. However, if
wanting to have user configurable/tuneable settings, the gain value (
range: 0-1) and the filter coefficient (range: 0-1) could be exposed in the
frontend setup menu under AVSync2 rather than hard-coded.
Excerpted code with modifications shown in mythplayer.cpp AVSync2 function:
.....
auto playspeed1000 = static_cast<int64_t>(1000.0F / play_speed);
bool reset = false;
// controller gain
float av_control_gain = 0.4;
// variable to carry forward prior adjustment for smoothing filter
calculation
float last_fix = 0;
// filter coefficient for the smoothing filter.
float sync_fc = 0.9;
.......
int sign = audio_adjustment < 0 ? -1 : 1;
// Use proportional control with configured gain & add an
exponential smoothing filter.
float fix_amount = (last_fix * sync_fc + (1 - sync_fc) *
audio_adjustment) * sign * av_control_gain;
last_fix = fix_amount;
//disable clamping of adjustment outputs
//if (fix_amount > avsync2adjustms)
// fix_amount = avsync2adjustms;
// Faster catch-up when off by more than 200 ms
//if (audio_adjustment * sign > 200)
// fix the sync within 15 - 20 frames
//fix_amount = audio_adjustment * sign / 15;
auto speedup1000 = static_cast<int64_t>(1000 * play_speed);
rtcbase -= static_cast<int64_t>(1000000 * fix_amount * sign /
speedup1000);
......
-Tim
