Mailing List Archive

Ryzen 4000 (Mobile) Softlocks/Micro-stutters
Hi All,

I'm currently using Xen 4.14 (Qubes 4.1 OS) on a Ryzen 7 4750U PRO, by default I'll experience softlocks where the mouse for example will jolt from time to time, in this state it's not usable.

Adding `dom0_max_vcpus=1 dom0_vcpus_pin` to Xen's CMDLINE results in no more jolting however performance isn't what it should be on an 8 core CPU, softlocks are still a problem within domU's, any sort of UI animation for example.

Reverting [this commit (8e2aa76dc1670e82eaa15683353853bc66bf54fc)](https://github.com/xen-project/xen/commit/8e2aa76dc1670e82eaa15683353853bc66bf54fc) results in even worse performance with or without the above changes to CMDLINE, and it's not usable at all.

Does anyone have any pointers?

Cheers
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
On 15.10.2020 02:38, Dylanger Daly wrote:
> I'm currently using Xen 4.14 (Qubes 4.1 OS) on a Ryzen 7 4750U PRO, by default I'll experience softlocks where the mouse for example will jolt from time to time, in this state it's not usable.

From what you say below I imply this is in Dom0?

> Adding `dom0_max_vcpus=1 dom0_vcpus_pin` to Xen's CMDLINE results in no more jolting however performance isn't what it should be on an 8 core CPU, softlocks are still a problem within domU's, any sort of UI animation for example.
>
> Reverting [this commit (8e2aa76dc1670e82eaa15683353853bc66bf54fc)](https://github.com/xen-project/xen/commit/8e2aa76dc1670e82eaa15683353853bc66bf54fc) results in even worse performance with or without the above changes to CMDLINE, and it's not usable at all.

You saying this surely has a reason, but making the connection would
help. I don't consider it surprising that a revert of an improvement
makes things worse. You having bothered to find a certain code change
also makes me suspect you've experimented with other scheduler
related settings - if so, please share all data you've got. (FAOD -
with the information provided I have no idea what to suggest, sorry.)

Jan
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
Hi Jan, thank you for responding.

Indeed this is for dom0, I only recently tried limiting a domU to 1 core and observed absolutely no softlocks, UI animations are smooth as butter with 1 core only.

Indeed I believe this is a CPU Scheduling issue, I've tried both the older credit and RTDS however both don't boot correctly.
The number of cores on this CPU is 8, 16 threads however Qubes by default disables SMT, sched_credit2_max_cpus_runqueue is 16 by default, I've tried testing with setting this to 7 or 8 however it'll either not boot, or nothing will change.

There are a number of credit2 tweak-ables so I'm hoping to play around and drop the `dom0_max_vcpus=1`, I suspect `sched_credit2_max_cpus_runqueue` is the main thing to play with.

I did manage to get it booting with sched_credit2_max_cpus_runqueue=7 but it ended up locking up shortly after X launched on dom0

??????? Original Message ???????

On Thursday, October 15th, 2020 at 7:18 PM, Jan Beulich <jbeulich@suse.com> wrote:

> On 15.10.2020 02:38, Dylanger Daly wrote:
>
> > I'm currently using Xen 4.14 (Qubes 4.1 OS) on a Ryzen 7 4750U PRO, by default I'll experience softlocks where the mouse for example will jolt from time to time, in this state it's not usable.
>
> From what you say below I imply this is in Dom0?
>
> > Adding `dom0_max_vcpus=1 dom0_vcpus_pin` to Xen's CMDLINE results in no more jolting however performance isn't what it should be on an 8 core CPU, softlocks are still a problem within domU's, any sort of UI animation for example.
> >
> > Reverting this commit (8e2aa76dc1670e82eaa15683353853bc66bf54fc) results in even worse performance with or without the above changes to CMDLINE, and it's not usable at all.
>
> You saying this surely has a reason, but making the connection would
>
> help. I don't consider it surprising that a revert of an improvement
>
> makes things worse. You having bothered to find a certain code change
>
> also makes me suspect you've experimented with other scheduler
>
> related settings - if so, please share all data you've got. (FAOD -
>
> with the information provided I have no idea what to suggest, sorry.)
>
> Jan
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
On 15.10.2020 11:14, Dylanger Daly wrote:
> Indeed this is for dom0, I only recently tried limiting a domU to 1 core and observed absolutely no softlocks, UI animations are smooth as butter with 1 core only.
>
> Indeed I believe this is a CPU Scheduling issue, I've tried both the older credit and RTDS however both don't boot correctly.

This wants reporting (with sufficient data, i.e. at least a serial log)
as separate issues.

> The number of cores on this CPU is 8, 16 threads however Qubes by default disables SMT, sched_credit2_max_cpus_runqueue is 16 by default, I've tried testing with setting this to 7 or 8 however it'll either not boot, or nothing will change.

Failure to boot, unless with insane command line options, should always
be reported to it can be fixed.

I'm afraid neither part of the reply gets you/us any closer to an
understanding of your softlockup issues. As a random thought, have you
tried disabling use of (deep) C-states? This is known to have helped
to work around errata on other hardware, so may be worth a try.

Jan
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
On 15/10/2020 01:38, Dylanger Daly wrote:
> Hi All,
>
> I'm currently using Xen 4.14 (Qubes 4.1 OS) on a Ryzen 7 4750U PRO, by
> default I'll experience softlocks where the mouse for example will
> jolt from time to time, in this state it's not usable.
>
> Adding `dom0_max_vcpus=1 dom0_vcpus_pin` to Xen's CMDLINE results in
> no more jolting however performance isn't what it should be on an 8
> core CPU, softlocks are still a problem within domU's, any sort of UI
> animation for example.
>
> Reverting this commit (8e2aa76dc1670e82eaa15683353853bc66bf54fc)
> <https://github.com/xen-project/xen/commit/8e2aa76dc1670e82eaa15683353853bc66bf54fc> results
> in even worse performance with or without the above changes to
> CMDLINE, and it's not usable at all.
>
> Does anyone have any pointers?

Does booting with sched=credit alter the symptoms?

~Andrew
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
> Does booting with sched=credit alter the symptoms?

Indeed I've tried this, the result is an observable delay, unusable performance, credit2 seems to be the only usable scheduler, I'm certain it has something to do with SMT being disabled, resulting in 8 cores instead of the expected 16 threads.

> As a random thought, have you tried disabling use of (deep) C-states?

Yeah I've tried both `processor.max_cstate=1|5`
I've also tried adding `0xC0010292` and `0xC0010296` MSRs into arch/x86/msr.c (guest_{rdmsr,wrmsr})

The above allowed me to use https://github.com/r4m0n/ZenStates-Linux/blob/master/zenstates.py

After removing `dom0_max_vcpus=1 dom0_vcpus_pin` from Xen's CMDLINE, and disabling C6 I observed no change.

> This wants reporting (with sufficient data, i.e. at least a serial log)

Hm, I'm not sure there's UART on this Laptop, can I save the boot log somewhere?
??????? Original Message ???????
On Thursday, October 15th, 2020 at 10:57 PM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:

> On 15/10/2020 01:38, Dylanger Daly wrote:
>
>> Hi All,
>>
>> I'm currently using Xen 4.14 (Qubes 4.1 OS) on a Ryzen 7 4750U PRO, by default I'll experience softlocks where the mouse for example will jolt from time to time, in this state it's not usable.
>>
>> Adding `dom0_max_vcpus=1 dom0_vcpus_pin` to Xen's CMDLINE results in no more jolting however performance isn't what it should be on an 8 core CPU, softlocks are still a problem within domU's, any sort of UI animation for example.
>>
>> Reverting [this commit (8e2aa76dc1670e82eaa15683353853bc66bf54fc)](https://github.com/xen-project/xen/commit/8e2aa76dc1670e82eaa15683353853bc66bf54fc) results in even worse performance with or without the above changes to CMDLINE, and it's not usable at all.
>>
>> Does anyone have any pointers?
>
> Does booting with sched=credit alter the symptoms?
>
> ~Andrew
Re: Ryzen 4000 (Mobile) Softlocks/Micro-stutters [ In reply to ]
On 20.10.2020 01:37, Dylanger Daly wrote:
>> This wants reporting (with sufficient data, i.e. at least a serial log)
>
> Hm, I'm not sure there's UART on this Laptop, can I save the boot log somewhere?

If the systems remains sufficiently usable "xl dmesg" will give you
the log. But you won't be able to get away without a serial-like
console (USB2 debug port may be an alternative, if you have a
suitable cable and if the USB topology in the laptop doesn't
prevent it functioning). Yes, laptops are always problematic in
this regard.

Jan