Mailing List Archive

[PATCH v3 0/9] Arm cache coloring
Shared caches in multi-core CPU architectures represent a problem for
predictability of memory access latency. This jeopardizes applicability
of many Arm platform in real-time critical and mixed-criticality
scenarios. We introduce support for cache partitioning with page
coloring, a transparent software technique that enables isolation
between domains and Xen, and thus avoids cache interference.

When creating a domain, a simple syntax (e.g. `0-3` or `4-11`) allows
the user to define assignments of cache partitions ids, called colors,
where assigning different colors guarantees no mutual eviction on cache
will ever happen. This instructs the Xen memory allocator to provide
the i-th color assignee only with pages that maps to color i, i.e. that
are indexed in the i-th cache partition.

The proposed implementation supports the dom0less feature.
The proposed implementation doesn't support the static-mem feature.
The solution has been tested in several scenarios, including Xilinx Zynq
MPSoCs.

Commits structure:
- [1-3] Coloring initialization, cache layout auto-probing and coloring
data for domains.
- [4-5] xl and Device Tree support for coloring.
- [6] New page allocator for domain memory that implement the cache
coloring mechanism.
- [7-9] Coloring support for Xen.

Global changes in v3:
- fixed a compilation error because of a forgotten "\"
- replaced some #ifdef with if ( IS_ENABLED )
- other minor changes (docs, typos, variable types, style, etc.)
- better acknowledged Luca Miccio as the original author
- removed #8 since the bootmodule address and size can be replaced without
the need of this particular revert
- removed #9 since it wasn't a clean revert and thanks to Julien things can
be done in a smarter way sticking with map_pages_to_xen() (see new #9)

Open points:
- The allocator proposed in #6 works only with order-0 pages and inserts
them in a sorted list using a linear search. This behavior can be slow if
large amount of memory is given to it, so the user is warned in the
documentation for that.
In a following patch, that I'm going to send separately, a simple buddy
allocator that indexes pages by color is presented. It can serve higher
order pages and doesn't need the linear search. Unfortunately, it has
some flaws that I will discuss there.
- I will address the latest v2 comments from Julien in v4

Acknowledgements
----------------

This work is sponsored by Xilinx Inc., and supported by University of
Modena and Reggio Emilia and Minerva Systems.

Carlo Nonato (6):
xen/arm: add cache coloring initialization
xen/arm: add cache coloring initialization for domains
tools/xl: add support for cache coloring configuration
xen/arm: add support for cache coloring configuration via device-tree
Revert "xen/arm: Remove unused BOOT_RELOC_VIRT_START"
xen/arm: add cache coloring support for Xen

Luca Miccio (3):
xen/arm: dump cache colors in domain info debug-key
xen/common: add cache coloring allocator for domains
xen/arm: add Xen cache colors command line parameter

docs/man/xl.cfg.5.pod.in | 10 +
docs/misc/arm/cache-coloring.rst | 229 ++++++++++++++
docs/misc/arm/device-tree/booting.txt | 4 +
docs/misc/xen-command-line.pandoc | 49 +++
tools/libs/light/libxl_create.c | 12 +
tools/libs/light/libxl_types.idl | 1 +
tools/xl/xl_parse.c | 52 +++-
xen/arch/arm/Kconfig | 34 +++
xen/arch/arm/Makefile | 1 +
xen/arch/arm/alternative.c | 9 +-
xen/arch/arm/arm64/head.S | 48 +++
xen/arch/arm/coloring.c | 411 ++++++++++++++++++++++++++
xen/arch/arm/domain.c | 9 +
xen/arch/arm/domain_build.c | 26 +-
xen/arch/arm/include/asm/coloring.h | 91 ++++++
xen/arch/arm/include/asm/config.h | 4 +-
xen/arch/arm/include/asm/domain.h | 4 +
xen/arch/arm/include/asm/mm.h | 19 +-
xen/arch/arm/include/asm/processor.h | 16 +
xen/arch/arm/mm.c | 95 +++++-
xen/arch/arm/p2m.c | 7 +-
xen/arch/arm/psci.c | 4 +-
xen/arch/arm/setup.c | 81 ++++-
xen/arch/arm/smpboot.c | 3 +-
xen/arch/arm/xen.lds.S | 2 +-
xen/common/Kconfig | 3 +
xen/common/page_alloc.c | 259 ++++++++++++++--
xen/include/public/arch-arm.h | 8 +
xen/include/xen/mm.h | 43 +++
29 files changed, 1487 insertions(+), 47 deletions(-)
create mode 100644 docs/misc/arm/cache-coloring.rst
create mode 100644 xen/arch/arm/coloring.c
create mode 100644 xen/arch/arm/include/asm/coloring.h

--
2.34.1
Re: [PATCH v3 0/9] Arm cache coloring [ In reply to ]
Hi Carlo,

Just in case you are waiting on my review before sending a new version.
Given that...

On 22/10/2022 16:51, Carlo Nonato wrote:
> Shared caches in multi-core CPU architectures represent a problem for
> predictability of memory access latency. This jeopardizes applicability
> of many Arm platform in real-time critical and mixed-criticality
> scenarios. We introduce support for cache partitioning with page
> coloring, a transparent software technique that enables isolation
> between domains and Xen, and thus avoids cache interference.
>
> When creating a domain, a simple syntax (e.g. `0-3` or `4-11`) allows
> the user to define assignments of cache partitions ids, called colors,
> where assigning different colors guarantees no mutual eviction on cache
> will ever happen. This instructs the Xen memory allocator to provide
> the i-th color assignee only with pages that maps to color i, i.e. that
> are indexed in the i-th cache partition.
>
> The proposed implementation supports the dom0less feature.
> The proposed implementation doesn't support the static-mem feature.
> The solution has been tested in several scenarios, including Xilinx Zynq
> MPSoCs.
>
> Commits structure:
> - [1-3] Coloring initialization, cache layout auto-probing and coloring
> data for domains.
> - [4-5] xl and Device Tree support for coloring.
> - [6] New page allocator for domain memory that implement the cache
> coloring mechanism.
> - [7-9] Coloring support for Xen.
>
> Global changes in v3:
> - fixed a compilation error because of a forgotten "\"
> - replaced some #ifdef with if ( IS_ENABLED )
> - other minor changes (docs, typos, variable types, style, etc.)
> - better acknowledged Luca Miccio as the original author
> - removed #8 since the bootmodule address and size can be replaced without
> the need of this particular revert
> - removed #9 since it wasn't a clean revert and thanks to Julien things can
> be done in a smarter way sticking with map_pages_to_xen() (see new #9)
>
> Open points:
> - The allocator proposed in #6 works only with order-0 pages and inserts
> them in a sorted list using a linear search. This behavior can be slow if
> large amount of memory is given to it, so the user is warned in the
> documentation for that.
> In a following patch, that I'm going to send separately, a simple buddy
> allocator that indexes pages by color is presented. It can serve higher
> order pages and doesn't need the linear search. Unfortunately, it has
> some flaws that I will discuss there.
> - I will address the latest v2 comments from Julien in v4

... some my comments have not been handled, I am not planning to review
this version. If there is small parts you want me to have a look before
you send a new version.

Cheers,

--
Julien Grall