From patchwork Thu May 2 16:55:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651894 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BBD07C25B76 for ; Thu, 2 May 2024 16:56:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715947.1117934 (Exim 4.92) (envelope-from ) id 1s2Zih-0006cv-Ax; Thu, 02 May 2024 16:55:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715947.1117934; Thu, 02 May 2024 16:55:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zih-0006cS-7r; Thu, 02 May 2024 16:55:43 +0000 Received: by outflank-mailman (input) for mailman id 715947; Thu, 02 May 2024 16:55:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zif-0006N3-92 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cd95efbb-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:39 +0200 (CEST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a56d7d457a1so1078769566b.1 for ; Thu, 02 May 2024 09:55:39 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cd95efbb-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668938; x=1715273738; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7GREHaaEsQD6UfemfISve1bYOvLW8rS5iYyct68ZwSw=; b=EqkbhmjnI5QoSSieWG72gWd8+4WMPVc+PQY79pbvAjCKVb6Cc2krGEkzrqTZt15UJ7 D/aaGH+7NEZzmVqH1z3BrB0rHKdZlXBlWryDohpTShvVsCBXanIN+c++GDzNGjR8QJlP n64GzWgrorq8uEyWT32Hi3bwQN+OKwW2ARDTvzn8oBZUqAUpQO6jtFjKbZ7K1d1lGT8P UwkdWiDVxFgcoTlgj3Mi4TR/rp9KuImRqyBgYUDFkeOGy0rJct8agmydMDG1ZSQh655S dDzafiqspXMKc91FI3GAuceW3dzWbfuvz6Q1XDZu7QykuDPw20C2CTUHUmeklbmv2lm9 vldQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668938; x=1715273738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7GREHaaEsQD6UfemfISve1bYOvLW8rS5iYyct68ZwSw=; b=Bjj3gM3UJnpkQ+q1sIc9h11aTWHD+becgTq4cvBHcoID0x9Kxb7hjy4/4ht6etqpZn qaV1AXIaYIAo51rPCdDeU3ghSJoLyXZ19YLwXm5GhzoTwrXXSeUfzdjk3Tuw2mEgyd1V BHjtQ1m03CxjG0Of5H5uZP2IZwPaA8tUpW0Xs1hMhnsZTMUN7bFkbM8Yuey+/hpX3dVW mi8ZqSnukSgpi5VemtEb/u3UjY6gCsVtW7PAKf2pMpSF6wnpCbwMrYxQNzCe40x6htNj uWSh1LoCjC9Xgr2DvDYdJa2MVzh8PieySuEvPvjozmcdAuRn5bPV3C9wk94lpsvZ5oYc 6CJg== X-Gm-Message-State: AOJu0Yzkftn3aKF60QzL+a0DpOjAB9fR1aVoRxluDd54KOfbo5Yb6j9l eYqvdsNgroWWdgrBwYcKzK4rGpZ3rLLXJOL4kr7aRQ+4g4ImBLZdJYfcyq/BAd12HF6UEO1IaA7 E X-Google-Smtp-Source: AGHT+IGWDc2DdScTkoTEys0xZwfnpda0hlZJeAncLA0A4bzwwXZSpr1rHSW9ZFbwc0DxtB10fwE7BA== X-Received: by 2002:a17:906:c048:b0:a59:146:1c4e with SMTP id bm8-20020a170906c04800b00a5901461c4emr53013ejb.58.1714668938113; Thu, 02 May 2024 09:55:38 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Marco Solieri Subject: [PATCH v8 01/13] xen/common: add cache coloring common code Date: Thu, 2 May 2024 18:55:21 +0200 Message-Id: <20240502165533.319988-2-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Last Level Cache (LLC) coloring allows to partition the cache in smaller chunks called cache colors. Since not all architectures can actually implement it, add a HAS_LLC_COLORING Kconfig option. MAX_LLC_COLORS_ORDER Kconfig option has a range maximum of 10 (2^10 = 1024) because that's the number of colors that fit in a 4 KiB page when integers are 4 bytes long. LLC colors are a property of the domain, so struct domain has to be extended. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- v8: - minor documentation fixes - "llc-coloring=on" is inferred from "llc-nr-ways" and "llc-size" usage - turned CONFIG_NR_LLC_COLORS to CONFIG_MAX_LLC_COLORS_ORDER, base-2 exponent - moved Kconfig options to common/Kconfig - don't crash if computed max_nr_colors is too large v7: - SUPPORT.md changes added to this patch - extended documentation to better address applicability of cache coloring - "llc-nr-ways" and "llc-size" params introduced in favor of "llc-way-size" - moved dump_llc_coloring_info() call in 'm' keyhandler (pagealloc_info()) v6: - moved almost all code in common - moved documentation in this patch - reintroduced range for CONFIG_NR_LLC_COLORS - reintroduced some stub functions to reduce the number of checks on llc_coloring_enabled - moved domain_llc_coloring_free() in same patch where allocation happens - turned "d->llc_colors" to pointer-to-const - llc_coloring_init() now returns void and panics if errors are found v5: - used - instead of _ for filenames - removed domain_create_llc_colored() - removed stub functions - coloring domain fields are now #ifdef protected v4: - Kconfig options moved to xen/arch - removed range for CONFIG_NR_LLC_COLORS - added "llc_coloring_enabled" global to later implement the boot-time switch - added domain_create_llc_colored() to be able to pass colors - added is_domain_llc_colored() macro --- SUPPORT.md | 7 ++ docs/misc/cache-coloring.rst | 116 ++++++++++++++++++++++++++++++ docs/misc/xen-command-line.pandoc | 37 ++++++++++ xen/common/Kconfig | 22 ++++++ xen/common/Makefile | 1 + xen/common/keyhandler.c | 3 + xen/common/llc-coloring.c | 111 ++++++++++++++++++++++++++++ xen/common/page_alloc.c | 3 + xen/include/xen/llc-coloring.h | 36 ++++++++++ xen/include/xen/sched.h | 5 ++ 10 files changed, 341 insertions(+) create mode 100644 docs/misc/cache-coloring.rst create mode 100644 xen/common/llc-coloring.c create mode 100644 xen/include/xen/llc-coloring.h diff --git a/SUPPORT.md b/SUPPORT.md index e10d46d924..94048adef2 100644 --- a/SUPPORT.md +++ b/SUPPORT.md @@ -381,6 +381,13 @@ by maintaining multiple physical to machine (p2m) memory mappings. Status, x86 HVM: Tech Preview Status, ARM: Tech Preview +### Cache coloring + +Allows to reserve Last Level Cache (LLC) partitions for Dom0, DomUs and Xen +itself. + + Status, Arm64: Experimental + ## Resource Management ### CPU Pools diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst new file mode 100644 index 0000000000..bcb45e9344 --- /dev/null +++ b/docs/misc/cache-coloring.rst @@ -0,0 +1,116 @@ +Xen cache coloring user guide +============================= + +The cache coloring support in Xen allows to reserve Last Level Cache (LLC) +partitions for Dom0, DomUs and Xen itself. Currently only ARM64 is supported. +Cache coloring realizes per-set cache partitioning in software and is applicable +to shared LLCs as implemented in Cortex-A53, Cortex-A72 and similar CPUs. + +To compile LLC coloring support set ``CONFIG_LLC_COLORING=y``. + +If needed, change the maximum number of colors with +``CONFIG_MAX_LLC_COLORS_ORDER=``. + +Runtime configuration is done via `Command line parameters`_. + +Background +********** + +Cache hierarchy of a modern multi-core CPU typically has first levels dedicated +to each core (hence using multiple cache units), while the last level is shared +among all of them. Such configuration implies that memory operations on one +core (e.g. running a DomU) are able to generate interference on another core +(e.g. hosting another DomU). Cache coloring realizes per-set cache-partitioning +in software and mitigates this, guaranteeing more predictable performances for +memory accesses. +Software-based cache coloring is particularly useful in those situations where +no hardware mechanisms (e.g., DSU-based way partitioning) are available to +partition caches. This is the case for e.g., Cortex-A53, A57 and A72 CPUs that +feature a L2 LLC cache shared among all cores. + +The key concept underlying cache coloring is a fragmentation of the memory +space into a set of sub-spaces called colors that are mapped to disjoint cache +partitions. Technically, the whole memory space is first divided into a number +of subsequent regions. Then each region is in turn divided into a number of +subsequent sub-colors. The generic i-th color is then obtained by all the +i-th sub-colors in each region. + +:: + + Region j Region j+1 + ..................... ............ + . . . + . . + _ _ _______________ _ _____________________ _ _ + | | | | | | | + | c_0 | c_1 | | c_n | c_0 | c_1 | + _ _ _|_____|_____|_ _ _|_____|_____|_____|_ _ _ + : : + : :... ... . + : color 0 + :........................... ... . + : + . . ..................................: + +How colors are actually defined depends on the function that maps memory to +cache lines. In case of physically-indexed, physically-tagged caches with linear +mapping, the set index is found by extracting some contiguous bits from the +physical address. This allows colors to be defined as shown in figure: they +appear in memory as subsequent blocks of equal size and repeats themselves after +``n`` different colors, where ``n`` is the total number of colors. + +If some kind of bit shuffling appears in the mapping function, then colors +assume a different layout in memory. Those kind of caches aren't supported by +the current implementation. + +**Note**: Finding the exact cache mapping function can be a really difficult +task since it's not always documented in the CPU manual. As said Cortex-A53, A57 +and A72 are known to work with the current implementation. + +How to compute the number of colors +################################### + +Given the linear mapping from physical memory to cache lines for granted, the +number of available colors for a specific platform is computed using three +parameters: + +- the size of the LLC. +- the number of the LLC ways. +- the page size used by Xen. + +The first two parameters can be found in the processor manual, while the third +one is the minimum mapping granularity. Dividing the cache size by the number of +its ways we obtain the size of a way. Dividing this number by the page size, +the number of total cache colors is found. So for example an Arm Cortex-A53 +with a 16-ways associative 1 MiB LLC can isolate up to 16 colors when pages are +4 KiB in size. + +Effective colors assignment +########################### + +When assigning colors, if one wants to avoid cache interference between two +domains, different colors needs to be used for their memory. + +Command line parameters +*********************** + +Specific documentation is available at `docs/misc/xen-command-line.pandoc`. + ++----------------------+-------------------------------+ +| **Parameter** | **Description** | ++----------------------+-------------------------------+ +| ``llc-coloring`` | Enable coloring at runtime | ++----------------------+-------------------------------+ +| ``llc-size`` | Set the LLC size | ++----------------------+-------------------------------+ +| ``llc-nr-ways`` | Set the LLC number of ways | ++----------------------+-------------------------------+ + +Auto-probing of LLC specs +######################### + +LLC size and number of ways are probed automatically by default. + +LLC specs can be manually set via the above command line parameters. This +bypasses any auto-probing and it's used to overcome failing situations, such as +flawed probing logic, or for debugging/testing purposes. diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index e760f3266e..906822a491 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1706,6 +1706,43 @@ This option is intended for debugging purposes only. Enable MSR_DEBUGCTL.LBR in hypervisor context to be able to dump the Last Interrupt/Exception To/From record with other registers. +### llc-coloring +> `= ` + +> Default: `false` + +Flag to enable or disable LLC coloring support at runtime. This option is +available only when `CONFIG_LLC_COLORING` is enabled. See the general +cache coloring documentation for more info. + +### llc-nr-ways +> `= ` + +> Default: `Obtained from hardware` + +Specify the number of ways of the Last Level Cache. This option is available +only when `CONFIG_LLC_COLORING` is enabled. LLC size and number of ways are used +to find the number of supported cache colors. By default the value is +automatically computed by probing the hardware, but in case of specific needs, +it can be manually set. Those include failing probing and debugging/testing +purposes so that it's possible to emulate platforms with different number of +supported colors. If set, also "llc-size" must be set, otherwise the default +will be used. Note that using these two options implies "llc-coloring=on". + +### llc-size +> `= ` + +> Default: `Obtained from hardware` + +Specify the size of the Last Level Cache. This option is available only when +`CONFIG_LLC_COLORING` is enabled. LLC size and number of ways are used to find +the number of supported cache colors. By default the value is automatically +computed by probing the hardware, but in case of specific needs, it can be +manually set. Those include failing probing and debugging/testing purposes so +that it's possible to emulate platforms with different number of supported +colors. If set, also "llc-nr-ways" must be set, otherwise the default will be +used. Note that using these two options implies "llc-coloring=on". + ### lock-depth-size > `= ` diff --git a/xen/common/Kconfig b/xen/common/Kconfig index cff3166ff9..276ef4e0f9 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -71,6 +71,9 @@ config HAS_IOPORTS config HAS_KEXEC bool +config HAS_LLC_COLORING + bool + config HAS_PIRQ bool @@ -513,4 +516,23 @@ config TRACEBUFFER to be collected at run time for debugging or performance analysis. Memory and execution overhead when not active is minimal. +config LLC_COLORING + bool "Last Level Cache (LLC) coloring" if EXPERT + depends on HAS_LLC_COLORING + depends on !NUMA + +config MAX_LLC_COLORS_ORDER + int "Maximum number of LLC colors (base-2 exponent)" + range 1 10 + default 7 + depends on LLC_COLORING + help + Controls the build-time size of various arrays associated with LLC + coloring. The value is a base-2 exponent. Refer to cache coloring + documentation for how to compute the number of colors supported by the + platform. This is only an upper bound. The runtime value is autocomputed + or manually set via cmdline parameters. + The default value corresponds to an 8 MiB 16-ways LLC, which should be + more than what's needed in the general case. + endmenu diff --git a/xen/common/Makefile b/xen/common/Makefile index d512cad524..7aa74a74ce 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -23,6 +23,7 @@ obj-y += keyhandler.o obj-$(CONFIG_KEXEC) += kexec.o obj-$(CONFIG_KEXEC) += kimage.o obj-$(CONFIG_LIVEPATCH) += livepatch.o livepatch_elf.o +obj-$(CONFIG_LLC_COLORING) += llc-coloring.o obj-$(CONFIG_MEM_ACCESS) += mem_access.o obj-y += memory.o obj-y += multicall.o diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c index 127ca50696..778f93e063 100644 --- a/xen/common/keyhandler.c +++ b/xen/common/keyhandler.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -303,6 +304,8 @@ static void cf_check dump_domains(unsigned char key) arch_dump_domain_info(d); + domain_dump_llc_colors(d); + rangeset_domain_printk(d); dump_pageframe_info(d); diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c new file mode 100644 index 0000000000..65ed96603a --- /dev/null +++ b/xen/common/llc-coloring.c @@ -0,0 +1,111 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Last Level Cache (LLC) coloring common code + * + * Copyright (C) 2022 Xilinx Inc. + */ +#include +#include +#include + +#define NR_LLC_COLORS (1 << CONFIG_MAX_LLC_COLORS_ORDER) + +static bool __ro_after_init llc_coloring_enabled; +boolean_param("llc-coloring", llc_coloring_enabled); + +static unsigned int __initdata llc_size; +size_param("llc-size", llc_size); +static unsigned int __initdata llc_nr_ways; +integer_param("llc-nr-ways", llc_nr_ways); +/* Number of colors available in the LLC */ +static unsigned int __ro_after_init max_nr_colors; + +static void print_colors(const unsigned int *colors, unsigned int num_colors) +{ + unsigned int i; + + printk("{ "); + for ( i = 0; i < num_colors; i++ ) + { + unsigned int start = colors[i], end = start; + + printk("%u", start); + + for ( ; i < num_colors - 1 && end + 1 == colors[i + 1]; i++, end++ ) + ; + + if ( start != end ) + printk("-%u", end); + + if ( i < num_colors - 1 ) + printk(", "); + } + printk(" }\n"); +} + +void __init llc_coloring_init(void) +{ + unsigned int way_size; + + if ( llc_size && llc_nr_ways ) + { + llc_coloring_enabled = true; + way_size = llc_size / llc_nr_ways; + } + else if ( !llc_coloring_enabled ) + return; + else + { + way_size = get_llc_way_size(); + if ( !way_size ) + panic("LLC probing failed and 'llc-size' or 'llc-nr-ways' missing\n"); + } + + /* + * The maximum number of colors must be a power of 2 in order to correctly + * map them to bits of an address. + */ + max_nr_colors = way_size >> PAGE_SHIFT; + + if ( max_nr_colors & (max_nr_colors - 1) ) + panic("Number of LLC colors (%u) isn't a power of 2\n", max_nr_colors); + + if ( max_nr_colors > NR_LLC_COLORS ) + { + printk(XENLOG_WARNING + "Number of LLC colors (%u) too big. Using configured max %u\n", + max_nr_colors, NR_LLC_COLORS); + max_nr_colors = NR_LLC_COLORS; + } else if ( max_nr_colors < 2 ) + panic("Number of LLC colors %u < 2\n", max_nr_colors); + + arch_llc_coloring_init(); +} + +void dump_llc_coloring_info(void) +{ + if ( !llc_coloring_enabled ) + return; + + printk("LLC coloring info:\n"); + printk(" Number of LLC colors supported: %u\n", max_nr_colors); +} + +void domain_dump_llc_colors(const struct domain *d) +{ + if ( !llc_coloring_enabled ) + return; + + printk("%u LLC colors: ", d->num_llc_colors); + print_colors(d->llc_colors, d->num_llc_colors); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 7c1bdfc046..d2032a79b0 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -126,6 +126,7 @@ #include #include #include +#include #include #include #include @@ -2625,6 +2626,8 @@ static void cf_check pagealloc_info(unsigned char key) } printk(" Dom heap: %lukB free\n", total << (PAGE_SHIFT-10)); + + dump_llc_coloring_info(); } static __init int cf_check pagealloc_keyhandler_init(void) diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h new file mode 100644 index 0000000000..c60c8050c5 --- /dev/null +++ b/xen/include/xen/llc-coloring.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Last Level Cache (LLC) coloring common header + * + * Copyright (C) 2022 Xilinx Inc. + */ +#ifndef __COLORING_H__ +#define __COLORING_H__ + +#include +#include + +#ifdef CONFIG_LLC_COLORING +void llc_coloring_init(void); +void dump_llc_coloring_info(void); +void domain_dump_llc_colors(const struct domain *d); +#else +static inline void llc_coloring_init(void) {} +static inline void dump_llc_coloring_info(void) {} +static inline void domain_dump_llc_colors(const struct domain *d) {} +#endif + +unsigned int get_llc_way_size(void); +void arch_llc_coloring_init(void); + +#endif /* __COLORING_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 132b841995..1bdb4767dd 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -629,6 +629,11 @@ struct domain /* Holding CDF_* constant. Internal flags for domain creation. */ unsigned int cdf; + +#ifdef CONFIG_LLC_COLORING + unsigned int num_llc_colors; + const unsigned int *llc_colors; +#endif }; static inline struct page_list_head *page_to_list( From patchwork Thu May 2 16:55:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0C08C25B74 for ; Thu, 2 May 2024 16:56:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715948.1117941 (Exim 4.92) (envelope-from ) id 1s2Zih-0006kn-MB; Thu, 02 May 2024 16:55:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715948.1117941; Thu, 02 May 2024 16:55:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zih-0006ij-HT; Thu, 02 May 2024 16:55:43 +0000 Received: by outflank-mailman (input) for mailman id 715948; Thu, 02 May 2024 16:55:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zif-0006N3-VL for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [2a00:1450:4864:20::52a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ce01ee55-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:39 +0200 (CEST) Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-572babec735so938445a12.0 for ; Thu, 02 May 2024 09:55:39 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:38 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ce01ee55-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668939; x=1715273739; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MCgujxNjMSkmhKzG/UmU41GDMYeB6voszmPxr4Qdo+o=; b=Yl1jixmoGr0qtFNK9iMplMDmSp+DGO0hnjggMSKf/sqMTMZSDB7xOAnausgc5rCZE8 fo9fpAD+1o0vFSfdt617ommsQaETQByiSL/qPaCbAmKQPSMlo/NsTlE5lbNku5JRn5kM 9RReyMmnK0N2U1g10osGaSfARxqRyitC2ApoQT+zdsKtpEeBbqASDh5PdKpdOIn0PA+1 HGD6U0TAveG9odokZ8sVSivxBjuIkMbwZORwmkXdqFFiCbzFyFlFnRit2LeWRfAJZ9lX XEYkE0ShYyllGrgO48kNwG9xRsmW2X0KWIkzcTT6Sk9JnIgYiCdPEhM2MBbGm/lQvsq0 up9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668939; x=1715273739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MCgujxNjMSkmhKzG/UmU41GDMYeB6voszmPxr4Qdo+o=; b=UnGxarK5N1LatJpnbiZYnbQSEsUiyurEcZ14Txe403xsexk6DXYA9ql5LDdi0NN4Hk 2bHxHVpsFCwsngGyWfF4PVQwNPvy0BKj8eb7liMrS1WDAZM6jJXh8wBZKevsR/NnSUZY vo2EYJLTgA5bPefmeA3+LJ2qsPozSbmxAGebT61+UCZzo/HvFJBjxAqOLpGfuJ4IqDZ2 kQBz+PT+SwnnCM458n2DdAraoT9haiTDYHd1lT5zY9HmRsWvczFEAi2SSGEyxnwKSL1X cv6TcCAZAN1RaWtGYuCwsuzHrShRln8dVjCl3AzyGocsbZibk6DVAg+v+qFlwy/XLtYZ 4HwQ== X-Gm-Message-State: AOJu0YxAK4tRqGmYtgRqRGJr5uQMmzQ5DSKwq8ERb7eD9R5Pg2jzhXzf ew6uhQwFmRrjXzjo6bsLurKvRNCYEdGIq8QZc9WhI7+sjQZjig4fEGaXvsFrWy5wQbh6eS++ANk Z X-Google-Smtp-Source: AGHT+IFUMs2AimGGb+Nekd0DlA/Awn9vWPGIcCYkVDdvV6g6eekdDqZurnlmAEDoy1p37abPu6BM/w== X-Received: by 2002:a17:906:5604:b0:a58:fcd2:759c with SMTP id f4-20020a170906560400b00a58fcd2759cmr212910ejq.34.1714668938966; Thu, 02 May 2024 09:55:38 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Marco Solieri Subject: [PATCH v8 02/13] xen/arm: add initial support for LLC coloring on arm64 Date: Thu, 2 May 2024 18:55:22 +0200 Message-Id: <20240502165533.319988-3-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 LLC coloring needs to know the last level cache layout in order to make the best use of it. This can be probed by inspecting the CLIDR_EL1 register, so the Last Level is defined as the last level visible by this register. Note that this excludes system caches in some platforms. Static memory allocation and cache coloring are incompatible because static memory can't be guaranteed to use only colors assigned to the domain. Panic during DomUs creation when both are enabled. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- v8: - no changes v7: - only minor changes v6: - get_llc_way_size() now checks for at least separate I/D caches v5: - used - instead of _ for filenames - moved static-mem check in this patch - moved dom0 colors parsing in next patch - moved color allocation and configuration in next patch - moved check_colors() in next patch - colors are now printed in short form v4: - added "llc-coloring" cmdline option for the boot-time switch - dom0 colors are now checked during domain init as for any other domain - fixed processor.h masks bit width - check for overflow in parse_color_config() - check_colors() now checks also that colors are sorted and unique --- docs/misc/cache-coloring.rst | 14 +++++ xen/arch/arm/Kconfig | 1 + xen/arch/arm/Makefile | 1 + xen/arch/arm/dom0less-build.c | 6 +++ xen/arch/arm/include/asm/processor.h | 16 ++++++ xen/arch/arm/llc-coloring.c | 77 ++++++++++++++++++++++++++++ xen/arch/arm/setup.c | 3 ++ xen/common/llc-coloring.c | 2 +- xen/include/xen/llc-coloring.h | 4 ++ 9 files changed, 123 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/llc-coloring.c diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst index bcb45e9344..404262b728 100644 --- a/docs/misc/cache-coloring.rst +++ b/docs/misc/cache-coloring.rst @@ -111,6 +111,20 @@ Auto-probing of LLC specs LLC size and number of ways are probed automatically by default. +In the Arm implementation, this is done by inspecting the CLIDR_EL1 register. +This means that other system caches that aren't visible there are ignored. + LLC specs can be manually set via the above command line parameters. This bypasses any auto-probing and it's used to overcome failing situations, such as flawed probing logic, or for debugging/testing purposes. + +Known issues and limitations +**************************** + +"xen,static-mem" isn't supported when coloring is enabled +######################################################### + +In the domain configuration, "xen,static-mem" allows memory to be statically +allocated to the domain. This isn't possible when LLC coloring is enabled, +because that memory can't be guaranteed to use only colors assigned to the +domain. diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index f8139a773a..36dfdc53c4 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -8,6 +8,7 @@ config ARM_64 depends on !ARM_32 select 64BIT select HAS_FAST_MULTIPLY + select HAS_LLC_COLORING config ARM def_bool y diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 7b1350e2ef..18ae566521 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -37,6 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o obj-y += irq.o obj-y += kernel.init.o obj-$(CONFIG_LIVEPATCH) += livepatch.o +obj-$(CONFIG_LLC_COLORING) += llc-coloring.o obj-y += mem_access.o obj-y += mm.o obj-y += monitor.o diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index 74f053c242..ca7519cf3b 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -880,7 +881,12 @@ void __init create_domUs(void) panic("No more domain IDs available\n"); if ( dt_find_property(node, "xen,static-mem", NULL) ) + { + if ( llc_coloring_enabled ) + panic("LLC coloring and static memory are incompatible\n"); + flags |= CDF_staticmem; + } if ( dt_property_read_bool(node, "direct-map") ) { diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h index 8e02410465..ef33ea198c 100644 --- a/xen/arch/arm/include/asm/processor.h +++ b/xen/arch/arm/include/asm/processor.h @@ -18,6 +18,22 @@ #define CTR_IDC_SHIFT 28 #define CTR_DIC_SHIFT 29 +/* CCSIDR Current Cache Size ID Register */ +#define CCSIDR_LINESIZE_MASK _AC(0x7, UL) +#define CCSIDR_NUMSETS_SHIFT 13 +#define CCSIDR_NUMSETS_MASK _AC(0x3fff, UL) +#define CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX 32 +#define CCSIDR_NUMSETS_MASK_FEAT_CCIDX _AC(0xffffff, UL) + +/* CSSELR Cache Size Selection Register */ +#define CSSELR_LEVEL_MASK _AC(0x7, UL) +#define CSSELR_LEVEL_SHIFT 1 + +/* CLIDR Cache Level ID Register */ +#define CLIDR_CTYPEn_SHIFT(n) (3 * ((n) - 1)) +#define CLIDR_CTYPEn_MASK _AC(0x7, UL) +#define CLIDR_CTYPEn_LEVELS 7 + #define ICACHE_POLICY_VPIPT 0 #define ICACHE_POLICY_AIVIVT 1 #define ICACHE_POLICY_VIPT 2 diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c new file mode 100644 index 0000000000..66c8db2baf --- /dev/null +++ b/xen/arch/arm/llc-coloring.c @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Last Level Cache (LLC) coloring support for ARM + * + * Copyright (C) 2022 Xilinx Inc. + */ +#include +#include + +#include +#include + +/* Return the LLC way size by probing the hardware */ +unsigned int __init get_llc_way_size(void) +{ + register_t ccsidr_el1; + register_t clidr_el1 = READ_SYSREG(CLIDR_EL1); + register_t csselr_el1 = READ_SYSREG(CSSELR_EL1); + register_t id_aa64mmfr2_el1 = READ_SYSREG(ID_AA64MMFR2_EL1); + uint32_t ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT; + uint32_t ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK; + unsigned int n, line_size, num_sets; + + for ( n = CLIDR_CTYPEn_LEVELS; n != 0; n-- ) + { + uint8_t ctype_n = (clidr_el1 >> CLIDR_CTYPEn_SHIFT(n)) & + CLIDR_CTYPEn_MASK; + + /* Unified cache (see Arm ARM DDI 0487J.a D19.2.27) */ + if ( ctype_n == 0b100 ) + break; + } + + if ( n == 0 ) + return 0; + + WRITE_SYSREG((n - 1) << CSSELR_LEVEL_SHIFT, CSSELR_EL1); + isb(); + + ccsidr_el1 = READ_SYSREG(CCSIDR_EL1); + + /* Arm ARM: (Log2(Number of bytes in cache line)) - 4 */ + line_size = 1U << ((ccsidr_el1 & CCSIDR_LINESIZE_MASK) + 4); + + /* If FEAT_CCIDX is enabled, CCSIDR_EL1 has a different bit layout */ + if ( (id_aa64mmfr2_el1 >> ID_AA64MMFR2_CCIDX_SHIFT) & 0x7 ) + { + ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX; + ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK_FEAT_CCIDX; + } + + /* Arm ARM: (Number of sets in cache) - 1 */ + num_sets = ((ccsidr_el1 >> ccsidr_numsets_shift) & ccsidr_numsets_mask) + 1; + + printk(XENLOG_INFO "LLC found: L%u (line size: %u bytes, sets num: %u)\n", + n, line_size, num_sets); + + /* Restore value in CSSELR_EL1 */ + WRITE_SYSREG(csselr_el1, CSSELR_EL1); + isb(); + + return line_size * num_sets; +} + +void __init arch_llc_coloring_init(void) +{ +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index d242674381..bbb3e1eea5 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -776,6 +777,8 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset, printk("Command line: %s\n", cmdline); cmdline_parse(cmdline); + llc_coloring_init(); + setup_mm(); vm_init(); diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index 65ed96603a..bcc651cc10 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -10,7 +10,7 @@ #define NR_LLC_COLORS (1 << CONFIG_MAX_LLC_COLORS_ORDER) -static bool __ro_after_init llc_coloring_enabled; +bool __ro_after_init llc_coloring_enabled; boolean_param("llc-coloring", llc_coloring_enabled); static unsigned int __initdata llc_size; diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index c60c8050c5..67b27c995b 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -11,10 +11,14 @@ #include #ifdef CONFIG_LLC_COLORING +extern bool llc_coloring_enabled; + void llc_coloring_init(void); void dump_llc_coloring_info(void); void domain_dump_llc_colors(const struct domain *d); #else +#define llc_coloring_enabled false + static inline void llc_coloring_init(void) {} static inline void dump_llc_coloring_info(void) {} static inline void domain_dump_llc_colors(const struct domain *d) {} From patchwork Thu May 2 16:55:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651896 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 831DDC25B75 for ; Thu, 2 May 2024 16:56:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715946.1117930 (Exim 4.92) (envelope-from ) id 1s2Zih-0006au-4e; Thu, 02 May 2024 16:55:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715946.1117930; Thu, 02 May 2024 16:55:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zih-0006an-1o; Thu, 02 May 2024 16:55:43 +0000 Received: by outflank-mailman (input) for mailman id 715946; Thu, 02 May 2024 16:55:41 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zif-0006Lf-Bn for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:41 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ce959dcb-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:40 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a4702457ccbso1157253466b.3 for ; Thu, 02 May 2024 09:55:40 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:39 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ce959dcb-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668939; x=1715273739; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=okh8duaE+EFMPEO4FhT3MzEyU6q3kS1zoVKRzTYnIo0=; b=YsLw4RvwdV3j1BdDTCzJ0VbrZynYZIzldbHHf4L9M7usWrlUNVQSWrWEU+XUG/sXiF Da63TUupIvjGKe0zFYseJdeIklpHgaeFFvC8TePKZOgXvPd5Z0Hd3R35EbHq8hLLxSnn /UG79/8YwLttDgQn8tb+IiW3SdTU6LaVl8eum5ocF58e0OHqQZFCvAm8SzNqWxdxQCEL 5k2QIeL+7ryXu7/H+V/dhX1v5UX6+NFzyGP9fZHpaNXOJazG9QBCKdW7bn3fSagCu9GX MrkUKnTzecoIYgJ/N2/pTrBsGzO0tBb0TaGfHHcKwu5ZkaLO3W8KzpeNffKplPRBskUH 8I2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668939; x=1715273739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=okh8duaE+EFMPEO4FhT3MzEyU6q3kS1zoVKRzTYnIo0=; b=v7h9OmUe9XXsbsYAXBs0SQh597BoVhnbBqU87k+ktrTF/TGU5BT+SbZlecz6FxjNw1 /dXqW/5kiJAF0+lAtae2kXRFUNV/4vTX+rbAJQylRRNbH/TBM7e2g09Z1n99dnTdAUZQ LqHHIzH3qLxoYB++DBbijoQTA+Z3fbbm2kk9PfsHhdnIKiK9tujPwsjknQ+10NWoNVwV GKcdYCKdvz00EinepZ8E8cM6h22kRWiGKz1URkUX3BQSKtWES4E65tp2xHhZNy3YJOFG prPJHTD8R6DMHzE5Hs5rTeXQNI3YyhMXGcjoAMbWRIDB8AVHztRtBu0XAqHHer2wKEz3 +mAw== X-Gm-Message-State: AOJu0YwEEbqCaapfzjXD3WYMvZNoygcjckW1XWV+RI+OKhus26+JoLiu dZZUg+IqjM81+UBxezxX1v2kboefuerNBCFeVrvG93/zV/O9IkH8HeghW+XCraGXZlkEcH91Mi9 Y X-Google-Smtp-Source: AGHT+IGHjOT87G7UTWtB/PiIPUk65gmJVKYGc/SIGIKgPC0DhCwF2tIf+GoOIE1hkPBxW5p8pnGwVw== X-Received: by 2002:a17:906:110c:b0:a59:719d:2735 with SMTP id h12-20020a170906110c00b00a59719d2735mr54793eja.64.1714668939702; Thu, 02 May 2024 09:55:39 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 03/13] xen/arm: permit non direct-mapped Dom0 construction Date: Thu, 2 May 2024 18:55:23 +0200 Message-Id: <20240502165533.319988-4-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Cache coloring requires Dom0 not to be direct-mapped because of its non contiguous mapping nature, so allocate_memory() is needed in this case. 8d2c3ab18cc1 ("arm/dom0less: put dom0less feature code in a separate module") moved allocate_memory() in dom0less_build.c. In order to use it in Dom0 construction bring it back to domain_build.c and declare it in domain_build.h. Take the opportunity to adapt the implementation of allocate_memory() so that it uses the host layout when called on the hwdom, via find_unallocated_memory(). Signed-off-by: Carlo Nonato --- v8: - patch adapted to new changes to allocate_memory() v7: - allocate_memory() now uses the host layout when called on the hwdom v6: - new patch --- xen/arch/arm/dom0less-build.c | 44 ----------- xen/arch/arm/domain_build.c | 97 ++++++++++++++++++++++++- xen/arch/arm/include/asm/domain_build.h | 1 + 3 files changed, 94 insertions(+), 48 deletions(-) diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index ca7519cf3b..c6bc4ee59c 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -49,50 +49,6 @@ bool __init is_dom0less_mode(void) return ( !dom0found && domUfound ); } -static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo) -{ - struct membanks *mem = kernel_info_get_mem(kinfo); - unsigned int i; - paddr_t bank_size; - - printk(XENLOG_INFO "Allocating mappings totalling %ldMB for %pd:\n", - /* Don't want format this as PRIpaddr (16 digit hex) */ - (unsigned long)(kinfo->unassigned_mem >> 20), d); - - mem->nr_banks = 0; - bank_size = MIN(GUEST_RAM0_SIZE, kinfo->unassigned_mem); - if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE), - bank_size) ) - goto fail; - - bank_size = MIN(GUEST_RAM1_SIZE, kinfo->unassigned_mem); - if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE), - bank_size) ) - goto fail; - - if ( kinfo->unassigned_mem ) - goto fail; - - for( i = 0; i < mem->nr_banks; i++ ) - { - printk(XENLOG_INFO "%pd BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n", - d, - i, - mem->bank[i].start, - mem->bank[i].start + mem->bank[i].size, - /* Don't want format this as PRIpaddr (16 digit hex) */ - (unsigned long)(mem->bank[i].size >> 20)); - } - - return; - -fail: - panic("Failed to allocate requested domain memory." - /* Don't want format this as PRIpaddr (16 digit hex) */ - " %ldKB unallocated. Fix the VMs configurations.\n", - (unsigned long)kinfo->unassigned_mem >> 10); -} - #ifdef CONFIG_VGICV2 static int __init make_gicv2_domU_node(struct kernel_info *kinfo) { diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 0784e4c5e3..ba2e48ecc7 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -416,7 +416,6 @@ static void __init allocate_memory_11(struct domain *d, } } -#ifdef CONFIG_DOM0LESS_BOOT bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo, gfn_t sgfn, paddr_t tot_size) { @@ -479,7 +478,6 @@ bool __init allocate_bank_memory(struct domain *d, struct kernel_info *kinfo, return true; } -#endif /* * When PCI passthrough is available we want to keep the @@ -968,6 +966,94 @@ out: return res; } +void __init allocate_memory(struct domain *d, struct kernel_info *kinfo) +{ + struct membanks *mem = kernel_info_get_mem(kinfo); + unsigned int i, nr_banks = 2; + paddr_t bank_start, bank_size; + struct membanks *hwdom_ext_regions = NULL; + + printk(XENLOG_INFO "Allocating mappings totalling %ldMB for %pd:\n", + /* Don't want format this as PRIpaddr (16 digit hex) */ + (unsigned long)(kinfo->unassigned_mem >> 20), d); + + mem->nr_banks = 0; + /* + * Use host memory layout for hwdom. Only case for this is when LLC coloring + * is enabled. + */ + if ( is_hardware_domain(d) ) + { + ASSERT(llc_coloring_enabled); + + hwdom_ext_regions = xzalloc_flex_struct(struct membanks, bank, + NR_MEM_BANKS); + if ( !hwdom_ext_regions ) + goto fail; + hwdom_ext_regions->max_banks = NR_MEM_BANKS; + + if ( find_unallocated_memory(kinfo, hwdom_ext_regions) ) + goto fail; + + nr_banks = hwdom_ext_regions->nr_banks; + } + + for ( i = 0; kinfo->unassigned_mem > 0 && nr_banks > 0; i++, nr_banks-- ) + { + if ( is_hardware_domain(d) ) + { + bank_start = hwdom_ext_regions->bank[i].start; + bank_size = hwdom_ext_regions->bank[i].size; + + if ( bank_size < min_t(paddr_t, kinfo->unassigned_mem, MB(128)) ) + continue; + } + else + { + if ( i == 0 ) + { + bank_start = GUEST_RAM0_BASE; + bank_size = GUEST_RAM0_SIZE; + } + else if ( i == 1 ) + { + bank_start = GUEST_RAM1_BASE; + bank_size = GUEST_RAM1_SIZE; + } + else + goto fail; + } + + bank_size = MIN(bank_size, kinfo->unassigned_mem); + if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(bank_start), + bank_size) ) + goto fail; + } + + if ( kinfo->unassigned_mem ) + goto fail; + + for( i = 0; i < mem->nr_banks; i++ ) + { + printk(XENLOG_INFO "%pd BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n", + d, + i, + mem->bank[i].start, + mem->bank[i].start + mem->bank[i].size, + /* Don't want format this as PRIpaddr (16 digit hex) */ + (unsigned long)(mem->bank[i].size >> 20)); + } + + xfree(hwdom_ext_regions); + return; + +fail: + panic("Failed to allocate requested domain memory." + /* Don't want format this as PRIpaddr (16 digit hex) */ + " %ldKB unallocated. Fix the VMs configurations.\n", + (unsigned long)kinfo->unassigned_mem >> 10); +} + static int __init handle_pci_range(const struct dt_device_node *dev, uint64_t addr, uint64_t len, void *data) { @@ -1188,7 +1274,7 @@ int __init make_hypervisor_node(struct domain *d, ext_regions->max_banks = NR_MEM_BANKS; - if ( is_domain_direct_mapped(d) ) + if ( domain_use_host_layout(d) ) { if ( !is_iommu_enabled(d) ) res = find_unallocated_memory(kinfo, ext_regions); @@ -2117,7 +2203,10 @@ static int __init construct_dom0(struct domain *d) /* type must be set before allocate_memory */ d->arch.type = kinfo.type; #endif - allocate_memory_11(d, &kinfo); + if ( is_domain_direct_mapped(d) ) + allocate_memory_11(d, &kinfo); + else + allocate_memory(d, &kinfo); find_gnttab_region(d, &kinfo); rc = process_shm_chosen(d, &kinfo); diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h index 45936212ca..943a6d438e 100644 --- a/xen/arch/arm/include/asm/domain_build.h +++ b/xen/arch/arm/include/asm/domain_build.h @@ -8,6 +8,7 @@ typedef __be32 gic_interrupt_t[3]; bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo, gfn_t sgfn, paddr_t tot_size); +void allocate_memory(struct domain *d, struct kernel_info *kinfo); int construct_domain(struct domain *d, struct kernel_info *kinfo); int domain_fdt_begin_node(void *fdt, const char *name, uint64_t unit); int make_chosen_node(const struct kernel_info *kinfo); From patchwork Thu May 2 16:55:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E634C04FFE for ; Thu, 2 May 2024 16:56:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715949.1117952 (Exim 4.92) (envelope-from ) id 1s2Zii-0006wL-C8; Thu, 02 May 2024 16:55:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715949.1117952; Thu, 02 May 2024 16:55:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zii-0006uI-3d; Thu, 02 May 2024 16:55:44 +0000 Received: by outflank-mailman (input) for mailman id 715949; Thu, 02 May 2024 16:55:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zig-0006Lf-F7 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:42 +0000 Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com [2a00:1450:4864:20::230]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id cf1a9237-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:41 +0200 (CEST) Received: by mail-lj1-x230.google.com with SMTP id 38308e7fff4ca-2dd6a7ae2dcso134698611fa.1 for ; Thu, 02 May 2024 09:55:41 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:40 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cf1a9237-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668941; x=1715273741; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OQIt6EqAhVSChAJtwNddHFJtbh/THZTAMqTRnNcVJEs=; b=jSFX0KHkYBXIaoPeifOO/6mr6iU+PP9auXhL2gwBjQCjkh4dj+l/VxIPDamJtctki0 D4FhyLWVzxaHTjTKF8aOjaZXtMtAfbZQLBB7rbuq4zCbn3EkqBYjYiHTlYjsBGlE3FCT LZ+zrXXx5vAU7AzQzbA2SXPdscKs0ZU3Va54675pHyKUimb0PGZ075GyIWAw2vzj9uca jKVKPyfftBh5usEXknMX64dpGH9jsNlTsbHqEVXOKA5E4fjAlfffGSdKykjnRpEGb+7+ occ7+VgAvKJBg0/MidhOnLs94HidALAiQNzx3SujaGpjtncMfBDskKHPwg/iH/xgaksA Z/og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668941; x=1715273741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OQIt6EqAhVSChAJtwNddHFJtbh/THZTAMqTRnNcVJEs=; b=wzGJcA0WxZ30ABiRoECSJXLuE3/PC9NMoFO5f+elrL33U2Adk8jJt1PLpSYEmyT2sA BPiWb5qN67byBiQNyhQgN7iVIsFd+0EPGpQpM5rtsD0Emk7gA7bNafB9KacKXhO9MY0/ ZFrayJ0O97fign5X3s7s2Ch0Nfbj5WvrZefW7bAJXWa6QoIeXZxwvYK6RtJmDcIqAjo/ YLOh7yE8ElRdQOV0ozssoSWHVhU1bPwP6r4CpO2DJZHC3T5GsYQaJz0QGhOYEB32PKrf 992lk4aR5Mq2iIwoWqqJL9KsxycIh7BnPpXzodadNpSZjsKJXcO7tgSUkBbVudHR2sru 3wZw== X-Gm-Message-State: AOJu0YzPvvuvmaOnMtbYLnXFaHsc0fZ+FACOuW9YsL1ILXX/+Qq5Qb8c FJh8JsZhWZTDwV1MEIbX69mZuhTooGXxwr0ht2WHiBuQcTkflxRUzI4U/lztED4KStTqsqsSMNt U X-Google-Smtp-Source: AGHT+IFfTaHyfpFw1bJI3YbMS0k968SxK3ibfMEZ2lxT+qyoIgawCS1WBjyfcSqxwSdRD47+M/u78g== X-Received: by 2002:a2e:b04a:0:b0:2df:b2d5:5935 with SMTP id d10-20020a2eb04a000000b002dfb2d55935mr287080ljl.28.1714668940688; Thu, 02 May 2024 09:55:40 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Marco Solieri Subject: [PATCH v8 04/13] xen/arm: add Dom0 cache coloring support Date: Thu, 2 May 2024 18:55:24 +0200 Message-Id: <20240502165533.319988-5-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add a command line parameter to allow the user to set the coloring configuration for Dom0. A common configuration syntax for cache colors is introduced and documented. Take the opportunity to also add: - default configuration notion. - function to check well-formed configurations. Direct mapping Dom0 isn't possible when coloring is enabled, so CDF_directmap flag is removed when creating it. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri Reviewed-by: Jan Beulich --- v8: - added bound check on dom0_num_colors - default colors array set just once v7: - parse_color_config() doesn't accept leading/trailing commas anymore - removed alloc_colors() helper v6: - moved domain_llc_coloring_free() in this patch - removed domain_alloc_colors() in favor of a more explicit allocation - parse_color_config() now accepts the size of the array to be filled - allocate_memory() moved in another patch v5: - Carlo Nonato as the new author - moved dom0 colors parsing (parse_colors()) in this patch - added dom0_set_llc_colors() to set dom0 colors after creation - moved color allocation and checking in this patch - error handling when allocating color arrays - FIXME: copy pasted allocate_memory() cause it got moved v4: - dom0 colors are dynamically allocated as for any other domain (colors are duplicated in dom0_colors and in the new array, but logic is simpler) --- docs/misc/cache-coloring.rst | 29 +++++++ docs/misc/xen-command-line.pandoc | 9 +++ xen/arch/arm/domain_build.c | 10 ++- xen/common/domain.c | 3 + xen/common/llc-coloring.c | 128 +++++++++++++++++++++++++++++- xen/include/xen/llc-coloring.h | 3 + 6 files changed, 180 insertions(+), 2 deletions(-) diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst index 404262b728..26b306a0ff 100644 --- a/docs/misc/cache-coloring.rst +++ b/docs/misc/cache-coloring.rst @@ -105,6 +105,35 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`. +----------------------+-------------------------------+ | ``llc-nr-ways`` | Set the LLC number of ways | +----------------------+-------------------------------+ +| ``dom0-llc-colors`` | Dom0 color configuration | ++----------------------+-------------------------------+ + +Colors selection format +*********************** + +Regardless of the memory pool that has to be colored (Xen, Dom0/DomUs), +the color selection can be expressed using the same syntax. In particular a +comma-separated list of colors or ranges of colors is used. +Ranges are hyphen-separated intervals (such as `0-4`) and are inclusive on both +sides. + +Note that: + +- no spaces are allowed between values. +- no overlapping ranges or duplicated colors are allowed. +- values must be written in ascending order. + +Examples: + ++-------------------+-----------------------------+ +| **Configuration** | **Actual selection** | ++-------------------+-----------------------------+ +| 1-2,5-8 | [1, 2, 5, 6, 7, 8] | ++-------------------+-----------------------------+ +| 4-8,10,11,12 | [4, 5, 6, 7, 8, 10, 11, 12] | ++-------------------+-----------------------------+ +| 0 | [0] | ++-------------------+-----------------------------+ Auto-probing of LLC specs ######################### diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 906822a491..2923e1fad4 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup. Specify a list of IO ports to be excluded from dom0 access. +### dom0-llc-colors +> `= List of [ | - ]` + +> Default: `All available LLC colors` + +Specify dom0 LLC color configuration. This option is available only when +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available +colors are used. + ### dom0_max_vcpus Either: diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index ba2e48ecc7..c0ed07051d 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2,6 +2,7 @@ #include #include #include +#include #include #include #include @@ -2250,6 +2251,7 @@ void __init create_dom0(void) .max_maptrack_frames = -1, .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version), }; + unsigned int flags = CDF_privileged; int rc; /* The vGIC for DOM0 is exactly emulating the hardware GIC */ @@ -2277,10 +2279,16 @@ void __init create_dom0(void) panic("SVE vector length error\n"); } - dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap); + if ( !llc_coloring_enabled ) + flags |= CDF_directmap; + + dom0 = domain_create(0, &dom0_cfg, flags); if ( IS_ERR(dom0) ) panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0)); + if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) ) + panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc); + if ( alloc_dom0_vcpu0(dom0) == NULL ) panic("Error creating domain 0 vcpu0\n"); diff --git a/xen/common/domain.c b/xen/common/domain.c index 6773f7fb90..a3cfb3bf37 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -1219,6 +1220,8 @@ void domain_destroy(struct domain *d) BUG_ON(!d->is_dying); + domain_llc_coloring_free(d); + /* May be already destroyed, or get_domain() can race us. */ if ( atomic_cmpxchg(&d->refcnt, 0, DOMAIN_DESTROYED) != 0 ) return; diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index bcc651cc10..26270cda9c 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -19,6 +19,66 @@ static unsigned int __initdata llc_nr_ways; integer_param("llc-nr-ways", llc_nr_ways); /* Number of colors available in the LLC */ static unsigned int __ro_after_init max_nr_colors; +/* Default coloring configuration */ +static unsigned int __ro_after_init default_colors[NR_LLC_COLORS]; + +static unsigned int __initdata dom0_colors[NR_LLC_COLORS]; +static unsigned int __initdata dom0_num_colors; + +/* + * Parse the coloring configuration given in the buf string, following the + * syntax below. + * + * COLOR_CONFIGURATION ::= COLOR | RANGE,...,COLOR | RANGE + * RANGE ::= COLOR-COLOR + * + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15,16. + */ +static int __init parse_color_config(const char *buf, unsigned int *colors, + unsigned int max_num_colors, + unsigned int *num_colors) +{ + const char *s = buf; + + *num_colors = 0; + + while ( *s != '\0' ) + { + unsigned int color, start, end; + + start = simple_strtoul(s, &s, 0); + + if ( *s == '-' ) /* Range */ + { + s++; + end = simple_strtoul(s, &s, 0); + } + else /* Single value */ + end = start; + + if ( start > end || (end - start) > (UINT_MAX - *num_colors) || + (*num_colors + (end - start)) >= max_num_colors ) + return -EINVAL; + + /* Colors are range checked in check_colors() */ + for ( color = start; color <= end; color++ ) + colors[(*num_colors)++] = color; + + if ( *s == ',' ) + s++; + else if ( *s != '\0' ) + break; + } + + return *s ? -EINVAL : 0; +} + +static int __init parse_dom0_colors(const char *s) +{ + return parse_color_config(s, dom0_colors, ARRAY_SIZE(dom0_colors), + &dom0_num_colors); +} +custom_param("dom0-llc-colors", parse_dom0_colors); static void print_colors(const unsigned int *colors, unsigned int num_colors) { @@ -43,9 +103,26 @@ static void print_colors(const unsigned int *colors, unsigned int num_colors) printk(" }\n"); } +static bool __init check_colors(const unsigned int *colors, + unsigned int num_colors) +{ + unsigned int i; + + for ( i = 0; i < num_colors; i++ ) + { + if ( colors[i] >= max_nr_colors ) + { + printk(XENLOG_ERR "LLC color %u >= %u\n", colors[i], max_nr_colors); + return false; + } + } + + return true; +} + void __init llc_coloring_init(void) { - unsigned int way_size; + unsigned int way_size, i; if ( llc_size && llc_nr_ways ) { @@ -79,6 +156,9 @@ void __init llc_coloring_init(void) } else if ( max_nr_colors < 2 ) panic("Number of LLC colors %u < 2\n", max_nr_colors); + for ( i = 0; i < max_nr_colors; i++ ) + default_colors[i] = i; + arch_llc_coloring_init(); } @@ -100,6 +180,52 @@ void domain_dump_llc_colors(const struct domain *d) print_colors(d->llc_colors, d->num_llc_colors); } +static void __init domain_set_default_colors(struct domain *d) +{ + printk(XENLOG_WARNING + "LLC color config not found for %pd, using all colors\n", d); + + d->llc_colors = default_colors; + d->num_llc_colors = max_nr_colors; +} + +int __init dom0_set_llc_colors(struct domain *d) +{ + typeof(*dom0_colors) *colors; + + if ( !dom0_num_colors ) + { + domain_set_default_colors(d); + return 0; + } + + if ( dom0_num_colors > max_nr_colors || + !check_colors(dom0_colors, dom0_num_colors) ) + { + printk(XENLOG_ERR "Bad LLC color config for %pd\n", d); + return -EINVAL; + } + + colors = xmalloc_array(typeof(*dom0_colors), dom0_num_colors); + if ( !colors ) + return -ENOMEM; + + memcpy(colors, dom0_colors, sizeof(*colors) * dom0_num_colors); + d->llc_colors = colors; + d->num_llc_colors = dom0_num_colors; + + return 0; +} + +void domain_llc_coloring_free(struct domain *d) +{ + if ( !llc_coloring_enabled || d->llc_colors == default_colors ) + return; + + /* free pointer-to-const using __va(__pa()) */ + xfree(__va(__pa(d->llc_colors))); +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index 67b27c995b..ee82932266 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -16,16 +16,19 @@ extern bool llc_coloring_enabled; void llc_coloring_init(void); void dump_llc_coloring_info(void); void domain_dump_llc_colors(const struct domain *d); +void domain_llc_coloring_free(struct domain *d); #else #define llc_coloring_enabled false static inline void llc_coloring_init(void) {} static inline void dump_llc_coloring_info(void) {} static inline void domain_dump_llc_colors(const struct domain *d) {} +static inline void domain_llc_coloring_free(struct domain *d) {} #endif unsigned int get_llc_way_size(void); void arch_llc_coloring_init(void); +int dom0_set_llc_colors(struct domain *d); #endif /* __COLORING_H__ */ From patchwork Thu May 2 16:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7A29C4345F for ; Thu, 2 May 2024 16:56:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715950.1117969 (Exim 4.92) (envelope-from ) id 1s2Zij-0007U9-LT; Thu, 02 May 2024 16:55:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715950.1117969; Thu, 02 May 2024 16:55:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zij-0007TA-HL; Thu, 02 May 2024 16:55:45 +0000 Received: by outflank-mailman (input) for mailman id 715950; Thu, 02 May 2024 16:55:44 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zih-0006N3-VT for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:43 +0000 Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [2a00:1450:4864:20::12c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id cfab38dc-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:42 +0200 (CEST) Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-51f3a49ff7dso919918e87.2 for ; Thu, 02 May 2024 09:55:42 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:41 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cfab38dc-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668942; x=1715273742; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9lNNmDdINfoHlllIflrjGEio9WD33GRBGAZScKnxGPA=; b=vIo2zAhWo5eT4uj4bhJHsRWmJ96FC1KhMzNfgWUQHJRriW1+BZI6M9DL5NQbPrNtcZ 5fKhFpddbpBBcuVtO8QgeExlq0N6fSh8Sd7z9JW5vTH8PtYCI9v9wV9fdbwsXQWy0PAl MXLAeUNhdQ/nl9v229ZGa6vDLMob3/X0JgFumCdMMCOLjMntwpit5KZTINPC5t6uQ+e2 +f3hwH2CXU+vepsngrNG++7j366Pv8af7st6jUbto74wfFD5cMp+I9iEefraM2e/T4BC gmgcD8QKw/Zp28T3EiS2W2YI7z8PdYWeGawI1q9fCNDBAqHhBVcmiKzhCT6h8prqYLn2 8I0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668942; x=1715273742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9lNNmDdINfoHlllIflrjGEio9WD33GRBGAZScKnxGPA=; b=Cx366My83m+MdyBKQgu+MEFX20N4SOVNJLrKvWYk4sWryJZcYn2JhYAto2SBL7sr/h /hQCGfxLGHnE3o9blCCX7niJ1BFWwntkiFsqrYmc8xFZ+WJrJ0pLu2aYzELjl5bCOwdd QppQmVCFPjne/Ixz+h6mg+EFelGFyCbUkygwL8xzIWs5g+4wlDXXto7SU0Bwtv/pQduI 2f45dBbyzpbAH+miKHH3GqoLEVFRrGn7Wajb77sGXjLbjhICzvN4Jo0Q7+JHlGfBuzeo gPamylXEAvm5gsJtd1IBvQyaFVKxt8yTAp9JjNvFc+5lXzsDCkJc+fSridXuOdeN677T laTA== X-Gm-Message-State: AOJu0YwnSCx5BeznvdWMGV1Zk8g1/cZ+uyLTwbmbzUBf+/In6h4QLcuG pGXcyD3DxYMZS63xYtvI9TgFt6vmP4OoatVrNV41Tp7QJSWxWxFK0QDZjQti69nasA89t4/exIo I X-Google-Smtp-Source: AGHT+IFE87a06WYupkqdQKi0ZTarJY7Cax3giuwXsFKojmeBItsHdWnxzl4JvHy/bNxWSAvrjg0bwg== X-Received: by 2002:a05:6512:32b9:b0:51e:11d5:bca5 with SMTP id q25-20020a05651232b900b0051e11d5bca5mr224999lfe.54.1714668941569; Thu, 02 May 2024 09:55:41 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Marco Solieri Subject: [PATCH v8 05/13] xen: extend domctl interface for cache coloring Date: Thu, 2 May 2024 18:55:25 +0200 Message-Id: <20240502165533.319988-6-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add a new domctl hypercall to allow the user to set LLC coloring configurations. Colors can be set only once, just after domain creation, since recoloring isn't supported. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri Reviewed-by: Jan Beulich --- v8: - fixed memory leak on error path of domain_set_llc_colors() v7: - -EOPNOTSUPP returned in case of hypercall called without llc_coloring_enabled - domain_set_llc_colors_domctl() renamed to domain_set_llc_colors() - added padding and input bound checks to domain_set_llc_colors() - removed alloc_colors() helper usage from domain_set_llc_colors() v6: - reverted the XEN_DOMCTL_INTERFACE_VERSION bump - reverted to uint32 for the guest handle - explicit padding added to the domctl struct - rewrote domain_set_llc_colors_domctl() to be more explicit v5: - added a new hypercall to set colors - uint for the guest handle v4: - updated XEN_DOMCTL_INTERFACE_VERSION --- xen/common/domctl.c | 10 ++++++++ xen/common/llc-coloring.c | 46 +++++++++++++++++++++++++++++++--- xen/include/public/domctl.h | 9 +++++++ xen/include/xen/llc-coloring.h | 2 ++ 4 files changed, 64 insertions(+), 3 deletions(-) diff --git a/xen/common/domctl.c b/xen/common/domctl.c index 2c0331bb05..30cca9a4ae 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -8,6 +8,7 @@ #include #include +#include #include #include #include @@ -864,6 +865,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) __HYPERVISOR_domctl, "h", u_domctl); break; + case XEN_DOMCTL_set_llc_colors: + if ( op->u.set_llc_colors.pad ) + ret = -EINVAL; + else if ( llc_coloring_enabled ) + ret = domain_set_llc_colors(d, &op->u.set_llc_colors); + else + ret = -EOPNOTSUPP; + break; + default: ret = arch_do_domctl(op, d, u_domctl); break; diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index 26270cda9c..ecfeb0ce82 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -4,6 +4,7 @@ * * Copyright (C) 2022 Xilinx Inc. */ +#include #include #include #include @@ -103,8 +104,7 @@ static void print_colors(const unsigned int *colors, unsigned int num_colors) printk(" }\n"); } -static bool __init check_colors(const unsigned int *colors, - unsigned int num_colors) +static bool check_colors(const unsigned int *colors, unsigned int num_colors) { unsigned int i; @@ -180,7 +180,7 @@ void domain_dump_llc_colors(const struct domain *d) print_colors(d->llc_colors, d->num_llc_colors); } -static void __init domain_set_default_colors(struct domain *d) +static void domain_set_default_colors(struct domain *d) { printk(XENLOG_WARNING "LLC color config not found for %pd, using all colors\n", d); @@ -226,6 +226,46 @@ void domain_llc_coloring_free(struct domain *d) xfree(__va(__pa(d->llc_colors))); } +int domain_set_llc_colors(struct domain *d, + const struct xen_domctl_set_llc_colors *config) +{ + unsigned int *colors; + + if ( d->num_llc_colors ) + return -EEXIST; + + if ( !config->num_llc_colors ) + { + domain_set_default_colors(d); + return 0; + } + + if ( config->num_llc_colors > max_nr_colors ) + return -EINVAL; + + colors = xmalloc_array(unsigned int, config->num_llc_colors); + if ( !colors ) + return -ENOMEM; + + if ( copy_from_guest(colors, config->llc_colors, config->num_llc_colors) ) + { + xfree(colors); + return -EFAULT; + } + + if ( !check_colors(colors, config->num_llc_colors) ) + { + printk(XENLOG_ERR "Bad LLC color config for %pd\n", d); + xfree(colors); + return -EINVAL; + } + + d->llc_colors = colors; + d->num_llc_colors = config->num_llc_colors; + + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index a33f9ec32b..d44eac8775 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1190,6 +1190,13 @@ struct xen_domctl_vmtrace_op { typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t); +struct xen_domctl_set_llc_colors { + /* IN LLC coloring parameters */ + uint32_t num_llc_colors; + uint32_t pad; + XEN_GUEST_HANDLE_64(uint32) llc_colors; +}; + struct xen_domctl { uint32_t cmd; #define XEN_DOMCTL_createdomain 1 @@ -1277,6 +1284,7 @@ struct xen_domctl { #define XEN_DOMCTL_vmtrace_op 84 #define XEN_DOMCTL_get_paging_mempool_size 85 #define XEN_DOMCTL_set_paging_mempool_size 86 +#define XEN_DOMCTL_set_llc_colors 87 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1339,6 +1347,7 @@ struct xen_domctl { struct xen_domctl_vuart_op vuart_op; struct xen_domctl_vmtrace_op vmtrace_op; struct xen_domctl_paging_mempool paging_mempool; + struct xen_domctl_set_llc_colors set_llc_colors; uint8_t pad[128]; } u; }; diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index ee82932266..b3801fca00 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -29,6 +29,8 @@ static inline void domain_llc_coloring_free(struct domain *d) {} unsigned int get_llc_way_size(void); void arch_llc_coloring_init(void); int dom0_set_llc_colors(struct domain *d); +int domain_set_llc_colors(struct domain *d, + const struct xen_domctl_set_llc_colors *config); #endif /* __COLORING_H__ */ From patchwork Thu May 2 16:55:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B336FC25B5C for ; Thu, 2 May 2024 16:56:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715951.1117973 (Exim 4.92) (envelope-from ) id 1s2Zik-0007Yu-3l; Thu, 02 May 2024 16:55:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715951.1117973; Thu, 02 May 2024 16:55:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zij-0007Xo-RW; Thu, 02 May 2024 16:55:45 +0000 Received: by outflank-mailman (input) for mailman id 715951; Thu, 02 May 2024 16:55:44 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zii-0006Lf-2b for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:44 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d01e4368-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:43 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a5966f5a76bso182206866b.1 for ; Thu, 02 May 2024 09:55:43 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:41 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d01e4368-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668942; x=1715273742; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3EUvo/9M8ITy6G3rHvhG7NG2DwAgBy3coT8I6ydQ2x4=; b=VS2N/fqYnnpQrOoXFuh4WnWwAMyilj52BvYFWiutiFyzngHCUGVMQo/ZO+mQ26abUp 3kqzhL/mFzAFqNNYXFJD2Mg4NHdMkkLk53mLoKVvn97sud+Vk+bAtdejAMybIkeeHdox bvgvazr/8lihylx4dFf1WkNxlqToN8qjq0/m2DCglGYBpm+tPtnmI2M5mXVNyhWTow/C ETIpbJ2DLEzSJNK6FSrk2KcICOx9po1PGh9ZSk4RscbbursZ2vP6CR2LtFDlX6VTyupd xtp7nmhpdhtoll48efxOIgvWN4GimRicXVY5FiM5792fQqjukKbGpsc05wjWo0H2DU1s wPFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668942; x=1715273742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3EUvo/9M8ITy6G3rHvhG7NG2DwAgBy3coT8I6ydQ2x4=; b=iOZLc3ERsqWRh7yM68RHux/WINVGJf+H9hPe1I4aZV3NXoVl08u2vlhFKl/qoSsnT/ 3FSZ4PcuID36/H4SRiEILzgGciaR+mvcRQFNN1INiiKKQwXYN0YweAM0eb3VwaF84TYx lCtkVyzju1ikNIBZ1dZ6dtBd2FN9wJ53QCfbXLudoQOJDr7WTScZwOJad0oqvqq8eOi2 I0ecpW/nSfrFatfkivSNRgtQg3WpWChOgBTDXDBeYVUZwe4fWk1J94Nt44Zegi/82jee KCaqLGdQi5EdY0Bx+YU5Jole588T4dbmhELCGo48rfBVwFIc/GAzqXudLY01WW4mGdEZ NBKg== X-Gm-Message-State: AOJu0Yz8IAvEFSklRsrEysq3hcS+FtGNy9ijk2fwK+x7tKZTqs6QtN1W Vm/1vHzv93M/x6YOl6fezr0HepPUidLZZMDi+sbiBJTCZQsJKlTtj3lN/AuVNdAaGYZ7lOjnrmc 5 X-Google-Smtp-Source: AGHT+IEaaUlZhh8N1o1seHM3dS9FR8bI1vP2vECN7M/57vevuCIUxNoc+9g8AGbYJ3CQ/WfMUyAYvw== X-Received: by 2002:a17:906:4906:b0:a55:5958:cb00 with SMTP id b6-20020a170906490600b00a555958cb00mr2219308ejq.38.1714668942177; Thu, 02 May 2024 09:55:42 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Anthony PERARD , Juergen Gross , Marco Solieri Subject: [PATCH v8 06/13] tools: add support for cache coloring configuration Date: Thu, 2 May 2024 18:55:26 +0200 Message-Id: <20240502165533.319988-7-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add a new "llc_colors" parameter that defines the LLC color assignment for a domain. The user can specify one or more color ranges using the same syntax used everywhere else for color config described in the documentation. The parameter is defined as a list of strings that represent the color ranges. Documentation is also added. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- v8: - warn the user in case of coloring not supported at hypervisor level v7: - removed unneeded NULL check before xc_hypercall_buffer_free() in xc_domain_set_llc_colors() v6: - no edits v5: - added LIBXL_HAVE_BUILDINFO_LLC_COLORS - moved color configuration in xc_domain_set_llc_colors() cause of the new hypercall v4: - removed overlapping color ranges checks during parsing - moved hypercall buffer initialization in libxenctrl --- docs/man/xl.cfg.5.pod.in | 10 +++++++++ tools/include/libxl.h | 5 +++++ tools/include/xenctrl.h | 9 ++++++++ tools/libs/ctrl/xc_domain.c | 35 +++++++++++++++++++++++++++++ tools/libs/light/libxl_create.c | 13 +++++++++++ tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 38 +++++++++++++++++++++++++++++++- 7 files changed, 110 insertions(+), 1 deletion(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index 8f2b375ce9..320644701b 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -3072,6 +3072,16 @@ raised. =back +=over 4 + +=item B + +Specify the Last Level Cache (LLC) color configuration for the guest. +B can be either a single color value or a hypen-separated closed +interval of colors (such as "0-4"). + +=back + =head3 x86 =over 4 diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 62cb07dea6..49521e5da4 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1368,6 +1368,11 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src); */ #define LIBXL_HAVE_BUILDINFO_HVM_SYSTEM_FIRMWARE +/* + * The libxl_domain_build_info has the llc_colors array. + */ +#define LIBXL_HAVE_BUILDINFO_LLC_COLORS 1 + /* * ERROR_REMUS_XXX error code only exists from Xen 4.5, Xen 4.6 and it * is changed to ERROR_CHECKPOINT_XXX in Xen 4.7 diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 4996855944..2ad8e2066b 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -2654,6 +2654,15 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32 int xc_domain_cacheflush(xc_interface *xch, uint32_t domid, xen_pfn_t start_pfn, xen_pfn_t nr_pfns); +/* + * Set LLC colors for a domain. + * It can only be used directly after domain creation. An attempt to use it + * afterwards will result in an error. + */ +int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid, + const unsigned int *llc_colors, + unsigned int num_llc_colors); + #if defined(__arm__) || defined(__aarch64__) int xc_dt_overlay(xc_interface *xch, void *overlay_fdt, uint32_t overlay_fdt_size, uint8_t overlay_op); diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c index f2d9d14b4d..d315cfa6c1 100644 --- a/tools/libs/ctrl/xc_domain.c +++ b/tools/libs/ctrl/xc_domain.c @@ -2180,6 +2180,41 @@ int xc_domain_soft_reset(xc_interface *xch, domctl.domain = domid; return do_domctl(xch, &domctl); } + +int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid, + const unsigned int *llc_colors, + unsigned int num_llc_colors) +{ + struct xen_domctl domctl = {}; + DECLARE_HYPERCALL_BUFFER(uint32_t, local); + int ret = -1; + + if ( num_llc_colors ) + { + size_t bytes = sizeof(uint32_t) * num_llc_colors; + + local = xc_hypercall_buffer_alloc(xch, local, bytes); + if ( local == NULL ) + { + PERROR("Could not allocate LLC colors for set_llc_colors"); + ret = -ENOMEM; + goto out; + } + memcpy(local, llc_colors, bytes); + set_xen_guest_handle(domctl.u.set_llc_colors.llc_colors, local); + } + + domctl.cmd = XEN_DOMCTL_set_llc_colors; + domctl.domain = domid; + domctl.u.set_llc_colors.num_llc_colors = num_llc_colors; + + ret = do_domctl(xch, &domctl); + +out: + xc_hypercall_buffer_free(xch, local); + + return ret; +} /* * Local variables: * mode: C diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c index 41252ec553..ff80625bd6 100644 --- a/tools/libs/light/libxl_create.c +++ b/tools/libs/light/libxl_create.c @@ -726,6 +726,19 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config, /* A new domain now exists */ *domid = local_domid; + ret = xc_domain_set_llc_colors(ctx->xch, local_domid, + b_info->llc_colors, + b_info->num_llc_colors); + if (ret < 0 && errno == EOPNOTSUPP) { + if (b_info->num_llc_colors > 0) + LOGED(WARN, local_domid, + "LLC coloring not enabled in the hypervisor"); + } else if (ret < 0) { + LOGED(ERROR, local_domid, "LLC colors allocation failed"); + rc = ERROR_FAIL; + goto out; + } + rc = libxl__is_domid_recent(gc, local_domid, &recent); if (rc) goto out; diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 470122e768..79118e1582 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -616,6 +616,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("ioports", Array(libxl_ioport_range, "num_ioports")), ("irqs", Array(uint32, "num_irqs")), ("iomem", Array(libxl_iomem_range, "num_iomem")), + ("llc_colors", Array(uint32, "num_llc_colors")), ("claim_mode", libxl_defbool), ("event_channels", uint32), ("kernel", string), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index ab09d0288b..ddfbcb00fd 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -1294,7 +1294,7 @@ void parse_config_data(const char *config_source, XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms, *usbctrls, *usbdevs, *p9devs, *vdispls, *pvcallsifs_devs; XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian, *dtdevs, - *mca_caps, *smbios; + *mca_caps, *smbios, *llc_colors; int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian, num_mca_caps; int num_smbios; int pci_power_mgmt = 0; @@ -1302,6 +1302,7 @@ void parse_config_data(const char *config_source, int pci_permissive = 0; int pci_seize = 0; int i, e; + int num_llc_colors; char *kernel_basename; libxl_domain_create_info *c_info = &d_config->c_info; @@ -1445,6 +1446,41 @@ void parse_config_data(const char *config_source, if (!xlu_cfg_get_long (config, "maxmem", &l, 0)) b_info->max_memkb = l * 1024; + if (!xlu_cfg_get_list(config, "llc_colors", &llc_colors, &num_llc_colors, 0)) { + int cur_index = 0; + + b_info->num_llc_colors = 0; + for (i = 0; i < num_llc_colors; i++) { + uint32_t start = 0, end = 0, k; + + buf = xlu_cfg_get_listitem(llc_colors, i); + if (!buf) { + fprintf(stderr, + "xl: Can't get element %d in LLC color list\n", i); + exit(1); + } + + if (sscanf(buf, "%" SCNu32 "-%" SCNu32, &start, &end) != 2) { + if (sscanf(buf, "%" SCNu32, &start) != 1) { + fprintf(stderr, "xl: Invalid LLC color range: %s\n", buf); + exit(1); + } + end = start; + } else if (start > end) { + fprintf(stderr, + "xl: Start LLC color is greater than end: %s\n", buf); + exit(1); + } + + b_info->num_llc_colors += (end - start) + 1; + b_info->llc_colors = (uint32_t *)realloc(b_info->llc_colors, + sizeof(*b_info->llc_colors) * b_info->num_llc_colors); + + for (k = start; k <= end; k++) + b_info->llc_colors[cur_index++] = k; + } + } + if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) { vcpus = l; if (libxl_cpu_bitmap_alloc(ctx, &b_info->avail_vcpus, l)) { From patchwork Thu May 2 16:55:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E6BABC4345F for ; Thu, 2 May 2024 16:56:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715953.1117998 (Exim 4.92) (envelope-from ) id 1s2Zim-0008DM-2o; Thu, 02 May 2024 16:55:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715953.1117998; Thu, 02 May 2024 16:55:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zil-0008Cp-Rk; Thu, 02 May 2024 16:55:47 +0000 Received: by outflank-mailman (input) for mailman id 715953; Thu, 02 May 2024 16:55:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zij-0006N3-K0 for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:45 +0000 Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [2a00:1450:4864:20::131]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d09ee2bf-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:44 +0200 (CEST) Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-5176f217b7bso14370030e87.0 for ; Thu, 02 May 2024 09:55:44 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:42 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d09ee2bf-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668943; x=1715273743; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wPiz9qg4rfggjlyChZDk0/7m5I+1YfXXIS22BJekSDM=; b=nFgSGVw9nfUaItCkluN54sq0cUwTgEvjU6xqtpLXk9cQkDYscp4tuW4JUdsBixnm+N Yeye2kS0+ttLMNM0gO5kF874+vgg75IheBqMfGe1Z6yLQ5neWxaKkA8L0CQxvdlG3Y1N WFiCXZtL72XXZszqyqKMiyF+XQFQMVGTP9HOr0b2lvFhQ3jYdHmM7nIceEaxAF35WpdM DidBo+WOt1/15lwSlbdiMGVms59BQ8OFl+e0Alj7NCygFtilkdiUC9yOb+PzDu2aHZ+D suWue6iue6WffqThlloOenAdlZsKn3DguLL4XE4KE5PwBxMeySBZ+n/FvLAZTyTrLKOM sWPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668943; x=1715273743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wPiz9qg4rfggjlyChZDk0/7m5I+1YfXXIS22BJekSDM=; b=acH4kB8YzlNN7UMt4ZGlUzjrK7nV+pCfA3bTcA08HfsBHWGgCClfvsBY4KYEGAJfeD BnHgrDzJ7Py+GSZsOZIZlcjbMpokT/5XAWp32I1sJa4JbrtyzV741UzOwZBXh2UcmBiR OJFIseiuZQe5lOlTbSg5Wd4zv+VhxyAq11KNUM78doQ9KEDSEHExMWXhhM4ACOG8KgsW 2+qkK7FQOHUIFCKHp+jHWC4KNzxia2/2kThx9jX7LPoY5R3IHvD7qFIkaPSgrvMWahu5 rShLWBLZXQm62HhhcZR/5p9E4cei+E0gIr4Fjx1oSJN7TL6oI5/OoK6abI0pqVU56rvI 66Kg== X-Gm-Message-State: AOJu0YyZ/XzaCz/0cevTpytCcGl9mGoGKQHafIZ7dr63ga254UQhguw9 a357cz2VRxsagFxD9K8EORedbGC0EAaKTzkbrjO7n/hYHp/XcSfRMGDSacjIldC9qY031SVFtf9 n X-Google-Smtp-Source: AGHT+IH3s4EhQ3jyWS2fovvunTeH+2VNzXP7zab5Chz7JFeYqEuat8WHR2Ki4VySNT+9d/sxhbjarg== X-Received: by 2002:a05:6512:36cf:b0:51d:4c8a:bbdb with SMTP id e15-20020a05651236cf00b0051d4c8abbdbmr310059lfs.3.1714668943288; Thu, 02 May 2024 09:55:43 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Marco Solieri Subject: [PATCH v8 07/13] xen/arm: add support for cache coloring configuration via device-tree Date: Thu, 2 May 2024 18:55:27 +0200 Message-Id: <20240502165533.319988-8-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add the "llc-colors" Device Tree attribute to express DomUs and Dom0less color configurations. Based on original work from: Luca Miccio Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- v8: - fixed memory leak on error path of domain_set_llc_colors_from_str() - realloc colors array after parsing from string to reduce memory usage v7: - removed alloc_colors() helper usage from domain_set_llc_colors_from_str() v6: - rewrote domain_set_llc_colors_from_str() to be more explicit v5: - static-mem check has been moved in a previous patch - added domain_set_llc_colors_from_str() to set colors after domain creation --- docs/misc/arm/device-tree/booting.txt | 4 +++ docs/misc/cache-coloring.rst | 48 +++++++++++++++++++++++++++ xen/arch/arm/dom0less-build.c | 10 ++++++ xen/common/llc-coloring.c | 42 +++++++++++++++++++++++ xen/include/xen/llc-coloring.h | 1 + xen/include/xen/xmalloc.h | 12 +++++++ 6 files changed, 117 insertions(+) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index bbd955e9c2..bbe49faadc 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -162,6 +162,10 @@ with the following properties: An integer specifying the number of vcpus to allocate to the guest. +- llc-colors + A string specifying the LLC color configuration for the guest. + Refer to docs/misc/cache_coloring.rst for syntax. + - vpl011 An empty property to enable/disable a virtual pl011 for the guest to diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst index 26b306a0ff..9527d0511e 100644 --- a/docs/misc/cache-coloring.rst +++ b/docs/misc/cache-coloring.rst @@ -12,6 +12,7 @@ If needed, change the maximum number of colors with ``CONFIG_MAX_LLC_COLORS_ORDER=``. Runtime configuration is done via `Command line parameters`_. +For DomUs follow `DomUs configuration`_. Background ********** @@ -147,6 +148,53 @@ LLC specs can be manually set via the above command line parameters. This bypasses any auto-probing and it's used to overcome failing situations, such as flawed probing logic, or for debugging/testing purposes. +DomUs configuration +******************* + +DomUs colors can be set either in the ``xl`` configuration file (documentation +at `docs/man/xl.cfg.pod.5.in`) or via Device Tree, also for Dom0less +configurations (documentation at `docs/misc/arm/device-tree/booting.txt`) using +the ``llc-colors`` option. For example: + +:: + + xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=1G dom0_max_vcpus=1 sched=null llc-coloring=on dom0-llc-colors=2-6"; + xen,dom0-bootargs "console=hvc0 earlycon=xen earlyprintk=xen root=/dev/ram0" + + dom0 { + compatible = "xen,linux-zimage" "xen,multiboot-module"; + reg = <0x0 0x1000000 0x0 15858176>; + }; + + dom0-ramdisk { + compatible = "xen,linux-initrd" "xen,multiboot-module"; + reg = <0x0 0x2000000 0x0 20638062>; + }; + + domU0 { + #address-cells = <0x1>; + #size-cells = <0x1>; + compatible = "xen,domain"; + memory = <0x0 0x40000>; + llc-colors = "4-8,10,11,12"; + cpus = <0x1>; + vpl011 = <0x1>; + + module@2000000 { + compatible = "multiboot,kernel", "multiboot,module"; + reg = <0x2000000 0xffffff>; + bootargs = "console=ttyAMA0"; + }; + + module@30000000 { + compatible = "multiboot,ramdisk", "multiboot,module"; + reg = <0x3000000 0xffffff>; + }; + }; + +**Note:** If no color configuration is provided for a domain, the default one, +which corresponds to all available colors is used instead. + Known issues and limitations **************************** diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index c6bc4ee59c..b77df9b642 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -807,6 +807,7 @@ void __init create_domUs(void) struct dt_device_node *node; const struct dt_device_node *cpupool_node, *chosen = dt_find_node_by_path("/chosen"); + const char *llc_colors_str = NULL; BUG_ON(chosen == NULL); dt_for_each_child_node(chosen, node) @@ -950,6 +951,10 @@ void __init create_domUs(void) #endif } + dt_property_read_string(node, "llc-colors", &llc_colors_str); + if ( !llc_coloring_enabled && llc_colors_str ) + panic("'llc-colors' found, but LLC coloring is disabled\n"); + /* * The variable max_init_domid is initialized with zero, so here it's * very important to use the pre-increment operator to call @@ -960,6 +965,11 @@ void __init create_domUs(void) panic("Error creating domain %s (rc = %ld)\n", dt_node_name(node), PTR_ERR(d)); + if ( llc_coloring_enabled && + (rc = domain_set_llc_colors_from_str(d, llc_colors_str)) ) + panic("Error initializing LLC coloring for domain %s (rc = %d)\n", + dt_node_name(node), rc); + d->is_console = true; dt_device_set_used_by(node, d->domain_id); diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index ecfeb0ce82..000cafbc74 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -4,6 +4,7 @@ * * Copyright (C) 2022 Xilinx Inc. */ +#include "xen/xmalloc.h" #include #include #include @@ -266,6 +267,47 @@ int domain_set_llc_colors(struct domain *d, return 0; } +int __init domain_set_llc_colors_from_str(struct domain *d, const char *str) +{ + int err; + unsigned int *colors, num_colors; + + if ( !str ) + { + domain_set_default_colors(d); + return 0; + } + + colors = xmalloc_array(unsigned int, max_nr_colors); + if ( !colors ) + return -ENOMEM; + + err = parse_color_config(str, colors, max_nr_colors, &num_colors); + if ( err ) + { + printk(XENLOG_ERR "Error parsing LLC color configuration"); + xfree(colors); + return err; + } + + if ( !check_colors(colors, num_colors) ) + { + printk(XENLOG_ERR "Bad LLC color config for %pd\n", d); + xfree(colors); + return -EINVAL; + } + + /* Adjust the size cause it was initially set to max_nr_colors */ + colors = xrealloc_array(colors, num_colors); + if ( !colors ) + return -ENOMEM; + + d->llc_colors = colors; + d->num_llc_colors = num_colors; + + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index b3801fca00..49ebd1e712 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -31,6 +31,7 @@ void arch_llc_coloring_init(void); int dom0_set_llc_colors(struct domain *d); int domain_set_llc_colors(struct domain *d, const struct xen_domctl_set_llc_colors *config); +int domain_set_llc_colors_from_str(struct domain *d, const char *str); #endif /* __COLORING_H__ */ diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h index 1b88a83be8..4bbf6ab7f3 100644 --- a/xen/include/xen/xmalloc.h +++ b/xen/include/xen/xmalloc.h @@ -34,6 +34,9 @@ ((_type *)_xmalloc_array(sizeof(_type), __alignof__(_type), _num)) #define xzalloc_array(_type, _num) \ ((_type *)_xzalloc_array(sizeof(_type), __alignof__(_type), _num)) +#define xrealloc_array(_ptr, _num) \ + ((typeof(_ptr))_xrealloc_array(_ptr, sizeof(typeof(*(_ptr))), \ + __alignof__(typeof(*(_ptr))), _num)) /* Allocate space for a structure with a flexible array of typed objects. */ #define xzalloc_flex_struct(type, field, nr) \ @@ -95,6 +98,15 @@ static inline void *_xzalloc_array( return _xzalloc(size * num, align); } +static inline void *_xrealloc_array( + void *ptr, unsigned long size, unsigned long align, unsigned long num) +{ + /* Check for overflow. */ + if ( size && num > UINT_MAX / size ) + return NULL; + return _xrealloc(ptr, size * num, align); +} + /* * Pooled allocator interface. */ From patchwork Thu May 2 16:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F77CC25B77 for ; Thu, 2 May 2024 16:56:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715952.1117981 (Exim 4.92) (envelope-from ) id 1s2Zik-0007kE-M6; Thu, 02 May 2024 16:55:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715952.1117981; Thu, 02 May 2024 16:55:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zik-0007hp-Ga; Thu, 02 May 2024 16:55:46 +0000 Received: by outflank-mailman (input) for mailman id 715952; Thu, 02 May 2024 16:55:45 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zij-0006Lf-8G for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:45 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d1013486-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:44 +0200 (CEST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a51addddbd4so947878966b.0 for ; Thu, 02 May 2024 09:55:44 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:43 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d1013486-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668944; x=1715273744; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jzaTNPxmLK/IqCNOK/3y/1wQIjSUY/Zy4G4Lp5Bcs1Q=; b=sgDBb+zj8XWJ3PJG/IdcqIhmZwejruC7M8BY0Q/Ric7F5upCVlZBicUv1Gzjv0XWUF hHGYY0AEXym6WALeb62wA+jjB+jR/l0PafNaqpwEswZPs6/yEt7IfdcH0nLCt92BFTqZ /4zaItUxlTdta6gq4j0NcMKPvqV8dtRkyNs/qr3XC6HJoenViLo6lHaZA5AVtr+7Ov1c CilMIG6mXKK/fsepZ4F0ULULDvfn2fnUkiEtFHQuoMaar3g3UjDH8fIvQv3mq+ENdxCA 0Vcyvthw+u/M0wTsKPXFZbVK6E5U4eGzigSj8BqP/5ZACF3hJ1MeN9G6siS33m0YXp/S ttrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668944; x=1715273744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jzaTNPxmLK/IqCNOK/3y/1wQIjSUY/Zy4G4Lp5Bcs1Q=; b=lL0yma5FdFVGiMsv73YeyxGNTAGRuQAT1hokl0il9QalF8c6A1Hr+2NuKak0AH69nm EMqD4TmbUKiEhCq3Yc/UEj2rNp5wKCZv6OPw8+XWfpb7VgEzNDAtttur0Lq2y0kiX3f8 M2PqyXUNuEP3hYQgQjpO4M0u6yMED3BggmHChsCPbJtJZVqLv0W5QEPbuMB4YmXp1VyF JGL9b+6G2w85MVZ+zhWkkWXtjcXDipb5Jt06DKMtAZb1MTsqRAEwjdm+flKPAPzXpsKi uD29fQwduafemHj83m5nwMcNFkCqdrIjs/bWjzdmw2gEJ7OHb0laJhzALCzl2PRgpiKy KRxg== X-Gm-Message-State: AOJu0Yz4mIlR3o7FBDw9lOJNNOdAuhDIq8rj5e1OOWTmmdvostmEM/gz 7yuLpEvNVQMvauKZ8SpPv3bAdP5SzJC7NlJLoFFxnVaSCnXt3pLE+5rne5IUEuHD4elbvUG0pen U X-Google-Smtp-Source: AGHT+IH8hfIaaSrExuqTbxGoNRBRtFoen/HYOr7HDoVeH46zsllKhxPxkt9DdQj/sz+Sb4NSSHJlqA== X-Received: by 2002:a17:906:7fd2:b0:a51:8145:6877 with SMTP id r18-20020a1709067fd200b00a5181456877mr65635ejs.37.1714668944045; Thu, 02 May 2024 09:55:44 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini Subject: [PATCH v8 08/13] xen/page_alloc: introduce preserved page flags macro Date: Thu, 2 May 2024 18:55:28 +0200 Message-Id: <20240502165533.319988-9-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 PGC_static and PGC_extra needs to be preserved when assigning a page. Define a new macro that groups those flags and use it instead of or'ing every time. To make preserved flags even more meaningful, they are kept also when switching state in mark_page_free(). Enforce the removal of PGC_extra before freeing new pages as this is considered an error and can cause ASSERT violations. Signed-off-by: Carlo Nonato --- v8: - fixed PGC_extra ASSERT fail in alloc_domheap_pages() by removing PGC_extra before freeing v7: - PGC_preserved used also in mark_page_free() v6: - preserved_flags renamed to PGC_preserved - PGC_preserved is used only in assign_pages() v5: - new patch --- xen/common/page_alloc.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index d2032a79b0..e71b571a3b 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -159,6 +159,7 @@ #endif #define PGC_no_buddy_merge PGC_static +#define PGC_preserved (PGC_extra | PGC_static) #ifndef PGT_TYPE_INFO_INITIALIZER #define PGT_TYPE_INFO_INITIALIZER 0 @@ -1426,11 +1427,11 @@ static bool mark_page_free(struct page_info *pg, mfn_t mfn) { case PGC_state_inuse: BUG_ON(pg->count_info & PGC_broken); - pg->count_info = PGC_state_free; + pg->count_info = PGC_state_free | (pg->count_info & PGC_preserved); break; case PGC_state_offlining: - pg->count_info = (pg->count_info & PGC_broken) | + pg->count_info = (pg->count_info & (PGC_broken | PGC_preserved)) | PGC_state_offlined; pg_offlined = true; break; @@ -2365,7 +2366,7 @@ int assign_pages( for ( i = 0; i < nr; i++ ) { - ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_static))); + ASSERT(!(pg[i].count_info & ~PGC_preserved)); if ( pg[i].count_info & PGC_extra ) extra_pages++; } @@ -2425,7 +2426,7 @@ int assign_pages( page_set_owner(&pg[i], d); smp_wmb(); /* Domain pointer must be visible before updating refcnt. */ pg[i].count_info = - (pg[i].count_info & (PGC_extra | PGC_static)) | PGC_allocated | 1; + (pg[i].count_info & PGC_preserved) | PGC_allocated | 1; page_list_add_tail(&pg[i], page_to_list(d, &pg[i])); } @@ -2484,6 +2485,11 @@ struct page_info *alloc_domheap_pages( } if ( assign_page(pg, order, d, memflags) ) { + unsigned long i; + + for ( i = 0; i < (1UL << order); i++ ) + pg[i].count_info &= ~PGC_extra; + free_heap_pages(pg, order, memflags & MEMF_no_scrub); return NULL; } @@ -2538,6 +2544,7 @@ void free_domheap_pages(struct page_info *pg, unsigned int order) { ASSERT(d->extra_pages); d->extra_pages--; + pg[i].count_info &= ~PGC_extra; } } From patchwork Thu May 2 16:55:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F219AC41513 for ; Thu, 2 May 2024 16:56:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715955.1118011 (Exim 4.92) (envelope-from ) id 1s2Zin-0000CN-T2; Thu, 02 May 2024 16:55:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715955.1118011; Thu, 02 May 2024 16:55:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zin-0000B7-MX; Thu, 02 May 2024 16:55:49 +0000 Received: by outflank-mailman (input) for mailman id 715955; Thu, 02 May 2024 16:55:47 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zil-0006Lf-Lc for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:47 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d1b26224-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:45 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a5557e3ebcaso367139466b.1 for ; Thu, 02 May 2024 09:55:45 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:44 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d1b26224-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668945; x=1715273745; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=70o7z5P/u9lC96jCkfPVLMG5TJG4lGrDIqh9ufbB1CA=; b=uckBMfCez2QqZ6UXAyGe67ysoXGSuaU5Ottdn691+jp1N2PLaedanOX7jkuSQILqYp ZU3X/fvoifSavT07cOV5R+uzrO4gCnadTvliWCZ0zMy1DwJFxAsEEXuqi9CTluBvY946 sibvyVjVVq+eToz8ctkHw5l9mOb8IHkVvJNHqQGHlInzqT5SufUThngHlAQrKuLoWRY7 DFX2j2Rm0Sq/y3HfLbMnJC3crosBVfYnIHRF1tFnfBmA2Rvo2Gp1UXrEwt1AADDh9C8K 2RUgVos+FbfMvwoQtDDpgg6bPplsBNwsDKPttK6o92Ck8H13KU2KpUSohn/A1CNXgfoW hJHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668945; x=1715273745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=70o7z5P/u9lC96jCkfPVLMG5TJG4lGrDIqh9ufbB1CA=; b=ofo3pDgmvvVqaHdI6PZSXhqyMb34qCJBWMOquWN3OYG65/3Xvm03Ty0kmT28csxonz 91aw0usPDmvOLSoLCH6Fh4XtiBHVnngljtiIG6id2pW1IA0QVTlsqeRejWJ6hcNqi3S7 RenH07mtKV9gOHRlvDw8cRq/cbxEYkffrBNdI6+biUjToVj0cRGohmoiv1TQDOvywqAY ODW46Am70YySsV7qiWtVI8JY+zqQqWb6LKUfgBlK9NFVlBcjM25fTlK0CUATVUku/7DA 37wUI1yc/AEc05ldY9qP2GuNQAd0/SaV4v07O922c6HXKp4+4zGS29/XXrbufE1zcL0s BBCQ== X-Gm-Message-State: AOJu0YxWQk9ppPfsaSyQRmDSpF1eYEP5qzd6ptk8zPSacw9EiOGdKIpR JfrZ4qDyVE+IgvQsRcV4DXA5HOidIrfPHmQYV6l66ad+Gu3okSQYVgGl9u4HtZzi5F7TbVjy0m5 X X-Google-Smtp-Source: AGHT+IGOA87V2E1Mf+hIXpVnN2hqZ/qwFf4DIW7+mPHoXDUmy4CYWAlF760gNoYDHNj0pS1S1hhl8g== X-Received: by 2002:a17:906:d293:b0:a51:abd8:8621 with SMTP id ay19-20020a170906d29300b00a51abd88621mr2580847ejb.19.1714668944927; Thu, 02 May 2024 09:55:44 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Marco Solieri Subject: [PATCH v8 09/13] xen: add cache coloring allocator for domains Date: Thu, 2 May 2024 18:55:29 +0200 Message-Id: <20240502165533.319988-10-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add a new memory page allocator that implements the cache coloring mechanism. The allocation algorithm enforces equal frequency distribution of cache partitions, following the coloring configuration of a domain. This allows for an even utilization of cache sets for every domain. Pages are stored in a color-indexed array of lists. Those lists are filled by a simple init function which computes the color of each page. When a domain requests a page, the allocator extracts the page from the list with the maximum number of free pages among those that the domain can access, given its coloring configuration. The allocator can only handle requests of order-0 pages. This allows for easier implementation and since cache coloring targets only embedded systems, it's assumed not to be a major problem. The buddy allocator must coexist with the colored one because the Xen heap isn't colored. For this reason a new Kconfig option and a command line parameter are added to let the user set the amount of memory reserved for the buddy allocator. Even when cache coloring is enabled, this memory isn't managed by the colored allocator. Colored heap information is dumped in the dump_heap() debug-key function. Based on original work from: Luca Miccio Signed-off-by: Marco Solieri Signed-off-by: Carlo Nonato --- v8: - requests that uses MEMF_* flags that can't be served are now going to fail - free_color_heap_page() is called directly from free_heap_pages() v7: - requests to alloc_color_heap_page() now fail if MEMF_bits is used v6: - colored allocator functions are now static v5: - Carlo Nonato as the new author - the colored allocator balances color usage for each domain and it searches linearly only in the number of colors (FIXME removed) - addedd scrub functionality - removed stub functions (still requires some macro definition) - addr_to_color turned to mfn_to_color for easier operations - removed BUG_ON in init_color_heap_pages() in favor of panic() - only non empty page lists are logged in dump_color_heap() v4: - moved colored allocator code after buddy allocator because it now has some dependencies on buddy functions - buddy_alloc_size is now used only by the colored allocator - fixed a bug that allowed the buddy to merge pages when they were colored - free_color_heap_page() now calls mark_page_free() - free_color_heap_page() uses of the frametable array for faster searches - added FIXME comment for the linear search in free_color_heap_page() - removed alloc_color_domheap_page() to let the colored allocator exploit some more buddy allocator code - alloc_color_heap_page() now allocs min address pages first - reduced the mess in end_boot_allocator(): use the first loop for init_color_heap_pages() - fixed page_list_add_prev() (list.h) since it was doing the opposite of what it was supposed to do - fixed page_list_add_prev() (non list.h) to check also for next existence - removed unused page_list_add_next() - moved p2m code in another patch --- docs/misc/cache-coloring.rst | 37 ++++++ docs/misc/xen-command-line.pandoc | 14 +++ xen/arch/arm/include/asm/mm.h | 5 + xen/common/Kconfig | 8 ++ xen/common/llc-coloring.c | 13 +++ xen/common/page_alloc.c | 188 +++++++++++++++++++++++++++++- xen/include/xen/llc-coloring.h | 4 + 7 files changed, 265 insertions(+), 4 deletions(-) diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst index 9527d0511e..f996627d0d 100644 --- a/docs/misc/cache-coloring.rst +++ b/docs/misc/cache-coloring.rst @@ -11,6 +11,9 @@ To compile LLC coloring support set ``CONFIG_LLC_COLORING=y``. If needed, change the maximum number of colors with ``CONFIG_MAX_LLC_COLORS_ORDER=``. +If needed, change the buddy allocator reserved size with +``CONFIG_BUDDY_ALLOCATOR_SIZE=``. + Runtime configuration is done via `Command line parameters`_. For DomUs follow `DomUs configuration`_. @@ -108,6 +111,8 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`. +----------------------+-------------------------------+ | ``dom0-llc-colors`` | Dom0 color configuration | +----------------------+-------------------------------+ +| ``buddy-alloc-size`` | Buddy allocator reserved size | ++----------------------+-------------------------------+ Colors selection format *********************** @@ -195,6 +200,17 @@ the ``llc-colors`` option. For example: **Note:** If no color configuration is provided for a domain, the default one, which corresponds to all available colors is used instead. +Colored allocator and buddy allocator +************************************* + +The colored allocator distributes pages based on color configurations of +domains so that each domains only gets pages of its own colors. +The colored allocator is meant as an alternative to the buddy allocator because +its allocation policy is by definition incompatible with the generic one. Since +the Xen heap is not colored yet, we need to support the coexistence of the two +allocators and some memory must be left for the buddy one. Buddy memory +reservation is configured via Kconfig or via command-line. + Known issues and limitations **************************** @@ -205,3 +221,24 @@ In the domain configuration, "xen,static-mem" allows memory to be statically allocated to the domain. This isn't possible when LLC coloring is enabled, because that memory can't be guaranteed to use only colors assigned to the domain. + +Cache coloring is intended only for embedded systems +#################################################### + +The current implementation aims to satisfy the need of predictability in +embedded systems with small amount of memory to be managed in a colored way. +Given that, some shortcuts are taken in the development. Expect worse +performances on larger systems. + +Colored allocator can only make use of order-0 pages +#################################################### + +The cache coloring technique relies on memory mappings and on the smallest +mapping granularity to achieve the maximum number of colors (cache partitions) +possible. This granularity is what is normally called a page and, in Xen +terminology, the order-0 page is the smallest one. The fairly simple +colored allocator currently implemented, makes use only of such pages. +It must be said that a more complex one could, in theory, adopt higher order +pages if the colors selection contained adjacent colors. Two subsequent colors, +for example, can be represented by an order-1 page, four colors correspond to +an order-2 page, etc. diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 2923e1fad4..d4d1e37272 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -270,6 +270,20 @@ and not running softirqs. Reduce this if softirqs are not being run frequently enough. Setting this to a high value may cause boot failure, particularly if the NMI watchdog is also enabled. +### buddy-alloc-size (arm64) +> `= ` + +> Default: `64M` + +Amount of memory reserved for the buddy allocator when colored allocator is +active. This options is available only when LLC coloring support is enabled. +The colored allocator is meant as an alternative to the buddy allocator, +because its allocation policy is by definition incompatible with the generic +one. Since the Xen heap systems is not colored yet, we need to support the +coexistence of the two allocators for now. This parameter, which is optional +and for expert only, it's used to set the amount of memory reserved to the +buddy allocator. + ### cet = List of [ shstk=, ibt= ] diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 48538b5337..68b7754bec 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -145,6 +145,11 @@ struct page_info #else #define PGC_static 0 #endif +#ifdef CONFIG_LLC_COLORING +/* Page is cache colored */ +#define _PGC_colored PG_shift(4) +#define PGC_colored PG_mask(1, 4) +#endif /* ... */ /* Page is broken? */ #define _PGC_broken PG_shift(7) diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 276ef4e0f9..16573400ed 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -535,4 +535,12 @@ config MAX_LLC_COLORS_ORDER The default value corresponds to an 8 MiB 16-ways LLC, which should be more than what's needed in the general case. +config BUDDY_ALLOCATOR_SIZE + int "Buddy allocator reserved memory size (MiB)" + default "64" + depends on LLC_COLORING + help + Amount of memory reserved for the buddy allocator to serve Xen heap, + working alongside the colored one. + endmenu diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index 000cafbc74..c598eb4bf7 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -27,6 +27,9 @@ static unsigned int __ro_after_init default_colors[NR_LLC_COLORS]; static unsigned int __initdata dom0_colors[NR_LLC_COLORS]; static unsigned int __initdata dom0_num_colors; +#define mfn_color_mask (max_nr_colors - 1) +#define mfn_to_color(mfn) (mfn_x(mfn) & mfn_color_mask) + /* * Parse the coloring configuration given in the buf string, following the * syntax below. @@ -308,6 +311,16 @@ int __init domain_set_llc_colors_from_str(struct domain *d, const char *str) return 0; } +unsigned int page_to_llc_color(const struct page_info *pg) +{ + return mfn_to_color(page_to_mfn(pg)); +} + +unsigned int get_max_nr_llc_colors(void) +{ + return max_nr_colors; +} + /* * Local variables: * mode: C diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index e71b571a3b..6a0f1262af 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -158,8 +158,12 @@ #define PGC_static 0 #endif -#define PGC_no_buddy_merge PGC_static -#define PGC_preserved (PGC_extra | PGC_static) +#ifndef PGC_colored +#define PGC_colored 0 +#endif + +#define PGC_no_buddy_merge (PGC_static | PGC_colored) +#define PGC_preserved (PGC_extra | PGC_static | PGC_colored) #ifndef PGT_TYPE_INFO_INITIALIZER #define PGT_TYPE_INFO_INITIALIZER 0 @@ -1457,6 +1461,8 @@ static bool mark_page_free(struct page_info *pg, mfn_t mfn) return pg_offlined; } +static void free_color_heap_page(struct page_info *pg, bool need_scrub); + /* Free 2^@order set of pages. */ static void free_heap_pages( struct page_info *pg, unsigned int order, bool need_scrub) @@ -1481,6 +1487,14 @@ static void free_heap_pages( pg[i].count_info |= PGC_need_scrub; poison_one_page(&pg[i]); } + + if ( pg->count_info & PGC_colored ) + { + /* Colored pages can be 0-order only, so ignore it */ + free_color_heap_page(pg, need_scrub); + spin_unlock(&heap_lock); + return; + } } avail[node][zone] += 1 << order; @@ -1945,6 +1959,156 @@ static unsigned long avail_heap_pages( return free_pages; } +/************************* + * COLORED SIDE-ALLOCATOR + * + * Pages are grouped by LLC color in lists which are globally referred to as the + * color heap. Lists are populated in end_boot_allocator(). + * After initialization there will be N lists where N is the number of + * available colors on the platform. + */ +static struct page_list_head *__ro_after_init _color_heap; +#define color_heap(color) (&_color_heap[color]) + +static unsigned long *__ro_after_init free_colored_pages; + +/* Memory required for buddy allocator to work with colored one */ +#ifdef CONFIG_LLC_COLORING +static unsigned long __initdata buddy_alloc_size = + MB(CONFIG_BUDDY_ALLOCATOR_SIZE); +size_param("buddy-alloc-size", buddy_alloc_size); + +#define domain_num_llc_colors(d) (d)->num_llc_colors +#define domain_llc_color(d, i) (d)->llc_colors[i] +#else +static unsigned long __initdata buddy_alloc_size; + +#define domain_num_llc_colors(d) 0 +#define domain_llc_color(d, i) 0 +#endif + +static void free_color_heap_page(struct page_info *pg, bool need_scrub) +{ + unsigned int color = page_to_llc_color(pg); + + free_colored_pages[color]++; + /* + * Head insertion allows re-using cache-hot pages in configurations without + * sharing of colors. + */ + page_list_add(pg, color_heap(color)); +} + +static struct page_info *alloc_color_heap_page(unsigned int memflags, + const struct domain *d) +{ + struct page_info *pg = NULL; + unsigned int i, color = 0; + unsigned long max = 0; + bool need_tlbflush = false; + uint32_t tlbflush_timestamp = 0; + bool need_scrub; + + if ( memflags & ~(MEMF_no_owner | MEMF_no_refcount | MEMF_no_scrub | + MEMF_no_tlbflush) ) + return NULL; + + spin_lock(&heap_lock); + + for ( i = 0; i < domain_num_llc_colors(d); i++ ) + { + unsigned long free = free_colored_pages[domain_llc_color(d, i)]; + + if ( free > max ) + { + color = domain_llc_color(d, i); + pg = page_list_first(color_heap(color)); + max = free; + } + } + + if ( !pg ) + { + spin_unlock(&heap_lock); + return NULL; + } + + need_scrub = pg->count_info & (PGC_need_scrub); + pg->count_info = PGC_state_inuse | (pg->count_info & PGC_preserved); + free_colored_pages[color]--; + page_list_del(pg, color_heap(color)); + + if ( !(memflags & MEMF_no_tlbflush) ) + accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp); + + init_free_page_fields(pg); + + spin_unlock(&heap_lock); + + if ( !(memflags & MEMF_no_scrub) ) + { + if ( need_scrub ) + scrub_one_page(pg); + else + check_one_page(pg); + } + + if ( need_tlbflush ) + filtered_flush_tlb_mask(tlbflush_timestamp); + + flush_page_to_ram(mfn_x(page_to_mfn(pg)), + !(memflags & MEMF_no_icache_flush)); + + return pg; +} + +static void __init init_color_heap_pages(struct page_info *pg, + unsigned long nr_pages) +{ + unsigned int i; + bool need_scrub = opt_bootscrub == BOOTSCRUB_IDLE; + + if ( buddy_alloc_size >= PAGE_SIZE ) + { + unsigned long buddy_pages = min(PFN_DOWN(buddy_alloc_size), nr_pages); + + init_heap_pages(pg, buddy_pages); + nr_pages -= buddy_pages; + buddy_alloc_size -= buddy_pages << PAGE_SHIFT; + pg += buddy_pages; + } + + if ( !_color_heap ) + { + unsigned int max_nr_colors = get_max_nr_llc_colors(); + + _color_heap = xmalloc_array(struct page_list_head, max_nr_colors); + free_colored_pages = xzalloc_array(unsigned long, max_nr_colors); + if ( !_color_heap || !free_colored_pages ) + panic("Can't allocate colored heap. Buddy reserved size is too low"); + + for ( i = 0; i < max_nr_colors; i++ ) + INIT_PAGE_LIST_HEAD(color_heap(i)); + } + + for ( i = 0; i < nr_pages; i++ ) + { + pg[i].count_info = PGC_colored; + free_color_heap_page(&pg[i], need_scrub); + } +} + +static void dump_color_heap(void) +{ + unsigned int color; + + printk("Dumping color heap info\n"); + for ( color = 0; color < get_max_nr_llc_colors(); color++ ) + if ( free_colored_pages[color] > 0 ) + printk("Color heap[%u]: %lu pages\n", + color, free_colored_pages[color]); +} + void __init end_boot_allocator(void) { unsigned int i; @@ -1964,7 +2128,13 @@ void __init end_boot_allocator(void) for ( i = nr_bootmem_regions; i-- > 0; ) { struct bootmem_region *r = &bootmem_region_list[i]; - if ( r->s < r->e ) + + if ( r->s >= r->e ) + continue; + + if ( llc_coloring_enabled ) + init_color_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s); + else init_heap_pages(mfn_to_page(_mfn(r->s)), r->e - r->s); } nr_bootmem_regions = 0; @@ -2460,7 +2630,14 @@ struct page_info *alloc_domheap_pages( if ( memflags & MEMF_no_owner ) memflags |= MEMF_no_refcount; - if ( !dma_bitsize ) + /* Only domains are supported for coloring */ + if ( d && llc_coloring_enabled ) + { + /* Colored allocation must be done on 0 order */ + if ( order || (pg = alloc_color_heap_page(memflags, d)) == NULL ) + return NULL; + } + else if ( !dma_bitsize ) memflags &= ~MEMF_no_dma; else if ( (dma_zone = bits_to_zone(dma_bitsize)) < zone_hi ) pg = alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d); @@ -2683,6 +2860,9 @@ static void cf_check dump_heap(unsigned char key) continue; printk("Node %d has %lu unscrubbed pages\n", i, node_need_scrub[i]); } + + if ( llc_coloring_enabled ) + dump_color_heap(); } static __init int cf_check register_heap_trigger(void) diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index 49ebd1e712..7f8218bfb2 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -33,6 +33,10 @@ int domain_set_llc_colors(struct domain *d, const struct xen_domctl_set_llc_colors *config); int domain_set_llc_colors_from_str(struct domain *d, const char *str); +struct page_info; +unsigned int page_to_llc_color(const struct page_info *pg); +unsigned int get_max_nr_llc_colors(void); + #endif /* __COLORING_H__ */ /* From patchwork Thu May 2 16:55:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651898 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A755BC25B78 for ; Thu, 2 May 2024 16:56:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715954.1118014 (Exim 4.92) (envelope-from ) id 1s2Zio-0000GK-9b; Thu, 02 May 2024 16:55:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715954.1118014; Thu, 02 May 2024 16:55:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zio-0000Em-1o; Thu, 02 May 2024 16:55:50 +0000 Received: by outflank-mailman (input) for mailman id 715954; Thu, 02 May 2024 16:55:47 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zil-0006N3-Ob for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:47 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d1eae1c0-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:46 +0200 (CEST) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a58a36008ceso1024925566b.0 for ; Thu, 02 May 2024 09:55:46 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:45 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d1eae1c0-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668945; x=1715273745; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F0HjLGde2czBrFNRqDQC6Fk0V43qRnD7BBzryFWROfc=; b=IvPCEZixSLI1xDOU6cbItVZYOj02qlFkX/iDCbyJeqIFEBpy1Q+VIVicj5+L/QIc44 BSXZmd/I1mD70SiMImkXQdexfD6my7rNex0qm3KlKX6iHWhY8ZOYeFPd+o84YvBe3oRA jlGMlhAeVcFO8xlqO4MfRJOS8Zla4cbyZRCHffydAkPNFwelQ3CAYEpyD6iUxOBaCv/b +fHMutPn2+1kytFBBbmV+6o9jRzLl0L/Apz0PigGPiHgpchDTGqVwU3n9rzVkI0WWAuR Y1OV3VuFCbwRo2T2fFTHsGQ3iw5g2E9MRtBKmiwEekjFy+TnqLipsRBPZkxJ2xk/8BqV Wjfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668945; x=1715273745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F0HjLGde2czBrFNRqDQC6Fk0V43qRnD7BBzryFWROfc=; b=i5QCRtl/eGMHcKU5Bx3cWdTxw1S9l/PtMP0riRg8wbqtOSHAgsMLWTNALaGLyuV+aA 7GWuCAFY6zFqFTwAj9SGd6Ghnnhjzs6Ti/LHPHhHzinfYslf488aoBntuFbHK0/N9kp6 LdjjXEO2mptp6SIqsNQIkz2uByduka6O5QgWrw04XrdYPAFy+Wl+oKMyicqTmzoYJqZE VP28onXFtDQoYe8x2M3hDkCHYicXw8XXAv2UcKlpIT50/1VfcHDbV3AHMT0z5im4BDEh BC+F2LdOoqLa3M33zvEX+x3UGLDlJTJe35Mu7d0E3RdZOxiGmiz5cEcTy2R63DQ4gTE/ Odkw== X-Gm-Message-State: AOJu0YzWXoU2tO6pI5En+IfP4vgWt+pONwpDxfWAY7wrGy+joNNOCG1e L8b38KcXticPlw5TEgyRJoldPDPSC2944JY8TzlARI0+jCDmcGuHGDGKyzQHoeIWuC9DPmwpwXG c X-Google-Smtp-Source: AGHT+IFj86uc1obXj9Kx3zkvsvH8RpMBzwS/P2urE04hZd8oYKwJRTnSvEd+QNKgfKfXBfspNETP/w== X-Received: by 2002:a17:906:eca8:b0:a58:c639:9518 with SMTP id qh8-20020a170906eca800b00a58c6399518mr47832ejb.76.1714668945596; Thu, 02 May 2024 09:55:45 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 10/13] xen/arm: use domain memory to allocate p2m page tables Date: Thu, 2 May 2024 18:55:30 +0200 Message-Id: <20240502165533.319988-11-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Cache colored domains can benefit from having p2m page tables allocated with the same coloring schema so that isolation can be achieved also for those kind of memory accesses. In order to do that, the domain struct is passed to the allocator and the MEMF_no_owner flag is used. This will be useful also when NUMA will be supported on Arm. Signed-off-by: Carlo Nonato Acked-by: Julien Grall --- v8: - no changes v7: - no changes v6: - Carlo Nonato as the only signed-off-by v5: - new patch --- xen/arch/arm/mmu/p2m.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c index 41fcca011c..d02a478cb8 100644 --- a/xen/arch/arm/mmu/p2m.c +++ b/xen/arch/arm/mmu/p2m.c @@ -32,7 +32,7 @@ static struct page_info *p2m_alloc_page(struct domain *d) */ if ( is_hardware_domain(d) ) { - pg = alloc_domheap_page(NULL, 0); + pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) printk(XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n"); } @@ -81,7 +81,7 @@ int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted) if ( d->arch.paging.p2m_total_pages < pages ) { /* Need to allocate more memory from domheap */ - pg = alloc_domheap_page(NULL, 0); + pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) { printk(XENLOG_ERR "Failed to allocate P2M pages.\n"); From patchwork Thu May 2 16:55:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD157C25B5C for ; Thu, 2 May 2024 16:56:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715956.1118021 (Exim 4.92) (envelope-from ) id 1s2Zip-0000PR-20; Thu, 02 May 2024 16:55:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715956.1118021; Thu, 02 May 2024 16:55:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zio-0000NP-MH; Thu, 02 May 2024 16:55:50 +0000 Received: by outflank-mailman (input) for mailman id 715956; Thu, 02 May 2024 16:55:48 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zim-0006N3-Ow for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:48 +0000 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [2a00:1450:4864:20::62f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d254c69c-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:46 +0200 (CEST) Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-a58a36008ceso1024928566b.0 for ; Thu, 02 May 2024 09:55:46 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:46 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d254c69c-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668946; x=1715273746; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N2LM23qNbULSHEplCwn2ygOn0A5pB1f/lEILoWDwehY=; b=EdhfhOOl01LJETvsGwmmXcWY/iJqkxkb1SF6GlwzkCVjor1eauEh/lWdehSJx7a0x9 OY1l/w3PrF+/WnydKz7nyyloQw3UN5j4bvbVTASYnUHcbKMO8mVMHX+eiINN1gtVTRTS CQZGsggcUbEurwfTOt21+pDImh56p6k9bHy6wotISDRX5voS3yKhlxsy5zRpDA5Sa7H+ DcQaa15gmFdPQSfdSwYPvfgw9l2Jx/zN7M64jUmoA1VazzIe/iwvgVzApT6VAhLvyN8M hsVpw862VfuqMq3J9HYVySbVAKtI9cE1ati9KbH2I0LU9bmNqJnsKlDskPZSW85w/krR YYNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668946; x=1715273746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N2LM23qNbULSHEplCwn2ygOn0A5pB1f/lEILoWDwehY=; b=A8063wtrAGAol44CPP3ppYahdjkwdtyx86X3hdce3TOdPWuUUEtGClCedLvvr3IZhC BiU5KEftGYQFtAR89lrAdeamVRyA+JHCC0CPsu8fdCkp7mca52CQjteQPPfQxXfhSRBr 3GcABCzCtSmAL4YLRO+emOo+TwXU+76o2sA/byjoC0kCl05zwUXUftC5MftHpAfBmG7W O0p3z1GtyzgIylJivrIaJ+unp/wSo0Z/UBQe/1iXdedC5aKM3UnVw8/BgDKhZMIuWf0k KEh5Z2X3BGNUfwBDECu28CFPZyy1M/iiSf33kLlasU9vAU3yAfpLbIIHFzuo/7YuS4ur Oe2Q== X-Gm-Message-State: AOJu0Yxv8Or+oNWTg9ptj3naAvhMHjLxyH9uJNw/tmu5eGDuEV5ybqj3 Yg/xAIK+51MJVI7HStiXyNeIHQHu3E06lMvLi/wT1t4MHZ+QpFEqaytnkZcGHjTHXH1eaRL/n+n Q X-Google-Smtp-Source: AGHT+IF4jfiVFArle54ykP0fTOLLkWR9rD9jCanhWbQyFsRvUeSfTo1P5MXi9h0x0Akki+DHOaZYOA== X-Received: by 2002:a17:906:a159:b0:a58:e71d:d74 with SMTP id bu25-20020a170906a15900b00a58e71d0d74mr73203ejb.13.1714668946329; Thu, 02 May 2024 09:55:46 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Luca Miccio , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Marco Solieri , Carlo Nonato Subject: [PATCH v8 11/13] xen/arm: add Xen cache colors command line parameter Date: Thu, 2 May 2024 18:55:31 +0200 Message-Id: <20240502165533.319988-12-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 From: Luca Miccio Add a new command line parameter to configure Xen cache colors. These colors are dumped together with other coloring info. Benchmarking the VM interrupt response time provides an estimation of LLC usage by Xen's most latency-critical runtime task. Results on Arm Cortex-A53 on Xilinx Zynq UltraScale+ XCZU9EG show that one color, which reserves 64 KiB of L2, is enough to attain best responsiveness: - Xen 1 color latency: 3.1 us - Xen 2 color latency: 3.1 us Since this is the most common target for Arm cache coloring, the default amount of Xen colors is set to one. More colors are instead very likely to be needed on processors whose L1 cache is physically-indexed and physically-tagged, such as Cortex-A57. In such cases, coloring applies to L1 also, and there typically are two distinct L1-colors. Therefore, reserving only one color for Xen would senselessly partitions a cache memory that is already private, i.e. underutilize it. Signed-off-by: Luca Miccio Signed-off-by: Marco Solieri Signed-off-by: Carlo Nonato Reviewed-by: Jan Beulich --- v8: - added bound check on xen_colors in llc_coloring_init() v7: - removed XEN_DEFAULT_COLOR - XEN_DEFAULT_NUM_COLORS is now used in a for loop to set xen default colors --- docs/misc/cache-coloring.rst | 2 ++ docs/misc/xen-command-line.pandoc | 10 ++++++++++ xen/common/llc-coloring.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+) diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst index f996627d0d..ad42d6b811 100644 --- a/docs/misc/cache-coloring.rst +++ b/docs/misc/cache-coloring.rst @@ -113,6 +113,8 @@ Specific documentation is available at `docs/misc/xen-command-line.pandoc`. +----------------------+-------------------------------+ | ``buddy-alloc-size`` | Buddy allocator reserved size | +----------------------+-------------------------------+ +| ``xen-llc-colors`` | Xen color configuration | ++----------------------+-------------------------------+ Colors selection format *********************** diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index d4d1e37272..2427f865c5 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2915,6 +2915,16 @@ mode. **WARNING: `x2apic_phys` is deprecated and superseded by `x2apic-mode`. The latter takes precedence if both are set.** +### xen-llc-colors (arm64) +> `= List of [ | - ]` + +> Default: `0: the lowermost color` + +Specify Xen LLC color configuration. This options is available only when +`CONFIG_LLC_COLORING` is enabled. +Two colors are most likely needed on platforms where private caches are +physically indexed, e.g. the L1 instruction cache of the Arm Cortex-A57. + ### xenheap_megabytes (arm32) > `= ` diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index c598eb4bf7..0ddd6946f1 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -11,6 +11,7 @@ #include #define NR_LLC_COLORS (1 << CONFIG_MAX_LLC_COLORS_ORDER) +#define XEN_DEFAULT_NUM_COLORS 1 bool __ro_after_init llc_coloring_enabled; boolean_param("llc-coloring", llc_coloring_enabled); @@ -27,6 +28,9 @@ static unsigned int __ro_after_init default_colors[NR_LLC_COLORS]; static unsigned int __initdata dom0_colors[NR_LLC_COLORS]; static unsigned int __initdata dom0_num_colors; +static unsigned int __ro_after_init xen_colors[NR_LLC_COLORS]; +static unsigned int __ro_after_init xen_num_colors; + #define mfn_color_mask (max_nr_colors - 1) #define mfn_to_color(mfn) (mfn_x(mfn) & mfn_color_mask) @@ -85,6 +89,13 @@ static int __init parse_dom0_colors(const char *s) } custom_param("dom0-llc-colors", parse_dom0_colors); +static int __init parse_xen_colors(const char *s) +{ + return parse_color_config(s, xen_colors, ARRAY_SIZE(xen_colors), + &xen_num_colors); +} +custom_param("xen-llc-colors", parse_xen_colors); + static void print_colors(const unsigned int *colors, unsigned int num_colors) { unsigned int i; @@ -163,6 +174,22 @@ void __init llc_coloring_init(void) for ( i = 0; i < max_nr_colors; i++ ) default_colors[i] = i; + if ( !xen_num_colors ) + { + unsigned int i; + + xen_num_colors = MIN(XEN_DEFAULT_NUM_COLORS, max_nr_colors); + + printk(XENLOG_WARNING + "Xen LLC color config not found. Using first %u colors\n", + xen_num_colors); + for ( i = 0; i < xen_num_colors; i++ ) + xen_colors[i] = i; + } + else if ( xen_num_colors > max_nr_colors || + !check_colors(xen_colors, xen_num_colors) ) + panic("Bad LLC color config for Xen\n"); + arch_llc_coloring_init(); } @@ -173,6 +200,8 @@ void dump_llc_coloring_info(void) printk("LLC coloring info:\n"); printk(" Number of LLC colors supported: %u\n", max_nr_colors); + printk(" Xen LLC colors (%u): ", xen_num_colors); + print_colors(xen_colors, xen_num_colors); } void domain_dump_llc_colors(const struct domain *d) From patchwork Thu May 2 16:55:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C2F8C25B10 for ; Thu, 2 May 2024 16:56:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715957.1118029 (Exim 4.92) (envelope-from ) id 1s2Ziq-0000bD-1T; Thu, 02 May 2024 16:55:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715957.1118029; Thu, 02 May 2024 16:55:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zip-0000ZI-Je; Thu, 02 May 2024 16:55:51 +0000 Received: by outflank-mailman (input) for mailman id 715957; Thu, 02 May 2024 16:55:49 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zin-0006N3-Oz for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:49 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d2f792ba-08a4-11ef-b4bb-af5377834399; Thu, 02 May 2024 18:55:48 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a595c61553cso245322166b.1 for ; Thu, 02 May 2024 09:55:48 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:46 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d2f792ba-08a4-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668947; x=1715273747; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=js2dKSvePAqOWBjeqArRJcnowTogkyqLrYPgrRYmXag=; b=RRXcZgWPMNtOTswwIbUkP7F+MWzCBWMztxI3r820+EGbYvBRm2BOB945+xFe3uMGJd RuQ+1ba75qFq+90sqIv/VU3LijYiEKPfikMRiExZ0P+ySe5C3oR4ykYSnVT6FnTmhA3i VmrGVC3SAwhBRgkPP62xIYv/W7Uu+umMIxo3DzrjPH6ck+0ZF6BRlRwSEoz48yULKmm6 tqj2dtNz5fM2nHKpyvgr8OkUlntuHQXEUmkmQX3IoJ3Eyn60nXIWkzj3DeqIzxjl+OiF 1VE+A0gdixMuVkYoqr0bZfd3ZGmUh933f6Xduv0fB8cd/Zc9J7KS+/0eL0aIIMdJMPra n4Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668947; x=1715273747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=js2dKSvePAqOWBjeqArRJcnowTogkyqLrYPgrRYmXag=; b=TlOPt6rjg5t72vDLtsL7o+Faro/eQF0x2mhXdIZuBy3AE5ky8oF7RtqZCbBrKrB3GY F12r5GSZ1ClN3CipXsJuKcuNdxyAdBmv0HQJKxDCQeidd0A3O+Dvb4RK4u1uwmrMxeFS LVPm0JshHa0iv8k5c6IaeDTXVoVD1lAp8aJAscip3rqfxc7stIGE+gqjkGc0clipEcLl Y6WEv6TsY3iW1PTIDnWEUegEnQ1BTSfhUOLnwSqnUfKDwW6f5OrEYdQR1P16L7FlQgpk LTaxY3dWowC9KNk9OJLrFXcAA7POTdAtVVKZp9Pvn7fTcROCj2NcLedaSQwJUHOg+vpn Suuw== X-Gm-Message-State: AOJu0YyXLAwjEkBhtFCn6pvQwmdG0NM/VmTqbi+M8wY4sH4csOiA09mp uPM9hp+A1/WiDfojJkKGIWJ2C2K5RKJsSJm2GJVYvOYoc18x/x1LsXRF1rNKLaFK/VgObsZexlQ W X-Google-Smtp-Source: AGHT+IHgyNFFlUmdIrnd/ng6aOPddiEn0HB2PTkDujnG/vTVRVBCqMLVj1yOyJ1Lyw3K+l8ARC4uGg== X-Received: by 2002:a17:906:fcc3:b0:a59:7766:e4ed with SMTP id qx3-20020a170906fcc300b00a597766e4edmr70061ejb.32.1714668947016; Thu, 02 May 2024 09:55:47 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v8 12/13] xen/arm: make consider_modules() available for xen relocation Date: Thu, 2 May 2024 18:55:32 +0200 Message-Id: <20240502165533.319988-13-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Cache coloring must physically relocate Xen in order to color the hypervisor and consider_modules() is a key function that is needed to find a new available physical address. 672d67f339c0 ("xen/arm: Split MMU-specific setup_mm() and related code out") moved consider_modules() under arm32. Move it to mmu/setup.c and make it non-static so that it can be used outside. Signed-off-by: Carlo Nonato --- v8: - patch adapted to new changes to consider_modules() v7: - moved consider_modules() to arm/mmu/setup.c v6: - new patch --- xen/arch/arm/arm32/mmu/mm.c | 115 +----------------------------- xen/arch/arm/include/asm/setup.h | 3 + xen/arch/arm/mmu/setup.c | 117 +++++++++++++++++++++++++++++++ 3 files changed, 121 insertions(+), 114 deletions(-) diff --git a/xen/arch/arm/arm32/mmu/mm.c b/xen/arch/arm/arm32/mmu/mm.c index 23150122f7..5d50db32eb 100644 --- a/xen/arch/arm/arm32/mmu/mm.c +++ b/xen/arch/arm/arm32/mmu/mm.c @@ -9,6 +9,7 @@ #include #include #include +#include static unsigned long opt_xenheap_megabytes __initdata; integer_param("xenheap_megabytes", opt_xenheap_megabytes); @@ -31,120 +32,6 @@ static void __init setup_directmap_mappings(unsigned long base_mfn, directmap_virt_end = XENHEAP_VIRT_START + nr_mfns * PAGE_SIZE; } -/* - * Returns the end address of the highest region in the range s..e - * with required size and alignment that does not conflict with the - * modules from first_mod to nr_modules. - * - * For non-recursive callers first_mod should normally be 0 (all - * modules and Xen itself) or 1 (all modules but not Xen). - */ -static paddr_t __init consider_modules(paddr_t s, paddr_t e, - uint32_t size, paddr_t align, - int first_mod) -{ - const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); -#ifdef CONFIG_STATIC_SHM - const struct membanks *shmem = bootinfo_get_shmem(); -#endif - const struct bootmodules *mi = &bootinfo.modules; - int i; - int nr; - - s = (s+align-1) & ~(align-1); - e = e & ~(align-1); - - if ( s > e || e - s < size ) - return 0; - - /* First check the boot modules */ - for ( i = first_mod; i < mi->nr_mods; i++ ) - { - paddr_t mod_s = mi->module[i].start; - paddr_t mod_e = mod_s + mi->module[i].size; - - if ( s < mod_e && mod_s < e ) - { - mod_e = consider_modules(mod_e, e, size, align, i+1); - if ( mod_e ) - return mod_e; - - return consider_modules(s, mod_s, size, align, i+1); - } - } - - /* Now check any fdt reserved areas. */ - - nr = fdt_num_mem_rsv(device_tree_flattened); - - for ( ; i < mi->nr_mods + nr; i++ ) - { - paddr_t mod_s, mod_e; - - if ( fdt_get_mem_rsv_paddr(device_tree_flattened, - i - mi->nr_mods, - &mod_s, &mod_e ) < 0 ) - /* If we can't read it, pretend it doesn't exist... */ - continue; - - /* fdt_get_mem_rsv_paddr returns length */ - mod_e += mod_s; - - if ( s < mod_e && mod_s < e ) - { - mod_e = consider_modules(mod_e, e, size, align, i+1); - if ( mod_e ) - return mod_e; - - return consider_modules(s, mod_s, size, align, i+1); - } - } - - /* - * i is the current bootmodule we are evaluating, across all - * possible kinds of bootmodules. - * - * When retrieving the corresponding reserved-memory addresses, we - * need to index the reserved_mem bank starting from 0, and only counting - * the reserved-memory modules. Hence, we need to use i - nr. - */ - nr += mi->nr_mods; - for ( ; i - nr < reserved_mem->nr_banks; i++ ) - { - paddr_t r_s = reserved_mem->bank[i - nr].start; - paddr_t r_e = r_s + reserved_mem->bank[i - nr].size; - - if ( s < r_e && r_s < e ) - { - r_e = consider_modules(r_e, e, size, align, i + 1); - if ( r_e ) - return r_e; - - return consider_modules(s, r_s, size, align, i + 1); - } - } - -#ifdef CONFIG_STATIC_SHM - nr += reserved_mem->nr_banks; - for ( ; i - nr < shmem->nr_banks; i++ ) - { - paddr_t r_s = shmem->bank[i - nr].start; - paddr_t r_e = r_s + shmem->bank[i - nr].size; - - if ( s < r_e && r_s < e ) - { - r_e = consider_modules(r_e, e, size, align, i + 1); - if ( r_e ) - return r_e; - - return consider_modules(s, r_s, size, align, i + 1); - } - } -#endif - - return e; -} - /* * Find a contiguous region that fits in the static heap region with * required size and alignment, and return the end address of the region diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 28fb659fe9..a72b6d5c63 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -278,6 +278,9 @@ struct init_info unsigned int cpuid; }; +paddr_t consider_modules(paddr_t s, paddr_t e, uint32_t size, paddr_t align, + int first_mod); + #endif /* * Local variables: diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c index f4bb424c3c..a8f93d4f82 100644 --- a/xen/arch/arm/mmu/setup.c +++ b/xen/arch/arm/mmu/setup.c @@ -6,7 +6,10 @@ */ #include +#include #include +#include +#include #include #include #include @@ -218,6 +221,120 @@ static void xen_pt_enforce_wnx(void) flush_xen_tlb_local(); } +/* + * Returns the end address of the highest region in the range s..e + * with required size and alignment that does not conflict with the + * modules from first_mod to nr_modules. + * + * For non-recursive callers first_mod should normally be 0 (all + * modules and Xen itself) or 1 (all modules but not Xen). + */ +paddr_t __init consider_modules(paddr_t s, paddr_t e, + uint32_t size, paddr_t align, + int first_mod) +{ + const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); +#ifdef CONFIG_STATIC_SHM + const struct membanks *shmem = bootinfo_get_shmem(); +#endif + const struct bootmodules *mi = &bootinfo.modules; + int i; + int nr; + + s = (s+align-1) & ~(align-1); + e = e & ~(align-1); + + if ( s > e || e - s < size ) + return 0; + + /* First check the boot modules */ + for ( i = first_mod; i < mi->nr_mods; i++ ) + { + paddr_t mod_s = mi->module[i].start; + paddr_t mod_e = mod_s + mi->module[i].size; + + if ( s < mod_e && mod_s < e ) + { + mod_e = consider_modules(mod_e, e, size, align, i+1); + if ( mod_e ) + return mod_e; + + return consider_modules(s, mod_s, size, align, i+1); + } + } + + /* Now check any fdt reserved areas. */ + + nr = fdt_num_mem_rsv(device_tree_flattened); + + for ( ; i < mi->nr_mods + nr; i++ ) + { + paddr_t mod_s, mod_e; + + if ( fdt_get_mem_rsv_paddr(device_tree_flattened, + i - mi->nr_mods, + &mod_s, &mod_e ) < 0 ) + /* If we can't read it, pretend it doesn't exist... */ + continue; + + /* fdt_get_mem_rsv_paddr returns length */ + mod_e += mod_s; + + if ( s < mod_e && mod_s < e ) + { + mod_e = consider_modules(mod_e, e, size, align, i+1); + if ( mod_e ) + return mod_e; + + return consider_modules(s, mod_s, size, align, i+1); + } + } + + /* + * i is the current bootmodule we are evaluating, across all + * possible kinds of bootmodules. + * + * When retrieving the corresponding reserved-memory addresses, we + * need to index the reserved_mem bank starting from 0, and only counting + * the reserved-memory modules. Hence, we need to use i - nr. + */ + nr += mi->nr_mods; + for ( ; i - nr < reserved_mem->nr_banks; i++ ) + { + paddr_t r_s = reserved_mem->bank[i - nr].start; + paddr_t r_e = r_s + reserved_mem->bank[i - nr].size; + + if ( s < r_e && r_s < e ) + { + r_e = consider_modules(r_e, e, size, align, i + 1); + if ( r_e ) + return r_e; + + return consider_modules(s, r_s, size, align, i + 1); + } + } + +#ifdef CONFIG_STATIC_SHM + nr += reserved_mem->nr_banks; + for ( ; i - nr < shmem->nr_banks; i++ ) + { + paddr_t r_s = shmem->bank[i - nr].start; + paddr_t r_e = r_s + shmem->bank[i - nr].size; + + if ( s < r_e && r_s < e ) + { + r_e = consider_modules(r_e, e, size, align, i + 1); + if ( r_e ) + return r_e; + + return consider_modules(s, r_s, size, align, i + 1); + } + } +#endif + + return e; +} + /* * Boot-time pagetable setup. * Changes here may need matching changes in head.S From patchwork Thu May 2 16:55:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlo Nonato X-Patchwork-Id: 13651900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B14FEC25B79 for ; Thu, 2 May 2024 16:56:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.715958.1118035 (Exim 4.92) (envelope-from ) id 1s2Ziq-0000gl-Dy; Thu, 02 May 2024 16:55:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 715958.1118035; Thu, 02 May 2024 16:55:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Ziq-0000f9-1M; Thu, 02 May 2024 16:55:52 +0000 Received: by outflank-mailman (input) for mailman id 715958; Thu, 02 May 2024 16:55:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s2Zin-0006Lf-UO for xen-devel@lists.xenproject.org; Thu, 02 May 2024 16:55:50 +0000 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [2a00:1450:4864:20::631]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d35bf878-08a4-11ef-909b-e314d9c70b13; Thu, 02 May 2024 18:55:48 +0200 (CEST) Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-a587831809eso1015932966b.1 for ; Thu, 02 May 2024 09:55:48 -0700 (PDT) Received: from carlo-ubuntu.mo54.unimo.it (nonato.mo54.unimo.it. [155.185.85.8]) by smtp.gmail.com with ESMTPSA id mq30-20020a170907831e00b00a5987fbfb83sm29103ejc.152.2024.05.02.09.55.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 09:55:47 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d35bf878-08a4-11ef-909b-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1714668948; x=1715273748; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WECgJiEg23J0x5KrTcZN6y0EvfDCy6GafmGxq0QIyEk=; b=lKJSluUBuvLwkgOk1Qq3XHcz7etstDbGtFKlkYZ0y/7mz47mk6V9M09vgOc+63JFSh zGafN/fwLNfnmXz9ZhiJwNzK03NbIjWSfuID6V8jjUILwaWyd0mmXR65EQOK1pR6dEvo SU6qNhs6koDQHvlQw27lLtRO5u3rXQPVqOWVOaPgnmH9np3lOu901EQhc5C57Gc0QtSI yHnzmvkQF9cEAU9kuOsokkJC76sjykTwxZE6TJvUaiAojHLTw3y/zSbMX0x6IADnq3oW wnXauT4lgMMqR/EPDPov9sFIaa013ufKQ+TH5iCzTcu65An76Srn+x7o+dbboo9FJq+u E8Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714668948; x=1715273748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WECgJiEg23J0x5KrTcZN6y0EvfDCy6GafmGxq0QIyEk=; b=UpuV5KR0nZFJOQ/h0bOyUv/9soOGoLV39HPaj1Ipb9Fa/FaVsnmzHq2F2w7+DmGm3t YB9MMl7HlGaTmE3w8yMxldc6AP4Nnwc1MrVbDmxQHAB9XW12qV4ELjPVj4pVN/x4neKR 2J+JoYDRxmtkU7A5lXGSU7YEogiQM/Muo45XCFbG6ZYrneizhKBe9HHtfTBtkiTZ7vXw nzUhcXs/642F84yz8xyJw3IHVsQpyclMI2f1Yk+o+0SNx19jo79ZxObmbUatXKDglM7S JAeYroFVu83w32kfAf1+TKZJ10Gni1L0XsTEuIOxA14ozCvta3NoQJdUc2lhUjeIzFfY gJ+w== X-Gm-Message-State: AOJu0YyQD/u1IZI9P0rHdpoW/j0xs9N697JnBAgfin0ayJoHSoEQnfkS Ga6Art1xvrnK+fLiishaH+zlJO66R4xVJu661w4ig69VaBFxH5aDUyKXEnHBMSvRX+rmpTkwLFu v X-Google-Smtp-Source: AGHT+IFxhylZqKSSf0raeqaYai/+rXwGChzuHGphLorcC4+60j+kdNQeosjYOMCjdKBfd3yfWGD+5g== X-Received: by 2002:a17:906:6c1:b0:a58:fabc:4a02 with SMTP id v1-20020a17090606c100b00a58fabc4a02mr48827ejb.39.1714668947881; Thu, 02 May 2024 09:55:47 -0700 (PDT) From: Carlo Nonato To: xen-devel@lists.xenproject.org Cc: andrea.bastoni@minervasys.tech, Carlo Nonato , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Marco Solieri Subject: [PATCH v8 13/13] xen/arm: add cache coloring support for Xen Date: Thu, 2 May 2024 18:55:33 +0200 Message-Id: <20240502165533.319988-14-carlo.nonato@minervasys.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240502165533.319988-1-carlo.nonato@minervasys.tech> References: <20240502165533.319988-1-carlo.nonato@minervasys.tech> MIME-Version: 1.0 Add the cache coloring support for Xen physical space. Since Xen must be relocated to a new physical space, some relocation functionalities must be brought back: - the virtual address of the new space is taken from 0c18fb76323b ("xen/arm: Remove unused BOOT_RELOC_VIRT_START"). - relocate_xen() and get_xen_paddr() are taken from f60658c6ae47 ("xen/arm: Stop relocating Xen"). setup_pagetables() must be adapted for coloring and for relocation. Runtime page tables are used to map the colored space, but they are also linked in boot tables so that the new space is temporarily available for relocation. This implies that Xen protection must happen after the copy. Finally, since the alternative framework needs to remap the Xen text and inittext sections, this operation must be done in a coloring-aware way. The function xen_remap_colored() is introduced for that. Signed-off-by: Carlo Nonato Signed-off-by: Marco Solieri --- v8: - moved xen_colored_map_size() to arm/llc-coloring.c v7: - added BUG_ON() checks to arch_llc_coloring_init() and create_llc_coloring_mappings() v6: - squashed with BOOT_RELOC_VIRT_START patch - consider_modules() moved in another patch - removed psci and smpboot code because of new idmap work already handles that - moved xen_remap_colored() in alternative.c since it's only used there - removed xen_colored_temp[] in favor of xen_xenmap[] usage for mapping - use of boot_module_find_by_kind() to remove the need of extra parameter in setup_pagetables() - moved get_xen_paddr() in arm/llc-coloring.c since it's only used there v5: - FIXME: consider_modules copy pasted since it got moved v4: - removed set_value_for_secondary() because it was wrongly cleaning cache - relocate_xen() now calls switch_ttbr_id() --- xen/arch/arm/alternative.c | 30 +++++++- xen/arch/arm/arm64/mmu/head.S | 58 +++++++++++++- xen/arch/arm/arm64/mmu/mm.c | 28 ++++++- xen/arch/arm/include/asm/mmu/layout.h | 3 + xen/arch/arm/llc-coloring.c | 63 ++++++++++++++++ xen/arch/arm/mmu/setup.c | 104 ++++++++++++++++++++++---- xen/arch/arm/setup.c | 10 ++- xen/common/llc-coloring.c | 18 +++++ xen/include/xen/llc-coloring.h | 13 ++++ 9 files changed, 305 insertions(+), 22 deletions(-) diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c index 016e66978b..8ca649b55e 100644 --- a/xen/arch/arm/alternative.c +++ b/xen/arch/arm/alternative.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -191,6 +192,27 @@ static int __apply_alternatives_multi_stop(void *xenmap) return 0; } +static void __init *xen_remap_colored(mfn_t xen_mfn, paddr_t xen_size) +{ + unsigned int i; + void *xenmap; + mfn_t *xen_colored_mfns, mfn; + + xen_colored_mfns = xmalloc_array(mfn_t, xen_size >> PAGE_SHIFT); + if ( !xen_colored_mfns ) + panic("Can't allocate LLC colored MFNs\n"); + + for_each_xen_colored_mfn ( xen_mfn, mfn, i ) + { + xen_colored_mfns[i] = mfn; + } + + xenmap = vmap(xen_colored_mfns, xen_size >> PAGE_SHIFT); + xfree(xen_colored_mfns); + + return xenmap; +} + /* * This function should only be called during boot and before CPU0 jump * into the idle_loop. @@ -209,8 +231,12 @@ void __init apply_alternatives_all(void) * The text and inittext section are read-only. So re-map Xen to * be able to patch the code. */ - xenmap = __vmap(&xen_mfn, 1U << xen_order, 1, 1, PAGE_HYPERVISOR, - VMAP_DEFAULT); + if ( llc_coloring_enabled ) + xenmap = xen_remap_colored(xen_mfn, xen_size); + else + xenmap = __vmap(&xen_mfn, 1U << xen_order, 1, 1, PAGE_HYPERVISOR, + VMAP_DEFAULT); + /* Re-mapping Xen is not expected to fail during boot. */ BUG_ON(!xenmap); diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S index fa40b696dd..7ad2c00fd5 100644 --- a/xen/arch/arm/arm64/mmu/head.S +++ b/xen/arch/arm/arm64/mmu/head.S @@ -427,6 +427,61 @@ fail: PRINT("- Boot failed -\r\n") b 1b ENDPROC(fail) +/* + * Copy Xen to new location and switch TTBR + * x0 ttbr + * x1 source address + * x2 destination address + * x3 length + * + * Source and destination must be word aligned, length is rounded up + * to a 16 byte boundary. + * + * MUST BE VERY CAREFUL when saving things to RAM over the copy + */ +ENTRY(relocate_xen) + /* + * Copy 16 bytes at a time using: + * x9: counter + * x10: data + * x11: data + * x12: source + * x13: destination + */ + mov x9, x3 + mov x12, x1 + mov x13, x2 + +1: ldp x10, x11, [x12], #16 + stp x10, x11, [x13], #16 + + subs x9, x9, #16 + bgt 1b + + /* + * Flush destination from dcache using: + * x9: counter + * x10: step + * x11: vaddr + * + * This is to ensure data is visible to the instruction cache + */ + dsb sy + + mov x9, x3 + ldr x10, =dcache_line_bytes /* x10 := step */ + ldr x10, [x10] + mov x11, x2 + +1: dc cvac, x11 + + add x11, x11, x10 + subs x9, x9, x10 + bgt 1b + + /* No need for dsb/isb because they are alredy done in switch_ttbr_id */ + b switch_ttbr_id + /* * Switch TTBR * @@ -452,7 +507,8 @@ ENTRY(switch_ttbr_id) /* * 5) Flush I-cache - * This should not be necessary but it is kept for safety. + * This should not be necessary in the general case, but it's needed + * for cache coloring because code is relocated in that case. */ ic iallu isb diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index 293acb67e0..076264b541 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include +#include #include #include @@ -126,27 +127,46 @@ void update_identity_mapping(bool enable) } extern void switch_ttbr_id(uint64_t ttbr); +extern void relocate_xen(uint64_t ttbr, void *src, void *dst, size_t len); typedef void (switch_ttbr_fn)(uint64_t ttbr); +typedef void (relocate_xen_fn)(uint64_t ttbr, void *src, void *dst, size_t len); void __init switch_ttbr(uint64_t ttbr) { - vaddr_t id_addr = virt_to_maddr(switch_ttbr_id); - switch_ttbr_fn *fn = (switch_ttbr_fn *)id_addr; + vaddr_t vaddr, id_addr; lpae_t pte; + if ( llc_coloring_enabled ) + vaddr = (vaddr_t)relocate_xen; + else + vaddr = (vaddr_t)switch_ttbr_id; + + id_addr = virt_to_maddr(vaddr); + /* Enable the identity mapping in the boot page tables */ update_identity_mapping(true); /* Enable the identity mapping in the runtime page tables */ - pte = pte_of_xenaddr((vaddr_t)switch_ttbr_id); + pte = pte_of_xenaddr(vaddr); pte.pt.table = 1; pte.pt.xn = 0; pte.pt.ro = 1; write_pte(&xen_third_id[third_table_offset(id_addr)], pte); /* Switch TTBR */ - fn(ttbr); + if ( llc_coloring_enabled ) + { + relocate_xen_fn *fn = (relocate_xen_fn *)id_addr; + + fn(ttbr, _start, (void *)BOOT_RELOC_VIRT_START, _end - _start); + } + else + { + switch_ttbr_fn *fn = (switch_ttbr_fn *)id_addr; + + fn(ttbr); + } /* * Disable the identity mapping in the runtime page tables. diff --git a/xen/arch/arm/include/asm/mmu/layout.h b/xen/arch/arm/include/asm/mmu/layout.h index a3b546465b..19c0ec63a5 100644 --- a/xen/arch/arm/include/asm/mmu/layout.h +++ b/xen/arch/arm/include/asm/mmu/layout.h @@ -30,6 +30,7 @@ * 10M - 12M Fixmap: special-purpose 4K mapping slots * 12M - 16M Early boot mapping of FDT * 16M - 18M Livepatch vmap (if compiled in) + * 16M - 24M Cache-colored Xen text, data, bss (temporary, if compiled in) * * 1G - 2G VMAP: ioremap and early_ioremap * @@ -74,6 +75,8 @@ #define BOOT_FDT_VIRT_START (FIXMAP_VIRT_START + FIXMAP_VIRT_SIZE) #define BOOT_FDT_VIRT_SIZE _AT(vaddr_t, MB(4)) +#define BOOT_RELOC_VIRT_START (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE) + #ifdef CONFIG_LIVEPATCH #define LIVEPATCH_VMAP_START (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE) #define LIVEPATCH_VMAP_SIZE _AT(vaddr_t, MB(2)) diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c index 66c8db2baf..c8b183da84 100644 --- a/xen/arch/arm/llc-coloring.c +++ b/xen/arch/arm/llc-coloring.c @@ -9,6 +9,7 @@ #include #include +#include /* Return the LLC way size by probing the hardware */ unsigned int __init get_llc_way_size(void) @@ -62,8 +63,70 @@ unsigned int __init get_llc_way_size(void) return line_size * num_sets; } +/** + * get_xen_paddr - get physical address to relocate Xen to + * + * Xen is relocated to as near to the top of RAM as possible and + * aligned to a XEN_PADDR_ALIGN boundary. + */ +static paddr_t __init get_xen_paddr(paddr_t xen_size) +{ + const struct membanks *mem = bootinfo_get_mem(); + paddr_t min_size, paddr = 0; + unsigned int i; + + min_size = (xen_size + (XEN_PADDR_ALIGN-1)) & ~(XEN_PADDR_ALIGN-1); + + /* Find the highest bank with enough space. */ + for ( i = 0; i < mem->nr_banks; i++ ) + { + const struct membank *bank = &mem->bank[i]; + paddr_t s, e; + + if ( bank->size >= min_size ) + { + e = consider_modules(bank->start, bank->start + bank->size, + min_size, XEN_PADDR_ALIGN, 0); + if ( !e ) + continue; + +#ifdef CONFIG_ARM_32 + /* Xen must be under 4GB */ + if ( e > GB(4) ) + e = GB(4); + if ( e < bank->start ) + continue; +#endif + + s = e - min_size; + + if ( s > paddr ) + paddr = s; + } + } + + if ( !paddr ) + panic("Not enough memory to relocate Xen\n"); + + printk("Placing Xen at 0x%"PRIpaddr"-0x%"PRIpaddr"\n", + paddr, paddr + min_size); + + return paddr; +} + +static paddr_t __init xen_colored_map_size(void) +{ + return ROUNDUP((_end - _start) * get_max_nr_llc_colors(), XEN_PADDR_ALIGN); +} + void __init arch_llc_coloring_init(void) { + struct bootmodule *xen_bootmodule = boot_module_find_by_kind(BOOTMOD_XEN); + + BUG_ON(!xen_bootmodule); + + xen_bootmodule->size = xen_colored_map_size(); + xen_bootmodule->start = get_xen_paddr(xen_bootmodule->size); } /* diff --git a/xen/arch/arm/mmu/setup.c b/xen/arch/arm/mmu/setup.c index a8f93d4f82..1a1b7f58a6 100644 --- a/xen/arch/arm/mmu/setup.c +++ b/xen/arch/arm/mmu/setup.c @@ -20,6 +20,11 @@ /* Override macros from asm/page.h to make them work with mfn_t */ #undef mfn_to_virt #define mfn_to_virt(mfn) __mfn_to_virt(mfn_x(mfn)) +#undef virt_to_mfn +#define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) + +#define virt_to_reloc_virt(virt) \ + (((vaddr_t)virt) - XEN_VIRT_START + BOOT_RELOC_VIRT_START) /* Main runtime page tables */ @@ -72,6 +77,7 @@ static void __init __maybe_unused build_assertions(void) /* 2MB aligned regions */ BUILD_BUG_ON(XEN_VIRT_START & ~SECOND_MASK); BUILD_BUG_ON(FIXMAP_ADDR(0) & ~SECOND_MASK); + BUILD_BUG_ON(BOOT_RELOC_VIRT_START & ~SECOND_MASK); /* 1GB aligned regions */ #ifdef CONFIG_ARM_32 BUILD_BUG_ON(XENHEAP_VIRT_START & ~FIRST_MASK); @@ -135,7 +141,12 @@ static void __init __maybe_unused build_assertions(void) lpae_t __init pte_of_xenaddr(vaddr_t va) { - paddr_t ma = va + phys_offset; + paddr_t ma; + + if ( llc_coloring_enabled ) + ma = virt_to_maddr(virt_to_reloc_virt(va)); + else + ma = va + phys_offset; return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL); } @@ -335,9 +346,44 @@ paddr_t __init consider_modules(paddr_t s, paddr_t e, return e; } +static void __init create_llc_coloring_mappings(void) +{ + lpae_t pte; + unsigned int i; + struct bootmodule *xen_bootmodule = boot_module_find_by_kind(BOOTMOD_XEN); + mfn_t start_mfn = maddr_to_mfn(xen_bootmodule->start), mfn; + + for_each_xen_colored_mfn ( start_mfn, mfn, i ) + { + pte = mfn_to_xen_entry(mfn, MT_NORMAL); + pte.pt.table = 1; /* level 3 mappings always have this bit set */ + xen_xenmap[i] = pte; + } + + for ( i = 0; i < XEN_NR_ENTRIES(2); i++ ) + { + vaddr_t va = BOOT_RELOC_VIRT_START + (i << XEN_PT_LEVEL_SHIFT(2)); + + pte = mfn_to_xen_entry(virt_to_mfn(xen_xenmap + + i * XEN_PT_LPAE_ENTRIES), + MT_NORMAL); + pte.pt.table = 1; + write_pte(&boot_second[second_table_offset(va)], pte); + } +} + /* - * Boot-time pagetable setup. + * Boot-time pagetable setup with coloring support * Changes here may need matching changes in head.S + * + * The cache coloring support consists of: + * - Create colored mapping that conforms to Xen color selection in xen_xenmap[] + * - Link the mapping in boot page tables using BOOT_RELOC_VIRT_START as vaddr + * - pte_of_xenaddr() takes care of translating addresses to the new space + * during runtime page tables creation + * - Relocate xen and update TTBR with the new address in the colored space + * (see switch_ttbr()) + * - Protect the new space */ void __init setup_pagetables(unsigned long boot_phys_offset) { @@ -347,6 +393,9 @@ void __init setup_pagetables(unsigned long boot_phys_offset) phys_offset = boot_phys_offset; + if ( llc_coloring_enabled ) + create_llc_coloring_mappings(); + arch_setup_page_tables(); #ifdef CONFIG_ARM_64 @@ -374,13 +423,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset) break; pte = pte_of_xenaddr(va); pte.pt.table = 1; /* third level mappings always have this bit set */ - if ( is_kernel_text(va) || is_kernel_inittext(va) ) - { - pte.pt.xn = 0; - pte.pt.ro = 1; - } - if ( is_kernel_rodata(va) ) - pte.pt.ro = 1; + pte.pt.xn = 0; /* Permissions will be enforced later. Allow execution */ xen_xenmap[i] = pte; } @@ -406,13 +449,48 @@ void __init setup_pagetables(unsigned long boot_phys_offset) ttbr = (uintptr_t) cpu0_pgtable + phys_offset; #endif - switch_ttbr(ttbr); - - xen_pt_enforce_wnx(); - #ifdef CONFIG_ARM_32 per_cpu(xen_pgtable, 0) = cpu0_pgtable; #endif + + if ( llc_coloring_enabled ) + ttbr = virt_to_maddr(virt_to_reloc_virt(THIS_CPU_PGTABLE)); + + switch_ttbr(ttbr); + + /* Protect Xen */ + for ( i = 0; i < XEN_NR_ENTRIES(3); i++ ) + { + vaddr_t va = XEN_VIRT_START + (i << PAGE_SHIFT); + lpae_t *entry = xen_xenmap + i; + + if ( !is_kernel(va) ) + break; + + pte = read_atomic(entry); + + if ( is_kernel_text(va) || is_kernel_inittext(va) ) + { + pte.pt.xn = 0; + pte.pt.ro = 1; + } else if ( is_kernel_rodata(va) ) { + pte.pt.ro = 1; + pte.pt.xn = 1; + } else { + pte.pt.xn = 1; + pte.pt.ro = 0; + } + + write_pte(entry, pte); + } + + /* + * We modified live page-tables. Ensure the TLBs are invalidated + * before setting enforcing the WnX permissions. + */ + flush_xen_tlb_local(); + + xen_pt_enforce_wnx(); } void *__init arch_vmap_virt_end(void) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index bbb3e1eea5..810dde3a49 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -754,8 +754,6 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset, /* Initialize traps early allow us to get backtrace when an error occurred */ init_traps(); - setup_pagetables(boot_phys_offset); - smp_clear_cpu_maps(); device_tree_flattened = early_fdt_map(fdt_paddr); @@ -779,6 +777,14 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset, llc_coloring_init(); + /* + * Page tables must be setup after LLC coloring initialization because + * coloring info are required in order to create colored mappings + */ + setup_pagetables(boot_phys_offset); + /* Device-tree was mapped in boot page tables, remap it in the new tables */ + device_tree_flattened = early_fdt_map(fdt_paddr); + setup_mm(); vm_init(); diff --git a/xen/common/llc-coloring.c b/xen/common/llc-coloring.c index 0ddd6946f1..323c125c48 100644 --- a/xen/common/llc-coloring.c +++ b/xen/common/llc-coloring.c @@ -33,6 +33,8 @@ static unsigned int __ro_after_init xen_num_colors; #define mfn_color_mask (max_nr_colors - 1) #define mfn_to_color(mfn) (mfn_x(mfn) & mfn_color_mask) +#define get_mfn_with_color(mfn, color) \ + (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color))) /* * Parse the coloring configuration given in the buf string, following the @@ -350,6 +352,22 @@ unsigned int get_max_nr_llc_colors(void) return max_nr_colors; } +mfn_t __init xen_colored_mfn(mfn_t mfn) +{ + unsigned int i, color = mfn_to_color(mfn); + + for ( i = 0; i < xen_num_colors; i++ ) + { + if ( color == xen_colors[i] ) + return mfn; + else if ( color < xen_colors[i] ) + return get_mfn_with_color(mfn, xen_colors[i]); + } + + /* Jump to next color space (max_nr_colors mfns) and use the first color */ + return get_mfn_with_color(mfn_add(mfn, max_nr_colors), xen_colors[0]); +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/llc-coloring.h b/xen/include/xen/llc-coloring.h index 7f8218bfb2..859ed2da4e 100644 --- a/xen/include/xen/llc-coloring.h +++ b/xen/include/xen/llc-coloring.h @@ -26,6 +26,17 @@ static inline void domain_dump_llc_colors(const struct domain *d) {} static inline void domain_llc_coloring_free(struct domain *d) {} #endif +/** + * Iterate over each Xen mfn in the colored space. + * @start_mfn: the first mfn that needs to be colored. + * @mfn: the current mfn. + * @i: loop index. + */ +#define for_each_xen_colored_mfn(start_mfn, mfn, i) \ + for ( i = 0, mfn = xen_colored_mfn(start_mfn); \ + i < (_end - _start) >> PAGE_SHIFT; \ + i++, mfn = xen_colored_mfn(mfn_add(mfn, 1)) ) + unsigned int get_llc_way_size(void); void arch_llc_coloring_init(void); int dom0_set_llc_colors(struct domain *d); @@ -37,6 +48,8 @@ struct page_info; unsigned int page_to_llc_color(const struct page_info *pg); unsigned int get_max_nr_llc_colors(void); +mfn_t xen_colored_mfn(mfn_t mfn); + #endif /* __COLORING_H__ */ /*