From patchwork Tue Sep 6 08:59:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12968232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DCFCBECAAA1 for ; Tue, 6 Sep 2022 21:50:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.400399.642127 (Exim 4.92) (envelope-from ) id 1oVgSj-0003Jh-2i; Tue, 06 Sep 2022 21:50:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 400399.642127; Tue, 06 Sep 2022 21:50:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgSi-0003HR-R7; Tue, 06 Sep 2022 21:50:28 +0000 Received: by outflank-mailman (input) for mailman id 400399; Tue, 06 Sep 2022 21:50:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgHs-0000Cs-BU for xen-devel@lists.xenproject.org; Tue, 06 Sep 2022 21:39:16 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 4506b3cb-2dc2-11ed-af93-0125da4c0113; Tue, 06 Sep 2022 10:59:52 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 897E81756; Tue, 6 Sep 2022 02:00:02 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id ED8C93F7B4; Tue, 6 Sep 2022 01:59:53 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4506b3cb-2dc2-11ed-af93-0125da4c0113 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH v7 1/9] xen/arm: introduce static shared memory Date: Tue, 6 Sep 2022 16:59:33 +0800 Message-Id: <20220906085941.944592-2-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906085941.944592-1-Penny.Zheng@arm.com> References: <20220906085941.944592-1-Penny.Zheng@arm.com> MIME-Version: 1.0 From: Penny Zheng This patch series introduces a new feature: setting up static shared memory on a dom0less system, through device tree configuration. This commit parses shared memory node at boot-time, and reserve it in bootinfo.reserved_mem to avoid other use. This commits proposes a new Kconfig CONFIG_STATIC_SHM to wrap static-shm-related codes, and this option depends on static memory( CONFIG_STATIC_MEMORY). That's because that later we want to reuse a few helpers, guarded with CONFIG_STATIC_MEMORY, like acquire_staticmem_pages, etc, on static shared memory. Signed-off-by: Penny Zheng --- v7 change: - add docs about checking for region overlap, etc - update docs to reflect that the host physical address is optional - end/bank_end may end up to be lower than paddr/mem->bank[i].start, So check that they don't overflow. - replace strcmp with strncmp since the string has already be validated - provide a dummy helper for process_shm_node() when !CONFIG_STATIC_SHM - remove useless #ifdef CONFIG_STATIC_SHM - change the order of property xen,shared-mem to xen,shared-mem = < [host physical address] [guest address] [size] > --- v6 change: - when host physical address is ommited, output the error message since xen doesn't support it at the moment - add the following check: 1) The shm ID matches and the region exactly match 2) The shm ID doesn't match and the region doesn't overlap - change it to "unsigned int" to be aligned with nr_banks - check the len of the property to confirm is it big enough to contain "paddr", "size", and "gaddr" - shm_id defined before nr_shm_domain, so we could re-use the existing hole and avoid increasing the size of the structure. - change "nr_shm_domain" to "nr_shm_borrowers", to not increment if the role is owner in parsing code - make "xen,shm_id" property as arbitrary string, with a strict limit on the number of characters, MAX_SHM_ID_LENGTH --- v5 change: - no change --- v4 change: - nit fix on doc --- v3 change: - make nr_shm_domain unsigned int --- v2 change: - document refinement - remove bitmap and use the iteration to check - add a new field nr_shm_domain to keep the number of shared domain --- docs/misc/arm/device-tree/booting.txt | 132 +++++++++++++++++++++ xen/arch/arm/Kconfig | 6 + xen/arch/arm/bootfdt.c | 165 ++++++++++++++++++++++++++ xen/arch/arm/include/asm/setup.h | 7 ++ 4 files changed, 310 insertions(+) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index 98253414b8..1c50fc6351 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -378,3 +378,135 @@ device-tree: This will reserve a 512MB region starting at the host physical address 0x30000000 to be exclusively used by DomU1. + +Static Shared Memory +==================== + +The static shared memory device tree nodes allow users to statically set up +shared memory on dom0less system, enabling domains to do shm-based +communication. + +- compatible + + "xen,domain-shared-memory-v1" + +- xen,shm-id + + An arbitrary string that represents the unique identifier of the shared + memory region, with a strict limit on the number of characters(\0 included), + `MAX_SHM_ID_LENGTH(16)`. e.g. "xen,shm-id = "my-shared-mem-1"". + +- xen,shared-mem + + An array takes a physical address, which is the base address of the + shared memory region in host physical address space, a size, and a guest + physical address, as the target address of the mapping. + e.g. xen,shared-mem = < [host physical address] [guest address] [size] > + + It shall also meet the following criteria: + 1) If the SHM ID matches with an existing region, the address range of the + region shall also exactly match. + 2) If the SHM ID does not match with any other existing region, it should + also not overlap with any other regions. + + The number of cells for the host address (and size) is the same as the + guest pseudo-physical address and they are inherited from the parent node. + + Host physical address is optional, when missing Xen decides the location + (currently unimplemented). + +- role (Optional) + + A string property specifying the ownership of a shared memory region, + the value must be one of the following: "owner", or "borrower" + A shared memory region could be explicitly backed by one domain, which is + called "owner domain", and all the other domains who are also sharing + this region are called "borrower domain". + If not specified, the default value is "borrower" and owner is + DOMID_IO, a system domain. + +As an example: + +chosen { + #address-cells = <0x1>; + #size-cells = <0x1>; + xen,xen-bootargs = "console=dtuart dtuart=serial0 bootscrub=0"; + + ...... + + /* this is for Dom0 */ + dom0-shared-mem@10000000 { + compatible = "xen,domain-shared-memory-v1"; + role = "owner"; + xen,shm-id = "my-shared-mem-0"; + xen,shared-mem = <0x10000000 0x10000000 0x10000000>; + } + + domU1 { + compatible = "xen,domain"; + #address-cells = <0x1>; + #size-cells = <0x1>; + memory = <0 131072>; + cpus = <2>; + vpl011; + + /* + * shared memory region identified as 0x0(xen,shm-id = <0x0>) + * is shared between Dom0 and DomU1. + */ + domU1-shared-mem@10000000 { + compatible = "xen,domain-shared-memory-v1"; + role = "borrower"; + xen,shm-id = "my-shared-mem-0"; + xen,shared-mem = <0x10000000 0x50000000 0x10000000>; + } + + /* + * shared memory region identified as 0x1(xen,shm-id = <0x1>) + * is shared between DomU1 and DomU2. + */ + domU1-shared-mem@50000000 { + compatible = "xen,domain-shared-memory-v1"; + xen,shm-id = "my-shared-mem-1"; + xen,shared-mem = <0x50000000 0x60000000 0x20000000>; + } + + ...... + + }; + + domU2 { + compatible = "xen,domain"; + #address-cells = <0x1>; + #size-cells = <0x1>; + memory = <0 65536>; + cpus = <1>; + + /* + * shared memory region identified as 0x1(xen,shm-id = <0x1>) + * is shared between domU1 and domU2. + */ + domU2-shared-mem@50000000 { + compatible = "xen,domain-shared-memory-v1"; + xen,shm-id = "my-shared-mem-1"; + xen,shared-mem = <0x50000000 0x70000000 0x20000000>; + } + + ...... + }; +}; + +This is an example with two static shared memory regions. + +For the static shared memory region identified as "my-shared-mem-0", host +physical address starting at 0x10000000 of 256MB will be reserved to be +shared between Dom0 and DomU1. It will get mapped at 0x10000000 in Dom0 guest +physical address space, and at 0x50000000 in DomU1 guest physical address space. +Dom0 is explicitly defined as the owner domain, and DomU1 is the borrower domain. + +For the static shared memory region identified as "my-shared-mem-1", host +physical address starting at 0x50000000 of 512MB will be reserved to be +shared between DomU1 and DomU2. It will get mapped at 0x60000000 in DomU1 guest +physical address space, and at 0x70000000 in DomU2 guest physical address space. +DomU1 and DomU2 are both the borrower domain, the owner domain is the default +owner domain DOMID_IO. diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 33e004d702..1fe5faf847 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -140,6 +140,12 @@ config TEE source "arch/arm/tee/Kconfig" +config STATIC_SHM + bool "Statically shared memory on a dom0less system" if UNSUPPORTED + depends on STATIC_MEMORY + help + This option enables statically shared memory on a dom0less system. + endmenu menu "ARM errata workaround via the alternative framework" diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c index ec81a45de9..b2cd425dac 100644 --- a/xen/arch/arm/bootfdt.c +++ b/xen/arch/arm/bootfdt.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -361,6 +362,168 @@ static int __init process_domain_node(const void *fdt, int node, size_cells, &bootinfo.reserved_mem, true); } +#ifdef CONFIG_STATIC_SHM +static int __init process_shm_node(const void *fdt, int node, + uint32_t address_cells, uint32_t size_cells) +{ + const struct fdt_property *prop, *prop_id, *prop_role; + const __be32 *cell; + paddr_t paddr, gaddr, size; + struct meminfo *mem = &bootinfo.reserved_mem; + unsigned int i; + int len; + bool owner = false; + const char *shm_id; + + if ( address_cells < 1 || size_cells < 1 ) + { + printk("fdt: invalid #address-cells or #size-cells for static shared memory node.\n"); + return -EINVAL; + } + + /* + * "xen,shm-id" property holds an arbitrary string with a strict limit + * on the number of characters, MAX_SHM_ID_LENGTH + */ + prop_id = fdt_get_property(fdt, node, "xen,shm-id", NULL); + if ( !prop_id ) + return -ENOENT; + shm_id = (const char *)prop_id->data; + if ( strnlen(shm_id, MAX_SHM_ID_LENGTH) == MAX_SHM_ID_LENGTH ) + { + printk("fdt: invalid xen,shm-id %s, it must be limited to %u characters\n", + shm_id, MAX_SHM_ID_LENGTH); + return -EINVAL; + } + + /* + * "role" property is optional and if it is defined explicitly, + * it must be either `owner` or `borrower`. + */ + prop_role = fdt_get_property(fdt, node, "role", NULL); + if ( prop_role ) + { + if ( !strcmp(prop_role->data, "owner") ) + owner = true; + else if ( strcmp(prop_role->data, "borrower") ) + { + printk("fdt: invalid `role` property for static shared memory node.\n"); + return -EINVAL; + } + } + + /* + * xen,shared-mem = ; + * Memory region starting from physical address #paddr of #size shall + * be mapped to guest physical address #gaddr as static shared memory + * region. + */ + prop = fdt_get_property(fdt, node, "xen,shared-mem", &len); + if ( !prop ) + return -ENOENT; + + if ( len != dt_cells_to_size(address_cells + size_cells + address_cells) ) + { + if ( len == dt_cells_to_size(size_cells + address_cells) ) + printk("fdt: host physical address must be chosen by users at the moment.\n"); + + printk("fdt: invalid `xen,shared-mem` property.\n"); + return -EINVAL; + } + + cell = (const __be32 *)prop->data; + device_tree_get_reg(&cell, address_cells, address_cells, &paddr, &gaddr); + size = dt_next_cell(size_cells, &cell); + + if ( !size ) + { + printk("fdt: the size for static shared memory region can not be zero\n"); + return -EINVAL; + } + + for ( i = 0; i < mem->nr_banks; i++ ) + { + /* + * Meet the following check: + * 1) The shm ID matches and the region exactly match + * 2) The shm ID doesn't match and the region doesn't overlap + * with an existing one + */ + if ( paddr == mem->bank[i].start && size == mem->bank[i].size ) + { + if ( strncmp(shm_id, mem->bank[i].shm_id, MAX_SHM_ID_LENGTH) == 0 ) + break; + else + { + printk("fdt: xen,shm-id %s does not match for all the nodes using the same region.\n", + shm_id); + return -EINVAL; + } + } + else + { + paddr_t end = paddr + size; + paddr_t bank_end = mem->bank[i].start + mem->bank[i].size; + + if ( (end <= paddr) || (bank_end <= mem->bank[i].start) ) + printk("fdt: static shared memory region %s overflow\n", shm_id); + + if ( (end <= mem->bank[i].start) || (paddr >= bank_end) ) + { + if ( strcmp(shm_id, mem->bank[i].shm_id) != 0 ) + continue; + else + { + printk("fdt: different shared memory region could not share the same shm ID %s\n", + shm_id); + return -EINVAL; + } + } + else + { + printk("fdt: shared memory region overlap with an existing entry %#"PRIpaddr" - %#"PRIpaddr"\n", + mem->bank[i].start, bank_end); + return -EINVAL; + } + } + } + + if ( i == mem->nr_banks ) + { + if ( i < NR_MEM_BANKS ) + { + /* Static shared memory shall be reserved from any other use. */ + safe_strcpy(mem->bank[mem->nr_banks].shm_id, shm_id); + mem->bank[mem->nr_banks].start = paddr; + mem->bank[mem->nr_banks].size = size; + mem->bank[mem->nr_banks].xen_domain = true; + mem->nr_banks++; + } + else + { + printk("Warning: Max number of supported memory regions reached.\n"); + return -ENOSPC; + } + } + /* + * keep a count of the number of borrowers, which later may be used + * to calculate the reference count. + */ + if ( !owner ) + mem->bank[i].nr_shm_borrowers++; + + return 0; +} +#else +static int __init process_shm_node(const void *fdt, int node, + uint32_t address_cells, uint32_t size_cells) +{ + printk("CONFIG_STATIC_SHM must be enabled for parsing static shared memory nodes\n"); + WARN(); + return -EINVAL; +} +#endif + static int __init early_scan_node(const void *fdt, int node, const char *name, int depth, u32 address_cells, u32 size_cells, @@ -386,6 +549,8 @@ static int __init early_scan_node(const void *fdt, process_chosen_node(fdt, node, name, address_cells, size_cells); else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") ) rc = process_domain_node(fdt, node, name, address_cells, size_cells); + else if ( depth <= 3 && device_tree_node_compatible(fdt, node, "xen,domain-shared-memory-v1") ) + rc = process_shm_node(fdt, node, address_cells, size_cells); if ( rc < 0 ) printk("fdt: node `%s': parsing failed\n", name); diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 5815ccf8c5..995eee1d09 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -23,10 +23,17 @@ typedef enum { } bootmodule_kind; +/* Indicates the maximum number of characters(\0 included) for shm_id */ +#define MAX_SHM_ID_LENGTH 16 + struct membank { paddr_t start; paddr_t size; bool xen_domain; /* whether the memory bank is bound to a Xen domain. */ +#ifdef CONFIG_STATIC_SHM + char shm_id[MAX_SHM_ID_LENGTH]; + unsigned int nr_shm_borrowers; +#endif }; struct meminfo { From patchwork Tue Sep 6 08:59:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12968229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3F7D3C38145 for ; Tue, 6 Sep 2022 21:50:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.400374.642077 (Exim 4.92) (envelope-from ) id 1oVgST-0001Kp-SW; Tue, 06 Sep 2022 21:50:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 400374.642077; Tue, 06 Sep 2022 21:50:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgST-0001Ke-OU; Tue, 06 Sep 2022 21:50:13 +0000 Received: by outflank-mailman (input) for mailman id 400374; Tue, 06 Sep 2022 21:50:11 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgIA-0000Cs-F6 for xen-devel@lists.xenproject.org; Tue, 06 Sep 2022 21:39:34 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 476269f8-2dc2-11ed-af93-0125da4c0113; Tue, 06 Sep 2022 10:59:56 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 69E5B1756; Tue, 6 Sep 2022 02:00:06 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C70633F7B4; Tue, 6 Sep 2022 01:59:56 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 476269f8-2dc2-11ed-af93-0125da4c0113 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu Subject: [PATCH v7 2/9] xen/arm: assign static shared memory to the default owner dom_io Date: Tue, 6 Sep 2022 16:59:34 +0800 Message-Id: <20220906085941.944592-3-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906085941.944592-1-Penny.Zheng@arm.com> References: <20220906085941.944592-1-Penny.Zheng@arm.com> MIME-Version: 1.0 From: Penny Zheng This commit introduces process_shm to cope with static shared memory in domain construction. DOMID_IO will be the default owner of memory pre-shared among multiple domains at boot time, when no explicit owner is specified. And DOMID_IO is a fake domain and is not described in the Device-Tree. Therefore When the owner of the shared region is DOMID_IO, we will only find the borrowers when parsing the Device-Tree. When we found the first borrower of the region, we need to assign the region to DOMID_IO This commit only considers allocating static shared memory to dom_io when owner domain is not explicitly defined in device tree, all the left, including the "borrower" code path, the "explicit owner" code path, shall be introduced later in the following patches. Signed-off-by: Penny Zheng Reviewed-by: Julien Grall --- v7 changes: - page_get_owner() can only be called for pages that are marked inuse, so use page_get_owner_and_reference() instead - reference shall be dropped using put_page() - adjust "d->max_pages" when acquire_domstatic_pages() fails - switch from assert to BUG_ON() because this is init code - refine commit message and in-code comments - rename allocate_shared_memory() to assign_shared_memory() --- v6 change: - use if-array to check psize, pbase and gbase are suitably aligned and valid - use mfn_valid to check (pbase, pbase + psize) - check d->max_pages will not overflow - refine acquire_shared_memory_bank to not reuse acquire_static_memory_bank, then input pbase and psize do not need to be used as a pointer. - use if-array to check if page owner is dom_io - in-code comment refinement --- v5 change: - refine in-code comment --- v4 change: - no changes --- v3 change: - refine in-code comment --- v2 change: - instead of introducing a new system domain, reuse the existing dom_io - make dom_io a non-auto-translated domain, then no need to create P2M for it - change dom_io definition and make it wider to support static shm here too - introduce is_shm_allocated_to_domio to check whether static shm is allocated yet, instead of using shm_mask bitmap - add in-code comment --- xen/arch/arm/domain_build.c | 160 ++++++++++++++++++++++++++++++++++++ xen/common/domain.c | 3 + 2 files changed, 163 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 3fd1186b53..be98ff83b1 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -759,6 +759,160 @@ static void __init assign_static_memory_11(struct domain *d, panic("Failed to assign requested static memory for direct-map domain %pd.", d); } + +#ifdef CONFIG_STATIC_SHM +/* + * This function checks whether the static shared memory region is + * already allocated to dom_io. + */ +static bool __init is_shm_allocated_to_domio(paddr_t pbase) +{ + struct page_info *page; + struct domain *d; + + page = maddr_to_page(pbase); + d = page_get_owner_and_reference(page); + if ( d == NULL ) + return false; + put_page(page); + + if ( d != dom_io ) + { + printk(XENLOG_ERR + "shm memory node has already been allocated to a specific owner %pd, Please check your configuration\n", + d); + return false; + } + + return true; +} + +static mfn_t __init acquire_shared_memory_bank(struct domain *d, + paddr_t pbase, paddr_t psize) +{ + mfn_t smfn; + unsigned long nr_pfns; + int res; + + /* + * Pages of statically shared memory shall be included + * into domain_tot_pages(). + */ + nr_pfns = PFN_DOWN(psize); + if ( (UINT_MAX - d->max_pages) < nr_pfns ) + { + printk(XENLOG_ERR "%pd: Over-allocation for d->max_pages: %lu.\n", + d, nr_pfns); + return INVALID_MFN; + } + d->max_pages += nr_pfns; + + smfn = maddr_to_mfn(pbase); + res = acquire_domstatic_pages(d, smfn, nr_pfns, 0); + if ( res ) + { + printk(XENLOG_ERR + "%pd: failed to acquire static memory: %d.\n", d, res); + d->max_pages -= nr_pfns; + return INVALID_MFN; + } + + return smfn; +} + +static int __init assign_shared_memory(struct domain *d, + uint32_t addr_cells, uint32_t size_cells, + paddr_t pbase, paddr_t psize) +{ + mfn_t smfn; + + dprintk(XENLOG_INFO, + "%pd: allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n", + d, pbase, pbase + psize); + + smfn = acquire_shared_memory_bank(d, pbase, psize); + if ( mfn_eq(smfn, INVALID_MFN) ) + return -EINVAL; + + /* + * DOMID_IO is auto-translated (i.e. it seems RAM 1:1). So we do not need + * to create mapping in the P2M. + */ + ASSERT(d == dom_io); + return 0; +} + +static int __init process_shm(struct domain *d, + const struct dt_device_node *node) +{ + struct dt_device_node *shm_node; + + dt_for_each_child_node(node, shm_node) + { + const struct dt_property *prop; + const __be32 *cells; + uint32_t addr_cells, size_cells; + paddr_t gbase, pbase, psize; + int ret = 0; + unsigned int i; + + if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") ) + continue; + + /* + * xen,shared-mem = ; + * TODO: pbase is optional. + */ + addr_cells = dt_n_addr_cells(shm_node); + size_cells = dt_n_size_cells(shm_node); + prop = dt_find_property(shm_node, "xen,shared-mem", NULL); + BUG_ON(!prop); + cells = (const __be32 *)prop->value; + device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase); + psize = dt_read_number(cells, size_cells); + if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) ) + { + printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n", + d, pbase, gbase); + return -EINVAL; + } + if ( !IS_ALIGNED(psize, PAGE_SIZE) ) + { + printk("%pd: size 0x%"PRIpaddr" is not suitably aligned\n", + d, psize); + return -EINVAL; + } + + for ( i = 0; i < PFN_DOWN(psize); i++ ) + if ( !mfn_valid(mfn_add(maddr_to_mfn(pbase), i)) ) + { + printk("%pd: invalid physical address 0x%"PRI_mfn"\n", + d, mfn_x(mfn_add(maddr_to_mfn(pbase), i))); + return -EINVAL; + } + + /* TODO: Consider owner domain is not the default dom_io. */ + /* + * DOMID_IO is a fake domain and is not described in the Device-Tree. + * Therefore when the owner of the shared region is DOMID_IO, we will + * only find the borrowers. + */ + if ( !is_shm_allocated_to_domio(pbase) ) + { + /* + * We found the first borrower of the region, the owner was not + * specified, so they should be assigned to dom_io. + */ + ret = assign_shared_memory(dom_io, addr_cells, size_cells, + pbase, psize); + if ( ret ) + return ret; + } + } + + return 0; +} +#endif /* CONFIG_STATIC_SHM */ #else static void __init allocate_static_memory(struct domain *d, struct kernel_info *kinfo, @@ -3236,6 +3390,12 @@ static int __init construct_domU(struct domain *d, else assign_static_memory_11(d, &kinfo, node); +#ifdef CONFIG_STATIC_SHM + rc = process_shm(d, node); + if ( rc < 0 ) + return rc; +#endif + /* * Base address and irq number are needed when creating vpl011 device * tree node in prepare_dtb_domU, so initialization on related variables diff --git a/xen/common/domain.c b/xen/common/domain.c index 7062393e37..fbd290ece7 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -783,6 +783,9 @@ void __init setup_system_domains(void) * This domain owns I/O pages that are within the range of the page_info * array. Mappings occur at the priv of the caller. * Quarantined PCI devices will be associated with this domain. + * + * DOMID_IO is also the default owner of memory pre-shared among multiple + * domains at boot time. */ dom_io = domain_create(DOMID_IO, NULL, 0); if ( IS_ERR(dom_io) ) From patchwork Tue Sep 6 08:59:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12968230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA1FDECAAA1 for ; Tue, 6 Sep 2022 21:50:29 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.400387.642088 (Exim 4.92) (envelope-from ) id 1oVgSb-000208-A2; Tue, 06 Sep 2022 21:50:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 400387.642088; Tue, 06 Sep 2022 21:50:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgSb-0001zw-6z; Tue, 06 Sep 2022 21:50:21 +0000 Received: by outflank-mailman (input) for mailman id 400387; Tue, 06 Sep 2022 21:50:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oVgHv-0000Cs-C8 for xen-devel@lists.xenproject.org; Tue, 06 Sep 2022 21:39:19 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 50df1d04-2dc2-11ed-af93-0125da4c0113; Tue, 06 Sep 2022 11:00:12 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C1BF139F; Tue, 6 Sep 2022 02:00:22 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 910463F7B4; Tue, 6 Sep 2022 02:00:13 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 50df1d04-2dc2-11ed-af93-0125da4c0113 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v7 7/9] xen/arm: create shared memory nodes in guest device tree Date: Tue, 6 Sep 2022 16:59:39 +0800 Message-Id: <20220906085941.944592-8-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906085941.944592-1-Penny.Zheng@arm.com> References: <20220906085941.944592-1-Penny.Zheng@arm.com> MIME-Version: 1.0 We expose the shared memory to the domU using the "xen,shared-memory-v1" reserved-memory binding. See Documentation/devicetree/bindings/reserved-memory/xen,shared-memory.txt in Linux for the corresponding device tree binding. To save the cost of re-parsing shared memory device tree configuration when creating shared memory nodes in guest device tree, this commit adds new field "shm_mem" to store shm-info per domain. For each shared memory region, a range is exposed under the /reserved-memory node as a child node. Each range sub-node is named xen-shmem@
and has the following properties: - compatible: compatible = "xen,shared-memory-v1" - reg: the base guest physical address and size of the shared memory region - xen,id: a string that identifies the shared memory region. Signed-off-by: Penny Zheng Reviewed-by: Stefano Stabellini --- v7 changes: - allocate reg for worst case addrcells + sizecells - replace assert() with BUG_ON() since it is init code --- v6 change: - change "struct meminfo *mem" to "const struct meminfo *mem" - change "unsigned long i" to "unsigned int i" to match the type of nr_banks. - accroding to the Linux binding, "xen,id" is meant to be a string, not an integer --- v5 change: - no change --- v4 change: - no change --- v3 change: - move field "shm_mem" to kernel_info --- v2 change: - using xzalloc - shm_id should be uint8_t - make reg a local variable - add #address-cells and #size-cells properties - fix alignment --- xen/arch/arm/domain_build.c | 147 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/kernel.h | 1 + 2 files changed, 146 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index d0ff487cc6..3b7436030e 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -914,7 +914,22 @@ static int __init assign_shared_memory(struct domain *d, return ret; } -static int __init process_shm(struct domain *d, +static int __init append_shm_bank_to_domain(struct kernel_info *kinfo, + paddr_t start, paddr_t size, + const char *shm_id) +{ + if ( (kinfo->shm_mem.nr_banks + 1) > NR_MEM_BANKS ) + return -ENOMEM; + + kinfo->shm_mem.bank[kinfo->shm_mem.nr_banks].start = start; + kinfo->shm_mem.bank[kinfo->shm_mem.nr_banks].size = size; + safe_strcpy(kinfo->shm_mem.bank[kinfo->shm_mem.nr_banks].shm_id, shm_id); + kinfo->shm_mem.nr_banks++; + + return 0; +} + +static int __init process_shm(struct domain *d, struct kernel_info *kinfo, const struct dt_device_node *node) { struct dt_device_node *shm_node; @@ -928,6 +943,7 @@ static int __init process_shm(struct domain *d, int ret = 0; unsigned int i; const char *role_str; + const char *shm_id; bool owner_dom_io = true; if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") ) @@ -972,6 +988,9 @@ static int __init process_shm(struct domain *d, if ( dt_property_read_string(shm_node, "role", &role_str) == 0 ) owner_dom_io = false; + dt_property_read_string(shm_node, "xen,shm-id", &shm_id); + BUG_ON((strlen(shm_id) <= 0) || (strlen(shm_id) >= MAX_SHM_ID_LENGTH)); + /* * DOMID_IO is a fake domain and is not described in the Device-Tree. * Therefore when the owner of the shared region is DOMID_IO, we will @@ -999,6 +1018,14 @@ static int __init process_shm(struct domain *d, if ( ret ) return ret; } + + /* + * Record static shared memory region info for later setting + * up shm-node in guest device tree. + */ + ret = append_shm_bank_to_domain(kinfo, gbase, psize, shm_id); + if ( ret ) + return ret; } return 0; @@ -1329,6 +1356,117 @@ static int __init make_memory_node(const struct domain *d, return res; } +#ifdef CONFIG_STATIC_SHM +static int __init make_shm_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + const struct meminfo *mem) +{ + unsigned int i = 0; + int res = 0; + + if ( mem->nr_banks == 0 ) + return -ENOENT; + + /* + * For each shared memory region, a range is exposed under + * the /reserved-memory node as a child node. Each range sub-node is + * named xen-shmem@
. + */ + dt_dprintk("Create xen-shmem node\n"); + + for ( ; i < mem->nr_banks; i++ ) + { + uint64_t start = mem->bank[i].start; + uint64_t size = mem->bank[i].size; + /* Placeholder for xen-shmem@ + a 64-bit number + \0 */ + char buf[27]; + const char compat[] = "xen,shared-memory-v1"; + /* Worst case addrcells + sizecells */ + __be32 reg[4]; + __be32 *cells; + unsigned int len = (addrcells + sizecells) * sizeof(__be32); + + snprintf(buf, sizeof(buf), "xen-shmem@%"PRIx64, mem->bank[i].start); + res = fdt_begin_node(fdt, buf); + if ( res ) + return res; + + res = fdt_property(fdt, "compatible", compat, sizeof(compat)); + if ( res ) + return res; + + cells = reg; + dt_child_set_range(&cells, addrcells, sizecells, start, size); + + res = fdt_property(fdt, "reg", reg, len); + if ( res ) + return res; + + dt_dprintk("Shared memory bank %u: %#"PRIx64"->%#"PRIx64"\n", + i, start, start + size); + + res = fdt_property_string(fdt, "xen,id", mem->bank[i].shm_id); + if ( res ) + return res; + + res = fdt_end_node(fdt); + if ( res ) + return res; + } + + return res; +} +#else +static int __init make_shm_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + struct meminfo *mem) +{ + ASSERT_UNREACHABLE(); +} +#endif + +static int __init make_resv_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + struct meminfo *mem) +{ + int res = 0; + /* Placeholder for reserved-memory\0 */ + char resvbuf[16] = "reserved-memory"; + + if ( mem->nr_banks == 0 ) + /* No shared memory provided. */ + return 0; + + dt_dprintk("Create reserved-memory node\n"); + + res = fdt_begin_node(fdt, resvbuf); + if ( res ) + return res; + + res = fdt_property(fdt, "ranges", NULL, 0); + if ( res ) + return res; + + res = fdt_property_cell(fdt, "#address-cells", addrcells); + if ( res ) + return res; + + res = fdt_property_cell(fdt, "#size-cells", sizecells); + if ( res ) + return res; + + res = make_shm_memory_node(d, fdt, addrcells, sizecells, mem); + if ( res ) + return res; + + res = fdt_end_node(fdt); + + return res; +} + static int __init add_ext_regions(unsigned long s, unsigned long e, void *data) { struct meminfo *ext_regions = data; @@ -3106,6 +3244,11 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo) if ( ret ) goto err; + ret = make_resv_memory_node(d, kinfo->fdt, addrcells, sizecells, + &kinfo->shm_mem); + if ( ret ) + goto err; + /* * domain_handle_dtb_bootmodule has to be called before the rest of * the device tree is generated because it depends on the value of @@ -3482,7 +3625,7 @@ static int __init construct_domU(struct domain *d, assign_static_memory_11(d, &kinfo, node); #ifdef CONFIG_STATIC_SHM - rc = process_shm(d, node); + rc = process_shm(d, &kinfo, node); if ( rc < 0 ) return rc; #endif diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/kernel.h index c4dc039b54..2cc506b100 100644 --- a/xen/arch/arm/include/asm/kernel.h +++ b/xen/arch/arm/include/asm/kernel.h @@ -19,6 +19,7 @@ struct kernel_info { void *fdt; /* flat device tree */ paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */ struct meminfo mem; + struct meminfo shm_mem; /* kernel entry point */ paddr_t entry;