From patchwork Fri May 6 07:24:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 123FCC433EF for ; Fri, 6 May 2022 07:26:18 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322646.544048 (Exim 4.92) (envelope-from ) id 1nmsLf-0007Py-CC; Fri, 06 May 2022 07:25:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322646.544048; Fri, 06 May 2022 07:25:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLf-0007Pn-7m; Fri, 06 May 2022 07:25:59 +0000 Received: by outflank-mailman (input) for mailman id 322646; Fri, 06 May 2022 07:25:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLd-0007P7-E9 for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:25:57 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id c3096221-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:25:53 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E184714BF; Fri, 6 May 2022 00:25:52 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5EF3B3FA27; Fri, 6 May 2022 00:25:50 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c3096221-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH v2 1/9] xen/arm: introduce static shared memory Date: Fri, 6 May 2022 15:24:54 +0800 Message-Id: <20220506072502.2177828-2-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 From: Penny Zheng This patch serie introduces a new feature: setting up static shared memory on a dom0less system, through device tree configuration. This commit parses shared memory node at boot-time, and reserve it in bootinfo.reserved_mem to avoid other use. This commits proposes a new Kconfig CONFIG_STATIC_SHM to wrap static-shm-related codes, and this option depends on static memory( CONFIG_STATIC_MEMORY). That's because that later we want to reuse a few helpers, guarded with CONFIG_STATIC_MEMORY, like acquire_staticmem_pages, etc, on static shared memory. Signed-off-by: Penny Zheng --- v2 change: - document refinement - remove bitmap and use the iteration to check - add a new field nr_shm_domain to keep the number of shared domain --- docs/misc/arm/device-tree/booting.txt | 120 ++++++++++++++++++++++++++ xen/arch/arm/Kconfig | 6 ++ xen/arch/arm/bootfdt.c | 68 +++++++++++++++ xen/arch/arm/include/asm/setup.h | 3 + 4 files changed, 197 insertions(+) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index a94125394e..e63ce171fc 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -355,3 +355,123 @@ device-tree: This will reserve a 512MB region starting at the host physical address 0x30000000 to be exclusively used by DomU1. + +Static Shared Memory +==================== + +The static shared memory device tree nodes allow users to statically set up +shared memory on dom0less system, enabling domains to do shm-based +communication. + +- compatible + + "xen,domain-shared-memory-v1" + +- xen,shm-id + + An u8 value represents the unique identifier of the shared memory region. + The maximum identifier shall be "xen,shm-id = <0xff>". + +- xen,shared-mem + + An array takes a physical address, which is the base address of the + shared memory region in host physical address space, a size, and a guest + physical address, as the target address of the mapping. The number of cells + for the host address (and size) is the same as the guest pseudo-physical + address and they are inherited from the parent node. + +- role (Optional) + + A string property specifying the ownership of a shared memory region, + the value must be one of the following: "owner", or "borrower" + A shared memory region could be explicitly backed by one domain, which is + called "owner domain", and all the other domains who are also sharing + this region are called "borrower domain". + If not specified, the default value is "borrower" and owner is + "dom_shared", a system domain. + +As an example: + +chosen { + #address-cells = <0x1>; + #size-cells = <0x1>; + xen,xen-bootargs = "console=dtuart dtuart=serial0 bootscrub=0"; + + ...... + + /* this is for Dom0 */ + dom0-shared-mem@10000000 { + compatible = "xen,domain-shared-memory-v1"; + role = "owner"; + xen,shm-id = <0x0>; + xen,shared-mem = <0x10000000 0x10000000 0x10000000>; + } + + domU1 { + compatible = "xen,domain"; + #address-cells = <0x1>; + #size-cells = <0x1>; + memory = <0 131072>; + cpus = <2>; + vpl011; + + /* + * shared memory region identified as 0x0(xen,shm-id = <0x0>) + * is shared between Dom0 and DomU1. + */ + domU1-shared-mem@10000000 { + compatible = "xen,domain-shared-memory-v1"; + role = "borrower"; + xen,shm-id = <0x0>; + xen,shared-mem = <0x10000000 0x10000000 0x50000000>; + } + + /* + * shared memory region identified as 0x1(xen,shm-id = <0x1>) + * is shared between DomU1 and DomU2. + */ + domU1-shared-mem@50000000 { + compatible = "xen,domain-shared-memory-v1"; + xen,shm-id = <0x1>; + xen,shared-mem = <0x50000000 0x20000000 0x60000000>; + } + + ...... + + }; + + domU2 { + compatible = "xen,domain"; + #address-cells = <0x1>; + #size-cells = <0x1>; + memory = <0 65536>; + cpus = <1>; + + /* + * shared memory region identified as 0x1(xen,shm-id = <0x1>) + * is shared between domU1 and domU2. + */ + domU2-shared-mem@50000000 { + compatible = "xen,domain-shared-memory-v1"; + xen,shm-id = <0x1>; + xen,shared-mem = <0x50000000 0x20000000 0x70000000>; + } + + ...... + }; +}; + +This is an example with two static shared memory regions. + +For the static shared memory region identified as 0x0, host physical +address starting at 0x10000000 of 256MB will be reserved to be shared between +Dom0 and DomU1. It will get mapped at 0x10000000 in Dom0 guest physical address +space, and at 0x50000000 in DomU1 guest physical address space. Dom0 is +explicitly defined as the owner domain, and DomU1 is the borrower domain. + +For the static shared memory region identified as 0x1, host physical +address starting at 0x50000000 of 512MB will be reserved to be shared between +DomU1 and DomU2. It will get mapped at 0x60000000 in DomU1 guest physical +address space, and at 0x70000000 in DomU2 guest physical address space. DomU1 +and DomU2 are both the borrower domain, the owner domain is the default owner +domain dom_shared. diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index ecfa6822e4..5ee9921f56 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -106,6 +106,12 @@ config TEE source "arch/arm/tee/Kconfig" +config STATIC_SHM + bool "Statically shared memory on a dom0less system" if UNSUPPORTED + depends on STATIC_MEMORY + help + This option enables statically shared memory on a dom0less system. + endmenu menu "ARM errata workaround via the alternative framework" diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c index e318ef9603..9bd08776a7 100644 --- a/xen/arch/arm/bootfdt.c +++ b/xen/arch/arm/bootfdt.c @@ -363,6 +363,70 @@ static int __init process_domain_node(const void *fdt, int node, size_cells, &bootinfo.reserved_mem, true); } +#ifdef CONFIG_STATIC_SHM +static int __init process_shm_node(const void *fdt, int node, + u32 address_cells, u32 size_cells) +{ + const struct fdt_property *prop; + const __be32 *cell; + paddr_t paddr, size; + struct meminfo *mem = &bootinfo.reserved_mem; + unsigned long i; + + if ( address_cells < 1 || size_cells < 1 ) + { + printk("fdt: invalid #address-cells or #size-cells for static shared memory node.\n"); + return -EINVAL; + } + + prop = fdt_get_property(fdt, node, "xen,shared-mem", NULL); + if ( !prop ) + return -ENOENT; + + /* + * xen,shared-mem = ; + * Memory region starting from physical address #paddr of #size shall + * be mapped to guest physical address #gaddr as static shared memory + * region. + */ + cell = (const __be32 *)prop->data; + device_tree_get_reg(&cell, address_cells, size_cells, &paddr, &size); + for ( i = 0; i < mem->nr_banks; i++ ) + { + /* + * A static shared memory region could be shared between multiple + * domains. + */ + if ( paddr == mem->bank[i].start && size == mem->bank[i].size ) + break; + } + + if ( i == mem->nr_banks ) + { + if ( i < NR_MEM_BANKS ) + { + /* Static shared memory shall be reserved from any other use. */ + mem->bank[mem->nr_banks].start = paddr; + mem->bank[mem->nr_banks].size = size; + mem->bank[mem->nr_banks].xen_domain = true; + mem->nr_banks++; + } + else + { + printk("Warning: Max number of supported memory regions reached.\n"); + return -ENOSPC; + } + } + /* + * keep a count of the number of domains, which later may be used to + * calculate the number of the reference count. + */ + mem->bank[i].nr_shm_domain++; + + return 0; +} +#endif + static int __init early_scan_node(const void *fdt, int node, const char *name, int depth, u32 address_cells, u32 size_cells, @@ -383,6 +447,10 @@ static int __init early_scan_node(const void *fdt, process_chosen_node(fdt, node, name, address_cells, size_cells); else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") ) rc = process_domain_node(fdt, node, name, address_cells, size_cells); +#ifdef CONFIG_STATIC_SHM + else if ( depth <= 3 && device_tree_node_compatible(fdt, node, "xen,domain-shared-memory-v1") ) + rc = process_shm_node(fdt, node, address_cells, size_cells); +#endif if ( rc < 0 ) printk("fdt: node `%s': parsing failed\n", name); diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 7a1e1d6798..38e02ced36 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -27,6 +27,9 @@ struct membank { paddr_t start; paddr_t size; bool xen_domain; /* whether the memory bank is bound to a Xen domain. */ +#ifdef CONFIG_STATIC_SHM + unsigned long nr_shm_domain; +#endif }; struct meminfo { From patchwork Fri May 6 07:24:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 905C6C433FE for ; Fri, 6 May 2022 07:26:15 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322647.544059 (Exim 4.92) (envelope-from ) id 1nmsLg-0007gE-LE; Fri, 06 May 2022 07:26:00 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322647.544059; Fri, 06 May 2022 07:26:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLg-0007g7-HI; Fri, 06 May 2022 07:26:00 +0000 Received: by outflank-mailman (input) for mailman id 322647; Fri, 06 May 2022 07:25:59 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLe-0007P7-RG for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:25:58 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id c590b3ba-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:25:57 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 102A81063; Fri, 6 May 2022 00:25:57 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6713B3FA27; Fri, 6 May 2022 00:25:53 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c590b3ba-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu Subject: [PATCH v2 2/9] xen/arm: allocate static shared memory to the default owner dom_io Date: Fri, 6 May 2022 15:24:55 +0800 Message-Id: <20220506072502.2177828-3-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 From: Penny Zheng This commit introduces process_shm to cope with static shared memory in domain construction. DOMID_IO will be the default owner of memory pre-shared among multiple domains at boot time, when no explicit owner is specified. This commit only considers allocating static shared memory to dom_io when owner domain is not explicitly defined in device tree, all the left, including the "borrower" code path, the "explicit owner" code path, shall be introduced later in the following patches. Signed-off-by: Penny Zheng --- v2 change: - instead of introducing a new system domain, reuse the existing dom_io - make dom_io a non-auto-translated domain, then no need to create P2M for it - change dom_io definition and make it wider to support static shm here too - introduce is_shm_allocated_to_domio to check whether static shm is allocated yet, instead of using shm_mask bitmap - add in-code comment --- xen/arch/arm/domain_build.c | 133 +++++++++++++++++++++++++++++++++++- xen/common/domain.c | 18 ++++- 2 files changed, 148 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 1472ca4972..e97bb6eeba 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -527,7 +527,13 @@ static mfn_t __init acquire_static_memory_bank(struct domain *d, mfn_t smfn; int res; - device_tree_get_reg(cell, addr_cells, size_cells, pbase, psize); + /* + * acquire_static_memory_bank() is also used for acquiring static shared + * memory, in such case, we don't need to parse 'cell' here since it is + * already parsed in process_shm(). + */ + if ( cell ) + device_tree_get_reg(cell, addr_cells, size_cells, pbase, psize); ASSERT(IS_ALIGNED(*pbase, PAGE_SIZE) && IS_ALIGNED(*psize, PAGE_SIZE)); if ( PFN_DOWN(*psize) > UINT_MAX ) { @@ -751,6 +757,125 @@ static void __init assign_static_memory_11(struct domain *d, panic("Failed to assign requested static memory for direct-map domain %pd.", d); } + +#ifdef CONFIG_STATIC_SHM +/* + * This function checks whether the static shared memory region is + * already allocated to dom_io. + */ +static bool __init is_shm_allocated_to_domio(paddr_t pbase) +{ + struct page_info *page; + + page = maddr_to_page(pbase); + ASSERT(page); + + if ( page_get_owner(page) == NULL ) + return false; + + ASSERT(page_get_owner(page) == dom_io); + return true; +} + +static mfn_t __init acquire_shared_memory_bank(struct domain *d, + u32 addr_cells, u32 size_cells, + paddr_t *pbase, paddr_t *psize) +{ + /* + * Pages of statically shared memory shall be included + * in domain_tot_pages(). + */ + d->max_pages += PFN_DOWN(*psize); + + return acquire_static_memory_bank(d, NULL, addr_cells, size_cells, + pbase, psize); + +} + +/* + * Func allocate_shared_memory is supposed to be only called + * from the owner. + */ +static int __init allocate_shared_memory(struct domain *d, + u32 addr_cells, u32 size_cells, + paddr_t pbase, paddr_t psize) +{ + mfn_t smfn; + + dprintk(XENLOG_INFO, + "Allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n", + pbase, pbase + psize); + + smfn = acquire_shared_memory_bank(d, addr_cells, size_cells, &pbase, + &psize); + if ( mfn_eq(smfn, INVALID_MFN) ) + return -EINVAL; + + /* + * DOMID_IO is the domain, like DOMID_XEN, that is not auto-translated. + * It sees RAM 1:1 and we do not need to create P2M mapping for it + */ + ASSERT(d == dom_io); + return 0; +} + +static int __init process_shm(struct domain *d, + const struct dt_device_node *node) +{ + struct dt_device_node *shm_node; + int ret = 0; + const struct dt_property *prop; + const __be32 *cells; + u32 shm_id; + u32 addr_cells, size_cells; + paddr_t gbase, pbase, psize; + + dt_for_each_child_node(node, shm_node) + { + if ( !dt_device_is_compatible(shm_node, "xen,domain-shared-memory-v1") ) + continue; + + if ( !dt_property_read_u32(shm_node, "xen,shm-id", &shm_id) ) + { + printk("Shared memory node does not provide \"xen,shm-id\" property.\n"); + return -ENOENT; + } + + addr_cells = dt_n_addr_cells(shm_node); + size_cells = dt_n_size_cells(shm_node); + prop = dt_find_property(shm_node, "xen,shared-mem", NULL); + if ( !prop ) + { + printk("Shared memory node does not provide \"xen,shared-mem\" property.\n"); + return -ENOENT; + } + cells = (const __be32 *)prop->value; + /* xen,shared-mem = ; */ + device_tree_get_reg(&cells, addr_cells, size_cells, &pbase, &psize); + ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE)); + gbase = dt_read_number(cells, addr_cells); + + /* TODO: Consider owner domain is not the default dom_io. */ + /* + * Per static shared memory region could be shared between multiple + * domains. + * In case re-allocating the same shared memory region, we check + * if it is already allocated to the default owner dom_io before + * the actual allocation. + */ + if ( !is_shm_allocated_to_domio(pbase) ) + { + /* Allocate statically shared pages to the default owner dom_io. */ + ret = allocate_shared_memory(dom_io, addr_cells, size_cells, + pbase, psize); + if ( ret ) + return ret; + } + } + + return 0; +} +#endif /* CONFIG_STATIC_SHM */ #else static void __init allocate_static_memory(struct domain *d, struct kernel_info *kinfo, @@ -3149,6 +3274,12 @@ static int __init construct_domU(struct domain *d, else assign_static_memory_11(d, &kinfo, node); +#ifdef CONFIG_STATIC_SHM + rc = process_shm(d, node); + if ( rc < 0 ) + return rc; +#endif + /* * Base address and irq number are needed when creating vpl011 device * tree node in prepare_dtb_domU, so initialization on related variables diff --git a/xen/common/domain.c b/xen/common/domain.c index 8d2c2a9897..0c41ecb197 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -777,8 +777,22 @@ void __init setup_system_domains(void) /* * Initialise our DOMID_IO domain. - * This domain owns I/O pages that are within the range of the page_info - * array. Mappings occur at the priv of the caller. + * DOMID_IO is used for mapping memory and MMIO regions when no explicit + * Domain need to be specified. + * + * For instance, DOMID_IO is the owner of memory pre-shared among + * multiple domains at boot time, when no explicit owner is specified. + * + * Also, DOMID_IO is used to restrict page-table updates to mapping I/O + * memory. Although no Foreign Domain need to be specified to map I/O + * pages, DOMID_IO is useful to ensure that no mappings to the OS's own + * heap are accidentally installed. (e.g., in Linux this could cause + * havoc as reference counts aren't adjusted on the I/O-mapping code + * path). This only makes sense as HYPERVISOR_mmu_update()'s and + * HYPERVISOR_update_va_mapping_otherdomain()'s "foreigndom" argument. + * For HYPERVISOR_mmu_update() context it can be specified by any + * calling domain, otherwise it's only permitted if the caller is + * privileged. * Quarantined PCI devices will be associated with this domain. */ dom_io = domain_create(DOMID_IO, NULL, 0); From patchwork Fri May 6 07:24:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 88F75C433F5 for ; Fri, 6 May 2022 07:26:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322648.544070 (Exim 4.92) (envelope-from ) id 1nmsLi-0007yU-Ue; Fri, 06 May 2022 07:26:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322648.544070; Fri, 06 May 2022 07:26:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLi-0007xu-Qm; Fri, 06 May 2022 07:26:02 +0000 Received: by outflank-mailman (input) for mailman id 322648; Fri, 06 May 2022 07:26:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLi-0007P7-8H for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:02 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id c78a1812-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:26:01 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5D0E41063; Fri, 6 May 2022 00:26:00 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 89F2B3FA27; Fri, 6 May 2022 00:25:57 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c78a1812-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 3/9] xen/arm: allocate static shared memory to a specific owner domain Date: Fri, 6 May 2022 15:24:56 +0800 Message-Id: <20220506072502.2177828-4-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 If owner property is defined, then owner domain of a static shared memory region is not the default dom_io anymore, but a specific domain. This commit implements allocating static shared memory to a specific domain when owner property is defined. Coding flow for dealing borrower domain will be introduced later in the following commits. Signed-off-by: Penny Zheng --- v2 change: - P2M mapping is restricted to normal domain - in-code comment fix --- xen/arch/arm/domain_build.c | 55 +++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index e97bb6eeba..f43378227a 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -798,9 +798,11 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d, */ static int __init allocate_shared_memory(struct domain *d, u32 addr_cells, u32 size_cells, - paddr_t pbase, paddr_t psize) + paddr_t pbase, paddr_t psize, + paddr_t gbase) { mfn_t smfn; + int ret = 0; dprintk(XENLOG_INFO, "Allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n", @@ -813,10 +815,20 @@ static int __init allocate_shared_memory(struct domain *d, /* * DOMID_IO is the domain, like DOMID_XEN, that is not auto-translated. - * It sees RAM 1:1 and we do not need to create P2M mapping for it + * It sees RAM 1:1 and we do not need to create P2M mapping for it. */ - ASSERT(d == dom_io); - return 0; + if ( d != dom_io ) + { + ret = guest_physmap_add_pages(d, gaddr_to_gfn(gbase), smfn, PFN_DOWN(psize)); + if ( ret ) + { + printk(XENLOG_ERR + "Failed to map shared memory to %pd.\n", d); + return ret; + } + } + + return ret; } static int __init process_shm(struct domain *d, @@ -829,6 +841,8 @@ static int __init process_shm(struct domain *d, u32 shm_id; u32 addr_cells, size_cells; paddr_t gbase, pbase, psize; + const char *role_str; + bool owner_dom_io = true; dt_for_each_child_node(node, shm_node) { @@ -855,19 +869,36 @@ static int __init process_shm(struct domain *d, ASSERT(IS_ALIGNED(pbase, PAGE_SIZE) && IS_ALIGNED(psize, PAGE_SIZE)); gbase = dt_read_number(cells, addr_cells); - /* TODO: Consider owner domain is not the default dom_io. */ + /* + * "role" property is optional and if it is defined explicitly, + * then the owner domain is not the default "dom_io" domain. + */ + if ( dt_property_read_string(shm_node, "role", &role_str) == 0 ) + owner_dom_io = false; + /* * Per static shared memory region could be shared between multiple * domains. - * In case re-allocating the same shared memory region, we check - * if it is already allocated to the default owner dom_io before - * the actual allocation. + * So when owner domain is the default dom_io, in case re-allocating + * the same shared memory region, we check if it is already allocated + * to the default owner dom_io before the actual allocation. */ - if ( !is_shm_allocated_to_domio(pbase) ) + if ( (owner_dom_io && !is_shm_allocated_to_domio(pbase)) || + (!owner_dom_io && strcmp(role_str, "owner") == 0) ) { - /* Allocate statically shared pages to the default owner dom_io. */ - ret = allocate_shared_memory(dom_io, addr_cells, size_cells, - pbase, psize); + struct domain *od = d; + paddr_t o_gbase = gbase; + + if ( owner_dom_io ) + { + od = dom_io; + /* For dom_io, GFN is always equal to PFN. */ + o_gbase = pbase; + } + + /* Allocate statically shared pages to the owner domain. */ + ret = allocate_shared_memory(od, addr_cells, size_cells, + pbase, psize, o_gbase); if ( ret ) return ret; } From patchwork Fri May 6 07:24:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C208C4332F for ; Fri, 6 May 2022 07:26:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322649.544081 (Exim 4.92) (envelope-from ) id 1nmsLm-0008KI-7b; Fri, 06 May 2022 07:26:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322649.544081; Fri, 06 May 2022 07:26:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLm-0008K7-4S; Fri, 06 May 2022 07:26:06 +0000 Received: by outflank-mailman (input) for mailman id 322649; Fri, 06 May 2022 07:26:05 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLl-0007P7-8u for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:05 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id c97ddbd2-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:26:04 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AB3E51063; Fri, 6 May 2022 00:26:03 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D8BBA3FA27; Fri, 6 May 2022 00:26:00 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c97ddbd2-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 4/9] xen/arm: introduce put_page_nr and get_page_nr Date: Fri, 6 May 2022 15:24:57 +0800 Message-Id: <20220506072502.2177828-5-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 Later, we need to add the right amount of references, which should be the number of borrower domains, to the owner domain. Since we only have get_page() to increment the page reference by 1, a loop is needed per page, which is inefficient and time-consuming. To save the loop time, this commit introduces a set of new helpers put_page_nr() and get_page_nr() to increment/drop the page reference by nr. Signed-off-by: Penny Zheng --- v2 change: - new commit --- xen/arch/arm/include/asm/mm.h | 4 ++++ xen/arch/arm/mm.c | 36 +++++++++++++++++++++++++---------- 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 424aaf2823..c737d51e4d 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -347,6 +347,10 @@ void free_init_memory(void); int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn, unsigned int order); +extern bool get_page_nr(struct page_info *page, const struct domain *domain, + unsigned long nr); +extern void put_page_nr(struct page_info *page, unsigned long nr); + extern void put_page_type(struct page_info *page); static inline void put_page_and_type(struct page_info *page) { diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 7b1f2f4906..e565979f3c 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -1537,7 +1537,8 @@ long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg) return 0; } -struct domain *page_get_owner_and_reference(struct page_info *page) +static struct domain *page_get_owner_and_nr_reference(struct page_info *page, + unsigned long nr) { unsigned long x, y = page->count_info; struct domain *owner; @@ -1545,13 +1546,14 @@ struct domain *page_get_owner_and_reference(struct page_info *page) do { x = y; /* + * Consider the minimum case(nr = 1): * Count == 0: Page is not allocated, so we cannot take a reference. * Count == -1: Reference count would wrap, which is invalid. */ if ( unlikely(((x + 1) & PGC_count_mask) <= 1) ) return NULL; } - while ( (y = cmpxchg(&page->count_info, x, x + 1)) != x ); + while ( (y = cmpxchg(&page->count_info, x, x + nr)) != x ); owner = page_get_owner(page); ASSERT(owner); @@ -1559,36 +1561,50 @@ struct domain *page_get_owner_and_reference(struct page_info *page) return owner; } -void put_page(struct page_info *page) +struct domain *page_get_owner_and_reference(struct page_info *page) +{ + return page_get_owner_and_nr_reference(page, 1); +} + +void put_page_nr(struct page_info *page, unsigned long nr) { unsigned long nx, x, y = page->count_info; do { - ASSERT((y & PGC_count_mask) != 0); + ASSERT(((y - nr) & PGC_count_mask) >= 0); x = y; - nx = x - 1; + nx = x - nr; } while ( unlikely((y = cmpxchg(&page->count_info, x, nx)) != x) ); if ( unlikely((nx & PGC_count_mask) == 0) ) - { free_domheap_page(page); - } } -bool get_page(struct page_info *page, const struct domain *domain) +void put_page(struct page_info *page) { - const struct domain *owner = page_get_owner_and_reference(page); + put_page_nr(page, 1); +} + +bool get_page_nr(struct page_info *page, const struct domain *domain, + unsigned long nr) +{ + const struct domain *owner = page_get_owner_and_nr_reference(page, nr); if ( likely(owner == domain) ) return true; if ( owner != NULL ) - put_page(page); + put_page_nr(page, nr); return false; } +bool get_page(struct page_info *page, const struct domain *domain) +{ + return get_page_nr(page, domain, 1); +} + /* Common code requires get_page_type and put_page_type. * We don't care about typecounts so we just do the minimum to make it * happy. */ From patchwork Fri May 6 07:24:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44818C433EF for ; Fri, 6 May 2022 07:26:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322650.544092 (Exim 4.92) (envelope-from ) id 1nmsLq-0000K9-La; Fri, 06 May 2022 07:26:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322650.544092; Fri, 06 May 2022 07:26:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLq-0000Jx-I8; Fri, 06 May 2022 07:26:10 +0000 Received: by outflank-mailman (input) for mailman id 322650; Fri, 06 May 2022 07:26:08 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLo-0007P7-KV for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:08 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id cb80e8d5-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:26:07 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0219A1063; Fri, 6 May 2022 00:26:07 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 30C683FA27; Fri, 6 May 2022 00:26:03 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cb80e8d5-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 5/9] xen/arm: Add additional reference to owner domain when the owner is allocated Date: Fri, 6 May 2022 15:24:58 +0800 Message-Id: <20220506072502.2177828-6-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 Borrower domain will fail to get a page ref using the owner domain during allocation, when the owner is created after borrower. So here, we decide to get and add the right amount of reference, which is the number of borrowers, when the owner is allocated. Signed-off-by: Penny Zheng --- v2 change: - new commit --- xen/arch/arm/domain_build.c | 62 +++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index f43378227a..b3ba0c501d 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -792,6 +792,34 @@ static mfn_t __init acquire_shared_memory_bank(struct domain *d, } +static int __init acquire_nr_borrower_domain(struct domain *d, + paddr_t pbase, paddr_t psize, + unsigned long *nr_borrowers) +{ + unsigned long bank; + + /* Iterate reserved memory to find requested shm bank. */ + for ( bank = 0 ; bank < bootinfo.reserved_mem.nr_banks; bank++ ) + { + paddr_t bank_start = bootinfo.reserved_mem.bank[bank].start; + paddr_t bank_size = bootinfo.reserved_mem.bank[bank].size; + + if ( pbase == bank_start && psize == bank_size ) + break; + } + + if ( bank == bootinfo.reserved_mem.nr_banks ) + return -ENOENT; + + if ( d == dom_io ) + *nr_borrowers = bootinfo.reserved_mem.bank[bank].nr_shm_domain; + else + /* Exclude the owner domain itself. */ + *nr_borrowers = bootinfo.reserved_mem.bank[bank].nr_shm_domain - 1; + + return 0; +} + /* * Func allocate_shared_memory is supposed to be only called * from the owner. @@ -803,6 +831,8 @@ static int __init allocate_shared_memory(struct domain *d, { mfn_t smfn; int ret = 0; + unsigned long nr_pages, nr_borrowers, i; + struct page_info *page; dprintk(XENLOG_INFO, "Allocate static shared memory BANK %#"PRIpaddr"-%#"PRIpaddr".\n", @@ -817,6 +847,7 @@ static int __init allocate_shared_memory(struct domain *d, * DOMID_IO is the domain, like DOMID_XEN, that is not auto-translated. * It sees RAM 1:1 and we do not need to create P2M mapping for it. */ + nr_pages = PFN_DOWN(psize); if ( d != dom_io ) { ret = guest_physmap_add_pages(d, gaddr_to_gfn(gbase), smfn, PFN_DOWN(psize)); @@ -828,6 +859,37 @@ static int __init allocate_shared_memory(struct domain *d, } } + /* + * Get the right amount of references per page, which is the number of + * borrow domains. + */ + ret = acquire_nr_borrower_domain(d, pbase, psize, &nr_borrowers); + if ( ret ) + return ret; + + /* + * Instead of let borrower domain get a page ref, we add as many + * additional reference as the number of borrowers when the owner + * is allocated, since there is a chance that owner is created + * after borrower. + */ + page = mfn_to_page(smfn); + for ( i = 0; i < nr_pages; i++ ) + { + if ( !get_page_nr(page + i, d, nr_borrowers) ) + { + dprintk(XENLOG_ERR, + "Failed to add %lu references to page %"PRI_mfn".\n", + nr_borrowers, mfn_x(smfn) + i); + goto fail; + } + } + + return 0; + + fail: + while ( --i >= 0 ) + put_page_nr(page + i, nr_borrowers); return ret; } From patchwork Fri May 6 07:24:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2D43C433F5 for ; Fri, 6 May 2022 07:26:32 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322656.544103 (Exim 4.92) (envelope-from ) id 1nmsLw-0000yF-0p; Fri, 06 May 2022 07:26:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322656.544103; Fri, 06 May 2022 07:26:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLv-0000xz-Qd; Fri, 06 May 2022 07:26:15 +0000 Received: by outflank-mailman (input) for mailman id 322656; Fri, 06 May 2022 07:26:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLt-00078X-PQ for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:13 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id cd75e8c0-cd0d-11ec-a406-831a346695d4; Fri, 06 May 2022 09:26:10 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4E7161063; Fri, 6 May 2022 00:26:10 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7BDE93FA27; Fri, 6 May 2022 00:26:07 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cd75e8c0-cd0d-11ec-a406-831a346695d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 6/9] xen/arm: add P2M type parameter in guest_physmap_add_pages Date: Fri, 6 May 2022 15:24:59 +0800 Message-Id: <20220506072502.2177828-7-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 In order to cover the scenario where users intend to set up guest p2m foreign mapping with nr_pages, this commit adds a new P2M type parameter in guest_physmap_add_pages. Signed-off-by: Penny Zheng Reviewed-by: Stefano Stabellini --- v2 change: - no change --- xen/arch/arm/domain_build.c | 5 +++-- xen/arch/arm/include/asm/p2m.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index b3ba0c501d..089b9e99fc 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -507,7 +507,7 @@ static bool __init append_static_memory_to_bank(struct domain *d, else sgfn = gaddr_to_gfn(mfn_to_maddr(smfn)); - res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages); + res = guest_physmap_add_pages(d, sgfn, smfn, nr_pages, p2m_ram_rw); if ( res ) { dprintk(XENLOG_ERR, "Failed to map pages to DOMU: %d", res); @@ -850,7 +850,8 @@ static int __init allocate_shared_memory(struct domain *d, nr_pages = PFN_DOWN(psize); if ( d != dom_io ) { - ret = guest_physmap_add_pages(d, gaddr_to_gfn(gbase), smfn, PFN_DOWN(psize)); + ret = guest_physmap_add_pages(d, gaddr_to_gfn(gbase), smfn, nr_pages, + p2m_ram_rw); if ( ret ) { printk(XENLOG_ERR diff --git a/xen/arch/arm/include/asm/p2m.h b/xen/arch/arm/include/asm/p2m.h index 8cce459b67..58590145b0 100644 --- a/xen/arch/arm/include/asm/p2m.h +++ b/xen/arch/arm/include/asm/p2m.h @@ -317,9 +317,10 @@ guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn, static inline int guest_physmap_add_pages(struct domain *d, gfn_t gfn, mfn_t mfn, - unsigned int nr_pages) + unsigned int nr_pages, + p2m_type_t t) { - return p2m_insert_mapping(d, gfn, nr_pages, mfn, p2m_ram_rw); + return p2m_insert_mapping(d, gfn, nr_pages, mfn, t); } mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn); From patchwork Fri May 6 07:25:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7844C433F5 for ; Fri, 6 May 2022 07:27:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322668.544114 (Exim 4.92) (envelope-from ) id 1nmsMh-0002b6-AK; Fri, 06 May 2022 07:27:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322668.544114; Fri, 06 May 2022 07:27:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsMh-0002az-68; Fri, 06 May 2022 07:27:03 +0000 Received: by outflank-mailman (input) for mailman id 322668; Fri, 06 May 2022 07:27:02 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLz-00078X-QD for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:19 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id cf6aa5d6-cd0d-11ec-a406-831a346695d4; Fri, 06 May 2022 09:26:14 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A3E7C1063; Fri, 6 May 2022 00:26:13 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C7FB53FA27; Fri, 6 May 2022 00:26:10 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cf6aa5d6-cd0d-11ec-a406-831a346695d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 7/9] xen/arm: set up shared memory foreign mapping for borrower domain Date: Fri, 6 May 2022 15:25:00 +0800 Message-Id: <20220506072502.2177828-8-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 This commit sets up shared memory foreign mapping for borrower domain. If owner domain is the default dom_io, all shared domain are treated as borrower domain. Signed-off-by: Penny Zheng Reviewed-by: Stefano Stabellini --- v2 change: - remove guest_physmap_add_shm, since for borrower domain, we only do P2M foreign memory mapping now. --- xen/arch/arm/domain_build.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 089b9e99fc..8d299a3616 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -965,6 +965,16 @@ static int __init process_shm(struct domain *d, if ( ret ) return ret; } + + if ( owner_dom_io || (strcmp(role_str, "borrower") == 0) ) + { + /* Set up P2M foreign mapping for borrower domain. */ + ret = guest_physmap_add_pages(d, _gfn(PFN_UP(gbase)), + _mfn(PFN_UP(pbase)), PFN_DOWN(psize), + p2m_map_foreign_rw); + if ( ret ) + return ret; + } } return 0; From patchwork Fri May 6 07:25:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C30ECC433EF for ; Fri, 6 May 2022 07:27:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322694.544136 (Exim 4.92) (envelope-from ) id 1nmsN8-0003oJ-QH; Fri, 06 May 2022 07:27:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322694.544136; Fri, 06 May 2022 07:27:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsN8-0003oC-NN; Fri, 06 May 2022 07:27:30 +0000 Received: by outflank-mailman (input) for mailman id 322694; Fri, 06 May 2022 07:27:29 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsLy-0007P7-JD for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:18 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id d157c0c9-cd0d-11ec-8fc4-03012f2f19d4; Fri, 06 May 2022 09:26:17 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F0FB61063; Fri, 6 May 2022 00:26:16 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 298193FA27; Fri, 6 May 2022 00:26:13 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d157c0c9-cd0d-11ec-8fc4-03012f2f19d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Penny Zheng Subject: [PATCH v2 8/9] xen/arm: create shared memory nodes in guest device tree Date: Fri, 6 May 2022 15:25:01 +0800 Message-Id: <20220506072502.2177828-9-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 We expose the shared memory to the domU using the "xen,shared-memory-v1" reserved-memory binding. See Documentation/devicetree/bindings/reserved-memory/xen,shared-memory.txt in Linux for the corresponding device tree binding. To save the cost of re-parsing shared memory device tree configuration when creating shared memory nodes in guest device tree, this commit adds new field "shm_mem" to store shm-info per domain. For each shared memory region, a range is exposed under the /reserved-memory node as a child node. Each range sub-node is named xen-shmem@
and has the following properties: - compatible: compatible = "xen,shared-memory-v1" - reg: the base guest physical address and size of the shared memory region - xen,id: a string that identifies the shared memory region. Signed-off-by: Penny Zheng --- v2 change: - using xzalloc - shm_id should be uint8_t - make reg a local variable - add #address-cells and #size-cells properties - fix alignment --- xen/arch/arm/domain_build.c | 144 ++++++++++++++++++++++++++++++ xen/arch/arm/include/asm/domain.h | 1 + xen/arch/arm/include/asm/setup.h | 1 + 3 files changed, 146 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 8d299a3616..f08606d2c0 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -894,6 +894,26 @@ static int __init allocate_shared_memory(struct domain *d, return ret; } +static int __init append_shm_bank_to_domain(struct domain *d, + paddr_t start, paddr_t size, + u32 shm_id) +{ + /* Allocate memory at first insertion. */ + if ( d->arch.shm_mem == NULL ) + { + d->arch.shm_mem = xzalloc(struct meminfo); + if ( d->arch.shm_mem == NULL ) + return -ENOMEM; + } + + d->arch.shm_mem->bank[d->arch.shm_mem->nr_banks].start = start; + d->arch.shm_mem->bank[d->arch.shm_mem->nr_banks].size = size; + d->arch.shm_mem->bank[d->arch.shm_mem->nr_banks].shm_id = shm_id; + d->arch.shm_mem->nr_banks++; + + return 0; +} + static int __init process_shm(struct domain *d, const struct dt_device_node *node) { @@ -975,6 +995,14 @@ static int __init process_shm(struct domain *d, if ( ret ) return ret; } + + /* + * Record static shared memory region info for later setting + * up shm-node in guest device tree. + */ + ret = append_shm_bank_to_domain(d, gbase, psize, shm_id); + if ( ret ) + return ret; } return 0; @@ -1305,6 +1333,117 @@ static int __init make_memory_node(const struct domain *d, return res; } +#ifdef CONFIG_STATIC_SHM +static int __init make_shm_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + struct meminfo *mem) +{ + unsigned long i = 0; + int res = 0; + + if ( mem->nr_banks == 0 ) + return -ENOENT; + + /* + * For each shared memory region, a range is exposed under + * the /reserved-memory node as a child node. Each range sub-node is + * named xen-shmem@
. + */ + dt_dprintk("Create xen-shmem node\n"); + + for ( ; i < mem->nr_banks; i++ ) + { + uint64_t start = mem->bank[i].start; + uint64_t size = mem->bank[i].size; + uint8_t shm_id = mem->bank[i].shm_id; + /* Placeholder for xen-shmem@ + a 64-bit number + \0 */ + char buf[27]; + const char compat[] = "xen,shared-memory-v1"; + __be32 reg[4]; + __be32 *cells; + unsigned int len = (addrcells + sizecells) * sizeof(__be32); + + snprintf(buf, sizeof(buf), "xen-shmem@%"PRIx64, mem->bank[i].start); + res = fdt_begin_node(fdt, buf); + if ( res ) + return res; + + res = fdt_property(fdt, "compatible", compat, sizeof(compat)); + if ( res ) + return res; + + cells = reg; + dt_child_set_range(&cells, addrcells, sizecells, start, size); + + res = fdt_property(fdt, "reg", reg, len); + if ( res ) + return res; + + dt_dprintk("Shared memory bank %lu: %#"PRIx64"->%#"PRIx64"\n", + i, start, start + size); + + res = fdt_property_cell(fdt, "xen,id", shm_id); + if ( res ) + return res; + + res = fdt_end_node(fdt); + if ( res ) + return res; + } + + return res; +} +#else +static int __init make_shm_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + struct meminfo *mem) +{ + ASSERT_UNREACHABLE(); +} +#endif + +static int __init make_resv_memory_node(const struct domain *d, + void *fdt, + int addrcells, int sizecells, + struct meminfo *mem) +{ + int res = 0; + /* Placeholder for reserved-memory\0 */ + char resvbuf[16] = "reserved-memory"; + + if ( mem == NULL ) + /* No shared memory provided. */ + return 0; + + dt_dprintk("Create reserved-memory node\n"); + + res = fdt_begin_node(fdt, resvbuf); + if ( res ) + return res; + + res = fdt_property(fdt, "ranges", NULL, 0); + if ( res ) + return res; + + res = fdt_property_cell(fdt, "#address-cells", addrcells); + if ( res ) + return res; + + res = fdt_property_cell(fdt, "#size-cells", sizecells); + if ( res ) + return res; + + res = make_shm_memory_node(d, fdt, addrcells, sizecells, mem); + if ( res ) + return res; + + res = fdt_end_node(fdt); + + return res; +} + static int __init add_ext_regions(unsigned long s, unsigned long e, void *data) { struct meminfo *ext_regions = data; @@ -3041,6 +3180,11 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo) if ( ret ) goto err; + ret = make_resv_memory_node(d, kinfo->fdt, addrcells, sizecells, + d->arch.shm_mem); + if ( ret ) + goto err; + /* * domain_handle_dtb_bootmodule has to be called before the rest of * the device tree is generated because it depends on the value of diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h index ed63c2b6f9..d0d1e12cfe 100644 --- a/xen/arch/arm/include/asm/domain.h +++ b/xen/arch/arm/include/asm/domain.h @@ -104,6 +104,7 @@ struct arch_domain #endif bool directmap; + struct meminfo *shm_mem; } __cacheline_aligned; struct arch_vcpu diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 38e02ced36..b5909acf68 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -29,6 +29,7 @@ struct membank { bool xen_domain; /* whether the memory bank is bound to a Xen domain. */ #ifdef CONFIG_STATIC_SHM unsigned long nr_shm_domain; + uint8_t shm_id ; /* ID identifier of a static shared memory bank. */ #endif }; From patchwork Fri May 6 07:25:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Penny Zheng X-Patchwork-Id: 12840661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BAFAC433EF for ; Fri, 6 May 2022 07:27:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.322678.544125 (Exim 4.92) (envelope-from ) id 1nmsMo-00030V-JC; Fri, 06 May 2022 07:27:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 322678.544125; Fri, 06 May 2022 07:27:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsMo-00030O-El; Fri, 06 May 2022 07:27:10 +0000 Received: by outflank-mailman (input) for mailman id 322678; Fri, 06 May 2022 07:27:09 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nmsM4-00078X-Qy for xen-devel@lists.xenproject.org; Fri, 06 May 2022 07:26:24 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id d3333901-cd0d-11ec-a406-831a346695d4; Fri, 06 May 2022 09:26:20 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 04C201063; Fri, 6 May 2022 00:26:20 -0700 (PDT) Received: from a011292.shanghai.arm.com (a011292.shanghai.arm.com [10.169.190.94]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 765BD3FA27; Fri, 6 May 2022 00:26:17 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d3333901-cd0d-11ec-a406-831a346695d4 From: Penny Zheng To: xen-devel@lists.xenproject.org Cc: wei.chen@arm.com, Penny Zheng , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH v2 9/9] xen/arm: enable statically shared memory on Dom0 Date: Fri, 6 May 2022 15:25:02 +0800 Message-Id: <20220506072502.2177828-10-Penny.Zheng@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220506072502.2177828-1-Penny.Zheng@arm.com> References: <20220506072502.2177828-1-Penny.Zheng@arm.com> MIME-Version: 1.0 From: Penny Zheng To add statically shared memory nodes in Dom0, user shall put according static shared memory configuration under /chosen node. This commit adds shm-processing function process_shm in construct_dom0 to enable statically shared memory on Dom0. Signed-off-by: Penny Zheng Reviewed-by: Stefano Stabellini --- v2 change: - no change --- xen/arch/arm/domain_build.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index f08606d2c0..9da0a0c88b 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2628,6 +2628,11 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo, if ( res ) return res; } + + res = make_resv_memory_node(d, kinfo->fdt, addrcells, sizecells, + d->arch.shm_mem); + if ( res ) + return res; } res = fdt_end_node(kinfo->fdt); @@ -3639,6 +3644,9 @@ static int __init construct_dom0(struct domain *d) { struct kernel_info kinfo = {}; int rc; +#ifdef CONFIG_STATIC_SHM + const struct dt_device_node *chosen = dt_find_node_by_path("/chosen"); +#endif /* Sanity! */ BUG_ON(d->domain_id != 0); @@ -3673,6 +3681,12 @@ static int __init construct_dom0(struct domain *d) allocate_memory_11(d, &kinfo); find_gnttab_region(d, &kinfo); +#ifdef CONFIG_STATIC_SHM + rc = process_shm(d, chosen); + if ( rc < 0 ) + return rc; +#endif + /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */ rc = gic_map_hwdom_extra_mappings(d); if ( rc < 0 )