From patchwork Tue May 19 17:20:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 11558255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1B2A13B1 for ; Tue, 19 May 2020 17:22:45 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F3FA2075F for ; Tue, 19 May 2020 17:22:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="gSJKewAq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F3FA2075F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5uu-0003uo-RP; Tue, 19 May 2020 17:20:36 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5ut-0003uj-Gw for xen-devel@lists.xenproject.org; Tue, 19 May 2020 17:20:35 +0000 X-Inumbo-ID: 0cb3400c-99f5-11ea-b07b-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0cb3400c-99f5-11ea-b07b-bc764e2007e4; Tue, 19 May 2020 17:20:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=/nUeUpuVV0k6Tn2LGbWheh4wHL0r38S9VrtLJqRIuNc=; b=gSJKewAqElCrYrsymiW8f6VAGR qilPwMQ04D3whhl1Nh9PPboEHO56jG0QnHneY+CsGa992Bt5kP4cc4hFs1HDF7d9LJJtuxTQsHpbR A+ghRUv/gfL+SUYEwZWam2xlf+BSpfcdb9WfgRMxPg5prTlrvJeo84ijOmRPorOIqxwU=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5ur-0006gi-IC; Tue, 19 May 2020 17:20:33 +0000 Received: from 54-240-197-227.amazon.com ([54.240.197.227] helo=ufe34d9ed68d054.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jb5ur-0003ie-81; Tue, 19 May 2020 17:20:33 +0000 From: Julien Grall To: xen-devel@lists.xenproject.org Subject: [PATCH v2 for-4.14 1/3] xen/arm: Allow a platform to override the DMA width Date: Tue, 19 May 2020 18:20:26 +0100 Message-Id: <20200519172028.31169-2-julien@xen.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200519172028.31169-1-julien@xen.org> References: <20200519172028.31169-1-julien@xen.org> X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Corey Minyard , minyard@acm.org, Andrew Cooper , Julien Grall , roman@zededa.com, George Dunlap , jeff.kubascik@dornerworks.com, Jan Beulich , Volodymyr Babchuk Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Julien Grall At the moment, Xen is assuming that all the devices are at least 32-bit DMA capable. However, some SoC have devices that may be able to access a much restricted range. For instance, the RPI has devices that can only access the first 1GB of RAM. The structure platform_desc is now extended to allow a platform to override the DMA width. The new is used to implement arch_get_dma_bit_size(). The prototype is now moved in asm-arm/mm.h as the function is not NUMA specific. The implementation is done in platform.c so we don't have to include platform.h everywhere. This should be fine as the function is not expected to be called in hotpath. Signed-off-by: Julien Grall Reviewed-by: Volodymyr Babchuk Tested-by: Corey Minyard --- Cc: Jan Beulich Cc: Andrew Cooper Cc: George Dunlap Changes in v2: - Add Corey's tested-by - Add Volodymyr's reviewed-by I noticed that arch_get_dma_bit_size() is only called when there is more than one NUMA node. I am a bit unsure what is the reason behind it. The goal for Arm is to use arch_get_dma_bit_size() when deciding how low the first Dom0 bank should be allocated. --- xen/arch/arm/platform.c | 5 +++++ xen/include/asm-arm/mm.h | 2 ++ xen/include/asm-arm/numa.h | 5 ----- xen/include/asm-arm/platform.h | 2 ++ 4 files changed, 9 insertions(+), 5 deletions(-) diff --git a/xen/arch/arm/platform.c b/xen/arch/arm/platform.c index 8eb0b6e57a5a..4db5bbb4c51d 100644 --- a/xen/arch/arm/platform.c +++ b/xen/arch/arm/platform.c @@ -155,6 +155,11 @@ bool platform_device_is_blacklisted(const struct dt_device_node *node) return (dt_match_node(blacklist, node) != NULL); } +unsigned int arch_get_dma_bitsize(void) +{ + return ( platform && platform->dma_bitsize ) ? platform->dma_bitsize : 32; +} + /* * Local variables: * mode: C diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h index 7df91280bc77..f8ba49b1188f 100644 --- a/xen/include/asm-arm/mm.h +++ b/xen/include/asm-arm/mm.h @@ -366,6 +366,8 @@ int arch_acquire_resource(struct domain *d, unsigned int type, unsigned int id, return -EOPNOTSUPP; } +unsigned int arch_get_dma_bitsize(void); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/include/asm-arm/numa.h b/xen/include/asm-arm/numa.h index 490d1f31aa14..31a6de4e2346 100644 --- a/xen/include/asm-arm/numa.h +++ b/xen/include/asm-arm/numa.h @@ -25,11 +25,6 @@ extern mfn_t first_valid_mfn; #define node_start_pfn(nid) (mfn_x(first_valid_mfn)) #define __node_distance(a, b) (20) -static inline unsigned int arch_get_dma_bitsize(void) -{ - return 32; -} - #endif /* __ARCH_ARM_NUMA_H */ /* * Local variables: diff --git a/xen/include/asm-arm/platform.h b/xen/include/asm-arm/platform.h index ed4d30a1be7c..997eb2521631 100644 --- a/xen/include/asm-arm/platform.h +++ b/xen/include/asm-arm/platform.h @@ -38,6 +38,8 @@ struct platform_desc { * List of devices which must not pass-through to a guest */ const struct dt_device_match *blacklist_dev; + /* Override the DMA width (32-bit by default). */ + unsigned int dma_bitsize; }; /* From patchwork Tue May 19 17:20:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 11558251 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8B2613B1 for ; Tue, 19 May 2020 17:22:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B61E42075F for ; Tue, 19 May 2020 17:22:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="Qie2SCxk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B61E42075F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5uy-0003vA-36; Tue, 19 May 2020 17:20:40 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5uw-0003uu-AI for xen-devel@lists.xenproject.org; Tue, 19 May 2020 17:20:38 +0000 X-Inumbo-ID: 0d1efad6-99f5-11ea-a95a-12813bfff9fa Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0d1efad6-99f5-11ea-a95a-12813bfff9fa; Tue, 19 May 2020 17:20:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ph7bs82KFKJYE5DJAdCJd2/AW9SaWrzXSk1YjKd+pXk=; b=Qie2SCxkxvamT9ybAE0Z2M7mXp Sm/gE4ceiUNR1CbEJ1JA90DTuyW1hT5y9YJP0p1bdDhMdjsEmXI12ZHzpEpGPmk6s0IsPHKJjRbvz FbjG++WkKM0FMVKiVoeXhLrSSKkpabQ8As9uRP2HoqR3/FKsIik1D7PywBnYcbHNOXZo=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5us-0006gn-NE; Tue, 19 May 2020 17:20:34 +0000 Received: from 54-240-197-227.amazon.com ([54.240.197.227] helo=ufe34d9ed68d054.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jb5us-0003ie-Dx; Tue, 19 May 2020 17:20:34 +0000 From: Julien Grall To: xen-devel@lists.xenproject.org Subject: [PATCH v2 for-4.14 2/3] xen/arm: Take into account the DMA width when allocating Dom0 memory banks Date: Tue, 19 May 2020 18:20:27 +0100 Message-Id: <20200519172028.31169-3-julien@xen.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200519172028.31169-1-julien@xen.org> References: <20200519172028.31169-1-julien@xen.org> X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Corey Minyard , Julien Grall , roman@zededa.com, jeff.kubascik@dornerworks.com, minyard@acm.org Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Julien Grall At the moment, Xen is assuming that all the devices are at least 32-bit DMA capable. However, some SoCs have devices that may be able to access a much restricted range. For instance, the Raspberry PI 4 has devices that can only access the first GB of RAM. The function arch_get_dma_bit_size() will return the lowest DMA width on the platform. Use it to decide what is the limit for the low memory. Signed-off-by: Julien Grall Tested-by: Corey Minyard --- Changes in v2: - Remove left-over in the comment - Add Corey's tested-by --- xen/arch/arm/domain_build.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 430708753642..3d7a75c31881 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -211,10 +211,12 @@ fail: * the ramdisk and DTB must be placed within a certain proximity of * the kernel within RAM. * 3. For dom0 we want to place as much of the RAM as we reasonably can - * below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit) - * or when a device assigned to dom0 can only do 32-bit DMA access. - * 4. For 32-bit dom0 the kernel must be located below 4GB. - * 5. We want to have a few largers banks rather than many smaller ones. + * below 4GB, so that it can be used by non-LPAE enabled kernels (32-bit). + * 4. Some devices assigned to dom0 can only do 32-bit DMA access or + * even be more restricted. We want to allocate as much of the RAM + * as we reasonably can that can be accessed from all the devices.. + * 5. For 32-bit dom0 the kernel must be located below 4GB. + * 6. We want to have a few largers banks rather than many smaller ones. * * For the first two requirements we need to make sure that the lowest * bank is sufficiently large. @@ -245,9 +247,9 @@ fail: * we give up. * * For 32-bit domain we require that the initial allocation for the - * first bank is under 4G. For 64-bit domain, the first bank is preferred - * to be allocated under 4G. Then for the subsequent allocations we - * initially allocate memory only from below 4GB. Once that runs out + * first bank is part of the low mem. For 64-bit, the first bank is preferred + * to be allocated in the low mem. Then for subsequent allocation, we + * initially allocate memory only from low mem. Once that runs out out * (as described above) we allow higher allocations and continue until * that runs out (or we have allocated sufficient dom0 memory). */ @@ -262,6 +264,7 @@ static void __init allocate_memory_11(struct domain *d, int i; bool lowmem = true; + unsigned int lowmem_bitsize = min(32U, arch_get_dma_bitsize()); unsigned int bits; /* @@ -282,7 +285,7 @@ static void __init allocate_memory_11(struct domain *d, */ while ( order >= min_low_order ) { - for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ ) + for ( bits = order ; bits <= lowmem_bitsize; bits++ ) { pg = alloc_domheap_pages(d, order, MEMF_bits(bits)); if ( pg != NULL ) @@ -296,24 +299,26 @@ static void __init allocate_memory_11(struct domain *d, order--; } - /* Failed to allocate bank0 under 4GB */ + /* Failed to allocate bank0 in the lowmem region. */ if ( is_32bit_domain(d) ) panic("Unable to allocate first memory bank\n"); - /* Try to allocate memory from above 4GB */ - printk(XENLOG_INFO "No bank has been allocated below 4GB.\n"); + /* Try to allocate memory from above the lowmem region */ + printk(XENLOG_INFO "No bank has been allocated below %u-bit.\n", + lowmem_bitsize); lowmem = false; got_bank0: /* - * If we failed to allocate bank0 under 4GB, continue allocating - * memory from above 4GB and fill in banks. + * If we failed to allocate bank0 in the lowmem region, + * continue allocating from above the lowmem and fill in banks. */ order = get_allocation_size(kinfo->unassigned_mem); while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS ) { - pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0); + pg = alloc_domheap_pages(d, order, + lowmem ? MEMF_bits(lowmem_bitsize) : 0); if ( !pg ) { order --; From patchwork Tue May 19 17:20:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 11558253 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10D6B60D for ; Tue, 19 May 2020 17:22:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E24882075F for ; Tue, 19 May 2020 17:22:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=xen.org header.i=@xen.org header.b="VlugWGv3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E24882075F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5v0-0003vU-Aq; Tue, 19 May 2020 17:20:42 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5uy-0003vI-Fx for xen-devel@lists.xenproject.org; Tue, 19 May 2020 17:20:40 +0000 X-Inumbo-ID: 0dc478a8-99f5-11ea-b07b-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0dc478a8-99f5-11ea-b07b-bc764e2007e4; Tue, 19 May 2020 17:20:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=SOZlp61CWrsQD7qa0dasRDDLg9x7AQZ16sVemh7nqoo=; b=VlugWGv3QcqQsn6sakAdRX3wC+ jaNvLHe2n3WAdZIvmI1rzb9fMEeBAk8bDRF0s/TStPZ2dxwYkgJiPS1FrEXBtGhFAKZptrSLBDCWq j0xnHURlEPqkzKUhfrclevpmamAyl9anxhdxDUvs1w1f2zjt7hZqvMLmoQmMxoYfRMnE=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jb5ut-0006gs-Sx; Tue, 19 May 2020 17:20:35 +0000 Received: from 54-240-197-227.amazon.com ([54.240.197.227] helo=ufe34d9ed68d054.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1jb5ut-0003ie-Js; Tue, 19 May 2020 17:20:35 +0000 From: Julien Grall To: xen-devel@lists.xenproject.org Subject: [PATCH v2 for-4.14 3/3] xen/arm: plat: Allocate as much as possible memory below 1GB for dom0 for RPI Date: Tue, 19 May 2020 18:20:28 +0100 Message-Id: <20200519172028.31169-4-julien@xen.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200519172028.31169-1-julien@xen.org> References: <20200519172028.31169-1-julien@xen.org> X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Corey Minyard , Julien Grall , roman@zededa.com, jeff.kubascik@dornerworks.com, minyard@acm.org Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Julien Grall The raspberry PI 4 has devices that can only DMA into the first GB of the RAM. Therefore we want allocate as much as possible memory below 1GB for dom0. Use the recently introduced dma_bitsize field to specify the DMA width supported. Signed-off-by: Julien Grall Reported-by: Corey Minyard Tested-by: Corey Minyard --- Changes in v2: - 1G is 30 bits not 10! - Add Corey's tested-by --- xen/arch/arm/platforms/brcm-raspberry-pi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/xen/arch/arm/platforms/brcm-raspberry-pi.c b/xen/arch/arm/platforms/brcm-raspberry-pi.c index b697fa2c6c0e..f5ae58a7d5f2 100644 --- a/xen/arch/arm/platforms/brcm-raspberry-pi.c +++ b/xen/arch/arm/platforms/brcm-raspberry-pi.c @@ -43,6 +43,7 @@ static const struct dt_device_match rpi4_blacklist_dev[] __initconst = PLATFORM_START(rpi4, "Raspberry Pi 4") .compatible = rpi4_dt_compat, .blacklist_dev = rpi4_blacklist_dev, + .dma_bitsize = 30, PLATFORM_END /*