From patchwork Sat Feb 6 00:38:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 12071545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D968C433E0 for ; Sat, 6 Feb 2021 00:39:04 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BE37864FEA for ; Sat, 6 Feb 2021 00:39:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE37864FEA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.81999.151592 (Exim 4.92) (envelope-from ) id 1l8BcS-0007O2-To; Sat, 06 Feb 2021 00:38:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 81999.151592; Sat, 06 Feb 2021 00:38:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l8BcS-0007Nv-P6; Sat, 06 Feb 2021 00:38:36 +0000 Received: by outflank-mailman (input) for mailman id 81999; Sat, 06 Feb 2021 00:38:34 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l8BcQ-0007Nq-Qn for xen-devel@lists.xenproject.org; Sat, 06 Feb 2021 00:38:34 +0000 Received: from mail.kernel.org (unknown [198.145.29.99]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5f739bcf-dcd0-4450-ba04-385f4720d55c; Sat, 06 Feb 2021 00:38:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D82B64FEA; Sat, 6 Feb 2021 00:38:33 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5f739bcf-dcd0-4450-ba04-385f4720d55c DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1612571913; bh=/K2eQTpPEn3hhRchfomiYUpVmHyLfVWO94ver0PRSKA=; h=Date:From:To:cc:Subject:From; b=gPxsO5UT8dIV5AMJKdeJsxtW8wortS8LIw1g/ewQcozPyQfTTqrK/phXWFVK0Jl7M onyn710PQ0CArG5U8VF67vs5LAmZ0PxQxsouBuPOZfxdI2RbasiH/bb/Y7PvjGr7ZV dyFL+XPCahGlmy7wPQ9E+D6pmAVX5rPf53UKmJE2GQvJSyyCPfDtYdGfocxfg8CCys /SP1UfY2dVdh/ImB2BnmszyTilUPfOaWkz1oOcoI2xGR3dIVnFgaBjUx812CPfzBA7 c7E26HPQ0WhS5i/AqeIUeE9WBy0Ss/ZnN6t7F6IuhzsqegCv2Q0Tne7MfSdSTY995w SGCYtJPQT9a5Q== Date: Fri, 5 Feb 2021 16:38:32 -0800 (PST) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s To: julien@xen.org cc: lucmiccio@gmail.com, sstabellini@kernel.org, xen-devel@lists.xenproject.org, Bertrand.Marquis@arm.com, Volodymyr_Babchuk@epam.com Subject: [PATCH] xen/arm: fix gnttab_need_iommu_mapping Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Commit 91d4eca7add broke gnttab_need_iommu_mapping on ARM. The offending chunk is: #define gnttab_need_iommu_mapping(d) \ - (is_domain_direct_mapped(d) && need_iommu(d)) + (is_domain_direct_mapped(d) && need_iommu_pt_sync(d)) On ARM we need gnttab_need_iommu_mapping to be true for dom0 when it is directly mapped, like the old check did, but the new check is always false. In fact, need_iommu_pt_sync is defined as dom_iommu(d)->need_sync and need_sync is set as: if ( !is_hardware_domain(d) || iommu_hwdom_strict ) hd->need_sync = !iommu_use_hap_pt(d); iommu_hwdom_strict is actually supposed to be ignored on ARM, see the definition in docs/misc/xen-command-line.pandoc: This option is hardwired to true for x86 PVH dom0's (as RAM belonging to other domains in the system don't live in a compatible address space), and is ignored for ARM. But aside from that, the issue is that iommu_use_hap_pt(d) is true, hence, hd->need_sync is false, and gnttab_need_iommu_mapping(d) is false too. As a consequence, when using PV network from a domU on a system where IOMMU is on from Dom0, I get: (XEN) smmu: /smmu@fd800000: Unhandled context fault: fsr=0x402, iova=0x8424cb148, fsynr=0xb0001, cb=0 [ 68.290307] macb ff0e0000.ethernet eth0: DMA bus error: HRESP not OK The fix is to go back to the old implementation of gnttab_need_iommu_mapping. However, we don't even need to specify && need_iommu(d) since we don't actually need to check for the IOMMU to be enabled (iommu_map does it for us at the beginning of the function.) This fix is preferrable to changing the implementation of need_sync or iommu_use_hap_pt because "need_sync" is not really the reason why we want gnttab_need_iommu_mapping to return true. Signed-off-by: Stefano Stabellini Backport: 4.12+ --- It is incredible that it was missed for this long, but it takes a full PV drivers test using DMA from a non-coherent device to trigger it, e.g. wget from a domU to an HTTP server on a different machine, ping or connections to dom0 won't trigger the bug. It is interesting that given that IOMMU is on for dom0, Linux could have just avoided using swiotlb-xen and everything would have just worked. It is worth considering introducing a feature flag (e.g. XENFEAT_ARM_dom0_iommu) to let dom0 know that the IOMMU is on and swiotlb-xen is not necessary. Then Linux can avoid initializing swiotlb-xen and just rely on the IOMMU for translation. diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h index 6f585b1538..2a154d1851 100644 --- a/xen/include/asm-arm/grant_table.h +++ b/xen/include/asm-arm/grant_table.h @@ -88,8 +88,7 @@ int replace_grant_host_mapping(unsigned long gpaddr, mfn_t mfn, #define gnttab_status_gfn(d, t, i) \ (((i) >= nr_status_frames(t)) ? INVALID_GFN : (t)->arch.status_gfn[i]) -#define gnttab_need_iommu_mapping(d) \ - (is_domain_direct_mapped(d) && need_iommu_pt_sync(d)) +#define gnttab_need_iommu_mapping(d) (is_domain_direct_mapped(d)) #endif /* __ASM_GRANT_TABLE_H__ */ /*