From patchwork Tue Mar 5 11:18:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13582159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8220BC54798 for ; Tue, 5 Mar 2024 11:19:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AF866B00D9; Tue, 5 Mar 2024 06:19:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1383B6B00DA; Tue, 5 Mar 2024 06:19:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA54A6B00DB; Tue, 5 Mar 2024 06:19:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D41EE6B00D9 for ; Tue, 5 Mar 2024 06:19:40 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 73DFE120CEE for ; Tue, 5 Mar 2024 11:19:40 +0000 (UTC) X-FDA: 81862740120.01.7F092E5 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf19.hostedemail.com (Postfix) with ESMTP id 200FC1A0024 for ; Tue, 5 Mar 2024 11:19:37 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CEpa5Pm+; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709637578; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qYb+zBT+l4bRumlZjw61WjNIKyLOpqGiPdeR0ak+ylo=; b=BSZ/Mfhs4U7fJKa4GdErTr4G2RN8zD25S2FeTdKS8QPhzFEEcye/9ICPoXEzu9kZNuj/s1 OzCUhhWgqLknRtWOMyDod9fs/Wsw5Gd4/aYN9BGdYl9HwWJhUBGT/4pG+TXXELU/BHGIqK q8m7xds8X7Lk/th9IfDJudo+2v9QDjg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CEpa5Pm+; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709637578; a=rsa-sha256; cv=none; b=qMUsiShp0DXvtFl7WVT99wQaimlWS8AZ+AKN1z2X/qtXwcI5Q8gpR+tz3UpaDu5BMLP6Kf 7Q50KwX3WLs6cLiNBWkuyJHWUuCP5Z2I7byQo3q0s+2UXfBUzyowXKuqVv6elr78NUM44x xggR+iHbM2ngTdhNNv4TRtR5b4jlAhI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B88C0CE1870; Tue, 5 Mar 2024 11:19:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59047C43390; Tue, 5 Mar 2024 11:19:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709637574; bh=inT0MN5YzB4Hs5AmOhFFcCz6zSy1Lq0FeGslm64Slg8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CEpa5Pm+Qqhj2EfkdTwTrAYyAvXIl0YrVxvx+DWKOg/8mx3vqZQn32kmJlhI1Jn5n GkNRqePnBQ708Sr6/JNHoy038LyclUT7V9ppbAO8kM3l2Gld5oEnVCacbFXZ6lGe2x 8BOHhnFPUHmAF/ZNgkUh9zkKatGZOG1TSdpOKgPf8s6lDWYji4TtdmCdYu2MElxBbE isrwPMzFErClEesG4gO7SMHRYcWtqsTO09CRwC5DnOQrJiJQcEenGg/0iPLenz8go1 a5fPge+AvbZx/ANhEW4V7LdpvOPRHNZyJ32k6pOXqocpU3f+5jaKdRBPtPTL+AfaFI ckvCzL1UiaHcg== From: Leon Romanovsky To: Christoph Hellwig , Robin Murphy , Marek Szyprowski , Joerg Roedel , Will Deacon , Jason Gunthorpe , Chaitanya Kulkarni Cc: Leon Romanovsky , Jonathan Corbet , Jens Axboe , Keith Busch , Sagi Grimberg , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?utf-8?b?SsOpcsO0bWUgR2xp?= =?utf-8?b?c3Nl?= , Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, kvm@vger.kernel.org, linux-mm@kvack.org, Bart Van Assche , Damien Le Moal , Amir Goldstein , "josef@toxicpanda.com" , "Martin K. Petersen" , "daniel@iogearbox.net" , Dan Williams , "jack@suse.com" , Zhu Yanjun Subject: [RFC RESEND 10/16] RDMA/umem: Prevent UMEM ODP creation with SWIOTLB Date: Tue, 5 Mar 2024 13:18:41 +0200 Message-ID: <8c6d5e7db2d1a01888cc7b9b9850b05e19c75c64.1709635535.git.leon@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 200FC1A0024 X-Stat-Signature: 8tg9xbs5br4z5wgnsahmrqeg5zjqcet5 X-Rspam-User: X-HE-Tag: 1709637577-522532 X-HE-Meta: U2FsdGVkX1/RDYPmcVVXNrJKfOxlRaNQFtY89SCO1hFnysniw2xpkghkz+9wnCDfpN8O9k/2hymQ/4b+8U6Vcv+Qu8WgH5kTmxiPXv5B7dFNxoCbqKU4zCP1gbR31VVAzJD4VagQzjUqsR/w3VjU4YyrNU46KnFESIWat4ZYnykK1VyDr3NSpS+SaIyUxLg3/ne5OaiOWURR8QXNghkjaw5c2+l2MSJX1kkB80AXiGISe4UAe6RmQOxMOz0SnpvBvdNzFET8PszpuNEQhYnFyRJMl9IsIDb8xm67EI8IL9fWSDntCx0WL+8tlP8DB5pvpkDfHS0l+z3eucfiy2KaROBVHTRKAHblowMDZcnEeZFBBBVl8QZi/8dzcbpnhnKliSVcCK8rsSwwXDi0yiWRi2VRFK0/AcW0UIOJjgSCTfqaB6E5WfcTYDDinpDzqo7pt4pLMhKGOVWnbxrsLSi/w+xRm3Xo+GtS7RReVP7/N1xZ3z0LJqfEEFGK7r2UGIeTDrdslmCTvOlQ3IWfHiu2e1IApec5cXf62t6E+D1KRE6LJIywo+3sI8So0TzLJb/c5uyaZErIb982Aruhph01uYlUgg/PQdiw4ngv1x4vS82SyF/NqVKOsWYS9iP6iGHDo1keZf4+kPoWWKEYM/0yS40lAzC7O5D4XSV0H4qGXbppJnUnLJax2L9FxCNl/BhaQAv1bi07MhqXgODDyYFQFzGVjBqPq++ZW8H7aygJEKL9UCvYKTscDt16awT3m4s2WS6CKLqPPDkpV8v3szi058GxsL7jIIZzbNmMew4rt0spV0YKpeAQLd2ncAVyGhOSIoMBvrkyQ60Tvf+P/WNhkYkKpxFcmqt9pM8qOfx5UFl3zcsE9P5DOLRZDdiZfH0aPK+QplU9x8pdoUHWHMPhVhSU0KtMeZni+7Cn6blh3Vqwt+pp9mT7YryEJ7N7XYnbDYjTCk8hnDkNU9oLFMZ HnbbGxLS gNGIFpp3+DvfNVVdr7jZAV6YUDg9acqoCusOzk/Hma4HFD5/kOcMtDsUQQMxSWJzQ5gmimPgY01tD3KEBz73L+YEJOC9l5CuXBRSATwFlwGvw7yMY7SQoACm1E4QgDzeaIAcbOb7q3fNV4hc+PHPVOc4Iv1vvE6SWtN15xub5EtLReWoHKI7HAbjffw+785rd0TgWyBjH/BWZXpJqtdPuRNT3eVttTfuWnukxTNyVIosti3B4AriCzjGqsA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Leon Romanovsky RDMA UMEM never supported DMA addresses returned from SWIOTLB, as these addresses should be programmed to the hardware which is not aware that it is bounce buffers and not real ones. Instead of silently leave broken system for the users who didn't know it, let's be explicit and return an error to them. Signed-off-by: Leon Romanovsky --- Documentation/core-api/dma-attributes.rst | 7 +++ drivers/infiniband/core/umem_odp.c | 77 +++++++++++------------ include/linux/dma-mapping.h | 6 ++ kernel/dma/direct.h | 4 +- kernel/dma/mapping.c | 4 ++ 5 files changed, 58 insertions(+), 40 deletions(-) diff --git a/Documentation/core-api/dma-attributes.rst b/Documentation/core-api/dma-attributes.rst index 1887d92e8e92..b337ec65d506 100644 --- a/Documentation/core-api/dma-attributes.rst +++ b/Documentation/core-api/dma-attributes.rst @@ -130,3 +130,10 @@ accesses to DMA buffers in both privileged "supervisor" and unprivileged subsystem that the buffer is fully accessible at the elevated privilege level (and ideally inaccessible or at least read-only at the lesser-privileged levels). + +DMA_ATTR_NO_TRANSLATION +----------------------- + +This attribute is used to indicate to the DMA-mapping subsystem that the +buffer is not subject to any address translation. This is used for devices +that doesn't need buffer bouncing or fixing DMA addresses. diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 1301009a6b78..57c56000f60e 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -50,51 +50,50 @@ static inline int ib_init_umem_odp(struct ib_umem_odp *umem_odp, const struct mmu_interval_notifier_ops *ops) { + size_t page_size = 1UL << umem_odp->page_shift; struct ib_device *dev = umem_odp->umem.ibdev; + size_t ndmas, npfns; + unsigned long start; + unsigned long end; int ret; umem_odp->umem.is_odp = 1; mutex_init(&umem_odp->umem_mutex); - if (!umem_odp->is_implicit_odp) { - size_t page_size = 1UL << umem_odp->page_shift; - unsigned long start; - unsigned long end; - size_t ndmas, npfns; - - start = ALIGN_DOWN(umem_odp->umem.address, page_size); - if (check_add_overflow(umem_odp->umem.address, - (unsigned long)umem_odp->umem.length, - &end)) - return -EOVERFLOW; - end = ALIGN(end, page_size); - if (unlikely(end < page_size)) - return -EOVERFLOW; - - ndmas = (end - start) >> umem_odp->page_shift; - if (!ndmas) - return -EINVAL; - - npfns = (end - start) >> PAGE_SHIFT; - umem_odp->pfn_list = kvcalloc( - npfns, sizeof(*umem_odp->pfn_list), GFP_KERNEL); - if (!umem_odp->pfn_list) - return -ENOMEM; - - - umem_odp->iova.dev = dev->dma_device; - umem_odp->iova.size = end - start; - umem_odp->iova.dir = DMA_BIDIRECTIONAL; - ret = ib_dma_alloc_iova(dev, &umem_odp->iova); - if (ret) - goto out_pfn_list; - - ret = mmu_interval_notifier_insert(&umem_odp->notifier, - umem_odp->umem.owning_mm, - start, end - start, ops); - if (ret) - goto out_free_iova; - } + if (umem_odp->is_implicit_odp) + return 0; + + start = ALIGN_DOWN(umem_odp->umem.address, page_size); + if (check_add_overflow(umem_odp->umem.address, + (unsigned long)umem_odp->umem.length, &end)) + return -EOVERFLOW; + end = ALIGN(end, page_size); + if (unlikely(end < page_size)) + return -EOVERFLOW; + + ndmas = (end - start) >> umem_odp->page_shift; + if (!ndmas) + return -EINVAL; + + npfns = (end - start) >> PAGE_SHIFT; + umem_odp->pfn_list = + kvcalloc(npfns, sizeof(*umem_odp->pfn_list), GFP_KERNEL); + if (!umem_odp->pfn_list) + return -ENOMEM; + + umem_odp->iova.dev = dev->dma_device; + umem_odp->iova.size = end - start; + umem_odp->iova.dir = DMA_BIDIRECTIONAL; + umem_odp->iova.attrs = DMA_ATTR_NO_TRANSLATION; + ret = ib_dma_alloc_iova(dev, &umem_odp->iova); + if (ret) + goto out_pfn_list; + + ret = mmu_interval_notifier_insert(&umem_odp->notifier, + umem_odp->umem.owning_mm, start, + end - start, ops); + if (ret) + goto out_free_iova; return 0; diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 91cc084adb53..89945e707a9b 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -62,6 +62,12 @@ */ #define DMA_ATTR_PRIVILEGED (1UL << 9) +/* + * DMA_ATTR_NO_TRANSLATION: used to indicate that the buffer should not be mapped + * through address translation. + */ +#define DMA_ATTR_NO_TRANSLATION (1UL << 10) + /* * A dma_addr_t can hold any valid DMA or bus address for the platform. It can * be given to a device to use as a DMA source or target. It is specific to a diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index 1c30e1cd607a..1c9ec204c999 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -92,6 +92,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, if (is_swiotlb_force_bounce(dev)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; + if (attrs & DMA_ATTR_NO_TRANSLATION) + return DMA_MAPPING_ERROR; return swiotlb_map(dev, phys, size, dir, attrs); } @@ -99,7 +101,7 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; - if (is_swiotlb_active(dev)) + if (is_swiotlb_active(dev) && !(attrs & DMA_ATTR_NO_TRANSLATION)) return swiotlb_map(dev, phys, size, dir, attrs); dev_WARN_ONCE(dev, 1, diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index f989c64622c2..49b1fde510c5 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -188,6 +188,10 @@ int dma_alloc_iova(struct dma_iova_attrs *iova) struct device *dev = iova->dev; const struct dma_map_ops *ops = get_dma_ops(dev); + if (dma_map_direct(dev, ops) && is_swiotlb_force_bounce(dev) && + iova->attrs & DMA_ATTR_NO_TRANSLATION) + return -EOPNOTSUPP; + if (dma_map_direct(dev, ops) || !ops->alloc_iova) { iova->addr = 0; return 0;