From patchwork Fri Jan 17 10:03:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13943119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C58D4C02185 for ; Fri, 17 Jan 2025 10:04:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59A1D280007; Fri, 17 Jan 2025 05:04:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 54A4A280001; Fri, 17 Jan 2025 05:04:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C46F280007; Fri, 17 Jan 2025 05:04:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1CFF2280001 for ; Fri, 17 Jan 2025 05:04:17 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D062DC188B for ; Fri, 17 Jan 2025 10:04:16 +0000 (UTC) X-FDA: 83016508512.14.D6AC55B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 1E4B440009 for ; Fri, 17 Jan 2025 10:04:14 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mRCFjuqn; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737108255; a=rsa-sha256; cv=none; b=oNHuulxckYkG/ksW0OJwMu2cu89b1hY3zdpk8YJDDS+Sa8mTWpiSP51b7qcqNjFSC0Cjd7 v2XHlVoUaEYGXaGiqVlody/UfzFxZ/SeEVE6gtHhBRY1WFVfc/e1jqNLDOhG20X7XWTKcx fcdfyQLY0o/QrdtAt7P5loi5xe8y70k= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mRCFjuqn; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of leon@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=leon@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737108255; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OUFgoi3FQNM98+ha+bc/i/4AE2ysI0eOkSmgUOa4jHI=; b=i6RYi09qjxrSEWFLVe5RFf5uuyF9ovt6hfpWmp56YW//GaHo1oXNnCKXwmmzM73B4XS4i8 7h+0lJzLeGq+e0FlymkaNnvgW2x5k3AtCSFBqlZqT0mO9SytCi1kuzGxiXtQ3JO5ikXZ+Z 3x8PtsL5jOQMEcOvj7lKgyzRSrgYnmQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id ADF225C5714; Fri, 17 Jan 2025 10:03:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 658ACC4CEE3; Fri, 17 Jan 2025 10:04:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737108254; bh=b6d3Ock46XbthcVFLofuUHJCITAgXKG9siQAvYGaAp8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mRCFjuqnJtaIWwe5/GpzaWd63p9tgJQU/oErGC9PUU1ISFs/68kuynU5ej1dZAsGI Gv1xe8qaH5EJ+rljZi5lBb7NfaOU2iB3qSgNLFgHwskkOTeGkQ1huEgjNMNe4YT7Uc qtgiYU/SAt406jHKheWu9G1MOrALkqESIaaXid3bfWfQrJrX7Qvlm0XmmPWL85HcfN fxT61GZZAgV9QxkIM6rzDebaoDwtghxnRq33HIu0tg6fTJw6v3DuUP/BQKz/tibbN/ YMemYu8n/YzF8sIDBocLOnHplH6tMC+0ZCNvntu9J383dqY7xIUgaiA3bXhz7xQsmQ 1t/Kl/NZpZhpg== From: Leon Romanovsky To: Christoph Hellwig , Jason Gunthorpe , Robin Murphy Cc: Jens Axboe , Joerg Roedel , Will Deacon , Sagi Grimberg , "Keith Busch" , "Bjorn Helgaas" , "Logan Gunthorpe" , "Yishai Hadas" , "Shameer Kolothum" , "Kevin Tian" , "Alex Williamson" , "Marek Szyprowski" , =?utf-8?b?SsOpcsO0bWUgR2xp?= =?utf-8?b?c3Nl?= , "Andrew Morton" , "Jonathan Corbet" , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, "Randy Dunlap" Subject: [PATCH v6 02/17] dma-mapping: move the PCI P2PDMA mapping helpers to pci-p2pdma.h Date: Fri, 17 Jan 2025 12:03:33 +0200 Message-ID: <15e9becd1a061b538b44cbe02a47beeed0f53771.1737106761.git.leon@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1E4B440009 X-Stat-Signature: m9b5kzf6mhcdjawqg973boqpt6e9cuy6 X-Rspam-User: X-HE-Tag: 1737108254-598004 X-HE-Meta: U2FsdGVkX1/s1LT1XTEr9K2vgtQiqWexKaCggpwvQ3G/8ZHzg7oRqaSAa61JCQiiYC7Yc5UB7lHhN0gZN+XMC09migA6RkWpdz6RgTHJnx0vRBkJ6uLMdoi1AVsZRfhgRphGWokKFm1NpvP7uJ9IOr4fLoKxqJtYi3zxu21gb+A7PPs2vMHFBvw35RNvKtB2SxFyGroyHYcGH9yC6o9lreVToTBsKF4dulfQPNacp0tJj5+8lQfus0NNXmhf4GUfru7YXha46wI+j+hWcbQPzQ3iIO4MKOCUbF9eenpYsmuYA7CTeBccvvG4Pz8CSQ9tzOFOQ6vglWvNS/G2lTYkvj72d6eIQkOYojiE4AsuS8bwHcVDwu1iqFAXYebLtTilOsR7O7FhKtf/gadFdsaMTe8jCHQ+LVO6d22XqZFAAv8862Y7HBJq2vWFi7XzlYm+pwGvQ+EdkYhQO4IS2f+uB4pCoaUyV4/ASVDPJm+3+EuhoSzibsIE7SOqscolZhld/ZL3iVBGfCPcA2Y2FO0elNI2KTW8nb1tqHYy0byL+4dbOvXZoYCqYH+2j2VrE8uEIGzf2K7Ksy2Map8iyhP5s8mUxlJrrlzic3hCZCkcMsBcYoxD7w/4MsPe1I7iuVPWgi6SDOiD6YALSfkgSq0Eh/2ZuAlICP1Aej5sQ6wrRte6Awb5AfbXip9Hwu8XzEILoECkmr0tsPZfljMdYOEXWJuOkUoHYLZqgxUH2IVLxycjn8mbqkZha1ZW9vgU6nK7uV83i6pZUAV8NuvII5T74OUM0xaUQ+yCv6W2UEkeGp2bQfmtib7rEFLA8eFDb4ct5lR9NSzL1+QUVoBLg1D+r+HmJt5EdEEIRA6DQsKNFdbjN69TbQPS+qNQzfO/JT91pVWdfCxx+0h9VjwvYoB4RS4Ix8KcamObtvcLMbyTwKZotwnJXWxIRRHxdC+qp3LSX1lqX4abNDNBX9VwZPL 0uyRxWnA D752UkXBLer+O3TNCxbrTl7WD+3svtE5IMEYlghGgLeV0T8IXrkPO0jPjWakkmMKN5kBxZ5MVjY9MfAqU17lbhSutxkNKQhd4Rerl22fzRU00O/FdjIcpmysqH2CPI4EkfVjpv9cq9nzY9PGTpBxsnfMHDQ+nV7FK1EAC9thSES3Za9WYpvRqGvNdJ7EoUrvG2zddJmPiSp3tqz9MvD42o1CLk1WzsExRE+ptmA4SCl2SI2m85gMAJn3AtKEL7erUATc0RNpXC5ddTpp2BPRdRoeUPOrXTC6sCfxXcCR0IIXYrLazKIa+Xgo5ezek7cH5Mslk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Christoph Hellwig To support the upcoming non-scatterlist mapping helpers, we need to go back to have them called outside of the DMA API. Thus move them out of dma-map-ops.h, which is only for DMA API implementations to pci-p2pdma.h, which is for driver use. Note that the core helper is still not exported as the mapping is expected to be done only by very highlevel subsystem code at least for now. Signed-off-by: Christoph Hellwig Reviewed-by: Logan Gunthorpe Acked-by: Bjorn Helgaas Signed-off-by: Leon Romanovsky --- drivers/iommu/dma-iommu.c | 1 + include/linux/dma-map-ops.h | 85 ------------------------------------- include/linux/pci-p2pdma.h | 84 ++++++++++++++++++++++++++++++++++++ kernel/dma/direct.c | 1 + 4 files changed, 86 insertions(+), 85 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 5746ffaf0061..853247c42f7d 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 63dd480e209b..f48e5fb88bd5 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -434,89 +434,4 @@ static inline void debug_dma_dump_mappings(struct device *dev) #endif /* CONFIG_DMA_API_DEBUG */ extern const struct dma_map_ops dma_dummy_ops; - -enum pci_p2pdma_map_type { - /* - * PCI_P2PDMA_MAP_UNKNOWN: Used internally for indicating the mapping - * type hasn't been calculated yet. Functions that return this enum - * never return this value. - */ - PCI_P2PDMA_MAP_UNKNOWN = 0, - - /* - * Not a PCI P2PDMA transfer. - */ - PCI_P2PDMA_MAP_NONE, - - /* - * PCI_P2PDMA_MAP_NOT_SUPPORTED: Indicates the transaction will - * traverse the host bridge and the host bridge is not in the - * allowlist. DMA Mapping routines should return an error when - * this is returned. - */ - PCI_P2PDMA_MAP_NOT_SUPPORTED, - - /* - * PCI_P2PDMA_BUS_ADDR: Indicates that two devices can talk to - * each other directly through a PCI switch and the transaction will - * not traverse the host bridge. Such a mapping should program - * the DMA engine with PCI bus addresses. - */ - PCI_P2PDMA_MAP_BUS_ADDR, - - /* - * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: Indicates two devices can talk - * to each other, but the transaction traverses a host bridge on the - * allowlist. In this case, a normal mapping either with CPU physical - * addresses (in the case of dma-direct) or IOVA addresses (in the - * case of IOMMUs) should be used to program the DMA engine. - */ - PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, -}; - -struct pci_p2pdma_map_state { - struct dev_pagemap *pgmap; - enum pci_p2pdma_map_type map; - u64 bus_off; -}; - -/* helper for pci_p2pdma_state(), do not use directly */ -void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state, - struct device *dev, struct page *page); - -/** - * pci_p2pdma_state - check the P2P transfer state of a page - * @state: P2P state structure - * @dev: device to transfer to/from - * @page: page to map - * - * Check if @page is a PCI P2PDMA page, and if yes of what kind. Returns the - * map type, and updates @state with all information needed for a P2P transfer. - */ -static inline enum pci_p2pdma_map_type -pci_p2pdma_state(struct pci_p2pdma_map_state *state, struct device *dev, - struct page *page) -{ - if (IS_ENABLED(CONFIG_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { - if (state->pgmap != page->pgmap) - __pci_p2pdma_update_state(state, dev, page); - return state->map; - } - return PCI_P2PDMA_MAP_NONE; -} - -/** - * pci_p2pdma_bus_addr_map - map a PCI_P2PDMA_MAP_BUS_ADDR P2P transfer - * @state: P2P state structure - * @paddr: physical address to map - * - * Map a physically contigous PCI_P2PDMA_MAP_BUS_ADDR transfer. - */ -static inline dma_addr_t -pci_p2pdma_bus_addr_map(struct pci_p2pdma_map_state *state, phys_addr_t paddr) -{ - WARN_ON_ONCE(state->map != PCI_P2PDMA_MAP_BUS_ADDR); - return paddr + state->bus_off; -} - #endif /* _LINUX_DMA_MAP_OPS_H */ diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index 2c07aa6b7665..e839f52b512b 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -104,4 +104,88 @@ static inline struct pci_dev *pci_p2pmem_find(struct device *client) return pci_p2pmem_find_many(&client, 1); } +enum pci_p2pdma_map_type { + /* + * PCI_P2PDMA_MAP_UNKNOWN: Used internally for indicating the mapping + * type hasn't been calculated yet. Functions that return this enum + * never return this value. + */ + PCI_P2PDMA_MAP_UNKNOWN = 0, + + /* + * Not a PCI P2PDMA transfer. + */ + PCI_P2PDMA_MAP_NONE, + + /* + * PCI_P2PDMA_MAP_NOT_SUPPORTED: Indicates the transaction will + * traverse the host bridge and the host bridge is not in the + * allowlist. DMA Mapping routines should return an error when + * this is returned. + */ + PCI_P2PDMA_MAP_NOT_SUPPORTED, + + /* + * PCI_P2PDMA_BUS_ADDR: Indicates that two devices can talk to + * each other directly through a PCI switch and the transaction will + * not traverse the host bridge. Such a mapping should program + * the DMA engine with PCI bus addresses. + */ + PCI_P2PDMA_MAP_BUS_ADDR, + + /* + * PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: Indicates two devices can talk + * to each other, but the transaction traverses a host bridge on the + * allowlist. In this case, a normal mapping either with CPU physical + * addresses (in the case of dma-direct) or IOVA addresses (in the + * case of IOMMUs) should be used to program the DMA engine. + */ + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, +}; + +struct pci_p2pdma_map_state { + struct dev_pagemap *pgmap; + enum pci_p2pdma_map_type map; + u64 bus_off; +}; + +/* helper for pci_p2pdma_state(), do not use directly */ +void __pci_p2pdma_update_state(struct pci_p2pdma_map_state *state, + struct device *dev, struct page *page); + +/** + * pci_p2pdma_state - check the P2P transfer state of a page + * @state: P2P state structure + * @dev: device to transfer to/from + * @page: page to map + * + * Check if @page is a PCI P2PDMA page, and if yes of what kind. Returns the + * map type, and updates @state with all information needed for a P2P transfer. + */ +static inline enum pci_p2pdma_map_type +pci_p2pdma_state(struct pci_p2pdma_map_state *state, struct device *dev, + struct page *page) +{ + if (IS_ENABLED(CONFIG_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { + if (state->pgmap != page->pgmap) + __pci_p2pdma_update_state(state, dev, page); + return state->map; + } + return PCI_P2PDMA_MAP_NONE; +} + +/** + * pci_p2pdma_bus_addr_map - map a PCI_P2PDMA_MAP_BUS_ADDR P2P transfer + * @state: P2P state structure + * @paddr: physical address to map + * + * Map a physically contigous PCI_P2PDMA_MAP_BUS_ADDR transfer. + */ +static inline dma_addr_t +pci_p2pdma_bus_addr_map(struct pci_p2pdma_map_state *state, phys_addr_t paddr) +{ + WARN_ON_ONCE(state->map != PCI_P2PDMA_MAP_BUS_ADDR); + return paddr + state->bus_off; +} + #endif /* _LINUX_PCI_P2P_H */ diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index e289ad27d1b5..c9b3893257d4 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "direct.h" /*