From patchwork Tue Jul 2 09:09:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 13719168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 736A6C3064D for ; Tue, 2 Jul 2024 09:10:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD9DA6B009D; Tue, 2 Jul 2024 05:10:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D87E56B009E; Tue, 2 Jul 2024 05:10:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C293A6B009F; Tue, 2 Jul 2024 05:10:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A27D16B009D for ; Tue, 2 Jul 2024 05:10:18 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4F0B61C388C for ; Tue, 2 Jul 2024 09:10:18 +0000 (UTC) X-FDA: 82294241316.15.20CB428 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf30.hostedemail.com (Postfix) with ESMTP id 0FC608002B for ; Tue, 2 Jul 2024 09:10:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jZsLMwcx; spf=pass (imf30.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719911405; a=rsa-sha256; cv=none; b=q78VTjrEXYtg1XGU3nItX0P+iSImTKPRpKbno+mIijQtXEF+3TOcMyPPrDU5/YALTC2pmP I33Pzjufb2Qq67JxauXLu2azaF+agfOM3dulg16ntwdTXqgq+RzvgimgtGv8c7ee9JCzNS YsGNmRAZRJ6Y80yKPC99xCcdx0ACq98= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jZsLMwcx; spf=pass (imf30.hostedemail.com: domain of leon@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=leon@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719911405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/Tk6HO/OFPztC8ZmvMZj3eONbvN9mSpoSMmb7Ix2/1w=; b=x2cMC4S6T6VZvpglBMM9xYbcH7FMenUZNi4ulU3N4IemQGu1mMBJVZ+A3QO0s87WaanNzH XhNkL+49vw7GsDxmNXcTdOuWpX256h2FjYLXO02Dpva5uNldrdF/QjpDmE7Ed5bm8QLuJa r43euWLnvgOskMb4ZK43TP75iJBIMe8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id F02CACE1D06; Tue, 2 Jul 2024 09:10:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CB762C116B1; Tue, 2 Jul 2024 09:10:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719911412; bh=GdqxFreVifTUH7yLunaT0Kp4VH1OcNBKz7jklxCA7IE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jZsLMwcxfIhbNdXAH7Erajavnd9zt8wZwqJVAhbFF8wuHrgj9ZFLnS0M824YeY1WI lsBBR7Huv75yN3CNdjq3wGOj6d6XRdgf43l4OFc9s6OcECQ4HjS13MEjvrsvp/qDtC V9bMyC4eCtQlzz7bAocuoWalQBZeKMuoUk0+t2JOvKUiwHjZ7NOHkhxRQ2bfGbsofJ 9EiPqR5DPCSASk+2Ji+dPRehAt5lU8Xk41chq5PYN3dkl0zeRpGP7rNQIh7zxcQ/Ux TOXwMCzKrQG/bRPwo4SyChwOnWE10AWLJZqnWPQAFqWJhu2cqfYHV+DULsrE24NbAI pEN9z2lU0w7oA== From: Leon Romanovsky To: Jens Axboe , Jason Gunthorpe , Robin Murphy , Joerg Roedel , Will Deacon , Keith Busch , Christoph Hellwig , "Zeng, Oak" , Chaitanya Kulkarni Cc: Leon Romanovsky , Sagi Grimberg , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , Marek Szyprowski , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 02/18] dma-mapping: provide an interface to allocate IOVA Date: Tue, 2 Jul 2024 12:09:32 +0300 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: rq4qbghus7fratnwjq7hym7q8uwe8eaj X-Rspamd-Queue-Id: 0FC608002B X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719911415-995556 X-HE-Meta: U2FsdGVkX185LZjQbaPXux0D83LDKFqmvIMxEQssD0inSjgJRGijaMOl4bmZL+ddPH30PfFxCakHl5HIWkh9iT42/rhacy+pi+LhDh6RFOiOTe5GkTRakLFBghwedWoUbBa9dJ4ZZNyU+OTin3VTyHcxUuaqmF4e1pxQQBBnC23Koeg0pVfGkrOTOy/9Nxy6B6hkVMxTaIu9PEtEMlX0pRF756G6rhQ0W9EGyA6+uZFJ2ceHlMt89Z4x5n1obJW2pk4lTxM37olXHHzOX0z1EA44qtLIpjMxkfmLcMiYio0PjqttQwQodMke3jMbLOVuwdsuzI4kORN380SnGc6un4XQjTYP0l7hgOTJ6S1XRLk6wpr18ygOz4AYPmQmtl5R8vx5F6TDsP0uqo1l8vBo0wqz1L9+3BfI/PRmpBKKKq7mlhksZ+fyhEZS63zN5hbcUS6neY/QTNQ6M2lnOp+FyMa1ov4O+wKKbmJMa1bJTRZ0GQ8vuYcjn/pjyCvzrZaqPt++1/VXwvHopNfl1o4k8ZhRD8SrILKOmDwO19hmStgIf7H7+kZkiimjq7mE+2TqRhGQY+7ebspDFZ2JcCDEzma+ZCqXFhJhEC3gm7/lq0Up8idIDUBe3CWK4l7/lUmO0Oa3zRaT5kuHpZUZzaLujPfO5DOk5s8wM6BrykWucRirUGS7UhRwhQNvsIiv3snyoSqZQtbi/9OhMV2SRC4aVccpItDIBQePJPj73pUI6KO30wwD/DvT/S0jQL7D3BIixat0QwX5BMUIGhzPHLj4+CuQp+xEz0AJjb4GYGHYATUa9xMJPV7JFjMkgd7o3bolKw+vW6EK+J1i0nAte8IresiOs5dZ5UUvlzMeS3KN8yARIX7dTJFlqD9ASTOj+5PJrVKGQ9gdUD1i14uiEm//aGu0vAbdHx35aZDOoePvYgMWF1s8z0pafsUYJakcz0rZ9JX9ACNwD/gx9BAg7rC gTh6n4+y CKvg7LYz8hksGOb5GMy2YVKKQxkKl4aLa66CVklnZ+Hq6MNcmd+rzU5kkVqUzB2FJExyblfIsXyi5ApS+W3ZJ9DM4coeuh+rnSGwguIHckuxTVA1gJ4ZwS12YOOqFVPsE/ooSwrU9iZD3sO7IS1dZ5aUYHWtuTE13MfLm3BqyjMsnr+EghoYw++cWQhiJ9KqUdOOwwQQ/flS98c8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Leon Romanovsky Existing .map_page() callback provides two things at the same time: allocates IOVA and links DMA pages. That combination works great for most of the callers who use it in control paths, but less effective in fast paths. These advanced callers already manage their data in some sort of database and can perform IOVA allocation in advance, leaving range linkage operation to be in fast path. Provide an interface to allocate/deallocate IOVA and next patch link/unlink DMA ranges to that specific IOVA. Signed-off-by: Leon Romanovsky --- include/linux/dma-map-ops.h | 3 +++ include/linux/dma-mapping.h | 20 +++++++++++++++++ kernel/dma/mapping.c | 44 +++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 02a1c825896b..23e5e2f63a1c 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -86,6 +86,9 @@ struct dma_map_ops { size_t (*max_mapping_size)(struct device *dev); size_t (*opt_mapping_size)(void); unsigned long (*get_merge_boundary)(struct device *dev); + + dma_addr_t (*alloc_iova)(struct device *dev, size_t size); + void (*free_iova)(struct device *dev, dma_addr_t dma_addr, size_t size); }; #ifdef CONFIG_DMA_OPS diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 49b99c6e7ec5..673ddcf140ff 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -90,6 +90,16 @@ struct dma_memory_type { struct dev_pagemap *p2p_pgmap; }; +struct dma_iova_attrs { + /* OUT field */ + dma_addr_t addr; + /* IN fields */ + struct device *dev; + size_t size; + enum dma_data_direction dir; + unsigned long attrs; +}; + #ifdef CONFIG_DMA_API_DEBUG void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr); void debug_dma_map_single(struct device *dev, const void *addr, @@ -115,6 +125,9 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr) return 0; } +int dma_alloc_iova(struct dma_iova_attrs *iova); +void dma_free_iova(struct dma_iova_attrs *iova); + dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs); @@ -166,6 +179,13 @@ int dma_mmap_noncontiguous(struct device *dev, struct vm_area_struct *vma, void dma_get_memory_type(struct page *page, struct dma_memory_type *type); #else /* CONFIG_HAS_DMA */ +static inline int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + return -EOPNOTSUPP; +} +static inline void dma_free_iova(struct dma_iova_attrs *iova) +{ +} static inline dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 877e43b39c06..0c8f51010d08 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -924,3 +924,47 @@ void dma_get_memory_type(struct page *page, struct dma_memory_type *type) type->type = DMA_MEMORY_TYPE_NORMAL; } EXPORT_SYMBOL_GPL(dma_get_memory_type); + +/** + * dma_alloc_iova - Allocate an IOVA space + * @iova: IOVA attributes + * + * Allocate an IOVA space for the given IOVA attributes. The IOVA space + * is allocated to the worst case when whole range is going to be used. + */ +int dma_alloc_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->alloc_iova) { + /* dma_map_direct(..) check is for HMM range fault callers */ + iova->addr = 0; + return 0; + } + + iova->addr = ops->alloc_iova(dev, iova->size); + if (dma_mapping_error(dev, iova->addr)) + return -ENOMEM; + + return 0; +} +EXPORT_SYMBOL_GPL(dma_alloc_iova); + +/** + * dma_free_iova - Free an IOVA space + * @iova: IOVA attributes + * + * Free an IOVA space for the given IOVA attributes. + */ +void dma_free_iova(struct dma_iova_attrs *iova) +{ + struct device *dev = iova->dev; + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (dma_map_direct(dev, ops) || !ops->free_iova || !iova->addr) + return; + + ops->free_iova(dev, iova->addr, iova->size); +} +EXPORT_SYMBOL_GPL(dma_free_iova);