From patchwork Fri Mar 19 13:25:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79560C433E0 for ; Fri, 19 Mar 2021 13:31:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 532B764F3B for ; Fri, 19 Mar 2021 13:31:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230377AbhCSNbO (ORCPT ); Fri, 19 Mar 2021 09:31:14 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14016 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229806AbhCSNan (ORCPT ); Fri, 19 Mar 2021 09:30:43 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T269c5zPkbj; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:06 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 1/6] iommu: Move IOVA power-of-2 roundup into allocator Date: Fri, 19 Mar 2021 21:25:43 +0800 Message-ID: <1616160348-29451-2-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Move the IOVA size power-of-2 rcache roundup into the IOVA allocator. This is to eventually make it possible to be able to configure the upper limit of the IOVA rcache range. Signed-off-by: John Garry --- drivers/iommu/dma-iommu.c | 8 ------ drivers/iommu/iova.c | 51 ++++++++++++++++++++++++++------------- 2 files changed, 34 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index af765c813cc8..15b7270a5c2a 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -429,14 +429,6 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain, shift = iova_shift(iovad); iova_len = size >> shift; - /* - * Freeing non-power-of-two-sized allocations back into the IOVA caches - * will come back to bite us badly, so we have to waste a bit of space - * rounding up anything cacheable to make sure that can't happen. The - * order of the unadjusted size will still match upon freeing. - */ - if (iova_len < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) - iova_len = roundup_pow_of_two(iova_len); dma_limit = min_not_zero(dma_limit, dev->bus_dma_limit); diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index e6e2fa85271c..e62e9e30b30c 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -179,7 +179,7 @@ iova_insert_rbtree(struct rb_root *root, struct iova *iova, static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, - struct iova *new, bool size_aligned) + struct iova *new, bool size_aligned, bool fast) { struct rb_node *curr, *prev; struct iova *curr_iova; @@ -188,6 +188,15 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long align_mask = ~0UL; unsigned long high_pfn = limit_pfn, low_pfn = iovad->start_pfn; + /* + * Freeing non-power-of-two-sized allocations back into the IOVA caches + * will come back to bite us badly, so we have to waste a bit of space + * rounding up anything cacheable to make sure that can't happen. The + * order of the unadjusted size will still match upon freeing. + */ + if (fast && size < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) + size = roundup_pow_of_two(size); + if (size_aligned) align_mask <<= fls_long(size - 1); @@ -288,21 +297,10 @@ void iova_cache_put(void) } EXPORT_SYMBOL_GPL(iova_cache_put); -/** - * alloc_iova - allocates an iova - * @iovad: - iova domain in question - * @size: - size of page frames to allocate - * @limit_pfn: - max limit address - * @size_aligned: - set if size_aligned address range is required - * This function allocates an iova in the range iovad->start_pfn to limit_pfn, - * searching top-down from limit_pfn to iovad->start_pfn. If the size_aligned - * flag is set then the allocated address iova->pfn_lo will be naturally - * aligned on roundup_power_of_two(size). - */ -struct iova * -alloc_iova(struct iova_domain *iovad, unsigned long size, +static struct iova * +__alloc_iova(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, - bool size_aligned) + bool size_aligned, bool fast) { struct iova *new_iova; int ret; @@ -312,7 +310,7 @@ alloc_iova(struct iova_domain *iovad, unsigned long size, return NULL; ret = __alloc_and_insert_iova_range(iovad, size, limit_pfn + 1, - new_iova, size_aligned); + new_iova, size_aligned, fast); if (ret) { free_iova_mem(new_iova); @@ -321,6 +319,25 @@ alloc_iova(struct iova_domain *iovad, unsigned long size, return new_iova; } + +/** + * alloc_iova - allocates an iova + * @iovad: - iova domain in question + * @size: - size of page frames to allocate + * @limit_pfn: - max limit address + * @size_aligned: - set if size_aligned address range is required + * This function allocates an iova in the range iovad->start_pfn to limit_pfn, + * searching top-down from limit_pfn to iovad->start_pfn. If the size_aligned + * flag is set then the allocated address iova->pfn_lo will be naturally + * aligned on roundup_power_of_two(size). + */ +struct iova * +alloc_iova(struct iova_domain *iovad, unsigned long size, + unsigned long limit_pfn, + bool size_aligned) +{ + return __alloc_iova(iovad, size, limit_pfn, size_aligned, false); +} EXPORT_SYMBOL_GPL(alloc_iova); static struct iova * @@ -433,7 +450,7 @@ alloc_iova_fast(struct iova_domain *iovad, unsigned long size, return iova_pfn; retry: - new_iova = alloc_iova(iovad, size, limit_pfn, true); + new_iova = __alloc_iova(iovad, size, limit_pfn, true, true); if (!new_iova) { unsigned int cpu; From patchwork Fri Mar 19 13:25:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D2FEC433E0 for ; Fri, 19 Mar 2021 13:31:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E56AD64F10 for ; Fri, 19 Mar 2021 13:31:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230241AbhCSNap (ORCPT ); Fri, 19 Mar 2021 09:30:45 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14018 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230251AbhCSNaT (ORCPT ); Fri, 19 Mar 2021 09:30:19 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T25scqzPkWX; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:07 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 2/6] iova: Add a per-domain count of reserved nodes Date: Fri, 19 Mar 2021 21:25:44 +0800 Message-ID: <1616160348-29451-3-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org To help learn if the domain has regular IOVA nodes, add a count of reserved nodes, calculated at init time. Signed-off-by: John Garry --- drivers/iommu/iova.c | 2 ++ include/linux/iova.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index e62e9e30b30c..cecc74fb8663 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -717,6 +717,8 @@ reserve_iova(struct iova_domain *iovad, * or need to insert remaining non overlap addr range */ iova = __insert_new_range(iovad, pfn_lo, pfn_hi); + if (iova) + iovad->reserved_node_count++; finish: spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); diff --git a/include/linux/iova.h b/include/linux/iova.h index c834c01c0a5b..fd3217a605b2 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -95,6 +95,7 @@ struct iova_domain { flush-queues */ atomic_t fq_timer_on; /* 1 when timer is active, 0 when not */ + int reserved_node_count; }; static inline unsigned long iova_size(struct iova *iova) From patchwork Fri Mar 19 13:25:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E57D3C43332 for ; Fri, 19 Mar 2021 13:31:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C8B2A64F3B for ; Fri, 19 Mar 2021 13:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230290AbhCSNar (ORCPT ); Fri, 19 Mar 2021 09:30:47 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14021 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230281AbhCSNaW (ORCPT ); Fri, 19 Mar 2021 09:30:22 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T25CSXzPk0h; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:07 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 3/6] iova: Allow rcache range upper limit to be configurable Date: Fri, 19 Mar 2021 21:25:45 +0800 Message-ID: <1616160348-29451-4-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Some LLDs may request DMA mappings whose IOVA length exceeds that of the current rcache upper limit. This means that allocations for those IOVAs will never be cached, and always must be allocated and freed from the RB tree per DMA mapping cycle. This has a significant effect on performance, more so since commit 4e89dce72521 ("iommu/iova: Retry from last rb tree node if iova search fails"), as discussed at [0]. Allow the range of cached IOVAs to be increased, by providing an API to set the upper limit, which is capped at IOVA_RANGE_CACHE_MAX_SIZE. For simplicity, the full range of IOVA rcaches is allocated and initialized at IOVA domain init time. Setting the range upper limit will fail if the domain is already live (before the tree contains IOVA nodes). This must be done to ensure any IOVAs cached comply with rule of size being a power-of-2. [0] https://lore.kernel.org/linux-iommu/20210129092120.1482-1-thunder.leizhen@huawei.com/ Signed-off-by: John Garry --- drivers/iommu/iova.c | 37 +++++++++++++++++++++++++++++++++++-- include/linux/iova.h | 11 ++++++++++- 2 files changed, 45 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index cecc74fb8663..d4f2f7fbbd84 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -49,6 +49,7 @@ init_iova_domain(struct iova_domain *iovad, unsigned long granule, iovad->flush_cb = NULL; iovad->fq = NULL; iovad->anchor.pfn_lo = iovad->anchor.pfn_hi = IOVA_ANCHOR; + iovad->rcache_max_size = IOVA_RANGE_CACHE_DEFAULT_SIZE; rb_link_node(&iovad->anchor.node, NULL, &iovad->rbroot.rb_node); rb_insert_color(&iovad->anchor.node, &iovad->rbroot); init_iova_rcaches(iovad); @@ -194,7 +195,7 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, * rounding up anything cacheable to make sure that can't happen. The * order of the unadjusted size will still match upon freeing. */ - if (fast && size < (1 << (IOVA_RANGE_CACHE_MAX_SIZE - 1))) + if (fast && size < (1 << (iovad->rcache_max_size - 1))) size = roundup_pow_of_two(size); if (size_aligned) @@ -901,7 +902,7 @@ static bool iova_rcache_insert(struct iova_domain *iovad, unsigned long pfn, { unsigned int log_size = order_base_2(size); - if (log_size >= IOVA_RANGE_CACHE_MAX_SIZE) + if (log_size >= iovad->rcache_max_size) return false; return __iova_rcache_insert(iovad, &iovad->rcaches[log_size], pfn); @@ -946,6 +947,38 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache, return iova_pfn; } +void iova_rcache_set_upper_limit(struct iova_domain *iovad, + unsigned long iova_len) +{ + unsigned int rcache_index = order_base_2(iova_len) + 1; + struct rb_node *rb_node = &iovad->anchor.node; + unsigned long flags; + int count = 0; + + spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); + if (rcache_index <= iovad->rcache_max_size) + goto out; + + while (1) { + rb_node = rb_prev(rb_node); + if (!rb_node) + break; + count++; + } + + /* + * If there are already IOVA nodes present in the tree, then don't + * allow range upper limit to be set. + */ + if (count != iovad->reserved_node_count) + goto out; + + iovad->rcache_max_size = min_t(unsigned long, rcache_index, + IOVA_RANGE_CACHE_MAX_SIZE); +out: + spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); +} + /* * Try to satisfy IOVA allocation range from rcache. Fail if requested * size is too big or the DMA limit we are given isn't satisfied by the diff --git a/include/linux/iova.h b/include/linux/iova.h index fd3217a605b2..952b81b54ef7 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -25,7 +25,8 @@ struct iova { struct iova_magazine; struct iova_cpu_rcache; -#define IOVA_RANGE_CACHE_MAX_SIZE 6 /* log of max cached IOVA range size (in pages) */ +#define IOVA_RANGE_CACHE_DEFAULT_SIZE 6 +#define IOVA_RANGE_CACHE_MAX_SIZE 10 /* log of max cached IOVA range size (in pages) */ #define MAX_GLOBAL_MAGS 32 /* magazines per bin */ struct iova_rcache { @@ -74,6 +75,7 @@ struct iova_domain { unsigned long start_pfn; /* Lower limit for this domain */ unsigned long dma_32bit_pfn; unsigned long max32_alloc_size; /* Size of last failed allocation */ + unsigned long rcache_max_size; /* Upper limit of cached IOVA RANGE */ struct iova_fq __percpu *fq; /* Flush Queue */ atomic64_t fq_flush_start_cnt; /* Number of TLB flushes that @@ -158,6 +160,8 @@ int init_iova_flush_queue(struct iova_domain *iovad, struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn); void put_iova_domain(struct iova_domain *iovad); void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad); +void iova_rcache_set_upper_limit(struct iova_domain *iovad, + unsigned long iova_len); #else static inline int iova_cache_get(void) { @@ -238,6 +242,11 @@ static inline void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad) { } + +static inline void iova_rcache_set_upper_limit(struct iova_domain *iovad, + unsigned long iova_len) +{ +} #endif #endif From patchwork Fri Mar 19 13:25:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34700C433DB for ; Fri, 19 Mar 2021 13:31:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09B2564F3B for ; Fri, 19 Mar 2021 13:31:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230231AbhCSNal (ORCPT ); Fri, 19 Mar 2021 09:30:41 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14020 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230250AbhCSNaT (ORCPT ); Fri, 19 Mar 2021 09:30:19 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T26YgmzPkdk; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:07 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 4/6] iommu: Add iommu_dma_set_opt_size() Date: Fri, 19 Mar 2021 21:25:46 +0800 Message-ID: <1616160348-29451-5-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add a function which allows the max optimised IOMMU DMA size to be set. Signed-off-by: John Garry --- drivers/iommu/dma-iommu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 15b7270a5c2a..a5dfbd6c0496 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -447,6 +447,21 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain, return (dma_addr_t)iova << shift; } +__maybe_unused +static void iommu_dma_set_opt_size(struct device *dev, size_t size) +{ + struct iommu_domain *domain = iommu_get_dma_domain(dev); + struct iommu_dma_cookie *cookie = domain->iova_cookie; + struct iova_domain *iovad = &cookie->iovad; + unsigned long shift, iova_len; + + shift = iova_shift(iovad); + iova_len = size >> shift; + iova_len = roundup_pow_of_two(iova_len); + + iova_rcache_set_upper_limit(iovad, iova_len); +} + static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, dma_addr_t iova, size_t size, struct page *freelist) { From patchwork Fri Mar 19 13:25:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A64B9C4332E for ; Fri, 19 Mar 2021 13:31:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83FFD64F59 for ; Fri, 19 Mar 2021 13:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230274AbhCSNaq (ORCPT ); Fri, 19 Mar 2021 09:30:46 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14019 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230258AbhCSNaU (ORCPT ); Fri, 19 Mar 2021 09:30:20 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T24tC4zPkFh; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:08 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 5/6] dma-mapping/iommu: Add dma_set_max_opt_size() Date: Fri, 19 Mar 2021 21:25:47 +0800 Message-ID: <1616160348-29451-6-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add a function to allow the max size which we want to optimise DMA mappings for. Signed-off-by: John Garry --- drivers/iommu/dma-iommu.c | 2 +- include/linux/dma-map-ops.h | 1 + include/linux/dma-mapping.h | 5 +++++ kernel/dma/mapping.c | 11 +++++++++++ 4 files changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index a5dfbd6c0496..d35881fcfb9c 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -447,7 +447,6 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain, return (dma_addr_t)iova << shift; } -__maybe_unused static void iommu_dma_set_opt_size(struct device *dev, size_t size) { struct iommu_domain *domain = iommu_get_dma_domain(dev); @@ -1278,6 +1277,7 @@ static const struct dma_map_ops iommu_dma_ops = { .map_resource = iommu_dma_map_resource, .unmap_resource = iommu_dma_unmap_resource, .get_merge_boundary = iommu_dma_get_merge_boundary, + .set_max_opt_size = iommu_dma_set_opt_size, }; /* diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 51872e736e7b..fed7a183b3b9 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -64,6 +64,7 @@ struct dma_map_ops { u64 (*get_required_mask)(struct device *dev); size_t (*max_mapping_size)(struct device *dev); unsigned long (*get_merge_boundary)(struct device *dev); + void (*set_max_opt_size)(struct device *dev, size_t size); }; #ifdef CONFIG_DMA_OPS diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 2a984cb4d1e0..91fe770145d4 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -144,6 +144,7 @@ u64 dma_get_required_mask(struct device *dev); size_t dma_max_mapping_size(struct device *dev); bool dma_need_sync(struct device *dev, dma_addr_t dma_addr); unsigned long dma_get_merge_boundary(struct device *dev); +void dma_set_max_opt_size(struct device *dev, size_t size); #else /* CONFIG_HAS_DMA */ static inline dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, @@ -257,6 +258,10 @@ static inline unsigned long dma_get_merge_boundary(struct device *dev) { return 0; } +static inline void dma_set_max_opt_size(struct device *dev, size_t size) +{ +} + #endif /* CONFIG_HAS_DMA */ struct page *dma_alloc_pages(struct device *dev, size_t size, diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index b6a633679933..59e6acb1c471 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -608,3 +608,14 @@ unsigned long dma_get_merge_boundary(struct device *dev) return ops->get_merge_boundary(dev); } EXPORT_SYMBOL_GPL(dma_get_merge_boundary); + +void dma_set_max_opt_size(struct device *dev, size_t size) +{ + const struct dma_map_ops *ops = get_dma_ops(dev); + + if (!ops || !ops->set_max_opt_size) + return; + + ops->set_max_opt_size(dev, size); +} +EXPORT_SYMBOL_GPL(dma_set_max_opt_size); From patchwork Fri Mar 19 13:25:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12151089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48309C43603 for ; Fri, 19 Mar 2021 13:31:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1FD3264F65 for ; Fri, 19 Mar 2021 13:31:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbhCSNap (ORCPT ); Fri, 19 Mar 2021 09:30:45 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14017 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbhCSNaT (ORCPT ); Fri, 19 Mar 2021 09:30:19 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T24JxyzPk7t; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:08 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 6/6] scsi: hisi_sas: Set max optimal DMA size for v3 hw Date: Fri, 19 Mar 2021 21:25:48 +0800 Message-ID: <1616160348-29451-7-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1616160348-29451-1-git-send-email-john.garry@huawei.com> References: <1616160348-29451-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org For IOMMU strict mode, more than doubles throughput in some scenarios. Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index 4580e081e489..2f77b418bbeb 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -4684,6 +4684,8 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto err_out_regions; } + dma_set_max_opt_size(dev, PAGE_SIZE * HISI_SAS_SGE_PAGE_CNT); + shost = hisi_sas_shost_alloc_pci(pdev); if (!shost) { rc = -ENOMEM;