From patchwork Thu May 4 15:54:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 9712245 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1EC5B60362 for ; Thu, 4 May 2017 15:54:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1089728174 for ; Thu, 4 May 2017 15:54:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0563B2863C; Thu, 4 May 2017 15:54:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8A81D28174 for ; Thu, 4 May 2017 15:54:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7CF0A890BB; Thu, 4 May 2017 15:54:20 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id D267A6E09F for ; Thu, 4 May 2017 15:54:18 +0000 (UTC) Received: by mail-wm0-x241.google.com with SMTP id y10so4237735wmh.0 for ; Thu, 04 May 2017 08:54:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ursulin-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oq9LhwrsvTRXNuLpz8TRPzTr0ZRshGWBoGB6Que0cg4=; b=ucoxDGa2EbkEHNCyPV4pOEP4aOSYjFkx2M+zdEEnRPjgWyfULMvrm+FlGOU02J0mQw D0otoBaPkwG7FsFYT9MFsuWz6kF2u9tliEEKkyWAeXLOGuULQtkUqIDEJ3rK0gHHxBAu 3dKAPBsyOkJQNC7vMpjXpS0lJNfuBfoZhzD/S3EAQ84vIT/hAIo8PVbnAbBHA8no3byO 7P3c53lpGn8wNOOsr1bLb6SXWDq7UuXmjv1iRk2mnNaruhG/zXMpxPocQR/Jb7kN/WMl vPXt6UT2duIFv+pLH8UCPvc0VWGr01lEMwjH4QdQuih3eZb+GNOm1RPgNF2D0zp3SP2x SZ4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oq9LhwrsvTRXNuLpz8TRPzTr0ZRshGWBoGB6Que0cg4=; b=fqLxsEf99Xuh30cq+crWb34tRz2aUe4y/OqcRg9A323GJRV2RaaRTq4T1pla2kGc3/ rwFR/vRMxo3BPmGIiz3DMwmGWq1E3ZEFg/pCtzrbyEAesRHmQ/NvbJP/iEUHZjFuRtm3 csFouyhyk44sFncAKTc4A0cILc+lCnoJwCW6xcH0+XHoX5XnDAref0Xr84yNJVEf85MF 2U7sSdP0yXVGRWL9WKQWgCC/LhtwtXJtm8/H9z6AlwaiRqChUifF7enWq18S0SR0xvuf PdIAMAWE9j7+g+it0xMbZG4NCHst4Yq64mHDqPs3K+Nd7pyr1YVV5bUtqtN4pFW4NVYN Y1Iw== X-Gm-Message-State: AN3rC/5awCkLcJDI1bRF2xdQTsOubWjPBsvGuJ/S4G0Tiqj+SMTQ4HY/ tUsb4RyxWJkIjA== X-Received: by 10.28.174.195 with SMTP id x186mr2435414wme.95.1493913257418; Thu, 04 May 2017 08:54:17 -0700 (PDT) Received: from t460p.intel ([94.15.199.60]) by smtp.gmail.com with ESMTPSA id y3sm2064342wrd.23.2017.05.04.08.54.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 May 2017 08:54:16 -0700 (PDT) From: Tvrtko Ursulin X-Google-Original-From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Thu, 4 May 2017 16:54:05 +0100 Message-Id: <20170504155405.7425-4-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170504155405.7425-1-tvrtko.ursulin@linux.intel.com> References: <20170504155405.7425-1-tvrtko.ursulin@linux.intel.com> Cc: linux-kernel@vger.kernel.org Subject: [Intel-gfx] [PATCH 4/4] drm/i915: Use __sg_alloc_table_from_pages for userptr allocations X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Tvrtko Ursulin With the addition of __sg_alloc_table_from_pages we can control the maximum coallescing size and eliminate a separate path for allocating backing store here. Similar to 871dfbd67d4e ("drm/i915: Allow compaction upto SWIOTLB max segment size") this enables more compact sg lists to be created and so has a beneficial effect on workloads with many and/or large objects of this class. v2: * Rename helper to i915_sg_segment_size and fix swiotlb override. * Commit message update. v3: * Actually include the swiotlb override fix. v4: * Regroup parameters a bit. (Chris Wilson) v5: * Rebase for swiotlb_max_segment. * Add DMA map failure handling as in abb0deacb5a6 ("drm/i915: Fallback to single PAGE_SIZE segments for DMA remapping"). v6: Handle swiotlb_max_segment() returning 1. (Joonas Lahtinen) v7: Rebase. Signed-off-by: Tvrtko Ursulin Cc: Chris Wilson Cc: linux-kernel@vger.kernel.org Reviewed-by: Chris Wilson (v4) Cc: Joonas Lahtinen --- drivers/gpu/drm/i915/i915_drv.h | 15 +++++++ drivers/gpu/drm/i915/i915_gem.c | 6 +-- drivers/gpu/drm/i915/i915_gem_userptr.c | 79 ++++++++++++--------------------- 3 files changed, 45 insertions(+), 55 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index b20ed16da0ad..320c16df1c9c 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -2676,6 +2676,21 @@ static inline struct scatterlist *__sg_next(struct scatterlist *sg) (((__iter).curr += PAGE_SIZE) < (__iter).max) || \ ((__iter) = __sgt_iter(__sg_next((__iter).sgp), false), 0)) +static inline unsigned int i915_sg_segment_size(void) +{ + unsigned int size = swiotlb_max_segment(); + + if (size == 0) + return SCATTERLIST_MAX_SEGMENT; + + size = rounddown(size, PAGE_SIZE); + /* swiotlb_max_segment_size can return 1 byte when it means one page. */ + if (size < PAGE_SIZE) + size = PAGE_SIZE; + + return size; +} + static inline const struct intel_device_info * intel_info(const struct drm_i915_private *dev_priv) { diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index f9c6b9b5002c..b2727905ef2b 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2336,7 +2336,7 @@ i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) struct sgt_iter sgt_iter; struct page *page; unsigned long last_pfn = 0; /* suppress gcc warning */ - unsigned int max_segment; + unsigned int max_segment = i915_sg_segment_size(); int ret; gfp_t gfp; @@ -2347,10 +2347,6 @@ i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) GEM_BUG_ON(obj->base.read_domains & I915_GEM_GPU_DOMAINS); GEM_BUG_ON(obj->base.write_domain & I915_GEM_GPU_DOMAINS); - max_segment = swiotlb_max_segment(); - if (!max_segment) - max_segment = rounddown(UINT_MAX, PAGE_SIZE); - st = kmalloc(sizeof(*st), GFP_KERNEL); if (st == NULL) return ERR_PTR(-ENOMEM); diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 58ccf8b8ca1c..d003076702ad 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -399,64 +399,42 @@ struct get_pages_work { struct task_struct *task; }; -#if IS_ENABLED(CONFIG_SWIOTLB) -#define swiotlb_active() swiotlb_nr_tbl() -#else -#define swiotlb_active() 0 -#endif - -static int -st_set_pages(struct sg_table **st, struct page **pvec, int num_pages) -{ - struct scatterlist *sg; - int ret, n; - - *st = kmalloc(sizeof(**st), GFP_KERNEL); - if (*st == NULL) - return -ENOMEM; - - if (swiotlb_active()) { - ret = sg_alloc_table(*st, num_pages, GFP_KERNEL); - if (ret) - goto err; - - for_each_sg((*st)->sgl, sg, num_pages, n) - sg_set_page(sg, pvec[n], PAGE_SIZE, 0); - } else { - ret = sg_alloc_table_from_pages(*st, pvec, num_pages, - 0, num_pages << PAGE_SHIFT, - GFP_KERNEL); - if (ret) - goto err; - } - - return 0; - -err: - kfree(*st); - *st = NULL; - return ret; -} - static struct sg_table * -__i915_gem_userptr_set_pages(struct drm_i915_gem_object *obj, - struct page **pvec, int num_pages) +__i915_gem_userptr_alloc_pages(struct drm_i915_gem_object *obj, + struct page **pvec, int num_pages) { - struct sg_table *pages; + unsigned int max_segment = i915_sg_segment_size(); + struct sg_table *st; int ret; - ret = st_set_pages(&pages, pvec, num_pages); - if (ret) + st = kmalloc(sizeof(*st), GFP_KERNEL); + if (!st) + return ERR_PTR(-ENOMEM); + +alloc_table: + ret = __sg_alloc_table_from_pages(st, pvec, num_pages, + 0, num_pages << PAGE_SHIFT, + max_segment, + GFP_KERNEL); + if (ret) { + kfree(st); return ERR_PTR(ret); + } - ret = i915_gem_gtt_prepare_pages(obj, pages); + ret = i915_gem_gtt_prepare_pages(obj, st); if (ret) { - sg_free_table(pages); - kfree(pages); + sg_free_table(st); + + if (max_segment > PAGE_SIZE) { + max_segment = PAGE_SIZE; + goto alloc_table; + } + + kfree(st); return ERR_PTR(ret); } - return pages; + return st; } static int @@ -540,7 +518,8 @@ __i915_gem_userptr_get_pages_worker(struct work_struct *_work) struct sg_table *pages = ERR_PTR(ret); if (pinned == npages) { - pages = __i915_gem_userptr_set_pages(obj, pvec, npages); + pages = __i915_gem_userptr_alloc_pages(obj, pvec, + npages); if (!IS_ERR(pages)) { __i915_gem_object_set_pages(obj, pages); pinned = 0; @@ -661,7 +640,7 @@ i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj) pages = __i915_gem_userptr_get_pages_schedule(obj); active = pages == ERR_PTR(-EAGAIN); } else { - pages = __i915_gem_userptr_set_pages(obj, pvec, num_pages); + pages = __i915_gem_userptr_alloc_pages(obj, pvec, num_pages); active = !IS_ERR(pages); } if (active)