From patchwork Mon Oct 10 10:53:18 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 9369341 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8960E607FD for ; Mon, 10 Oct 2016 10:53:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7989C29521 for ; Mon, 10 Oct 2016 10:53:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E15A2957D; Mon, 10 Oct 2016 10:53:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 97E522957C for ; Mon, 10 Oct 2016 10:53:31 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2BC276E433; Mon, 10 Oct 2016 10:53:31 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mail-lf0-x243.google.com (mail-lf0-x243.google.com [IPv6:2a00:1450:4010:c07::243]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2D1126E433 for ; Mon, 10 Oct 2016 10:53:30 +0000 (UTC) Received: by mail-lf0-x243.google.com with SMTP id x79so5914557lff.2 for ; Mon, 10 Oct 2016 03:53:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ursulin-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id; bh=kWpjukJi2ON+rzI0xzMNCgtD0yj0VrN5dklrYO1EQ+M=; b=wH0fMdSRAw+Xd9CU1GrtxzFKfz+kKcLuhatz2GfaKHZ6V4QojgkF7X9x0OcWgQFx6F k/zRrsJ/SI7MH4ueqIeEpgE3k6BbhZMJ8d0/KCjyYYF/P28087gxJss932cwY1EAUz6R tE4xHQo50pvEOmv5n79eq4qsp0t0U+QZA84Vk601xH1/tmZtmQpVaU4UQxmddBIx/xHR ey/f9fIo2IBmLXWZ/BcLC4A9Fn+mzBOWnlX3zLx4wJPs5ZWUyTuR8sVwr2g7GlNlZVvi dpyWmoX+LoCGTgi0lIy6CmuSrhs1iscP2v6iklaunburwtnyraUaHq5wdrYcGTYeGGgr nfHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=kWpjukJi2ON+rzI0xzMNCgtD0yj0VrN5dklrYO1EQ+M=; b=K7+NMl96so09VNzEXjVa1izs6D0YRyKKWcE9RlDzw5PxaK7HigVJFXqQCMPhbaET/I qfTdYW/rZRTJfUE8mLn4u6Y0FlbFzhzXD2jbKiZVv5JfPHKM0J3nY65cvJg0QpycoqVn LL2vWWah0s+/iuYOod9oqLdMrCzrWjoHn2i4Xq046M7Y3PSyaSpMAALEuhdXjiqoLtC/ qV6hGNcXD0KJkZNuhODOwGaR9sCbmc0Xea1CH9hlFsRWuxk/J5ZaWpJQLf68jO6wmadt s4Jcuo4K4kd17D90/vha75j0U3zJXaM3PxQPVAZcSIJHKBf23LEsjABzrhZEQhSiJUQ/ iZ+Q== X-Gm-Message-State: AA6/9Rnyv5W3Jd5e5/OAo72y4xRt98x/UeA81y4tyB+DxWHfhyZQgH2IxA4vb9Iq/H82wg== X-Received: by 10.194.24.163 with SMTP id v3mr14144937wjf.180.1476096806626; Mon, 10 Oct 2016 03:53:26 -0700 (PDT) Received: from e31.Home ([2a02:c7d:9b6d:e300:916a:6cab:ac67:71c2]) by smtp.gmail.com with ESMTPSA id n5sm38989374wjv.35.2016.10.10.03.53.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2016 03:53:26 -0700 (PDT) From: Tvrtko Ursulin X-Google-Original-From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Date: Mon, 10 Oct 2016 11:53:18 +0100 Message-Id: <1476096798-26192-1-git-send-email-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.7.4 Subject: [Intel-gfx] [RFC] drm/i915: Introduce i915_alloc_sg_table X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Tvrtko Ursulin There are currently two places in the code which build the sg table for the object backing store and in the future there will be one more. Consolidate that into a single helper which takes a caller defined context and callbacks. !!! Compile tested only. !!! Signed-off-by: Tvrtko Ursulin Cc: Chris Wilson --- drivers/gpu/drm/i915/i915_gem.c | 162 ++++++++++++++++---------------- drivers/gpu/drm/i915/i915_gem_gtt.c | 84 +++++++++++++++++ drivers/gpu/drm/i915/i915_gem_gtt.h | 14 +++ drivers/gpu/drm/i915/i915_gem_userptr.c | 54 ++++------- 4 files changed, 198 insertions(+), 116 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index a89a88922448..79dcd912759f 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2208,19 +2208,69 @@ i915_gem_object_put_pages(struct drm_i915_gem_object *obj) return 0; } +struct i915_gem_get_pages_gtt_ctx { + struct drm_i915_private *dev_priv; + gfp_t gfp; + struct address_space *mapping; + unsigned int page_count; + bool shrunk_all; +}; + +static struct page * +i915_gem_gtt_get_page(void *context, unsigned int page_num) +{ + struct i915_gem_get_pages_gtt_ctx *ctx = context; + struct page *page; + + if (!ctx->shrunk_all) + page = shmem_read_mapping_page_gfp(ctx->mapping, page_num, + ctx->gfp); + else + page = shmem_read_mapping_page(ctx->mapping, page_num); + + /* Check that the i965g/gm workaround works. */ + WARN_ON(page && !IS_ERR(page) && (ctx->gfp & __GFP_DMA32) && + (page_to_pfn(page) >= 0x00100000UL)); + + return page; +} + +static bool +i915_gem_gtt_get_page_failed(void *context, unsigned int page_num, + unsigned int err_cnt, int err) +{ + struct i915_gem_get_pages_gtt_ctx *ctx = context; + + if (err_cnt > 1) + return true; + + if (err_cnt == 0) { + i915_gem_shrink(ctx->dev_priv, ctx->page_count, + I915_SHRINK_BOUND | I915_SHRINK_UNBOUND | + I915_SHRINK_PURGEABLE); + } else { + i915_gem_shrink_all(ctx->dev_priv); + ctx->shrunk_all = true; + } + + return false; +} + +static void +i915_gem_gtt_put_page(void *context, struct page *page) +{ + put_page(page); +} + static int i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) { struct drm_i915_private *dev_priv = to_i915(obj->base.dev); - int page_count, i; - struct address_space *mapping; + struct i915_gem_get_pages_gtt_ctx ctx; struct sg_table *st; - struct scatterlist *sg; struct sgt_iter sgt_iter; struct page *page; - unsigned long last_pfn = 0; /* suppress gcc warning */ int ret; - gfp_t gfp; /* Assert that the object is not currently in any GPU domain. As it * wasn't in the GTT, there shouldn't be any way it could have been in @@ -2229,78 +2279,44 @@ i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) BUG_ON(obj->base.read_domains & I915_GEM_GPU_DOMAINS); BUG_ON(obj->base.write_domain & I915_GEM_GPU_DOMAINS); - st = kmalloc(sizeof(*st), GFP_KERNEL); - if (st == NULL) - return -ENOMEM; - - page_count = obj->base.size / PAGE_SIZE; - if (sg_alloc_table(st, page_count, GFP_KERNEL)) { - kfree(st); - return -ENOMEM; - } + ctx.dev_priv = dev_priv; + ctx.page_count = obj->base.size / PAGE_SIZE; + ctx.shrunk_all = false; /* Get the list of pages out of our struct file. They'll be pinned * at this point until we release them. * * Fail silently without starting the shrinker */ - mapping = obj->base.filp->f_mapping; - gfp = mapping_gfp_constraint(mapping, ~(__GFP_IO | __GFP_RECLAIM)); - gfp |= __GFP_NORETRY | __GFP_NOWARN; - sg = st->sgl; - st->nents = 0; - for (i = 0; i < page_count; i++) { - page = shmem_read_mapping_page_gfp(mapping, i, gfp); - if (IS_ERR(page)) { - i915_gem_shrink(dev_priv, - page_count, - I915_SHRINK_BOUND | - I915_SHRINK_UNBOUND | - I915_SHRINK_PURGEABLE); - page = shmem_read_mapping_page_gfp(mapping, i, gfp); - } - if (IS_ERR(page)) { - /* We've tried hard to allocate the memory by reaping - * our own buffer, now let the real VM do its job and - * go down in flames if truly OOM. - */ - i915_gem_shrink_all(dev_priv); - page = shmem_read_mapping_page(mapping, i); - if (IS_ERR(page)) { - ret = PTR_ERR(page); - goto err_pages; - } - } -#ifdef CONFIG_SWIOTLB - if (swiotlb_nr_tbl()) { - st->nents++; - sg_set_page(sg, page, PAGE_SIZE, 0); - sg = sg_next(sg); - continue; - } -#endif - if (!i || page_to_pfn(page) != last_pfn + 1) { - if (i) - sg = sg_next(sg); - st->nents++; - sg_set_page(sg, page, PAGE_SIZE, 0); - } else { - sg->length += PAGE_SIZE; - } - last_pfn = page_to_pfn(page); - - /* Check that the i965g/gm workaround works. */ - WARN_ON((gfp & __GFP_DMA32) && (last_pfn >= 0x00100000UL)); + ctx.mapping = obj->base.filp->f_mapping; + ctx.gfp = mapping_gfp_constraint(ctx.mapping, + ~(__GFP_IO | __GFP_RECLAIM)); + ctx.gfp |= __GFP_NORETRY | __GFP_NOWARN; + + st = i915_alloc_sg_table(ctx.page_count, &ctx, + i915_gem_gtt_get_page, + i915_gem_gtt_get_page_failed, + i915_gem_gtt_put_page); + if (IS_ERR(st)) { + ret = PTR_ERR(st); + /* shmemfs first checks if there is enough memory to allocate + * the page and reports ENOSPC should there be insufficient, + * along with the usual ENOMEM for a genuine allocation failure. + * + * We use ENOSPC in our driver to mean that we have run out of + * aperture space and so want to translate the error from + * shmemfs back to our usual understanding of ENOMEM. + */ + if (ret == -ENOSPC) + ret = -ENOMEM; + return ret; } -#ifdef CONFIG_SWIOTLB - if (!swiotlb_nr_tbl()) -#endif - sg_mark_end(sg); + obj->pages = st; ret = i915_gem_gtt_prepare_object(obj); if (ret) - goto err_pages; + goto err; if (i915_gem_object_needs_bit17_swizzle(obj)) i915_gem_object_do_bit_17_swizzle(obj); @@ -2311,24 +2327,12 @@ i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) return 0; -err_pages: - sg_mark_end(sg); +err: for_each_sgt_page(page, sgt_iter, st) put_page(page); sg_free_table(st); kfree(st); - /* shmemfs first checks if there is enough memory to allocate the page - * and reports ENOSPC should there be insufficient, along with the usual - * ENOMEM for a genuine allocation failure. - * - * We use ENOSPC in our driver to mean that we have run out of aperture - * space and so want to translate the error from shmemfs back to our - * usual understanding of ENOMEM. - */ - if (ret == -ENOSPC) - ret = -ENOMEM; - return ret; } diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 0bb4232f66bc..441910eafdea 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -3703,3 +3703,87 @@ void i915_vma_unpin_and_release(struct i915_vma **p_vma) i915_vma_unpin(vma); i915_vma_put(vma); } + +struct sg_table * +i915_alloc_sg_table(unsigned int page_count, void *context, + i915_page_alloc_f page_alloc, + i915_page_alloc_err_f page_alloc_error, + i915_page_put_f page_put) +{ + struct sg_table *st; + struct scatterlist *sg; + struct page *page; + struct sgt_iter sgt_iter; + unsigned long pfn; + unsigned long last_pfn = 0; /* suppress gcc warning */ + unsigned int err_cnt = 0; + unsigned int i; + int err; + + st = kmalloc(sizeof(*st), GFP_KERNEL); + if (st == NULL) + return ERR_PTR(-ENOMEM); + + if (sg_alloc_table(st, page_count, GFP_KERNEL)) { + kfree(st); + return ERR_PTR(-ENOMEM); + } + + sg = st->sgl; + st->nents = 0; + + for (i = 0; i < page_count; i++) { +retry: + page = page_alloc(context, i); + if (!page || IS_ERR(page)) { + err = page ? PTR_ERR(page) : -ENOMEM; + + if (!page_alloc_error || + page_alloc_error(context, i, err_cnt, err)) + goto err_page_alloc; + else + goto retry; + + err_cnt++; + } + +#ifdef CONFIG_SWIOTLB + if (swiotlb_nr_tbl()) { + st->nents++; + sg_set_page(sg, page, PAGE_SIZE, 0); + sg = sg_next(sg); + continue; + } +#endif + pfn = page_to_pfn(page); + + if (!i || pfn != last_pfn + 1) { + if (i) + sg = sg_next(sg); + st->nents++; + sg_set_page(sg, page, PAGE_SIZE, 0); + } else { + sg->length += PAGE_SIZE; + } + + last_pfn = pfn; + } + +#ifdef CONFIG_SWIOTLB + if (!swiotlb_nr_tbl()) +#endif + sg_mark_end(sg); + + return st; + +err_page_alloc: + sg_mark_end(sg); + if (page_put) { + for_each_sgt_page(page, sgt_iter, st) + page_put(context, page); + } + sg_free_table(st); + kfree(st); + + return ERR_PTR(err); +} diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h index ec78be2f8c77..17768c99605e 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.h +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h @@ -730,4 +730,18 @@ static inline struct page *i915_vma_first_page(struct i915_vma *vma) return sg_page(vma->pages->sgl); } +typedef struct page * +(*i915_page_alloc_f)(void *context, unsigned int page_num); +typedef bool +(*i915_page_alloc_err_f)(void *context, unsigned int page_num, + unsigned int err_cnt, int err); +typedef void +(*i915_page_put_f)(void *context, struct page *page); + +struct sg_table * +i915_alloc_sg_table(unsigned int page_count, void *context, + i915_page_alloc_f page_alloc, + i915_page_alloc_err_f page_alloc_error, + i915_page_put_f page_put); + #endif diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index e537930c64b5..cddebe50916a 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -397,54 +397,34 @@ struct get_pages_work { struct task_struct *task; }; -#if IS_ENABLED(CONFIG_SWIOTLB) -#define swiotlb_active() swiotlb_nr_tbl() -#else -#define swiotlb_active() 0 -#endif +struct i915_gem_get_pages_userptr_ctx { + struct page **pvec; +}; -static int -st_set_pages(struct sg_table **st, struct page **pvec, int num_pages) +static struct page * +i915_gem_userptr_get_page(void *context, unsigned int page_num) { - struct scatterlist *sg; - int ret, n; - - *st = kmalloc(sizeof(**st), GFP_KERNEL); - if (*st == NULL) - return -ENOMEM; + struct i915_gem_get_pages_userptr_ctx *ctx = context; - if (swiotlb_active()) { - ret = sg_alloc_table(*st, num_pages, GFP_KERNEL); - if (ret) - goto err; - - for_each_sg((*st)->sgl, sg, num_pages, n) - sg_set_page(sg, pvec[n], PAGE_SIZE, 0); - } else { - ret = sg_alloc_table_from_pages(*st, pvec, num_pages, - 0, num_pages << PAGE_SHIFT, - GFP_KERNEL); - if (ret) - goto err; - } - - return 0; - -err: - kfree(*st); - *st = NULL; - return ret; + return ctx->pvec[page_num]; } static int __i915_gem_userptr_set_pages(struct drm_i915_gem_object *obj, struct page **pvec, int num_pages) { + struct i915_gem_get_pages_userptr_ctx ctx; + struct sg_table *st; int ret; - ret = st_set_pages(&obj->pages, pvec, num_pages); - if (ret) - return ret; + ctx.pvec = pvec; + st = i915_alloc_sg_table(num_pages, &ctx, + i915_gem_userptr_get_page, + NULL, NULL); + if (IS_ERR(st)) + return PTR_ERR(st); + + obj->pages = st; ret = i915_gem_gtt_prepare_object(obj); if (ret) {