From patchwork Wed Jun 24 08:02:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11622563 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CFB5912 for ; Wed, 24 Jun 2020 08:03:30 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1537320FC3 for ; Wed, 24 Jun 2020 08:03:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1537320FC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4D2096E17C; Wed, 24 Jun 2020 08:03:29 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id B526789F49 for ; Wed, 24 Jun 2020 08:03:25 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 21599198-1500050 for multiple; Wed, 24 Jun 2020 09:02:53 +0100 From: Chris Wilson To: linux-mm@kvack.org Date: Wed, 24 Jun 2020 09:02:47 +0100 Message-Id: <20200624080248.3701-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/2] mm/mmu_notifier: Mark up direct reclaim paths with MAYFAIL X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jason Gunthorpe , intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Andrew Morton , Chris Wilson Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" When direct reclaim enters the shrinker and tries to reclaim pages, it has to opportunitically unmap them [try_to_unmap_one]. For direct reclaim, the calling context is unknown and may include attempts to unmap one page of a dma object while attempting to allocate more pages for that object. Pass the information along that we are inside an opportunistic unmap that can allow that page to remain referenced and mapped, and let the callback opt in to avoiding a recursive wait. Signed-off-by: Chris Wilson Cc: Andrew Morton CC: Jason Gunthorpe --- include/linux/mmu_notifier.h | 15 ++++++++++++++- mm/mmu_notifier.c | 3 +++ mm/rmap.c | 5 +++-- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index fc68f3570e19..ee1ad008951c 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -48,7 +48,8 @@ enum mmu_notifier_event { MMU_NOTIFY_RELEASE, }; -#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) +#define MMU_NOTIFIER_RANGE_BLOCKABLE BIT(0) +#define MMU_NOTIFIER_RANGE_MAYFAIL BIT(1) struct mmu_notifier_ops { /* @@ -169,6 +170,12 @@ struct mmu_notifier_ops { * a non-blocking behavior then the same applies to * invalidate_range_end. * + * If mayfail is set then the callback may return -EAGAIN while still + * holding its page references. This flag is set inside direct + * reclaim paths that are opportunistically trying to unmap pages + * from unknown contexts. The callback must be prepared to handle + * the matching invalidate_range_end even after failing the + * invalidate_range_start. */ int (*invalidate_range_start)(struct mmu_notifier *subscription, const struct mmu_notifier_range *range); @@ -397,6 +404,12 @@ mmu_notifier_range_blockable(const struct mmu_notifier_range *range) return (range->flags & MMU_NOTIFIER_RANGE_BLOCKABLE); } +static inline bool +mmu_notifier_range_mayfail(const struct mmu_notifier_range *range) +{ + return (range->flags & MMU_NOTIFIER_RANGE_MAYFAIL); +} + static inline void mmu_notifier_release(struct mm_struct *mm) { if (mm_has_notifiers(mm)) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 352bb9f3ecc0..95b89cee7af4 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -493,6 +493,9 @@ static int mn_hlist_invalidate_range_start( _ret = ops->invalidate_range_start(subscription, range); if (!mmu_notifier_range_blockable(range)) non_block_end(); + if (_ret == -EAGAIN && + mmu_notifier_range_mayfail(range)) + _ret = 0; if (_ret) { pr_info("%pS callback failed with %d in %sblockable context.\n", ops->invalidate_range_start, _ret, diff --git a/mm/rmap.c b/mm/rmap.c index 5fe2dedce1fc..912b737a3353 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1406,8 +1406,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, - address, + mmu_notifier_range_init(&range, + MMU_NOTIFY_CLEAR, MMU_NOTIFIER_RANGE_MAYFAIL, + vma, vma->vm_mm, address, min(vma->vm_end, address + page_size(page))); if (PageHuge(page)) { /* From patchwork Wed Jun 24 08:02:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11622561 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADD8A912 for ; Wed, 24 Jun 2020 08:03:26 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95F0320FC3 for ; Wed, 24 Jun 2020 08:03:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95F0320FC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chris-wilson.co.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 68F2A89E69; Wed, 24 Jun 2020 08:03:25 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D1BDB89E69 for ; Wed, 24 Jun 2020 08:03:23 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 21599206-1500050 for multiple; Wed, 24 Jun 2020 09:03:20 +0100 From: Chris Wilson To: linux-mm@kvack.org Date: Wed, 24 Jun 2020 09:02:48 +0100 Message-Id: <20200624080248.3701-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624080248.3701-1-chris@chris-wilson.co.uk> References: <20200624080248.3701-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/2] drm/i915/gem: Use mmu_notifier_range_mayfail() to avoid waiting inside reclaim X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Chris Wilson Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" The direct reclaim path may trigger the mmu_notifier callback as part of try_to_unmap_one. As this is purely an opportunitistic attempt to reclaim pages, and will be called from any allocation context under unknown conditions (that include attempting to allocate pages for the userptr object itself and subsequently trying to reclaim parts of the partially acquired object) we have to be careful never to wait on anything being held by the calling context. Since that is unknown, we have to avoid waiting from inside direct reclaim. Signed-off-by: Chris Wilson --- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c index 9c53eb883400..72cfb91230ea 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_userptr.c @@ -103,6 +103,7 @@ userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, it = interval_tree_iter_first(&mn->objects, range->start, end); while (it) { struct drm_i915_gem_object *obj; + unsigned int flags; if (!mmu_notifier_range_blockable(range)) { ret = -EAGAIN; @@ -126,9 +127,12 @@ userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, } spin_unlock(&mn->lock); - ret = i915_gem_object_unbind(obj, - I915_GEM_OBJECT_UNBIND_ACTIVE | - I915_GEM_OBJECT_UNBIND_BARRIER); + flags = (I915_GEM_OBJECT_UNBIND_ACTIVE | + I915_GEM_OBJECT_UNBIND_BARRIER); + if (mmu_notifier_range_mayfail(range)) + flags = 0; + + ret = i915_gem_object_unbind(obj, flags); if (ret == 0) ret = __i915_gem_object_put_pages(obj); i915_gem_object_put(obj);