From patchwork Tue Aug 20 08:18:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11103267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CE65112C for ; Tue, 20 Aug 2019 08:19:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C4BAB23A85 for ; Tue, 20 Aug 2019 08:19:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="YWxeLKHf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4BAB23A85 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2FCB86B000D; Tue, 20 Aug 2019 04:19:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 281ED6B000E; Tue, 20 Aug 2019 04:19:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1749A6B0010; Tue, 20 Aug 2019 04:19:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id E5D3C6B000D for ; Tue, 20 Aug 2019 04:19:11 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 94C308248AC0 for ; Tue, 20 Aug 2019 08:19:11 +0000 (UTC) X-FDA: 75842106102.14.verse71_15f7320553661 X-Spam-Summary: 2,0,0,8870e7b88d981156,d41d8cd98f00b204,daniel.vetter@ffwll.ch,:linux-kernel@vger.kernel.org::dri-devel@lists.freedesktop.org:intel-gfx@lists.freedesktop.org:daniel.vetter@ffwll.ch:jgg@ziepe.ca:chris@chris-wilson.co.uk:akpm@linux-foundation.org:rientjes@google.com:jglisse@redhat.com:mhocko@suse.com:christian.koenig@amd.com:gregkh@linuxfoundation.org:rppt@linux.vnet.ibm.com:daniel.vetter@intel.com,RULES_HIT:41:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1500:1515:1516:1518:1535:1544:1593:1594:1605:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3873:3874:4119:4250:4321:4385:4605:5007:6119:6261:6653:6742:8660:9010:10004:11026:11473:11658:11914:12043:12114:12296:12297:12438:12517:12519:12555:12679:12895:12986:13148:13161:13229:13230:13846:13894:14096:14097:14181:14394:14659:14721:21063:21080:21222:21433:21444:21451:21627:21740:30029:30054:30055:30056:3 0064:300 X-HE-Tag: verse71_15f7320553661 X-Filterd-Recvd-Size: 8242 Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Tue, 20 Aug 2019 08:19:10 +0000 (UTC) Received: by mail-ed1-f67.google.com with SMTP id p28so5331807edi.3 for ; Tue, 20 Aug 2019 01:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wgia7qPSXicnYInuL6+gfr2cdFlvKPNwuVbgz5GenZE=; b=YWxeLKHfVQ6n9wGxXuRJAn+g9qtmHMwaPzmYfWe8BYeYvBhhpUOFVNbgUl+9qLpP7T PHmTFDTMVPwsO/1+PqoKETqfMC0QoEs4IGQgsATXmRfGgRNSHR4/V73LAd/yZFEVWK1y s1kPTUoWam517FLtcOhvQSucIJxk+akolMA3M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wgia7qPSXicnYInuL6+gfr2cdFlvKPNwuVbgz5GenZE=; b=QtSX/ZnsEelvMVNw+0DZw6nVFjPerPujdjPZovOne6belRB270V3v5uT0jFS6p6uLk uG1dS1aAbyYzXBCc8mPcj+xhEMkKa1B5ztAkB5sKGFBlxM/IkZJ2qUOLgb/bsl6DIRex PsNS9sBXyB9IW0pBOMtmcf/Iyr9uKBRyq3kCJMDFp9vNdDpTfujfvacyBRXykxwEqqEe lGPiYL9cI6OggcHfCwypDIZNSCy8OEd1yWYW072PlA/wvpNf0extmLe854jP/WHU129S 5nvHIVfWvEYPcoZBCsXGzuDZQRZ0EaLfC9hVVQNUnyVEqb0utV2mSDlwEut3XjRPNvY4 EYVQ== X-Gm-Message-State: APjAAAXBS42NNWCOSXL0qVRYitLHhAuS52nWUwBGwqCfVfRQjb57qDZ5 rc3s3MGFTbbFJ5Qf4gXpmNmlYA== X-Google-Smtp-Source: APXvYqyu/f8bVTg3bBh5ZePWB9S7yhhKk9kHLLvI/rN0zBooM1KCXKcqbAcEp76HXAnJ5MeGWWnWww== X-Received: by 2002:aa7:d813:: with SMTP id v19mr29744820edq.45.1566289149643; Tue, 20 Aug 2019 01:19:09 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:569e:0:3106:d637:d723:e855]) by smtp.gmail.com with ESMTPSA id fj15sm2469623ejb.78.2019.08.20.01.19.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2019 01:19:08 -0700 (PDT) From: Daniel Vetter To: LKML Cc: Linux MM , DRI Development , Intel Graphics Development , Daniel Vetter , Jason Gunthorpe , Chris Wilson , Andrew Morton , David Rientjes , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Michal Hocko , =?utf-8?q?Christian_K=C3=B6nig?= , Greg Kroah-Hartman , Mike Rapoport , Daniel Vetter Subject: [PATCH 1/4] mm, notifier: Add a lockdep map for invalidate_range_start/end Date: Tue, 20 Aug 2019 10:18:59 +0200 Message-Id: <20190820081902.24815-2-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190820081902.24815-1-daniel.vetter@ffwll.ch> References: <20190820081902.24815-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a similar idea to the fs_reclaim fake lockdep lock. It's fairly easy to provoke a specific notifier to be run on a specific range: Just prep it, and then munmap() it. A bit harder, but still doable, is to provoke the mmu notifiers for all the various callchains that might lead to them. But both at the same time is really hard to reliable hit, especially when you want to exercise paths like direct reclaim or compaction, where it's not easy to control what exactly will be unmapped. By introducing a lockdep map to tie them all together we allow lockdep to see a lot more dependencies, without having to actually hit them in a single challchain while testing. On Jason's suggestion this is is rolled out for both invalidate_range_start and invalidate_range_end. They both have the same calling context, hence we can share the same lockdep map. Note that the annotation for invalidate_ranage_start is outside of the mm_has_notifiers(), to make sure lockdep is informed about all paths leading to this context irrespective of whether mmu notifiers are present for a given context. We don't do that on the invalidate_range_end side to avoid paying the overhead twice, there the lockdep annotation is pushed down behind the mm_has_notifiers() check. v2: Use lock_map_acquire/release() like fs_reclaim, to avoid confusion with this being a real mutex (Chris Wilson). v3: Rebase on top of Glisse's arg rework. v4: Also annotate invalidate_range_end (Jason Gunthorpe) Also annotate invalidate_range_start_nonblock, I somehow missed that one in the first version. Cc: Jason Gunthorpe Cc: Chris Wilson Cc: Andrew Morton Cc: David Rientjes Cc: "Jérôme Glisse" Cc: Michal Hocko Cc: "Christian König" Cc: Greg Kroah-Hartman Cc: Daniel Vetter Cc: Mike Rapoport Cc: linux-mm@kvack.org Signed-off-by: Daniel Vetter Reviewed-by: Jason Gunthorpe --- include/linux/mmu_notifier.h | 8 ++++++++ mm/mmu_notifier.c | 9 +++++++++ 2 files changed, 17 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index b6c004bd9f6a..39a86b77a939 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -42,6 +42,10 @@ enum mmu_notifier_event { #ifdef CONFIG_MMU_NOTIFIER +#ifdef CONFIG_LOCKDEP +extern struct lockdep_map __mmu_notifier_invalidate_range_start_map; +#endif + /* * The mmu notifier_mm structure is allocated and installed in * mm->mmu_notifier_mm inside the mm_take_all_locks() protected @@ -310,19 +314,23 @@ static inline void mmu_notifier_change_pte(struct mm_struct *mm, static inline void mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); if (mm_has_notifiers(range->mm)) { range->flags |= MMU_NOTIFIER_RANGE_BLOCKABLE; __mmu_notifier_invalidate_range_start(range); } + lock_map_release(&__mmu_notifier_invalidate_range_start_map); } static inline int mmu_notifier_invalidate_range_start_nonblock(struct mmu_notifier_range *range) { + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); if (mm_has_notifiers(range->mm)) { range->flags &= ~MMU_NOTIFIER_RANGE_BLOCKABLE; return __mmu_notifier_invalidate_range_start(range); } + lock_map_release(&__mmu_notifier_invalidate_range_start_map); return 0; } diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 16f1cbc775d0..d12e3079e7a4 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -21,6 +21,13 @@ /* global SRCU for all MMs */ DEFINE_STATIC_SRCU(srcu); +#ifdef CONFIG_LOCKDEP +struct lockdep_map __mmu_notifier_invalidate_range_start_map = { + .name = "mmu_notifier_invalidate_range_start" +}; +EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_start_map); +#endif + /* * This function allows mmu_notifier::release callback to delay a call to * a function that will free appropriate resources. The function must be @@ -197,6 +204,7 @@ void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range, struct mmu_notifier *mn; int id; + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist) { /* @@ -220,6 +228,7 @@ void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range, mn->ops->invalidate_range_end(mn, range); } srcu_read_unlock(&srcu, id); + lock_map_release(&__mmu_notifier_invalidate_range_start_map); } EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_end); From patchwork Tue Aug 20 08:19:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11103271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6399112C for ; Tue, 20 Aug 2019 08:19:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 71BDF23AC8 for ; Tue, 20 Aug 2019 08:19:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="H3IaTIIK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71BDF23AC8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 718F46B000E; Tue, 20 Aug 2019 04:19:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 67E246B0010; Tue, 20 Aug 2019 04:19:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5719C6B0269; Tue, 20 Aug 2019 04:19:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 2EB0E6B000E for ; Tue, 20 Aug 2019 04:19:13 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id D91996C2D for ; Tue, 20 Aug 2019 08:19:12 +0000 (UTC) X-FDA: 75842106144.18.leg41_1626500992f0d X-Spam-Summary: 2,0,0,978cbe459e45f00e,d41d8cd98f00b204,daniel.vetter@ffwll.ch,:linux-kernel@vger.kernel.org::dri-devel@lists.freedesktop.org:intel-gfx@lists.freedesktop.org:daniel.vetter@ffwll.ch:jgg@ziepe.ca:chris@chris-wilson.co.uk:akpm@linux-foundation.org:rientjes@google.com:jglisse@redhat.com:mhocko@suse.com:christian.koenig@amd.com:gregkh@linuxfoundation.org:rppt@linux.vnet.ibm.com:daniel.vetter@intel.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3874:4250:5007:6261:6653:6742:7903:10004:10400:10450:10455:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13069:13311:13357:13846:13894:13972:14096:14097:14181:14394:14659:14721:19904:19999:21080:21444:21451:21627:30054:30055:30064,0,RBL:209.85.208.65:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201,CacheIP:none,Bayes ian:0.5, X-HE-Tag: leg41_1626500992f0d X-Filterd-Recvd-Size: 5053 Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 20 Aug 2019 08:19:12 +0000 (UTC) Received: by mail-ed1-f65.google.com with SMTP id x19so5328226eda.12 for ; Tue, 20 Aug 2019 01:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HRs5HpBA3qSqFe5qnt1LL6h0B7I2hzSM+r7A2DpfdeU=; b=H3IaTIIKcnZxd8XaCn8QCH9mcrNGjOx/ak2ybd66H0ki331clYHFNQzkO0bKWcuXsr Mq6+jU/jrdI7HQWeHyhaTDSfFkUDTPsmPa++H2SDWnl1ZAvHNh4YpUIn825O4i0/plh8 z09eHCYqJKERfRBDZg/Pr+cpNjod1qv2vMJ8w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HRs5HpBA3qSqFe5qnt1LL6h0B7I2hzSM+r7A2DpfdeU=; b=MOly1xQAuQ9lDXdmc9hCk6E0Z9YmQE0sxTdzCIRwcnIoBEIcyzBKRNuNRfh70TDBn7 JukuaxudTwHoDjTBDn7C/mG2IFnmT6DtmO1teyuAK7wHLM47PyJa5TWO2O5VOSzOzkG+ /Oxof98NBvVvjfXXLcf4e2BFtyoCIeAzEjrGa+qirMS82FDEX71csfERYpOAVR7Qc/16 PU/uX+o9azSLHeBj2FQfTEJUtfzP91IPUZl7sbXD69h12hcFXb9d3CVWG0PfGcgc4ElK WwnjzOkjF1nTuRa3cvCHL9UIgqqud5Xol1202xY35H2viXxL1Ic5K906t9T7+SWy3d4N GnaA== X-Gm-Message-State: APjAAAXdWYrIFa65+5DnHWWBvMDoRBlokeZ+4XC9Eb7GApIOLH9+2OCY c++gCkeTqRNKFBZ50FK/0skOPQ== X-Google-Smtp-Source: APXvYqzwk/VVcEGb/uEJb+MjHFqWnLqThVvSK3PFeDiC0L0cxhC/+XJhzn783w4kD3r5dKVWby8hcQ== X-Received: by 2002:aa7:c498:: with SMTP id m24mr12784881edq.277.1566289150887; Tue, 20 Aug 2019 01:19:10 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:569e:0:3106:d637:d723:e855]) by smtp.gmail.com with ESMTPSA id fj15sm2469623ejb.78.2019.08.20.01.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2019 01:19:10 -0700 (PDT) From: Daniel Vetter To: LKML Cc: Linux MM , DRI Development , Intel Graphics Development , Daniel Vetter , Jason Gunthorpe , Chris Wilson , Andrew Morton , David Rientjes , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Michal Hocko , =?utf-8?q?Christian_K=C3=B6nig?= , Greg Kroah-Hartman , Mike Rapoport , Daniel Vetter Subject: [PATCH 2/4] mm, notifier: Prime lockdep Date: Tue, 20 Aug 2019 10:19:00 +0200 Message-Id: <20190820081902.24815-3-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190820081902.24815-1-daniel.vetter@ffwll.ch> References: <20190820081902.24815-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We want to teach lockdep that mmu notifiers can be called from direct reclaim paths, since on many CI systems load might never reach that level (e.g. when just running fuzzer or small functional tests). Motivated by a discussion with Jason. I've put the annotation into mmu_notifier_register since only when we have mmu notifiers registered is there any point in teaching lockdep about them. Also, we already have a kmalloc(, GFP_KERNEL), so this is safe. Cc: Jason Gunthorpe Cc: Chris Wilson Cc: Andrew Morton Cc: David Rientjes Cc: "Jérôme Glisse" Cc: Michal Hocko Cc: "Christian König" Cc: Greg Kroah-Hartman Cc: Daniel Vetter Cc: Mike Rapoport Cc: linux-mm@kvack.org Signed-off-by: Daniel Vetter Reviewed-by: Jason Gunthorpe --- mm/mmu_notifier.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index d12e3079e7a4..538d3bb87f9b 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -256,6 +256,13 @@ static int do_mmu_notifier_register(struct mmu_notifier *mn, BUG_ON(atomic_read(&mm->mm_users) <= 0); + if (IS_ENABLED(CONFIG_LOCKDEP)) { + fs_reclaim_acquire(GFP_KERNEL); + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); + lock_map_release(&__mmu_notifier_invalidate_range_start_map); + fs_reclaim_release(GFP_KERNEL); + } + ret = -ENOMEM; mmu_notifier_mm = kmalloc(sizeof(struct mmu_notifier_mm), GFP_KERNEL); if (unlikely(!mmu_notifier_mm)) From patchwork Tue Aug 20 08:19:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11103277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78D10112C for ; Tue, 20 Aug 2019 08:19:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2C32123AFB for ; Tue, 20 Aug 2019 08:19:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="JaM5/xgb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C32123AFB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E23AD6B0010; Tue, 20 Aug 2019 04:19:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D833C6B0269; Tue, 20 Aug 2019 04:19:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BFB866B026A; Tue, 20 Aug 2019 04:19:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id A0F1D6B0010 for ; Tue, 20 Aug 2019 04:19:14 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 52CAE180AD7C1 for ; Tue, 20 Aug 2019 08:19:14 +0000 (UTC) X-FDA: 75842106228.13.pigs06_165c6a81d211f X-Spam-Summary: 2,0,0,c3976733c7ea77f5,d41d8cd98f00b204,daniel.vetter@ffwll.ch,:linux-kernel@vger.kernel.org::dri-devel@lists.freedesktop.org:intel-gfx@lists.freedesktop.org:daniel.vetter@ffwll.ch:jgg@ziepe.ca:peterz@infradead.org:mingo@redhat.com:akpm@linux-foundation.org:mhocko@suse.com:rientjes@google.com:christian.koenig@amd.com:jglisse@redhat.com:yamada.masahiro@socionext.com:wvw@google.com:andriy.shevchenko@linux.intel.com:tglx@linutronix.de:jannh@google.com:feng.tang@intel.com:keescook@chromium.org:rdunlap@infradead.org:daniel.vetter@intel.com,RULES_HIT:1:2:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4050:4321:5007:6261:6653:6742:6743:7875:7903:7974:9108:9163:10004:11026:11232:11473:11658:11914:12043:12114:12291:12296:12297:12438:12517:12519:12555:12663:12895:12986:13846:13894:14394:1465 9:14877: X-HE-Tag: pigs06_165c6a81d211f X-Filterd-Recvd-Size: 11044 Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Tue, 20 Aug 2019 08:19:13 +0000 (UTC) Received: by mail-ed1-f65.google.com with SMTP id w5so5343695edl.8 for ; Tue, 20 Aug 2019 01:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kABFBkDTbJpVEXzQpdNq3PP8kqHWnvhar8F2JgU77ew=; b=JaM5/xgbKK+AhR1C9ILOXmNQGXVZ8JqAzbqOYr/Sa1NOIXaPoTdYMIex1wNRTaNR1H 0lgnk4VUA78HLzrtZV5dKh3c6EzlClYvBxOr7DMSJuv+ORoS+Pjd0tQRpx09SD06vEsJ LDkFvzrnWxx7gq7wfm+5tQPki349I1dF4VUSc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kABFBkDTbJpVEXzQpdNq3PP8kqHWnvhar8F2JgU77ew=; b=oX8Uor1i41U270E8kAGRUGe8bR7E3uRpQBxGXFKLOonwne4IbCNDORGLPlZFXAO3gY HT8i6f1OOpvcTkQbDfuRz4WACdY1KnlQIp1pJ7XywlSahntLIr5wP7gnUCg0RsL3gqr/ f0nvZF3t3OsT3oIDqR2aeR78VASJxUnaugBkpPnCDRh8U5CyIk6yr/3MDSjoj5bi/Qlb iwPwfd7a4eOGfQivBRBwfoaLRTAiMpZhPCvwmZz4/7Ce+oP4Z8DnjwWcnhxnkrZ+9Qcc jZgdN8HbMmWXw15v6xNjQxkNz8s2FTl85H+IfxOAKOwVucfplYM6n3nWV3msF4wCPDvy 9bzg== X-Gm-Message-State: APjAAAUtckYqowA+5X6X11ElcEWudL253RC/3rmauejXOCu0YduVykd0 Edldr7DeMX1ZKrTpQTLhzRtDTg== X-Google-Smtp-Source: APXvYqwxVfdMyfH6Sr7FjkFh00kzLLgvHcBayZz8QBpzQUnopXikBPcilruIA53DE03NvsKBscOJvw== X-Received: by 2002:a05:6402:789:: with SMTP id d9mr28959642edy.25.1566289152351; Tue, 20 Aug 2019 01:19:12 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:569e:0:3106:d637:d723:e855]) by smtp.gmail.com with ESMTPSA id fj15sm2469623ejb.78.2019.08.20.01.19.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2019 01:19:11 -0700 (PDT) From: Daniel Vetter To: LKML Cc: Linux MM , DRI Development , Intel Graphics Development , Daniel Vetter , Jason Gunthorpe , Peter Zijlstra , Ingo Molnar , Andrew Morton , Michal Hocko , David Rientjes , =?utf-8?q?Christian_K=C3=B6nig?= , =?utf-8?b?SsOp?= =?utf-8?b?csO0bWUgR2xpc3Nl?= , Masahiro Yamada , Wei Wang , Andy Shevchenko , Thomas Gleixner , Jann Horn , Feng Tang , Kees Cook , Randy Dunlap , Daniel Vetter Subject: [PATCH 3/4] kernel.h: Add non_block_start/end() Date: Tue, 20 Aug 2019 10:19:01 +0200 Message-Id: <20190820081902.24815-4-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190820081902.24815-1-daniel.vetter@ffwll.ch> References: <20190820081902.24815-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In some special cases we must not block, but there's not a spinlock, preempt-off, irqs-off or similar critical section already that arms the might_sleep() debug checks. Add a non_block_start/end() pair to annotate these. This will be used in the oom paths of mmu-notifiers, where blocking is not allowed to make sure there's forward progress. Quoting Michal: "The notifier is called from quite a restricted context - oom_reaper - which shouldn't depend on any locks or sleepable conditionals. The code should be swift as well but we mostly do care about it to make a forward progress. Checking for sleepable context is the best thing we could come up with that would describe these demands at least partially." Peter also asked whether we want to catch spinlocks on top, but Michal said those are less of a problem because spinlocks can't have an indirect dependency upon the page allocator and hence close the loop with the oom reaper. Suggested by Michal Hocko. v2: - Improve commit message (Michal) - Also check in schedule, not just might_sleep (Peter) v3: It works better when I actually squash in the fixup I had lying around :-/ v4: Pick the suggestion from Andrew Morton to give non_block_start/end some good kerneldoc comments. I added that other blocking calls like wait_event pose similar issues, since that's the other example we discussed. Cc: Jason Gunthorpe Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Andrew Morton Cc: Michal Hocko Cc: David Rientjes Cc: "Christian König" Cc: Daniel Vetter Cc: "Jérôme Glisse" Cc: linux-mm@kvack.org Cc: Masahiro Yamada Cc: Wei Wang Cc: Andy Shevchenko Cc: Thomas Gleixner Cc: Jann Horn Cc: Feng Tang Cc: Kees Cook Cc: Randy Dunlap Cc: linux-kernel@vger.kernel.org Acked-by: Christian König (v1) Signed-off-by: Daniel Vetter Acked-by: Peter Zijlstra (Intel) --- include/linux/kernel.h | 25 ++++++++++++++++++++++++- include/linux/sched.h | 4 ++++ kernel/sched/core.c | 19 ++++++++++++++----- 3 files changed, 42 insertions(+), 6 deletions(-) diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 4fa360a13c1e..82f84cfe372f 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -217,7 +217,9 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset); * might_sleep - annotation for functions that can sleep * * this macro will print a stack trace if it is executed in an atomic - * context (spinlock, irq-handler, ...). + * context (spinlock, irq-handler, ...). Additional sections where blocking is + * not allowed can be annotated with non_block_start() and non_block_end() + * pairs. * * This is a useful debugging help to be able to catch problems early and not * be bitten later when the calling function happens to sleep when it is not @@ -233,6 +235,25 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset); # define cant_sleep() \ do { __cant_sleep(__FILE__, __LINE__, 0); } while (0) # define sched_annotate_sleep() (current->task_state_change = 0) +/** + * non_block_start - annotate the start of section where sleeping is prohibited + * + * This is on behalf of the oom reaper, specifically when it is calling the mmu + * notifiers. The problem is that if the notifier were to block on, for example, + * mutex_lock() and if the process which holds that mutex were to perform a + * sleeping memory allocation, the oom reaper is now blocked on completion of + * that memory allocation. Other blocking calls like wait_event() pose similar + * issues. + */ +# define non_block_start() \ + do { current->non_block_count++; } while (0) +/** + * non_block_end - annotate the end of section where sleeping is prohibited + * + * Closes a section opened by non_block_start(). + */ +# define non_block_end() \ + do { WARN_ON(current->non_block_count-- == 0); } while (0) #else static inline void ___might_sleep(const char *file, int line, int preempt_offset) { } @@ -241,6 +262,8 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset); # define might_sleep() do { might_resched(); } while (0) # define cant_sleep() do { } while (0) # define sched_annotate_sleep() do { } while (0) +# define non_block_start() do { } while (0) +# define non_block_end() do { } while (0) #endif #define might_sleep_if(cond) do { if (cond) might_sleep(); } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index 9f51932bd543..c5630f3dca1f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -974,6 +974,10 @@ struct task_struct { struct mutex_waiter *blocked_on; #endif +#ifdef CONFIG_DEBUG_ATOMIC_SLEEP + int non_block_count; +#endif + #ifdef CONFIG_TRACE_IRQFLAGS unsigned int irq_events; unsigned long hardirq_enable_ip; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2b037f195473..57245770d6cc 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3700,13 +3700,22 @@ static noinline void __schedule_bug(struct task_struct *prev) /* * Various schedule()-time debugging checks and statistics: */ -static inline void schedule_debug(struct task_struct *prev) +static inline void schedule_debug(struct task_struct *prev, bool preempt) { #ifdef CONFIG_SCHED_STACK_END_CHECK if (task_stack_end_corrupted(prev)) panic("corrupted stack end detected inside scheduler\n"); #endif +#ifdef CONFIG_DEBUG_ATOMIC_SLEEP + if (!preempt && prev->state && prev->non_block_count) { + printk(KERN_ERR "BUG: scheduling in a non-blocking section: %s/%d/%i\n", + prev->comm, prev->pid, prev->non_block_count); + dump_stack(); + add_taint(TAINT_WARN, LOCKDEP_STILL_OK); + } +#endif + if (unlikely(in_atomic_preempt_off())) { __schedule_bug(prev); preempt_count_set(PREEMPT_DISABLED); @@ -3813,7 +3822,7 @@ static void __sched notrace __schedule(bool preempt) rq = cpu_rq(cpu); prev = rq->curr; - schedule_debug(prev); + schedule_debug(prev, preempt); if (sched_feat(HRTICK)) hrtick_clear(rq); @@ -6570,7 +6579,7 @@ void ___might_sleep(const char *file, int line, int preempt_offset) rcu_sleep_check(); if ((preempt_count_equals(preempt_offset) && !irqs_disabled() && - !is_idle_task(current)) || + !is_idle_task(current) && !current->non_block_count) || system_state == SYSTEM_BOOTING || system_state > SYSTEM_RUNNING || oops_in_progress) return; @@ -6586,8 +6595,8 @@ void ___might_sleep(const char *file, int line, int preempt_offset) "BUG: sleeping function called from invalid context at %s:%d\n", file, line); printk(KERN_ERR - "in_atomic(): %d, irqs_disabled(): %d, pid: %d, name: %s\n", - in_atomic(), irqs_disabled(), + "in_atomic(): %d, irqs_disabled(): %d, non_block: %d, pid: %d, name: %s\n", + in_atomic(), irqs_disabled(), current->non_block_count, current->pid, current->comm); if (task_stack_end_corrupted(current)) From patchwork Tue Aug 20 08:19:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11103283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26713912 for ; Tue, 20 Aug 2019 08:19:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E786723AFB for ; Tue, 20 Aug 2019 08:19:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="k0TQLN1N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E786723AFB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A24726B0269; Tue, 20 Aug 2019 04:19:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D9BB6B026A; Tue, 20 Aug 2019 04:19:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C4A36B026B; Tue, 20 Aug 2019 04:19:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 6D0EA6B0269 for ; Tue, 20 Aug 2019 04:19:15 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1BCE2181AC9B4 for ; Tue, 20 Aug 2019 08:19:15 +0000 (UTC) X-FDA: 75842106270.29.ant77_16814918fcd26 X-Spam-Summary: 2,0,0,d877fc2f75d2db67,d41d8cd98f00b204,daniel.vetter@ffwll.ch,:linux-kernel@vger.kernel.org::dri-devel@lists.freedesktop.org:intel-gfx@lists.freedesktop.org:daniel.vetter@ffwll.ch:jgg@ziepe.ca:akpm@linux-foundation.org:mhocko@suse.com:rientjes@google.com:christian.koenig@amd.com:jglisse@redhat.com:daniel.vetter@intel.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2911:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:4321:4425:5007:6261:6653:6742:7903:9010:9108:10004:10400:11026:11658:11914:12043:12297:12438:12517:12519:12555:12663:12895:13846:13894:14181:14394:14659:14721:21080:21433:21444:21451:21627:21740:30012:30054:30055:30064:30080,0,RBL:209.85.208.65:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL: none,Cus X-HE-Tag: ant77_16814918fcd26 X-Filterd-Recvd-Size: 5221 Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 20 Aug 2019 08:19:14 +0000 (UTC) Received: by mail-ed1-f65.google.com with SMTP id g8so5341623edm.6 for ; Tue, 20 Aug 2019 01:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qZjzrRlZLlBFvOCMrk8fj5ec19AHD7NWtYEFTJfXwxg=; b=k0TQLN1Nox1/M2ISzbAc4zHTlgcsKT0BXPGsZZPpOBqREaDXPeFdyc1XMaGAKvIGx9 QxjK/4tpciokI/Wa88F954/rhr+3602xdbDv1jh7RSBhbC+T9zEzICLRcmUaOI2umUXJ dsXauR/20FNGp0zT6Wb3KfuJRD0AlIWDKNULA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qZjzrRlZLlBFvOCMrk8fj5ec19AHD7NWtYEFTJfXwxg=; b=IpblqVt18nnV2sf8hvweFY6GQ8pNfR//Sme24doKRJ3SQoCO5VHZui+ePJhdUNvKl8 1rccO7fcINKAKu7proI8xS94YLP/yW0mhJDT8UTdHGsZU7wrM2kSxhXjyjJGegJXAAU9 30ebTRMXqOrKrhEw5BTDiQTdBBGYzGhxWPWKnYWo+gtGvTwbqp2ST4eSwQQ4tUgPCsPL tTPjOeQfbwdbvzbaIiXYbzeu+Ru1zRfK+kQPjCXgBpWk5Pw57qEcGE+GrAHhVPA1Wbs0 vhstdShO94g4evRxe/wAurTgxLbqSdP5IwB1Xya4V2PhvVIztx3el395EVc6Y7jqSwBl 460g== X-Gm-Message-State: APjAAAVJHbHnMcYsYmYvNiffVKXF24mDwgwwuHaQenLcP7T0223bs9K8 RwitPCF6iTelJbd6V6i08w/Zeg== X-Google-Smtp-Source: APXvYqwpUTAeK0tqJjN6py6Pg1Ep5nMFRaQeoDTkIIz0n/eBEVwxM4rYPXYBIaOECS38AYG5tsgjYg== X-Received: by 2002:a17:906:f2d0:: with SMTP id gz16mr24236150ejb.21.1566289153507; Tue, 20 Aug 2019 01:19:13 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:569e:0:3106:d637:d723:e855]) by smtp.gmail.com with ESMTPSA id fj15sm2469623ejb.78.2019.08.20.01.19.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2019 01:19:12 -0700 (PDT) From: Daniel Vetter To: LKML Cc: Linux MM , DRI Development , Intel Graphics Development , Daniel Vetter , Jason Gunthorpe , Andrew Morton , Michal Hocko , David Rientjes , =?utf-8?q?Christian_K=C3=B6nig?= , =?utf-8?b?SsOp?= =?utf-8?b?csO0bWUgR2xpc3Nl?= , Daniel Vetter Subject: [PATCH 4/4] mm, notifier: Catch sleeping/blocking for !blockable Date: Tue, 20 Aug 2019 10:19:02 +0200 Message-Id: <20190820081902.24815-5-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190820081902.24815-1-daniel.vetter@ffwll.ch> References: <20190820081902.24815-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We need to make sure implementations don't cheat and don't have a possible schedule/blocking point deeply burried where review can't catch it. I'm not sure whether this is the best way to make sure all the might_sleep() callsites trigger, and it's a bit ugly in the code flow. But it gets the job done. Inspired by an i915 patch series which did exactly that, because the rules haven't been entirely clear to us. v2: Use the shiny new non_block_start/end annotations instead of abusing preempt_disable/enable. v3: Rebase on top of Glisse's arg rework. v4: Rebase on top of more Glisse rework. Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Michal Hocko Cc: David Rientjes Cc: "Christian König" Cc: Daniel Vetter Cc: "Jérôme Glisse" Cc: linux-mm@kvack.org Reviewed-by: Christian König Reviewed-by: Jérôme Glisse Signed-off-by: Daniel Vetter --- mm/mmu_notifier.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 538d3bb87f9b..856636d06ee0 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -181,7 +181,13 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist) { if (mn->ops->invalidate_range_start) { - int _ret = mn->ops->invalidate_range_start(mn, range); + int _ret; + + if (!mmu_notifier_range_blockable(range)) + non_block_start(); + _ret = mn->ops->invalidate_range_start(mn, range); + if (!mmu_notifier_range_blockable(range)) + non_block_end(); if (_ret) { pr_info("%pS callback failed with %d in %sblockable context.\n", mn->ops->invalidate_range_start, _ret,