From patchwork Fri Oct 23 12:21:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852863 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DCA716BC for ; Fri, 23 Oct 2020 12:22:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DACD120795 for ; Fri, 23 Oct 2020 12:22:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="SBvkme8o" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DACD120795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CAD1D6B005D; Fri, 23 Oct 2020 08:22:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5C126B0062; Fri, 23 Oct 2020 08:22:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4ACD6B0068; Fri, 23 Oct 2020 08:22:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id 84CE26B005D for ; Fri, 23 Oct 2020 08:22:27 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 100AE180AD815 for ; Fri, 23 Oct 2020 12:22:27 +0000 (UTC) X-FDA: 77403103134.06.birds69_0e0e24027259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id E82F3100C0CB6 for ; Fri, 23 Oct 2020 12:22:26 +0000 (UTC) X-Spam-Summary: 1,0,0,973bc7f4a7868f3c,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:2:41:69:152:355:379:541:800:960:968:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1434:1437:1515:1516:1518:1535:1593:1594:1605:1606:1730:1747:1777:1792:2194:2198:2199:2200:2393:2559:2562:2731:2911:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4119:4321:4425:5007:6119:6261:6653:6691:6742:7875:7903:9592:10004:11026:11473:11658:11914:12043:12114:12291:12296:12297:12438:12517:12519:12555:12663:12683:12700:12737:12895:13153:13161:13180:13221:13228:13229:13846:13894:13972:14096:14097:14394:14659:21080:21324:21433:21444:21450:21451:21627:21987:21990:30012:30054:30055:30064:30070:30091,0,RBL:209.85.221.65:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yrypsrcickh5hh64necdh71hfzxyccmcynyede99trar89uzy1xsnkn4y55xf.8a6xagu564m8mkmzzgnan6k481k99bo5u5k3b4iz3kuu59ufyspjtrb85k97p96.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:no ne,Domai X-HE-Tag: birds69_0e0e24027259 X-Filterd-Recvd-Size: 8872 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:25 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id n15so1596454wrq.2 for ; Fri, 23 Oct 2020 05:22:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IgQVmlN66kTzdCBU8ZoKnJMQUq1PyY1GSJDM/XX0ojY=; b=SBvkme8oy7kGO+RfY6ceOp15GpaJIvk6RoLhvV/WueIm6Pg+oG/7v+xKkGaetsoFKT 4Jrw4O5tvC4Old6EBSgoM8lbYnG5Co58mGGq/XlAbL+NJ+J1nzTnHFdqYbcE/fjum1hb sSY+nGawYtOErvna7ZMT/xH6GPplKKq36FBvY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IgQVmlN66kTzdCBU8ZoKnJMQUq1PyY1GSJDM/XX0ojY=; b=WDoay85WCz/vwguGpYwwaBwaw+m46MSw+7zv827bRE4G++zjVmbIgbH9g+OGSBoEDp OnAJHXDBZlAflmP3EZ5o/gGamZJTHzU3Ch+PN1j/NjKzvLev1cBT5bnomdxC2EK3KWzu BCuTCuJ3AFHS9U0kCP4L4fsSPdc/4JVpWewfJr5ZcNwRqfpOUEC7/msKH/oeSnqZ4kp0 hEwIEdFXfWQxYgedUZQ+54gsEeY+ZxHk9roO8ybegItC3qCgAm+rCcyh3WlRa64D1wFz aA2XgxQBZ5OikY4vQnAC6u4XlG2jCeOjofzDDNro6Wc8ViKv4EBzMTbAX1VQ0z0hYR4O 0qkg== X-Gm-Message-State: AOAM532NPGWgRk2QCKNE3aKbkU2iBvu9p5zrvsMqJ+jGc+OdXp3DcdOU aph1Zps1xW1sHGY8ujdToZ8n+g== X-Google-Smtp-Source: ABdhPJznoMnt3wkQ258dPqEZNcn4LIp7cxpxLk7m7dixQUbSTt/QJMXc3d+QHpQiaKAR3urrcsOBqA== X-Received: by 2002:adf:e8c7:: with SMTP id k7mr2365623wrn.102.1603455744516; Fri, 23 Oct 2020 05:22:24 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:23 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , linux-fsdevel@vger.kernel.org, Dave Chinner , Qian Cai , linux-xfs@vger.kernel.org, =?utf-8?q?Thomas_Hellstr=C3=B6m?= , Andrew Morton , Jason Gunthorpe , linux-mm@kvack.org, linux-rdma@vger.kernel.org, Maarten Lankhorst , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Daniel Vetter Subject: [PATCH 03/65] mm: Track mmu notifiers in fs_reclaim_acquire/release Date: Fri, 23 Oct 2020 14:21:14 +0200 Message-Id: <20201023122216.2373294-3-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: fs_reclaim_acquire/release nicely catch recursion issues when allocating GFP_KERNEL memory against shrinkers (which gpu drivers tend to use to keep the excessive caches in check). For mmu notifier recursions we do have lockdep annotations since 23b68395c7c7 ("mm/mmu_notifiers: add a lockdep map for invalidate_range_start/end"). But these only fire if a path actually results in some pte invalidation - for most small allocations that's very rarely the case. The other trouble is that pte invalidation can happen any time when __GFP_RECLAIM is set. Which means only really GFP_ATOMIC is a safe choice, GFP_NOIO isn't good enough to avoid potential mmu notifier recursion. I was pondering whether we should just do the general annotation, but there's always the risk for false positives. Plus I'm assuming that the core fs and io code is a lot better reviewed and tested than random mmu notifier code in drivers. Hence why I decide to only annotate for that specific case. Furthermore even if we'd create a lockdep map for direct reclaim, we'd still need to explicit pull in the mmu notifier map - there's a lot more places that do pte invalidation than just direct reclaim, these two contexts arent the same. Note that the mmu notifiers needing their own independent lockdep map is also the reason we can't hold them from fs_reclaim_acquire to fs_reclaim_release - it would nest with the acquistion in the pte invalidation code, causing a lockdep splat. And we can't remove the annotations from pte invalidation and all the other places since they're called from many other places than page reclaim. Hence we can only do the equivalent of might_lock, but on the raw lockdep map. With this we can also remove the lockdep priming added in 66204f1d2d1b ("mm/mmu_notifiers: prime lockdep") since the new annotations are strictly more powerful. v2: Review from Thomas Hellstrom: - unbotch the fs_reclaim context check, I accidentally inverted it, but it didn't blow up because I inverted it immediately - fix compiling for !CONFIG_MMU_NOTIFIER v3: Unbreak the PF_MEMALLOC_ context flags. Thanks to Qian for the report and Dave for explaining what I failed to see. Cc: linux-fsdevel@vger.kernel.org Cc: Dave Chinner Cc: Qian Cai Cc: linux-xfs@vger.kernel.org Cc: Thomas Hellström (Intel) Cc: Andrew Morton Cc: Jason Gunthorpe Cc: linux-mm@kvack.org Cc: linux-rdma@vger.kernel.org Cc: Maarten Lankhorst Cc: Christian König Signed-off-by: Daniel Vetter --- mm/mmu_notifier.c | 7 ------- mm/page_alloc.c | 31 ++++++++++++++++++++----------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 4fc918163dd3..6bf798373eb0 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -612,13 +612,6 @@ int __mmu_notifier_register(struct mmu_notifier *subscription, mmap_assert_write_locked(mm); BUG_ON(atomic_read(&mm->mm_users) <= 0); - if (IS_ENABLED(CONFIG_LOCKDEP)) { - fs_reclaim_acquire(GFP_KERNEL); - lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); - lock_map_release(&__mmu_notifier_invalidate_range_start_map); - fs_reclaim_release(GFP_KERNEL); - } - if (!mm->notifier_subscriptions) { /* * kmalloc cannot be called under mm_take_all_locks(), but we diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 780c8f023b28..2edd3fd447fa 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -57,6 +57,7 @@ #include #include #include +#include #include #include #include @@ -4207,10 +4208,8 @@ should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_fla static struct lockdep_map __fs_reclaim_map = STATIC_LOCKDEP_MAP_INIT("fs_reclaim", &__fs_reclaim_map); -static bool __need_fs_reclaim(gfp_t gfp_mask) +static bool __need_reclaim(gfp_t gfp_mask) { - gfp_mask = current_gfp_context(gfp_mask); - /* no reclaim without waiting on it */ if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) return false; @@ -4219,10 +4218,6 @@ static bool __need_fs_reclaim(gfp_t gfp_mask) if (current->flags & PF_MEMALLOC) return false; - /* We're only interested __GFP_FS allocations for now */ - if (!(gfp_mask & __GFP_FS)) - return false; - if (gfp_mask & __GFP_NOLOCKDEP) return false; @@ -4241,15 +4236,29 @@ void __fs_reclaim_release(void) void fs_reclaim_acquire(gfp_t gfp_mask) { - if (__need_fs_reclaim(gfp_mask)) - __fs_reclaim_acquire(); + gfp_mask = current_gfp_context(gfp_mask); + + if (__need_reclaim(gfp_mask)) { + if (gfp_mask & __GFP_FS) + __fs_reclaim_acquire(); + +#ifdef CONFIG_MMU_NOTIFIER + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); + lock_map_release(&__mmu_notifier_invalidate_range_start_map); +#endif + + } } EXPORT_SYMBOL_GPL(fs_reclaim_acquire); void fs_reclaim_release(gfp_t gfp_mask) { - if (__need_fs_reclaim(gfp_mask)) - __fs_reclaim_release(); + gfp_mask = current_gfp_context(gfp_mask); + + if (__need_reclaim(gfp_mask)) { + if (gfp_mask & __GFP_FS) + __fs_reclaim_release(); + } } EXPORT_SYMBOL_GPL(fs_reclaim_release); #endif From patchwork Fri Oct 23 12:21:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852869 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E2601580 for ; Fri, 23 Oct 2020 12:22:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D74B821527 for ; Fri, 23 Oct 2020 12:22:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="GgfqsG68" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D74B821527 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E82B56B0062; Fri, 23 Oct 2020 08:22:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E0B2C6B0068; Fri, 23 Oct 2020 08:22:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6D486B006C; Fri, 23 Oct 2020 08:22:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 946966B0062 for ; Fri, 23 Oct 2020 08:22:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2D0A11EE6 for ; Fri, 23 Oct 2020 12:22:28 +0000 (UTC) X-FDA: 77403103176.01.fear51_0d108a627259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id ED40510229464 for ; Fri, 23 Oct 2020 12:22:27 +0000 (UTC) X-Spam-Summary: 1,0,0,0e42daab42bd67ea,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:355:379:541:800:960:973:982:988:989:1260:1311:1314:1345:1359:1431:1437:1515:1535:1544:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:3138:3139:3140:3141:3142:3355:3865:3867:3868:3870:3871:3872:3874:4118:4321:4605:5007:6119:6261:6653:6742:6743:7875:7903:8603:8660:8957:9592:10004:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13148:13161:13180:13229:13230:13846:13894:14096:14181:14394:14721:21080:21433:21444:21451:21627:21939:21987:21990:30012:30054:30055:30064:30069:30070,0,RBL:209.85.221.65:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yftnceexaskuajx8yuoufrunrmaycn4sfxuhcegazmtx67qh3f45gh5wxqbe9.5s4whpmhxznxmaibxhn3781977egbbdkmc3eaoti1uzwke5h9nksyukk7mpoxhs.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: fear51_0d108a627259 X-Filterd-Recvd-Size: 7656 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:27 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id b8so1598712wrn.0 for ; Fri, 23 Oct 2020 05:22:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Md4YAXpjpdGTyHyAgh2ay+AKQEd8Ro2fcgd8T+fjD4o=; b=GgfqsG68s8RU3qED2SP+ReACq0NGvEMRdgsY6b6ebZqvXlKJoL/Mx/M9P+m8eu/R51 JiE6HK/crlCj0SnKoHO/QCCI3VkmWtqVIAyu4QXV/TZTZHYUHURHT9eDDCsbRIL7hGE7 uWyhojoODyHuN4DFjcfQ46NFY2Khc8hMcQmkE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Md4YAXpjpdGTyHyAgh2ay+AKQEd8Ro2fcgd8T+fjD4o=; b=mEKhhQL7473x8f4UaQ9oBdGcUkIGpjalZdaAf4d1X7wc4zDCMx6gWrloUxqFM2DMZr 8egToRPOTf6dCIuAUgKNTnmrkfsXdXCbhS6ZwXVuZeG/4YZkc9lo+C82mQ4pvdp8fMXs 3WBGuK/Q9rD9zgcZsuYh77W5a1w5wUhuClrJc4mQMjeNgLs4gcskbz0uMxApvB6F96QU DwLi6MY5Qavzwk09TBx8n1CvO9AWe6TC5wD/rGBKzumqb4OVHZusTywyMEzraX34pGT0 c/IGr3sTvGR9kNe8SpA3zKjN9OwRzjeGoWkCVrXjL6pmzGNTRfd50NRHC6T7qatFbWZ7 DrNg== X-Gm-Message-State: AOAM532wZ5o1KcIj+Q+0CecQZ+/UEy+pqzXtTw2lN5NLzL1axiuyMplJ NcnqgWw7N2ygLfQUmGEyTg91zA== X-Google-Smtp-Source: ABdhPJwY3k8N8hWJAzZAqbhvx4fpSD2M0kW3wHfJxLbpQ9x79iQZHoLA9FsW4Ur15mgIydU2Qpf3mw== X-Received: by 2002:adf:f4ca:: with SMTP id h10mr2271950wrp.89.1603455746195; Fri, 23 Oct 2020 05:22:26 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:25 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , "Paul E . McKenney" , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Peter Zijlstra , Ingo Molnar , Vlastimil Babka , Mathieu Desnoyers , Sebastian Andrzej Siewior , Michel Lespinasse , Waiman Long , Thomas Gleixner , Randy Dunlap , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Dave Chinner , Qian Cai , linux-xfs@vger.kernel.org, Daniel Vetter Subject: [PATCH 04/65] mm: Extract might_alloc() debug check Date: Fri, 23 Oct 2020 14:21:15 +0200 Message-Id: <20201023122216.2373294-4-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Extracted from slab.h, which seems to have the most complete version including the correct might_sleep() check. Roll it out to slob.c. Motivated by a discussion with Paul about possibly changing call_rcu behaviour to allocate memory, but only roughly every 500th call. There are a lot fewer places in the kernel that care about whether allocating memory is allowed or not (due to deadlocks with reclaim code) than places that care whether sleeping is allowed. But debugging these also tends to be a lot harder, so nice descriptive checks could come in handy. I might have some use eventually for annotations in drivers/gpu. Note that unlike fs_reclaim_acquire/release gfpflags_allow_blocking does not consult the PF_MEMALLOC flags. But there is no flag equivalent for GFP_NOWAIT, hence this check can't go wrong due to memalloc_no*_save/restore contexts. Cc: Paul E. McKenney Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Andrew Morton Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Vlastimil Babka Cc: Mathieu Desnoyers Cc: Sebastian Andrzej Siewior Cc: Michel Lespinasse Cc: Daniel Vetter Cc: Waiman Long Cc: Thomas Gleixner Cc: Randy Dunlap Cc: linux-mm@kvack.org Cc: linux-fsdevel@vger.kernel.org Cc: Dave Chinner Cc: Qian Cai Cc: linux-xfs@vger.kernel.org Signed-off-by: Daniel Vetter Acked-by: Vlastimil Babka Acked-by: Paul E. McKenney --- include/linux/sched/mm.h | 16 ++++++++++++++++ mm/slab.h | 5 +---- mm/slob.c | 6 ++---- 3 files changed, 19 insertions(+), 8 deletions(-) diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index f889e332912f..2b0037abac0b 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -205,6 +205,22 @@ static inline void fs_reclaim_acquire(gfp_t gfp_mask) { } static inline void fs_reclaim_release(gfp_t gfp_mask) { } #endif +/** + * might_alloc - Marks possible allocation sites + * @gfp_mask: gfp_t flags that would be use to allocate + * + * Similar to might_sleep() and other annotations this can be used in functions + * that might allocate, but often dont. Compiles to nothing without + * CONFIG_LOCKDEP. Includes a conditional might_sleep() if @gfp allows blocking. + */ +static inline void might_alloc(gfp_t gfp_mask) +{ + fs_reclaim_acquire(gfp_mask); + fs_reclaim_release(gfp_mask); + + might_sleep_if(gfpflags_allow_blocking(gfp_mask)); +} + /** * memalloc_noio_save - Marks implicit GFP_NOIO allocation scope. * diff --git a/mm/slab.h b/mm/slab.h index 6cc323f1313a..fedd789b2270 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -492,10 +492,7 @@ static inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s, { flags &= gfp_allowed_mask; - fs_reclaim_acquire(flags); - fs_reclaim_release(flags); - - might_sleep_if(gfpflags_allow_blocking(flags)); + might_alloc(flags); if (should_failslab(s, flags)) return NULL; diff --git a/mm/slob.c b/mm/slob.c index 7cc9805c8091..8d4bfa46247f 100644 --- a/mm/slob.c +++ b/mm/slob.c @@ -474,8 +474,7 @@ __do_kmalloc_node(size_t size, gfp_t gfp, int node, unsigned long caller) gfp &= gfp_allowed_mask; - fs_reclaim_acquire(gfp); - fs_reclaim_release(gfp); + might_alloc(gfp); if (size < PAGE_SIZE - minalign) { int align = minalign; @@ -597,8 +596,7 @@ static void *slob_alloc_node(struct kmem_cache *c, gfp_t flags, int node) flags &= gfp_allowed_mask; - fs_reclaim_acquire(flags); - fs_reclaim_release(flags); + might_alloc(flags); if (c->size < PAGE_SIZE) { b = slob_alloc(c->size, flags, c->align, node, 0); From patchwork Fri Oct 23 12:21:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22C8592C for ; Fri, 23 Oct 2020 12:22:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B10F924248 for ; Fri, 23 Oct 2020 12:22:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="ZLQYcJfG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B10F924248 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD2FC6B0068; Fri, 23 Oct 2020 08:22:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C82476B006C; Fri, 23 Oct 2020 08:22:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B71E56B006E; Fri, 23 Oct 2020 08:22:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id 86B9F6B0068 for ; Fri, 23 Oct 2020 08:22:57 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0D3212470 for ; Fri, 23 Oct 2020 12:22:57 +0000 (UTC) X-FDA: 77403104394.23.stove18_181025627259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id DACB037604 for ; Fri, 23 Oct 2020 12:22:56 +0000 (UTC) X-Spam-Summary: 1,0,0,cd575377b4c8f7ef,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:1:2:41:69:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3865:3867:3868:3871:3872:3874:4050:4250:4321:4385:5007:6117:6119:6261:6653:6742:7875:7903:9592:11026:11232:11473:11658:11914:12043:12050:12291:12296:12297:12438:12517:12519:12555:12683:12895:13161:13229:13846:13894:14394:14659:21080:21324:21433:21444:21451:21627:21990:30003:30012:30051:30054:30055:30064:30070:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04ygn1qefpxtwmrymzbicdewwwet8opusiykr5yienwa7yixmqhhkf51s6gdpzh.5meyduh9nua17b5fbtssy6t1r5w4cfogjwrjccrouu41os3dnydpnqgzodhehty.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:1:0,LFtime:2 5,LUA_SU X-HE-Tag: stove18_181025627259 X-Filterd-Recvd-Size: 10496 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:56 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id h7so1587648wre.4 for ; Fri, 23 Oct 2020 05:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=13dXuBKsrPtV1DlWwyiIYqTF1cPCadFdp8ckLAWd0AI=; b=ZLQYcJfGXFA882t746W/3+RPIsXyZnhi4SwDIyg6MTBPgmo0ug4tuTNhsmPDGRsV1R om7kV/lJ7tQL8I335IWN4eoYkgJc5G62sDYmr2kL5A/r0tu/lpbPN15e0H+jst1xf+hU NWlcqaPiMhTfjeLoSXdopaVX6OqVUgSNKKf3k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=13dXuBKsrPtV1DlWwyiIYqTF1cPCadFdp8ckLAWd0AI=; b=dudw0cb2HemP94/fWK9+vtk7CDMXQCSwTgkK9inQnWljteGpxUsYX0PfTSr/G96DS/ Tbw7O4PCN9KIqPYcVS45np6DAVoUfCwg3ced8bW0Z6b/qF5845tUvXHamOTLsaaFw9Tu LV2q/iRSzmfhVYHf1g99FT9lDssQ27F5GWllHtBHzDSOy6PatyySPsi9vlrYW+NyNesG TpscHqq6ddm38eWtfzBj9MEfxKsGEUVOyjpXRpPP+MAcox1ZbAq6nekkwZO1UGu2HmeV 23htUpgiPjvE8yzsaq09OVbfyvyllE8Tjc9t4Jjd3JMWy1wTy+q2NO6lF/N6mQuHPije 7Mmw== X-Gm-Message-State: AOAM532RPD/TJLeDqnyxto0TJBn59ceIQuax8FPtWkfLDG9YLBOPRyYL TOW/SeLeKLZ3u0xwxsM8fIpGtA== X-Google-Smtp-Source: ABdhPJyqSAn0aM35YnoVtNUbn1vJJ8l+XxiU6chHGaoO67vyxZzJbEVkyOyc9YZmKmVc8U3bDNN/Yw== X-Received: by 2002:adf:e685:: with SMTP id r5mr2560251wrm.340.1603455774974; Fri, 23 Oct 2020 05:22:54 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:54 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Gerald Schaefer , Daniel Vetter , Jason Gunthorpe , Dan Williams , Kees Cook , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Niklas Schnelle , linux-s390@vger.kernel.org Subject: [PATCH 29/65] s390/pci: Remove races against pte updates Date: Fri, 23 Oct 2020 14:21:40 +0200 Message-Id: <20201023122216.2373294-29-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed: - gpu drivers dynamically manage their memory nowadays, invalidating ptes with unmap_mapping_range when buffers get moved - contiguous dma allocations have moved from dedicated carvetouts to cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE) - even /dev/mem now invalidates mappings when the kernel requests that iomem region when CONFIG_IO_STRICT_DEVMEM is set, see commit 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea. Fix this. Since zpci_memcpy_from|toio seems to not do anything nefarious with locks we just need to open code get_pfn and follow_pfn and make sure we drop the locks only after we're done. The write function also needs the copy_from_user move, since we can't take userspace faults while holding the mmap sem. v2: Move VM_IO | VM_PFNMAP checks around so they keep returning EINVAL like before (Gerard) v3: Polish commit message (Niklas) Reviewed-by: Gerald Schaefer Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Dan Williams Cc: Kees Cook Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Niklas Schnelle Cc: Gerald Schaefer Cc: linux-s390@vger.kernel.org Cc: Niklas Schnelle Signed-off-by: Daniel Vetter --- arch/s390/pci/pci_mmio.c | 98 +++++++++++++++++++++++----------------- 1 file changed, 57 insertions(+), 41 deletions(-) diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 401cf670a243..1a6adbc68ee8 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -119,33 +119,15 @@ static inline int __memcpy_toio_inuser(void __iomem *dst, return rc; } -static long get_pfn(unsigned long user_addr, unsigned long access, - unsigned long *pfn) -{ - struct vm_area_struct *vma; - long ret; - - mmap_read_lock(current->mm); - ret = -EINVAL; - vma = find_vma(current->mm, user_addr); - if (!vma) - goto out; - ret = -EACCES; - if (!(vma->vm_flags & access)) - goto out; - ret = follow_pfn(vma, user_addr, pfn); -out: - mmap_read_unlock(current->mm); - return ret; -} - SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, const void __user *, user_buffer, size_t, length) { u8 local_buf[64]; void __iomem *io_addr; void *buf; - unsigned long pfn; + struct vm_area_struct *vma; + pte_t *ptep; + spinlock_t *ptl; long ret; if (!zpci_is_enabled()) @@ -158,7 +140,7 @@ SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, * We only support write access to MIO capable devices if we are on * a MIO enabled system. Otherwise we would have to check for every * address if it is a special ZPCI_ADDR and would have to do - * a get_pfn() which we don't need for MIO capable devices. Currently + * a pfn lookup which we don't need for MIO capable devices. Currently * ISM devices are the only devices without MIO support and there is no * known need for accessing these from userspace. */ @@ -176,21 +158,37 @@ SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, } else buf = local_buf; - ret = get_pfn(mmio_addr, VM_WRITE, &pfn); + ret = -EFAULT; + if (copy_from_user(buf, user_buffer, length)) + goto out_free; + + mmap_read_lock(current->mm); + ret = -EINVAL; + vma = find_vma(current->mm, mmio_addr); + if (!vma) + goto out_unlock_mmap; + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) + goto out_unlock_mmap; + ret = -EACCES; + if (!(vma->vm_flags & VM_WRITE)) + goto out_unlock_mmap; + + ret = follow_pte_pmd(vma->vm_mm, mmio_addr, NULL, &ptep, NULL, &ptl); if (ret) - goto out; - io_addr = (void __iomem *)((pfn << PAGE_SHIFT) | + goto out_unlock_mmap; + + io_addr = (void __iomem *)((pte_pfn(*ptep) << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK)); - ret = -EFAULT; if ((unsigned long) io_addr < ZPCI_IOMAP_ADDR_BASE) - goto out; - - if (copy_from_user(buf, user_buffer, length)) - goto out; + goto out_unlock_pt; ret = zpci_memcpy_toio(io_addr, buf, length); -out: +out_unlock_pt: + pte_unmap_unlock(ptep, ptl); +out_unlock_mmap: + mmap_read_unlock(current->mm); +out_free: if (buf != local_buf) kfree(buf); return ret; @@ -274,7 +272,9 @@ SYSCALL_DEFINE3(s390_pci_mmio_read, unsigned long, mmio_addr, u8 local_buf[64]; void __iomem *io_addr; void *buf; - unsigned long pfn; + struct vm_area_struct *vma; + pte_t *ptep; + spinlock_t *ptl; long ret; if (!zpci_is_enabled()) @@ -287,7 +287,7 @@ SYSCALL_DEFINE3(s390_pci_mmio_read, unsigned long, mmio_addr, * We only support read access to MIO capable devices if we are on * a MIO enabled system. Otherwise we would have to check for every * address if it is a special ZPCI_ADDR and would have to do - * a get_pfn() which we don't need for MIO capable devices. Currently + * a pfn lookup which we don't need for MIO capable devices. Currently * ISM devices are the only devices without MIO support and there is no * known need for accessing these from userspace. */ @@ -306,22 +306,38 @@ SYSCALL_DEFINE3(s390_pci_mmio_read, unsigned long, mmio_addr, buf = local_buf; } - ret = get_pfn(mmio_addr, VM_READ, &pfn); + mmap_read_lock(current->mm); + ret = -EINVAL; + vma = find_vma(current->mm, mmio_addr); + if (!vma) + goto out_unlock_mmap; + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) + goto out_unlock_mmap; + ret = -EACCES; + if (!(vma->vm_flags & VM_WRITE)) + goto out_unlock_mmap; + + ret = follow_pte_pmd(vma->vm_mm, mmio_addr, NULL, &ptep, NULL, &ptl); if (ret) - goto out; - io_addr = (void __iomem *)((pfn << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK)); + goto out_unlock_mmap; + + io_addr = (void __iomem *)((pte_pfn(*ptep) << PAGE_SHIFT) | + (mmio_addr & ~PAGE_MASK)); if ((unsigned long) io_addr < ZPCI_IOMAP_ADDR_BASE) { ret = -EFAULT; - goto out; + goto out_unlock_pt; } ret = zpci_memcpy_fromio(buf, io_addr, length); - if (ret) - goto out; - if (copy_to_user(user_buffer, buf, length)) + +out_unlock_pt: + pte_unmap_unlock(ptep, ptl); +out_unlock_mmap: + mmap_read_unlock(current->mm); + + if (!ret && copy_to_user(user_buffer, buf, length)) ret = -EFAULT; -out: if (buf != local_buf) kfree(buf); return ret; From patchwork Fri Oct 23 12:21:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852935 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C04E617D4 for ; Fri, 23 Oct 2020 12:23:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 788102168B for ; Fri, 23 Oct 2020 12:23:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="A2ERDcZ2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 788102168B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C61386B006C; Fri, 23 Oct 2020 08:22:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BE9A66B006E; Fri, 23 Oct 2020 08:22:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A87676B0070; Fri, 23 Oct 2020 08:22:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id 74AE96B006C for ; Fri, 23 Oct 2020 08:22:58 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 12039246E for ; Fri, 23 Oct 2020 12:22:58 +0000 (UTC) X-FDA: 77403104436.05.drain85_3309a8a27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id E054818025895 for ; Fri, 23 Oct 2020 12:22:57 +0000 (UTC) X-Spam-Summary: 1,0,0,6514d629abe83690,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:69:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1544:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:3138:3139:3140:3141:3142:3355:3865:3866:3867:3873:3874:4119:4321:4385:4605:5007:6119:6261:6609:6653:6742:7875:7903:8603:9010:9036:9592:10004:11026:11473:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13846:13894:14181:14394:14659:14721:21080:21444:21451:21627:21773:21795:21990:30051:30054:30055:30064:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yfng6d4cxhwpftm9kicm49k8qehyc6r3rzzf8cuys9gjpfqyrbk3tt4kmrgys.1jrec6p63i1mshaozq5o3djq5kycheoqdzig9u4ap6ffo5og3reu7d8xny13mo7.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:152,LUA_SUMMARY:non e X-HE-Tag: drain85_3309a8a27259 X-Filterd-Recvd-Size: 8325 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:57 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id n18so1578613wrs.5 for ; Fri, 23 Oct 2020 05:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5HSDx6z2iUpxgLk7STJ2LBrJWLho+Vrj3pPui9pXoI8=; b=A2ERDcZ2h8YmUWNBZ/rO5ci+Zbio7dzc5zE6ueA1G5iHXv/nSSfH5nKWJnGE2tP6SM lAIybYIl2XJ1d4QLTooEx1cWiTwgcTzCJgLxu3BjxHhuhmR3A1IWUsjqRJ6BxOK5uu1H ORlRa3CHR0RumFl54NPYbFU425Dlijl+XroEk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5HSDx6z2iUpxgLk7STJ2LBrJWLho+Vrj3pPui9pXoI8=; b=WnDl1DXKn9BIHWf16prtrx1WmTjjCIDbU4HtAWav9U1nEbBQ/uaZKf0wm6Wbg/Pv/v 0ipkkA19M1qAe9N6I8wS30/ZelKQetDmD6Oqaz8CNrIn72xsrJEjit1F7BcX1geeorZ5 fTcpowclgXSWXnYg5ySDgP9GOhuEkSgqTN/vbdMcu9RaDcRyn43euYzCatTuI51tSs2+ 87WK9M0LgazqUEJNwUHeqH5jN8RxKt+oy1B8fz0hYwZW8+cApHgbxIG4rIGkbfrPt1Q2 JdXax26OecTKq5lN1408qFaVCYMXpXl9EqJjkfow6V96xNROrEoKWT6xMfXBbJY/BNLF Ba2Q== X-Gm-Message-State: AOAM531ghXuGvcCWNEXpvEPgyfXApi4hCAZ0tKgZbdscnkGRYor24O4X bUopKZSeghdDoqUGfMw9qu6cZg== X-Google-Smtp-Source: ABdhPJwB9oBURoZEUIU0XwnGncVrFo4aEZO/vR71X+2pBVqd1kDG6aUbhmExOYLy9L+jfKme/7oc4w== X-Received: by 2002:adf:e589:: with SMTP id l9mr2327940wrm.110.1603455776358; Fri, 23 Oct 2020 05:22:56 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:55 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , John Hubbard , Daniel Vetter , Jason Gunthorpe , Inki Dae , Joonyoung Shim , Seung-Woo Kim , Kyungmin Park , Kukjin Kim , Krzysztof Kozlowski , Andrew Morton , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 30/65] drm/exynos: Stop using frame_vector helpers Date: Fri, 23 Oct 2020 14:21:41 +0200 Message-Id: <20201023122216.2373294-30-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames. Reviewed-by: John Hubbard Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Inki Dae Cc: Joonyoung Shim Cc: Seung-Woo Kim Cc: Kyungmin Park Cc: Kukjin Kim Cc: Krzysztof Kozlowski Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Signed-off-by: Daniel Vetter --- v2: Use unpin_user_pages_dirty_lock (John) --- drivers/gpu/drm/exynos/Kconfig | 1 - drivers/gpu/drm/exynos/exynos_drm_g2d.c | 47 +++++++++++-------------- 2 files changed, 20 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/exynos/Kconfig b/drivers/gpu/drm/exynos/Kconfig index 6417f374b923..43257ef3c09d 100644 --- a/drivers/gpu/drm/exynos/Kconfig +++ b/drivers/gpu/drm/exynos/Kconfig @@ -88,7 +88,6 @@ comment "Sub-drivers" config DRM_EXYNOS_G2D bool "G2D" depends on VIDEO_SAMSUNG_S5P_G2D=n || COMPILE_TEST - select FRAME_VECTOR help Choose this option if you want to use Exynos G2D for DRM. diff --git a/drivers/gpu/drm/exynos/exynos_drm_g2d.c b/drivers/gpu/drm/exynos/exynos_drm_g2d.c index 967a5cdc120e..ecede41af9b9 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_g2d.c +++ b/drivers/gpu/drm/exynos/exynos_drm_g2d.c @@ -205,7 +205,8 @@ struct g2d_cmdlist_userptr { dma_addr_t dma_addr; unsigned long userptr; unsigned long size; - struct frame_vector *vec; + struct page **pages; + unsigned int npages; struct sg_table *sgt; atomic_t refcount; bool in_pool; @@ -378,7 +379,6 @@ static void g2d_userptr_put_dma_addr(struct g2d_data *g2d, bool force) { struct g2d_cmdlist_userptr *g2d_userptr = obj; - struct page **pages; if (!obj) return; @@ -398,15 +398,9 @@ static void g2d_userptr_put_dma_addr(struct g2d_data *g2d, dma_unmap_sgtable(to_dma_dev(g2d->drm_dev), g2d_userptr->sgt, DMA_BIDIRECTIONAL, 0); - pages = frame_vector_pages(g2d_userptr->vec); - if (!IS_ERR(pages)) { - int i; - - for (i = 0; i < frame_vector_count(g2d_userptr->vec); i++) - set_page_dirty_lock(pages[i]); - } - put_vaddr_frames(g2d_userptr->vec); - frame_vector_destroy(g2d_userptr->vec); + unpin_user_pages_dirty_lock(g2d_userptr->pages, g2d_userptr->npages, + true); + kvfree(g2d_userptr->pages); if (!g2d_userptr->out_of_list) list_del_init(&g2d_userptr->list); @@ -474,35 +468,34 @@ static dma_addr_t *g2d_userptr_get_dma_addr(struct g2d_data *g2d, offset = userptr & ~PAGE_MASK; end = PAGE_ALIGN(userptr + size); npages = (end - start) >> PAGE_SHIFT; - g2d_userptr->vec = frame_vector_create(npages); - if (!g2d_userptr->vec) { + g2d_userptr->pages = kvmalloc_array(npages, sizeof(*g2d_userptr->pages), + GFP_KERNEL); + if (!g2d_userptr->pages) { ret = -ENOMEM; goto err_free; } - ret = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE, - g2d_userptr->vec); + ret = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE, + g2d_userptr->pages); if (ret != npages) { DRM_DEV_ERROR(g2d->dev, "failed to get user pages from userptr.\n"); if (ret < 0) - goto err_destroy_framevec; - ret = -EFAULT; - goto err_put_framevec; - } - if (frame_vector_to_pages(g2d_userptr->vec) < 0) { + goto err_destroy_pages; + npages = ret; ret = -EFAULT; - goto err_put_framevec; + goto err_unpin_pages; } + g2d_userptr->npages = npages; sgt = kzalloc(sizeof(*sgt), GFP_KERNEL); if (!sgt) { ret = -ENOMEM; - goto err_put_framevec; + goto err_unpin_pages; } ret = sg_alloc_table_from_pages(sgt, - frame_vector_pages(g2d_userptr->vec), + g2d_userptr->pages, npages, offset, size, GFP_KERNEL); if (ret < 0) { DRM_DEV_ERROR(g2d->dev, "failed to get sgt from pages.\n"); @@ -538,11 +531,11 @@ static dma_addr_t *g2d_userptr_get_dma_addr(struct g2d_data *g2d, err_free_sgt: kfree(sgt); -err_put_framevec: - put_vaddr_frames(g2d_userptr->vec); +err_unpin_pages: + unpin_user_pages(g2d_userptr->pages, npages); -err_destroy_framevec: - frame_vector_destroy(g2d_userptr->vec); +err_destroy_pages: + kvfree(g2d_userptr->pages); err_free: kfree(g2d_userptr); From patchwork Fri Oct 23 12:21:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852943 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E477616BC for ; Fri, 23 Oct 2020 12:23:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 93C58221F9 for ; Fri, 23 Oct 2020 12:23:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="HLKP8/sU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93C58221F9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 313E16B006E; Fri, 23 Oct 2020 08:23:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 27C396B0070; Fri, 23 Oct 2020 08:23:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EC136B0071; Fri, 23 Oct 2020 08:23:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id CF3EF6B006E for ; Fri, 23 Oct 2020 08:22:59 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 68300181AC9CC for ; Fri, 23 Oct 2020 12:22:59 +0000 (UTC) X-FDA: 77403104478.27.shelf54_111210627259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 42AB93D66B for ; Fri, 23 Oct 2020 12:22:59 +0000 (UTC) X-Spam-Summary: 1,0,0,4c7d519167449f0b,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:2919:3138:3139:3140:3141:3142:3352:3865:3871:3872:4385:5007:6261:6653:6742:10004:10400:11026:11473:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13069:13161:13229:13311:13357:13846:13894:14096:14097:14181:14394:14659:14721:21080:21251:21444:21451:21627:30054:30055:30064:30090,0,RBL:209.85.128.67:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04yf37uamomnqwckhyuq1jnn3q74fyp4qq1ashnry3t6p5dkrmk3gfme9f7bzxu.im8zaxgg4a997skdohecskk58xfa13yrucn4w1oy9n1wqwe5uh3suc4pzfrwq7u.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: shelf54_111210627259 X-Filterd-Recvd-Size: 5098 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:58 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id k18so1287340wmj.5 for ; Fri, 23 Oct 2020 05:22:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VujIMeu4s9glQHD7csuDCyNoNKl6Zu7MMMJFOASMnXs=; b=HLKP8/sUzUrEFqrvdfp4nIWF3Fj3rdBIXl3euidd+sV2PDSfKG6erI11swR+hCO22K SrU2t7SID5EyIcOumwAw8pvvF8+cj7hxL+5Xc9w7THMCq1a4kc7ovtRIlSLFE6Z6wvW9 xJUHAA5lVFAEQmCV4toX9HY0sQbKAmWG/AZ00= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VujIMeu4s9glQHD7csuDCyNoNKl6Zu7MMMJFOASMnXs=; b=QTLGpVQexqnvkTU6/XykjWEmeIj2plhaL1n4W6iLTFRwiH9w+RIf40Yv9/80a37/Jc SuX4HsXHoWl+Wtl6c45R4XIQC6ECGcBrF7zirD6NK6nwDpLpftORJNJmgV/NFbeLrKtm bO3WgJ/fYsKn9HHbQTaY3jn4KGdABYui/C3bcMy24FeSxCqNQW5CQNstXV9Cy4/+OzQj KB9qzKiXdT8PVrVwJZZvl1v6L5NnfZpdpOL4OyKUXVdqqO5SUB3c5Bz4fjH5v7bGQOP8 fbe4RDefzrELOG7lmQ704t0pVS3+CWdOWDUZLfYUSM4Hv5ByyFWGOcogEekxAqdxvMf7 +fuQ== X-Gm-Message-State: AOAM531AJ9QZjJnKezRmhDQ8Q4ieI50UGNlFrMUWCuZRCz+TgOsSG++c E6VTckYyeEo+PXNVYwGcPyRlkw== X-Google-Smtp-Source: ABdhPJzooQe5/oLZHxZZgmc8taLUFT3g7cbiBkIxYHgxSDDcYbP1QZs6l4Y9xlT/uD3YRe4MsGFoyA== X-Received: by 2002:a1c:b389:: with SMTP id c131mr2156367wmf.73.1603455777607; Fri, 23 Oct 2020 05:22:57 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:57 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Inki Dae , Joonyoung Shim , Seung-Woo Kim , Kyungmin Park , Kukjin Kim , Krzysztof Kozlowski , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 31/65] drm/exynos: Use FOLL_LONGTERM for g2d cmdlists Date: Fri, 23 Oct 2020 14:21:42 +0200 Message-Id: <20201023122216.2373294-31-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The exynos g2d interface is very unusual, but it looks like the userptr objects are persistent. Hence they need FOLL_LONGTERM. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Inki Dae Cc: Joonyoung Shim Cc: Seung-Woo Kim Cc: Kyungmin Park Cc: Kukjin Kim Cc: Krzysztof Kozlowski Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Signed-off-by: Daniel Vetter --- drivers/gpu/drm/exynos/exynos_drm_g2d.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/exynos/exynos_drm_g2d.c b/drivers/gpu/drm/exynos/exynos_drm_g2d.c index ecede41af9b9..1e0c5a7f206e 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_g2d.c +++ b/drivers/gpu/drm/exynos/exynos_drm_g2d.c @@ -475,7 +475,8 @@ static dma_addr_t *g2d_userptr_get_dma_addr(struct g2d_data *g2d, goto err_free; } - ret = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE, + ret = pin_user_pages_fast(start, npages, + FOLL_FORCE | FOLL_WRITE | FOLL_LONGTERM, g2d_userptr->pages); if (ret != npages) { DRM_DEV_ERROR(g2d->dev, From patchwork Fri Oct 23 12:21:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852951 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2227B1580 for ; Fri, 23 Oct 2020 12:23:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B000D24641 for ; Fri, 23 Oct 2020 12:23:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="LYdtpI+a" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B000D24641 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ADBEA6B0070; Fri, 23 Oct 2020 08:23:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A62046B0071; Fri, 23 Oct 2020 08:23:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E2B06B0072; Fri, 23 Oct 2020 08:23:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id 4D5596B0070 for ; Fri, 23 Oct 2020 08:23:01 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DD7F9181AC9CC for ; Fri, 23 Oct 2020 12:23:00 +0000 (UTC) X-FDA: 77403104520.28.scale81_3306d9727259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id C02876C05 for ; Fri, 23 Oct 2020 12:23:00 +0000 (UTC) X-Spam-Summary: 1,0,0,cd565c0771ef6f6c,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:2:41:69:152:355:379:541:800:960:966:968:973:981:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1593:1594:1605:1606:1730:1747:1777:1792:2194:2196:2199:2200:2393:2553:2559:2562:2892:2895:3138:3139:3140:3141:3142:3865:3866:3867:3870:3872:3874:4120:4250:4321:4385:4605:5007:6119:6261:6653:6742:7514:7875:7903:8603:9010:9592:10004:11026:11233:11473:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13846:13894:14394:14659:21080:21444:21451:21627:21740:21795:21990:30051:30054:30055:30064:30070:30075:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04y8kx1ujbjw6779a9ekch84t3nwsopqkhi95nhwgi354yd356onz77gehwohxa.yfcaxi8ewnax7xibafk1zy6pk5uqzupzgpmp3tyezdi4iqo8tn4rcfxphd3qi8i.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_ru les:0:0: X-HE-Tag: scale81_3306d9727259 X-Filterd-Recvd-Size: 9117 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:22:59 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id i1so1602557wro.1 for ; Fri, 23 Oct 2020 05:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gmqIPIW7gzq47FEKtDbSWASgt80RL8xNX11AC8WH7PY=; b=LYdtpI+apmqLLojb4ZRznc2h7zEGrjYo0kiHbPMbjYXTyXBt0NUjmyOsiywqpaG/s7 Cy6xi2FNMSJPeBshdxr6tV9DvH7OUgVOIZF6i5xz+WGJdPjj5Wmv9+u/HMnuj0j54dQV hyGDyfysSef8S5DeX6pkx3fBFXC4QHoZhmshw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gmqIPIW7gzq47FEKtDbSWASgt80RL8xNX11AC8WH7PY=; b=jG3zscpS8kHpHDfFGY4mljfbGMHHs8W6PgJWQ4jpbMSH40ysg6oAm4JEdjGPuW+wGK RdUlU0TtpYupuLxX1/JhpzHJKL1H35RzO7LQdXYur1KtnbgG7zP5EveFb98Iq481Wing qrXVNfP6uyhGCDN4Tsz3NfzZ9UGdEJVGwLo9pejfh4C6JrHC/38QkwvE+5zorCWZ0uuM 4sF/VD6WH8czDQsoF8lU4tkQKkGdOzL8DbaPUnjQO86r4TIT9/0QHqUwbEKpy6jljBJ7 r+5K65D5LTel7OXiyuQiWOwKaX6FOkBhLmM30Fpgkpw87QRbMb2h4OLzW0ybrTgJ+Ge9 KwqA== X-Gm-Message-State: AOAM53262YcvZoW0u08+MrfOG86YDUgqJivPy6leH8T1QAvM4tzjukbm lBZsSY1G7BO/vBEt/87kMVUldQ== X-Google-Smtp-Source: ABdhPJwAvy6PTF3ck+c1/RE/B9F9j73Zp1tpK8EzOO/oIow6Yd/I7IeTXqp84vUzY1bxGcWAx70jgA== X-Received: by 2002:adf:e4c4:: with SMTP id v4mr2299916wrm.79.1603455779015; Fri, 23 Oct 2020 05:22:59 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:58 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , John Hubbard , Daniel Vetter , Jason Gunthorpe , Andrew Morton , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Oded Gabbay , Omer Shpigelman , Ofir Bitton , Tomer Tayar , Moti Haimovski , Greg Kroah-Hartman , Pawel Piskorski Subject: [PATCH 32/65] misc/habana: Stop using frame_vector helpers Date: Fri, 23 Oct 2020 14:21:43 +0200 Message-Id: <20201023122216.2373294-32-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames. Reviewed-by: John Hubbard Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Oded Gabbay Cc: Omer Shpigelman Cc: Ofir Bitton Cc: Tomer Tayar Cc: Moti Haimovski Cc: Daniel Vetter Cc: Greg Kroah-Hartman Cc: Pawel Piskorski Signed-off-by: Daniel Vetter --- v2: Use unpin_user_pages_dirty_lock (John) v3: Update kerneldoc (Oded) --- drivers/misc/habanalabs/Kconfig | 1 - drivers/misc/habanalabs/common/habanalabs.h | 6 ++- drivers/misc/habanalabs/common/memory.c | 49 ++++++++------------- 3 files changed, 22 insertions(+), 34 deletions(-) diff --git a/drivers/misc/habanalabs/Kconfig b/drivers/misc/habanalabs/Kconfig index 8eb5d38c618e..2f04187f7167 100644 --- a/drivers/misc/habanalabs/Kconfig +++ b/drivers/misc/habanalabs/Kconfig @@ -6,7 +6,6 @@ config HABANA_AI tristate "HabanaAI accelerators (habanalabs)" depends on PCI && HAS_IOMEM - select FRAME_VECTOR select DMA_SHARED_BUFFER select GENERIC_ALLOCATOR select HWMON diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index edbd627b29d2..41af090b3e6a 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -870,7 +870,8 @@ struct hl_ctx_mgr { * struct hl_userptr - memory mapping chunk information * @vm_type: type of the VM. * @job_node: linked-list node for hanging the object on the Job's list. - * @vec: pointer to the frame vector. + * @pages: pointer to struct page array + * @npages: size of @pages array * @sgt: pointer to the scatter-gather table that holds the pages. * @dir: for DMA unmapping, the direction must be supplied, so save it. * @debugfs_list: node in debugfs list of command submissions. @@ -881,7 +882,8 @@ struct hl_ctx_mgr { struct hl_userptr { enum vm_type_t vm_type; /* must be first */ struct list_head job_node; - struct frame_vector *vec; + struct page **pages; + unsigned int npages; struct sg_table *sgt; enum dma_data_direction dir; struct list_head debugfs_list; diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c index 5ff4688683fd..327b64479f97 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/misc/habanalabs/common/memory.c @@ -1281,45 +1281,41 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size, return -EFAULT; } - userptr->vec = frame_vector_create(npages); - if (!userptr->vec) { + userptr->pages = kvmalloc_array(npages, sizeof(*userptr->pages), + GFP_KERNEL); + if (!userptr->pages) { dev_err(hdev->dev, "Failed to create frame vector\n"); return -ENOMEM; } - rc = get_vaddr_frames(start, npages, FOLL_FORCE | FOLL_WRITE, - userptr->vec); + rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE, + userptr->pages); if (rc != npages) { dev_err(hdev->dev, "Failed to map host memory, user ptr probably wrong\n"); if (rc < 0) - goto destroy_framevec; + goto destroy_pages; + npages = rc; rc = -EFAULT; - goto put_framevec; - } - - if (frame_vector_to_pages(userptr->vec) < 0) { - dev_err(hdev->dev, - "Failed to translate frame vector to pages\n"); - rc = -EFAULT; - goto put_framevec; + goto put_pages; } + userptr->npages = npages; rc = sg_alloc_table_from_pages(userptr->sgt, - frame_vector_pages(userptr->vec), - npages, offset, size, GFP_ATOMIC); + userptr->pages, + npages, offset, size, GFP_ATOMIC); if (rc < 0) { dev_err(hdev->dev, "failed to create SG table from pages\n"); - goto put_framevec; + goto put_pages; } return 0; -put_framevec: - put_vaddr_frames(userptr->vec); -destroy_framevec: - frame_vector_destroy(userptr->vec); +put_pages: + unpin_user_pages(userptr->pages, npages); +destroy_pages: + kvfree(userptr->pages); return rc; } @@ -1405,8 +1401,6 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u64 size, */ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) { - struct page **pages; - hl_debugfs_remove_userptr(hdev, userptr); if (userptr->dma_mapped) @@ -1414,15 +1408,8 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) userptr->sgt->nents, userptr->dir); - pages = frame_vector_pages(userptr->vec); - if (!IS_ERR(pages)) { - int i; - - for (i = 0; i < frame_vector_count(userptr->vec); i++) - set_page_dirty_lock(pages[i]); - } - put_vaddr_frames(userptr->vec); - frame_vector_destroy(userptr->vec); + unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true); + kvfree(userptr->pages); list_del(&userptr->job_node); From patchwork Fri Oct 23 12:21:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852961 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D2EF1580 for ; Fri, 23 Oct 2020 12:23:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA39A221F9 for ; Fri, 23 Oct 2020 12:23:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="I4RFwd2e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA39A221F9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B26326B0071; Fri, 23 Oct 2020 08:23:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB44A6B0072; Fri, 23 Oct 2020 08:23:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92A876B0073; Fri, 23 Oct 2020 08:23:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 6094B6B0071 for ; Fri, 23 Oct 2020 08:23:02 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F18018565E4B for ; Fri, 23 Oct 2020 12:23:01 +0000 (UTC) X-FDA: 77403104562.16.idea08_18009d327259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id CE92C1016458D for ; Fri, 23 Oct 2020 12:23:01 +0000 (UTC) X-Spam-Summary: 1,0,0,822dff460899fae2,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:4321:5007:6261:6653:6742:7514:8603:10004:10400:11026:11657:11658:11914:12043:12296:12297:12517:12519:12555:12895:12986:13069:13161:13229:13311:13357:13846:13894:14096:14097:14181:14394:14659:14721:21080:21251:21444:21451:21627:21990:30054:30055:30064:30090,0,RBL:209.85.128.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yrebsyxpezx3kyffq53gj3dhz1xypjyabm1pa4y4abm39ar6x4yhtyzmszam4.4i6fuecf1bpak4kby9rp15yagffkqfacsm5tfq7ejc84nwr7paw6gpcuo38kzzg.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: idea08_18009d327259 X-Filterd-Recvd-Size: 5142 Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:01 +0000 (UTC) Received: by mail-wm1-f66.google.com with SMTP id q5so1306158wmq.0 for ; Fri, 23 Oct 2020 05:23:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fm+L1OZ5vnKeOGH1XwjpH65CAXQE2qGCwQHeV0BfCbo=; b=I4RFwd2eW9iNL5qH7GC2GN+bOEbVbW0io2ha60AXzDjB/0yYw4W1JK6yrbHeRAL0iY ISW+gqIaC+vP2a2lhS4YPj3Em5r0tFkHvuJ7lEUT60inNY0XgrlVSxFy5WXKshps7csM du6jwmFk8hHJwEqDf96AP/6FrBQrEHzL9do1E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fm+L1OZ5vnKeOGH1XwjpH65CAXQE2qGCwQHeV0BfCbo=; b=U1PBdMtqir37k2tP+suKu+5ms2+YHJrOWwTsj2SctLAzCHpJV8w5iSUJK5F78NxDoW PN5ggdm6cLu4XqpgKZB8zdc1Q+xq9fFc5e1sVAW63mSd0ltFXCU/eEyDCVvpYqd4JCUM +CiXAjf/EBY2ZHugs62eJcGxxIpGMrE2MvpXvHUUeY5hzV0+nMcXuLvnsebaRUoaO5Bc b8tWvgxFSXwEImjSwuf9av6UHv6D22+zrAsU87a5r6Zzj73y1yuREAXKztLs2A6W8X9R qAv7whv9Wx6jPbjjJC65Ypwh01HHRgKlMSxViwlD6NCPwLK4/MQkzfL/raZj09UnIXF8 o/gw== X-Gm-Message-State: AOAM531M0J5xE9ElHk/wn5rbOZEYtHQeKAUXg+CX8m3d9NPEoJfJ2LUl lFmmNdhlH2+TI5FuOQOBcxqqOQ== X-Google-Smtp-Source: ABdhPJyGjziGYIZ7LCrUaPvnmjitFLFb8BHzU6RIkZE3cezG0NmOKu92ESUHVoVpHKIYbtJBpxbBNA== X-Received: by 2002:a1c:7409:: with SMTP id p9mr1939536wmc.167.1603455780340; Fri, 23 Oct 2020 05:23:00 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.22.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:22:59 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Oded Gabbay , Omer Shpigelman , Ofir Bitton , Tomer Tayar , Moti Haimovski , Greg Kroah-Hartman , Pawel Piskorski Subject: [PATCH 33/65] misc/habana: Use FOLL_LONGTERM for userptr Date: Fri, 23 Oct 2020 14:21:44 +0200 Message-Id: <20201023122216.2373294-33-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: These are persistent, not just for the duration of a dma operation. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Oded Gabbay Cc: Omer Shpigelman Cc: Ofir Bitton Cc: Tomer Tayar Cc: Moti Haimovski Cc: Daniel Vetter Cc: Greg Kroah-Hartman Cc: Pawel Piskorski Signed-off-by: Daniel Vetter --- drivers/misc/habanalabs/common/memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/misc/habanalabs/common/memory.c b/drivers/misc/habanalabs/common/memory.c index 327b64479f97..767d3644c033 100644 --- a/drivers/misc/habanalabs/common/memory.c +++ b/drivers/misc/habanalabs/common/memory.c @@ -1288,7 +1288,8 @@ static int get_user_memory(struct hl_device *hdev, u64 addr, u64 size, return -ENOMEM; } - rc = pin_user_pages_fast(start, npages, FOLL_FORCE | FOLL_WRITE, + rc = pin_user_pages_fast(start, npages, + FOLL_FORCE | FOLL_WRITE | FOLL_LONGTERM, userptr->pages); if (rc != npages) { From patchwork Fri Oct 23 12:21:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852971 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 419271580 for ; Fri, 23 Oct 2020 12:23:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E455924182 for ; Fri, 23 Oct 2020 12:23:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="AEIa/7Rt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E455924182 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6EA206B0072; Fri, 23 Oct 2020 08:23:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5F6A56B0073; Fri, 23 Oct 2020 08:23:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 422716B0074; Fri, 23 Oct 2020 08:23:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id EAB116B0072 for ; Fri, 23 Oct 2020 08:23:03 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 80EB5181AC9CC for ; Fri, 23 Oct 2020 12:23:03 +0000 (UTC) X-FDA: 77403104646.18.sail58_4e1192227259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 585BA101E3F51 for ; Fri, 23 Oct 2020 12:23:03 +0000 (UTC) X-Spam-Summary: 1,0,0,c428a0de5957f9f5,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:69:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:4118:4250:4321:4385:5007:6261:6653:6742:7903:9592:10004:10400:11026:11658:11914:12043:12294:12296:12297:12438:12517:12519:12555:12683:12895:12986:13846:13894:14181:14394:14659:14721:21080:21251:21324:21444:21451:21627:21740:21990:30012:30054:30055:30064:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04y8b3jacdddntnp9encjanffdbawopayq4q5uj4xos4z58cy4b5a19xzaau59p.esiui85oo8xshohogknh8qxome4kyy68ggyw9fcefx4gyd4hfpnhw199atydhsb.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: sail58_4e1192227259 X-Filterd-Recvd-Size: 7337 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:02 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id j7so1563248wrt.9 for ; Fri, 23 Oct 2020 05:23:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cXs1UvblKDwz1YQCIe37x2wwQ5T1EiKP8bli0y6YQUo=; b=AEIa/7RtMPcTkOS17xaoZ+tZ2WHJwDB/uLw+Jxwkknjzz0o8Pvb3jK8xNmI5+rkacH RvXPWyHDVHuGWcl22HeQn+NbqKDCOAf9Q9GDKtS71H17YcqVr/FxJsbfr3QL5ROHJrug tpVEoVnRUaDsKK3bbvDbpWdN2Mh/1JfyJptkk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cXs1UvblKDwz1YQCIe37x2wwQ5T1EiKP8bli0y6YQUo=; b=cDtYENIPfJ1fxI8M6jjElf174Ui3a9NV6FKqIIG0TLBX8bAZ2RpOB945NCTPIrAarK HjZ/1ZGfHVtse7ixrrpQJno4kATHNjFE6EB6THY+K3t0jms/Exvn2mCbltgUO3VODyyo 37ilYqn9dbul8v6MAATNSVAwqbhufjjLdQW5F8tBVXfz7+/CCWD69Tms8i93NY+vn9vg li1j1qWgFwp8Drdjgs4TvNpyCMai5oUSlt4+JNaq3Zc6d/JfuHUuDb0nR0bV2K2eH17Y dIWCTwKJcxg4D6icvjaH5ur0ISGcH5YBB9tTWYaT1iCETgBrhK/HTBFuIR9Q+YulFoGd 1OGw== X-Gm-Message-State: AOAM530oNshNl3ZHYz4VZ0aoTs3EmPqp74UxdjeKD/W8qFWtJYbe6fM+ EYswKUkKJW5Tki/YMPlzSnBifA== X-Google-Smtp-Source: ABdhPJwugsOja41agH4JeM0mB6q18uXcawtusiYBwFAiX2c/bgyl3tIOiZ8ZGxA9kMQgDSsksw1WPw== X-Received: by 2002:adf:fd8a:: with SMTP id d10mr2236246wrr.85.1603455781768; Fri, 23 Oct 2020 05:23:01 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:01 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Pawel Osciak , Marek Szyprowski , Kyungmin Park , Tomasz Figa , Mauro Carvalho Chehab , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 34/65] mm/frame-vector: Use FOLL_LONGTERM Date: Fri, 23 Oct 2020 14:21:45 +0200 Message-Id: <20201023122216.2373294-34-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is used by media/videbuf2 for persistent dma mappings, not just for a single dma operation and then freed again, so needs FOLL_LONGTERM. Unfortunately current pup_locked doesn't support FOLL_LONGTERM due to locking issues. Rework the code to pull the pup path out from the mmap_sem critical section as suggested by Jason. By relying entirely on the vma checks in pin_user_pages and follow_pfn (for vm_flags and vma_is_fsdax) we can also streamline the code a lot. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Pawel Osciak Cc: Marek Szyprowski Cc: Kyungmin Park Cc: Tomasz Figa Cc: Mauro Carvalho Chehab Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Signed-off-by: Daniel Vetter --- v2: Streamline the code and further simplify the loop checks (Jason) --- mm/frame_vector.c | 50 ++++++++++++++--------------------------------- 1 file changed, 15 insertions(+), 35 deletions(-) diff --git a/mm/frame_vector.c b/mm/frame_vector.c index 10f82d5643b6..d44779e56313 100644 --- a/mm/frame_vector.c +++ b/mm/frame_vector.c @@ -38,7 +38,6 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, struct vm_area_struct *vma; int ret = 0; int err; - int locked; if (nr_frames == 0) return 0; @@ -48,40 +47,25 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, start = untagged_addr(start); - mmap_read_lock(mm); - locked = 1; - vma = find_vma_intersection(mm, start, start + 1); - if (!vma) { - ret = -EFAULT; - goto out; - } - - /* - * While get_vaddr_frames() could be used for transient (kernel - * controlled lifetime) pinning of memory pages all current - * users establish long term (userspace controlled lifetime) - * page pinning. Treat get_vaddr_frames() like - * get_user_pages_longterm() and disallow it for filesystem-dax - * mappings. - */ - if (vma_is_fsdax(vma)) { - ret = -EOPNOTSUPP; - goto out; - } - - if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) { + ret = pin_user_pages_fast(start, nr_frames, + FOLL_FORCE | FOLL_WRITE | FOLL_LONGTERM, + (struct page **)(vec->ptrs)); + if (ret > 0) { vec->got_ref = true; vec->is_pfns = false; - ret = pin_user_pages_locked(start, nr_frames, - gup_flags, (struct page **)(vec->ptrs), &locked); - goto out; + goto out_unlocked; } + mmap_read_lock(mm); vec->got_ref = false; vec->is_pfns = true; do { unsigned long *nums = frame_vector_pfns(vec); + vma = find_vma_intersection(mm, start, start + 1); + if (!vma) + break; + while (ret < nr_frames && start + PAGE_SIZE <= vma->vm_end) { err = follow_pfn(vma, start, &nums[ret]); if (err) { @@ -92,17 +76,13 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, start += PAGE_SIZE; ret++; } - /* - * We stop if we have enough pages or if VMA doesn't completely - * cover the tail page. - */ - if (ret >= nr_frames || start < vma->vm_end) + /* Bail out if VMA doesn't completely cover the tail page. */ + if (start < vma->vm_end) break; - vma = find_vma_intersection(mm, start, start + 1); - } while (vma && vma->vm_flags & (VM_IO | VM_PFNMAP)); + } while (ret < nr_frames); out: - if (locked) - mmap_read_unlock(mm); + mmap_read_unlock(mm); +out_unlocked: if (!ret) ret = -EFAULT; if (ret > 0) From patchwork Fri Oct 23 12:21:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852973 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D2AA92C for ; Fri, 23 Oct 2020 12:23:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 368ED24631 for ; Fri, 23 Oct 2020 12:23:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="GWMzMlOo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 368ED24631 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B2D536B0073; Fri, 23 Oct 2020 08:23:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A8CE86B0075; Fri, 23 Oct 2020 08:23:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88BA96B0078; Fri, 23 Oct 2020 08:23:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0233.hostedemail.com [216.40.44.233]) by kanga.kvack.org (Postfix) with ESMTP id 4BE026B0073 for ; Fri, 23 Oct 2020 08:23:05 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E1D588565E33 for ; Fri, 23 Oct 2020 12:23:04 +0000 (UTC) X-FDA: 77403104688.05.bee39_401631527259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id C721518001721 for ; Fri, 23 Oct 2020 12:23:04 +0000 (UTC) X-Spam-Summary: 1,0,0,402fe1091a1d02ee,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:1:2:41:69:152:334:355:368:369:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:1801:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3280:3865:3866:3867:3870:3871:3872:3873:3874:4051:4321:4605:5007:6117:6261:6609:6653:6742:7875:7903:9592:10004:11026:11232:11473:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13846:13894:14096:14097:14394:14659:21080:21092:21444:21450:21451:21627:21773:21795:21990:30003:30034:30051:30054:30055:30064:30089:30090,0,RBL:209.85.221.68:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yrymzoj9birguhqy8zqrafdkqqkocp7c44ut3cspe3r1pnndpimh9dc6k8ujw.e5jk4bxg3h731c9ji5pyg539r4fwwntnwgh867iaht8wxn9wsr8b7nb6hjq8st7.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules :0:0:0,L X-HE-Tag: bee39_401631527259 X-Filterd-Recvd-Size: 12174 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:04 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id n6so1554303wrm.13 for ; Fri, 23 Oct 2020 05:23:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6WV52rOTICG5gX8Y1zv7jCBKQNXPiDy4YHeczZNq5R8=; b=GWMzMlOoM30W5hlxnVdv3gz53647+4fdHXYymDsOfzhTNsk0z0MfJMjx4w6nZ94KSK sfbyQuNDRBSJ7z7dwGyDhR7Ol6tkx4xZWRKmmTirMfsl0A+bAFoETnuyiBBVhYpf0FVO 4tjClYte6Izd5GcvbhbzB5NvdIfaex+qSjX5g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6WV52rOTICG5gX8Y1zv7jCBKQNXPiDy4YHeczZNq5R8=; b=doh5RTMxlP0kiEThkuBqCLfJghOceskkk1aqBn6v077TI30+D9ySNGQYYD8GAXJRVd 37Anb+APp3Oyl/eoYwmN0MmWTnsKFrV/btEdIRcPcmukperAdT+ffa26HV/NHaEyDgOr T+Ja/AWNkAGi1mAhYivh25NJ8qneMsn5GTSv6wNV2e5pRm46fowL9274R2s2S30+2pME VUr6AyuKCZ5U7Cs0RRDZ6Na+Sktz3rUo0LW4CoaKAqqAzGQX2qiOWaxrLeKy6QSMto02 hXK6hOi/MCLQ6c0zzyWB1y9RHSJZEsMMEFSOmTkkvGKgh6EICeN10gqLZDNJScsd38ZP SFxw== X-Gm-Message-State: AOAM530zl+xkG+dJACHRCLE5J8tSHZa3QbG7ydIqBTFnQsQ/aHqXgi8k QwPTxZnAAOEtcw8Vd+gKSmoBrA== X-Google-Smtp-Source: ABdhPJyf0rFBdStZWr3ETjOPszhGAdp3As8uqEIf5/xUW6NvJZpiJwjwaGUkv0kEBGdr8o9LmQTEEg== X-Received: by 2002:adf:e549:: with SMTP id z9mr2523394wrm.12.1603455783113; Fri, 23 Oct 2020 05:23:03 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:02 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , John Hubbard , Mauro Carvalho Chehab , Daniel Vetter , Jason Gunthorpe , Pawel Osciak , Marek Szyprowski , Kyungmin Park , Tomasz Figa , Mauro Carvalho Chehab , Andrew Morton , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 35/65] media: videobuf2: Move frame_vector into media subsystem Date: Fri, 23 Oct 2020 14:21:46 +0200 Message-Id: <20201023122216.2373294-35-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's the only user. This also garbage collects the CONFIG_FRAME_VECTOR symbol from all over the tree (well just one place, somehow omap media driver still had this in its Kconfig, despite not using it). Reviewed-by: John Hubbard Acked-by: Mauro Carvalho Chehab Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Pawel Osciak Cc: Marek Szyprowski Cc: Kyungmin Park Cc: Tomasz Figa Cc: Mauro Carvalho Chehab Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Daniel Vetter Signed-off-by: Daniel Vetter --- v3: - Create a new frame_vector.h header for this (Mauro) --- drivers/media/common/videobuf2/Kconfig | 1 - drivers/media/common/videobuf2/Makefile | 1 + .../media/common/videobuf2}/frame_vector.c | 2 + drivers/media/platform/omap/Kconfig | 1 - include/linux/mm.h | 42 ----------------- include/media/frame_vector.h | 47 +++++++++++++++++++ include/media/videobuf2-core.h | 1 + mm/Kconfig | 3 -- mm/Makefile | 1 - 9 files changed, 51 insertions(+), 48 deletions(-) rename {mm => drivers/media/common/videobuf2}/frame_vector.c (99%) create mode 100644 include/media/frame_vector.h diff --git a/drivers/media/common/videobuf2/Kconfig b/drivers/media/common/videobuf2/Kconfig index edbc99ebba87..d2223a12c95f 100644 --- a/drivers/media/common/videobuf2/Kconfig +++ b/drivers/media/common/videobuf2/Kconfig @@ -9,7 +9,6 @@ config VIDEOBUF2_V4L2 config VIDEOBUF2_MEMOPS tristate - select FRAME_VECTOR config VIDEOBUF2_DMA_CONTIG tristate diff --git a/drivers/media/common/videobuf2/Makefile b/drivers/media/common/videobuf2/Makefile index 77bebe8b202f..54306f8d096c 100644 --- a/drivers/media/common/videobuf2/Makefile +++ b/drivers/media/common/videobuf2/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 videobuf2-common-objs := videobuf2-core.o +videobuf2-common-objs += frame_vector.o ifeq ($(CONFIG_TRACEPOINTS),y) videobuf2-common-objs += vb2-trace.o diff --git a/mm/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c similarity index 99% rename from mm/frame_vector.c rename to drivers/media/common/videobuf2/frame_vector.c index d44779e56313..6590987c14bd 100644 --- a/mm/frame_vector.c +++ b/drivers/media/common/videobuf2/frame_vector.c @@ -8,6 +8,8 @@ #include #include +#include + /** * get_vaddr_frames() - map virtual addresses to pfns * @start: starting user address diff --git a/drivers/media/platform/omap/Kconfig b/drivers/media/platform/omap/Kconfig index f73b5893220d..de16de46c0f4 100644 --- a/drivers/media/platform/omap/Kconfig +++ b/drivers/media/platform/omap/Kconfig @@ -12,6 +12,5 @@ config VIDEO_OMAP2_VOUT depends on VIDEO_V4L2 select VIDEOBUF2_DMA_CONTIG select OMAP2_VRFB if ARCH_OMAP2 || ARCH_OMAP3 - select FRAME_VECTOR help V4L2 Display driver support for OMAP2/3 based boards. diff --git a/include/linux/mm.h b/include/linux/mm.h index 16b799a0522c..acd60fbf1a5a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1743,48 +1743,6 @@ int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc); int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, struct task_struct *task, bool bypass_rlim); -/* Container for pinned pfns / pages */ -struct frame_vector { - unsigned int nr_allocated; /* Number of frames we have space for */ - unsigned int nr_frames; /* Number of frames stored in ptrs array */ - bool got_ref; /* Did we pin pages by getting page ref? */ - bool is_pfns; /* Does array contain pages or pfns? */ - void *ptrs[]; /* Array of pinned pfns / pages. Use - * pfns_vector_pages() or pfns_vector_pfns() - * for access */ -}; - -struct frame_vector *frame_vector_create(unsigned int nr_frames); -void frame_vector_destroy(struct frame_vector *vec); -int get_vaddr_frames(unsigned long start, unsigned int nr_pfns, - unsigned int gup_flags, struct frame_vector *vec); -void put_vaddr_frames(struct frame_vector *vec); -int frame_vector_to_pages(struct frame_vector *vec); -void frame_vector_to_pfns(struct frame_vector *vec); - -static inline unsigned int frame_vector_count(struct frame_vector *vec) -{ - return vec->nr_frames; -} - -static inline struct page **frame_vector_pages(struct frame_vector *vec) -{ - if (vec->is_pfns) { - int err = frame_vector_to_pages(vec); - - if (err) - return ERR_PTR(err); - } - return (struct page **)(vec->ptrs); -} - -static inline unsigned long *frame_vector_pfns(struct frame_vector *vec) -{ - if (!vec->is_pfns) - frame_vector_to_pfns(vec); - return (unsigned long *)(vec->ptrs); -} - struct kvec; int get_kernel_pages(const struct kvec *iov, int nr_pages, int write, struct page **pages); diff --git a/include/media/frame_vector.h b/include/media/frame_vector.h new file mode 100644 index 000000000000..1ed0cd64510d --- /dev/null +++ b/include/media/frame_vector.h @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef _MEDIA_FRAME_VECTOR_H +#define _MEDIA_FRAME_VECTOR_H + +/* Container for pinned pfns / pages in frame_vector.c */ +struct frame_vector { + unsigned int nr_allocated; /* Number of frames we have space for */ + unsigned int nr_frames; /* Number of frames stored in ptrs array */ + bool got_ref; /* Did we pin pages by getting page ref? */ + bool is_pfns; /* Does array contain pages or pfns? */ + void *ptrs[]; /* Array of pinned pfns / pages. Use + * pfns_vector_pages() or pfns_vector_pfns() + * for access */ +}; + +struct frame_vector *frame_vector_create(unsigned int nr_frames); +void frame_vector_destroy(struct frame_vector *vec); +int get_vaddr_frames(unsigned long start, unsigned int nr_pfns, + unsigned int gup_flags, struct frame_vector *vec); +void put_vaddr_frames(struct frame_vector *vec); +int frame_vector_to_pages(struct frame_vector *vec); +void frame_vector_to_pfns(struct frame_vector *vec); + +static inline unsigned int frame_vector_count(struct frame_vector *vec) +{ + return vec->nr_frames; +} + +static inline struct page **frame_vector_pages(struct frame_vector *vec) +{ + if (vec->is_pfns) { + int err = frame_vector_to_pages(vec); + + if (err) + return ERR_PTR(err); + } + return (struct page **)(vec->ptrs); +} + +static inline unsigned long *frame_vector_pfns(struct frame_vector *vec) +{ + if (!vec->is_pfns) + frame_vector_to_pfns(vec); + return (unsigned long *)(vec->ptrs); +} + +#endif /* _MEDIA_FRAME_VECTOR_H */ diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index bbb3f26fbde9..d045e3a5a1d8 100644 --- a/include/media/videobuf2-core.h +++ b/include/media/videobuf2-core.h @@ -18,6 +18,7 @@ #include #include #include +#include #define VB2_MAX_FRAME (32) #define VB2_MAX_PLANES (8) diff --git a/mm/Kconfig b/mm/Kconfig index 6c974888f86f..da6c943fe9f1 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -815,9 +815,6 @@ config DEVICE_PRIVATE memory; i.e., memory that is only accessible from the device (or group of devices). You likely also want to select HMM_MIRROR. -config FRAME_VECTOR - bool - config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS diff --git a/mm/Makefile b/mm/Makefile index d5649f1c12c0..a025fd6c6afd 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -111,7 +111,6 @@ obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o obj-$(CONFIG_USERFAULTFD) += userfaultfd.o obj-$(CONFIG_IDLE_PAGE_TRACKING) += page_idle.o -obj-$(CONFIG_FRAME_VECTOR) += frame_vector.o obj-$(CONFIG_DEBUG_PAGE_REF) += debug_page_ref.o obj-$(CONFIG_HARDENED_USERCOPY) += usercopy.o obj-$(CONFIG_PERCPU_STATS) += percpu-stats.o From patchwork Fri Oct 23 12:21:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852985 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB1DF92C for ; Fri, 23 Oct 2020 12:23:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6936124650 for ; Fri, 23 Oct 2020 12:23:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="kxcHr9Xa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6936124650 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1149A6B0074; Fri, 23 Oct 2020 08:23:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C8A86B0075; Fri, 23 Oct 2020 08:23:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF6036B0078; Fri, 23 Oct 2020 08:23:06 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id C0BB46B0074 for ; Fri, 23 Oct 2020 08:23:06 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 56354362C for ; Fri, 23 Oct 2020 12:23:06 +0000 (UTC) X-FDA: 77403104772.05.join94_500d8eb27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 38E821800E38C for ; Fri, 23 Oct 2020 12:23:06 +0000 (UTC) X-Spam-Summary: 1,0,0,c53b4172011d50c6,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1544:1593:1594:1605:1711:1730:1747:1777:1792:1981:2194:2198:2199:2200:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4119:4321:4605:5007:6117:6119:6261:6653:6742:7875:7903:7974:10004:11026:11473:11658:11914:12043:12050:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13161:13229:13846:13894:14096:14097:14181:14394:14659:14721:21080:21444:21451:21627:21990:30002:30054:30055:30064:30070:30090,0,RBL:209.85.221.68:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04y8f7mi83cdc47t36ewi8dzyg547ocef5tchd9qp9i36ipe96acph6ow96km84.nu5kz884ehcpo8p8fixoqrffsmuenzyn6d6jr1gbx6qq1bxtby667t5yhftomps.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,C ustom_ru X-HE-Tag: join94_500d8eb27259 X-Filterd-Recvd-Size: 8294 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:05 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id e17so1556979wru.12 for ; Fri, 23 Oct 2020 05:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hPo/Bw6tNC6m8oopvfckb4iPuUPePa6aSZ/PC8jTJPc=; b=kxcHr9XaGgdgUN/6f9tRbwBS2R7y8JekVykqE3uxv65hBKf0VpkNLgGxd8FIvRj750 nO7ickZUiLfE5Y0mI+ot8RAg1bvzM+pU+EnQr+/RtXjGmgjqqxLTca/kFkBVNTJKhQjr XwcE78fbKjpMmELfqkY2vXpXpf6p2WAexMqJ0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hPo/Bw6tNC6m8oopvfckb4iPuUPePa6aSZ/PC8jTJPc=; b=A5y0mwEh0YZyEqQ+1WjBI9i94YeXcHmM3MEi/YjCnSa4Qss60IFkXP+G1d55j5mrcl +m/OdhOCQC0Fuv5lQMNRrPsaRwU7HetAa8xfBhJ7b1nPu4f995ez27tfpPWQYRf3KmOe BTM5dzWID4u3wRgQu3N+43x0RFoKHKA5qyZMuSZ0dfgo75KqVh/T5Dw6TYAH1rVPloMF FVyq4EwhUuacpePp7GBzRk9/6kp+XKAig7WfYbWCMcxKO5H92ZmoK3GysP8Nl3FRtR15 AePsZIF0MM54ZdhmdFCDghMAjRBViarBK7eyLSA6QFk+Mm0OVDZPM/4y8mp5Ydutdfsf WH/w== X-Gm-Message-State: AOAM531k9D6F0MxukWk9uP0XI50QRbl7rapW+Y44+MbdpTt/coiGgiNs GwAlm/laLewScIhSkWIaV0D2Wg== X-Google-Smtp-Source: ABdhPJwnPss/3+FXK73SGyn1ajFye9TH2uvE+XJgRwBdhY+CfV9NESKavPXc5txtoFVhV9XkRhrz0Q== X-Received: by 2002:adf:e8d0:: with SMTP id k16mr2306084wrn.362.1603455784579; Fri, 23 Oct 2020 05:23:04 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:03 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Dan Williams , Kees Cook , Benjamin Herrensmidt , Dave Airlie , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Chris Wilson Subject: [PATCH 36/65] mm: Close race in generic_access_phys Date: Fri, 23 Oct 2020 14:21:47 +0200 Message-Id: <20201023122216.2373294-36-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed: - gpu drivers dynamically manage their memory nowadays, invalidating ptes with unmap_mapping_range when buffers get moved - contiguous dma allocations have moved from dedicated carvetouts to cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE) - even /dev/mem now invalidates mappings when the kernel requests that iomem region when CONFIG_IO_STRICT_DEVMEM is set, see 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea. Fix this. Since ioremap might need to manipulate pagetables too we need to drop the pt lock and have a retry loop if we raced. While at it, also add kerneldoc and improve the comment for the vma_ops->access function. It's for accessing, not for moving the memory from iomem to system memory, as the old comment seemed to suggest. References: 28b2ee20c7cb ("access_process_vm device memory infrastructure") Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Dan Williams Cc: Kees Cook Cc: Benjamin Herrensmidt Cc: Dave Airlie Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Chris Wilson Signed-off-by: Daniel Vetter --- v2: Fix inversion in the retry check (John). v4: While at it, use offset_in_page (Chris Wilson) --- include/linux/mm.h | 3 ++- mm/memory.c | 46 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 45 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index acd60fbf1a5a..2a16631c1fda 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -566,7 +566,8 @@ struct vm_operations_struct { vm_fault_t (*pfn_mkwrite)(struct vm_fault *vmf); /* called by access_process_vm when get_user_pages() fails, typically - * for use by special VMAs that can switch between memory and hardware + * for use by special VMAs. See also generic_access_phys() for a generic + * implementation useful for any iomem mapping. */ int (*access)(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); diff --git a/mm/memory.c b/mm/memory.c index eeae590e526a..1b46eae3b703 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4840,28 +4840,68 @@ int follow_phys(struct vm_area_struct *vma, return ret; } +/** + * generic_access_phys - generic implementation for iomem mmap access + * @vma: the vma to access + * @addr: userspace addres, not relative offset within @vma + * @buf: buffer to read/write + * @len: length of transfer + * @write: set to FOLL_WRITE when writing, otherwise reading + * + * This is a generic implementation for &vm_operations_struct.access for an + * iomem mapping. This callback is used by access_process_vm() when the @vma is + * not page based. + */ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write) { resource_size_t phys_addr; unsigned long prot = 0; void __iomem *maddr; - int offset = addr & (PAGE_SIZE-1); + pte_t *ptep, pte; + spinlock_t *ptl; + int offset = offset_in_page(addr); + int ret = -EINVAL; + + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) + return -EINVAL; + +retry: + if (follow_pte(vma->vm_mm, addr, &ptep, &ptl)) + return -EINVAL; + pte = *ptep; + pte_unmap_unlock(ptep, ptl); - if (follow_phys(vma, addr, write, &prot, &phys_addr)) + prot = pgprot_val(pte_pgprot(pte)); + phys_addr = (resource_size_t)pte_pfn(pte) << PAGE_SHIFT; + + if ((write & FOLL_WRITE) && !pte_write(pte)) return -EINVAL; maddr = ioremap_prot(phys_addr, PAGE_ALIGN(len + offset), prot); if (!maddr) return -ENOMEM; + if (follow_pte(vma->vm_mm, addr, &ptep, &ptl)) + goto out_unmap; + + if (!pte_same(pte, *ptep)) { + pte_unmap_unlock(ptep, ptl); + iounmap(maddr); + + goto retry; + } + if (write) memcpy_toio(maddr + offset, buf, len); else memcpy_fromio(buf, maddr + offset, len); + ret = len; + pte_unmap_unlock(ptep, ptl); +out_unmap: iounmap(maddr); - return len; + return ret; } EXPORT_SYMBOL_GPL(generic_access_phys); #endif From patchwork Fri Oct 23 12:21:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11852993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2640817D4 for ; Fri, 23 Oct 2020 12:23:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C303924641 for ; Fri, 23 Oct 2020 12:23:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="K0cS0ybR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C303924641 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 951CD6B0075; Fri, 23 Oct 2020 08:23:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 903316B0078; Fri, 23 Oct 2020 08:23:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72DFC6B007B; Fri, 23 Oct 2020 08:23:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id 3FE966B0075 for ; Fri, 23 Oct 2020 08:23:08 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D5C14362C for ; Fri, 23 Oct 2020 12:23:07 +0000 (UTC) X-FDA: 77403104814.27.face98_17043aa27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id B19443D669 for ; Fri, 23 Oct 2020 12:23:07 +0000 (UTC) X-Spam-Summary: 1,0,0,ab94e39333ef757b,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:2:41:152:355:379:541:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1593:1594:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2553:2559:2562:2693:2894:2901:2903:3138:3139:3140:3141:3142:3743:3865:3866:3867:3868:3870:3871:3872:3873:3874:4049:4120:4250:4321:4385:4605:5007:6117:6119:6261:6653:6742:7875:7903:8784:9165:10004:11026:11473:11658:11914:12043:12050:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13161:13229:13846:13894:14096:14097:14394:14659:21080:21324:21444:21451:21627:21740:21795:21939:21966:21990:30003:30025:30051:30054:30055:30064:30070:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04y8jcji15jhgjn4hrgk7kteexkydocpk9jqqrm89ubyt746xxgi8i49feodqs6.qtib8zhubx9687qjtuiyxbnir695xbe1z59d3wqzdwfgqeqfa1ep5h1gxt1seda.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0 .5,Netch X-HE-Tag: face98_17043aa27259 X-Filterd-Recvd-Size: 9409 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:07 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id h5so1575429wrv.7 for ; Fri, 23 Oct 2020 05:23:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jw8O4zHShCAt4bi/mdSwE8pqaQ/Ci7nnAmM3jj6tKas=; b=K0cS0ybRn+oowL57X6Nq5A66q23biV6d8Mh7nGNYpraDsikoql8NXlqFHPQVPGQZEq ZhmiEFHW8LcIIsLLB04p9O5em1Gy5Y5hrhfhfjs2YroaC/WrwZiQyslSzwbv6NtWleY1 RjNBNdVMAomvkHW50DBDpEmjI0t0ipkA/M7JE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jw8O4zHShCAt4bi/mdSwE8pqaQ/Ci7nnAmM3jj6tKas=; b=O1trmQFJDZdu/o2rympg1IPM7TMuxzwrPX1oeth4kMeCX45LTMM7XRM3uqECI0t2cX FEF/QMPu+txqhBmk59xgOiGvNgIaUJJZBjBVVUe0GM7HrlHZra2GEfR88JfZJUjHGUB+ K54Yiz2r4T+GkvggCPIYn7BD/y/SbdojqmiGzYrvdDZrRJRIyj5NFKzC/79LjuAxkWe8 AoeQUDGA8k342RaPvFJjV3IRfwldirjTfTK4ZpDUHtP1Z3QP4CtvUDr7eEvwYdAGSAWN T70gfg+8r/OhJGl3rPij3KLFXENN44qgXWlsyevDkZdTLeQCVVDfMv6IADzgOEULGmBe yHsw== X-Gm-Message-State: AOAM533t+Fo7YZ5mnwRL2XTe+2N3fhEouXwVnEnVqyUvWOB7ghDFmK6y ZJV87MwqBIU9QcjxwvV+Otcljw== X-Google-Smtp-Source: ABdhPJyHNrzVnBA/1XZbDotdv4KJ/xMo2zJVR986ilnB2l49QrGJnJFipT6hqx9IUcARiSC9qfVVjQ== X-Received: by 2002:a5d:6551:: with SMTP id z17mr2352471wrv.266.1603455785885; Fri, 23 Oct 2020 05:23:05 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:05 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH 37/65] mm: Add unsafe_follow_pfn Date: Fri, 23 Oct 2020 14:21:48 +0200 Message-Id: <20201023122216.2373294-37-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed: - gpu drivers dynamically manage their memory nowadays, invalidating ptes with unmap_mapping_range when buffers get moved - contiguous dma allocations have moved from dedicated carvetouts to cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE) - even /dev/mem now invalidates mappings when the kernel requests that iomem region when CONFIG_IO_STRICT_DEVMEM is set, see 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea. Unfortunately there's some users where this is not fixable (like v4l userptr of iomem mappings) or involves a pile of work (vfio type1 iommu). For now annotate these as unsafe and splat appropriately. This patch adds an unsafe_follow_pfn, which later patches will then roll out to all appropriate places. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: kvm@vger.kernel.org Signed-off-by: Daniel Vetter --- include/linux/mm.h | 2 ++ mm/memory.c | 32 +++++++++++++++++++++++++++++++- mm/nommu.c | 17 +++++++++++++++++ security/Kconfig | 13 +++++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 2a16631c1fda..ec8c90928fc9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1653,6 +1653,8 @@ int follow_pte_pmd(struct mm_struct *mm, unsigned long address, pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp); int follow_pfn(struct vm_area_struct *vma, unsigned long address, unsigned long *pfn); +int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address, + unsigned long *pfn); int follow_phys(struct vm_area_struct *vma, unsigned long address, unsigned int flags, unsigned long *prot, resource_size_t *phys); int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 1b46eae3b703..9a2ec07ff20b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4788,7 +4788,12 @@ EXPORT_SYMBOL(follow_pte_pmd); * @address: user virtual address * @pfn: location to store found PFN * - * Only IO mappings and raw PFN mappings are allowed. + * Only IO mappings and raw PFN mappings are allowed. Note that callers must + * ensure coherency with pte updates by using a &mmu_notifier to follow updates. + * If this is not feasible, or the access to the @pfn is only very short term, + * use follow_pte_pmd() instead and hold the pagetable lock for the duration of + * the access instead. Any caller not following these requirements must use + * unsafe_follow_pfn() instead. * * Return: zero and the pfn at @pfn on success, -ve otherwise. */ @@ -4811,6 +4816,31 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address, } EXPORT_SYMBOL(follow_pfn); +/** + * unsafe_follow_pfn - look up PFN at a user virtual address + * @vma: memory mapping + * @address: user virtual address + * @pfn: location to store found PFN + * + * Only IO mappings and raw PFN mappings are allowed. + * + * Returns zero and the pfn at @pfn on success, -ve otherwise. + */ +int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address, + unsigned long *pfn) +{ +#ifdef CONFIG_STRICT_FOLLOW_PFN + pr_info("unsafe follow_pfn usage rejected, see CONFIG_STRICT_FOLLOW_PFN\n"); + return -EINVAL; +#else + WARN_ONCE(1, "unsafe follow_pfn usage\n"); + add_taint(TAINT_USER, LOCKDEP_STILL_OK); + + return follow_pfn(vma, address, pfn); +#endif +} +EXPORT_SYMBOL(unsafe_follow_pfn); + #ifdef CONFIG_HAVE_IOREMAP_PROT int follow_phys(struct vm_area_struct *vma, unsigned long address, unsigned int flags, diff --git a/mm/nommu.c b/mm/nommu.c index 75a327149af1..3db2910f0d64 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -132,6 +132,23 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address, } EXPORT_SYMBOL(follow_pfn); +/** + * unsafe_follow_pfn - look up PFN at a user virtual address + * @vma: memory mapping + * @address: user virtual address + * @pfn: location to store found PFN + * + * Only IO mappings and raw PFN mappings are allowed. + * + * Returns zero and the pfn at @pfn on success, -ve otherwise. + */ +int unsafe_follow_pfn(struct vm_area_struct *vma, unsigned long address, + unsigned long *pfn) +{ + return follow_pfn(vma, address, pfn); +} +EXPORT_SYMBOL(unsafe_follow_pfn); + LIST_HEAD(vmap_area_list); void vfree(const void *addr) diff --git a/security/Kconfig b/security/Kconfig index 7561f6f99f1d..48945402e103 100644 --- a/security/Kconfig +++ b/security/Kconfig @@ -230,6 +230,19 @@ config STATIC_USERMODEHELPER_PATH If you wish for all usermode helper programs to be disabled, specify an empty string here (i.e. ""). +config STRICT_FOLLOW_PFN + bool "Disable unsafe use of follow_pfn" + depends on MMU + help + Some functionality in the kernel follows userspace mappings to iomem + ranges in an unsafe matter. Examples include v4l userptr for zero-copy + buffers sharing. + + If this option is switched on, such access is rejected. Only enable + this option when you must run userspace which requires this. + + If in doubt, say Y. + source "security/selinux/Kconfig" source "security/smack/Kconfig" source "security/tomoyo/Kconfig" From patchwork Fri Oct 23 12:21:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853009 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A1701580 for ; Fri, 23 Oct 2020 12:23:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1AF0024182 for ; Fri, 23 Oct 2020 12:23:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="braem6Gm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AF0024182 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 706E66B0078; Fri, 23 Oct 2020 08:23:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6DF1A6B007B; Fri, 23 Oct 2020 08:23:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57F0A6B007D; Fri, 23 Oct 2020 08:23:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id 2B2836B0078 for ; Fri, 23 Oct 2020 08:23:09 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C9D5F180AD815 for ; Fri, 23 Oct 2020 12:23:08 +0000 (UTC) X-FDA: 77403104856.17.chess17_47167ef27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id ACCE0180D0185 for ; Fri, 23 Oct 2020 12:23:08 +0000 (UTC) X-Spam-Summary: 1,0,0,866f420f8f987de5,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:4117:4321:5007:6261:6653:6742:6743:7875:8603:9036:10004:10400:11026:11232:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13095:13161:13229:13846:13894:14181:14394:14659:14721:14819:21080:21433:21444:21451:21627:21740:21773:30054:30055:30064:30090,0,RBL:209.85.128.68:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04yfke64grw7tz9fgytqs3i9k1iuhopsxews6q98sj98igcjprexqiufggpkkzn.p3macecgk7crd4tpf6gj8pc86bj9rupfq3nan5mxo7ypr8tzisshrcm9fyk7u1k.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMAR Y:none X-HE-Tag: chess17_47167ef27259 X-Filterd-Recvd-Size: 6856 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:08 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id d78so1229971wmd.3 for ; Fri, 23 Oct 2020 05:23:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fFEiUvLl06c3j38++d5NzLL6L7leWWZ4E8u91rYq/3A=; b=braem6GmR1a0W1U9n+P0ZjCixCoaNwItneKf4XXmwiM+nUddiP16QJccIkWq1nfeRZ 9G1sNxdp8zuH24WxTyNX0vImxMRYFV400Sbo4uC5sqzrP2mUvG1zzNo4wpm+ElGlMrv0 BVEObXOiwYJO5XRjv1iDH7j7YCFz5rnvbq8qg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fFEiUvLl06c3j38++d5NzLL6L7leWWZ4E8u91rYq/3A=; b=BD07TgkLh0mrVwzaF3UbbotZ8kWqHXwZZtIoh5T1xAvu6rC5whMNO5KyTzra3xN7Hp bcmd2RwEZl9WVyRK5AMTtRsdY9H/kr7St3DRAVk+1me8f8pHfN1683gi80JTlypWBycr 0jQfMYLoKMe22Wc6bcU6gSd5J17zbTPrb51qL0SUELAXJqq4r27KMDHIuxGwAGmeiFZB sQPuoWkq+0q1yK/ogmwzsroykP9CX+tD+M2TY/9Sw+LWh7Vv3wO3myq/2KpvSzsARNa8 Jzmpz5ai4OpeyEaS27xPFhbY+d15KuHkgw7pgvtgVmnBikcLxWrspd/KwyJA/JknGuTi kMwQ== X-Gm-Message-State: AOAM531mCl79aioUuDtipcaii9275BA8dMkgruKVk44M015kImYgjIFb uel6QrmffY1VpCBKx1t9kFtN/A== X-Google-Smtp-Source: ABdhPJx8R5/3dC+619rMxj0G4kaFtGbCPSV2VZg+ru6CCCIKgtbWbWS3poGnwIzyRcabuPIieQR+6w== X-Received: by 2002:a1c:2042:: with SMTP id g63mr1996821wmg.174.1603455787244; Fri, 23 Oct 2020 05:23:07 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:06 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Pawel Osciak , Marek Szyprowski , Kyungmin Park , Tomasz Figa , Laurent Dufour , Vlastimil Babka , Daniel Jordan , Michel Lespinasse Subject: [PATCH 38/65] media/videbuf1|2: Mark follow_pfn usage as unsafe Date: Fri, 23 Oct 2020 14:21:49 +0200 Message-Id: <20201023122216.2373294-38-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The media model assumes that buffers are all preallocated, so that when a media pipeline is running we never miss a deadline because the buffers aren't allocated or available. This means we cannot fix the v4l follow_pfn usage through mmu_notifier, without breaking how this all works. The only real fix is to deprecate userptr support for VM_IO | VM_PFNMAP mappings and tell everyone to cut over to dma-buf memory sharing for zerocopy. userptr for normal memory will keep working as-is, this only affects the zerocopy userptr usage enabled in 50ac952d2263 ("[media] videobuf2-dma-sg: Support io userptr operations on io memory"). Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Pawel Osciak Cc: Marek Szyprowski Cc: Kyungmin Park Cc: Tomasz Figa Cc: Laurent Dufour Cc: Vlastimil Babka Cc: Daniel Jordan Cc: Michel Lespinasse Signed-off-by: Daniel Vetter --- v3: - Reference the commit that enabled the zerocopy userptr use case to make it abundandtly clear that this patch only affects that, and not normal memory userptr. The old commit message already explained that normal memory userptr is unaffected, but I guess that was not clear enough. --- drivers/media/common/videobuf2/frame_vector.c | 2 +- drivers/media/v4l2-core/videobuf-dma-contig.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/media/common/videobuf2/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c index 6590987c14bd..e630494da65c 100644 --- a/drivers/media/common/videobuf2/frame_vector.c +++ b/drivers/media/common/videobuf2/frame_vector.c @@ -69,7 +69,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, break; while (ret < nr_frames && start + PAGE_SIZE <= vma->vm_end) { - err = follow_pfn(vma, start, &nums[ret]); + err = unsafe_follow_pfn(vma, start, &nums[ret]); if (err) { if (ret == 0) ret = err; diff --git a/drivers/media/v4l2-core/videobuf-dma-contig.c b/drivers/media/v4l2-core/videobuf-dma-contig.c index 52312ce2ba05..821c4a76ab96 100644 --- a/drivers/media/v4l2-core/videobuf-dma-contig.c +++ b/drivers/media/v4l2-core/videobuf-dma-contig.c @@ -183,7 +183,7 @@ static int videobuf_dma_contig_user_get(struct videobuf_dma_contig_memory *mem, user_address = untagged_baddr; while (pages_done < (mem->size >> PAGE_SHIFT)) { - ret = follow_pfn(vma, user_address, &this_pfn); + ret = unsafe_follow_pfn(vma, user_address, &this_pfn); if (ret) break; From patchwork Fri Oct 23 12:21:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA19D17E6 for ; Fri, 23 Oct 2020 12:23:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5770F24248 for ; Fri, 23 Oct 2020 12:23:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="URbjYqZv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5770F24248 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BA0046B007B; Fri, 23 Oct 2020 08:23:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B27486B007D; Fri, 23 Oct 2020 08:23:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EE436B007E; Fri, 23 Oct 2020 08:23:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0132.hostedemail.com [216.40.44.132]) by kanga.kvack.org (Postfix) with ESMTP id 734AA6B007B for ; Fri, 23 Oct 2020 08:23:10 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1EED7362C for ; Fri, 23 Oct 2020 12:23:10 +0000 (UTC) X-FDA: 77403104940.09.slope24_5d0a36c27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id F3A5F180AD81D for ; Fri, 23 Oct 2020 12:23:09 +0000 (UTC) X-Spam-Summary: 1,0,0,19fe33f43a2e79e2,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2194:2199:2393:2553:2559:2562:2693:2901:2903:3138:3139:3140:3141:3142:3353:3865:3867:3871:3874:4321:5007:6117:6119:6261:6653:6742:7903:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13069:13161:13229:13311:13357:13846:13894:14096:14097:14181:14394:14659:14721:21080:21444:21451:21627:21939:21990:30012:30054:30055:30064:30090,0,RBL:209.85.128.67:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yrksatr4j8q76k8zmmrojncejrjopgci3af69iyo1jieyda4xx1wpwupz8ca6.hkhebehsqj36hkza8y6t7q74nff7jituttyqihwmabubz1bbqqd7cgb7bdxa34a.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: slope24_5d0a36c27259 X-Filterd-Recvd-Size: 5371 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:09 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id d3so1295191wma.4 for ; Fri, 23 Oct 2020 05:23:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hp7lChw6Uow+58XPniSHemIeAoNErbB/sdK6im80Iug=; b=URbjYqZvij6vMp6zpIk/3WlniapSCelkh8qPhM9XSJmSi3JP0Pkq6JfYZq+gC0V2Ja Jq8vCQ4Ho/GAm6SQXrzJhiajzuA2XAwJXKB1Cll6tMyrVwJYz4NCmG2PdfE0KkUS7bh0 l5/AnqPDLrJU4JyaIv3jCsFac60g5JtxDcFVQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hp7lChw6Uow+58XPniSHemIeAoNErbB/sdK6im80Iug=; b=OnKY9gaLHmm91lNT/5RTP1BXK9RICbtu6vDiHoW7L8UNJmZsE/Mc+cEQFSedIIKFM9 1QgurDpz6DAFNLYZIS95uxvgRsvXsR+ZORw0bJkNMQ8xcgazw9Km2ChQtA52p28WRgDL SC36eGeznzGs1T0nyytyWXcx8mPHr6N3dvpdSLKHZQjd9Zr5khdh9n4rMW6YFElzzO5f O+N/uHPXYW4GFFAQ+fIGWlzEYF8sbEOMROWRU47UW7BcRFTfy0B3hQEf/ItSNuj5Hh9b bZcedziqW1PXLipXzO2ebu8bc9YFp6SK3gQmTs+UpyZOCbl0KVnMQXFDrxnKrSlrvS0/ Q54g== X-Gm-Message-State: AOAM533p4ffmxlWF4ojuxlsF0ulh8F4bw78GKwGG6TvPfoQ1LwCidCJn saOK6ARiXqDd8nayqAX3Urs9Vw== X-Google-Smtp-Source: ABdhPJwqZWBmXRQ8vY8ZKJrjdPXGP2WILf15a1TrVl8Pgis8iT7Odd2hX51Nk7fKP5ClvyukHiQ2vg== X-Received: by 2002:a1c:4d05:: with SMTP id o5mr1994877wmh.94.1603455788583; Fri, 23 Oct 2020 05:23:08 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:08 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Alex Williamson , Cornelia Huck , kvm@vger.kernel.org Subject: [PATCH 39/65] vfio/type1: Mark follow_pfn as unsafe Date: Fri, 23 Oct 2020 14:21:50 +0200 Message-Id: <20201023122216.2373294-39-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The code seems to stuff these pfns into iommu pts (or something like that, I didn't follow), but there's no mmu_notifier to ensure that access is synchronized with pte updates. Hence mark these as unsafe. This means that with CONFIG_STRICT_FOLLOW_PFN, these will be rejected. Real fix is to wire up an mmu_notifier ... somehow. Probably means any invalidate is a fatal fault for this vfio device, but then this shouldn't ever happen if userspace is reasonable. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Alex Williamson Cc: Cornelia Huck Cc: kvm@vger.kernel.org Signed-off-by: Daniel Vetter --- drivers/vfio/vfio_iommu_type1.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 5fbf0c1f7433..a4d53f3d0a35 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -421,7 +421,7 @@ static int follow_fault_pfn(struct vm_area_struct *vma, struct mm_struct *mm, { int ret; - ret = follow_pfn(vma, vaddr, pfn); + ret = unsafe_follow_pfn(vma, vaddr, pfn); if (ret) { bool unlocked = false; @@ -435,7 +435,7 @@ static int follow_fault_pfn(struct vm_area_struct *vma, struct mm_struct *mm, if (ret) return ret; - ret = follow_pfn(vma, vaddr, pfn); + ret = unsafe_follow_pfn(vma, vaddr, pfn); } return ret; From patchwork Fri Oct 23 12:21:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19FE216BC for ; Fri, 23 Oct 2020 12:23:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C832E24182 for ; Fri, 23 Oct 2020 12:23:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="C2WlQm6Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C832E24182 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2EADC6B007D; Fri, 23 Oct 2020 08:23:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1FFD56B007E; Fri, 23 Oct 2020 08:23:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09D9D6B0080; Fri, 23 Oct 2020 08:23:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id D310A6B007D for ; Fri, 23 Oct 2020 08:23:11 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7E235180AD815 for ; Fri, 23 Oct 2020 12:23:11 +0000 (UTC) X-FDA: 77403104982.17.worm47_1105fe927259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 635E4180D0180 for ; Fri, 23 Oct 2020 12:23:11 +0000 (UTC) X-Spam-Summary: 1,0,0,748bc48eb0361dc1,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3872:5007:6261:6653:6742:7903:10004:10400:11026:11473:11658:11914:12043:12297:12438:12517:12519:12555:12663:12679:12895:13069:13095:13161:13184:13229:13311:13357:13846:13894:14181:14394:14659:14721:21080:21433:21444:21451:21627:21990:30054:30055:30064:30070:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yfzziscqnkhsp3ytksmbbk45f3syc7eqo7uf5bc6thm9pu9tw1jbet4zkujd9.n4oksd71jerhjgj161i8eotz5s4s3hdeon94wcydjfeep7r37cnczq5bbbxz8j1.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: worm47_1105fe927259 X-Filterd-Recvd-Size: 5174 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:10 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id x7so1596762wrl.3 for ; Fri, 23 Oct 2020 05:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5W4UqFTJMoDRzu7XpmV7m/mIx7tbXZK4TFNNVek8jNg=; b=C2WlQm6Y7/BLr8qzfIPZbRoqOaxbXAT3hL2x0db/Vq7QIa9HiiDV68fCYfgycZtA+x GcQT5YHcVe38UBZQ7KU9yFXVhUTzN+GFKBdcS0mcPQCEf6Uxqwp+fW97FHKul4o73JoP PYYcx5UtzedMnl7YfV4Rz5BNzjRWhqXORNHQA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5W4UqFTJMoDRzu7XpmV7m/mIx7tbXZK4TFNNVek8jNg=; b=a6VJ+wXGdOyJ3nUYQnreEhobW1fqXpylQv3424MLuP/EES/gLcDU/ddf50XcpF5TwW LSvmj/UlxuuYUIhwefSwZh0xw9mLUWvJWKFglo6DGBVbiJVFGaT1Agbu43XESp8mJpNy jDEDAeWY9kIaOOHbPJrl/40A2vjakCgT2YdTxkUwmKGePuzL3ZB0T3blzN49xl6ouUK1 o7SLK7DFZL8Nul/p4OOQ6D8e6e5I5guzeb/J9Hx9r05HS/Me9f8wS5rAn/f2UxPRYddG eq6yNMBQ0zO2QOcUCZ4r+BDQH0vKqNuelG3uKlYQqT6LDw5JQh0K0Wtc1Sb71EdQu1+X tJyw== X-Gm-Message-State: AOAM532n+N+l59XxaUh08FyuAy6evC/AZTkIUPtumqQD+VT0/kzrWy6Q Bmrc8Nh5vz6Kq5B1AiULzRctKw== X-Google-Smtp-Source: ABdhPJw3Pb3vhwtwwYNFMVzcx8Z/llu6fU2csM/RXOrVwDY9ekmNXTKA2S1lYyLiAIKDJCOf9S0bYg== X-Received: by 2002:a5d:6cae:: with SMTP id a14mr2454800wra.187.1603455789882; Fri, 23 Oct 2020 05:23:09 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:09 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Bjorn Helgaas , linux-pci@vger.kernel.org Subject: [PATCH 40/65] PCI: Obey iomem restrictions for procfs mmap Date: Fri, 23 Oct 2020 14:21:51 +0200 Message-Id: <20201023122216.2373294-40-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's three ways to access PCI BARs from userspace: /dev/mem, sysfs files, and the old proc interface. Two check against iomem_is_exclusive, proc never did. And with CONFIG_IO_STRICT_DEVMEM, this starts to matter, since we don't want random userspace having access to PCI BARs while a driver is loaded and using it. Fix this by adding the same iomem_is_exclusive() check we already have on the sysfs side in pci_mmap_resource(). References: 90a545e98126 ("restrict /dev/mem to idle io memory ranges") Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org Signed-off-by: Daniel Vetter --- v2: Improve commit message (Bjorn) --- drivers/pci/proc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/pci/proc.c b/drivers/pci/proc.c index d35186b01d98..3a2f90beb4cb 100644 --- a/drivers/pci/proc.c +++ b/drivers/pci/proc.c @@ -274,6 +274,11 @@ static int proc_bus_pci_mmap(struct file *file, struct vm_area_struct *vma) else return -EINVAL; } + + if (dev->resource[i].flags & IORESOURCE_MEM && + iomem_is_exclusive(dev->resource[i].start)) + return -EINVAL; + ret = pci_mmap_page_range(dev, i, vma, fpriv->mmap_state, write_combine); if (ret < 0) From patchwork Fri Oct 23 12:21:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A10991580 for ; Fri, 23 Oct 2020 12:23:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 514F924248 for ; Fri, 23 Oct 2020 12:23:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="ZYi8sWlY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 514F924248 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6974A6B007E; Fri, 23 Oct 2020 08:23:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5CBFB6B0080; Fri, 23 Oct 2020 08:23:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46CDE6B0081; Fri, 23 Oct 2020 08:23:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0147.hostedemail.com [216.40.44.147]) by kanga.kvack.org (Postfix) with ESMTP id 181146B007E for ; Fri, 23 Oct 2020 08:23:13 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BBC21181AC9CC for ; Fri, 23 Oct 2020 12:23:12 +0000 (UTC) X-FDA: 77403105024.05.bell02_62157ed27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 9F0BD18001721 for ; Fri, 23 Oct 2020 12:23:12 +0000 (UTC) X-Spam-Summary: 1,0,0,d46aba4f1ba9ad0e,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:968:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3871:3872:3874:4321:4401:5007:6261:6653:6742:10004:10400:11658:11914:12043:12297:12517:12519:12555:12895:12986:13069:13311:13357:13846:13894:14096:14097:14181:14394:14659:14721:21080:21444:21451:21627:21990:30054:30055:30064:30089:30090,0,RBL:209.85.128.65:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04y8fap4so3eo6rnh1z3wbi8k945fopr8qepnh5cm1s4zcdsut3dxmknpfir91h.xcijiuh1854f4ygnbe6yc45okxb8pfh73j9gju1ead4oe5y4pte6tthgtzygsj4.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: bell02_62157ed27259 X-Filterd-Recvd-Size: 4930 Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:12 +0000 (UTC) Received: by mail-wm1-f65.google.com with SMTP id c194so1234711wme.2 for ; Fri, 23 Oct 2020 05:23:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r2DrHs8sznEAEmv81CQMiKCncSgZZXtZ6EFyoHCS3Oo=; b=ZYi8sWlYqHhW266a7Wh4e6eQQ7DyJpqW3Nom0/Vr/JEW76wTqmHKXtiUU2akIxM9qE 8aR9Gox2HMVTwkNkPC+2hvbT31wupa4IhJNXHm/GZLFwhyKkoZlckNfJvIscTIBwTBdu K9kLhfXuXSyNypnF/UEvOEGMAa8S7zhjmlrlI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r2DrHs8sznEAEmv81CQMiKCncSgZZXtZ6EFyoHCS3Oo=; b=uPN6La+G+AUCIhbBZvy8rtpKOtvXpzQcLhTT1Eh/BGzIvpNPWimelU7A4tH8O6BjNS QUf3x1JpvpIqC9Ti5l6DmxBNJwE8k21ZAUHKQJaWvhhSajWvcPU1LGrI4ei0D06FFisw y7BQb77KVzX0gUnJEvFz2joSgNrYxiKF/Gg124hZkp0u2AIxdKkvNsTpu1gNUPes33O3 5+PQvm+/SZJbDincZ8ibOBoe5VPuzEP4tdC45X283eDmwrm5m8bLYDKYxlcgCOiu3CCH nii77ph/6oEr+/3TVCwTMmlL/xzw54lTqsUsjXjC/qGqyfhaxNxhHh3PUTJOzT0MPhQW c/Qg== X-Gm-Message-State: AOAM5324+5W5ruyuLumoIemW8jH2p++91ds0KSJmWVMkJvvMqjwUty7U PrsBiUQzBmJLttUcP3YWvBdePg== X-Google-Smtp-Source: ABdhPJxMGQvA/dZohSXbnMcyrWAeGFzAxThKHUObw664xzBQpaV3+wC1rwIhI7OjqV8yoZkn2jUtDA== X-Received: by 2002:a1c:4445:: with SMTP id r66mr2002851wma.140.1603455791088; Fri, 23 Oct 2020 05:23:11 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:10 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org Subject: [PATCH 41/65] /dev/mem: Only set filp->f_mapping Date: Fri, 23 Oct 2020 14:21:52 +0200 Message-Id: <20201023122216.2373294-41-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we care about pagecache maintenance, we need to make sure that both f_mapping and i_mapping point at the right mapping. But for iomem mappings we only care about the virtual/pte side of things, so f_mapping is enough. Also setting inode->i_mapping was confusing me as a driver maintainer, since in e.g. drivers/gpu we don't do that. Per Dan this seems to be copypasta from places which do care about pagecache consistency, but not needed. Hence remove it for slightly less confusion. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Reviewed-by: Dan Williams Signed-off-by: Daniel Vetter --- drivers/char/mem.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/char/mem.c b/drivers/char/mem.c index abd4ffdc8cde..5502f56f3655 100644 --- a/drivers/char/mem.c +++ b/drivers/char/mem.c @@ -864,7 +864,6 @@ static int open_port(struct inode *inode, struct file *filp) * revocations when drivers want to take over a /dev/mem mapped * range. */ - inode->i_mapping = devmem_inode->i_mapping; filp->f_mapping = inode->i_mapping; return 0; From patchwork Fri Oct 23 12:21:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1AA6B1580 for ; Fri, 23 Oct 2020 12:23:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C01F421527 for ; Fri, 23 Oct 2020 12:23:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="e5hphnls" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C01F421527 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C11756B0080; Fri, 23 Oct 2020 08:23:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BEA5F6B0081; Fri, 23 Oct 2020 08:23:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A88D96B0082; Fri, 23 Oct 2020 08:23:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 7B5896B0080 for ; Fri, 23 Oct 2020 08:23:14 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 29CF01DED for ; Fri, 23 Oct 2020 12:23:14 +0000 (UTC) X-FDA: 77403105108.12.copy93_110f52d27259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 0287C18054902 for ; Fri, 23 Oct 2020 12:23:13 +0000 (UTC) X-Spam-Summary: 1,0,0,3a9466429d0fc9bb,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:1:2:41:69:152:355:379:541:800:960:966:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1500:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2553:2559:2562:2894:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4052:4321:4385:4605:5007:6119:6261:6653:6742:7875:7903:8603:9592:9707:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12517:12519:12555:12683:12895:12986:13846:13894:14394:14659:21080:21433:21444:21451:21611:21627:21987:21990:30012:30029:30054:30055:30064:30070:30089:30090,0,RBL:209.85.221.67:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04y8o7r7zmtdnurmkuqpiexf5umwsycu1cjr35s14oapkidfw17yj1bus7d1q98.np9u6cpywb9c7xcm3jpo8rysarh9uinourmbo33jdom3wsfcm1xdbswdxqy9u88.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNS BL:none, X-HE-Tag: copy93_110f52d27259 X-Filterd-Recvd-Size: 13106 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:13 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id n15so1599437wrq.2 for ; Fri, 23 Oct 2020 05:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Eh+QFd2rUGVvEdUIq62q2ZCwC5wIklTNSjxdtgUk2/Y=; b=e5hphnlsMYsasyR/YI69NuX59uzEdX2qcimMYImOZm1M/LT2BHrGVpvWCoO3kcKnbz Ov4evujMm+wUa0eaaQI03YI0d3JG8wQ5tVEwmAcc8BYd5ts9ASXWcikA5h+6IZPLJgkT Q5UjRznd6tk9kE/aYdy5AZJulxidfuYuKArx4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Eh+QFd2rUGVvEdUIq62q2ZCwC5wIklTNSjxdtgUk2/Y=; b=aLsdrJ9ibJoYCU9p6QHRuPzGsmsl3CqIaMfaAHyX6ziRU9kFhC2yeV6nHcGrh1VaZE D/mCTagWRUHHeuBXGwrPs8vHvKlQtyfki8ToRGICC1NG2OV/631EIdLdJdvVnmMST6K+ WxaSWy+dqRItHkPgVEU2rAdINV68J692IkXP0MthSZNKUbBVkKCx1gv/CL9y+tH7hCrO cwCM3mFGryealPPXDw0CiP1oQqV1vF1zjyRmyz2+EpVNcBgXRNwVe+XQRjOaIEtwURa7 mmak9R/2RKojGDFon/A+dcZZG1hON2h9T5AJ0m9AbjWza4TEYyOtKzIm62uzMYWB6r+k VMDw== X-Gm-Message-State: AOAM530j7MHtHEoYUhNh9ELUW+fwufyftGGgZuRz4H0Se8q0oH9YNQHq AgR9pZKE5G/B7EUp7DsmTMVsUQ== X-Google-Smtp-Source: ABdhPJwnOL4dnmc6lyGNP0x67dHlVVixkc5nrJQk3+7ioO7moXdzTELeRqRSEuYEwVn0xorg8Qblnw== X-Received: by 2002:adf:e549:: with SMTP id z9mr2524031wrm.12.1603455792411; Fri, 23 Oct 2020 05:23:12 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:11 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Greg Kroah-Hartman , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Arnd Bergmann , David Hildenbrand , "Rafael J. Wysocki" Subject: [PATCH 42/65] resource: Move devmem revoke code to resource framework Date: Fri, 23 Oct 2020 14:21:53 +0200 Message-Id: <20201023122216.2373294-42-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We want all iomem mmaps to consistently revoke ptes when the kernel takes over and CONFIG_IO_STRICT_DEVMEM is enabled. This includes the pci bar mmaps available through procfs and sysfs, which currently do not revoke mappings. To prepare for this, move the code from the /dev/kmem driver to kernel/resource.c. Reviewed-by: Greg Kroah-Hartman Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Arnd Bergmann Cc: Greg Kroah-Hartman Cc: Daniel Vetter Cc: David Hildenbrand Cc: "Rafael J. Wysocki" Signed-off-by: Daniel Vetter --- v3: - add barrier for consistency and document why we don't have to check for NULL (Jason) v4 - Adjust comments to reflect the general nature of this iomem revoke code now (Dan) --- drivers/char/mem.c | 85 +--------------------------------- include/linux/ioport.h | 6 +-- kernel/resource.c | 101 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 102 insertions(+), 90 deletions(-) diff --git a/drivers/char/mem.c b/drivers/char/mem.c index 5502f56f3655..53338aad8d28 100644 --- a/drivers/char/mem.c +++ b/drivers/char/mem.c @@ -31,9 +31,6 @@ #include #include #include -#include -#include -#include #ifdef CONFIG_IA64 # include @@ -809,42 +806,6 @@ static loff_t memory_lseek(struct file *file, loff_t offset, int orig) return ret; } -static struct inode *devmem_inode; - -#ifdef CONFIG_IO_STRICT_DEVMEM -void revoke_devmem(struct resource *res) -{ - /* pairs with smp_store_release() in devmem_init_inode() */ - struct inode *inode = smp_load_acquire(&devmem_inode); - - /* - * Check that the initialization has completed. Losing the race - * is ok because it means drivers are claiming resources before - * the fs_initcall level of init and prevent /dev/mem from - * establishing mappings. - */ - if (!inode) - return; - - /* - * The expectation is that the driver has successfully marked - * the resource busy by this point, so devmem_is_allowed() - * should start returning false, however for performance this - * does not iterate the entire resource range. - */ - if (devmem_is_allowed(PHYS_PFN(res->start)) && - devmem_is_allowed(PHYS_PFN(res->end))) { - /* - * *cringe* iomem=relaxed says "go ahead, what's the - * worst that can happen?" - */ - return; - } - - unmap_mapping_range(inode->i_mapping, res->start, resource_size(res), 1); -} -#endif - static int open_port(struct inode *inode, struct file *filp) { int rc; @@ -864,7 +825,7 @@ static int open_port(struct inode *inode, struct file *filp) * revocations when drivers want to take over a /dev/mem mapped * range. */ - filp->f_mapping = inode->i_mapping; + filp->f_mapping = iomem_get_mapping(); return 0; } @@ -995,48 +956,6 @@ static char *mem_devnode(struct device *dev, umode_t *mode) static struct class *mem_class; -static int devmem_fs_init_fs_context(struct fs_context *fc) -{ - return init_pseudo(fc, DEVMEM_MAGIC) ? 0 : -ENOMEM; -} - -static struct file_system_type devmem_fs_type = { - .name = "devmem", - .owner = THIS_MODULE, - .init_fs_context = devmem_fs_init_fs_context, - .kill_sb = kill_anon_super, -}; - -static int devmem_init_inode(void) -{ - static struct vfsmount *devmem_vfs_mount; - static int devmem_fs_cnt; - struct inode *inode; - int rc; - - rc = simple_pin_fs(&devmem_fs_type, &devmem_vfs_mount, &devmem_fs_cnt); - if (rc < 0) { - pr_err("Cannot mount /dev/mem pseudo filesystem: %d\n", rc); - return rc; - } - - inode = alloc_anon_inode(devmem_vfs_mount->mnt_sb); - if (IS_ERR(inode)) { - rc = PTR_ERR(inode); - pr_err("Cannot allocate inode for /dev/mem: %d\n", rc); - simple_release_fs(&devmem_vfs_mount, &devmem_fs_cnt); - return rc; - } - - /* - * Publish /dev/mem initialized. - * Pairs with smp_load_acquire() in revoke_devmem(). - */ - smp_store_release(&devmem_inode, inode); - - return 0; -} - static int __init chr_dev_init(void) { int minor; @@ -1058,8 +977,6 @@ static int __init chr_dev_init(void) */ if ((minor == DEVPORT_MINOR) && !arch_has_dev_port()) continue; - if ((minor == DEVMEM_MINOR) && devmem_init_inode() != 0) - continue; device_create(mem_class, NULL, MKDEV(MEM_MAJOR, minor), NULL, devlist[minor].name); diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 6c2b06fe8beb..8ffb61b36606 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -302,11 +302,7 @@ struct resource *devm_request_free_mem_region(struct device *dev, struct resource *request_free_mem_region(struct resource *base, unsigned long size, const char *name); -#ifdef CONFIG_IO_STRICT_DEVMEM -void revoke_devmem(struct resource *res); -#else -static inline void revoke_devmem(struct resource *res) { }; -#endif +extern struct address_space *iomem_get_mapping(void); #endif /* __ASSEMBLY__ */ #endif /* _LINUX_IOPORT_H */ diff --git a/kernel/resource.c b/kernel/resource.c index 841737bbda9e..a800acbc578c 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -18,12 +18,15 @@ #include #include #include +#include #include #include #include #include #include +#include #include +#include #include @@ -1112,6 +1115,58 @@ resource_size_t resource_alignment(struct resource *res) static DECLARE_WAIT_QUEUE_HEAD(muxed_resource_wait); +static struct inode *iomem_inode; + +#ifdef CONFIG_IO_STRICT_DEVMEM +static void revoke_iomem(struct resource *res) +{ + /* pairs with smp_store_release() in iomem_init_inode() */ + struct inode *inode = smp_load_acquire(&iomem_inode); + + /* + * Check that the initialization has completed. Losing the race + * is ok because it means drivers are claiming resources before + * the fs_initcall level of init and prevent iomem_get_mapping users + * from establishing mappings. + */ + if (!inode) + return; + + /* + * The expectation is that the driver has successfully marked + * the resource busy by this point, so devmem_is_allowed() + * should start returning false, however for performance this + * does not iterate the entire resource range. + */ + if (devmem_is_allowed(PHYS_PFN(res->start)) && + devmem_is_allowed(PHYS_PFN(res->end))) { + /* + * *cringe* iomem=relaxed says "go ahead, what's the + * worst that can happen?" + */ + return; + } + + unmap_mapping_range(inode->i_mapping, res->start, resource_size(res), 1); +} +struct address_space *iomem_get_mapping(void) +{ + /* + * This function is only called from file open paths, hence guaranteed + * that fs_initcalls have completed and no need to check for NULL. But + * since revoke_iomem can be called before the initcall we still need + * the barrier to appease checkers. + */ + return smp_load_acquire(&iomem_inode)->i_mapping; +} +#else +static void revoke_iomem(struct resource *res) {} +struct address_space *iomem_get_mapping(void) +{ + return NULL; +} +#endif + /** * __request_region - create a new busy resource region * @parent: parent resource descriptor @@ -1179,7 +1234,7 @@ struct resource * __request_region(struct resource *parent, write_unlock(&resource_lock); if (res && orig_parent == &iomem_resource) - revoke_devmem(res); + revoke_iomem(res); return res; } @@ -1713,4 +1768,48 @@ static int __init strict_iomem(char *str) return 1; } +static int iomem_fs_init_fs_context(struct fs_context *fc) +{ + return init_pseudo(fc, DEVMEM_MAGIC) ? 0 : -ENOMEM; +} + +static struct file_system_type iomem_fs_type = { + .name = "iomem", + .owner = THIS_MODULE, + .init_fs_context = iomem_fs_init_fs_context, + .kill_sb = kill_anon_super, +}; + +static int __init iomem_init_inode(void) +{ + static struct vfsmount *iomem_vfs_mount; + static int iomem_fs_cnt; + struct inode *inode; + int rc; + + rc = simple_pin_fs(&iomem_fs_type, &iomem_vfs_mount, &iomem_fs_cnt); + if (rc < 0) { + pr_err("Cannot mount iomem pseudo filesystem: %d\n", rc); + return rc; + } + + inode = alloc_anon_inode(iomem_vfs_mount->mnt_sb); + if (IS_ERR(inode)) { + rc = PTR_ERR(inode); + pr_err("Cannot allocate inode for iomem: %d\n", rc); + simple_release_fs(&iomem_vfs_mount, &iomem_fs_cnt); + return rc; + } + + /* + * Publish iomem revocation inode initialized. + * Pairs with smp_load_acquire() in revoke_iomem(). + */ + smp_store_release(&iomem_inode, inode); + + return 0; +} + +fs_initcall(iomem_init_inode); + __setup("iomem=", strict_iomem); From patchwork Fri Oct 23 12:21:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A951992C for ; Fri, 23 Oct 2020 12:23:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5EA2124654 for ; Fri, 23 Oct 2020 12:23:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="X9fvYwj+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EA2124654 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1FDA06B0081; Fri, 23 Oct 2020 08:23:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 18A036B0082; Fri, 23 Oct 2020 08:23:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0022D6B0083; Fri, 23 Oct 2020 08:23:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0092.hostedemail.com [216.40.44.92]) by kanga.kvack.org (Postfix) with ESMTP id C9B336B0081 for ; Fri, 23 Oct 2020 08:23:15 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6DD7F181AC9CC for ; Fri, 23 Oct 2020 12:23:15 +0000 (UTC) X-FDA: 77403105150.03.bikes06_5f0b67327259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 4C0DE28A4E8 for ; Fri, 23 Oct 2020 12:23:15 +0000 (UTC) X-Spam-Summary: 1,0,0,29229150f3791ecd,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3354:3865:3866:3867:3870:3871:3874:4118:4321:4605:5007:6261:6653:6742:6743:7903:8603:10004:10400:10450:10455:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12700:12737:12895:12986:13161:13229:13846:13894:14096:14097:14181:14394:14659:14721:19904:19999:21080:21324:21444:21451:21627:21990:30029:30054:30055:30064:30090,0,RBL:209.85.128.66:@ffwll.ch:.lbl8.mailshell.net-66.201.201.201 62.8.0.100;04yrfhh5t5oezdschkp1pobmktzmeopwnke1j1qiapge647dwmioek9gikdfbzf.63dnjyqynadyyy9ettdstfpjuozftrn3ss185ekdht5p7h3huxa9mxjeihhftwc.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:26,LUA_S UMMARY:n X-HE-Tag: bikes06_5f0b67327259 X-Filterd-Recvd-Size: 7119 Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:14 +0000 (UTC) Received: by mail-wm1-f66.google.com with SMTP id w23so143984wmi.4 for ; Fri, 23 Oct 2020 05:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkmJ6ZmKgmcd6yJ6bv+WDWEfAJ+atBjrBRrq3Oi6xEA=; b=X9fvYwj+CM1uPA1P8WC5HSgT+LzVZQ+OD1o+k+agSuOPsfZl3UPk7123oRH9IvG99j GfkfDQWLVGKw2EbuNrvibQZy+U+qtkpV3YLZht2b4ThR9SdJpdsR19OQn6pLGfNk0QFs ZOzs+xrJyUetOFvNliSHp5y1W0rt8j+J+9iN0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkmJ6ZmKgmcd6yJ6bv+WDWEfAJ+atBjrBRrq3Oi6xEA=; b=lgRL7azDiJOMtzK5Ea7Yrjdkbs/LA5Wo7vTgJKhre9Ta3lDxRKVSUX3G6cgUFkwhc8 XW/+yDYz+b4RVGxD0XJxzaQXo+wEeLIQ8/xehnnyVCH5tJIVRrmJgwFeb2r/TNK6P2Vy u4HcIRtOdnD/qRn0dckTeA1Fd4tHhv6ywE2aUWYeyQnqRR1KX0tmKbAodylb69giFR4U 8h273rRoZX302y/6hy+53q5DgxdKNy3PzwYZHbHpL0anOElL4PhnkBvWMMlFLtgFyIy+ 3Vbo41YCPfmAXuIgAFogXYLBOlmcGOrU4bG6RrPVv49rvtFAeTmCw3JYdF9E5uv/JD/A RFMg== X-Gm-Message-State: AOAM531f6TQOfmWPZLPhvhXBm7vNSqX6kg7UU7tG+F0o1kwew0an7Hp/ +lwK4EEwfF2cegF0qT8xK6wTnA== X-Google-Smtp-Source: ABdhPJyAocM64pOIP3LBDQAV0GBJxakbq+ElmbOKnaH+G79tsDURZz/ywBYbGb5p2YID6L6RLw4GKg== X-Received: by 2002:a1c:9a93:: with SMTP id c141mr2199856wme.168.1603455793854; Fri, 23 Oct 2020 05:23:13 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:13 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Greg Kroah-Hartman , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Bjorn Helgaas , linux-pci@vger.kernel.org, "Rafael J. Wysocki" , Christian Brauner , "David S. Miller" , Michael Ellerman , Sourabh Jain , Mauro Carvalho Chehab , Nayna Jain Subject: [PATCH 43/65] sysfs: Support zapping of binary attr mmaps Date: Fri, 23 Oct 2020 14:21:54 +0200 Message-Id: <20201023122216.2373294-43-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We want to be able to revoke pci mmaps so that the same access rules applies as for /dev/kmem. Revoke support for devmem was added in 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region"). The simplest way to achieve this is by having the same filp->f_mapping for all mappings, so that unmap_mapping_range can find them all, no matter through which file they've been created. Since this must be set at open time we need sysfs support for this. Add an optional mapping parameter bin_attr, which is only consulted when there's also an mmap callback, since without mmap support allowing to adjust the ->f_mapping makes no sense. Reviewed-by: Greg Kroah-Hartman Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Cc: Christian Brauner Cc: "David S. Miller" Cc: Michael Ellerman Cc: Sourabh Jain Cc: Daniel Vetter Cc: Mauro Carvalho Chehab Cc: Nayna Jain Signed-off-by: Daniel Vetter --- fs/sysfs/file.c | 11 +++++++++++ include/linux/sysfs.h | 2 ++ 2 files changed, 13 insertions(+) diff --git a/fs/sysfs/file.c b/fs/sysfs/file.c index eb6897ab78e7..9d8ccdb000e3 100644 --- a/fs/sysfs/file.c +++ b/fs/sysfs/file.c @@ -169,6 +169,16 @@ static int sysfs_kf_bin_mmap(struct kernfs_open_file *of, return battr->mmap(of->file, kobj, battr, vma); } +static int sysfs_kf_bin_open(struct kernfs_open_file *of) +{ + struct bin_attribute *battr = of->kn->priv; + + if (battr->mapping) + of->file->f_mapping = battr->mapping; + + return 0; +} + void sysfs_notify(struct kobject *kobj, const char *dir, const char *attr) { struct kernfs_node *kn = kobj->sd, *tmp; @@ -240,6 +250,7 @@ static const struct kernfs_ops sysfs_bin_kfops_mmap = { .read = sysfs_kf_bin_read, .write = sysfs_kf_bin_write, .mmap = sysfs_kf_bin_mmap, + .open = sysfs_kf_bin_open, }; int sysfs_add_file_mode_ns(struct kernfs_node *parent, diff --git a/include/linux/sysfs.h b/include/linux/sysfs.h index 34e84122f635..a17a474d1601 100644 --- a/include/linux/sysfs.h +++ b/include/linux/sysfs.h @@ -164,11 +164,13 @@ __ATTRIBUTE_GROUPS(_name) struct file; struct vm_area_struct; +struct address_space; struct bin_attribute { struct attribute attr; size_t size; void *private; + struct address_space *mapping; ssize_t (*read)(struct file *, struct kobject *, struct bin_attribute *, char *, loff_t, size_t); ssize_t (*write)(struct file *, struct kobject *, struct bin_attribute *, From patchwork Fri Oct 23 12:21:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11853025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E9FB1580 for ; Fri, 23 Oct 2020 12:23:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B80BC2463C for ; Fri, 23 Oct 2020 12:23:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="MXuVxECt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B80BC2463C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFFC96B0082; Fri, 23 Oct 2020 08:23:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A87236B0083; Fri, 23 Oct 2020 08:23:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B8386B0085; Fri, 23 Oct 2020 08:23:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id 5D0256B0082 for ; Fri, 23 Oct 2020 08:23:17 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 028308062E65 for ; Fri, 23 Oct 2020 12:23:17 +0000 (UTC) X-FDA: 77403105234.07.pie33_190338527259 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id DA7F51803F9A3 for ; Fri, 23 Oct 2020 12:23:16 +0000 (UTC) X-Spam-Summary: 1,0,0,15d0d23e44066e1c,d41d8cd98f00b204,daniel.vetter@ffwll.ch,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1431:1437:1515:1516:1518:1535:1544:1593:1594:1711:1730:1747:1777:1792:1981:2194:2198:2199:2200:2393:2553:2559:2562:2901:2902:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:4118:4250:4321:4605:5007:6119:6261:6653:6742:7875:7903:7996:8603:10004:11026:11232:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12663:12895:13161:13229:13846:13894:14096:14097:14181:14394:14659:14721:21080:21444:21451:21627:21740:21939:21990:30054:30055:30064:30070:30090,0,RBL:209.85.221.66:@ffwll.ch:.lbl8.mailshell.net-62.8.0.100 66.201.201.201;04y86ukjyu1y8heqwrqargauxz1m8opbp91n9y435qdwm1xa66ftattgxyrpat5.1uh8qixi3fhk814b68jfzu3b4qrz9z3dax7a9frjdh9zaeu8epk1e5zbrwxh1nx.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Cus tom_rule X-HE-Tag: pie33_190338527259 X-Filterd-Recvd-Size: 7769 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Fri, 23 Oct 2020 12:23:16 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id b8so1601833wrn.0 for ; Fri, 23 Oct 2020 05:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rl2aV6v4BUOFD3sWM4dGPxjWFUFBHT1H6eKln2146qM=; b=MXuVxECtJ78bOiKGDfTFAHp+CJhdgNky1rItm2A5X7tJL5vSSg4lbAVohhini2H0yd xh374IeyDf7uaUzvTG9zdianxpiWqa1UVuaj2eiOwSQsJ2XHInhGqMSD1spurc8v+95N ChQ4jEq74dxOuKfApkooMZETrU8kZ+CcHZQ1Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rl2aV6v4BUOFD3sWM4dGPxjWFUFBHT1H6eKln2146qM=; b=WELaW+q+Rk5zfMosS9fqNFb4orGDYWoFca5U+qvTadc4caXuhQZRjAy8Gwrkp/fZW/ ONN3FCzgPx0s6ReNcy3L8n307E+DLYQlaT49dFmZrcMqps/5HEUpliNJSzj9wk+UYfn4 vzrtfEcWyO8J7vv6VbGvM6m5QMR9Q5NgnBZS9LW30eFrDq9ScPCWX/GXS8lPxn3qMxhM KgqWUjH86N5swAhh17SQ1ZxMQSgu6c29vlLF+fXlM2jyek1+L7qSGBQNaqm+OB9zfFXh 3ZQHDCoA9sGsr5/j80OgtrGn+kVgLM7tZKuAiQpKPEJCvvGsp2PLc6JA+gFKzI1624A1 FHTQ== X-Gm-Message-State: AOAM530nm0eYJjuIjcQD86ZWJTTaPshvqS8rroSdX6Yajj4HrjGPOiGj 4aXCUD3nBo4TBHmqPuRRVjoPiA== X-Google-Smtp-Source: ABdhPJwV5FzPPp/4Ge7bj9kllxBQ6Gec4MZEynrcnFexeILKj2jkWpYJEMkJr7ScFwkgM+ZpKL2Tqg== X-Received: by 2002:a5d:6551:: with SMTP id z17mr2353056wrv.266.1603455795408; Fri, 23 Oct 2020 05:23:15 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id y4sm3056484wrp.74.2020.10.23.05.23.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Oct 2020 05:23:14 -0700 (PDT) From: Daniel Vetter To: DRI Development Cc: Intel Graphics Development , Daniel Vetter , Daniel Vetter , Jason Gunthorpe , Kees Cook , Dan Williams , Andrew Morton , John Hubbard , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Greg Kroah-Hartman , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-media@vger.kernel.org, Bjorn Helgaas , linux-pci@vger.kernel.org Subject: [PATCH 44/65] PCI: Revoke mappings like devmem Date: Fri, 23 Oct 2020 14:21:55 +0200 Message-Id: <20201023122216.2373294-44-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201023122216.2373294-1-daniel.vetter@ffwll.ch> References: <20201021163242.1458885-1-daniel.vetter@ffwll.ch> <20201023122216.2373294-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") /dev/kmem zaps ptes when the kernel requests exclusive acccess to an iomem region. And with CONFIG_IO_STRICT_DEVMEM, this is the default for all driver uses. Except there's two more ways to access PCI BARs: sysfs and proc mmap support. Let's plug that hole. For revoke_devmem() to work we need to link our vma into the same address_space, with consistent vma->vm_pgoff. ->pgoff is already adjusted, because that's how (io_)remap_pfn_range works, but for the mapping we need to adjust vma->vm_file->f_mapping. The cleanest way is to adjust this at at ->open time: - for sysfs this is easy, now that binary attributes support this. We just set bin_attr->mapping when mmap is supported - for procfs it's a bit more tricky, since procfs pci access has only one file per device, and access to a specific resources first needs to be set up with some ioctl calls. But mmap is only supported for the same resources as sysfs exposes with mmap support, and otherwise rejected, so we can set the mapping unconditionally at open time without harm. A special consideration is for arch_can_pci_mmap_io() - we need to make sure that the ->f_mapping doesn't alias between ioport and iomem space. There's only 2 ways in-tree to support mmap of ioports: generic pci mmap (ARCH_GENERIC_PCI_MMAP_RESOURCE), and sparc as the single architecture hand-rolling. Both approach support ioport mmap through a special pfn range and not through magic pte attributes. Aliasing is therefore not a problem. The only difference in access checks left is that sysfs PCI mmap does not check for CAP_RAWIO. I'm not really sure whether that should be added or not. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Kees Cook Cc: Dan Williams Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: Greg Kroah-Hartman Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org Signed-off-by: Daniel Vetter --- v2: - Totally new approach: Adjust filp->f_mapping at open time. Note that this now works on all architectures, not just those support ARCH_GENERIC_PCI_MMAP_RESOURCE --- drivers/pci/pci-sysfs.c | 4 ++++ drivers/pci/proc.c | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index 6d78df981d41..cee38fcb4a86 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -928,6 +928,7 @@ void pci_create_legacy_files(struct pci_bus *b) b->legacy_io->read = pci_read_legacy_io; b->legacy_io->write = pci_write_legacy_io; b->legacy_io->mmap = pci_mmap_legacy_io; + b->legacy_io->mapping = iomem_get_mapping(); pci_adjust_legacy_attr(b, pci_mmap_io); error = device_create_bin_file(&b->dev, b->legacy_io); if (error) @@ -940,6 +941,7 @@ void pci_create_legacy_files(struct pci_bus *b) b->legacy_mem->size = 1024*1024; b->legacy_mem->attr.mode = 0600; b->legacy_mem->mmap = pci_mmap_legacy_mem; + b->legacy_io->mapping = iomem_get_mapping(); pci_adjust_legacy_attr(b, pci_mmap_mem); error = device_create_bin_file(&b->dev, b->legacy_mem); if (error) @@ -1155,6 +1157,8 @@ static int pci_create_attr(struct pci_dev *pdev, int num, int write_combine) res_attr->mmap = pci_mmap_resource_uc; } } + if (res_attr->mmap) + res_attr->mapping = iomem_get_mapping(); res_attr->attr.name = res_attr_name; res_attr->attr.mode = 0600; res_attr->size = pci_resource_len(pdev, num); diff --git a/drivers/pci/proc.c b/drivers/pci/proc.c index 3a2f90beb4cb..9bab07302bbf 100644 --- a/drivers/pci/proc.c +++ b/drivers/pci/proc.c @@ -298,6 +298,7 @@ static int proc_bus_pci_open(struct inode *inode, struct file *file) fpriv->write_combine = 0; file->private_data = fpriv; + file->f_mapping = iomem_get_mapping(); return 0; }