From patchwork Tue Apr 4 13:58:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13200237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17822C761AF for ; Tue, 4 Apr 2023 13:59:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A84AB6B007D; Tue, 4 Apr 2023 09:59:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0DB2900002; Tue, 4 Apr 2023 09:59:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D4376B0080; Tue, 4 Apr 2023 09:59:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7ED306B007D for ; Tue, 4 Apr 2023 09:59:00 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4158BABD51 for ; Tue, 4 Apr 2023 13:59:00 +0000 (UTC) X-FDA: 80643864840.07.ABEDAD6 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 65F9920004 for ; Tue, 4 Apr 2023 13:58:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TxwFEXAt; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680616738; a=rsa-sha256; cv=none; b=VuaLikC8M0FZzzdjeDu/9ty+TuSFbfOeDZUCPH9C1ZS7LHms5KpKj3Qgyfu1hIJfrkWmUc I8R77yBllYW6m1A0rE5lNQn8IPfUMg0y7Juk8Ndv4Mg25xC4+sfuApoEFpgWxpO+UqELif OoG5FYFG42C9xWWA7UJSEOzb0h5GJj0= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TxwFEXAt; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680616738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fp3DXW4B3dj0QEKkbnSR4pkaFzIpRXPOad7xMgXDJaQ=; b=CzJhF2jiDMPKQp2OUpQx+VtLXnW2I200XXAr6JjUF3gHF3uH0/M1R1gYdPSQLZ7Fx7lIuf mkssr4rU0/A4znc2EzhzeQ3xi+Gf2YrfJku77lol7KYfyG5C4uL8Qro0oQsCB8OOrGNUGr 21K2A4lNj5u5yGiYkhzhKbroDjBL82M= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Fp3DXW4B3dj0QEKkbnSR4pkaFzIpRXPOad7xMgXDJaQ=; b=TxwFEXAtSCPKXbeHQbszWeprHo VxqiwXK91lJo0UbDX3RXW4XNyqSskCcZ+9c6WpaI49UbctxbS8wZqEs3nPaFmlxhiHDuV8vPlJax2 YyzP6h7vKnv5ar2DQ7nkf41EsuJBzI5y3wKFEZhWVvhRMEWky3yvIorFwMObUfedc0Ox57hBgNG4L hJdS5UNhY9mxtMwjqVdTEK1iqVrDNyxYcuLW4sE/QpfuP1jUoT8MUTWkYQUjLGPjz2slZ/+T3Dmu8 uN5UE/fYsoduKgaE2VcvgZ2054Ku5h1J2ox/V4pURx85nuWl/nVa5VEMNVMnWzhBlMND2LjjWjjuw v1Yks0fw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pjhBU-00FPcl-Co; Tue, 04 Apr 2023 13:58:52 +0000 From: "Matthew Wilcox (Oracle)" To: Suren Baghdasaryan Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Punit Agrawal Subject: [PATCH 4/6] mm: Move FAULT_FLAG_VMA_LOCK check down in handle_pte_fault() Date: Tue, 4 Apr 2023 14:58:48 +0100 Message-Id: <20230404135850.3673404-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230404135850.3673404-1-willy@infradead.org> References: <20230404135850.3673404-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 65F9920004 X-Rspamd-Server: rspam01 X-Stat-Signature: 51sf3wfexie9owgwcf1rbbbfotiaxsmb X-HE-Tag: 1680616738-468638 X-HE-Meta: U2FsdGVkX19KO7aOcREWZ45CX0AV/ZFlTyuoCZcF5LQaDpi0kNVxTumxA/REUBb+AtrpCXJtBj2F2Wl1xwaoadRQyisKlGkjsxAfdskYbxx+/BYjwjk3s0Y8C7PWmrEaX8K3AXLqKpMzO9dbzeCmum+SzoOF+MjrKBXn/cDhqUuuTdzGdzjDcu74NLX05GRf0XMjCGXMFs/P2SUu8fPAZhI3SJd2lAYCg4sdZvc0D8d98GNaYah4FwU8h6kClPi3HF2vGy3cN+Xc+B1nzUsRZMEeAdoQR7yXnSVJ3J6iFxrQyyruxzryGfGm5oVRTFN23d0IJwVIrisMgODMaVu0EA3eXkFRrom236d9GDnjiuaCezDCVGiWUNWZosRnrrIkqdcI638XfPkWxdrETl+x8k5sAlxUsXWZ4NxggC3WbfBaeo1n/JMhIUS0rIM6RXlM6yEWcFoLjDSy9FSJfikmn/bXH8LBxvkFxY2ouKSn4F+y0NJkLC7K/baqwDFwkGGFE5yF+THVFiEgJzKS2sRqjMr4iK7DpBd89sbBGOhq3u6zN3YrfIHnnY11uVTXsbsbe7Zp+eW0wanvOFtwzjTTllmSleOy8ziJs0oulnH72f66IRbv96ZHwMkCjE+fkzrtYOTLobphuwPQDIuvxGE99dG1KqEUHxWSyBX2Qx8vQ32zwGxt3BQjzMAGx4RviPPPOYy/sQqKMQytarq8hD7sG9aGf4uPJDTR/bPPnY/NBiTO6HlF4RPM+Y0KEtGLugJrXCnU0w+F2uP/rf/s3KJX3AR9bJvsGWTC9CiUG+gjqytlgbJ1M7Ysv+7yLYwixNoVYu/6a72EbQ1HnkZjp1Y+Qk5p/pp+8r65iEUHIqdC326V15S2idM2QdlQdh1hxTz+4Ucis31/FG4cG9ml6+inR10rlQaTEwM4ep8jQiWxVJZMav45EohoaL3ASpLmorwJyn9vyk8ZWD2kflAKJv/ sAaBNarS bcV72/nrRdH4oCIjgSFmVx9eIQiwyK1eIV6JJh8ua2xacht75MctkJOqOyLfYhr1vJ6AM6JTic5IXEVhDEwnXG+a5rYpy+dsEhZA1N4m4L14GFli1ez/uFR0YNT9Llmi1l3XfNWD7EgSbFqMY0ITxoFwgQKVkubucULLwi7De3kuhAwwU152G00tsZjLboQRcUFMZPrhSURNHYpubxQMHzri/akqoj1ki2grPj645laiPYidich/Zd5+1Vg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Call do_pte_missing(), do_wp_page() and do_numa_page() under the VMA lock. The first two gain the check, but do_numa_page() looks safe to run outside the mmap lock. Signed-off-by: Matthew Wilcox (Oracle) --- mm/memory.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a2e27403e4f1..dc2baddc6040 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3329,6 +3329,9 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; struct folio *folio = NULL; + if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) + return VM_FAULT_RETRY; + if (likely(!unshare)) { if (userfaultfd_pte_wp(vma, *vmf->pte)) { pte_unmap_unlock(vmf->pte, vmf->ptl); @@ -3644,6 +3647,8 @@ static vm_fault_t do_pte_missing(struct vm_fault *vmf) { if (vma_is_anonymous(vmf->vma)) return do_anonymous_page(vmf); + else if (vmf->flags & FAULT_FLAG_VMA_LOCK) + return VM_FAULT_RETRY; else return do_fault(vmf); } @@ -4896,9 +4901,6 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) { pte_t entry; - if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vmf->vma)) - return VM_FAULT_RETRY; - if (unlikely(pmd_none(*vmf->pmd))) { /* * Leave __pte_alloc() until later: because vm_ops->fault may @@ -4957,6 +4959,9 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) if (pte_protnone(vmf->orig_pte) && vma_is_accessible(vmf->vma)) return do_numa_page(vmf); + if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vmf->vma)) + return VM_FAULT_RETRY; + vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); spin_lock(vmf->ptl); entry = vmf->orig_pte;