From patchwork Tue Apr 4 13:58:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13200236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E3EEC6FD1D for ; Tue, 4 Apr 2023 13:58:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAAEF6B0078; Tue, 4 Apr 2023 09:58:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A5AF56B007D; Tue, 4 Apr 2023 09:58:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 949AE6B007E; Tue, 4 Apr 2023 09:58:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 840696B0078 for ; Tue, 4 Apr 2023 09:58:58 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5EA64C0DA4 for ; Tue, 4 Apr 2023 13:58:58 +0000 (UTC) X-FDA: 80643864756.21.B57C7C5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id D5EFA14000B for ; Tue, 4 Apr 2023 13:58:55 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=H5QsLN89; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680616736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+X4Ep4ykQThQZtLA9ICZLzgCY3jpfgFwwzOJniiRp5g=; b=b7LYa5TEJbJw3c7sgj7nYaOdRDk1GFZp+CASuj0CdziW8hjp5cnZWlqEfVrZt7ScYMij/z eEJVrEDQ76quV82LcCsc2uqoaQSwu6dqAkEn3wMhMPCGM0nOj7ZIysOAu2OR//2vWhs1Vw 4dLBCzbNQ3SHOodkWI9ClU9mhH1z3r0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=H5QsLN89; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680616736; a=rsa-sha256; cv=none; b=YmEp9P7x7CR2XD3PZA5420lY/Q0P5BSedBB1GZKDKigdYO/U9glRVYcECo95kL6PYTi9F8 vu8a+Mwm9HdyCWGDACbSJEfHscD0rFx9PDyYEMKyrUQ2QPh9hlLLwrGjAGwc+g1u/JG/sp jH10sti/acPti5DRbLZAyo4u6lFz/H4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=+X4Ep4ykQThQZtLA9ICZLzgCY3jpfgFwwzOJniiRp5g=; b=H5QsLN89pXHLK2DxF+z8kahswh xwcY3DxZ1JJ63xik1ZPiYMfP0hiCt2hH6DLAmD0RwO6stP30snJo6nQkAIRshQB46ggRt6TKnZb/a +ul0nI/Wjm8XXJH/i9Eu4lF3rCyYODJ1S2QhRkF10bcYOWMk/39U18ohLDwMV7wedlULvpuHvQy/o 6U9l368xFkHT9E1bYat91pOle+g7m42oWvMtNSpjvX9WZx8CYeRo/wjjTQq1RTbxeKwPBhBMI4yfb 3gGodTonrj+IEPRJd/Co6SGLo15XYOrccSYAxLWsBl8W5nG0TPiF/EIhsN8YcU/u863dAT14iRUlQ JBqsXQyQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pjhBU-00FPcj-9Z; Tue, 04 Apr 2023 13:58:52 +0000 From: "Matthew Wilcox (Oracle)" To: Suren Baghdasaryan Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Punit Agrawal Subject: [PATCH 3/6] mm: Move FAULT_FLAG_VMA_LOCK check into handle_pte_fault() Date: Tue, 4 Apr 2023 14:58:47 +0100 Message-Id: <20230404135850.3673404-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230404135850.3673404-1-willy@infradead.org> References: <20230404135850.3673404-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D5EFA14000B X-Stat-Signature: qenewz954fy5jccjxp174geme95oz4in X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680616735-721957 X-HE-Meta: U2FsdGVkX19Y7HMQ+oSfEPMTcZL+1TC51b3MQugeG724ItwvkxV5V/v81GGry9zKcBrbrOspnfHPUio8KXp56j07CPrQXV0sgGUmEJzHXiMunZ7xTpnLlTFjLW+vpCfrrS3fHV4sDZvxZekEtMnMttJl0Xc9G+EOwXLmbD+2pp8GuK9ztrRxjokP3wVX5A82LbCawjnCziDL9p7bavg/C6hLYn4hNL+WqSgQYNVwtzACEUq7KG8g5nQ13SlSLTuf8mq+NpNs5zXSj4YuKkT/0Ig7joswmi2JhC0DmKTvHLPUdO4kCpLVp/LJFVfdpZJrBdTVKL7k7crBUZZRDbRdsfBvneE1nRj1tXUxcZ8YhRjNjrpOL4adeUYLqKvHqP8Fjbo66He6fe4MrXjVUmUJNRNMIKScX5dBXZ1y8Zt2TEA0tZWNN2VkSreVxuJXZLXCDHkft4dkbdASBvVdiJKuROoPZUSYsjUNkHkLcNjHtYbnO/pizsPj+adMmDTpJE7v/zWjySrErt/Fd2x3WOLHculhsbNzmbTS/eVqkxAdCfYhyjBcv4NEgu/QBk25w9rLYJQn28AmM1NjLpHulkjRBygHLzeUk7//2a4D380dPEPt8PAt/Gxt/PeHhyXaRj7NtgF4YD/G7Qa3IPFnn6Mn+m4VFiAHZsGpaXz1N4iq0mJ6o0T8FccuPgJgSUUzaaPuOcLrUS1lMoFZimjURF7+uE8tGGJPiodepaz1Q4uXgfPhf8r7o+U7gvmLxN+WaI5Sx3xCrUx5usgTPSZpjjK2luHUe5w8m0UH0pEutxOg0uCpuHuiqEHQaCShx3EoF4iBL9LMdzQEEf96Gljh5uXykwG6glAwHU/+YYyRFWj9pTuErQTkpuTwku3HRmTRgkJN3mj4WvvGs+y7d8ZGTPhpz4TGUzIIpyGvwZL96vh08V7hePtb3+LyKf4VEnlomlDkFD6sn4PU+RnB0eGCd65 Qhbf96lm NPdZwwCtU8lq+f7u7cfmStI1SWJyJ9yRu60RTI+4lNZa5ZJf2DGQugNlhFhwZGahiG5sdbGIWSk+ocfQo474VgtLpBdQjlyWUeks9ik/U5a2i4pbEBWsqRODDg5hb2pgMzpK1Dsc8g0I+6/kavNL1czhqviIHZNTas9qogEK/UcPkHQDbPNYV9lKAOp//pDc/89XyTkxUixAyRjGEXEPizeQqHi9zaR5DpQ7QH5MvBQ0QQNTS5qfL0hjt3A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Push the check down from __handle_mm_fault(). There's a mild upside to this patch in that we'll allocate the page tables while under the VMA lock rather than the mmap lock, reducing the hold time on the mmap lock, since the retry will find the page tables already populated. Signed-off-by: Matthew Wilcox (Oracle) --- mm/memory.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index fc1f0ef9a7a5..a2e27403e4f1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4896,6 +4896,9 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) { pte_t entry; + if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vmf->vma)) + return VM_FAULT_RETRY; + if (unlikely(pmd_none(*vmf->pmd))) { /* * Leave __pte_alloc() until later: because vm_ops->fault may @@ -5014,9 +5017,6 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, p4d_t *p4d; vm_fault_t ret; - if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) - return VM_FAULT_RETRY; - pgd = pgd_offset(mm, address); p4d = p4d_alloc(mm, pgd, address); if (!p4d)