From patchwork Mon Jul 24 18:54:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13325263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91BE2C001DF for ; Mon, 24 Jul 2023 18:54:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E9246B007E; Mon, 24 Jul 2023 14:54:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2997C8E0001; Mon, 24 Jul 2023 14:54:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13A5E6B0081; Mon, 24 Jul 2023 14:54:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 05ED06B007E for ; Mon, 24 Jul 2023 14:54:33 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B7ACCA0B23 for ; Mon, 24 Jul 2023 18:54:32 +0000 (UTC) X-FDA: 81047406384.02.6119172 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id 01D0F40005 for ; Mon, 24 Jul 2023 18:54:30 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EInUKGfN; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690224871; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TWTBYufoYSHkxfGZKIaPme2dTr/Pf10i6Hfp7/PcTWM=; b=23dJhHZSkxpLeZILOgdRq2kFStWbvDfCaT0zCU6vJ/aezr9SV7vlb4E2rFAI5mK0WU+LEB Z5mXDxxCQeu2ORIVy97EZVI221YcstpQFJAIKiyGqfeLgNTQdce8km2uUp8rXZ95H4BfcS 2IYtcLcbp1vq+WXjOUKvJ7YQXCI/NOc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EInUKGfN; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690224871; a=rsa-sha256; cv=none; b=rTEho9ZzhhQwgOql/qXpOi19E3rwV4xO0/LSdfi8+iVef4Cnr0+IZ5vGA2Fi0H7WKOknq8 bGw8YL8AbRmXWtDLVt5kxECLWsmIUMkw1AByjo+LWNEhdRDWsD/1nZsVCaVdLalkAngjmj uWR4KvNyezosyh87yRFRpPi8U4Km4Go= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TWTBYufoYSHkxfGZKIaPme2dTr/Pf10i6Hfp7/PcTWM=; b=EInUKGfNGCuTTNsKCFRp6uzlB9 1fJcmDCG3/jqX0aR3pLqeB7Hfy3fyyoldC/a2lX8d4E4Z+0RA6Dqfj4qjuAvlFXSEhACCF4KIS8YA CfoD9MK242rPZ7Zi1CapBwX3l2XrXyD8sHIWtl7fQT15/dJnIuqKFa9jwI3D0IFIxQXiGB/eIyGQW MgbFEpHyXrSqrRmNd7OZcOekNHH53oejJDKO5zaI1nU3ilkddAlysN4Cviibn2M2A7528Ffzmggtq yof80Uewlt6OaQZ2zNYLFnAa9mWwu7VMTOfLdGS0Qrn4Gfz4V8UeeqDrY3cehYVPDsJXVVYK+Ty+y iaQQTgqA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qO0hA-004iR5-7U; Mon, 24 Jul 2023 18:54:12 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Punit Agrawal , Suren Baghdasaryan Subject: [PATCH v3 03/10] mm: Move FAULT_FLAG_VMA_LOCK check from handle_mm_fault() Date: Mon, 24 Jul 2023 19:54:03 +0100 Message-Id: <20230724185410.1124082-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230724185410.1124082-1-willy@infradead.org> References: <20230724185410.1124082-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 01D0F40005 X-Stat-Signature: 336nez4sgpqzzpsfrxquto6935a9icfn X-HE-Tag: 1690224870-330530 X-HE-Meta: U2FsdGVkX18ICGvDFUqHN4mVsRbry2nAs+xgs53UDc4mcy5tbj5wfXH5nGFkogWnngTTXje9WDwnbwyM1XNo5b/+GpvC2iGnAqsSLuTgC7fEtqj68VDoyUc4zJFg566zKqt23C8agQvXTdNY/xUbKHH4mzePEHy6TaMYi/eZcv9hvYVrrluPcPi94L6fOOZNYxEC6XOnMG+sHrFXScoJws1HC0sp4a+5nN/DCbyf2Z2GMuGoGmUDq3q4vUiBCYUazthYv+6ff8ODhGyM2zHhoKrgxL4XRmJHCQwnb/9eoFdu9RPF3bQF/B+76ZG87hbvTsxjHfCEM8y+EScW8dKG5fztH4txhqrWoFaCwqj1dxNAT2PV8tye9QLS/tHQ/5T8TNH7rhde8GARWhfTDgvaO0ZNfq6JgEQVE+CFJMhFTEe1k6n8A16Q8awY1jG+Ta2+fdSCF/JTZivjRJcTMLHQ5SFgiBkgIPMNh17nDO2kkRGdMYDYrAO7GrEyFlApJoL85twBkNble8prjGAH31Hq5U6TNx6A3UW4mgWymRKPHzW2xhHQgrA//aoCDfvnfqHZsTU8RK5a4D7iAebuUj9f73UcvNGwOF5raE/c7XxqPEucCTxugHFJyr9oaQM59KGtz6CXva/sgouwpKF7MykoKlo+X3BFZW95CT32aNYiFSQ9kEdrZZ37Wc5deW2312shglBJGlm/X7C40goyOjtpxKRYfd9PdQbX3VPa3gWrXJsoYP9h1soDdq1tOMgfyeNthCZ0IL76YO3XRiGK/LzocUzERXRGs7bqJ2/ZXFuEDw65hSTfD/5rDCKTAwMPYQINPHHWQOvHQN+AaRRNhoy3I9DrI9FSfinKok42JnrbF8i7NG/0MhYCz1SYdt9dAbPjLq8VhUSSjhx4Vhg19QiEiWTC6O3GllhPtiJ3REjrQYw8lsDwkJrnDpVH1xHRT3rN7YXToxD+UkAyZWb/7aP WpGeXvmK npfXuVTD14LBKooYGIqQP6OIlV46scGNS0nNRbo7sQKSIkwIKDKXKp9LmTLhYbLtWO4c09gG+VrbGeyuiTFU3lOo06qMmqg1Mb0RjBSMTbHQqw9L4qj8EkWFL3gfwC2mTaek5jBl710N8Wr2r4zGXziHqcLqudGGoH+zNgmvQXJ6j8uzmRwYjzwjqxe4IXtBLqo1pMhobEY4B3y7UMLY76DyzfD60BdHQ4hdju8+VjWU4MtxoVMD2e+1qSw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Handle a little more of the page fault path outside the mmap sem. The hugetlb path doesn't need to check whether the VMA is anonymous; the VM_HUGETLB flag is only set on hugetlbfs VMAs. There should be no performance change from the previous commit; this is simply a step to ease bisection of any problems. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan --- mm/hugetlb.c | 6 ++++++ mm/memory.c | 18 +++++++++--------- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 154cc5b31572..e327a5a7602c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6089,6 +6089,12 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, int need_wait_lock = 0; unsigned long haddr = address & huge_page_mask(h); + /* TODO: Handle faults under the VMA lock */ + if (flags & FAULT_FLAG_VMA_LOCK) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + /* * Serialize hugepage allocation and instantiation, so that we don't * get spurious allocation failures if two CPUs race to instantiate diff --git a/mm/memory.c b/mm/memory.c index c7ad754dd8ed..5ca8902b6f67 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5112,10 +5112,10 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) } /* - * By the time we get here, we already hold the mm semaphore - * - * The mmap_lock may have been released depending on flags and our - * return value. See filemap_fault() and __folio_lock_or_retry(). + * On entry, we hold either the VMA lock or the mmap_lock + * (FAULT_FLAG_VMA_LOCK tells you which). If VM_FAULT_RETRY is set in + * the result, the mmap_lock is not held on exit. See filemap_fault() + * and __folio_lock_or_retry(). */ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags) @@ -5134,6 +5134,11 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, p4d_t *p4d; vm_fault_t ret; + if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + pgd = pgd_offset(mm, address); p4d = p4d_alloc(mm, pgd, address); if (!p4d) @@ -5361,11 +5366,6 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, goto out; } - if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) { - vma_end_read(vma); - return VM_FAULT_RETRY; - } - /* * Enable the memcg OOM handling for faults triggered in user * space. Kernel faults are handled more gracefully.