From patchwork Mon Jan 9 20:53:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13094284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93DCEC6379F for ; Mon, 9 Jan 2023 20:54:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32F7F8E0014; Mon, 9 Jan 2023 15:54:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DFC08E0001; Mon, 9 Jan 2023 15:54:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1803E8E0014; Mon, 9 Jan 2023 15:54:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 086DE8E0001 for ; Mon, 9 Jan 2023 15:54:30 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D90F21A0F57 for ; Mon, 9 Jan 2023 20:54:29 +0000 (UTC) X-FDA: 80336463858.28.E7E66CE Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 4A4408000B for ; Mon, 9 Jan 2023 20:54:28 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=EbLgVpiN; spf=pass (imf30.hostedemail.com: domain of 3A3-8YwYKCBgGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3A3-8YwYKCBgGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673297668; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mttq8CcSxh6pk/PDvvqo/9XfYX/awSV+YKJHUec4DWQ=; b=3f/hR/CSqDQ3F9duaDtLOuC+dF1gXPMtJYpUWXv7jIILq9dWbcieBYUdf2KsMLn5vrpasZ w+DWiqHWVSqyzwwy+qblmkR50HipisnyrY0icoa5dRTTKiVMMkFYkLi1AHLUhwLy2FwXf4 +e8V4YMLn29tXbS8z7Ndg0at0jqYaW8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=EbLgVpiN; spf=pass (imf30.hostedemail.com: domain of 3A3-8YwYKCBgGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3A3-8YwYKCBgGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673297668; a=rsa-sha256; cv=none; b=Qnfiy/Q1IjMbNhzE8HW65LXi8J7W3Fp4BsAEGjKt0kq3eq68fLeAftDIhecUAe7MpY6UF7 SyDdm1nzE5jfVMjv2a5RKDaer1FcTutHT7IyMfweWVYSOUKLXbgQTUa1ASxkwW+o0jTNha 2DXGHp4p1sRfrPKvWhmTdfV8jJsyBfc= Received: by mail-yb1-f201.google.com with SMTP id i10-20020a25f20a000000b006ea4f43c0ddso10381013ybe.21 for ; Mon, 09 Jan 2023 12:54:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mttq8CcSxh6pk/PDvvqo/9XfYX/awSV+YKJHUec4DWQ=; b=EbLgVpiNWauCxYsPhZzluNh8i7Jc9od95XU7q4baD/bZuTFbACugKjaBqwn3ZlwjPK DyA2dJROokooe1X9hWF9cqxx7R8dzaLC4TQ2VgG1K6ZzCrV6Cn+bNgWtTm8qKV2EPlQI C3EgQy0lx8995KoeqPGehRxKjsY9DJj6pO/ydpCP34y4VCDsA5lFDA9xaQObsm54nagL gf/rqX49ePSI64VKn0w7yUB28hwRSupVRZyvn8G3inMqKMvhkMvaFS5iHqspfhqeeQGW sJQNxcx4h1OW/3tP5B8wZxgrbXFTXvC0zscvDZLigjE3sBlwiKuCEIrG4zHiBAmIFXhN sp1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mttq8CcSxh6pk/PDvvqo/9XfYX/awSV+YKJHUec4DWQ=; b=Tql8ruPCZqb+8myY2zYuLkZ7yB+i6YqYJIpzUPvumkA1SOq/M+/wQctT0zNeXoLNM2 /SHRnhf9XWPOEf8Sq/xL5ZgCVf0JOobMTItIMOeX+/8cBfz3DGtAeOGn869ibsS7h+oy 3Vy6V5ixr1LlrbQL0XRFv5NQzpe7JhnnXeASDTvl1+pTwCrGdTkOXrH0qGsxjIwMsNOu 2CyvC/VvUQgcKsLq1BSLrP6FNuab5XnMcW23GEECKvZy13S1xqWugxqZaftzuOO5YA+o U8MKozQV/AyA9zoIEXKX4BlWfoUIDLkknMoL+hqHhuADQMGxxVGZMlIbXUFd9V32d4f6 K06g== X-Gm-Message-State: AFqh2kpoYlxXnQRvRBcHENSFuLzzrJ665npOnkyZMHGy9dgz5vS/bQ1w 4ricui9i3g2wkhgMBq7UMnZn/lSgm9c= X-Google-Smtp-Source: AMrXdXsB8fbMiTP1DYP5J5XEU88bTBTK0YYISNp2DJzOoqciXF/+cmYxWMxqmdOzbjRBqH/+xJjk8vQehVo= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:9393:6f7a:d410:55ca]) (user=surenb job=sendgmr) by 2002:a25:850e:0:b0:6f8:42d8:2507 with SMTP id w14-20020a25850e000000b006f842d82507mr8097864ybk.110.1673297667509; Mon, 09 Jan 2023 12:54:27 -0800 (PST) Date: Mon, 9 Jan 2023 12:53:13 -0800 In-Reply-To: <20230109205336.3665937-1-surenb@google.com> Mime-Version: 1.0 References: <20230109205336.3665937-1-surenb@google.com> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20230109205336.3665937-19-surenb@google.com> Subject: [PATCH 18/41] mm/khugepaged: write-lock VMA while collapsing a huge page From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, paulmck@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, hughlynch@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4A4408000B X-Stat-Signature: 88komr9ghq87nocbfowf6fwewkd14kz8 X-HE-Tag: 1673297668-985419 X-HE-Meta: U2FsdGVkX18lI/xHcOlkXg6qJOFd5gxZaDIoEU9PBuj8Riw+bgS9X9x7f3eRc9GQXusK9V/kaddGdORAwVYbgvpy73ZlKz5TyHbk2oHL7zDxASjlPgxuzcx2GIZ0oM8NimJy3V7VWj/4/AZ0OhwSqsmX0xGg7PE7bXf7ch7MS7SdrSHln825gsUBompaWYTGpCldRfvIgpaojEtg5ojK0i52Pu1/sybzr2pNzFOe3uI8Xh25ouLiVhNMN1sbJtPVl44Ft8475Y0z5t917kQGHVFTUtUbKFxmEbfeySaMfmPzt9EO25JFsnEY3bztTV8F84sRzPBq41ETHGInLGu8cZvvsIps2mdfoMKYpHrkpAfXis6hyBsCMdDjpDihNZi88pGmVsO/oXg3TlaxoAqU/SlzFCmyTNEQH6sBtc0uyscAlw2gW0AIvsaK72sxXeLo96hF9T+UwnkI8b8X72GF0e6oSi3Dhr59SmzGigtANflQOvaNpitvV68Qa1aaZpUgaidPTfLF0HdQaSMUg0Yy78j6ssE7PBA25nGiEdU5xxYCN9UKFjLcrqObzK4tRButCsj4nB69Cxnva+qRFcaOTB0lp2vkSXo4UuyOHeWTjbKwZleMYlCd7b9pNwZEQgA+ydFQkSeAACxZec5kQWpjvCew+9o3kQzq1HIWdWtdyPwrM5IDEbDTObHAPtbDOtkOSUTwzPTpHw/U++kNxzejr1DK47hK4jqPjj8piAPnV4UrF+BjLO8bKDuG1BtiJ6loPpXMz+I9QjpvRZGzDYCwQfJ1dHSOPSHQOjkgdKgKUAQoMdyooWvVTxE228tci5U1iJd172lkCtOpVXdFQR0iVj9pBOE/qM/2eMYVLEdVWyoyyi8nkV7ZpbJ79vEMTRuUoet9ixqiNQfRROb/GeSesBOUArezaoluQxCiKAAnONXPleNhW2A2+QVmKm3R191vD/0rWWqslbqBqZGF6Sg VfvQtiCS 6wqkftZXbFkavnJXj5w5fBbu5Zd1v5dc1X/MtbJKF+1wtRx3GTumWQTPNEhPK3+TR8upoVo0vyAkWx70hQN54L/uSl3UgI4iKcZbVPFBx1e6t/N7THKtBetfWCWwHIi3xjua8L/YZxHO8QSPGy7Q0X7ZJR+emsx1NTaL+6VYJTLC6TukJKOmotqZuH6DBRUmMfaab3KwjyAMXlxrg5wdFl4Qs50e+QzOlvK0Gs0ulty53Kr+clQ6qQY2yKgACKkE+ZjEYH+M3FOoAnlxNvgt+VThIuy6b4/ntxrcP5V+opkiEvxV64aGAtaFt4KzkppEBJ2uWIPAKxM5mVx+OU4887aYrY78GcpQGsC831SKvnQCePfk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Protect VMA from concurrent page fault handler while collapsing a huge page. Page fault handler needs a stable PMD to use PTL and relies on per-VMA lock to prevent concurrent PMD changes. pmdp_collapse_flush(), set_huge_pmd() and collapse_and_free_pmd() can modify a PMD, which will not be detected by a page fault handler without proper locking. Signed-off-by: Suren Baghdasaryan --- mm/khugepaged.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5376246a3052..d8d0647f0c2c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1032,6 +1032,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address, if (result != SCAN_SUCCEED) goto out_up_write; + vma_write_lock(vma); anon_vma_lock_write(vma->anon_vma); mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm, @@ -1503,6 +1504,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, goto drop_hpage; } + /* Lock the vma before taking i_mmap and page table locks */ + vma_write_lock(vma); + /* * We need to lock the mapping so that from here on, only GUP-fast and * hardware page walks can access the parts of the page tables that @@ -1690,6 +1694,7 @@ static int retract_page_tables(struct address_space *mapping, pgoff_t pgoff, result = SCAN_PTE_UFFD_WP; goto unlock_next; } + vma_write_lock(vma); collapse_and_free_pmd(mm, vma, addr, pmd); if (!cc->is_khugepaged && is_target) result = set_huge_pmd(vma, addr, pmd, hpage);