From patchwork Thu Aug 22 07:13:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30EBCC531DF for ; Thu, 22 Aug 2024 07:14:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1BF06B02A8; Thu, 22 Aug 2024 03:13:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7D0B6B02AA; Thu, 22 Aug 2024 03:13:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 857556B02AD; Thu, 22 Aug 2024 03:13:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 538F06B02A8 for ; Thu, 22 Aug 2024 03:13:58 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 999A181353 for ; Thu, 22 Aug 2024 07:13:57 +0000 (UTC) X-FDA: 82479016914.04.64C5854 Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) by imf14.hostedemail.com (Postfix) with ESMTP id C320410001D for ; Thu, 22 Aug 2024 07:13:55 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Zz9BVblg; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310770; a=rsa-sha256; cv=none; b=bDmtMh9bTIoZwhZBbx1dqPvKd+ttX5zl4dF6wd7vBi/e7UXjDi25wDMeOEN47YOcOaxSU1 TFOXdqg0YZquwuUuZeYqjMim+LXDDkxN4/wimAD+j/+8iNlSoRRtqy5VbMHUOJ6mpbCMOi /HckPGM8OqUyYDJnGoIa/pyDuHo3oIA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Zz9BVblg; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CO0xfYDzsVihapVFXw+qMWwHYfpsvq03XEdZOsUU02k=; b=GjgamGVcRkJHHguWxTcE1WNFEt7RvRMaVNF5JF4RCTejji0DFzLpVeBSDT5IK/qeYcVJvK VI5CjcYG3B7R0iIFv6pyDZYod5LVqqChPtwehVonAwU0QU3X5KnvyzW++cKo4bHJdgErIw PR8U3tSIALm7zp42Ws0vU6zJJ4CY8oA= Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3db14339fb0so294209b6e.2 for ; Thu, 22 Aug 2024 00:13:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310834; x=1724915634; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CO0xfYDzsVihapVFXw+qMWwHYfpsvq03XEdZOsUU02k=; b=Zz9BVblg4BwQzlvaOnH5GG0JOANPffW7EuxakRGbFQoAwXwHfBRj/W1sR4Em7ZaPUQ in019vvrHFa/yd5qJWFDyi4fvv68XN5pgL8NKWBFulGQ9SKiluABQDSpfcjtmqD1SJi7 bZVg6dgRHziwUssEqAFKNoaaMp/eESxBueChJlyR5fmQD3/gRGluF/tldRZNcyxMJFCU ITeuRPn8f7evmMSHiX1WRqF3d5A1AaACAWJRP0pcD+NGU0z24RNVU0FzKd0xsI8ht7eU 0YBPit0itWAw9bPb/6zVesywUg4vjUPEr+Hit8oBTeTVhOp6eONXGzJy9ZW5ACSSpwuF /rxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310834; x=1724915634; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CO0xfYDzsVihapVFXw+qMWwHYfpsvq03XEdZOsUU02k=; b=ojHYW3tZwOyWmJuFklncjee3w5rY5VD0ON0hKUS/Q4D0Jo4myM4/eyVqUufJyu1dC3 xRZp1uYF+ivJaHR/K6q7g7vji/mFB2BMc30iLHbtXLsS/F1qJN8x5JCnC5by4El3Isqi cb39Djmow1uwMBMUxYhK0L307PzGZDuv5i6ZF7aYhzaLXuVdiho1wxRIyGuoKhm0w+ZO eEYBywFN3YfqZY97Wx7IQ3Z93ZQt++5y2JbO7WP9qdYP7/9BQhiwnOMv73SKOI/1wJ+x CzG2TvSIJr/7Lc+S30mnd+Z8TvdDOWgGKjM2tksxhIAdsUq/yqRS8w/Oa/q2UhQQlRUw Ttmw== X-Forwarded-Encrypted: i=1; AJvYcCUdI2inkibtCKnwvAbKxSEPyZWwaOecoWJ/e6f0B42Ad3740d6RZ7C6gp0jQaZcUSFG4MPGtVQo8w==@kvack.org X-Gm-Message-State: AOJu0YyKcs/jQsAHHd/D+jEZ3tunJSxcoIdqA6Bh/KrVhcTLvjtJMdDE wS5QDTC9pkxle9jAMG3rJA535F1DrZeySqE2JE0urF9fahHQ//gPpX211lQHTjg= X-Google-Smtp-Source: AGHT+IHZvtclVw9AATG67WMtfqsJZD4bpWoHNpEdN87kwQejpgQ5CLrsmOGdaNcwygIvYs6P1O1W6A== X-Received: by 2002:a05:6808:148a:b0:3db:15ed:2a24 with SMTP id 5614622812f47-3de1950bb78mr6135912b6e.24.1724310834483; Thu, 22 Aug 2024 00:13:54 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:13:54 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 01/14] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Thu, 22 Aug 2024 15:13:16 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C320410001D X-Stat-Signature: 83fqmisqwa4bnp9fd6b8udab8473enko X-Rspam-User: X-HE-Tag: 1724310835-655384 X-HE-Meta: U2FsdGVkX19/3sr4cp3deWBQPjOnAao3uRcSucvqBvP+PplZoh1lDilh73Ndb76ZirPc78LnVoaaXcVw7LwaeIM2P4sUo3DS0gp3rwkTWaYiEqQSIkXMLllZf/BbQmnI5ADApXYqeFGZCyCosw3CpXRE1Zpr5dT+B862sN9C3GI1TahY4dQ4HiiWSmiqPH4Fx0KUL+GFNKMo0sB4uGsHRAub4iu7CO9DqDDHj9YTPZMHz6KkXER8uz6K9koEaRdFDNV+Bdp/0hRgynClnW9v1g1eMqgORl4cLkj2sAaVHGWd/0od5smanhYsAsMEOR8vonW3UrpFFE15LtUMQEWvc6Bni5JwWdE5hUioG0Mm8cj7PuQVOYQH/WzUuv4+6NT9oG4A/RWHig8mO3gt3knc03mf0aPm6hfwKbVlSGCe87KOsdH+RO3S0oMM+JgWV3VYDAOffLx7cQPtiMTnU/tIVawmZtzfGGhWgUFmtb1onHtfOfapo+JyaH5d/+udQBAegL8P5ChjBdkbLiXHD6M9idYmupA3OUxAvdR7nZGWBiAgVZVNV+Zn+fpx1sxXooklypnTbTveMR2+Xswe9hqPJRurvRepXTrSDu9f6BMXNaWDnQOSDL9+/r1iBuk2GEjM3gnY6EIHXDiYoFy8qZyYp2lJrYhihTX6HwoGtTM+wb6SPd+gxBaLKje2p7zvP2DaAa3MYgp5qVJkMqMTyvsW7DR416iXt3iplgikxdR0eNJACwynb61jgabJMpA28CT3n+uPqxA7CifbNPt8UyFdHq5ZMnnA/MADq4GA6VdbIaxO+5j49IkjG3ccgxd+GmGIAwLgnEHM+Dmwbl6fga+tXvwWRpms+UoNue0aRCvfq9evqRUXeq7ANSyCQE57X4m+r4FLOGM3D8tVRYkASoBxhrjHIIcLIK1XBnLYV5jqwXiTjvzrdFmRk822NfPCpkUz6D040St9mxCJVrBiPA6 9qINbxzC bsrjVhKPhqSwTBQ1Q5a1ayJksqjDlkN4bNoSuS9bEtNR1PM2arWV93X7+91KFVsHvF51icq2vQJHoD2+jACt45t9kX/mV+uPz0khsIdsfCftnJGHdm5oVI1/6A9oBT0t2vLa8Ps9+jXvUvunKZ3D1JAZvz7CweNi0da9vqXEDBr4oVOY6VutNjMAsc63+Q0cIOESdhCgAqjCjW7FBUraS7ceEKofpAdt66dv0cL/fExEX+5zHImBgW6cl/v+tVxdUvGIQl9r9RRZUIWNmn5PkzCSaPnytVOCRdEINfBJVR4vm4D7Lezlrb+KG1yGNsXMnXWvjOVdef045dcdCvIyu0OK515asL873WBxmxQ1z96u9uaRjyfLGF5oNea8Upk2/mNURJvAqZ8z0j7SJjX52f8Lbl50ZwBJviCGdUiEaNmG61dWjfqLVzyNLOZCxEeGknX9p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. This can help the caller recheck *pmd once the PTL is taken. In some cases, that is, either the mmap_lock for write, or pte_same() check on contents, is also enough to ensure that the pmd entry is stable. But in order to prevent the interface from being abused, we choose to pass in a dummy local variable instead of NULL. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng --- Documentation/mm/split_page_table_lock.rst | 7 ++++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 43 ++++++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index da29b066495d6..a00cb35ce065f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2954,6 +2954,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..9a1666574c959 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,33 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + BUG_ON(!pmdvalp); + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + *pmdvalp = pmdval; + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +383,22 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like + * pte_offset_map(); but when successful, it also outputs a pointer to the + * spinlock in ptlp - as pte_offset_map_lock() does, but in this case without + * locking it. This helps the caller to avoid a later pte_lockptr(mm, *pmd), + * which might by that time act on a changed *pmd: pte_offset_map_ro_nolock() + * provides the correct spinlock pointer for the page table that it returns. + * For readonly case, the caller does not need to recheck *pmd after the lock is + * taken, because the RCU lock will ensure that the PTE page will not be freed. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the + * pdmval. For cases where pte or pmd entries may be modified, that is, maywrite + * case, this can help the caller recheck *pmd once the lock is taken. In some + * cases, that is, either the mmap_lock for write, or pte_same() check on + * contents, is also enough to ensure that the pmd entry is stable. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Thu Aug 22 07:13:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3ACDC531DF for ; Thu, 22 Aug 2024 07:14:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E0966B02AB; Thu, 22 Aug 2024 03:14:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 069A78000C; Thu, 22 Aug 2024 03:14:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB1CA80009; Thu, 22 Aug 2024 03:14:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B14756B02AB for ; Thu, 22 Aug 2024 03:14:03 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 57C10A7F0A for ; Thu, 22 Aug 2024 07:14:03 +0000 (UTC) X-FDA: 82479017166.04.C3BE141 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf20.hostedemail.com (Postfix) with ESMTP id 74CD81C001B for ; Thu, 22 Aug 2024 07:14:01 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=deDHovkz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310776; a=rsa-sha256; cv=none; b=v0++XOofGQuDIa3AAci/Efq6M9o/jzDBNLlxn80G5RCfcR9nB9BvDiff6ugKqAGshYHaiK jKaR1ijvOIBMDtagYpx8eeRuK0I4ECf3jkkIcx62lFlUeSfr7lpI5PUWrx0eugWAbUOKdT nMqChobYsvv+RlCRFTRG3SPcQK3HESg= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=deDHovkz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310776; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d/m1IXPfk7xvDU+Qn41r4NiqBFnWc/gEWMU/opSilGI=; b=Nsvu+TJhKH2gt8jRmL6CQUvitOnSeq5rjeZrY2LOlXPdxeTYTFlnpKL84VOWGGptIqvYLK o6hovz6PDvMO1CnZvz/6IqF4WlYPNqZvq9Ty6UcmEpPq+qxXAdHJDiDZIU86z2U41SzcOZ cgPx+H6DHxMXzVVchIOMX1QtNdp7Lco= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-714187df604so1006291b3a.1 for ; Thu, 22 Aug 2024 00:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310840; x=1724915640; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d/m1IXPfk7xvDU+Qn41r4NiqBFnWc/gEWMU/opSilGI=; b=deDHovkzQGZGhdiJkun0B8NMbSOZ+d9G1oxDR4gDjuU/2ilfB563I48VwPxV3WLkN5 Or3b+WSTpmA4I543zdOwzgzbgFuFqzaiBBi4BDiye5zqFznoPz3/++S9RMOe7QPpowXc t1IvRqOw0WLlZvHArQ+XUW44HB1iKka9z3Je4WFVcIQfFOewwHdD9epcemxvN7zqLv9m yiuuyT4wmU5n2KnTPy34D3Vede5r9xLbbQOfZpstswxbGwK00u2KuSLzbY1HqjUnosxM gxPQsQLi6honPTYovg1HL0wJPl/TuUtXt5RBDf9x5q1uiSpznzUP/uw5yJtCOr633lud 9I9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310840; x=1724915640; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d/m1IXPfk7xvDU+Qn41r4NiqBFnWc/gEWMU/opSilGI=; b=Iz8i8KgOdGg4iLWfd9ESVlMysOZUaRSJYY1wUDsBzhabOv7SYHQjkeQ56QD6eqQc7K eaB9tVnd5iGa0duyqK8a7VJohZZVMh6E4THn88eS/PLXBdSwGAkRxtkhyKk2MXVpH4pU 165axnOStwFpSDPShViOFqKk/ljty5BHHIXeSkNZO61zj74MLFuSDwa6G07TPod24Jow 9xaKLa4dwMJqRtWsWaRd+/g2lQemiyEGCGoFp/m+sFoIgpHly834xl6T38+Y2vK4Q5oO zyDMFykYiduyuUL+UcQFJOdTGJ9Uvq13o/ui3tpZ16MK9yBUPqwTNQXz21I9/VudbdIS Gzsg== X-Forwarded-Encrypted: i=1; AJvYcCW9gamSgvw8HTOqQmiAHs42JRHJF1OAR6vbXkdO9zoXyweVnZJEZJMq2DL0gDDRssAkFeY4CfDQmQ==@kvack.org X-Gm-Message-State: AOJu0Yw8rS5rFNd/i0A4gFWNnfGBuKImnP7zvSJ3zyzD0Cx+GBQ5AYF2 gMOI+cuY/WoZB9ZAZs2cfconplGIjtTvYdc8dAnP2REW8WR0NlhM1Ey4D/GEDNM= X-Google-Smtp-Source: AGHT+IEhsa9RMy8FC6eeC7Dc/qzW9nWoiqPGY6DtNfgP/dDTOo2T0oKgHsMPC17UGp2n78kSJG5uNg== X-Received: by 2002:a05:6a20:6a2a:b0:1c4:9c08:5df3 with SMTP id adf61e73a8af0-1cae51cf01cmr3205085637.7.1724310840022; Thu, 22 Aug 2024 00:14:00 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:13:59 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 02/14] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:17 +0800 Message-Id: <7915acf5887e7bf0c5cc71ff30ad2fe8447d005d.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 74CD81C001B X-Stat-Signature: 9ss1oeeu644aredzpdhatoisfwxy5yfy X-Rspam-User: X-HE-Tag: 1724310841-358690 X-HE-Meta: U2FsdGVkX1/iHuFZKFhUB9ESKFM7EApR9AgWMGMsQID3q6ow+op3hGLRURa6mvmUCZjQnq9u2dZUaYPbT046S5r1IqzcGCLRnYY7lHhvD9Ggx8cF+S+KUYXlw2BktCk95HyNuAqmM2ICXsfrFrZKj/YqP3f3pzw96q/6nEdu4CY9HML37pMDi2+SelYHEs9GOuca7DTEaMGz8fBXSgZ6sbXmUBPArwXS1JBeCKxB3X3n/9XIbvKR/2e0X9UVv78REnxsocjNTJONYERaHUOG46NWaCXjrkjwvimiR+Mql+xwVHZxDeuQioDcys5i5YHyi9FG85ZYvtcK221NCbbasHYlq9jyMYFQLKWOCtNpV3hYgFC39I/ypgM0DtQUDZT/kt5N+OqU2J4MfnxVeohfY2SmHkmESAlsuH09ge6ujI9TCMvtkX306Dh2I7/wuCsmbym8qg1qYfmEpjih5gYM/7V/k0JB7kdnuPLFq/pM+x9aPpveOVqn0s0XwCo8w7vwddmNVxw7drUDtbT1NouAQHzsLa8QsktQIyrs+DEJtyoa1e2LbgWXa0EgqQ4vuGTi2Ky3nu3H2ufPfjIbk5ioJz5XV+1QY0jXlqim36Wfrtr2ga+ma0VC62/u9Mdic0IORPPtqwO3krr4eagwlNm4YMaWmZzW8mDgoUjv2LcqyDnIi4/D9ZUA0NSj6C37JzlmNY93vnzDwOobPERaaxDhovUXzzjI4uMQzivn4kwNrp54HmOTEfh5r6iakjKPmML57vHxQpE0pekRR2CLZFdy1XMxil7CYwQDr2Qbo0Ef3nwH8XPrIzhMUMJHMQsRHnZq1pm62xBLSvKMIPKgl9NBvjBFq7T7K7ZZdMoVV5LzvqiTPJwrmPDr2w4QdfiKMjhBffaymvmMajesoWMz9Q1TqOsC6O794GL+qlVw/4ljYYSB6WvXvyEx/PvCfLWg16mH7OXiDUM2N1nKt7QGMLD DUOH4E4G jUCP/+VY8+hDpCJLkaEmfrFVlp1nMDuQoc0llWTx5Cgn2AHildBcMT2kqPuoS7FVf8JA0vt2/NFj0d/xjPCHEse/tBV2EA61rbhMj56PPaF9ridU+OqWIoXyuMvEQnyMhYHcczFXScbv5fY16w47o0Cfk7I2ctqPayiV4aTPlHL9mS44aInvmf53fIFPUvSUe2sF+2G/P9XSK9oCVyXawFM+oHjbeHP+MXAgI/844V8dWYbzVyrzhIzXcmMrGT8YgH39FyBatchfo2L1EMfU6DKOeR4+yGTzByr0MLvidWG7HF2OTZ8tXvVf/WjsZlXSAPmGpkKChk1xNkvCrnyLGP9fyUFPQZ9HgDTTwqEJbGg3saM/6SiHyor3dAZn2VSjj3OaJ8FHptz2zMPSNXz4cRy4nIu02xNyIHYn4n4giC5M9GmhYrULAMGATvM8WCbsdZ1Aq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In do_adjust_pte(), we may modify the pte entry. At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the PTL held. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/arm/mm/fault-armv.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..de6c7d8a2ddfc 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -94,6 +94,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,16 +113,22 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; do_pte_lock(ptl); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + do_pte_unlock(ptl); + pte_unmap(pte); + goto again; + } ret = do_adjust_pte(vma, address, pfn, pte); From patchwork Thu Aug 22 07:13:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F8E2C5321E for ; Thu, 22 Aug 2024 07:14:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DB826B011F; Thu, 22 Aug 2024 03:14:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2161F6B0126; Thu, 22 Aug 2024 03:14:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0415F6B02B3; Thu, 22 Aug 2024 03:14:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CC43B6B02B1 for ; Thu, 22 Aug 2024 03:14:08 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 93DA01A13DD for ; Thu, 22 Aug 2024 07:14:08 +0000 (UTC) X-FDA: 82479017376.03.B0FBE7A Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by imf20.hostedemail.com (Postfix) with ESMTP id CA4D51C000C for ; Thu, 22 Aug 2024 07:14:06 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=lVp3FlNh; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310766; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O1mpKMHwwZ1CFkcRRT4IxjhKZ5nh5A3TknIoqewquLI=; b=FlD1WTVXqxPCb0bYowObipq5U/RKwVLRh/4cpCxIaWMm6HCaprKpokzTE2HlIyGjz5Bota +twsJv/QhuLMQWbCxsokIBBD8QgwGKgmJUrVfEiTEZ8O8V91MzmwbPr5k0DfPEj8YtDDuj O6Zn0N/TOPq0reKC0hhJgxUIUYMgYA8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310766; a=rsa-sha256; cv=none; b=nrwuY496XF6gJgRXkMoVUkOqJK3Am5tcIKrdRRALZb7+S8oRdr8/zobaP2U/WmHseIBicS Mx/FI8r8PwV0ZT35kcTSbbTRwReM/Gfbk15bwrhAMkpcwbAB5tItbrrL9/8/g9XI/FXh/2 O5V/YZy6LLunWNIeiJDR6MFB5lijh4A= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=lVp3FlNh; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3db22c02a42so272361b6e.3 for ; Thu, 22 Aug 2024 00:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310846; x=1724915646; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O1mpKMHwwZ1CFkcRRT4IxjhKZ5nh5A3TknIoqewquLI=; b=lVp3FlNhqcnR3ebTMJGsKCPrtHzMTP46SGOMPvwx1bEICs0za3fxyfrJi44LVXmod0 j4UC9zWC6hIgNBJQzoU/Rr4MhWGNYD9ZG+LRu21pNBPYUqFAfFKx0EvjclY2PiQmcaNq Y3dq4v9nvJfbMz02GXHp+Yi5IVWtYsMErlNn5v2m0VMb7CMqd+0ElJI9c2Zw6wn/gZY5 mNLa94jR8r1NYBeo51T/yqKpZ1IdT+hpyCPNdQYVDIyjgFbw0bhEyrNeE0JOX4k6cGle orPodP7Bfx1ri+DMKPYAeSC+nZDk/ZM4GV5uULouh6s/ed3j+zX0Wa3iy6E8YXvY8qWS IhHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310846; x=1724915646; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O1mpKMHwwZ1CFkcRRT4IxjhKZ5nh5A3TknIoqewquLI=; b=FbsckIDtlFMfO+lMYU3ua0KX2BeUUFlytlp+LPqX20JJ4CjwroYof6DIR5V1EERkDI tONuxkg3IRENB8tmkdO+cHfe9HSAx5xYLHmWkrXRfahWvHciDROyDlqkKhZzmXqZ7/W+ 5MvDA7Z1uOIO9jjjd3MiethZ+dZ+jNUh2INAlhC2C5mmunuicBXvLOErLfC5qZvA22Tz 6NvfR1pCshIBHS8HmoAyB0WHmjI1RgT/GOLSdN81u/vtUVl7da53S8Eqlmwaf4BcbVaf HjcExiuLbsgygF4YXaaOO9vYv73GDYyBKM0RfmcXjZnR5LhY0gqUZB+9dYin5hUp2xZA vanA== X-Forwarded-Encrypted: i=1; AJvYcCWEQQC4edPSGNb25AocyJApjKKBn5mOXOU7BQtDrp3j6CO2QJoEd3EVTqaRTWXAXEHYnC0BlXvhRw==@kvack.org X-Gm-Message-State: AOJu0YznBfVP+B0XJsphkVO23Hw+uYd44yVkLsEzheKB9Hi/Pl77YAps MsMd210cx6jCJ8Gu4QzrwFMHwYeGhC135LhWjrIgIY0cXRoVHXgbam1MtX9gbZA= X-Google-Smtp-Source: AGHT+IFDX2PqWoIoI6wCA8pFxmw/BmBf/PDUvtd9hijDBvTgLhn0KCKCIBnod1yVuIa/+/M4c6qbig== X-Received: by 2002:a05:6808:1a08:b0:3da:a16e:1764 with SMTP id 5614622812f47-3de194e0e97mr6243309b6e.4.1724310845744; Thu, 22 Aug 2024 00:14:05 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:05 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 03/14] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Thu, 22 Aug 2024 15:13:18 +0800 Message-Id: <89ae94f722996b4ee7ef20d44cd7f5e2f0495c47.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: CA4D51C000C X-Stat-Signature: wzbkdifiqaeaptdggxc6mydcftqu7n11 X-HE-Tag: 1724310846-455938 X-HE-Meta: U2FsdGVkX1/iFFYqGXJDpsw1TtR9VaFTBW+/rQNGIjzs+Q+lVxTqZSWBPThbcJsKOA/DnTND/iggb1bhXAm/C7IuBwU+C1mDoARvQkK6qUxrD6TTy93S5G+xSMqrXwXrGTIHLm1oNYR08jhj3X7nZvPqvyizkhSnnENkX4X9Qnmjexf6fiLLomajPjaP7ARCQUvauEJifAvWTKneJ/d3+perdig4hk+cVuCwwhegSN3Djgo2aqKkmHROb3zlxKOmVhLs9evL41hJUMEHQ8C34/Xp4kLZv5PkpcE89GTF0b5XdrkOfBMzjTFfQSShalg4G4MpnLt04wJBs+DtG8WHc9yu/qs5sY1y+747JarvQ2TuR7CWWzPbpw2NR/gPOQdgw4rjdRAf8DVb12klYEU9eJn9YdfBxZT3DMlBF14fzQKYXkYOxX5He+KDRphpYtBykEm0+upJ68mcuHtmnABpsrO+k9BKatWf7ti+GBubXvqO2Woktp2sP+xlymfaPLldfLra3jeRTCBt4pBQDiO2VJXWIykqx2Wqsr9Pg5Cs3NZVTjr4TcdWhommaB9K04/UztqC0UxjedyYFE9W7oljorpiMgtYWBB8KZUWq5f1txkZ6lnQ8BCMllGDscJztW1IZcDHu0mXglSBko+XXBPyAb9QopvMJjj6SO7riJnGK02CO9H0ZP1ELdM8sI9+4MxJQST3/b7T8GKAjsyQq38EY1mVXlWhvxKsRFa1WrYZG3OiOcOwalniRN4QJcCevbv0xYMARMYKU8vlYIK1jKDVk+9SDihm3lvL+VlmTtQpeFh7TXnOx898AJnMzjCW6K3FIwwqxM2loAwNCYxkyfo0Qc+AQmRdd9I5sk4OB28IDFb7mbxi8Al2wfUyw8sNwMTiBan2OsoQ0zxVASh9y5G4Q/eTc6ucbnHzzxZXLQW2eOMoGnwrv8TH6+d1JJIgS21NENBQ7wKCbWV+3R4VdUG B18VEUWc vBA6W1bs+05X4RJ1KHudormQlB79fcP5oFcGj8hK7mWx1crSlLcNtXVYhIJPkCwBUQGK+LxbXn89XuM7SQXWcXGdVDCWYeDq5yJLIGNmQ0I0F4hVmptZC20rbwOw/MRTrdngwIe/QAvwugqxxGi5AZJIQ5aF9uAD+gZvbMDQMdEnQQS2IEdIpelmRA36nAWfLM6rwYLGL7OYlelM52EV6/7ncLY8LcBRVNSyWCcALSmU8fiQVjj3aUuKzXY+X4Pt9lp6pZhOSXANr6OuQDsqjfoEyT0Akow5JTHA2Tnvx0vl5WbFEQ7YNPyiCOMMaIR+gcSvjMDSaQNgOVL8QNgD0HyqafqPV7+vzFYAOJ2pVTQkZ+E1xNBR0Au/cunsbRiO9KsmlOMOYBykhbyxRNF29wLNVF+osYMi7i4xwQeRav5P0NBavoyR7AacgvepWFV7Ba0gO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Thu Aug 22 07:13:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7DBEC5321E for ; Thu, 22 Aug 2024 07:14:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C5996B0266; Thu, 22 Aug 2024 03:14:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 524076B026A; Thu, 22 Aug 2024 03:14:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 350086B026B; Thu, 22 Aug 2024 03:14:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1061E6B0266 for ; Thu, 22 Aug 2024 03:14:15 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B4A29413C7 for ; Thu, 22 Aug 2024 07:14:14 +0000 (UTC) X-FDA: 82479017628.28.0DCC216 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf26.hostedemail.com (Postfix) with ESMTP id DBE61140014 for ; Thu, 22 Aug 2024 07:14:12 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=P1k+wyfA; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310836; a=rsa-sha256; cv=none; b=dHs9SH1SlgCpxEnsdIoqIKhpFG2OqfqRXU90pwL+ryqv6X5ID0/M7ULwxUWS/zwtyQElJk BKYJ8hL3guwUNyOTXaz34Lxq7rnYpscE5u+pFrMU4px24wBzD971Ecy6Cb1niEX8xMRYdI Mzaw99oxSa4KuNmVUWdi66x2H4Ga47E= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=P1k+wyfA; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310836; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jnbwyZ8nm7OA7xAj7wrWrabgK1t5zgYyH3DelQTt/t4=; b=tyfPfyBEqYsT4noYFgMiTBZ+VKBUeQttrJKoegDBaOf//BV1ujafeTCI9VcOFEGZeF8ZoF jDHsG009RcM+1x2/yaLVwNc/wEA6lUDo6VRx3rxX9ATGQ/akh4eUALIsBO/DoPr+25Xmz0 hDU8x9irtbkVBgN6UeBPLMnmJk348+k= Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7142a93ea9cso394257b3a.3 for ; Thu, 22 Aug 2024 00:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310851; x=1724915651; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jnbwyZ8nm7OA7xAj7wrWrabgK1t5zgYyH3DelQTt/t4=; b=P1k+wyfAMO9akuaG95nIZ2kM5xVyTfBEtHwRKLwgJkscCoOwfMALVLEshVGxdbx6ck yKAQsVRvBKlNyQUoJBX00+lDL/phadY68cPreaUtKV8NYP+okSnIaJGYzTdcsfbTdU3G J851DXMCK6/AR4uHPRBbI6a8cstpGw2qMzbq3YosRpzRLBXszRqHhE26CgWA4tq4m2na CwbuVCRxfN7VD/xRGTmImgJ+ix5CM0BvgXLeV+5uVRSCXLEOiufWO/7D7QycLWemzHqk cRbKjyp//tMvolNfo3aEUzlcgv+/k5M8ZGPO8g9+R/sgAafcyEKjtL4vHoBwdT/D6ZnN HHOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310851; x=1724915651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jnbwyZ8nm7OA7xAj7wrWrabgK1t5zgYyH3DelQTt/t4=; b=o76ehv4aDyixCq6TNHl/jtvDl4z16HsZepsuDXXBjBIvGw4S4EHZkznUbb7cgFHvL+ kM8NxokLDYtNJQBEQ+7DosDL5pF3lnFZvjPRCDt9yz2UzvC8RljAp1zGKqx207EQLhlZ iavHm0If2QjPIHhyUKF5jTzc42EaVlpbTLpf0m6bwDVlvDXNvXbOMIjpHcymoNPF92Nr frvsggN+FvtoouUMQPNaXPRc/5U4R6nfOwuhSqm9fi4Ph11mh7+uRVLOOXQRZ3kJGRr4 jszOE7w8M0t7R6UA9YVkyfk47u6sztt+R8zyq1wjphZNZnSyWeX7yoWdcex9MYx9o+rS X9UA== X-Forwarded-Encrypted: i=1; AJvYcCWch4dBchAPkNcDJJcQLNfDodvuvalUbGvb0v+8ZWPpEFfbrd4FfL+S3mjiWtfA/ADH4QEsLn6MVw==@kvack.org X-Gm-Message-State: AOJu0YyW9GzpLpP/Yl4hyJI7+nlp9PsDDhJ5wkxI3iTfLpiEahVff82A YyrZVtf0Hoc39HUJNRJ7CalUbKq6AhcqN5DOAvV33Mn7XXOXTPwaghMxPyUyx+U= X-Google-Smtp-Source: AGHT+IGXmNRuv+mQEijx6lEwkmYjHk6XirTogWRG+N5Gk1h3xzTz9eMzqUkZv9fzF0cNQyeHFRUhTA== X-Received: by 2002:a05:6a00:c93:b0:70d:3777:da8b with SMTP id d2e1a72fcca58-714235a5404mr6267050b3a.25.1724310851578; Thu, 22 Aug 2024 00:14:11 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:10 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 04/14] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Thu, 22 Aug 2024 15:13:19 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: DBE61140014 X-Rspamd-Server: rspam01 X-Stat-Signature: schomztzbcku1ke1m86b9fm7z9fcq7k3 X-HE-Tag: 1724310852-331930 X-HE-Meta: U2FsdGVkX1+w9/Dj/R0dl8eTAjFDvd3zT7SQvTXGdGAJaItAhMJUXf7HKp5dIomd7CVmhAd7DN2cLaAPRR/djW/nBQ5i+TSv3tAS1Vv6Ab5Z/ykCiztmWfAllWxspLpMOBUxFczPB91IRMCehQ4c6IjU0AqVoYv0O/YdnhxyORezMuQ9hDhR0/BbKx55F2aOTw/ZQcW/r6fto9rCjS11L2UGSgqzXdYir+UB25ekzGV86eqn3775e6dKLBzlysMqQuypYYsS5rvNrmrldnLcxF+0kUYbV0tLqXQW22ia1CQj5Nl8qbVb7HjZjFqA7D5Z1ZANFqGgLPlIhRpWVUsvTqtvqy2j4/VqeGSIYyX8wvdkL1uZS4UzVq9q7DDS8jpgZHIyUxPgDzStqM2Tt6wDUrAjUTzawD2GIS3kKMntB5kBM2abPJ/BbsARVOFXJYfeZjukLzMKBFrib7kpiv58aisaCOsUIfiO9W7PhaBziAoER3oe/ocAIcxvZbm349kCPwRvB8CU/9G5d5QOHjCF/uXnq+KHAP3tT/3ScD8ylcWyjIFYmoEWb07Xd0VUOUogZp+jf4SHxgQhI0KN6hk7u3gAS94lzIAe+VZk16NUCNhb4oiWuMAR+iLzQfGvfeJFtIjDQQAkjni7tVY5mg+l4ouTsGowYeHlIpW8WFV/JjWvk4Mx3eeXGLqOne4GSxngFYpua9Dr3pWnVF9yno71elds811KAY1DWRLopaG44xFP/3djDAKtTfu63Xwa65xzKMSMWQERnaIfpY6IqxWMLG9L6iTZyv2XltT+Fa0eiD1xc9RuZMZW/nUnbHFFS8jZ/bO4Yg3E6q1X+yjF454eK7mxDma5Wbnhlk00gZR/uH0J5q6vTvAiJ7W2trQXNAC0ltk2a/pyMYsGmmuuC58AcND3hof5hZOruLxtiNHH35EhVAQJozMqTa6LwJiW5+d7OxfArspMtg93WBC+Q+k cQwVzvSW mjzz3i7QuIYs7vxBJ2Mx2bCcJ9XL2SSZGGxkMjPhyHN9xm+nWeM6mNw14fnhvHir9lYm8oMGoa5IcsaYvvOoTx3aSKn8WrR2JQ0OeHs869w3tRq/2LXqk5LoxFxxD920QndZL34HB7QXYFFt/T1AJt/uKiLOOACTj4vt7EOwd/vzCbicVA3hXgkP1iX0TPnHCdGynKteNAK+v0rqWgS6HmjI+47INga9lLeO5O4bXls4VM9m7O30RuNVHMU3gSrkUFqtBVp9cXjkfZuAVuwER+Ru0MqPPLv+5p0dT+lB5efcj6MQRL3/DEIXxJTfYlpF9240UKLHwnhyUUjoJEwiaDbJ0rAXt+t7mDJv6MPOS2ccn8Tj9/Yzu9QXo7zJM11xSW4jS9xyakgIjcF+oMZPb69C95BfIlW5Fpm9JNO7qk0JT9PULJ5n4TWvV5GnLj/VSGqH4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 0f13126b43b08..c98da9af6b9bd 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3234,8 +3234,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Thu Aug 22 07:13:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8962C531DF for ; Thu, 22 Aug 2024 07:14:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EA408000C; Thu, 22 Aug 2024 03:14:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 76E3780009; Thu, 22 Aug 2024 03:14:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59CAB8000C; Thu, 22 Aug 2024 03:14:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 315CA80009 for ; Thu, 22 Aug 2024 03:14:20 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E1519413C2 for ; Thu, 22 Aug 2024 07:14:19 +0000 (UTC) X-FDA: 82479017838.25.6B64A16 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by imf14.hostedemail.com (Postfix) with ESMTP id 15A81100002 for ; Thu, 22 Aug 2024 07:14:17 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EKJlXDEr; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310841; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=toohyZrQNwk2kNUrhQFlaPWHYnggLi3YB3M65Yq8QzA=; b=GNYqPOuhHnfkk8Pe41lJrlTjY44uHgqBf4ShIaY4N++1B8BdFZDrHCMZBMghVyj+DRY+9R mmnGZWYrvgMJ+ZHNQvyw8KXvv1ww5DTHDYLQCV8DcIm/T+TkJ3ikriEyyPpnJH/y5I2QlB Jn/WRlvOfSQ+j++nFymwkKVZA4ly4WE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=EKJlXDEr; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310841; a=rsa-sha256; cv=none; b=jhhpmAOvAO2jNSySnNhx//a69xIrIqi6AYhD8Vac4vyIeAw5fjCWawNhlgWYYSi3jn2Tr3 iwGjlCj0pD81qm+l8vVYr4PdcZftP7C3ZC7PP0xHdatK6Tod4DLzEHvrDYby9gAmauMuPG DF2EjHZ/KU+lHI2rm1yDh+aa0K+C2mg= Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3db1d0fca58so279836b6e.3 for ; Thu, 22 Aug 2024 00:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310857; x=1724915657; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=toohyZrQNwk2kNUrhQFlaPWHYnggLi3YB3M65Yq8QzA=; b=EKJlXDErYzpCd3L/lnlr34armL9lGqt7nLsi9D7hmb+vOsj1ZwShdZEeqQcarHY6sb MyQtnJ9HWoLVGUNPBLF2dxmXife2OeAQxvcaSE35U90M+XyfylesMogAfxiFJCKHkEvS sMzb1PuN782BPiEitiViWOzPgfqI+ANJPN6dmtwAWI7c42T2+AFQ+ONZJrvDowjx+TFy COiWixOQTWDgh5MaQLyb91qRbQALkRfuE3GCHlOxdSYCb1TEX6Xwh8TrIFy7luzKuEGO 3Hwg/HNabsUvh3OeWEr0ypSu3Ay61LgZFZQJ74q2oGGgH6jSehCv1RZUXsvdVMfyvOMm qKCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310857; x=1724915657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=toohyZrQNwk2kNUrhQFlaPWHYnggLi3YB3M65Yq8QzA=; b=Qx4gypXLkIy65VuzvpsadtB9JnFAjtAodqNZZ3TuB3HIlr9FGXDNi95dDsFLlpty1k FIzHybYAY0tDrvtODdeuFiFWuCkpqso7lQPjdABhsiC6RkjQKoR1QYB/P1mo2LAtgM46 CKA/Wb5XvcFrCTRs4Nuc+jji1XgTY3+R9Yyo3TE3iAWLl1llwJN2/8iHsBwOxA1nGAKv btpOYGmCJB6q+AaviN5+2cLrHFHP4Tho1B5eSbO9B1pyE04wtecLWChydyVCmqCORxzs MQc6ZgPjjUVBhUt0+oF1kkCkBprITIv/5Yoj8rJRY3en+/JLafsoU4m+DQfXMyWU/bux 9V4g== X-Forwarded-Encrypted: i=1; AJvYcCUirZAGqLRpilfOToofWqlTv8jeVjnPbRrXDUqs2H9X1zMa/R0jbCF7oT2jSFka8vEoljmWjXew4w==@kvack.org X-Gm-Message-State: AOJu0Yy9J8MT2uVwFfGjBH+IDbVqd4AsDATfdBGeEqU9a7KruqUBZ6rt Y3DXsVNA/6kjS7HtwPPyXRzRESkG5uJu7pdXwQaI/2PQR1bkButgnV69DLsdcJs= X-Google-Smtp-Source: AGHT+IGgjighdWtbrKc2Se12Iutexgq8AwJLXy8Rdygo/b6qt6ZS1aEL6K5S14xFU0XlAJpbImifsA== X-Received: by 2002:a05:6808:179b:b0:3d9:38e2:5392 with SMTP id 5614622812f47-3de195c380emr6158328b6e.36.1724310857179; Thu, 22 Aug 2024 00:14:17 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:16 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 05/14] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Thu, 22 Aug 2024 15:13:20 +0800 Message-Id: <2b7dceaa12da273c6decf92e3bf2ebc4f4e85ec5.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 1s3nrsfzqwdb1iouuup9kxpa8p1xd8u3 X-Rspamd-Queue-Id: 15A81100002 X-Rspamd-Server: rspam11 X-HE-Tag: 1724310857-220935 X-HE-Meta: U2FsdGVkX19nDgcMwDndnFAQbEnyxgJRTsrl2MZS3KiUzrlc1cEwut1F7XLdzYEUXA2UCfSfMtdnBjNv0OJ5dGpFEBL3cWH7kypnqiMaIbWGL9AV/2cXREDme0mjc+4zbTcCgriq9Q+ogpKvC3AxuSaQTqLYSWc61wc/IyxFYSn6FkI6HEWjitndoqzmx4VLT4HWMA+fILlDsNlMWrtADTsHrHjD9ukSKNHUC69W+tT/sKAVbAcgADmQmTikz0z31ScHYYhJHYNluFuxZREqajk+/ZMMJj5PJhXNJmhY2fQg3OaZ53Ssc31c3y847h15s0kKrTkHA9Vwp9Su9AXuxfti0jslBYeSGU958ApbjhcM5yERUIZb/x2hc2CseBa0KwpYXGyUuBsMQSbPfwxbUf2gYxxbhAdXr7KTtVaKZl1MdUGNGPe8QwgzIqs9yDdRO0OQRwRDwfOnD+mVvrN8yhZ5iS8J+RaraVTdc1P9/tHBN+dRDKpFL+cPhM0tjPlBGKA3mfVFKf6ciCzGEpSkO7CLkCmno0H94VV22XBz3a06VRjXpCp6lEJz7/ZeFMypBxzG66nu40XNMW+G28SRakq8UdbWTvQSh8MyphimHnQT9+9JHAsNEVvoxB4tujAq+ux5+dmHal7jTVpVn2DZQm1/LYeB4zRaaxloA6+jKVyzbGvYc3fqzxSQ7lXxPeV+PBsLSSts1kQ9wZ4df2/uX7XqpcIHOdgR5ilhMnQeZ3uqdHnZrRJzRVsH8Xu/5l76hWkoCYtixCJ2/OzXudh1WTjeFBlk49JrzKEf2CjvjJQX7kKTfce8G2WgGEktPCEjJ3ykEwVX7DdYj0S2Pc4kaTSrZBgl7wQbaipaKKh4r5lREtPmp02P6Jd0t1phqZWBVPzdPUvbTeZGxHtvUqsY1qWC9SFpKYHUFmbVgPhaqCDDR14sZpGEX4l9tz34yG9mJzmW9Dn0kjuvvzWK/6m SFcOkGQ9 mAONOBF0ImNXc/GK3pvZ8nB+RQJje0B09kPMxMjtr9rtqMejQx7wzKuEw9i6mj00+wiouMA8H5Xy7XE0Za9lJOoeGtGe/gzXuC9OW+Gbujb3vuGVED1up/t+R/yXI8An2eAl0uNZ4jyPh9AWUucv5yvWWKy/zxHSPJVRvUK6jo/qx5EbAala1vO3oACiD1DkBvzNoizBtDt4OFaWMOCMK4ccPCC7jxJZbRdKRi0Ly4oIRMhlp2KCo/sHyCDb/7jxJsVcllOE0We25juSbYRRDZcNY2vJiwH++58wuh/DE7Hxp6YgXoXfh7Em1Po2rvVkRe0fL+qTR2klAOcq8K/pAY8GuA21OMk0F/NY6/hA2uGWV3FvFCA+eeHnRr+TgxudOv8T8GlLnhK6NzKmLMoF16s5QysfcGXewS6D8LLGQ7EHHf3PN00SVj3m+ZwCdztWPaKuS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4a83c40d90538..53bfa7f4b7f82 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Thu Aug 22 07:13:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 120BCC3DA4A for ; Thu, 22 Aug 2024 07:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA4128000D; Thu, 22 Aug 2024 03:14:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A074980009; Thu, 22 Aug 2024 03:14:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E3628000D; Thu, 22 Aug 2024 03:14:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5312880009 for ; Thu, 22 Aug 2024 03:14:26 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 14C951A0FDD for ; Thu, 22 Aug 2024 07:14:26 +0000 (UTC) X-FDA: 82479018132.15.7EC5273 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf20.hostedemail.com (Postfix) with ESMTP id 3D7321C001B for ; Thu, 22 Aug 2024 07:14:24 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="FT/YbMZC"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YTJMrM+GLhYLeZwUuiK6dImcBAbTGjqYt196yQPATUA=; b=2ZSe1v9Daq3dSdEBawOkERjAEfBJuHfwnNp3ww2wHnE9xdImVWY8MT7juFwmNM6Ur0dRby xFmsMiAKLlAKSUgTXLFX1f9t86vxIteLBou8mriws3mWOw1HZnn31TSxJeLys1grWlNUOd /CUGp1FP/QCdYp8y0XsP588SHRmiado= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310774; a=rsa-sha256; cv=none; b=g9GxJFz6DJ+nws8fP1910N75rzQ/MaqIV9gR42Vrv3K/y2srv6GmyMV/a4GyDe7Rv6KGBH FhVgSGJcLR/qQiOkS48evIXY8jqUwJty4M3TdDR/wfhHrnbR+DDKjSqCEm9bEW09HlMhrT jMnpxcf8kOdAvff3EeZQTs5zyr7J2qo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="FT/YbMZC"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-714186ce2f2so440769b3a.0 for ; Thu, 22 Aug 2024 00:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310863; x=1724915663; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YTJMrM+GLhYLeZwUuiK6dImcBAbTGjqYt196yQPATUA=; b=FT/YbMZCmX8vy1afyzGZ8OIkqB7ejguMMWhX2gOTEtbh1lqP9nzP9SKB+g1eI/BLQb g2F6Bw02ejDLVWcUucz0MfDibovB5TgWCtHibjGno66MVnBUlXdEA6BsmhK9AWJKcL5j oLq/PNKF7X9SUntvvAagD/Vd1afVaJ4Rwo0g4IMbOY80AP4EZsk0/HQWrvvqe8KGUR7M M6k3ANcuveopFVRxsA2W+ve/pifmXZalQpWIYvBCuaCiAQvWLHXNaa7uWuVM4J//jDZd X5sApl0EVQc4l4qcql6kfF6NIDVZ2qWjgZxN/KhSbDVLm/Tlmw7Y/nblzdLtYBOhlLlf pOIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310863; x=1724915663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YTJMrM+GLhYLeZwUuiK6dImcBAbTGjqYt196yQPATUA=; b=KXXGMcDbvL09adYvWPekeAqnnhxN4f4Npl/XyrnmtI6z9WGZ67aJqh4VsmqbKiTpJC JAB+ExmbwVLLmtVVDs1iJbpzPHWPxEdcVBkmsBhs6XNy/YgmMQwuWiA4PG2jF0hfFZik iKGAr6rSvIoxh1I9FOmCFHzKsKtdK36a6YldmBI02gKfX6tVtorYdBiLHNsMEnlyQp5N 21kqDBIhFtBi7PKbTrQioGZByw4xCFsGThF0vMM6T4rZ6ywtTCvGJcQsJ5MwqHbTqqOK WXkjdNU4rrwP2FQ+5EPHsZMBqx2Ew7FBmLf/4P78LZUHvGybfnwY8GUwxRxzFwasTsZE 3ONQ== X-Forwarded-Encrypted: i=1; AJvYcCVkHPV3MzZUl6BQk8JcM4pKx6LA8dhV8NHuIrFD56b2+yXw830wtw92oOxq5MnAcQo8vwGt5SM0Jg==@kvack.org X-Gm-Message-State: AOJu0YzGeYZlmPe7nJBKeXipleQh7Fiviawv2kQF7NIr4bV+xf/Z2Icb 6zo2/cCqJGZOQpUEjwTJGSIdZCUtie3OewnyFDRkQqzZPT7BIoGaNG/2j35+YUY= X-Google-Smtp-Source: AGHT+IEJaUpX6CQIEutPHebqIOZXfyMaKxdLW1UElMR3gb10gY3PgzFG7UEaK+UwdtskMN8KUpfIGw== X-Received: by 2002:a05:6a00:cd4:b0:714:1bce:913a with SMTP id d2e1a72fcca58-7142356e960mr5852557b3a.21.1724310862916; Thu, 22 Aug 2024 00:14:22 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:22 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 06/14] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:21 +0800 Message-Id: <5acabedfae7ded01b075960b4a91f2e15b4d76b5.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 3D7321C001B X-Stat-Signature: asxgnsa8uzeb9cu4hedy86bsdr8b3x6b X-Rspam-User: X-HE-Tag: 1724310863-585179 X-HE-Meta: U2FsdGVkX1+qBedf54p6VMR6/Dgl49AtwCtNQgwkx8pW19uZxvcLryFlYqGn8HtIkEx7vQogCrOQELOJ8c8E9PU2L9cvrzUZ3TTk2zSJBBP6brTrb3AsEcJOiYXYdKsh5jJQs2PCg5rSqQqu09VumlasVQZ8DKpML2tSfVkgNPLsePD1917S+GRTvIDDv//6eNUkkLQFmrPSkfAzcdw1rFDMO7zqta5Q6l7Bdpg2FPKDZVfBfXdoLKVwqwXESM5/BPJbDCnw++tL0Ecfh1c8nNHEYndeoktEVDBIwf/Qpur2AVO4wNdp07X/yw30Cy7+ieTJZgF7fSb/gpZMGmUvpRtmZwAHEODGs/+oDYTYZ9MRldmk6UO5sWTh4mMzD9105aYKBRhIQW62FkLZMC29hiNfU5oWA/FMi4zzA68uQNq8lQqbPYhMD/L3yvl0kOyn5A0PeBTWeJI4s8icy9dn27jOrLe2wxlSW+73yeJSi8IrxyN1LLoo1IhcfW/36hGWEk4Oc23TiSxyauxj4OmyN3qSFPjvBMsGxL01GQuYEE+b/TAlIGOg9k2OoxbaCivfiO5FufxtotSPtADKN8gBqkq2DRbTco0LYAxlHi2/3cl2w4B/gUmRadRYW8otd+BFyL0PM1dNLDzAwercUhQ0Ny+tXtPQ/12NJBKIA72U1DFJLGEw5M2zeYgr4h6S15AezL2Fr7afQgKoh7jcTbEAjGiTL3PSh37y1cE69qfG/NEwKGdRDJHQCT/dk0koMHqO6tiLrWsZNeyBPLSrIYxvxqbWGCTZO8eL5bAtrSrhHf6WZESz2NcNEeIb7NFBYOr1ob9S8qM1JEzjRZOjpn+3SxfeZDbqfXwmvoy3UaJyQYDHJ/JsL/sUCNqY57T6ThIJbtKReZZ1ewqg0KHbeXPTFVrwJoBZ99Zk5zwW2fYfhrxZ5GL5v/U1GKm25CKT08F/tWpfrTrVJkEbQjq1OBm ttiT0nDv jUlIwkWgW/PUq1ODBX8JrfbN+oh6OZzg5QhnFWrhGumXveu5lDilk4nr81OLUAuwpDbUuPvpFlLj6eyYmUigDnCiq0KAEXk2MCTBqWsq6nzk1cqrmztimtCgb40X3RvAFmeQQmXuPhcatNmZCtc0BExGDTc1OBIEpFLuU0mVq9GAActWDy1JOuOZeE4URGJpZmOepu2mYJ46oqocwtXeJaJDL+c9FIIWlVxN9LGxUKmveWCcFtStjeSDnVGI52xfr9W2y4SG86KD45LpTPLKIpSSUd4RIOPhTCdN2hLXbE92Jh7jbdIzyru1HY+2tbd13OuxwyKdkZGVbjrGTvYxZyEMv0p06sD920HLmRnfkoxL28mA8JHUMPkZGdr+xKrD05mIHAY6b9DEzc5Oe9Sulp1clzKFeyFon/ktUUnR1Wa8072Fx5Q5VxYZzazjCjSOkP1+D X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 93c0c25433d02..7b6071a0e21e2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5499,14 +5499,22 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte will be + * modified, but since we will use pte_same() to detect the + * change of the pte entry, there is no need to get pmdval, so + * just pass a dummy variable to it. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Thu Aug 22 07:13:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19DCDC3DA4A for ; Thu, 22 Aug 2024 07:14:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AFAF8000F; Thu, 22 Aug 2024 03:14:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43D5F80009; Thu, 22 Aug 2024 03:14:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21A148000F; Thu, 22 Aug 2024 03:14:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F1BDE80009 for ; Thu, 22 Aug 2024 03:14:31 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ADA511C1F9A for ; Thu, 22 Aug 2024 07:14:31 +0000 (UTC) X-FDA: 82479018342.11.3C15192 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf08.hostedemail.com (Postfix) with ESMTP id D07D316000F for ; Thu, 22 Aug 2024 07:14:29 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=V+IflJ1m; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310790; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jy8AvBGPXewIN0f06tSdN0J5TZbHLAwXxEIbEfEZm70=; b=vTog/IXBxGRVOwa2SPQ1mSJohHc1wAMKsw7kO7Zfwe/DpADR8g+tLHRPtgxlrRgDaZKfT7 /FE01qphVaYzVllUmUQhk6jbBXbbASaAfUDO5cV14HGAHpJ5COKGSSxKhkZh9/EggzvztM 96vtMMyKzFACb7H4BhwmssxbESQIdO0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310790; a=rsa-sha256; cv=none; b=MXiB2Y7B1NXW/mclX6bDLbux6CqY2RaK7ligrlMKQh33GBXBI32FgUT3v8XPg0ndAMvJq9 hl9FOmQPuMtwSz+jypjudv0bIGSjhDEuVFLcMO3n1XV3ECubOi7KzE/dc9U/F030UliR+f j9EArNr7bq0qTjQ/0WbVtDLRoYh7GzI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=V+IflJ1m; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-71431524f3aso398600b3a.0 for ; Thu, 22 Aug 2024 00:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310868; x=1724915668; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jy8AvBGPXewIN0f06tSdN0J5TZbHLAwXxEIbEfEZm70=; b=V+IflJ1mctHvzSkk0jlwM9stp+RNdKxZ/F5v7+76ZXEH8cm1N5nxPSDTVJAbchBMzS ihv9xOdwiFhwYFzh8Iy14Ctqsk2SsWJMedBCUZIQ//5zq49pVIiXwYPd5DdyPjnvX0M7 Jl7oS+s4oDPg7ZjuOwRmOxxrCuCMgwChfei/gAcgN3jbPc5i1qC1cyutlqtM96/V4RU5 K+ltSkKgP8uZslTlyfSHtpeTiY5HFEwScuJVssQKehrurcOdaEwDhVGcFCmjLUpEC/9e CTsJ6Dt91hVK7N9wKglZrVLWTxGIUzIgnCU1Kf4dnjPRcKMDw9tXE6CvqNl/ilTqc7pq QZYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310868; x=1724915668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jy8AvBGPXewIN0f06tSdN0J5TZbHLAwXxEIbEfEZm70=; b=w1FqdarfG174cA/2QAwOccwq7I2srXmXZ1wN89NiQKiZc1iJg0LqLfzyY1FBlvBzLl czkEom+Tuvz3+2JSDjTKXiC610XOvyH5C8+8eyLS9Y2WhCo1nhAn8F40OR0Xb87VIvzR HXOy7z+Zpq737uzrlWPHUhXMCgzQTB09spqpr4NgR+HDsNf0JQZ01GNiumWv3BooxhsM uM7tm9yenLBRTLjXvzVzSvhqqxy5kIoDVwJKvH95mMQByaQufJPWMKQdBgnq6Sn2XhTM tqvA6/hEkb7qvB046hUpQ4mJK7njvuhO9ZJc1WnORuUpfZaGg0IlTub9q4DEituueOqA nqcA== X-Forwarded-Encrypted: i=1; AJvYcCWxCuvszg9SI3/cF5hdHbOS0kB6u+fZw4asvkdhNHGDagvW3NkwcEF5cHvbqHLgjgrP002YRowbMg==@kvack.org X-Gm-Message-State: AOJu0Ywjk0dj1EsjM2KmRcoj+zdbfGNIXpPmsYcW5quYt4jMA9xUCD3I QPKXWQBuIYBslC3a17zjHdVabCCIOqQqleKvzv8dWxsi9U1el6XRzg1TzH004bo= X-Google-Smtp-Source: AGHT+IEV/3obR+4TCbDGR6rVI1h9FT7v1/u/+fxVcK1o1ztnph09CHD4OJqmjxMXnPNwi8/vaSiIPQ== X-Received: by 2002:a05:6a21:6b0a:b0:1c0:e728:a99e with SMTP id adf61e73a8af0-1cada078d94mr5247385637.26.1724310868515; Thu, 22 Aug 2024 00:14:28 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:28 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 07/14] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:22 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: D07D316000F X-Stat-Signature: e9fbkqr8sh3u1th8g9arjwnrnkrrdmbu X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724310869-657172 X-HE-Meta: U2FsdGVkX1+BfKV8noKjhlDBCI1isGIEXiK4RQCv24PntuGW5vcaGkAX9d2upixib+PS6VgNh7TF+RNY/DKI0IegsYYZgRh/D+Q04zCJQmzFvvhkxy8eg7f6C5iAPTyV2K2teJf/yU1RpTEp1AoFM9KsRE0pluuuWdTv8By2yq/sgu6bkRb4QAUybmQZxVhfeYaMgeFuvEqYhoRUrfM2UF74/9t1Bvpw4P19rrixY/lirDWffk2MKqIFYRgk6oyuhrYDfE9C91ffHjGnzJ34IbeeJfWRYV7yiWWFFi1+kMGCEDyr2XCkdE1ed1IcEU3cq2gtfAxK+t6MJ1Hp1mg+YrH8Xn8HyIociN1ElinZf5TUtmA7EVOogPH6gwVuJPgEGCw3CLhL7UjpT+7+pR39DOQJ8KEhR1r7ypIH/rXYxDM7F6OKqk+06r3RHICxodzHd3c9476QS8GwOK5aCpwOuLjsgHkl5x/fT9eEufrePAYoExOvdu/CWOH0b/o5DmUVn6yX+ZaGNG4wTVkA9lgmHo0SpVRka2vuJPGcTfwKA+BpBVmTspKlJXtUOYqqwF/60lZ1PsZwFuVfwarTgcehd1NvcDXoMvUNQZX2+z/lZoNyNLCtPnZ/LiMcPK4ct12ekRP+S50sOTb1FaqOHl8l3gfYgmZJPTJE1oN5fwGwT8H7UQkfSozw6o6gfy4AlwHQVmNUV12/K2UOoiEJ8fLfUtj1vhyUgv8zeNaybC1AumIF3vPIkv9CDJuBXHMpzoF195pBUSFsxJ0qSDYb5XCDOup9rkmtNz4WEPoeXFaohQuy8DoJQiV4tcCRBqrhKyDuMzt6nhZ9jgGQP6+LvqML5Je1tBsMFO9zF+6dT46kJwEhCMtYBZrHvRx4fdyf+qP7bzfx5vQIirobJL+IxLL1iMbV3myXows+KmvXYD8AmT7EJjfPLashF/07vVo7X6rddPm6GLwyJpQv87mN9kH sPr4dVCD U9w2EtTUEaLLLPGvfZLhhrzEm3rHv5x/6KcNxNOcvZ92L05aWcDz7gMCbcuzLGg58d6MvoYkNz/35Fqj44V4uKIGI7J+4TLKhIQbJRD8ep6yyRQfmNw6PWGuALywnQNLHZTtriBc65cqF6BRLjMVGc8w1Qtomx9vEcJsu3ciR0fE4zL8l65oze3QTmSVgU/JqHtRC7KMr9CQ4R9MsidcYH+N281LSnLDRNaHnckyVJz/58OQTm+5ZK1wZUnoF7n/HhhR0JFiD3cphW7awWAI/aKVhmvQKFvPWapniIlPMxkzkirpd+ixs9DWL7L7QXrQTQl18kcsysmYq9S91aBHMHxCYxKOgKsuX6Z04iiNx1mInaVHloHYWZI/vI/yFeY4pUr0K2HKFQi72SDyoNk7F57QtP6rSTrtvQZGATMd206eSg2XBZEcuw4QImFJC5HDH7Hck X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. For the case where the ptl is released first and then the pml is acquired, the PTE page may have been freed, so we must do pmd_same() check before reacquiring the ptl. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 53bfa7f4b7f82..15d3f7f3c65f2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1604,7 +1604,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1612,6 +1612,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1657,6 +1660,16 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); + /* + * We called pte_unmap() and release the ptl before acquiring + * the pml, which means we left the RCU critical section, so the + * PTE page may have been freed, so we must do pmd_same() check + * before reacquiring the ptl. + */ + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + spin_unlock(pml); + goto pmd_change; + } if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); } @@ -1688,6 +1701,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, pte_unmap_unlock(start_pte, ptl); if (pml && pml != ptl) spin_unlock(pml); +pmd_change: if (notified) mmu_notifier_invalidate_range_end(&range); drop_folio: From patchwork Thu Aug 22 07:13:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFB95C3DA4A for ; Thu, 22 Aug 2024 07:14:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C511A80009; Thu, 22 Aug 2024 03:14:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BDB3780010; Thu, 22 Aug 2024 03:14:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B8F080009; Thu, 22 Aug 2024 03:14:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 77D406B00AA for ; Thu, 22 Aug 2024 03:14:37 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 27F63C1311 for ; Thu, 22 Aug 2024 07:14:37 +0000 (UTC) X-FDA: 82479018594.25.FBB51D4 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf08.hostedemail.com (Postfix) with ESMTP id 49BFF160008 for ; Thu, 22 Aug 2024 07:14:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a+pDqIvP; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310785; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lG9D96+xW+VlDYZHTIMi8bcfezh9NEOiBopQjA6w29o=; b=QMOSesPyWHuAZqwViBsOGUtOEVTbFQpwp75sz5vY+0j/gvgqBo0MabWsU1h4IRSgr3J5kO 8vHGhnhpah6c/3wfKJeLYenXz5MSmbJ8e9Amomdfu7nZCgR67Nq0zHHD4y1AEjm1/S0J83 p5iErjG5ZBcFaC1Kwf7xqquClsI9S3I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310785; a=rsa-sha256; cv=none; b=kXfPMcbB4DwgFmUk7bTBfi8PN6EPTIRPOI6yPkLJZgk0IBLI6NlLNxUdNsILBMkGJ9c5af /b/j1bcoclHya4sd3WYSadlCaTkpXBo/xaXeRUIjSgdR898Ng4j/tKdgGk5E5K0kX6RMAF zr2ZxM1DV6WpyugbYK/1H12WKIrDbZo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a+pDqIvP; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-713edc53429so393938b3a.2 for ; Thu, 22 Aug 2024 00:14:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310874; x=1724915674; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lG9D96+xW+VlDYZHTIMi8bcfezh9NEOiBopQjA6w29o=; b=a+pDqIvPCxsBqzCvWy5kti4DLMAAHWZl0x7Jziu/FVPA2PYkf2lflYG4//NLuW5G5I Y6/MDBgqGBUz2tttr08+SScWdHjZc8ZOagA+HChHPdWd4IMJbsk4zgX8WGir2iPtu3jF X+m7ANyr1CjFjH4BHHMWRq/f56oTly0Bm2eMwUXhXKu1L83n/S7XNjqSHYBQAM+//20K NuoDv6myrZPgH4nlWiBfNJqlDscIDt5xNnu7abbYpjC0SvSvyOx2jOa/JmLvfSuUd7jQ qPnOchx2HovPbfLDGq5fSdFj5SsaAjIu0fDsowz8jQpXQBFneNp2mbwFyr+Z7cllhOXI qcOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310874; x=1724915674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lG9D96+xW+VlDYZHTIMi8bcfezh9NEOiBopQjA6w29o=; b=jupWh6J5+Uzeia+b6JW9kbfaA1M4NpAzY0ZoAAUJk3c877Yw/b5FxpOGmuh2UpZWcm DYs5s5IRlnS6X54kpd1I8E5736HU4Q9dgFZV4MFoHvCY+Kr4/wXY7eyZJS7hqWaqW+tI Xl6PV+ftTmOghjGeEbJTvfrgoJufEykUhF4CZUvIzCHrm/S4l7bUFPovjaX+KW8LR9Cf MQQa69R+sEW1oGCnWU+74p1TpChBWwV4dSh+208JitcBDq1hK1IkmrS6MUMcy7VuZRGw GNt/cMqzrcKx9XnVbt7b+/nVVLxWtldfyIFiXmt6wFbJ9n1MXKOmbeTQvTXoRCJ6i7Uk lJVQ== X-Forwarded-Encrypted: i=1; AJvYcCWl8oY3PCEaaGEBRz8SATg22GYT1u3Ok/7iP+vqaNgegG3l8gjeG+f43P/h1Llpu5I2Z40aOUrQaQ==@kvack.org X-Gm-Message-State: AOJu0YwahGbUSA+YjywyXmffqFf3WWpIUkCCbpc+UV8NuGYn0v9jmi0I zW2Nr9VNyEf6OR+l+7MtN7EhYZzIaqJsheD8bg9jL/Opz93sf8ytlHHX7DHiJU8= X-Google-Smtp-Source: AGHT+IE1Azw7tBspdYFUO9+EydXCiNKc3KqrNzafyoF+69WNl4yC4JRDYA6cJOAAfX+V+yHVonAuzQ== X-Received: by 2002:a05:6a20:9d91:b0:1c4:f209:f1ea with SMTP id adf61e73a8af0-1cad80f657emr5853305637.31.1724310873985; Thu, 22 Aug 2024 00:14:33 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:33 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 08/14] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:23 +0800 Message-Id: <71100c3867c4cf6f5f429ce9f2db8432066d0e99.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 49BFF160008 X-Stat-Signature: hr89kwa8eqc3i1nxnph36kab8nadyd8y X-Rspam-User: X-HE-Tag: 1724310875-380414 X-HE-Meta: U2FsdGVkX19iA0Gdgdrg65xitlGzJgNn3hM2ifJCQFX94LH7xPPztaU8mR4aFE8553pHpqy0xzK59rdCiz9mdZh2R9CxIyFKWsMr3KNLrikHj1tuJ2eIz50Bc295+aIFRyxmt1jX0LK+HjN+L6AsTFIISMBsPUrFHWwQdcjMWTwtY5bo0hn8NhXVlVXfAkDaEr+6mbIY8SuKkayTXaaFnRzm+98UcN+5AohWl3CqGY23IxcEuhy665BQT83rFESeD5uMfz4bL5kW2YOe0+GndPLyx98L6tI8Iqxh4wusapQ1QBdCHHO80P0uHrSbz04h/CV4NNFcAmDadoonbRK0bE4JyTOaWi4okFuXDmRluj2xYDMBe/KOWLimWc1jivXA/yFitMTsrSnhjcWWP0ymxXDm2Vf6h0mG8gWNxLEUsuZskFfUlsdt9IXyhYZITPw8sH+TBUIPK1qRLBOKb6kpX9b0YkzINNSwmk5v3nz5O2/qdDH99UYFLm/vx76cHjgzs5AM8ItCqOi31EnKOHx43gunbRKouo4nF9Vma1uEx40JHVr2PFzP1cq2zZBtL8ae0ty7IA1bTwpDzYRtFIftjx+Z0e/eTy60GDz0i03WO+2cFRIgTawPfb9JSnU8+4Lz3IjVdmhvY+2NxwIK6dfAI42XShzP+meJQiU9KcaskIRhOgZy4bVwoGX88vge6mOUvSR6lFeVXqcVZ60KpjP0fD97u0dgqxK4i2ZODTlWEgRLtfyuJP7AA5GwskoGJpk4MwmTOYLyruHzijiIH6eQMPiS4ZvQIcrZYegFd191kEnzFBkaqX4iO0bJTROLjhXqogBmlAghs1Os98vvEc+BiByXv6BgSkU9cBP+gbsDGkRo3Hj/eVeYCkk9MsOMzIcxOR3Uxv2450lvWun178JH+bHW9xqTwHjSnTOXo3yqbJq+5mlRBzAMx5h3Tck24YOOXE9Ye5MooFI6MXyE9R5 MvmT8sXC 00fevkVKak3/95aqnb3iZltvSCY1dklNaIb1u8qjjPSOyh5GTyu6+qR70WnnkXJU/Hq8hHvUa4Z/wb7PutwepXPl7Xbzvn6V1bpI6VCWI5c7wXPjFnU+c7sXSf52Q5snpjQCzmv5SghpPJ46dv/2slSvVL8PpptsH3dJFEpI7U3jbPcLD8P/3UhsEt/9RvA9svHNk/EcCPscF7XxmZ4Cq+S+1a/fWPPpm619L1us0+BPNmpXoKmiG/5IvUgkvmirQQzZ7Jf7V/2DT8EYC6T2dG0XB7LZhQ+xZ4O4KYOy42OZc5bkZhrBG36jPuEdYomWVKgK0BGVLp1RQ19lJVbgkS+vUQnpWYhWpSj87W2M6Bq+seQ3JQN5H1e+qVFXEeqLaki52usCVPkeyBo8bVepE4RA+xzFssl4YeQLQunQRWoliWP9JuhKoMUhoUaPrcEoB2tXi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). But since we already hold the write lock of mmap_lock, there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/memory.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 7b6071a0e21e2..30d98025b2a40 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1083,6 +1083,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t dummy_pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1108,7 +1109,15 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we already hold the write lock of mmap_lock, there is no + * need to get pmdval to do pmd_same() check, just pass a dummy variable + * to it. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &dummy_pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Thu Aug 22 07:13:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D4DCC3DA4A for ; Thu, 22 Aug 2024 07:14:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CAED80011; Thu, 22 Aug 2024 03:14:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DF7580010; Thu, 22 Aug 2024 03:14:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4D6080011; Thu, 22 Aug 2024 03:14:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BDD6F80010 for ; Thu, 22 Aug 2024 03:14:42 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 52029804FA for ; Thu, 22 Aug 2024 07:14:42 +0000 (UTC) X-FDA: 82479018804.15.B70A9B7 Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com [209.85.160.43]) by imf15.hostedemail.com (Postfix) with ESMTP id 80D06A0023 for ; Thu, 22 Aug 2024 07:14:40 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a48iCGNr; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.43 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kCwbwV/YmKb9rcRCZPyi2vEyGZwxl6QKnoBllHH347k=; b=y0F2gssNENW4OinfjDw4+GfLOu0ArGQy/0hsdoB2+jx2w+hBDkbP8cgPacSvkOwjclOq1d +LJJEQrTp+DgWJPhDhnOHLFdSCS1wS5K/Iij34HcGaz+QAYcbZUyT9ziooNAHBOb1dv9Zz /YXEndT3WGEW9VkUIV8RY9B2NBFH8bw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310801; a=rsa-sha256; cv=none; b=VX97/o1Dc8S2Pp3NovhtltXDlQjzGY7GrqsaaadEjjtjW2YrkG4gB1m6miesJbV4PPnBLi KaRLIox5lWiNwoUgGFb2ao+JAfE/jdn1iEuz7MoLoH4gNshlw93rLdDIB8TPYgY2ViYR+i Kq3L4wc/aNHvS4QVV9aq+PqboMmzXVM= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a48iCGNr; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.43 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-oa1-f43.google.com with SMTP id 586e51a60fabf-27012aa4a74so250299fac.0 for ; Thu, 22 Aug 2024 00:14:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310879; x=1724915679; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kCwbwV/YmKb9rcRCZPyi2vEyGZwxl6QKnoBllHH347k=; b=a48iCGNruWrskqiVZi6XSEg/8YppD8YWqO3YAtYG6guhUO/bbwo9/9ulMTvfjIo4Pr YBmJ06S4hzq3oArnA6q90geGipRoPCtkM9aEDzKDCZoJFERg6STaqdP+D0HMCbtfMYUz mB4ug8LFsgAX2glS4ilwyeZJnAQJlDT/EEVfhJR/xx4a7yZUUWmLty0JV3svJCQHzsPQ bHwE8Vmt0k5J3LrqDD6c6/MOhZSm00Ckk5Ri3um4jaKUZZMxiqE+cJ9DOERLgb+EGjCV VdZ0u7CugEgl4gjlieIBZc6tdNjRrcEiF1Vt/dWybMEKdsmvG/WPGJmeGdCOYKediTED ZUxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310879; x=1724915679; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kCwbwV/YmKb9rcRCZPyi2vEyGZwxl6QKnoBllHH347k=; b=KZTlnUgwtAE++UYNhAdB0XucUPy3d2XIncHcaxkCcnf4S6rwdz03OhdJhHfEzbKpBQ 1ljrC/rXxteRoaLgkdrRzsOx1h9SLNNj0Zk4DUgEYZMC7e7EkaDJVOJturPpwe0HY1Pc gXO4B4JDPrligFX/V2PXDv+Gn+h1vNZy8hBW+8usVn77294FOo6h5tGB7MkXp7p54dXf SnCNcCcY569C1LeCWGgFXs51gpmKJlHuo+TdUHmDcKyOOctflYx2Nqjwt1hbaUto4JoT 1xlQHnawATx+KYoQbTGgJZkkXYWhbuqI+RfYCzEMhY/KnmYlaygJgVyD2cZdb1xhJfAe 8QCQ== X-Forwarded-Encrypted: i=1; AJvYcCXcLG8IChnhSnmfRRd6jkT9bdPpx85kOfhKXXaKMbQSX38XLP4+4036hEVok+2VxITGAaW5079vpQ==@kvack.org X-Gm-Message-State: AOJu0YzoQrEQRD2YhqxfBJg8F1ITs9w+98jdV+MT7e0EzIQF/u+oWq66 sDasrSA6ICNeQyBFrCFIp7m64TVtXsn2NA/hGKFYKWbD/a7+HfClqirYonfq3jw= X-Google-Smtp-Source: AGHT+IFsTn+Q5A1WOhbyPTAjHclYiOvt3nhLZV/Hi5KOu9NOtbxQa/3JykhMvHE3PrFxCVBhO2k7lw== X-Received: by 2002:a05:6871:149:b0:268:afc3:648e with SMTP id 586e51a60fabf-273cffcb557mr1073020fac.42.1724310879532; Thu, 22 Aug 2024 00:14:39 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:39 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 09/14] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:24 +0800 Message-Id: <12097a187fee5f21860d4f87e7aa103f822e298e.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 80D06A0023 X-Stat-Signature: 3wo7wyi5f59pn1citimxnpah5xmho86c X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724310880-652006 X-HE-Meta: U2FsdGVkX1+Mrea9BBYvbIieWTIPVvAC44TWdYFTiHGx1TSji0KRcC38anTWZ6Ye/sNLpamx8ezR0aTAvRruTC+p5NgJuaVcJIJwFptT4RqsEZ6XbDHqCh1jq9gn4J7pgfiwsvbpALyc2+1sg1XtIrCAGFIVczYUjn12/QKpN5AtG0ucajIp0Qw097viAFJ0KpqLJfM9cKBTVOOIiX8l5//Nane+m5uaE8Ay7WekkVX+w+XRDSH3gAl90bSfyMqvOH9Q53W2HcE1eb79s2Zz39piTrR4Nfc2zoUBk+f6PmDWukeWCaM5Qc9ilG/cFbnp3YAVq2xhe+sl0uU7+lhRHJ99mQn7nxbsI2xeKWcJeqvbpUQvSqW72/8aTYdcjm78LqOnhplHwOi4gWLc02rHrSpmQSQJyjl3eOEfZ9CCHmgUlQ7AUuYk4+83dKOWPKxXG2nKqm9tv5idYMF4G/yAQuYvnchuGSxYbGuolTVQlYoQhZvp/qgLhkPKDXdqZunva0cE4jj0NFLpX1sjkPXi4ZzwHefhkPM7hnm6GCz5uhY3SSvupsHaOhGoCJZuDB7ShtN8UgY7ixr28WofPQ7PKVhUEojco4FwQSrG9PG/3SLka2XztDbI2gQzvQOfiDqmBTXTnDALGTEaPpth3pNeu5mPM0WIcLpXBpTHreYm++bSo9cseo3TqQjcxSro6VNxY5y78YQL+2l02cmB6HO5T0srA8drxhKcAkSSqTgSgQIRcqNA8OETxpzrUpynTBzkbYBPBESu5HEjPuRM0Fvf4O6I4i0OxRHnbF4qQlQFwTBAmtZfodW9Zwzl39Y5G/JPf/ivrsLiV6Gh76H61OIa1du6wskPPpEhnZbBf/v+Ei1AlCgA5jgz/4P0+HEQ4KSAoETpo20EPcnzLlkBjCBvf1Ea9iqRQVHVTlk+u8xzcV4FbksUUhSyNQIAhQbYULf5DSJr8aOwxZy9RJzKlZe sRp6ioa6 K59HI5/e8mhoHGRg7McR4DTQS7C0TimzhB6pH55+WoCCpi1smVTmOaw+VqcwNVHHg7iv+H4Mw1uT+GEl9nlnqBMVKlV8BOUQFytnIM3vCgLbEZz0JAzzN0AQG4xV76zrYqNOXHPK9u9oIOjyh2J/eBgdrUxmQSm3EYkwKJtfes3YsXSCDLPu5CZEfJB8lhSHgr6mmaFsVcKdTHFZPirplejLNonhtYz5qixQKfIW0CHrzzTFZqPsn6bzV7HisTbsM2837D31B5nkqoDdevI1HRuy4CQfaOnY+Gb/+khcGWKjfR2919IUg0GXz5/yVWCSsEoTVaNNB5Bvi3uh9j3TOXVF7rH/nYSfJITNDBN5nDfSLuczc2QptNhdIK/kb0U/UJ7/lWM3Rn9sPa/jYNBKJsVm7ZKuPdDyavpbhMAlGtHqbPdLuLVM7nSnb6v2xj0yD4sCwGB/ZeizL4ec= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). But since we already hold the exclusive mmap_lock, there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng --- mm/mremap.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..f96b025c09079 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t dummy_pmdval; int err = 0; /* @@ -175,7 +176,13 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Use the maywrite version to indicate that new_pte will be modified, + * but since we hold the exclusive mmap_lock, there is no need to + * recheck pmd_same() after acquiring the new_ptl. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Thu Aug 22 07:13:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E01A1C3DA4A for ; Thu, 22 Aug 2024 07:14:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 489256B00C4; Thu, 22 Aug 2024 03:14:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 413B06B00DE; Thu, 22 Aug 2024 03:14:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 265466B00D7; Thu, 22 Aug 2024 03:14:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 03C046B00BD for ; Thu, 22 Aug 2024 03:14:47 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C1F461A055A for ; Thu, 22 Aug 2024 07:14:47 +0000 (UTC) X-FDA: 82479019014.10.8FFBD9C Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by imf17.hostedemail.com (Postfix) with ESMTP id 021B840004 for ; Thu, 22 Aug 2024 07:14:45 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UuoXXJrK; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310820; a=rsa-sha256; cv=none; b=MwfCj25uhs7LrHY7vySsqunEhsB9OcaC37xmlat1Mg2VfZdhwCUeZ5TggeRhB1e/BuVZ+R hhUAtauZK4WInxAQruka1HbVYzv5ogmQfEUXMVlxrr0kYZv+fhLQxUt71jS/e0gWa8qn7D 3+FnGUPHvR4AIP9m9Of1LLhjJdLK76Y= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UuoXXJrK; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310820; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Crbz/R3rJcbpbcovfp9fh4ozsAEMZSaAxrx7zvZfRRw=; b=vYQe3chGM0sYDqXKtgnJoz3e6pv9AQ4mj1rbcYPARAJMthjMdjXGXX3SidvQX+7sgyd/Zn UE1w6znS1e+ZpWyZD05xT0tVPQLV5yh8z+HFr+E5oL6gqV5vMmv+Mu9E7ZjWogLnMbZDFl 9j8e0P8tEOW/tK5n3tQP7Q0ypZRLkFg= Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3dd16257b87so212308b6e.2 for ; Thu, 22 Aug 2024 00:14:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310885; x=1724915685; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Crbz/R3rJcbpbcovfp9fh4ozsAEMZSaAxrx7zvZfRRw=; b=UuoXXJrKYGm778yaWIRzZCkabDrPJ5rk9UCNPtRc+5KQ50zM5YqDvxQ8DkOc3GClGw V8QN0PehdKqQFEcf++ys826d8Ehhg3k7ZkTQrtz+vHzl05fsey2tKL23UkOurCEn5qMn 2aUD4PRVTXzBDpDpCbkd7tfIKIt221/j/IHUL5oI+5bQM0HHTWw5Q4pxMG4zMiDz1zQf h/YRsn8Np8MjlFMFE33wlI67UGRXes2W//fOu3JKjP/Ow1G5sOpDteNvkZcjz8quEf+4 73c7Br7bs46OctmkcMiAf86m5rXv478tE5GZ30ZZrKnPPcReKFx3qnx9hUUQzdsMSUWU eNLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310885; x=1724915685; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Crbz/R3rJcbpbcovfp9fh4ozsAEMZSaAxrx7zvZfRRw=; b=sAjcwW4RlbFwMCduzK8o4OxtETeTjp9dM9WGWlQHmbH/GyzlYIzobUubOlKIrCbZH6 ZyD2dMxT3O5G+xSEIG4SFyKk/M70krNTUlJpHdy/2h3xcSbAs7tznh5TOj2rCvMabmRm TqOqFXrGcXIhIIAdjPAPpGIgbWPO70wbBB/gJxkjQkNKiSBRX6npvNRIMkXv/VDrdsnc ksTJ3RJBSn2j3ohKF31LyUdeFmRQUIiUx9jzp8T8a+nns0JGQ+abNlj5WOp8DAhgDPyn CxjO+hib3LTOwTQrV3cFQGjccqjWYy2inQQLpLmMVbUI8w6Zq9enaKkfjYHtCh1o9jZr um5A== X-Forwarded-Encrypted: i=1; AJvYcCXDcnRvHSpSty/wUrayPhmdcayxo5xuR/HgIQqbOzUEpxQCD/4m5fP7qTyLbS58w3TnBKISgM8bwQ==@kvack.org X-Gm-Message-State: AOJu0YzPL5Qtu8fRH04K7ZGWGgs22TvYbLaBxLY4/y9+a0UypM+ymz/k 4EzZlv1lq2aGNoDaLGrFQuZ2zT5s38sHmWkI2rTT5TD/VGbuva+CigYbeRrJFXM= X-Google-Smtp-Source: AGHT+IF73IAzwqsx0nomJCVJryAV7cndKc0EgZ0+2TYoDy25bfQlq+9JSwXFUBQmZZcxXSkt111lxw== X-Received: by 2002:a05:6808:d46:b0:3d9:29c1:be5c with SMTP id 5614622812f47-3de194fbf2bmr5329807b6e.12.1724310885053; Thu, 22 Aug 2024 00:14:45 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:44 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 10/14] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:25 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 021B840004 X-Stat-Signature: 619hc5j7g3ugpfkibfoqwsiti7mwjbm5 X-Rspam-User: X-HE-Tag: 1724310885-57771 X-HE-Meta: U2FsdGVkX18wdPOhMWq+RABi5Hu/ra2txxD0pFpMLmdLlscUBZ/R2LFICdddIUc6fVBvCjLxf/YzbA602QAPOcf0FLWZBL4c1b7PSs0pSh39CbJhaXcz9l0UupLoohiMpoWsuDla8eSZlj/J00Coo7yLmag1b5jbgW1qz6UMZvL9p5scJ/pDSQZNGjuIMU2DdSZCUUBMMFAxAbQJthrTo2DeyBNVolRDpoR3/S2GzCNbuCvgi3jgjHfDZ9eU8g/RS1qtDaC6G/NjZnROwunKCtk9Ti5keaQON+iG6Z2sqkxAP50vYciwO+lBc31+QImB1DDemkU1obntzgQOdYKHDyXbM0laFMQyNGM18tazPKWnmB0H25WxLSCLvIszRs5OZOcvUZmoY38N1URpFpVhIPg6IKX4BCrR+Ho6gwxg/8EQDH7rmi81M4dM+ijpbv9xFuhBibrJ0/DPMXgqaHMeRIvNfLWyLkR+gelRWGe4H2tjTEkEqv1wMF1PsKu59zV3IgA+4GPmFaXhmG7/962/5O7w6nNrVHZFFx5NwxPRx32olxROziR+cwv/08/68D071gl8GOGTBL0s4C85zwwLYNgai7f3TPyWgHA4l/CnJ7sacN2FNr5Tmif5xHSdGWKAx+fo5SF24iXqfj5vu9wThWFrOtOEtLOrNyJXhVW8l6igaQzYJqY2m2DtI+z/u3SUga0W/rghdW39Q7ShY945VZyoLekolZrp5esyLqf/BGqMcmbXmAKCnnQpucrJnLoQFlFHXGVRVtW3mUDZUpym7Z8KTcpeeXOdiE+9JH8WLv2L/tCLbYX8tL9Jn07v67BmZDUyMRKjPswsGCLBupPUyFv7uff63USg3hpsFKucrjBzwzPXkOk9xB39nSSSAK9RWH7+FVW3bd/JuTWwXkWeQxpDnRHFKU5fB65m6h1yDXJWl3PDXFnfAEuqiKyhEoc6fhWKh7FzCw1U0Oo/SVC SQuDAp/5 gsVeG023m+y6FZ4ob5F70vB5VJMz2r25WXTF3z5yfHSyj3vuFyH3HhfdlaJ/hir2i9r2PWONEf33csleh1k8CT6GyB9NEZJ3Tun6rXeTR5j04xfgPbUPGeqYMNsz4wBFP+LuJZNQ9n0V83JFDsYQFrzjBTYWqteEaocjY1qnuNh8AkY0rtrztshGeMBPsxvUbkTqj5iep2NVcpXDVHsCW8cNNxV2nVUYSBuAv31c0I379mKMXgOk4F2xbxCft6/BuO3NBh0emgFqe7tlL3UHxbD5KWUuhu4jsBLYyNvBsOjZOAzsIClXepoPtzuMimjtgc420s8Nm8v5Tdw7Ugr0gr7QHs1Q3buRzf0N4wpWudOQtMmm/m5VG8MWTcDS6JOagUDQ9lJOAUK/joKeuXYSykvVtpslX2T5LkHgg3F08mafz+p9EzhqBKro2wEh//jmhvikg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng --- mm/page_vma_mapped.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..f1d73fd448708 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,9 +13,11 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; + pmd_t pmdval; if (pvmw->flags & PVMW_SYNC) { /* Use the stricter lookup */ @@ -25,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,10 +35,11 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, &pmdval, ptlp); if (!pvmw->pte) return false; + *pmdvalp = pmdval; ptent = ptep_get(pvmw->pte); @@ -69,6 +73,12 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } pvmw->ptl = *ptlp; spin_lock(pvmw->ptl); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pvmw->pmd)))) { + spin_unlock(pvmw->ptl); + goto again; + } + return true; } @@ -278,7 +288,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -307,6 +317,12 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!pvmw->ptl) { pvmw->ptl = ptl; spin_lock(pvmw->ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, pvmw->ptl); + pvmw->ptl = NULL; + pvmw->pte = NULL; + goto restart; + } } goto this_pte; } while (pvmw->address < end); From patchwork Thu Aug 22 07:13:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03556C5321E for ; Thu, 22 Aug 2024 07:14:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B0FD6B028A; Thu, 22 Aug 2024 03:14:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 439606B028B; Thu, 22 Aug 2024 03:14:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2176880012; Thu, 22 Aug 2024 03:14:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EA49D6B028A for ; Thu, 22 Aug 2024 03:14:53 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A3EBE1613A4 for ; Thu, 22 Aug 2024 07:14:53 +0000 (UTC) X-FDA: 82479019266.28.D8E5E12 Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by imf06.hostedemail.com (Postfix) with ESMTP id D200B18000F for ; Thu, 22 Aug 2024 07:14:51 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=fHWXY8Nl; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7wvuIpVK8M95P7HYfgMO/FZeMoPfoEIc6BtRbN48KOY=; b=Lwry6g8T673+l+LnUqrLLvp3sv/s7ZswF0f+FhcgCYJTi/sVvYHeZYGNFCKFjr29GayJjh XHDhMMETVme3VUN1hvvOUvk9josq661IiMT7ACsCPp6wmBPNXgewg/acMfSyH0r7Tg7+Wf UcbizOYS0KXIkonjo2NyCmjqnCNZ5hs= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=fHWXY8Nl; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.167.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310784; a=rsa-sha256; cv=none; b=QXtkkGDlx4x0zGgIvyauC2Kecx6UxBLpEfIzROu/8k6JmNYkVJDAacJNKupOUEcTGgEPbM rCJu6hGk6sRDGZmEzdR7kUrUZX8NTnFciS4lW7/Rye9VqzN3CX3UL/AxB3iipaAIpQby4i gopT4UkA8rcTHl0EW1nM/kDbcwUTjUA= Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-3db157cb959so285395b6e.0 for ; Thu, 22 Aug 2024 00:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310891; x=1724915691; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7wvuIpVK8M95P7HYfgMO/FZeMoPfoEIc6BtRbN48KOY=; b=fHWXY8Nl4otGgw2QU0rZP4V0+Gt5rq/DuvUdwh6+tvOHOZCzAO9ax6RrotTsv3ldMC rgH1c2Kl0Lw4TdDPoQSpcupdZi+kWsK4KJpAg1HEEc2sRu0+jqSy9yim1PZXSgc4HpFg FYEh1b6t6kvhVk0x2kaMFBSJbSt/+nBePhcNWe9Uo6vdUIW+54P99dIMNZPjN3vq+HM3 oN/lQyO3ruRIR7O840R9LbX4DmH0UfaZsHQCchJY+EDBA7+nmoGzDlUY7zHsUkU4qd0O eUmQO7dwRiEks7A5MQ31ECWuiPWZ2MD9g8sYIv8JqKhKYY/i7ybQHjIwe1c7bZQLZSZa SmQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310891; x=1724915691; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7wvuIpVK8M95P7HYfgMO/FZeMoPfoEIc6BtRbN48KOY=; b=qubhFylA8ncdvB38fj6/QuQyLJZ5eRajiWkPvBoFoUVmcBCOBKeKbdJoJWlV2oVMWP zIzlXkXlj6Ok9hX6ETBjBrZ5xw0YiF7Q8uSNbb+5oBLnLcivlvNnarYUkZR7yB0m8ba+ A7XmV+HtlkNANPxEEPARrMukyofmy8nx6Z8q8/+ybw790ts5JkAU1Yjrr1NG90uaoun3 XBpPpscBnzhjOo40icy/qcyf4n7dLMkcnwLh09keYrtaiGp5s8EjmqBwwPmZe9l6RItX k3uNE4zgSXT/F9sxLogL6z/B/MEYRPwixBHeIUoGZOFmUpWLwMS821SlMQF8o4VhfoV9 ft3g== X-Forwarded-Encrypted: i=1; AJvYcCXA2YarYqt5dwFsOeZGI50PWv8JlC13YryIcR/SbxEDzRkTof+ECk3zUcfOlC0MBEwYKGx1Xj270Q==@kvack.org X-Gm-Message-State: AOJu0YwK4yfaFrYu+lesPRsj3iTtAGG8gTkTjJfIcwTCfs0MxLHIs1zZ akEje7m9i0v7aRyQSHSvvCnPSKw0VAi9kAvcLM9XsDqOt/RppGBo0O9xIphG7W8= X-Google-Smtp-Source: AGHT+IH3dTXEnimLAtKAfnTveP36n8fQueYjLUMvugh9RpWMjyLI+ldrjyQ+VF2Ub3qToXnGQhbMbg== X-Received: by 2002:a05:6808:3847:b0:3d6:2b12:7dc0 with SMTP id 5614622812f47-3de22ff6ae0mr1325714b6e.20.1724310890896; Thu, 22 Aug 2024 00:14:50 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:50 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 11/14] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:26 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D200B18000F X-Stat-Signature: yqhf4z4ayofpbh6kbeit75xifxnkbb1c X-Rspam-User: X-HE-Tag: 1724310891-619771 X-HE-Meta: U2FsdGVkX18s4xJzeE43bgkm6Gl/sGJidlAd5gMv7C4yd8ympC3gUDXguBctABpm8DnN9HjEqSyh4YNeiM+ikv5f4mVYoNDaIf4kgx5FbzPaYGS4HOdKdBpW5q9kl88siQzNEizAO4ANXCHok3bxobdOBBugU0QdcvT/0UtFmo5z+sxeHZNbj2Epji4bbwzRlQO1xl+l6JL6/Q23mHzNMJjKSfkktr98u13Jx4Z9YK0i/H5bqYH3Lm7P3vOB5EgmKaT/lr/TmRCpdXj/fhT9YhHFzWdAZ9KgT7H8WU9ndHxgBI2Xp6arrgshXY7LbHFQcUcr5+8+nl4Tyrs5NpqSbYSMo/mZGjSzaA1Tj2zuzRJh5Lmmjz31l/+D+ov00vnlAd52WWKfRkwZ6buhaR4K0NXL/le2ODVFrZDoZYh6dHwraBvR2IZKXvOhSlux+abgz16a596SpTAMLtsz9a2pH8oMGBapcpuq1AadM5ulrzccBQWA4g5iSg/WOmp2TK+FDaoXPl5UZAuyLf9rSH/QquT66oX9Z3NUmZq4lyy8Y3KqG+Yn+vUWv/rKHsNcuIthKAaYh+N9d7xriP5aB7btIyN5OYkpNKOJZezIAdftowO+UKIefiSuXZBHubDXDzT+Q74Y+SWExTuN2d1i3O1q6Gu7eyvn2AeP+6u/a/bS9g8KWyFvyrWNM84jwFLrbMDBWdu5GM0ITBhgrnOC2LkNLnuOKaA/SkfOhVOsdaXUhWCAm5BmcBO/dwZ0A1ONKddvl5ipOMlUzr23RqhRWKprMb9JO2r8RJ657RBITrLD2vnQxdt6MO68xManj+PEtMLmmCBacumw9ox6KGbqdt3AiNWVchrZVQKlyRF006Bl+O8uPMNCwJBJiEzEwGmYSRgBC0YgCWlD84XAtXaS9ALMvil/ZdtLyqroKwt6NkLei1Je03/X0mPj7o+/6yEcygZp8GDOaRAnVPcXpAiU6/z 88gasp7N JtQqh2JwAv3DIm/Hn/qXyLJj7ppMtDG7awqLjMEffHYb1WA7WIMTRWfSkS3QWwp3AbKkFwZCCX3m029dXwOPB1ADQjzVwsKeZNQ4UFm7FnKav8ja1x4an/k0mK6Fj+Tglkj4zq5CxCeYPAzMquAbP6gVpQWHTduDtn37flljtjovvOaI5ldYGlpZmKabzHbiiWW1QC+d8jasOzWcTOPBr53ik2qYZ2FFYAaf/CHE2EArQCzbXxM3/I1WxroCpd9Slc2rwYdTICGxpjkNvH9LJS/7jy/7/WQ40d90FcyZNQx9e/0Lrpug6ZrziwXFkOQ4fvoWehjCoOPWyMQbkaq5Qpwo3yJqTnloWXL9GLp9LNt7j01eBukUMQqjkeKtVxCV+jm7NcMK7U3AmSz+HX0n6NxpDIXPyvMHQJiS5/y8DqHRMob8eC+0LhthvmY5BALBtImeZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we already do the pte_same() check, there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 6ef42d9cd482e..414ee744257b7 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Thu Aug 22 07:13:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93EBBC5321E for ; Thu, 22 Aug 2024 07:15:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B682780015; Thu, 22 Aug 2024 03:15:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ACAC980012; Thu, 22 Aug 2024 03:15:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F4BA80015; Thu, 22 Aug 2024 03:15:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 634D780012 for ; Thu, 22 Aug 2024 03:15:00 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 238DE1412BF for ; Thu, 22 Aug 2024 07:15:00 +0000 (UTC) X-FDA: 82479019560.25.C9BF329 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf27.hostedemail.com (Postfix) with ESMTP id 47AD640007 for ; Thu, 22 Aug 2024 07:14:58 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=eV5j5Vlm; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310881; a=rsa-sha256; cv=none; b=DBxfAqaRMSbOFRvv68jIVRhwlGkZH0PV+0IntjqWylbYVjFYjObcPvpyHHnVE8NbMONSEh gcYIPiagpFE6nF71lKKx3Y2PasRg/9Wf+yGE+7libIqXG83HtBQaV6P4/KhT7w64hlA3k7 AgkV9cAXHLQLhI758gz0vViGP+4TLGw= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=eV5j5Vlm; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310881; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TKuz+eT/BjM4oBQXjKjArJ5u0hGAcRe4llal02NX1d4=; b=xo5lpFfuh8WQJpGgk9mbZ07PcD1BUljiD+AL84TdLZo1SV9ClDuFymnoa61xXv2ZnTEYid eK9E8Nlm+MMoU0Wit4GAmvTrprNWFIUIJ+u7rqR9J0BwUTMQEgii/ARQoBHOllf+tjUpHB Sv2QbFxJo8Wi4SLTt7x5/Velg8vYp5o= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2021c03c13aso3533285ad.1 for ; Thu, 22 Aug 2024 00:14:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310897; x=1724915697; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TKuz+eT/BjM4oBQXjKjArJ5u0hGAcRe4llal02NX1d4=; b=eV5j5Vlmmzl5itiJGqqWi8UX/VuLF/LmeRCDFW/ELhMMtMG/K3Kuw3Hi8jlMrpgS/q KVWe14mXY/vZn8QB+1hmNK9IbnlO0f2l83jGB8AK20R0zHmD/rRzSXm2rKkO7eFfadjW ZPjd0Fdvwr/fwSLTjecZ78DVfSciRGt/r7+sv777x+fvM0gZtEEA9GRAZv/dI1M2QDYD XZoNW35x1nFEqEeBr+qQ9s9MmKgm75OVw5DqXCVPNSxWAj6WvgHzkAMgCFX/SZTXkVsH 39PnhayXQygiNHgCx2epbLAX3q1flzGUnVSfcI4HjrGeC7VUGb1QCiBbGQiyXTvceDIF lw4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310897; x=1724915697; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TKuz+eT/BjM4oBQXjKjArJ5u0hGAcRe4llal02NX1d4=; b=fo1I1ykOMU4E7Hy37vadr6vnfY/i9Ug7kmMdedRndqoUqSnzG3v2PwHE5Vigw9oUtu FI7wnGsuCmdhb/DTwnvj1Df+FoftnVtiNAE2UznoYCPaTRwJknVSx461fpii+8kTeXgk 8W1MdDvpgX2XQ3IesVQRPcqhn4rg03UWJUBugZhvvE/sphiLo4/jusUuC5w3fDNMbNGg W5pBNFhKTMy4HRvQiBH5hMEcp9zMK5uMQ54fuNR513Br0bRTVmoQA/UxaDQszL07w770 UgWD17VKHpYBG/iDxGjD1t++dKBLGBDjFZnnxKDM9ux0gP3/sRoAvGcTZfTk8VU8H1MA 2piQ== X-Forwarded-Encrypted: i=1; AJvYcCVXLnJfyqC+YjhIE+Y448ORg0dUoe9O93VoMxJ97vaRYwHP/GMRKDLhwY88g7Pq7rhsw0XbdXd/2Q==@kvack.org X-Gm-Message-State: AOJu0YxZgXLUd8Ype3jvnQYnpzJ7+mUsUEsTu+MARUcifUdO2x7G+I7p 9i+PvulwRRREuV1N0vMc8EXQZRsHF+By7m9wvX9mPdS4pPa+XpWdizUbl11bSHY= X-Google-Smtp-Source: AGHT+IF6H0tDfgQqKcPaVFXqnNnvtvkdaP6bepcTCK17Vph5rD33YztktUCyS9gQ0weOULsun7jJQw== X-Received: by 2002:a17:902:e88d:b0:1ff:43a8:22f2 with SMTP id d9443c01a7336-2037fe188b8mr31159405ad.24.1724310896888; Thu, 22 Aug 2024 00:14:56 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:14:56 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 12/14] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:27 +0800 Message-Id: <0e2980e9a31d3503a5dfb350c04f4ede4d05220e.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 47AD640007 X-Rspamd-Server: rspam01 X-Stat-Signature: kpuq7ed7wy1cb71z17ehx8z67f9h3nya X-HE-Tag: 1724310898-322790 X-HE-Meta: U2FsdGVkX186tGkeSsD3S0lJDqQgWpHtFWm00tLMunooNoZf94F/gEBjtjfkmWdnFuD31v4Wjl8BzMu0F1t1g/ZUuwj4At84RLip8ft+UnAecY6mHsFAgU32rZaQ/CJEoOGCAGDbSDljZSQU6yyBELMoHLn6DWTYtDb8YWfFfOJaGezw5eFT/hbxuMxg/CNHMWEo+oFzelcuMJjyZGV2LnaInbx7FiM6RgAk6KfXLfO6bkiyfJSMrqXiFRde7w/6eGeSrhg7iQF6qOxeca38FcOkWVUhkTrLHWKJxonM1YxdHkARLKGUot75rG+Uv7frtV2x56dytrrG+N5wbVljJDEsxTKN8GZJpV4Q/1b3TVuxg/xVkgt5Z+FntlGW/9E3KXagaWxLLIfLftYdJNcPNvgEux3oNefnn1LmDHAy077HmDUQSOGWDm8CRMQL3R5D7BWvu+jKNl4+E1TChavYSnoF0dffSwTAKjtM4xfnoLojSuvHgQT0UW5C/c3TG+DVbpXsv1n+NVtQQ5krRjFgWUT4PKUwsVtRuw/piTmIMNDqisseAzSOewsexRrmXqYvLj9FnBO82t+Jg5X9pdTYTCjtPE7KrN0XEgsitMj1jfNMpeFV0zrE7kXl/cnRs63jOE2wjAr7ySGU9+6XAJBUxtSOl4xlMBS4UCiiaQbfzlr3x70T7ybLWeys7vZEypnzWvvMTamO6YGINHj/Xk5Hk1VqGz5IcvIW0CaVegHzkppv/UFMUL5nXXUXVZ0dfckA71JeAJ15cqE6ZKGnrWaYqThkoOva5myc9xeha0ty86kR5s5hqjZ7GddFrwrfUAsYCbBPej7dNbLZrVhgkVV4xiusZ7ndcHSTpbiwO+1QNF5KYk/VyEPHSWwEruwoTWws05JKlPBqtz7zOsNCQaM0stgb8BmIUJhFqKknMakJ/ziV4k9dzttt8Q0dlnBq5gb3Sqo1aVgJrLLTj3TkOXD BTTV3A+I ZM8koq/uEOeEhmjLlj+MOFlyxx5awwIy9I/LVczO6hUKigp7b5s2QzomzT9msNNiCapt7C29LZQRTf9fwtSPR9R2k38HMXFF2c7UlaMGaRaLbTESbVUP96PzO+TvgKbfskqJ1YuP1szt/TMSvvMIi/gx/+geNuF1C4xsL4BApSq73eSSYhv9B/76bD11iPN+alP/Spszu5qzXIbMNmMGSrh0Jd99pW2Iz7DNnAEV3CScoU9bniwbdy5Q3d+GyNphfMRpGG/dCAssd826cWoSNHXN5E59TM3gLQ46UzujBoCkhy0FFZUUwImbnau5hDVjrbMJAjnNrkQy9z0sWChfgf0oZaZi6R8vfSm90p3U6vJ/TQIAlweZsk6nCv5fgQdbztiC/FOFQpxfinMqcSuJiqbftX7qvgHNsB/l10fSNpkDBycwnzSd0RufGXwDId3SD0Gey X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 5dc96a843466a..89ef1ac8eb1a6 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3396,8 +3396,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3405,6 +3407,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Thu Aug 22 07:13:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F6EEC531DF for ; Thu, 22 Aug 2024 07:15:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9290080012; Thu, 22 Aug 2024 03:15:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 845526B0290; Thu, 22 Aug 2024 03:15:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 461C06B00A5; Thu, 22 Aug 2024 03:15:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 216E06B00CE for ; Thu, 22 Aug 2024 03:15:06 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CFDEA1412C1 for ; Thu, 22 Aug 2024 07:15:05 +0000 (UTC) X-FDA: 82479019770.16.6683DC7 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf15.hostedemail.com (Postfix) with ESMTP id 043EFA0013 for ; Thu, 22 Aug 2024 07:15:03 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=fw8gDEab; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310840; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gFdM7jnff0z8zF8oYknH78hRu/cQudWIIbslP9zGsAs=; b=r1oyXecFAJ0Ks/7ThHNIFOmQ0IS8V0BYex51cv1YIA4A+uFKN2M2SwrmN9brolpXR/jAa9 aYnQpxWAZl5Uyg3AK4l5Iv4iD0PMG7TN2Nz9NUCnp1Y8E1cUc17No17mES6PjFp6Tydjn4 iwH0GGn2x/VNIR+n0CQDwcCVJyGYwes= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=fw8gDEab; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310840; a=rsa-sha256; cv=none; b=yWCqGeaMAJRqkYdlflhxWVYT/gCdjEprUyioMSePF8mDGzc8N8vXzon88YfJ9Z9B67fMYa Yx7r2i8TAhLiTsAvqv++MNLGQmOjRLaojiatF2cKl87qNnSuGt5nvPjgn/BNUwdMWJN7D1 lNWYylKVMnKloTJQP4uyt8PhBnqXVfQ= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-7142a93ea9cso394533b3a.3 for ; Thu, 22 Aug 2024 00:15:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310903; x=1724915703; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gFdM7jnff0z8zF8oYknH78hRu/cQudWIIbslP9zGsAs=; b=fw8gDEabvoan/h/LJYIZ53R758as0n+ebnL+dU+quZjiZSFlk5pnbRQ22q2OiQcTzT CT7GrsjsGmX0FfGIq6aR0M/0PxdPWjkVp/RUPYzhhguAQyTKo1NAJB/4kwh/8SeYSbtM OH/bKhjf4YUG8Gn4cA608FySqVAn9Q7gTexj1GO7HfQVMisJbaEQe6yRv8JiU2jzX38K qUpfKD57veNtme4yhHZa+mJH+5itI/Eq3YS3n+lvr5KQLEAHmxduKuWzDRy0BnnicJ9+ Ffo5OW2Yjm1YJv89zW3SC0WukXjmaxJXzf7egGX9lftxzXNvdYOceR9bJrDkjxIn0v9L UgnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310903; x=1724915703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gFdM7jnff0z8zF8oYknH78hRu/cQudWIIbslP9zGsAs=; b=EAMK0MHEc1/tBUXcbtUiYLAwBJSvrKd06HxcYuDc5DMMFN/XGU/4vVaukd4BXV2z0e N7A/BFHfxGuetA59qKLJM3F35wZTIFPYcwLaLHihld68RSB2JlUDMI2KNa2LCwn1I7Xn cgCK9DJ+xOTSiRWVPGV0yCmfH6348OzG/usIKNiPhch6lEhLfNGHU/YvdVzt1zdCPj4D krmBba/fGfIsRQbGyQ6JbUMQuW4RwEJ8Uf+vwIi4PskeaM87tw6aotq45AwOvZ3C/k5u mKKI0ISYxrluKqbLMYQRoQQJYLngrJwPsKm10Ck6KJEsjv1PriZEeXFMjIyZqF3le/z/ 5LpA== X-Forwarded-Encrypted: i=1; AJvYcCV6pXH9W00DE80/3QBUYrteXM//0Qp7+6V3Dsq/JxYEbaE+z5ixXgGwKTSToSNbRJcO3u0iulY0qQ==@kvack.org X-Gm-Message-State: AOJu0YxAPfGL9LGq+mZhTiqoAmLOKu85EzVT+K7lNCg+kLemVcp8ieLj CEJkQHgNVs26Eaxw4CCBlx9el3ZjwPjO3LliJ6KyjYaXPMNBzLUuJx48TFATeZE= X-Google-Smtp-Source: AGHT+IElSFOoF4KMPSsF0Gy5tbhJtfEKgqkCJ+pjJm8bK93AAf5vkcbpbvWiZRIPotD+TEYG0fABhg== X-Received: by 2002:a05:6a00:b53:b0:70d:2fb5:f996 with SMTP id d2e1a72fcca58-714234922a8mr5431869b3a.11.1724310902611; Thu, 22 Aug 2024 00:15:02 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.14.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:15:02 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 13/14] mm: pgtable: remove pte_offset_map_nolock() Date: Thu, 22 Aug 2024 15:13:28 +0800 Message-Id: <061a61c5871948680488ad02fd1e03547948a500.1724310149.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 5oey9dowysjx8w6gtiqe1eq9ogu8a1tb X-Rspam-User: X-Rspamd-Queue-Id: 043EFA0013 X-Rspamd-Server: rspam02 X-HE-Tag: 1724310903-297752 X-HE-Meta: U2FsdGVkX19kOe5Qnwnim9HhM9Q4Po+zGjlwfKSHfoLBcpKy8hXeA4pJqistx6GZ13WIf6YEyKiu4Pevdfd2hhx9yKBBOqHgIivvIY/6kPpT+VKv9qcH4/7/jQzL83v4rsEq6DUo9o4zzhd22NwL4dhGWwZBkEOZGp0ZYgww/MUYFzRCINhnX6K96tGh7ZyveSiU30dfBBSzzsKRW4JbC3p1rkyjXp8qIH905ClArcQhti28KgAXANiAipsSWe8AY3Kq5v8+c345UQFaScZiUNPxm9SIa7sU3JqiVklrgaSL0Jb/Jt5eTQ4iQj0cJ8DKwewap1iKP8wj5GlJLX2j9N6rmmjnT8nw5mwlaYw3zHyM9NX0JrAzWPQA9rAY5vEWNPx2bT0whD1ASEa9SwUrNUyaaxuzn6hGPjdSMe6FtBOB0LhtA6cgUb4Oh/JzwHPvsaqkO8QO4nqhZITjf2mBLSQK3rnXmZMG+/iIzTEo3n89MU9jiAl5H0R8FcZbimuBZYfupUMkX7CCy2p/W+N5YP9lOkgwkHuf5mpJIIq2h3/eqvgRtkKcG15e7NI0MxZEYjU7abcmid2zOGvHONT8/FcrC2k+HV2vyjbgJ4O56vAUnNMRjmVb/UZaknbYQbIho9fihjjyeukD+xXmIT7YrGNGFzcWQKMeFPUv5RPbFxXg+JVnUSC6ycYlGo1dLMCv3T7J/XRIQRXGbWttRRGcxBIZOOPBIHCzeF+GoHrcjeq3tyad4wUzFlRF0bGT0z1GnNUlIh2iqiDwxXLaUizBtEvdDpFoxNfajRhzHlTCFaUWmTf8HhYr2HlBl612QElwvqdoNKIh3obwdBgaK1WvSKNkt4A1HPI73D+2xiZWK/lVnWbiH92QlR83FEtDqtCZxZiJHOU2ZVITfM9y9K483JvuIkCg6NLxMwQlpdkRE2OrdJGuaLSUgkV/7bfXBE5m0lJf9Eq85RqgIcC2qfy 2u3fMWx7 dZgM/YkqSLIgzlAcdRwUX7+jD+MTP++8k0usMiZWRqnQvzO2LGBX0yk/LhkJ3bV/HJMD8u11YU2sd2LUDvCJt+oN8/HWAe3ZA6qy7VqXA8wsT8w0gL2t10OwXFCim08mLNlpdVjDniHZiqK1iSwmknnS+sxDbb8aLQD2EX5mek8KGLy+MwjP2L8nlMPXkOlYpq9MwPXhUC1N3bQxwkJnoN04ztgAj65yN0BD0ZhOl7Sp6fl+rorDMDYqfD9RPB2RIHgrVEXmQZHxSMx4E6eu0T+19eU8ofeV4f6A69L/lACcELPu54ltVuCJI43+cSY7gMJv7ViG7IQkiWMbD3GeSxLWADdPU9GkGEuPbfqDJI4FqusaiVNp/LAu7chzl/k5EwvNTjklyQbQHrakDwJkvyhxvjU1sB7VTZzeK3abIIalvN3nNPlLMwAXNXCrDKl5AU+hA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index a00cb35ce065f..1d3e8882f6ba1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2952,8 +2952,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 9a1666574c959..3b6dd8bb5b134 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -374,15 +362,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like * pte_offset_map(); but when successful, it also outputs a pointer to the * spinlock in ptlp - as pte_offset_map_lock() does, but in this case without From patchwork Thu Aug 22 07:13:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13772843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE60CC5321E for ; Thu, 22 Aug 2024 07:15:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F381E80019; Thu, 22 Aug 2024 03:15:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC2AD80017; Thu, 22 Aug 2024 03:15:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC66F80019; Thu, 22 Aug 2024 03:15:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A58A180017 for ; Thu, 22 Aug 2024 03:15:11 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 63F32C1313 for ; Thu, 22 Aug 2024 07:15:11 +0000 (UTC) X-FDA: 82479020022.04.51D7206 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf30.hostedemail.com (Postfix) with ESMTP id 859A880005 for ; Thu, 22 Aug 2024 07:15:09 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ghycqUH8; spf=pass (imf30.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724310828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3XLeH1tU+payUby5GohREupClIvRGZdbeLsp/8eA5FY=; b=7RucurS0I/C53HPAOz/5nUP1xa/Roj8deowa/ElFojcHgIAlRmpexIjaqNgoeVrxzHryUa bqaCBqm9f13VZcSar5uEXwOFKpSXIRPsno8OTpA3dTy+AOYn6xnMTTxyLRFKvDkKTq2LrD eDLzu+dbF2d6ud6GwaBhqg6Mqbcl0VY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724310828; a=rsa-sha256; cv=none; b=4JJzlcaJP9KKjhkzjgj8p/leMuFc5UU+7oeOLzzztyCZslrF67EkBvMohcGVrc++1CwZNl 3ZLyBrMbAJEuJYuWGRNXZRpdZ45M25+9lYVhasgnOXXPuTMHc29fmglNsBkn6HTHsquMLJ UYs38nD/3euFQ4cwaSJc9tPGeZgOyj0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ghycqUH8; spf=pass (imf30.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-7142448aaf9so338314b3a.1 for ; Thu, 22 Aug 2024 00:15:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724310908; x=1724915708; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3XLeH1tU+payUby5GohREupClIvRGZdbeLsp/8eA5FY=; b=ghycqUH8+Ii2t8ghynmqCUWL4IF713smQOxyQgeTV1JWWKx/P9yT/jGlLqPMSGCBeo 1OZgD0lzPRKVgNX4Q17H771Z7vLjcyu2XBSaddJnCWVgM1YaO3YTI7igsvBO98igkJXR J5U9am2KbCDQJkoT79PW1cH8wbtV1hqBEAMv3KpKVqGpakhWhwZwQzsv82JwE92g8adH eaxp5iL3YMAc0pgl+S7Z34bpqcdQoPb+CAfKOhq51H8zjBsiXO1j0kgCOLM6y8KPAcMQ KhoZsM4JYYmxqFn3lbtymfNP7xKQQUKHI2mn4q0jtW336JAvADcJ/Zq2foNO9UneiDNr XJNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724310908; x=1724915708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3XLeH1tU+payUby5GohREupClIvRGZdbeLsp/8eA5FY=; b=DWdwk0GIZSuMcv+syxM3jW19b5iXA2byV08wqjNsdzksQHPCBOVrGWb41KeCY4KoMY ZbLhk24IC/MmEdRqtDv1cVa2cyGID2uaKgkopIF7uUPUbjiZfwAtTJ6WZBp8KWVu2DpO gM28mczZH8NXqW3TiCmIaj4TJdNGk78x/DCgfeftBbTXHJVb3W4zwo5LXzwtNC+V+wLL WSSHFcJb8LBqxB5Ip+GI5ImZFDi0wD1wlLv3Th7Xs28i+P6A1iBob9ADp3GS3bfYLbTP WV8MPUcCHxXQt0uMx9B5nf7a05fnL50nVfhMzziWiuE9sCd6bNmC0WwT2G0kTOi4FQYD ENfA== X-Forwarded-Encrypted: i=1; AJvYcCWZNjhTofxDgywD8rMNCJqDigshlysTNs3LbjODxyPv0RipH8x8VQIO90eCeOLPXAGC3nJNWrce/g==@kvack.org X-Gm-Message-State: AOJu0Yz2qBtc57pwONdtkRGWpsieg1CgwIiybvIpXVLTtdtZJ/p4yaKl fNpfWi+f3NWnwJE+nYzvXrW7BDA3Y9FmLV10ioNjKzPAY4BumapQox8U1PDvszPmLz9NRwOCqlR DhSQ= X-Google-Smtp-Source: AGHT+IGjO58kdb6JjGpOfA0UnXFn2ByeOkp/uIZzxQt7YtlV+9PVIdBNBOEj4fOrvZDCpXHiWUn49Q== X-Received: by 2002:aa7:8244:0:b0:710:d294:a2b with SMTP id d2e1a72fcca58-7143176773amr3032765b3a.8.1724310908168; Thu, 22 Aug 2024 00:15:08 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([240e:473:c90:f96:d029:ea8a:4e6d:d272]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ac994a3sm695095a12.16.2024.08.22.00.15.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Aug 2024 00:15:07 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v2 14/14] mm: khugepaged: retract_page_tables() use pte_offset_map_rw_nolock() Date: Thu, 22 Aug 2024 15:13:29 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 7ttawarwk4xc8yoaq6q99yeb66ta1h8k X-Rspamd-Queue-Id: 859A880005 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724310909-90157 X-HE-Meta: U2FsdGVkX19TKmuFVRMH0bAFPEb+VCaGBff8PiFVj0sdH2s6z4LRZiHnDiGco3/47AUmlRiAB68s2QfUK0yO10xfc4P7fgUU2iBhCO13tnGbRVQO0tqMbko2HrqBZGBMQdmFEp4Aiu9jZQ9y5V72cDshDVpIx+ojVrDBbhDRYRESFAHbplSRs2ZU0hFTQT+rj/N2xfFAqi4SZHs5hHDRimZlwYxaERVr2mUIDI9s4NI+7wJKGhbrqdsTSyGu3nPxiqnSTrDiBlt5RGr5dsiJPeXHeSl+0Wsz5DXZnd6kjMbBMCLh2uN0xY53tiGKb6aCBz89e6Abvx58Bl+qxxrsRNfh/M74BNvdbe1RClBRA6YVR9N+JXi0lB7X8RMpXdLHnLS2SHsOAGt/0k0JtID2abxsYCC6E86fVhrjKBGb3LShEB+wrgnVd19At7Yl1sLLbpySnqeXDGGCRPkDJjqFm8vqA9gI6LkDolxhj+efJQdYq96yA3SX2tLtkqvv48Kz3q+BlRr61y6ZwC2TucHxx6piwhHEVN/XPMIqtKptsnZFSQDPjvHKHGSKUy0Z5LASGsRfOD7YjuZDJPEb2m7GW60/IZgvoDF5udXRH6QJdRIcTTF/jjyOJtlNDHtGfLpkrDT0sWvbS+QQPbmmdxHTAOyWaEx0NmX+hja7+T1VdmJwRmQW2HcAa7WcDnOIdVz1nRhSm6udRVYbbaBUs0nuupczQfST75jSKastGRUxsInwuAvl/9r3SFBqmdYAzbPzzqOlrZzAoEtq28A2NjmQEn4ggkJjDBQkFuQkqaHMyYoZoUd+kx4m7mqQTOZGLOlA8+bkNj5ELsM/MIbuPHu0pv0UqSpV2bJzjW+DiEJ/BSJbn8Qpc9CUVHPWdO+LA/6wx8U8jKRr9a1gSijjRJhVJ6bm7cTk8OCo9CX+tEzAMZMfiadzVTG5vlt+DGU7gNSY1GyeRE/CQKUy1i083z7 sU5qNrck c2gHP5WUhwXxYj2NMu5/c+keEau3Xh+hZttFvzTCOYMEKRY0BgCWLdwYE65b6FUsntIG70GPGUszP5JbBf4UVdJQFDHoMxH0YyRjRArepdHVah6eGF/dLiIi3CWaOlADMGzkXYg1sAvULOfE+eOqLyAmgtY/kKuBb8yDGebaX88/dkIB8i+HG2pBCy7qwJqAjqlCZPcY2Y/XCwNU78Kilh7eRW3SkyYKBOqvaE2c8Ic0AdG+YL0NqOyGjl0A5P+f2d8lR/KIo3ubCK4Nq+W3mfDW7IWE8Mom2JRhTF69ostDwDhXjJOWA4MLTaiNytCo93kl54cc1m4FWj2hFpQaCC4iFv2rrdHAymcOHVvbTyjRvlCQMrEeao5uhdO93PMrZrX4Pb2q8ZexVtXp61YntihSXnAXohU/PgqOA93h92ag7z6WCZ9tz8bhaQ+KgIF6INzrI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In retract_page_tables(), we may modify the pmd entry after acquiring the pml and ptl, so we should also check whether the pmd entry is stable. Using pte_offset_map_rw_nolock() + pmd_same() to do it. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 15d3f7f3c65f2..799412041d262 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1723,6 +1723,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) spinlock_t *pml; spinlock_t *ptl; bool skipped_uffd = false; + pte_t *pte; /* * Check vma->anon_vma to exclude MAP_PRIVATE mappings that @@ -1758,11 +1759,25 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) addr, addr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); + pte = pte_offset_map_rw_nolock(mm, pmd, addr, &pgt_pmd, &ptl); + if (!pte) { + mmu_notifier_invalidate_range_end(&range); + continue; + } + pml = pmd_lock(mm, pmd); - ptl = pte_lockptr(mm, pmd); if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + if (ptl != pml) + spin_unlock(pml); + mmu_notifier_invalidate_range_end(&range); + continue; + } + pte_unmap(pte); + /* * Huge page lock is still held, so normally the page table * must remain empty; and we have already skipped anon_vma