From patchwork Tue Sep 24 06:09:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A022CF9C72 for ; Tue, 24 Sep 2024 06:10:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D56A6B0085; Tue, 24 Sep 2024 02:10:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 868116B0088; Tue, 24 Sep 2024 02:10:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 688DD6B0089; Tue, 24 Sep 2024 02:10:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4978A6B0085 for ; Tue, 24 Sep 2024 02:10:38 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BC5DF8182E for ; Tue, 24 Sep 2024 06:10:37 +0000 (UTC) X-FDA: 82598607714.03.0C23487 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf07.hostedemail.com (Postfix) with ESMTP id E2DBF4000B for ; Tue, 24 Sep 2024 06:10:35 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=eEUo1TDT; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; b=0OCzCEIJVfl6xhpgVuhUN4Mr5kvhE9+5nxnWoaafPb0bw+QhH3kZoGeJxSZsQGAQLAnSKf 6d5A3yocJogQL8OkLwAWSB4fp7OXXN1Wqe1vdVHlaVKs8WIHJnMyqXZeRX1uNtPtlEclp5 cjyf/eUZXaC8bgnS8DlKHWZCaFg9X18= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158118; a=rsa-sha256; cv=none; b=GUf0ta8rq2X0pKxbGyeBCsiNedo6EIZmuoWyrdeD0sW601msHhImD/VFbDLr+52pYvOq6/ rmoM+vX1lCXd64ElVJYh+IHfoEYNe8cCFluYGv3zttG2NzfiYHGqQT26jRNAGEtmyksKrq aIFGSJ2RgXGUdJ1TNHazDIpRKS/ltt4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=eEUo1TDT; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-20573eb852aso38676125ad.1 for ; Mon, 23 Sep 2024 23:10:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158234; x=1727763034; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; b=eEUo1TDT0YsEGsSimwsvavkTReiN7JFNNojmwYHjGqRTjTdQmF+NveLOPKzpowu6O0 9nhIDWtZ6fFZGEbcxzqm/j0c86B+NoyOri3l7cAHTFq5U0JfUw6yEwdy7jWKDhWA3q32 wWEDLGHOWi4kmDv9lGdJIXAobjQ9nYs9ecBuUXE5Ryssmkv3LRNmcT73wkD8ekzEPCc7 6viLIjOmUloM+itxyg42MdrX9D/fZgcaPKhywVNXQZwhidffwJ0KbU16KfnwlUqOC/mH XqyHSLjS3MkVQ19TAWWgQp4KR0aMoqcKdOhlIKoYj5ukP6xcXwW7ehTv+cRDc8+NX2Jy vArQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158234; x=1727763034; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; b=B6t87E/8TuZVDe8mdH7ELgFAXFb2a5GfyRqPahEnY2VUvwXJ/Tuv+cuKWCphFQgqTQ hI33KsSk/6SAAvtpJAYQEyqIZFdDqJAaMVcgMVbEMzqOrCvU0i3k1iREXQnEONlljvFy /6uVmnhqhGFwBoEcLcNl9pqnal/NP6oT+trCGpaHR4so80ej394CzXorYEdE8yqNtOIc v7nn34E00ksKaXQnM8CWG2iA3maB0U8TV2UQKvMxZik2E0XryD40lMt0tcu27jVKxN4e spaBoYXqni5MHy/+zfWNXXVQWLlpyo1bxAyGtZrzxYqf12ZuiqX8ojRdCDIBWbpUc11Q dXng== X-Forwarded-Encrypted: i=1; AJvYcCUOsv4WlzxxPzLCibRsiqr9u//Zh9bhWjGJ4/gm22JyaWnrMt667quoLKygSRdO/S5zB5DqzUsp4A==@kvack.org X-Gm-Message-State: AOJu0Yw/8j0az2/wXAyTUZjxqY5a84yF+rlqKrIbGu0iwDtShLKO7gSD LexIyqsVUqZWjazpCfHxJLGuvNg9IUxeeL8l2TjVD3Q69U7urmqb4QSramJ2TxU= X-Google-Smtp-Source: AGHT+IFurxEbSfxWnkgUvSmQpfQ7iOb3of4fLdIGxM8iu8OEamvgIzhTlXEYzcUx/o7HLvoMu2wctg== X-Received: by 2002:a17:902:ce8c:b0:207:6d2:1aa5 with SMTP id d9443c01a7336-20aed0b299cmr27872445ad.13.1727158234439; Mon, 23 Sep 2024 23:10:34 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:33 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 01/13] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Tue, 24 Sep 2024 14:09:53 +0800 Message-Id: <1a4fea06f8cada72553a8d8992a92e9c09f2c9d4.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: E2DBF4000B X-Stat-Signature: m1rsk4sy655h9um3aj1m6wbnmqxnfkm5 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727158235-675915 X-HE-Meta: U2FsdGVkX189+0YIvfVGOoADiQS6vrojRX1mmkP2ewu6ReM5UK8mzy3xCpz7HycUniZgGfiIQyQdLpYSRAbdDk3JfjDnqlPx56YkKYErYjc5KLs7hpXlwlxO0kr3zWGhzRYcK1UqeTlUldrtzyR1hE5nwLsdzF/TeSV7plO/oSa1XJFeoEmAxEYND5IhkI08OOcIWfmiISMi4AtZBeWEivg2AbwuqrtmyH4E+dGnupodRL3+0hCwXt48ZcPkxIjDB4PEjqq4jlEwYGJiIkRa3yeoeuDNxEI/k0eYizyL63b4ucGlXazjXVH2T7lCcLgPBO99KkfgLfdXMebXm4LcBjfhiZ1LwIHP8dN7sdVa/Hm9frbf9KRNc5ILdwLxIkzkUh9NWU/c2CbzouFK7/5UBS+GZYQYR7VpOoTNTkHlmIC0gt0JRp+VPl04IDf+R/Lm23sTOqWFe+T3fcgN2ADJBeSu769wDVusesgamO01fhACGHuISekVVAR/vfq6U+qLaJBvIUVJbkBp27v/fhI276WKlbuvHhCaTFv7ZxaW77vEx5hwbHBASC9MbYc/oLEP6RrrKDh/rQlEZtviy16H+bQVP+6eOb3J71cB7oeGFPVFlXEHyL7oo1VUWPpEVdFxP3r9WZ6LABN5PWB1mxX/98bjQ8xH9YlAAz61noU2ez9tjRw1MK3+J9BB48NGLc48S/aS+ouspWcA0VvB4jbaNOGue6imIw9ls0DYYgN1wvXJuBAERHgeMIQsyIIZQ9z6X6BNuSUZVVM6zip98S+VbTQsRlqjKtqENqgZoCdqYISLHurjK7V0Hn1WyM0FH4rHNCUbFButZr53mpGIHfS6wIbTtB/GhuVLWUfEVXmI9x8VtHAvSeKkxXE7iVm+z8OEcCWoT/fmBSNBLrrrXhm2/KHLP063agq2iG0croUmwNY0/mY++lp1vskhWYIC7Wl5QrsMFLtzZzBANE0lELw KhRQKJO1 W1GYMDWY2Es2aLx4ULov0dj9SxGAQW4AEK75vfyHrHk2l/A8//RLqIPLsxk+R3/9NKfHkNpjoH/pzt2gFuioVVlTFFpZXZdUBMIQudw3caFz/V0+TAksGLNCOhBA2x7ILypZiRiQ1uM4ywnAkuDgEiUtLgOPu/HAbRPFF8Houyl+vmSBWqAtM0QrLFOuIQG9ur/nh/EhnLVRdzQayb8bfNYOxbpGVWcqcxDC2t+Lf+5O+PB7UP+JFDvVJ1pCLGBJdIRnereRW94j9SMIubE+6FEMTYzhQVN/N8Y2H+VfuAREB2mqcd1bJ6QfN5Vyrqat/U3cOBqeBhXJ/L1rzORxtUKX3CEJjR2VsHnab8pG7bDRAIQsWw9qxF14KjSNKmtnXdUO6dRWeyHOIS2wCAko8MlFx4KsmeBZqI6QdtCuoo1iHt8C0Sb7JGWf+Cw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. It is applicable for may-write cases where any modification operations to the page table may happen after the corresponding spinlock is held afterwards. But the users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will be read-only/read-write protected. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 7 +++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 50 ++++++++++++++++++++++ 3 files changed, 62 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index 546a9406859ad..9a4550cd830c9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3017,6 +3017,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..262b7065a5a2e 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,33 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + VM_WARN_ON_ONCE(!pmdvalp); + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + *pmdvalp = pmdval; + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +383,29 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_ro_nolock() provides the correct spinlock + * pointer for the page table that it returns. Even after grabbing the spinlock, + * we might be looking either at a page table that is still mapped or one that + * was unmapped and is about to get freed. But for R/O access this is sufficient. + * So it is only applicable for read-only cases where any modification operations + * to the page table are not allowed even if the corresponding spinlock is held + * afterwards. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the pdmval. + * It is applicable for may-write cases where any modification operations to the + * page table may happen after the corresponding spinlock is held afterwards. + * But the users should make sure the page table is stable like checking pte_same() + * or checking pmd_same() by using the output pmdval before performing the write + * operations. + * + * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will + * be read-only/read-write protected. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Tue Sep 24 06:09:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4569CCF9C71 for ; Tue, 24 Sep 2024 06:10:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D41986B0089; Tue, 24 Sep 2024 02:10:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CCB0E6B008A; Tue, 24 Sep 2024 02:10:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B444E6B008C; Tue, 24 Sep 2024 02:10:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 953766B0089 for ; Tue, 24 Sep 2024 02:10:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4B2C381856 for ; Tue, 24 Sep 2024 06:10:45 +0000 (UTC) X-FDA: 82598608050.12.80C2DC9 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf29.hostedemail.com (Postfix) with ESMTP id E339612000A for ; Tue, 24 Sep 2024 06:10:41 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=XuHwaBlJ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158207; a=rsa-sha256; cv=none; b=TJsxw15pXGcrp2gIuC+yq2Vu3wCzH53AQVBXUR1R2k6ZCikw3UOo9rfqDdF2lZXXC9CFtN J6fTihBDmo4quY0AU4TWYbuuaAGF0E+1T3cpz+JSJ6x9LPg6QHEctCQCH6st7ZeoM89L36 wDlx39h973O/tad25AwduMM2o3q0F60= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=XuHwaBlJ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=txlUqhEzY2QkF+uTmFi5qff/qBpAZHnEVVUWZYVpe9FwZ8eIDvfCOqf4YbmaDBzg0WweQY uyLZ5EQGZb4Z3gFyTBE9sbRb2tECrZ4mFHFKTx6r4oZh+6f1VJtoRBClfGZIGiOkeUDLVK BTIqc1b1fTKTLMS96oGml8QQ1AueSyE= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-206aee40676so41320235ad.0 for ; Mon, 23 Sep 2024 23:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158240; x=1727763040; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=XuHwaBlJr87JEuPEZxcrxdfVn7xGjF/rmUNWJzjlXefv80pBgWgeryjLDajUAneV0x ReRS9z8C6MfCrkusSj44CAJ/sPIGifVeL1RuKVPvEd6SOK72KnzdInbLQ5UgT1jkqulU aSTTFfoTfoxdnwr4EC8oK5Uk7U6ZZQJaqJzdUiKNzbyCC/ttEcmRAtzdc6dfcU/5IO51 ADWl6Xd8WmIzulz42V5TZ7Odr27GDsmbjW8zTNPTUFBtNMx/w7fGwRXk8okQBBotujFL VpyuhagLSgvCpUdX6AWopnZMKRpOi3cUIz3RYkEGx2pYVYmEKJLN5h3sLkqT08nd1VlI tBIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158240; x=1727763040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=IYsNpkciKkLSrX5e8uTzgKYybqP3UsYGry52/P77y/AUvl5kc/bxuPngBT+QVUh9sF 08g3KlavOU3yRxpQDmGEL+RWTQZzC6q2DLXj0u7K4Q37ULQXZbhWI04L++By0oGj4BkX R0b4x5XM5998+2CdxDtcAMYfEptmXN1G+QIfl0KnF4vEM6DP3hw/LaUej++ce87l76Cf VVXXt3ruz0Yd4dlq4IQCyBOVkQVvmK1UQ4x8usNGtfGB6avl3FK2kE2itpajRH7RiYN7 k1k8iyRU16ywPpqUXho+fv0RoOqzNqUDGf6QCCYz7w4CRM+jCd2Cb/ieeaH7kqlf32Av GuQQ== X-Forwarded-Encrypted: i=1; AJvYcCV3nfiPrDGaBT/yACnQiArD0jKLXlMKkzW8o3mO0/APjk+iY9TLJiT8jNP0kEENBGr64WliyI4T/g==@kvack.org X-Gm-Message-State: AOJu0YwrxGV96Re6+c65QlTX7gjrR9I1wMWTpOdau/doE6iy+zutwte2 QiwbjxwPSCQb8VJUKyibb+K5ksYOGeKxovsRAcVwyMKp2zm/Hik685P3xXkBgXw= X-Google-Smtp-Source: AGHT+IElcy6/dcdtPcpdDMoiIIDu5/E2AQfQDwpdj/dfRLSSVL2tlA+KvKWV7BmaFFYnOD/6ZCb7Sg== X-Received: by 2002:a17:903:2311:b0:206:a935:2f8 with SMTP id d9443c01a7336-208d832a910mr203885105ad.2.1727158240521; Mon, 23 Sep 2024 23:10:40 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:40 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 02/13] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:54 +0800 Message-Id: <597a93841525dbd58fbebcfdc547f2080694835b.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: E339612000A X-Rspamd-Server: rspam01 X-Stat-Signature: wmbpd4n5d7n1dcsh6cgb8m6eswq8hkif X-HE-Tag: 1727158241-506374 X-HE-Meta: U2FsdGVkX1++quHHByU02miTq1SWuA5+DYDQyiYwx7RFLy1uCtGtjg3gjy0zA8v7D4naM9egyBdrNa/KJPPze4pz+M9/QKPMEj+G3IK+P/DOPwABbnIpW2F3rPo/IZA8baQ5T3IpQPFjdZyDTW/tLH7MPA7db8NENcqgHDeRlmSkDJp3Jg8H7bIzixFOLNCu8GrE0VvuSAl0a2M7u6HprdaOXQqU3+Wf+ujCiqCC56qG1Bo+YgHKvhPjPVmk7tZlu1jMfiK0u+tLTcYYyvi1yH1Cwau8UkgO/2ohZH8wvyCSFm1BdOhlRphOlXtgbmxjk2diCVhkmfSSUB8+sN0GK+aOjGdKjTgBilNSAzREbPS2XqJ4yMOfjj7UQ3t6pKmGHRW9QkuIUvpdXxxJSvFtmVYAfD9jg/ojC4hqmWXU/Rqgowcce0rO00y34MQBCHVLFeBjgW07LHNHzyF00S/3yUdDnjxV2QguwkiGbKTijDymOLqYp4srPZV8wOP9oPzIPyh1+RPe7I4ZP9cNpx06s0v4zxoMQJvXFxUFRJuayvQmBDgodNIihrCg6fWjXwnHpGOXtGMFbwoFjF4pcX/0CMTfYFi7c6I7290v7rZ2BWTuNUCSdKIksvA+8XpUBz5QhRRw37jyyQ1ktlfPKS0Gb7reEx0UcIS20n6W/jZ0UDFtxZUqG8Tc+hAKCR6U/p4nmZUqbtaHDxSZOvezPQZjlkpcNJKsGQl3rR9rWte47oihNogbARbnFnGsTwMyQjpW3MiwdvjkyJRzmGosSHgfD9yk9kxSz720bwWGQ21elVqPcp6Ih48/TltZlp4C7j8U3j3ulOjLbLnGHp5ANjNVyIERBUmHpjYj0VaybOfPYqL7NECVDDzXZILTL8tip9vEaVd3kU/AAWlxWJrX1u1nCfhz7+5Jx+xM5S+ScEivL844vk/3WHIFLRKkMlBIs9Qv4/IqPccrOp6S59FgwtF oXFelNGz hafI630FfWP1n5GA7Bm5hA2WIi6+TiXxBymw/usgfzNWxg1boav1+UzMw1+37SUG0xdomHPanK5NMMB5SiOoCw4eC5q8Zoaj7bXcsPjq4a+Lsnmg737+OuZuQLpRuJDm2wNZXIYfTS+xPEtlNuPerOoOZKFKRuohxurnkLJHVioUA3F18anvNeFIroNPnCvS1isQvD7bl6fw8g9X2k4WG2mfnC8m17YZTJqCyfbrcvfYNWqxS8QWOj5lNdGDEIUcZJ5STyrGqx8gmwccHMnq15zy/xN/4NzMJqYlJAfAZ4HU4jPpiM+b2/M1VFo9VTYuW2XPtirRm0hBxZ+SKhEb3NwdiHN7IjqrgXaPp7rxr76sf9TyvPRAHlLJ57H7h16+H8g4cU0QTqqq806Dth47vq8cwldmSyjK43sfCWal+222/XEd6rm9Ix3FsF8GtHR7EjXaUDu00R4SPn+I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Tue Sep 24 06:09:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1C64CF9C6B for ; Tue, 24 Sep 2024 06:10:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 48F2E6B008C; Tue, 24 Sep 2024 02:10:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 418056B0092; Tue, 24 Sep 2024 02:10:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 291E06B0093; Tue, 24 Sep 2024 02:10:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 05AB26B008C for ; Tue, 24 Sep 2024 02:10:50 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 825AB8185B for ; Tue, 24 Sep 2024 06:10:50 +0000 (UTC) X-FDA: 82598608260.08.8A3D68E Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf26.hostedemail.com (Postfix) with ESMTP id A7641140009 for ; Tue, 24 Sep 2024 06:10:48 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=APEjLCx0; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158234; a=rsa-sha256; cv=none; b=2Z6qvdaTOBB8H7RRyYxSTPmZFd/0OFdtKMRNQCfg3EGOnmzbPbIZowjV3WsChOzHS8i1+a 6M2yaFu8y9PS0XHPBnpV5VoV7hlVAWhCU6vRF3LTz+p5T5PKTEiQOJOkKda56aGgpFflp8 8zNqeSCc0RP5q7rziS1K0GQdrOValYQ= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=APEjLCx0; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158234; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=HIrn78hoObIGwGEWI9V1EyLaAbd4LUUR+nRP6Qm1VSM2jTzMR/DpUAI83BIVfw1kxSURkr oVpOINrqrEDQjLlCE6DTcT0Yj/kb+e7D2LJJPKtnyjx9W+hhP0wD8N54MRa/9jy6DRv2ZF uJ0IrhmnorlJyqTsaFquLdbMov2NoEQ= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2068bee21d8so51317845ad.2 for ; Mon, 23 Sep 2024 23:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158247; x=1727763047; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=APEjLCx0rS/kePaDF3kXqjA3QnEKbiFuWipA3ubLkrX9yXqS6ilADTELTP32muqLU7 LKJorxSpIIswSToOV8qgzLdA+9p1J765zJnit7aKGEUX0tWXPk/7lkdpI6tI2K86MGv8 vAF7mfH6BTe2D/sp4NSE54z5mmEeWhyRoPNr4rHgdUWr45OZEMDDxGpxAoz9uHQFj4nG 7TUYcuCN0qaYuL0X7WTdtmoOMxNpI0thH2WQGkU/D8BQ/PlAZx/xv76gHO0cSWf2QHNY t+Z2MHqrqcnEDHFmkSc4nNbt59ez6+Nc9XNgPrByxWyLSQOWynqMPlm6RNQj14mFoTlL uSDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158247; x=1727763047; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=FrBsV1yvmWyxwosVOtebLoEXkGoVGwk0AJWzUoK+SRDp3jAY/h33lMQJrTfc5Yyvyy 7SgOWmAZ787ijBzBF53l2kBvSI7d662z1yrqQcOSeBXGJQmA+CB42XtN7Wvz/kUrIFkT NjhIEyf7jYSerJZYhcOoCnolYIjHo7nTv6XT/ADj5kuUVF1oWJjUy8FSWJv21Ngk7cOE 4I/FgNxTnRzt/f+qbA24REMiL34CNmLv7CS5uFu2AEZB2UCiqeKg57vP41UH+BOMLe+V EvmQrB+Tdf8JcxTm21Ry1J5ozyH+2GFL7xLb07kjkZfmDzvwEG1WEiPLa031FUJCPh8B IEuQ== X-Forwarded-Encrypted: i=1; AJvYcCUkER8C04xRH2tzqETTQSUW4ab9ZZDoxoHV6rKFln0KWyVHinH9bIYzEU1ZZwYD4iGsFwH2VoGHMg==@kvack.org X-Gm-Message-State: AOJu0YwSTawOEvE9FDmWyHwt2qahntu0GqtKs6gqwG/4czAmRwn+K48x WusU/tDoboWJiRomajn3yj49NdbyhdU8i80PBCM7Vv2bjOZFswu94B1xC9gKr2c= X-Google-Smtp-Source: AGHT+IGm87aqttqV2Aef0OLuq60VY3Mp4w41J7dgMXq2Y+1L52786e2U6HUOab2ZHlW+VZaRsZgruQ== X-Received: by 2002:a17:903:22c5:b0:205:6f2d:adf7 with SMTP id d9443c01a7336-208d83a8387mr177369015ad.21.1727158246651; Mon, 23 Sep 2024 23:10:46 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:46 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 03/13] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:55 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: a8yi8rsy4s6jfe5z1ax3we34rqmjd6xm X-Rspamd-Queue-Id: A7641140009 X-Rspamd-Server: rspam02 X-HE-Tag: 1727158248-468553 X-HE-Meta: U2FsdGVkX1+Ek2jpOHJccJZ43tuCoqguFMh3KuTI5i7otsPczLyDdhozJDyAtCluCC3wJX/V4EbFhkEywdzGLR/3SH3JJzeYjL9VaDP5SfuiWHukFteYUriplAqBmm38/JNTfSetMoiTjOarJ0TVKYwKVpdHAPNt1Q4OcPhaJX0AGGWx2XZ1aLaUUsEiS3Q3WpZrqsKFN8EVsRzVpDx6EB127QpVwrf16oXtXe1nx5Vy8a7TYgmDV63nKd3MqLiUwnZacWzxs+O6jx5XWXOAt6kqFBgBpHl/euhVxMEJ8kpfq996n5Ku2JuBpsJMxRnCxoSFLjvBowE2mkfIfWb8cEyRdrx1iL+7AZVk3c6vUSgJ1HslepkCsAkBBFcLCqD6KD0MRjjY1j7FrpkbQM9RBam95Cvbp4sA0P6vrIAql13tgtwiRGf7TfXVMtCDwgLe5ppY04W574xTulAH/dXMKQAwyqtb5kUZlhoZd7j1WrHFVWHGUnq7udd33R6Lgf6vy96PLxGnLyK3jmAroeqjaoNFn5QOd4XaoO6aQ7Fwz9jeCogDyjgBsn9fMi3hx57E2Cohc6Huz6FZevs+Dn09KHYrVFD1/0v9DjOAX3k3dwAzaM3gfaapyMJbpbur/V1XC6dDDW0d75PRMBS6FQpHOHyOpcTQDvcTLRsrHlCB3W728zSK11mIS6SYGkIg95WeooK5rrufHdJ1vldDHn3KtJH1ivEKGLWrCMCVK5SXXdErM6A6qRiap+DEr9WFU+lGDq73gtihJGeXisyUooegfKoRaRCBBRRfF0uPE8XiGvKY3/5VeJjBbKuvsG4vx3YyueeA2smb/fG61lFQr+/GpbISTT7Oz3oeNx3EO2pcrGf3UEvo6hg+Eizvt0c9tZZJBDmm+npxlz/roT85SVzSfK3XpB1AqAIv0H8UuZ6LM5Dx2JhoGylpFaDAs/FbSHxybdQVBPqPBraDgifGBu8 WqwPZB3B jasQGuZnMNTIzoqz/d0GJaElDTFi+VNqV1fRdnzPuMtVgf7WFkKzK39cJ9BHy4urAwWeECGFCQlw5eZHxoPbrwkleLYaNVQ9e1D2FfvbHtAfzyrTyWE8nF/UaNtdiNAbCJtgDc0c/CxofLsR0i1Dyy6xH0IspuPaiAu0MQVaCn+ToVBj2Z/L9Wem3JIIf34o1YdTU7jg1zIqJfgw2Xxazhjv4PklYUGcPg2FunBRNDLFcZl2r87wMTVlwpzlhpjUhzr/JI3DQCltKqsI4nT3tkQPBJRXRt29dW4VS/2TV0zQRGCF305tIsz87Vu4NkmDjqSX1olq8DZNmwKVlHjnaeXmDF9HbpAitKeDNgk7tWHPD7F5nSida9h7a6aKILzNjV5oKzXv7MMESd1i4dGb8YYzhhqkaSm/kW/VduRGfxfMFv1t8uDEzgNyV4cvpBYuBXuGF9jKJPBcXS5k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3e46ca45e13dc..6c6ff8722550f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3271,8 +3271,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Tue Sep 24 06:09:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 783B1CF9C71 for ; Tue, 24 Sep 2024 06:17:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C42E6B00A9; Tue, 24 Sep 2024 02:17:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A3CC6B00AA; Tue, 24 Sep 2024 02:17:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7C2E6B00AB; Tue, 24 Sep 2024 02:17:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CAB1C6B00A9 for ; Tue, 24 Sep 2024 02:17:57 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 729451A1543 for ; Tue, 24 Sep 2024 06:17:57 +0000 (UTC) X-FDA: 82598626194.06.D00BBE4 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) by imf14.hostedemail.com (Postfix) with ESMTP id 9BB6210000A for ; Tue, 24 Sep 2024 06:17:55 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=OyoBGJD6; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.166.51 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158588; a=rsa-sha256; cv=none; b=D4uQxLXM0e5Qq6/utyutOY8fnl98rdYajzipriwGieg6dwB40ISHVyDJlG85cW8RJeEROr QHJjXqQ7fYmNBZLxYqo8nY/WBioCoW38lPJ+qXYH/fcQgtqmUtk9wEKoJNDnsr6QvRGIc8 IjwlG24HRNXGiKoo+2yj/gTTHTQagxQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=OyoBGJD6; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.166.51 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=Ux5aNP/oLGqsIM9y+xHY8KPmXx/0xGZ9RPGpTCoY21QI1NCoNSNdgapv0fD66K5kjBu9u0 Jzuhqz/5ap81fy7SY5zSMuwrDgmkFh+LN87a8vh4Xex7ptQrL9hE38AO6Bx6i/mLxx9VSn yw2v0WpG9biQV1ewpXkLc+gDLdfP2uU= Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-82aef2c1e5fso198624739f.1 for ; Mon, 23 Sep 2024 23:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158674; x=1727763474; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=OyoBGJD6rp936tP7qbev6JYkJbfAWQ0HWMuMUupk3AmgcGRdSu2K2YcoCchWSMQZCz RIplzj/IYB5r6M91rgRmaKSltizuiKy2iV5+7Quiqo5kUZnqGYJd/hS6AS7iAcgKZkXh 8r88IXuDg+m2h/6SS3iN5TOyTJIyrelegcfXffouAgY15NNkQiFoeogDuE8QTxiiLFwW 01y7N6bKpEtU4yEFr/tdtEUgVPqN9Hmo+yK15lKqbJ/zUXr4Nf3Ezs+5wjBbmStqYVb2 wB/rJ6qK2KtXoEdCWt7OUtF+iOpHnf1+xb2KnLgrugaO8QTvGvHPHLVJpGee9joOC5wx PjGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158674; x=1727763474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=BOrBV6B/kJmKXTdyQ2KNuKUEDefkn9Fu+KtpHocpwE0xfdvVmbBUTtWpkQLTsrGqL1 lUgN3tkb7Bhd47oDp59Oe9DH6PNzb4zOERk6IL8Hs18HC4rQ9HgDv5xGj2kgV6WnLgQj St+1AGSBax4nWff659pIC/Zu3g5a3bU4g80JRG3Dt3mbLE/QEgnA6GNI8xAxqZTwZMSl N0MJEGFcXBHMIf4KsOzy3Un5WTAsxaavOtesRJAv3egdkzwTvrMWOxY64VDS6neL9x7v qDGxEuS6h2asonF6G7x/F7qFw+iWuHzi8nLV8CIrv32HoI+1CKGzqg10ZYKiUI3PRu/3 ShSg== X-Forwarded-Encrypted: i=1; AJvYcCWrpus4X2rLNWp6u+NHeRE2TdW4JRlWlbJkOOuqBAMMVW4s5OfYSE1vJgsTufiaw10QnYxQp9LJGw==@kvack.org X-Gm-Message-State: AOJu0YxJwxjrzReH6RxjNd9vGuwkEEMH27epSrnC4CeBEdPp3fKI6YsV 5VQ2E1yVwMNgTwIolSngIckLIJQRj3xICW+pQrfHtsGg3CdXVDAmLmjku59gqsOKBEjC34Wu1vZ sHak= X-Google-Smtp-Source: AGHT+IEXIad82sT4qTFz/mgMdJh1108B7x8i2K/1dZJvW9IHButimBDIW5JZoOsyX1eaQQUsC7JSYg== X-Received: by 2002:a17:902:c94c:b0:205:4a37:b2ac with SMTP id d9443c01a7336-208d83c8698mr211905735ad.34.1727158253643; Mon, 23 Sep 2024 23:10:53 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:53 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 04/13] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:56 +0800 Message-Id: <69e1c4483c2d679d3e2c8446264e5878e5b52a97.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 9BB6210000A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ongt9yjzyy3jc9j8nwiaf83ju6igpax3 X-HE-Tag: 1727158675-888837 X-HE-Meta: U2FsdGVkX18i5InrweiS4yjZ4BuaYgUREGNc1F1N0xyIZzDZXdviGYdW4/sPxgkssV++5ZN4P+0K4das2d6RpEpwj5uATb8hqYsFM4Ap6YU3oUwlO6B3aUrpWo5rdJtR6L6UT98OXw18FaXyAu/XsDjh2csUjl9YvuJ9W4+FD+Saod6+uSXg82EKzWxBQZ+mY/fshpnMuCIIuxaPqJZZABBAHH58aDT5dwMY+N3d0jcsEbRDwkIOJFinBM7RPFppoNQtf4zHkBxaWwEV4z5g2GpVI4QgAqf3vmgYCuA0U9msFPHKZJ7j3E/D8wmbiM5hjWSxkpYNUiMQpfQehuNWHgrjqu/vKrbHjjcELQfUKFscRgp/hpk1peo4htjbkSSUnINz46iP+C8OV7JSfDqpvSm7S9T4kv5/xYYrcpGZWzv44nPcq1YQR9zUX7U7RcRT9Wwo1A8dt9H//XuXKiM93bH1h3WVFc9ffkpooZQgTeYS8UoW5X9nD2mns67HAhNypc1Vz44sTLax7+1VXZZBMZMYoXcaUuCfvIpyRPN6aokif25Tgcg8CGnficyVMcwkUWP0qvwr0lxpHzqYoKSJWu7ljWUi8qyL9h2EEwh5VjKmdMWJRylcQzgDSMiAtFkEwL+i8LE1UBdVhiXjwCMYtq88Up5vbYI257DeWB7qIX7Dk4bPlF21Z24S59PePtVHsZKw+Xwu0ozFsvV84E6sKCnNL08K9eL+pQQ2/T5qiCRwWn46v5XcCc5RQXNjhIOoS0ghiVnubbevwrVFhaTGxRYQOd5vVMT6ysFjWpPzYhsnjFB3XfkWMj4fY/n3w2pFKJJszNOJ9n5VHWvSi+Lqab62Ujpi3WgOaKxPwMm8w8EyqlZRbz3qUhcjddvQ3Q3CjuEGoi4UUcsAN/bmBk3l/gvBnRzvEzqC7O2kXYK0B0KdV38k9+9lMbPa0cFQ04S4PPiCStkDVDJ98sklF3F xGiyvPkH l54/NyD25Ex60WmPhzG7qK9Q86CLllm4feCLgyLQAoLdrsBX+Btfs+5vkBE3CSGtesFAXiXQ2tj13rKc5DOpKWqJkzTBUdU7Z0BM4FFbwGC+0GkH5bvjIieDSnN3IROYZTnmFftlV2IHNiJxz1s2ZxpDNTib9PkDZIf0Ykm2rl38b92vwuBObBCI3G+wFo2t8+c2vS/jB7CwOhrfxB6/Ifyk91nUk+p9+t/JUFbcgCkXukrKf1QWbSZrfqjBIn2ymcrrBLcBmuWQt+tNhEpvwAj1ZvGWR4z+AnmIzDB6oytQyyBq8a4KIrcTIvFU5TXASQikntxYn34t0crS26yeQiwpwT/UOexfH83IijFEfY4hKkotJ0GG2x0EkIKZbItwM8YRRKra/RbxnUMoroh7VY1b6m3Y5OMf4VrTJcZ9v++s7EfaiOTD7Bp2hd8TSrQ1YqTG8Il60AGI4E1Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff6..6498721d4783a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Tue Sep 24 06:09:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1D3FCF9C71 for ; Tue, 24 Sep 2024 06:11:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 74E6B6B0093; Tue, 24 Sep 2024 02:11:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D6F06B0095; Tue, 24 Sep 2024 02:11:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 577E36B0096; Tue, 24 Sep 2024 02:11:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 32DC06B0093 for ; Tue, 24 Sep 2024 02:11:08 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AA1F8A17F1 for ; Tue, 24 Sep 2024 06:11:07 +0000 (UTC) X-FDA: 82598608974.09.75B0048 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf26.hostedemail.com (Postfix) with ESMTP id C8C83140010 for ; Tue, 24 Sep 2024 06:11:05 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Eh59EGpj; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158205; a=rsa-sha256; cv=none; b=TbDXBrZFpWCJZSL95hUbv8A4VZldNpnGfhZSJ0d644zRDV6ZeA+AP//dHNwQFytDmKJ46Y e0me5hZ7azP/pXFz8dpZ2tLJmfWtuqE5O2hRP4dQb+Uzxr7dLChN96R94KBV0ianvRnjxo 6xOQNiYC5rDpKnyFJv3apiuEzXx+iZw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Eh59EGpj; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158205; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; b=r5ZrmaPlp42KsIyNNkx5sWMAGa9JMgkX1LEu/l5r7cmoxbaJdr3ee2dAERwr6l5bHNGAuh r/jYvsmh0sB30jGqRwIAD/D4R2Z2A9kZHSs4sT3Kx/XVYnfDovoe10vTMpIGaNR26jhOHh 7AaP5kqY624/T48dLkx0YWHy3WS09PE= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2059112f0a7so45870965ad.3 for ; Mon, 23 Sep 2024 23:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158264; x=1727763064; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; b=Eh59EGpjabFg6lCs/McEIspZS6BbKE4VgT5DGEgOmJKH2hGWYkaq/n2EhEOVAsKPFM Mu+mxs+85RN8pzBp7Wfixb8mnjHYod10jlXGP08DIqJPFUW9raiA/GiQbiZbpPU1W3HX 8zw9t/uwAYBH9IP4UrSNIJFA6u6nWiEyyle7IpjkA3BrDihd7NshraX4Y0BVIYQn+IVP jcqZBwXXLXnNEe7ZbU8VraadwEw6IEuPgW2dFpQXq7CLNrSy6Ll6z7+4IYI/3bwU3EoR uGPSbfwnrrxip4fQBQGNp/1SCq0QBDKQru+hkVhJZR7sKSRUsdOot7+lxXM9nqz5Mb3P NNiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158264; x=1727763064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; b=uNa6BIFM1RJA5o9eJOmvry5XLxBjFsvOJ7KXEFokS11n8Eu/dO0NaJTPtLGFCpBkHR BRsqnLOJLPyLmpJ5y0ugesMp57Afy1hRIblgCKpy4X8t4PCDu38sr9zzV+unOQ8vC86f b359RpdPmjjyn1sutl/73k20GQcefWSNJ5XKVpXMFeN27x5VEhVOHosqM39/HNiqWBjH qftDoDu8BBO+NMxMGuTf9NyAqM3sjj8EykQZ+GzVhzaQ28LemqS+ccnORkUss6rsQYd3 983979poEu6JXiL0cEYL5s6RfqAZMMEP5VzUnCWmLeMZEbzi0k0ojBLC4coGGl8p97HA kydQ== X-Forwarded-Encrypted: i=1; AJvYcCWZgdEYV4W9BDNgOo5Ef3e4u/ko6QEv3h2Q42zgVMWK/6Jdn7xaidQh07bkXuRafQBnU4ETb1IzhQ==@kvack.org X-Gm-Message-State: AOJu0Yz6xr9wg0GZj19YT5mmZIfM9c+xkCyNycoSX4sgI1q6rKYCx+ne CbZEftlzB/DHeDiz8Yc72YxQ7jfw8nuGuGwtcexZiZcm4jNdLqtTTHz4RJob294= X-Google-Smtp-Source: AGHT+IGIKI1reF9Jfo+dWimmN1cyjgiGD8qn8mcbfIJc2tOglwlNFzTXQQXYEG1VSSXByHdLorgo7Q== X-Received: by 2002:a17:903:944:b0:205:4e15:54c8 with SMTP id d9443c01a7336-208d8440f7amr165611365ad.61.1727158259613; Mon, 23 Sep 2024 23:10:59 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:59 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 05/13] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:57 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 43ks3uoh9ocpju6yft77ohqaixjwgawi X-Rspamd-Queue-Id: C8C83140010 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1727158265-657511 X-HE-Meta: U2FsdGVkX18EJbYd5zVGjacDKI3wAoSZvhklZ/z6BKCUm42ihT1mHmImXBS4JCHG5lbUYRnfsL5q6Geh3mN1ROq7rxdCNRFfpdN8WiBohxdwEKfOI/2xwFjiNB8G0H6Ezfwy8zavvIhgGspY0ve4SMdjewMNJRe3oUpsxHxWFw4m70B09EUgLW6vroEJRMTvsC0ikSIh+TYaOVetEOnRlCgWiduNfHYPO9qgLkrfjRMsMm9LaKIy2F4TCpFUdhaZTyyruMsqGh8UjQb/ukWzcHEVGZmiZmaKXtEgw++UaGcriVMGeNNTokl1AWn+g4k37BSBV1KzSDp0j5trQCd5Ihcad5IadX+HUe/BduuFDOrvR5IR1Bbsyc5E35gukAt36I+NrBHAlH1F9Lh+7nCXldluFNgX+0pdd80jL/gnOi7eS1XjG1vrWe4JXWTrVz7mgW3wTHQA/vKMUEsun0DUdSsVXJ+nJbrPdkEyneREYvl+WbM4CeQNUBGjScbSdAeR2wLzSbbTqY6189Msjj22i3koS4qupYkgemM9hth9zj0seanxysetNNxQgYkAN81n9VD4wBrzZgOc8FBgHRTZBEjAAUxqFXHI4FtVn3yU1oB3eSdn8dubVX9HtZ5L7caZEANIgAjEuvLA4FAunqUerZin+Q5YkUjbzGk3p2Cqd/x2bVx2ktAIbUFNidn1qjI+g576j7jocVnJXG+Y7/pKVodoi/n0pN5tuYfpdRfBnIffALQpErh9ZvzlKpvi3tx7Wkqoy2HShISq8z2h4OI7gIfqNI1zS9b7piusO4RkTJX/XgvnqdnrXwyk6hctzgkXpqcIAYKG2iL3yFTXReZwMDXL+/HrXQqjhc2P4qDwPJ5zLFME2IMdmNBbqj+eBoHn6OoC3t0J7AsfIg6gE5JSkoN/+MOWv8x+J+wXQJJu2FQgA6ZqEbOYfJ4gg5gqjPrMYp1IhbKqcIDt0UrDvZx xR6TroCp hH7NQHY6WFsnijSgqShpfnvyhXRz22lZB9qGffaZFA1vSobCnhC7vQi38vwepHMzh/Oa3Lo5FVPvCRSW7zRiQ7RJ55yJOGpk1AZJK8Avw2RhKFPc3lzSEXvFmzPSLKxy9PUN625ea/P1fURnTV+LTOYuapKJrWOe6oMbW6ibvtZEzdDEGNOUQm6/6MGeLYEa/kiCvXjSp1GdLKSHYv8uvYsC6f9JtSuF2KEaR8q8PX+ph6nPk8Tqd8KnXpfq9TfuJ0VHVrDZK5v+3mj6k5LJEErQZcE9TATODViwoV79qDYzf7yvMn4lh33RTqgPdILxSeAjBAXripYQOFrt4Q9IKienvyhHT9VFUyUhFc5sVQYk/g3SaQHX+DAKufsSM5EzxRqvZP5g1WUZJFFlkLXeFDcNTtB7Ae4feEYNNxUwdkQ/Uvt3xCCsll7ebJ+szjzHfRIVkPEhqjBkLx28= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In do_adjust_pte(), we may modify the pte entry. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. All callers of update_mmu_cache_range() hold the vmf->ptl, so we can determined whether split PTE locks is being used by doing the following, just as we do elsewhere in the kernel. ptl != vmf->ptl And then we can delete the do_pte_lock() and do_pte_unlock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- Hi David and Muchun, I did not remove your Acked-by and Reviewed-by tags since there is no functional change. arch/arm/mm/fault-armv.c | 53 +++++++++++++++++----------------------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..2bec87c3327d2 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -61,32 +61,8 @@ static int do_adjust_pte(struct vm_area_struct *vma, unsigned long address, return ret; } -#if defined(CONFIG_SPLIT_PTE_PTLOCKS) -/* - * If we are using split PTE locks, then we need to take the page - * lock here. Otherwise we are using shared mm->page_table_lock - * which is already locked, thus cannot take it. - */ -static inline void do_pte_lock(spinlock_t *ptl) -{ - /* - * Use nested version here to indicate that we are already - * holding one similar spinlock. - */ - spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); -} - -static inline void do_pte_unlock(spinlock_t *ptl) -{ - spin_unlock(ptl); -} -#else /* !defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void do_pte_lock(spinlock_t *ptl) {} -static inline void do_pte_unlock(spinlock_t *ptl) {} -#endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ - static int adjust_pte(struct vm_area_struct *vma, unsigned long address, - unsigned long pfn) + unsigned long pfn, struct vm_fault *vmf) { spinlock_t *ptl; pgd_t *pgd; @@ -94,6 +70,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,20 +89,33 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; - do_pte_lock(ptl); + /* + * If we are using split PTE locks, then we need to take the page + * lock here. Otherwise we are using shared mm->page_table_lock + * which is already locked, thus cannot take it. + */ + if (ptl != vmf->ptl) { + spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + goto again; + } + } ret = do_adjust_pte(vma, address, pfn, pte); - do_pte_unlock(ptl); + if (ptl != vmf->ptl) + spin_unlock(ptl); pte_unmap(pte); return ret; @@ -133,7 +123,8 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, static void make_coherent(struct address_space *mapping, struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned long pfn) + unsigned long addr, pte_t *ptep, unsigned long pfn, + struct vm_fault *vmf) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *mpnt; @@ -160,7 +151,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma, if (!(mpnt->vm_flags & VM_MAYSHARE)) continue; offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT; - aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn); + aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn, vmf); } flush_dcache_mmap_unlock(mapping); if (aliases) @@ -203,7 +194,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma, __flush_dcache_folio(mapping, folio); if (mapping) { if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); + make_coherent(mapping, vma, addr, ptep, pfn, vmf); else if (vma->vm_flags & VM_EXEC) __flush_icache_all(); } From patchwork Tue Sep 24 06:09:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1AC4CF9C6B for ; Tue, 24 Sep 2024 06:11:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 729106B0096; Tue, 24 Sep 2024 02:11:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 68CC26B0098; Tue, 24 Sep 2024 02:11:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B5596B0099; Tue, 24 Sep 2024 02:11:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 21AF26B0096 for ; Tue, 24 Sep 2024 02:11:09 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C71EBAC79E for ; Tue, 24 Sep 2024 06:11:08 +0000 (UTC) X-FDA: 82598609016.25.13C9875 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf19.hostedemail.com (Postfix) with ESMTP id EBB511A0006 for ; Tue, 24 Sep 2024 06:11:06 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IaPGsugJ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158170; a=rsa-sha256; cv=none; b=5bcrJWuf2SLYfpqSNiYv/eJT4xs+81ppkjZTW2rhSP8UWw3crGXumjQlaxRM9KkS20Rryt SFc5aQP+Eij1SpBHkqOd+LAJ7xEvlMUlmIlPUghbEE2oIbvTFEeJi3v0cdpPE4FC2b81d1 CBeHs3fPmspdyUJ+svi//b3wW+yF488= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IaPGsugJ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=Hea/Z3DKl4YtQpCuu59i8LR7Z1fAVzP9LYZoQrJARgS0Ecb/zkX7FkcEjJCH5ScNHm5DxR 2k/LWwrqkNtmloCniWPSzuhkX8JJIye2trrAgKAN5+0ZPO1Bwo0YvWiwyMKAQzjQ+c+uxC KiB89BFTu1ubMWGcZ3B846SAr14Gj4Y= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2059204f448so43655415ad.0 for ; Mon, 23 Sep 2024 23:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158266; x=1727763066; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=IaPGsugJlB90ba6Vm6VOpO9bLc3FBH8JJrhwoIBCLf1r/0GcAtWWfaR6G4L55B2I8J LbD1mmhfUNPKQnayaw4L6NQrgy/z/ehNuYj9/e/EhNsHYTXpKcCb2CWPZid4tvvxK/wx yQLDepj1jBkftwu7CZ/pnYH6CeeOJHz0TVptNG/BhOxKvwdqOVezqZRzkodW32FVjJOz jjJW+6uCKIt+MGMqUSN6emQwdXmCKrstx6tt9B0wHbha8GfyVKyahYUhFBPHdxc/xtsC U9yIo+8yazVaz24U6BDwY9uTU91vJqJsmFtZorrOQ7NUG+ChvJZczrYKQneV+reNER56 hkVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158266; x=1727763066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=i6F1Eexhvnow0/Znv5xg83Yv5Vk0wAGFnVookp1nJbnxFAtwBBnuY8ki0oHCIs2Et6 Q+jCf7wQL2Ses9c9Lj9Fp4zG1xl36UTvZLViWDXFvo7Wo8HWoWcN5p0dcVidhzbmIm6w BOfcniL1YrzKsYVptP1ttoQIf22CPAqhndC+YbcuMkqIYOLNRO7CwrtwbERu1G0grq3i 2F6HTHD+q+p+AA7G0ky4xXvui5BeEmlMAnbJ4H6PunUiRdGkMpkz6obsbNymC5fUFvoX cQhd6GOqbwCKcgoShxLuNzeAnaC2aAP90MDfJBiW5srePFkcsqJ6mpmaJ/AHM31U8GaO xcdg== X-Forwarded-Encrypted: i=1; AJvYcCU4AW1wE3fnJPKoGz51Y565PHXXWQFOWmLviDtzwKyQ0M9SR0ptCfxnsixkoqQher807hPKEsl+UA==@kvack.org X-Gm-Message-State: AOJu0YzXNPMU6m4CbRVlLhdZLeIaa+7Pq5lV8Rotz6A1TlsMqzXz0dfk e+xt7zTyY0J/BgYY+Ga1GUftYbHtezTbr46/KpjXXBNiv2yhKtLpjCNglyYVn78= X-Google-Smtp-Source: AGHT+IE0gKFIx3/KW4QPIJwQt51ZMLGla2X2YU2KRRy7vcX0w9t/15+gAAip1FE70+J7SP+04M4CVg== X-Received: by 2002:a17:902:d511:b0:205:7574:3b87 with SMTP id d9443c01a7336-208d980bb34mr182109485ad.15.1727158265764; Mon, 23 Sep 2024 23:11:05 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:05 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 06/13] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:58 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: EBB511A0006 X-Stat-Signature: i3nhws7toh1g436s9xjfuxbaczmsgngk X-Rspam-User: X-HE-Tag: 1727158266-61533 X-HE-Meta: U2FsdGVkX1+Ekw9E/vEfrj3QF19LtJl0MS80Y5ofYRqS9e/UqlSWzdz/qgBLsFSGss6bEL0JIfNplQ9SBAPPcssOKiw/8BiGl8UPVp2vbfFU8Hv7fTE9SjGpUEH+v0ZOcq8WxaNwoSkGTS75zkrixYHZ6POYXnhMMB3nNvxQZb1hn98SNrnCzZZ2zFZZ4+SWyh+cPxX9X49Lf0AbG5ySjgISejieNuQIJa4M9MEDQE4QiLyX/wkhEEbHAUVrbD60fvy3Y7Rzr8MwrljyuEWmrZbb4Y4KK41c8lvG8EPVYUjD07/6mmTWaMfeztf7soQ8OyWGkkJAUN/H/FPYgTKtJznoOS8KlMs3jbPZA51h7h1uXtfP/O1NKvx9G8/H0gijeDoeptiVDzNuTB8IwpIKfQNolvZ47LNkTKKTPctIBjm+orUc7aLLYgKTsCxXVvnLbVGephnzoMbRUk07SXz0KsPnHOjykt0UedGeEOoYdJGvjlvTDJ8DrlueQGIOAnwjTNY/2xw7tlXWDZjjo7Voz6SAe6EABau+2ufuhAuQWm3ZC3enNVuxh6Lls1sAT1TDBJTODaAF3H7V+S+rVWPAYHmozDXhpPTNesXGJ6VrhsFFfQuR32EQokBWS4bZFV2XMKIjYjzCrdpAXZo8ShnI602DjHosSJBRD8WqIkDM+oLDd5G31hub1c+qgyKUZEISk9OJjJ2vDNNB5tTgffkPX3Q9J38ficGbjgN87tc+5bcwwo6HbiJTEK8JkuSOAghDOaO7lSVyzzTRimPRqMrTBCICHIXlu2M3SeYAV9Z5Kf52BI3NrFgLbEm2wriNuQytw+IwtYfarU4vYS/xXqIQVG/OohV9rNGZrStbMl4Ul+QJ10Tgnfik8bN5GU9RpKYjxPa9IYg8EG0yE4d9Ut9qncUzWz9vhrfUNGwHKXnzxnWjvDzCHNfxaJ4Oxeg8xgfNkI6TN7pSzT6R1c40/fZ wC1faVcy mmDr4bXp5HdIcq8X8Vl45beGvBvoQcyU7MwQfAIC1JQ+OEXkmlDuqOzUh3h48BYhM0Kf+fJyWpSrbsUgHHPPO+2OSjUDaVJYLwsILO8WoXu827ZhFckivw0M0+MCmomxxaNIUvzpae2Mko7xrY3FLygpGQ8LA+ILMgW5tM/K7Tb+bh2b69V2VbnabTSwBR1DF9yTB4C87ruEe0aFxZwptiMzph54bfVDf94/H+ECFnbLZ6uNcQ+Ru9wynuVnV2/d6FQM9OXPSCt/cEfA+6lTXH9d+Ay04tmIq68dt5euGJ9GyogCZuUIe0+rzyV/nnMEwCgtnq1+rZbJdnOq4EVzRZ3uK8P31uiNfEti1wvn37K0sschHIziw5K8s/nKSMVd3FFpsOqS6R1WwK9U9NXEENhHC57hNLF3vbxCQv4zTBxL0eTLcPhRftPlYYLVAPc0Ttm1dWUYTABS4JUA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a245d28787034..6432b636d1ba7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5750,14 +5750,24 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte may be + * modified, but since we will use pte_same() to detect the + * change of the !pte_none() entry, there is no need to recheck + * the pmdval. Here we chooes to pass a dummy variable instead + * of NULL, which helps new user think about why this place is + * special. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Tue Sep 24 06:09:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D78ACF9C71 for ; Tue, 24 Sep 2024 06:11:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3EB66B0099; Tue, 24 Sep 2024 02:11:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC68D6B009A; Tue, 24 Sep 2024 02:11:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D41186B009B; Tue, 24 Sep 2024 02:11:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B2D1E6B0099 for ; Tue, 24 Sep 2024 02:11:15 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4B7F3141810 for ; Tue, 24 Sep 2024 06:11:15 +0000 (UTC) X-FDA: 82598609310.07.F2A7405 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf19.hostedemail.com (Postfix) with ESMTP id 70B431A000F for ; Tue, 24 Sep 2024 06:11:13 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BRuIzasA; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158116; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; b=6odYzg6DYOko/HiMxoCfXr3N+Y25DGUFR8pp1TM/4Fd2EI/oQ+ez9QsuqyVG9CN3JVORFu CnhoI/URVKJ1DfR2TGbnwogDeZYfOBcon9j3OwPWIBWGlyJ21zoOu8TsdfywzntZaJ7xBz RqnvU76ppgiUePAUIBVV4s/DYCGqxfs= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BRuIzasA; spf=pass (imf19.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158116; a=rsa-sha256; cv=none; b=nMQji2Qa7IuCjM92dLqrRxp6/RLh8qKFhtovJacK4erHbMtY5fIwXuunBVHBl3mDNJO9Cc B+/jIe6E9J/BG1bOauJRsvrYgLV4CdYgIfCe0juiEhhcHGOFBB6kt6wtqaLLTxbx0i/+0r MZlCRmTK+fnDNyO7xLVQmkKo7ORYQ8E= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-206e614953aso49438485ad.1 for ; Mon, 23 Sep 2024 23:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158272; x=1727763072; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; b=BRuIzasADsWYMWpZxGf0jO99iPZmsxsDX1IcN6BBieHMCeNPhED3wgSFgfcXFMtuX7 Xo1bMlFB2gyuCXtRTloIll+VApPrTmcXSO8TTq5RP1mYAItfU7wSNRBKzI/xgAb0j0tS GvFHiIdZtts0/ZkoHuaiHQBYAhT3Ua0/YkvNo4TrvXueTXKACqldaDCYOFb9LNyHpqnT 2l7Vy4t8Snj1wvXQNHFkf9Zaf2k2TI1w8iVz7np2XjrqJn80i4j8dzj+QKNes09f6FXc q31Z0t+dwJ4PpTvkzx+2b1Nl41glvf0tORs64oHzDyuECghEcUFlvByzCwVAN2d3ppx0 GAhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158272; x=1727763072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; b=eMhUSPdlAM+8RqQAx+/uN2wSHugl9omNmRIMDEhS5FAYNl5qy4Bd7wjziWX2d8CfuK 3m5tXdtBoUKPc1uQ0QKK6x1sy9+W36vQkE8WITTvVaf7ACAOIvcvOG9LYL7hLXfLoCcL +OS0bICO8YD+KDFnZYAYcTvtAynueunIPbxzQLxDRvuH2/pUjWd07hlektJnjA3YIGnD T3jjdn5i1ZzFatfbVvt6dSPCVukXQxqwKRqgZ3VfDi5Ddm/NPQcKUZRoIYTC+6XimWhB yuYbD5u9V3k9gG+bjcbsUYsRgpFYNzv/tHq+AH04lcOKvxEUbkI7FxAH363zsfMRTnLC O/tg== X-Forwarded-Encrypted: i=1; AJvYcCWwFVYu9LoHT+SxIsxyFPFo+AmMe/hWyKm5ITVwiFMrelj0P+F/GqlZywUO+OURKkNO1++k/+5TCA==@kvack.org X-Gm-Message-State: AOJu0YysLFJRNwaFHOLHpUyVrH1JC1WSUV9dT1Zc3l3uJpDv0Yj5vvIf 8kxxMgQ441hHCQuCy0vxwCjStGsdf1c1Cp+cXY68FK69dSfuCpHUBHBr9COl37I= X-Google-Smtp-Source: AGHT+IGXhl7eW49ih0T9YK4dpESD/v76/2vHb7m+lE9WqCGFaYPlARnoD/c7DbHxV22Za7qr6kwfpA== X-Received: by 2002:a17:902:f54f:b0:205:755c:dde6 with SMTP id d9443c01a7336-208d83cc8b6mr212103045ad.30.1727158272107; Mon, 23 Sep 2024 23:11:12 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:11 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 07/13] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:59 +0800 Message-Id: <07d975c50fe09c246e087303b39998430b1a66bd.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 70B431A000F X-Stat-Signature: 377hqqhpw38xhqrms1yggiwoeh9jbsxk X-Rspam-User: X-HE-Tag: 1727158273-204345 X-HE-Meta: U2FsdGVkX1+XMFiP3mWMdWX42rw4q2H6lqei+VfFZYZfZD+yw5LcEwd3Wiky0aWt5s6pTh6BqX64CQtu8Wy80J5YqBfdehAwt9jgQedQlAiLGuLSHapQRaIzTX274EYgvJ2+mYOHh5PKoOf2wNKEFe8sD5LUelt/4o9eoDry0dp0WBmQTH7PLCnH1foVjNspYp97Eq5i92V8O/Kj5dUoLkTdRmJMd33U2R4wgYsrm9dTJXcaoKG0p0g/mcULpYS0WOGhg/u2SLi8wLDqRbs+rtCivk4mQyOrmJHQLi6MGTvEEJBhXtJW47pmoPrAt1EufOvmbInGAF7dtk5koEV9I/x2UpunjFNRMHVbhSOY3ZaA43LEy7uwRCudC1sF4ADZujC7/hpEeSSEL0sjSrSzbPx9fK5zqhdYETzfjHuacjtwI54UvrjA8JAr279slz+5WDbtFsILqIME/jg64sCDdeJxBxirniPiKirUZiY2wJoviAKIXkwaX5ghaJ56x7C9MRThzJ1WNWWe0cliwXShORVQqqLNnrakDzjAcKw6Un1ZiPem9HgiWTIhqbxURfcNmBWQzlP4uSf9+eoyF5qTPTbx88EBbwP8WANivqRrh8j7hD3O9xXMXZPUoDGkv+1xT9TIUDlkQmRw02FFaOrNjspJ70htdR+rBNVznJEtyx8lYwd9NFPp8YRmC5D4xeFGKQoMh4taHrn8O4s7/3vHMGTK9Dtfpi/g6gFQwNnZhELRjViv8OQiILGdbsMpf/BaIxtzSHi+8aCyi5CXXmZIvMA2D9+6nZp8DisULmwt01KyQSxA5pajsmDAvMuHiPTLBhsETjBPmMb+Ddgc0dtbDQW+oQufinwUlug/HsX4Ck8NlTh68aN1D1JFgBTm5U9lHmno8h2Va0S7pPdS0eCWVf58a+YkRrEut5T3xzATB1DfM4NigdMqxR92gswGzFMmGeNql1vf9rn27wFxjOS Pe8AXYZy Q4H6QeQ1LZidyv/Sq8XvoLIvfj/SvHPGu+L5Mi+YPUi/wGdmbNDS3sXmGRA414l3asz/yRdO2Omsr9nrwDqRzknB8YLMvmgdFOgZoaiUfPptcqv2JpDHPkLFQaW5MrA+5Kxrj4/a1+Ovq771nrZLPIm47cGLjPG6FttrcbXDKQkAPRtH4wsy5pRYhrSloj/2cX0IBFL1taXGrTNjq5SvxtvZMYXeYri0sh0uKqSRHXSjKfeVQse87T+dNW9Q37R++pPlyX2b57qN5M+jmn9AwRySaH31e+Pyow91jKmVgCmLrcbcv2Xkh1VndHolDWrxDhsHgyMN77XMZ5dB2ZVE22sUec/xdrdbVIjdvk0M3YVbt6IIN+W5lJ70FzqGwPY/D6YxDsKvgK9aDWhferXNsM9jgdEQoRdlHTDsLY8h/rUji5AEK9CuniWysWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6498721d4783a..8ab79c13d077f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1605,7 +1605,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1613,6 +1613,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1645,7 +1648,6 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, nr_ptes++; } - pte_unmap(start_pte); if (!pml) spin_unlock(ptl); @@ -1658,13 +1660,19 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); - if (ptl != pml) + if (ptl != pml) { spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + spin_unlock(pml); + goto abort; + } + } } pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd); pmdp_get_lockless_sync(); if (ptl != pml) spin_unlock(ptl); + pte_unmap(start_pte); spin_unlock(pml); mmu_notifier_invalidate_range_end(&range); From patchwork Tue Sep 24 06:10:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5757CF9C71 for ; Tue, 24 Sep 2024 06:11:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37E326B009B; Tue, 24 Sep 2024 02:11:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32DA96B009C; Tue, 24 Sep 2024 02:11:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CE666B009D; Tue, 24 Sep 2024 02:11:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EC4956B009B for ; Tue, 24 Sep 2024 02:11:21 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A7B08816DB for ; Tue, 24 Sep 2024 06:11:21 +0000 (UTC) X-FDA: 82598609562.22.DEA8C5F Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf05.hostedemail.com (Postfix) with ESMTP id C6AC210000F for ; Tue, 24 Sep 2024 06:11:19 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rz+GP4hH; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; b=RpXp9QJozs7a1iaID/mKVn4S99HK7j0Q8Bhoffgy/9Ch4EXztubCYihx57Yvmesf4/qWaK rKHOxl30wUhx3nq9kn5loQiQFDKZ8aWt1nGbXLRgGogKQ3LiYs8ZMA+xrEyd94yDREJ0gp HlcAFUx9cQImIcs11xGMcK3TO/aSdRM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158160; a=rsa-sha256; cv=none; b=6ARnjGiUkp8ciXONfMsyJ0wOPnkQYrd3eRM+dw8ne9Urxs0xXue+zuotqBVldQq6TXI9jx 2tUAQA2Qb36ucUKxxspwEPBEWBKHJee1m4d1TD9XtMs4UcsiEp7qb6FgkDtZS59uHcyFGq T9rn7XvziZeCuZd5CycyVipkJsNDlyM= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rz+GP4hH; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2054e22ce3fso50850475ad.2 for ; Mon, 23 Sep 2024 23:11:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158278; x=1727763078; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; b=Rz+GP4hHN/mx/CJHP+0csw3UaUu9SMrfwfF8gdHuwbs0XNRR+Zx5jqbXLQ1gReiuG9 iEnJhDAIOKKMGvcmFbi3cqe398ar/Le6iEQOmgF8r/becy1TkWED/MmutpfLYzTyeArb SKKl0YM9p6kfjhuwV++hOAauwTZDp8/iQI1ewTHalNv42N8TaWjDJpqBV+++6ec17A9r BfepOyZvOc/MBLR1ZM5PItYt05Ciaa9aafz6WTuz9XPSE4AfcD518lyzzGDOCJVPpyIi vdsE9P5v/tOoWB0szw52iQ6DpzGfTe8U72AeNyrwlWB6yLxEVjXpTGvQsfPcnEtUcbIG BwFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158278; x=1727763078; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; b=qAW/ebsOkTB4BhNF66vwX/2kNqtpvWcHm7r1Wcso5uWOJKoL5caR1zjCOC+4K2SAa1 52cSWORyibi0bzetBS++A4tYFShofutKIpoUFCG3qeJo/EoGuUym4dUWL2cl6UbsVzKD DwjIGTtb2KvZBshnigBAlsoMcp82bV+fYO4cJBB/WQIRBSD/5L+LWSrluVHyboAkCa/p GFTLLXw7bDCvyw9jW4k1fEXRu26ih2ORjaQy2CpvjUUZd9J0YkEE5QJbRASZozTgB9Ce n46ILmA6g+qTqcGaaF1/Lb2ERG1Y8Rr2jezyTEJuRgqAltUkVgzkAhtWsqeYnonta9U/ lo1A== X-Forwarded-Encrypted: i=1; AJvYcCXUcZQNmAVrgxkbpfsja6ML1bSJGljZJkn9FtHvQ+EdFY2zF3Eq8irfpIuTMpU4e0b1+5ViVIcQFw==@kvack.org X-Gm-Message-State: AOJu0YyjQyXWqZGfm1QW+E6OKCaA9zlnOTgGVz6R1DzJjwEsrvGsfbCF EGy6Xc/i/e7ymt4GD5mmttMuzLRc64kkRN1mqCAiOcRxfworZfGeuA2ijZy9ZpM= X-Google-Smtp-Source: AGHT+IHw+OqtuQ5LCBY4YmqC9EJn7stUTlaTSc5oItaKMOqNO6mqJktvnwlzFLYt5xYGl9LymCl5gw== X-Received: by 2002:a17:902:d50f:b0:207:1709:380 with SMTP id d9443c01a7336-208d838ede3mr189786005ad.27.1727158278506; Mon, 23 Sep 2024 23:11:18 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:17 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 08/13] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:00 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: o9zasriwriaxzcp8ds1p98yzdq3gqxu9 X-Rspamd-Queue-Id: C6AC210000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1727158279-860445 X-HE-Meta: U2FsdGVkX19ENyhVYiXx3OggBALiYDZfLrzxgWyNLNqZKmh9tpJSPShikNXgHGW51NcXxPk2+N0wrzuDsNDqa2hkdDOHuaxtfGAbWfiGm8fFXNlAqDKEMlH4pRsmAMEllxj6VivqXPh2AsZUY+CHteSfHMYhmiF/ii6YEfvm5OESV5Bwj2zGx7gyh/MV0o/K4nJ41EQ+iVZzyHwTA45Qx8ZxWtYZELywHXx1UYsbrjxs1qSVMbdyZJ60DUiQYLkuBbLKdDtKGU8VatHCA6bPupKRoE+Sleu2sz5WL8328xUJO3910d7Yo1hlUxfOVnzUtufnOxRY+O8FWFdALOzeKoQuzbVKaKN6QUQgD8o9O9VnDRJBruAvldrY9odEhRlCYgAF3ZeweJWa2L4vlPrN1PK+fEf+Jzua200Du/I+RROXqia12ety1/KGVgIvEk3lPJGPl02nGOtdLL75SfhRa1o/rMC7SqyjmLu/RvYWDRHHBG+Z+x8dVSsXnPW7cO5MQrRrcaowpbBE9H08qvnr3iZaW3+f7hOp4g2EUzUafuYMtjoUqkaLGYeK/BKM/Z44mnVOtjYRcvw4mOR883zD5PR3CvHCbcpEKH1LVCXhtIeK6inQtoW25bMQWTpr0t7ZjG4teR26IJj9xMcWo7T+wjq45cAUg6xYcC94QS90qA12rAO9ijhyoIQhrLCwrRUNjrBmr8kfJYX9W6/GfinDlphB0IEpn7b+L0nDi1gs5wk6dCSaH0xcctS6LFUF0AkJH9Z/t2LuiAk1vAJ02DW/jFwB/5HidBDVSCBFnY0YGrEsD4LzIybgRNQMte8bfLPQP3afeXUnHGL4BsJumwcZDFU7x/UQbaeMZX2qoqv7g0co6CFRvIPNGEKw2mM5uAnMlbN2SgH2I0fDx3miNoamoFt5BsmyJ81MsekcUf+Z3ug5LORAH7wsyLazuQyeK831AeeWY91SL65+ViLsC7x M+Xsmi6U zVMFhHI639NLb6Z35wvdvZvr+HXSIuuemOOD+vwPczMtavZ+toKtGsJwVhGsUh2Hekdb0ww/HqnGLE75926kyW0LteNpFNAswpq+e+Ew1rnibOZ9tc6yaCkMcVN1NOSwsn86KR5Tju1KOpC3t3xigVHGVD8M/8PR8OctvJGhFKwDTibHDZhp5cQlBVdEyibgF7+zAb26Inuu4RjtVA3UHO6QfLDs4KDKHg8nuemXfe5wTZrdwMjicXs5mlcN5FiaaT6xp3Oe5qQFNKWNHWymwwelgwcmHH3O5QlJYO/B6EDQD6/9uVhNNTVQ21IffuAEN+9P8EBepRAtMMH4Nh8/YQ5z3YmZTIk15Czh8jRZyDqrRR90tbnK4pPdOWqEasAiKSxdA5GQBZwJPK3k0wd1Vw8Ilj4ZNL97rjhhmaKvBCVUag9vz55xZUq1q2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we already hold the exclusive mmap_lock, and the copy_pte_range() and retract_page_tables() are using vma->anon_vma to be exclusive, so the PTE page is stable, there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/memory.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6432b636d1ba7..c19cf14e1c565 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1086,6 +1086,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t dummy_pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1111,7 +1112,15 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * We already hold the exclusive mmap_lock, the copy_pte_range() and + * retract_page_tables() are using vma->anon_vma to be exclusive, so + * the PTE page is stable, and there is no need to get pmdval and do + * pmd_same() check. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &dummy_pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Tue Sep 24 06:10:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63026CF9C71 for ; Tue, 24 Sep 2024 06:11:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E52EB6B009D; Tue, 24 Sep 2024 02:11:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E03EE6B009E; Tue, 24 Sep 2024 02:11:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA3C16B009F; Tue, 24 Sep 2024 02:11:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A74C46B009D for ; Tue, 24 Sep 2024 02:11:27 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6D862C180B for ; Tue, 24 Sep 2024 06:11:27 +0000 (UTC) X-FDA: 82598609814.07.4082674 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf21.hostedemail.com (Postfix) with ESMTP id 947981C0008 for ; Tue, 24 Sep 2024 06:11:25 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=NC7k5pQz; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158225; a=rsa-sha256; cv=none; b=n44eAn1MlgIkARuSdMUaOXqn6YNOexZpoRU0bWUW+QwygYOo2mliE8Fc6y4IO7xOUycX65 xAjO2FxDnokRzpLQ84ILlExSnR3ybdcorDmqOMZKmTs9SxItJ9v78VmZRlLBPpjkgIVHpe Q/HBBrkEtx65sFTrmigZzeF6AxhC/dw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=NC7k5pQz; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158225; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; b=cDmHejFIpTq6Fo2AL364ymjWHRzCOyO37jDCsNEdNTxTzr+OKFrs8Ofe4hqj/N8y6GvCwe rX9gIMT6UScdvMtBFFvuGqta6utgkf4qoW49BX4n+I4+hdIRnv5S19kDZ8skaEQm4EylM8 OwK4mtp5vBPN6X9/GTU5Tn1vovfl+Mc= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2055136b612so61986375ad.0 for ; Mon, 23 Sep 2024 23:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158284; x=1727763084; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; b=NC7k5pQzcVeOJFo880WXhgW7hAJmJEWmpY7tlJelbfEuSuon5lmzica30E1oB8YNTC LLYKSc076cjO9F+W0U1WaDc1yghm1nKa7k7FNb0KZR7QJHEwQiJZiIkFS5JZN3tBTki5 Dbjcd3iqPSkfbkqdBB4SmpJzVqBCuy/5BpnWfYeAb9onjLbAii9C8+n+j08YqJ5MCP8y p94jIMP8tqdDkRvMVxJYuLyppNG7edIcfAk1/19GDP6+kJECe8uaWvL1HkLmGpTIcaTv 0TBBttZ5KcrHPNQQusTu1GwuHpYtuA/XuwDT0BPZzyHxsj6+sp/zmEQkPw0rYfx5sgPZ JGtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158284; x=1727763084; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; b=SItsLHeAgFq5fKmGFsS+O1wHEPyROTApFVehjNpQm6ixybN2SWYOwX4WCZjRY71uam d3ACW41Su1UAYtwH8o6LLrfcFT9kuqKorQBT3IAmrKv3DxJ8h8A6kvvyHGca+CYcueUq XH7C9YZXPrN3FEmbsXO6VEjLedhyC5kPsvSUtuNLCJq6+Ro8LaHLGX9fr1CPoE0JfBsr X5U7XvS8ZWKR+XrHsXg2Hh2oKTC7C8Octc373CMFDU5vjWb/TaXVsvAB01018BAipOnv cwTSrEYmdo7Z/rLH3HYP604qo2pRMwrOBIEaTIXUHHLdWPJ+CcqqaHm7m4XVHfvpSzuC i7IQ== X-Forwarded-Encrypted: i=1; AJvYcCX90u0MWWZbcXv+SaUCpVZierE0KmjP3r5lAMs2l2fq3+KyWzKzV115V7cRH/uy876FtPiwwq4Qvg==@kvack.org X-Gm-Message-State: AOJu0YxYuQhdJKHu01ncqEa6vUF49EPRnNAXnz4deL4ojE98UKIdE1xA iUjEgFfES2ytuSWy31lNEU0nA580Tz1g95sieSDOycgzWmbwqI5XIFzJRNYoCgU= X-Google-Smtp-Source: AGHT+IEhn+iGj1vkzgTAalYoZbG5psMaFd6djM60OU+3J7JbD8pdYTlvGGhg/wgUQ8cNcQFEVSSWnw== X-Received: by 2002:a17:902:d483:b0:206:aac4:b844 with SMTP id d9443c01a7336-208d97e3f78mr215574535ad.6.1727158284398; Mon, 23 Sep 2024 23:11:24 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:24 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 09/13] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:01 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: pzyj5tz7sdb373rwgegfyt6wxt6rgwc8 X-Rspamd-Queue-Id: 947981C0008 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1727158285-495822 X-HE-Meta: U2FsdGVkX18XCG835c2/Tx6MGvx5G61n/DVsZwbV4KIybZKsgVjlgPF9JICNB3PzKA99UxVmc4UhPKcsGnJcxR+bSKU9xRtAyY4zb9dR9pQr3v8VlTLNkdY4wBiAjcZTMHJNAuVSiL+MRMmdtEU1I7W2ojVNUIQeEN8JLeDocGD20LUnQJXyEjgrFKGB3plo6YgEBl8x/rgkWeag5pkR67PDx1kQN61HowJaBCmtGEhbm9c4VrRgMC1Qp2YLSmd7wFYd2YWcm2L/FpPTw08EcW5dC5CcBc9yHPr4q0Etd7LFG+eUT/pmp2GkJqutByl+ayf4jh5ob3Ugi7bEEPDHJt8Lki1uhTmhOW7JKe4+6sIPNwFCW8OHCDI1K4zz8Trfxac/g+GEXh5RYPt0TRuy9I2YIWj75x24dLtekteUTzpDohL/J/i6Qey6joApApQhBFQRWHsDHCwC40OvgBWSdAlRRhzKXGpZgfV8oMaxGzCrJNoijmjEpILO0yFCc1XQd+QaJ/fb7kuhm4J/Nr5Iru1zROaEd62jUcZnotus8tjUP1/XFljzI/gMJdHM8fPo11+kmqVoImO8xamQRvd+a7uELCK4ZtXL3IFC+ZN8+nMlzcb5VKBSsAm1g+01tBNVQ+Ml1fHHNV7ATlpgwejYVjxNUjMd5R8VQtDu2a/mzZKcaPWl7+qKM0CW4Imqe+2Cdh7XP85tkC34U7Inot8DdDBU/1Ntqz7mM1GGkU+dQAhH/1dOukHjOdc9b2kfEt3UdMzVfTMHyYyAzxkpKSJfNRFSLuILvba6xbIuvo+Lkr6xhan/nmXoTgH9I0mct1NKwGsj1xyxqO8CCBPGF+VC5WKmM0Is1Tfrrt0op0n217BUabjP6ptgg+YREhoUNd/M4QRhybLp3nNAJSdFvL3nz0ZG110VrHEHXGPA/yUQsJU6lxoxQqaIDdMwjKgD40WI6RjbkIP1HsaV6rorVPF If8Ct1l4 1ErB8c+dt2sTu6brnqZwgnEVvoq6jQ+xZJ/peJpjfidiEF4eKx/xHl3Cp02QSJBx4D7r5BJ3qonV9cS0OCwqLmqDnmnUMj64OjkxRxkvniNSztiMaTWVxmHgPBr51LEIl+GhEcmIRj3m9/V5/pMwXViT2RpSP9JsfNY8caO/kB0/YzPOtEtHeB3t69MKDbQ+kTShEO7B60/JSY+Spz14TwRmei92no1sP6xla79WzAxt1D5oKB7eCByZ30Lcq9gprKVsxb+1jBjcc8/c5TC7/2sAyF2LsJ8jlVQtg+lFMNYb3odd2jLixrU5OeB/j3Xv/CrGM+Rn4/IVrNADrkzNlE18ov/3uFFSn3tKJ/o9Kq3AES+pCmk+5HocR/plkhj480cGjg82c2p1fv2PYOT4PZGJ1iDj6w9XoDsGgiyr66O1vBQw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Now new_pte is none, so hpage_collapse_scan_file() path can not find this by traversing file->f_mapping, so there is no concurrency with retract_page_tables(). In addition, we already hold the exclusive mmap_lock, so this new_pte page is stable, so there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..9dffd4a5b4d18 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t dummy_pmdval; int err = 0; /* @@ -175,7 +176,15 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Now new_pte is none, so hpage_collapse_scan_file() path can not find + * this by traversing file->f_mapping, so there is no concurrency with + * retract_page_tables(). In addition, we already hold the exclusive + * mmap_lock, so this new_pte page is stable, so there is no need to get + * pmdval and do pmd_same() check. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Tue Sep 24 06:10:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C03B2CF9C6B for ; Tue, 24 Sep 2024 06:11:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57AA36B009F; Tue, 24 Sep 2024 02:11:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 502806B00A0; Tue, 24 Sep 2024 02:11:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A3636B00A1; Tue, 24 Sep 2024 02:11:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 188406B009F for ; Tue, 24 Sep 2024 02:11:34 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8EB1DC1808 for ; Tue, 24 Sep 2024 06:11:33 +0000 (UTC) X-FDA: 82598610066.18.89D77FA Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf06.hostedemail.com (Postfix) with ESMTP id BFC20180006 for ; Tue, 24 Sep 2024 06:11:31 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WT+R5YZ8; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158277; a=rsa-sha256; cv=none; b=p6LILTCyYN8WOV5VcZbuZFpfBvdo0IVxLdttB/l4Ak8fwc0w2KLPfpNzu/eO3u+yb0mf1T TlG3LjqjnQnDo6i5Xwz+6KLKzojJ+uiSJSAhLe2czOF28slc0yGXXEZG2+J5q2v2yb+po/ 9/DI2rgVl4pCFSErFBWy3wn62J7tAcM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WT+R5YZ8; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; b=wDi2JU/blrxadqZRG4JDj3UxjhzsdLO54vEEhefqK/2UdYzMMs34n2RH/0fTMRpjcnJj9A 04odsGBpca+vJisUtgJPdxVXsVx5BxIdkSRSf92Btia6KvT6ByKFF+b3QS82dTqjXwQTI3 UNHRM+Sh2+k1LQCXvLOTisVPFlBvISI= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2053f6b8201so44772395ad.2 for ; Mon, 23 Sep 2024 23:11:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158290; x=1727763090; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; b=WT+R5YZ8nYQwwOGYgWXACGLKGdUqIyjcrhcwuJ3M1zdZsJ2NWlDDYXOLIBWAYUqgK8 Gmlvv8HtrVBYosw7o/l1mAqJGZ+Kj+KtaNoC4et6QwXN/aoKNVdwU1HRwts+q8/4tkit nFQLxdT0COFmfAiWvrkoXK/Mmg9d4NfCy0brd0o4htp5jZVu3e8JUWni35VAq9UX77Q/ ydmB+TANTymz0wxlfX1idkvU7JY1kQscoJkVGGxPnxPEtgtqK0ZgLhrRaVPc2jLKDkzm TusGR1TiHkNTiFH5aAa4R3c38Xwp48GOAmjD3PIQcbAGGFYDDA1wMuDSXQ6N/JH0WEvS HyiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158290; x=1727763090; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; b=n+bTZ1ZOTizjZHhfHdBxwz/+i46ChH1FYXGLRNd/I3Qk32mo/V6gsWXkDri9ea4lOD YVQ3lQV+eGl0GMP4mNeEvi06Wy8b2P7aB+MgmNV3uQeKyVABnoUIRmU1t3jfGaakwh/Y VCmZVHBfPqKw/3DPGUreDEgrxcWHLqzhI5ZO1yDbUPj/zrN38DoSRTnC+vzj4TSHPszB 9rkMRovruY99CV4Dep7BJSwcq1tMKyRk43DSrF7wEhu6LrTALgVPORJZMF3PVuOKORz9 TAjEM2sq997mlT+1p1LjHs6iTJajLYBHE/EE+9aGzPBq/2Ye/FBRKtj8+JHVuaTfv9BE qcvQ== X-Forwarded-Encrypted: i=1; AJvYcCXRr5KSWiEI0gsj+FWh2eTFJ+qtUYjnYvKPky9JZt8ftHEsckyyBaxjb7lD0j3cT0BwHBNuPGfGow==@kvack.org X-Gm-Message-State: AOJu0Yz+u/nJL3QqPvtq6L6wEJE93z3mq4GmxUxPyHmhv8i5WUiJDhel EEEZw2CIRfwXrsu/2yETZnc3y04PllBJua9nqwD6Z0Og2sucgZQzjHytSnxyUnw= X-Google-Smtp-Source: AGHT+IGuU8VFNPNElLsd/e1qu4NWf2AelC2dd7GFKvCMDCyRlgITelf6+XB9J1CmvT70UykrnIRqmQ== X-Received: by 2002:a17:902:f542:b0:201:f70a:7492 with SMTP id d9443c01a7336-208d98603b2mr202721885ad.53.1727158290456; Mon, 23 Sep 2024 23:11:30 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:30 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 10/13] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:02 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 8kyrzxz5hwmz8o6gwejgggnzosozak7p X-Rspamd-Queue-Id: BFC20180006 X-Rspamd-Server: rspam02 X-HE-Tag: 1727158291-56258 X-HE-Meta: U2FsdGVkX1/XfGU5n3cStVryuSy0KdNycYrK7gFKKZ8Js/9Va8aP5ftloMzf9cmGs/cshuKk0Fr9YtEKk9E0RmyDCTvX3UxUW9hvKUQtvFKgsP6fwv9HO4M2eOHs88dkz/sY92iSLjg9fHrxI9Cw1C+UwQAkXVSZGhqIlB4vHcMO6/ZiTNFpEgSTrFcjdUmFEez1UdSeusV9q4eM4lB5jehabLZrnCjjCiKcJj7CpY75WHBpN41/9mk8atUAuRuIPsJIAN6LDnst8QBMY+r8EUoGIgTachmNt7FJiX9RY64KdnzUxHeT5AVnsFjtHmjXD92ICgy8HfBTq8ui0gxbXtv/TdTT+aejW9M+uKeAVbIWcAVzchFtdaG4X7GXmHW5l/PJDHJYMRjPTMFFWTWI+Tkmldv/QFjuFR+PowF3+WHD3f8iRDWR+XXLSWmnxX2yFohtk/D/vE0jeQ3kPQiS6POuhXaglVlJcJEk4JmCU04Sfr0u/6cLq8lKT8kkI888I7/wBpsj3s3toiijp++/eOqhzE2hMW9QgFUr8dpmbPcPGlXay4n/UQ8eUSgg+/C5eWgc+9hDX4JRYbyzg6++B+zvv4iwgY3fZNXy30+FinmDHD25f52YTAWveL4stiFdrIhVEu2tI1rC3toRHiJF1Zm4DgmNGCg0koGjbqIHTecuczFym88LHbQkEj2I0Tt98pQm/iOoJjwilONdqZOmfQNgQ8FSsQMij+0XnGScFr07CQlbeMHIO0GuD9GJRRi8Z+XPrenrtzpsBl+jiXqmFS+F/zfyHiUXuSaGqrOlWxV/+za5LwJ7A4f3luvrCeBlj7MVAMCFxx9NbT7Dx5NLz0E/eow51oIVJgPzmJ1R1CdEFl8ilURfl52jGCDMkvNae0hfXvQMqKiT+yKunUyi+bm6SS6oDcgToFMbDXj030YZS/SM6vsma5kDUs6blfJ0ax66OcetsqcIfUmpbef jtdzuhVU vXoF6fKa4aJ/hERAzH3dhwB3R0rk7NuwEGrVf4a+YlcfstYsL30x1GLhr2nGQsf6AQeY1262q0dJLGhgSAXnNypouT/x5FbS26cqHQ4zWqUPIbMrzwutuFlma2+OKdUv08H8rwygIui61QOrBedcHjZrHM4SwWnavQMQgeYctpsH1Y2L1J3KYJV4IPly7PgCNIn/4S5mj/8kCeeuVeSN1ykqJdGd8/B6MndYVU49KdSGicX3+VyxpClQlZ8NPzbsAspWGDk9RYx8mp0H5S+UPINJRa567BK1Q8gTXMVR4gkmKM3bifP10YNch4FI/uIPqICW3i9pGhxEtj+J0agptd7+VXzSRyiX+58lx/0M4RVFIt0+OdhtfdVtUC/5LgOmp5MDwOagUGdB6fj1nloqb7uSsyF9A/wJqraJWvuNDU82t9F2lUCAayB+mrQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng --- mm/page_vma_mapped.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..6410f29b37c1b 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,9 +13,11 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; + pmd_t pmdval; if (pvmw->flags & PVMW_SYNC) { /* Use the stricter lookup */ @@ -25,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,10 +35,11 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, &pmdval, ptlp); if (!pvmw->pte) return false; + *pmdvalp = pmdval; ptent = ptep_get(pvmw->pte); @@ -67,8 +71,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } else if (!pte_present(ptent)) { return false; } + spin_lock(*ptlp); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, *ptlp); + goto again; + } pvmw->ptl = *ptlp; - spin_lock(pvmw->ptl); + return true; } @@ -278,7 +287,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -307,6 +316,12 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!pvmw->ptl) { pvmw->ptl = ptl; spin_lock(pvmw->ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, pvmw->ptl); + pvmw->ptl = NULL; + pvmw->pte = NULL; + goto restart; + } } goto this_pte; } while (pvmw->address < end); From patchwork Tue Sep 24 06:10:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB1DFCF9C6B for ; Tue, 24 Sep 2024 06:11:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E0B76B00A1; Tue, 24 Sep 2024 02:11:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 467876B00A2; Tue, 24 Sep 2024 02:11:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E2446B00A3; Tue, 24 Sep 2024 02:11:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0E4A66B00A1 for ; Tue, 24 Sep 2024 02:11:40 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BE1D71C70EC for ; Tue, 24 Sep 2024 06:11:39 +0000 (UTC) X-FDA: 82598610318.18.8DFB158 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf22.hostedemail.com (Postfix) with ESMTP id DD7B7C0005 for ; Tue, 24 Sep 2024 06:11:37 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UwTWWbpo; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158210; a=rsa-sha256; cv=none; b=W+v4Z7EC3uWiae4dszsNssRj90CGlrtcSwhngYOks1nOs3HEQkr9nXQZnx3gGIR4Kkkvvn sqBR6tQHq0NPYc5Okr9Ce0JUNBXgODrMFdQ++BSUxp3+mEyKq8bzEQer8IIIRTuQXk1V/j qLFatYJUgGc+ahs5RXqqEUlUM0G9aNE= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UwTWWbpo; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=zXP/y0uMeTaxRjCjpoPlGoBBepizp9eT2p/oKgpkn2nZf8T0IDJO8hmBAmP9NaQsjL7EML 0PNwNQrvh4jvff4PILx1+QXXXsn9zSQqox33Y2m0ad/M9iLxaY+uY85Fyi12a+K6R6UYRA w9rH4zp04Fc8vkJkLkOXaqZPCFXGy6Y= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-206bd1c6ccdso48965525ad.3 for ; Mon, 23 Sep 2024 23:11:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158297; x=1727763097; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=UwTWWbpopMQe/EV1ZvE5tJKFY1ttAZ4xR9sRufHnsHf8vYgHsD/bELErJbjnR2XpXx WwGaCur7L2RCA4JD+j/YEUa02sAEZ4udEHBxtjjSbDvJOD7ysDFz8tTUTjuYgI6w1d4w mshvs50EKNwD4IIXowPmRLJzpeWMxaC5Dhc7yp8wlRqnv5SIgIgHa4L6mHCe6xKJxfHR iCAZ5XQxvNeSVfdD9qS6L/x9tf0anXa9SaRY2iEzpjmigeZpp4aaidZmXhqfWYs3cVkv IZ2MJ6uEfEFJlY/Jkv7uxqoo7dn48TO9+rC0jKoxgqR6ucTDIslOd8QsbSMHPEAvG3od zL8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158297; x=1727763097; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=Q35ZFyyj0m0f+a4B4txsaPmx9UI+29n+cm9MLjs1Ef4NG4Bf/KmB8HQnyT7fkMNa/T MxBeMes+cPa4hH3O3boPPD9q719JiPbxf9+1A6kSByJszvv2fZQcVBeZBroMiH9LpXce JwPepwiBsI4UexQSJ6z6zQ7ptR2inZaUnKcseidcRxLwJhFWmvTms9Q4SyBX+JTb+rf9 jJvWckNhXaAtbTAWByBkzydeRNrFCHJy1DiLioYoivyOKJkIR33mVSoPeCuACjbtP0TG XnzUYcv+oTDLlTarVrxsS66JTP07X9rrVytPYJjDAZNk/17skr1A7Kr/igJEXIh+KlBX 62Ww== X-Forwarded-Encrypted: i=1; AJvYcCXIN+xTNlXuLJVsKUSLrl9wYgR3Tt4zmqV9iv1iyBMttYOJ6kRa5/M4TqcEhM2HyNtbEbVjG8UZrQ==@kvack.org X-Gm-Message-State: AOJu0YwaDi6hQgEHX7gfHwCR4RTbkuRp+M+Ytf4Ap2QNoP4vncWIuarb oHEpghlGSRCONHfmLh9ZnSjlK0COaUlrPGEoDEbbMcMtb8v+40WSpCWzqw71CS4= X-Google-Smtp-Source: AGHT+IFBI5Ned/vbTuuRRzdnv79MFASKoi0/m2fLpf4am0H6y8GNq4qW/RgfzVT9no9zcYiBkg/Xzw== X-Received: by 2002:a17:902:ea07:b0:20a:f013:ddb5 with SMTP id d9443c01a7336-20af013e29dmr19152265ad.59.1727158296610; Mon, 23 Sep 2024 23:11:36 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:36 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 11/13] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:03 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: DD7B7C0005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: dobcoc5j9qrcxag8mp7akp68dcbydpz8 X-HE-Tag: 1727158297-490880 X-HE-Meta: U2FsdGVkX1+/MAW19ihirY4CRcW9po3rLNHGJihMUTFx+aaiJ+f5wBGDNePqASYAsBcR75c2+Ow/kvjldaWdSqI2ulZitSKq/2q9vvb0MFB4i2c219jC9UcuHaXPIrpCwdGOSyf7rB5+NXpj01BMB4yLFLuZaANSIHzs6xIoMkg1smgW8aTvuB3Us1qZhFGAiYknM8E/w4JF8HODAh3pwxiTCrj2gMcTMBdfbUOUn55Sat1uRjA3HermMDY4j0jkU1IlzD/jXqB7KipQeW9P5eyqrtP/JOGzhILKhWoY7Fug18sChDtzkbkc5I2OIvXCR/pyTOBTE8Ipjxj9lipwL/hxmHHroARbBWMO2dffAsLm9lBbyIIKpdY+k0Orhnry4rGTNnFnNSUFU9rcRumQkzdEFRdXvbh3jfcYlWi3YTZjdavrj1PoihVY+grJSM39OVS34ddKDjIeTKA53Y1ojlkpDh3K+6jnLH+l9xcDmUM8NnPQBk6ly1pq0UHlTS3/ONaBLYfY6ueD83HLAt3ORlKeqLiV/BRh7pWq7q0WlIWs8vTT+/MF3XNyJ98HD3iKjJPRuyx/tfyb5o24zTixOfhG0/nim/j3B27o2ah8CmkbMzLr+2A588R6jYUgo7dapWovUSwH4hVr1o2tQGdhItE7Q6FEuF8F4dZfRLptFc3gY4M31UEx+BXyWyWaDbmE5pnK8fs4B9oZnlv+P3ytqlp8+v/MpoB0QWI+3rKcdLKGTdJvZFuxGjGVivZDOcDCIbVlAOuwOwC9IvfnYIf0ft9lL/nJXhFg8OtDqG5b5Qc3vceIFmpNmc8NwW6N9iJZKfJAOckrg8n7gH461hFLM5r1WYmv8Yz61uF9BzXmJMBLAoc8jvOPrin0+9QlSYVnAQ7IzFKU+XVAIDR0sYFzWQ5EQCIzvclijGX5JgMeZcZONuj1y15qSp7PQSHTpHhdIHfALce8GvSysgcZG2q ZM2fgflm Twh5ETE24Ue6YjXJIJh+H819ZY+yiPuPOky70P/FsAH4MCeSughldU5D+/SCOuAfysYrYHPoJzFyUQMVQTu/i7WGJ71xGzfDAJe7JTw6TfyRlRiojOuXcYUYSYwBiZplTlk8HX7BeHbn587h7fIXzqwwa34qtmTMGY3otejsPiRa/UDiL89AvHv0cYnwS1SajblSarOMcBx7zZTqbCowvyCFQyCpVTAttWC1htT5KwCdfGlSB9X79j6u84uj68z2pZq2P0ySEy9NMBS8reJjHZkkHxJ7GzIQOXsQOiO6i9GZXQq6D1VQsWQyOBI4n4rv/UkwVc5lrfwcLPl5gguXyNpzNMtpRmBErLNfHK3SGc7pBkPidVnW3nYm0Z3rFGSJDIDI7SlyUOyG6j0gUQEDKTLFD2HsE2gPE+RL+DlXHh/D8poY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will use pte_same() to detect the change of the pte entry, there is no need to get pmdval, so just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ce13c40626472..48b87c62fc3dd 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Tue Sep 24 06:10:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2B0ECF9C6B for ; Tue, 24 Sep 2024 06:11:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 505A26B00A3; Tue, 24 Sep 2024 02:11:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B4E26B00A4; Tue, 24 Sep 2024 02:11:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32F296B00A5; Tue, 24 Sep 2024 02:11:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0FD956B00A3 for ; Tue, 24 Sep 2024 02:11:46 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BCD2D1617F5 for ; Tue, 24 Sep 2024 06:11:45 +0000 (UTC) X-FDA: 82598610570.07.E2AF191 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf15.hostedemail.com (Postfix) with ESMTP id E56A9A0002 for ; Tue, 24 Sep 2024 06:11:43 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PwciraW2; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; b=ATnLsG4UduX28K4IIwjonWiuMauK1tdOwiSxyVUydpP0FLvmhwh70FpStg+FakwPGdPhst 91d4EY6IbFpa+5JH7MkQlXCyaTBbUUFB6nTWOF7UAxAzGBUk6qM3WqY615YeMUFxjbX+Zy XQi7k34Jrynflt2+B2agt2j42YhJngU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158186; a=rsa-sha256; cv=none; b=5qOVqhAHGgf8Q5q1diHqfOWKbX9cnf6huIjz+Zqs/iSPx4HDgDe/Ok94v9h60tbuD0Cnq/ ecqUHhREk0XYhIYp1XSswEIKma1IivfyOIPusEE2CMDaJXA5WTQr0eYKQjPjrYVZ8X+ac7 ehNKl9TPbmq8w3BoLLOwC/1SE4+aRJ0= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PwciraW2; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2053525bd90so47137615ad.0 for ; Mon, 23 Sep 2024 23:11:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158303; x=1727763103; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; b=PwciraW2+m9PlVgbSMMkGVMZ2whfQGtHFlIp4YYqb8MUc0fDG+n7OEhDnGCYV5k/Bn jNP66u+tcn8lNmMWIv6MdEnywcCAi7gYSsNXDRfGrCc8CfCrDyoQy42xbVQrk8EIv7aP gDojxZ4qdm5NYjgF5STgSRdGqW8uK3gFTgnVbvW2bSsne9RM8mwxQBZCJ0eProfMxvkF DxLkbaUMkJJ0ry23bVHQIJddE3yiyJPf50k92YsTqP4orMMnzE19vxxa7VUzMMd/Qrwm xHIIlaHY8k8WKs59mnQV1dmCQ9hwKSaonKUb0RBe/5agMSjuOptQ6mnDbbrASEz8gegn tTHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158303; x=1727763103; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; b=q6mzgPVxKyxOQKjMAWokVJ48+nLqbxfwL1yNE9QJat66cMZVFoOeBIW6MoU5F0uOX8 01YmqKqq68e7ZNaF5NMa89kniqCtDqJYIAd1yOn71WQKwNGxwUjNz6BHx4yTBXrHhe+Q CNTgAxm4WGk1Q3X5SmeDvqQIHEgpqK5zyHdVmFiNTTQOn3EXtp7ykYQ3fsRXYoFWsldj lQLTWjenI62pgo2rSt+IZuedfvKInZVUPpVNL8qDPgPdbgqJQzvATgfgyLLaV2HIuc1Q nSMDwiBrq0VOktJwQXjBwAW+YUU3BKN3oyP1VEs+wrA/X0MqctG/hlUsM/W6Zf5l+nBu xqlA== X-Forwarded-Encrypted: i=1; AJvYcCU2AVkruaTITXAL6iGgcrwDJ1JdDy5nJFCHkx/C84Gh8xbTAinQfTn3nIa6K9ZpqEVxY0xiLLbHtg==@kvack.org X-Gm-Message-State: AOJu0YzLDsyTX2MGDLjkxYb6mHM/0ivlgsbhyoiE9eqPYJAAdzEkJ2YS ip/G2ro5w3IFD2Bp/fQ9/cMjdvDwtvpEhCEnSjS957zEhVbc7AC27txdRjMvo7g= X-Google-Smtp-Source: AGHT+IFl5AsK8a8BNnydzmhf2ct6wAj0Fi3O6gM7MY+2JS5/M4Off4jZQYnq6Yk2iDK5ETk+W1TbBw== X-Received: by 2002:a17:902:ec87:b0:206:b915:58e with SMTP id d9443c01a7336-208d83af7b0mr191498925ad.22.1727158302727; Mon, 23 Sep 2024 23:11:42 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:42 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 12/13] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:04 +0800 Message-Id: <6efef5316d7468d13ed5f344452a0b872481972b.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: E56A9A0002 X-Stat-Signature: tzs55r6msusbwhjw83d4wqthhgh3mz1r X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727158303-67412 X-HE-Meta: U2FsdGVkX18LZ22qLLBUyt2u+Y75oSfdgGU1Oe/DYHC+8Zl9c1pGT0UrpjxvwICY6S4iX9MB1uOAH3c058nSZi8MHY+FXYpHpExgpl1OctOnv6lcdwoeOIJSbjy866uN+i9s3zFw+8QsPmKsHYX0rvL77HCsSJhgUTxeCMvNeqsydb/KW9E+8u4xe9nkg1QsmOkQqUcavRpzgQoVAl2ULsgpI7Dvx4awhIjJo9tedFiWshQR8Obo3M7LGBONn+EepUVI9bXMsevDu9qnncXj9zTsklIZwXICrayCzkaO+Ke10tibj7AzGt4PkjJneElbsTHWwOlTzDQfuhDgyYpJtW74knlUkae9lKbKq00BhwZeBQXpEvtoFhrtzNUAxaNE/8F2DFOoD9XevSEmrt7O4d6AGTtZ+ZCEGOGBtO7VgOEEfzcnzGbaOckY38rygB32+Vh61CT4wtrOGwyeWvLy8sJxjQTH/+bzRy50s/dugM9pau8Ml59P2b78mlJtTUl71MA7B9pAZWYLEmT8Xl41p1i5SlFLfXXdlgK+Sr0C1PhnfjTnxCRxcU+mKD4cKqli7fFn6nmMc6ZBO+fgPFJJGlftEd0JTkG5fTTkVizb0ALMoIC/FZksmN7ZgClBKCFv7UcDDD2y10IPNJ6tXtT7hDJ1dfgi4Qy45I0oI0l5QruHizLb2YI5mgTzAGV9Be+TJopjDiq2NsxXSQV2J2juWS8QqYNpUMxM4995KDBAv0kQXCs6PPmEeyb5IDKly/wox/aeb9sWFt6jNocE+Oery/twJr2vlR2Fp+GM09wRg3AZc37k+sxYpmTbGRKYCvja70d119IhE2P0EaDZjX0oNusZG3yT0Be+1WI4Bc5S/tkJ4Ts72vk+cb8YTypEJ7RhO/TbGAc1OF9l6Gw4ZE8Xg2NCjyIQ4FDeU9JF4AuOG2jmdizfbVwuZbo6pVgpO1xrbc/yb36/IgqhufxyS6G Dyugpgg8 9WI++REWZoCyHMa4GaNrGOFMBH6uTXPYXeuFQuz5jRYHJLqmClrGe5MfkyI6IG0vMu7berbaMeaIHw0KOH2ft8eeesCR8Lb0waskJq6yBxyDzrmYbuzMt+abhILHUnZE9COYsaVpuCS17qPBsm/uExb6hGM2HMjg2gYUpfO2R/QWCY16BUaXu5Bfn0DFyCn0vNIkMiRLppJDz5Hn7x/s38kvnZeyhHto4+132ytFz1yHRsq3l8v9dmB1SW2/GQs6N8ndxZuVMOFESnI37d9sd4IW/8LAHojj/mSmmjabnuPzYAlfCl0Dc94+tRsQkH8SApOwR/2wSV+qmVrKR5JUZzCk0wVZ9r8rMn56uhEQd992wuOQPr8utp7noCUYP0ST7YEKLVeo9SsT/fFq/4VtSYxqCpoXeU3urWVuIzWJmCnAyisLV6QjVZCkAYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 749cdc110c745..bdca94e663bc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3375,8 +3375,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3384,6 +3386,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Tue Sep 24 06:10:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0314CF9C6B for ; Tue, 24 Sep 2024 06:11:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79CAF6B00A5; Tue, 24 Sep 2024 02:11:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 727A06B00A6; Tue, 24 Sep 2024 02:11:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 578306B00A7; Tue, 24 Sep 2024 02:11:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 390376B00A5 for ; Tue, 24 Sep 2024 02:11:52 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EFAD6817F9 for ; Tue, 24 Sep 2024 06:11:51 +0000 (UTC) X-FDA: 82598610822.20.D3FC437 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 0D4AD180011 for ; Tue, 24 Sep 2024 06:11:49 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=iBG8FHOt; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158192; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; b=TDDOWYC5M/y5kFxOG69QtwpD+32trgyUSj2gfHi0if0GnDNy9bZAGVUDRPFEW8535R5FLu lGfIbKSN0gbuUyT4k0DCsB7NZWr20KDJEhVieUSLwPZ3FtmZ0GKp7eXEXSA2G7+Cn7Qlp+ u3PyI+NEHm98RbKtjnuCQR6USDNDvFM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158192; a=rsa-sha256; cv=none; b=Iu7wAXwXelpqd0iPpxWTIZ0qQef6l610O/cmtawnVn7uex4eEFT5iYDahXn/jGjH9k7fch EohPkWkG7RGiEMfWPkHVCyz/fuX38fw7/0mP4D3WQcVl8f0N/6SwtVHmfW4th3r8HQT/b6 DJ4vdMxmruyyRFZ8UVoAISM+ZvVqouI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=iBG8FHOt; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-20551eeba95so44280735ad.2 for ; Mon, 23 Sep 2024 23:11:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158309; x=1727763109; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; b=iBG8FHOt3kKI7UiLsbMyc68t/EC8p3jR5zgvRbgtC+MN6IapY3LDwHxNBInT5InSCV BrdD5PdAXsctGxEtuM2pjApud6TbmAduFc7n/idKPcOQRuJsy1ANi0ugnY32TCV+W71X sxeFj1G1jN/UJ8Q+qQ8MN/L/G+HSyI9fhXiWJrHLzfUhvD1G1hIomwqFYRm6HRsL2cbE 8ybmDFrMwo9H010lTSabysWErE5UKvLKhWtTdH1BYS7xlqSzhhLde4EvTlD/iYtgWEdS z05T66h31NMfEwuJiROxqUU3y6WTBzxTAUKrymU7bLfUyPH8zRqlPHNKddm4wv2ZecrW SavA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158309; x=1727763109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; b=Xkg/Hs4f/Q2oS+/pfFRy7IkGGt6zuWBOjJnxiI06fLrlteoO+5N7Iv05iuPxISM7rO eHnSlq3wjoWa7DzB5jV0PzPSwKkR0eZUGrMGeshcBd475oyfCRixv/1k3xPoqjlB0D9C iAiqYA0plOo9209JzY7yhQn9VXCJBnFxnCHa1lw//HSDpIaCRJBUPEutn3InqotWXxGo c4X/p+bYfbTfKJ8PrnBlBtSdDPVkjPdEv4keXyf2u2bv2rmHWAx+uSaI3ayDlzza2HmT P2D+5ZU1es+rHXhnHMptEDUPImYE2TVvutABFN5tQslBG57lkC9lQTI1N1kCL0CaeqN5 5seA== X-Forwarded-Encrypted: i=1; AJvYcCWd6NeoEXYEzMLwdguxri4DT97FJIVxLfyNtRyAhcEfy6F/dYfkRSiqIaCtmKDJ36GYAhCa7vsZZQ==@kvack.org X-Gm-Message-State: AOJu0YxJnc1n3ruzfBAVpCepxEXoiRqRS6OfjQ9vFbuTTJTNl1X1JWpA j3mPvx324HXsJsjcLXs6fO8jvQHMJ1afq3gTmDNd6vi2cbkddPsmb5tkw4jtFtqOTz1HoiE7ID2 mETc= X-Google-Smtp-Source: AGHT+IGkJQOlzg3g752NhI9RquMqodCfvTZ7obYXRPzPmN4sDO5P6Za1vWoZJJ9nTCdoHXRNF+rvtA== X-Received: by 2002:a17:902:ec8a:b0:202:311c:1a59 with SMTP id d9443c01a7336-208d83b6ce8mr195135205ad.27.1727158308785; Mon, 23 Sep 2024 23:11:48 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:48 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 13/13] mm: pgtable: remove pte_offset_map_nolock() Date: Tue, 24 Sep 2024 14:10:05 +0800 Message-Id: <8eb7fcecf9ed8268980d0bd040c0a4f349cbca8f.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 0D4AD180011 X-Stat-Signature: w9n584pigi3z1nb3hp9derht4nttzdcd X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727158309-491149 X-HE-Meta: U2FsdGVkX18OFSA95jQJAvU9QJFnxQIaHu1TxIgbAuVZWftqD78ZKi8f/ZOQv+P3/FqL9jf+kABlBg8GgUXh8CY/bTtMZCflKJf8w6BdZJzMFaa3EvRv4c66utA15yInTfNL4x5J0NtQ4/L2Hzl/ncOc/Xb1uo2H+vBiT0QPQ5Gr0h3ULlfoCOt3u+Gx+8cTE4yyIo/5Vi4h3g6MV25iHAHAg7uTiQMpLqAaAn8kbTgHxexRsPXN4w4JNNF/OzlcIIrDevJ1j0uqgW/bVGofFlTcZDVfLv0nvj3WQ/8HqVLddseDDi5yt4P31Hb6zqADCvTg43zSZKBunq5xLKItasNdfGHscv/u9JH+JH9khx8bdFAYFHXQRodiDO4bM7kzqh+SpfQsPtUG9sDSj6ve3Jx+ks41bjwzfO7UdNv6uMeUBlD1wgkTyKf2Tz2nIpK37atcjS8JsqjJGpt/kVQgnNi/TgBL7riaQtEXTSja5Tsnjqc0n1DrwpgZxW3iDyQO2jVJNRm0ocH98yHXYigNuLiFREoyvST/HOyarsdkVYLni6xAIegwSOfxucWX59yasWdHeHXR///sQyMNIOcC9jOAEGoTtHkYcEcoo3hFLH1L/FPdRXnlqbe79jVb3swozww3HYuxgzdt6wLriNjjD6djXeGB9kR16sk6nKhWyJgGTT39ljk6oDGCtZiBOT47X1bsIf2yf+36DkcaniOXL3gOFrnvwdQGFAL5Zqwtsv6p8FoReOBra/1++JudPp3tIeh1KXGYTAU1tPklXeEj/ZAzVcObGYcOp2gZHfnrgEmDVA/B9VxTLcsKz8Uf2Od/r0RkYLLPRoyCMsQztThkxrBYYs1ipBhAd5WQaOPOFuaHG/S28UbLGKKTwD3ITHZmzjMnWvVk6yV+f0AGfy6XS81F7eZbqiJgNZ6huj1K7vXC5g7kbHJsZSQZoukYNW+8TTvNx2UlBP2yxe7ih3o GDA7DeLB 7eU0V7MiiYzyiKnz3Xvn+glgFbsSS7mktwSKX6N/qwSeyROrfkXgxkjI2WGXryGkyOtCJVJtt6sSR+QC31E2zz7WiGUhad+LQBYdF0ynbboMSXELzdXpEjD1wNpHsFun3H0CFxmQTN/uWne7u4Uot0prtg6HbJ2acPywfMFMoyIGVypG8frXvFxjsGOF8FPZdvRz1HJCLQ+TUQ70o6W/oaUBHSKGxmRRN6d3FxGniNOh3nNCHXqV1j72Hu4M6G5V2lS8wvvaDjH0qZ2wnDNpywh9ncLDEsHoh4YjfyoBqAlbagama5W00KkJNpGMIlq8Zm1lgUxrtot1ciwV7/in544TOZFlfhakcrqjgdh46vFMjapr+Th7ytXBEU15YrJnNdSZxv+bhrCXsWcbJXohZvYuAl5jbmEbPnq0tXdEo9xwYSnGA4YMOOlKGog== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 9a4550cd830c9..e2a4502ab019b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,8 +3015,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 262b7065a5a2e..c68aa655b7872 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -374,15 +362,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); * but when successful, it also outputs a pointer to the spinlock in ptlp - as * pte_offset_map_lock() does, but in this case without locking it. This helps