From patchwork Wed Sep 4 08:40:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F4DECA0ED3 for ; Wed, 4 Sep 2024 08:40:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF3916B020A; Wed, 4 Sep 2024 04:40:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B7D886B020C; Wed, 4 Sep 2024 04:40:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D0426B020B; Wed, 4 Sep 2024 04:40:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7C7BB6B0206 for ; Wed, 4 Sep 2024 04:40:53 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 140C2C0CA4 for ; Wed, 4 Sep 2024 08:40:53 +0000 (UTC) X-FDA: 82526410386.14.CFEDF6A Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf14.hostedemail.com (Postfix) with ESMTP id 1C7F5100004 for ; Wed, 4 Sep 2024 08:40:50 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=SlQDIomu; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439227; a=rsa-sha256; cv=none; b=yXTABA/jcY/tK/eA3f3H2+ZrdThq6WatRBSprUyUynwpa1a9+m4/zQAM49A4rh357MS622 QstjUJNC4UNk+wR4NKhN3Cm71K1QP0s5MrpY0t2j2Gsb2LQBGNidX3nfBJ7+m6d39EP5/X 4OTo499Dp6WKCFqm+9QSfiXOCSfN++8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=SlQDIomu; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3ZoTvF0msswRhisxHHwhwQPLPrn7vj0AmYqzYo7APLw=; b=Q3PnZKC9ANWF3gAAEPAtFQz+LAut60SiXXakqa46olaJI377eWgEBbOMZUzWJN+Kgx2Taq GHX7on4UHieEYH0vXc79W6GxpLASgfec3cAFSmpHgytqBa3xWEVjCc/CDBJ5IloEAr22nk xFO9CujccLW4/BAcpMd0NunT1TuFBd4= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2055136b612so36608815ad.0 for ; Wed, 04 Sep 2024 01:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439250; x=1726044050; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3ZoTvF0msswRhisxHHwhwQPLPrn7vj0AmYqzYo7APLw=; b=SlQDIomuJ5jufXUySAHI3Ydq5+awYW1pG1j7XcYgxjEhTW8qX5uVCpiuFPecqf+P0e LQw5x4U940s37BcMaj+d0WTM8EqxbeJD/uQuLndZqrZcaeRVZMNMezrHZ09C2SwgPIrY IMcnxqhzF5DNnj+8j8UBKPgF72Eww4XC/UceSrwQGqirby1eREPSCvLnOpErsFfl4WFj rUsh/lcZWVDy0Rx4QJ+gZgOyPlniiBWAU8pe39bNMLDVogdZaXaIrg0myDW4alulIaSk N5CAHPgpW28c5eEWzfA52IAIwzidMvQEz+oZ6FSmLLmbZGRsUjMZff5ajwE3UFAfqYD9 gyLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439250; x=1726044050; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3ZoTvF0msswRhisxHHwhwQPLPrn7vj0AmYqzYo7APLw=; b=IwHmAXGLlss0T3Nb52cXDbJSrbiV3OTMrveviOogVhBhKseENOT0h/WOheApJByN/N j9oMnKR5XtD3B7w6sa+34OyTJSOLE+6hu5iyO26Z3I659QsveiyLWqfyyXbS215GBZNW Y2TuN5n1XkDBWU9yS9px0Hkvc0elLeDOgYz1G4vweyQ68h0dBoicwIVwgkZjTUrYmTIe LoT7OGoTIGRerLlYnfoijUXHa81/8hFXaJFlHuec/b1rvtv0DRZqgZTm7sfDWaJBd0MH BIYRzsGDM3H6ZvLCC3FhCfCB/4QPTMfNY+K1VDdopLu5HVOUizMIkwtOSJMxnK3gjoxH QOpg== X-Forwarded-Encrypted: i=1; AJvYcCV0IR2oKlUcPewUD0jDi9VqWAcPGfprb852C3M1F0+qQiLQBzb91HZZbw//xsiWqUJrK9UD7mcWyw==@kvack.org X-Gm-Message-State: AOJu0YyvbGicQxogwKNzsA8z94qsImm7ETkR8fxmocal9zc5OWBkkFQN qe5rdgx2fhaI+rKhUenQFywVxu2G5fmykxccqgxiD8AF+7UxIOAkuJpARQ+vVk0= X-Google-Smtp-Source: AGHT+IHLu4af/0RdmwIJrtep3I3PIUPy/4FB7EG8zsCDexMsa/uyvNrQYTXwy/3KiLtDhw4LWBklXg== X-Received: by 2002:a17:902:ce0b:b0:202:3dcf:8c38 with SMTP id d9443c01a7336-20546b35d3fmr190359795ad.44.1725439249775; Wed, 04 Sep 2024 01:40:49 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.40.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:40:49 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 01/14] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Wed, 4 Sep 2024 16:40:09 +0800 Message-Id: <20240904084022.32728-2-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 1C7F5100004 X-Rspamd-Server: rspam01 X-Stat-Signature: 1gqb5x4u17w3rx7e8bjkzozwmtu1t97f X-HE-Tag: 1725439250-839016 X-HE-Meta: U2FsdGVkX1/SkJtGQDP34yZuKhWz+72/akEEdFy6jeY/CPRZwz1HsC9FZdO4XNeiDwwbIWqSQ6DSze0amhzOEsDh/AGxJyhTYSfA/7RZ0Ipc4k7gRO78Dvq9JHkzM9QX6QY+GwBvjQ4vjnaJbI4w0SlUEUCITkHVAa211NCV/KirLU+821bGhqb5ALOEbcnOewNtqLvSSDlj8VaiNsAb0hg4TQ5zWTA3d3xGyn7MnfuhBui11dnkI8ABt0uS8A/o31vwzsQPVHbmn7YF0u8uabe90tD1uvlOmhtBBL/DfAvNy9/KdpiAzJmNctERvo/+IDawU1BocK87Kl/lFvEQhRnZ3VsVqiChVnK0ZkSlHCRHECMz0PnHZ3wDY0wBsQ5eXTC3wSItvupflglYRzMw4C1hip6zhq9P77Y4Z9lesdujZGGN2DHCxx+ajCcjN6qUlbDCPpmS5AO5f0ANRaQUaDuylahZY9vgbriS9PDP4ERZ1jEBk8OYfRSiw3QTLy3TlFkApkJWzFDAJdAx19WppqlNnCq6zfe0zpvqULGMch9GoCifVXVOA9OWJy94HZb52abfYL32LXG+QhR9XuQkWfWqG3tjkgIbUwP7FhiR/wLc0EyhWlOLR8evc/eHc3wKQZcBVf3hxsGzbA873imElLe7PSghxw35aX5lyD9AElP7NM9AXOpIjjosvv8O5Jy6SYwDG9i/4v+GbxuR7pleJVu4lkaaTVAKwEw6XCea5kXAa1pApv82DoD8kzY2AptLq0TMapp1e79yZ8je6riYo3PXaeKIO8lQ1qBnC8QkuaPODCmwTMWBem9lL2PzLrtrXaEXNZ175WyJ0bPWIqYDjlYAP/LYyq2tsE7046XiYRwiitPKi616O1Bkt3XsedEEmPkrXkcrWyuHZWE8rJ+5/c4Oafe0yFfAbSWg91DbkJqKoCseJlQxS3y5+lwIf/Bx0z4aoJZ+mSfsrY/TPA4 w1UiGI8C yL4+o0fyzQxr5nF0ZfcdaxQIQ2w5cCAUsYWfaInknc9gCW+VqEC76TDe11BmhKZO7o1p9j8KQHbs0mQNGvp9fTse7UzzHOddPKZZaSMvh0PkpYFefMFWbSYDxKcCFZFqx9F2rTsiKZOiX/zF7PYrcjHQVZ6JYM+bezIjepwu1Me/WXtJNBwXhrhMigrnvY5u9ZjcN0G8/+ihHAEcHWoeWfBCPUqya5hLCMnFpIuBRuhpKA/SnfNiemH4yy6tV7OGUGjADIg9WqogTd8RraQOhr00rRMJsB9Gq/BEGiJLgWGM5i01MttAldFmFb8txr96uqKT4CXP069d7UxZbamyOgfgpqdu5pmr74SdpLKGZq9wf8Zd5J1LWF2L9jV0T/nGNVMhfC7cq9ce8kmNeEeacy2ln6IlkvO7lPh7PYirhwyRONb5olsagdQucIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. It is applicable for may-write cases where any modification operations to the page table may happen after the corresponding spinlock is held afterwards. But the users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will be read-only/read-write protected. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng --- Documentation/mm/split_page_table_lock.rst | 7 +++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 50 ++++++++++++++++++++++ 3 files changed, 62 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index a7c74a840249a..1fde9242231c9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3006,6 +3006,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..262b7065a5a2e 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,33 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + VM_WARN_ON_ONCE(!pmdvalp); + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + *pmdvalp = pmdval; + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +383,29 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_ro_nolock() provides the correct spinlock + * pointer for the page table that it returns. Even after grabbing the spinlock, + * we might be looking either at a page table that is still mapped or one that + * was unmapped and is about to get freed. But for R/O access this is sufficient. + * So it is only applicable for read-only cases where any modification operations + * to the page table are not allowed even if the corresponding spinlock is held + * afterwards. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the pdmval. + * It is applicable for may-write cases where any modification operations to the + * page table may happen after the corresponding spinlock is held afterwards. + * But the users should make sure the page table is stable like checking pte_same() + * or checking pmd_same() by using the output pmdval before performing the write + * operations. + * + * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will + * be read-only/read-write protected. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Wed Sep 4 08:40:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFDD5CA0ED3 for ; Wed, 4 Sep 2024 08:41:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DDE46B026D; Wed, 4 Sep 2024 04:41:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 365DC6B026E; Wed, 4 Sep 2024 04:41:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E05A6B0270; Wed, 4 Sep 2024 04:41:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EC3B76B026D for ; Wed, 4 Sep 2024 04:40:59 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 88FC6120D6B for ; Wed, 4 Sep 2024 08:40:59 +0000 (UTC) X-FDA: 82526410638.04.F328DDD Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf27.hostedemail.com (Postfix) with ESMTP id B5F9540008 for ; Wed, 4 Sep 2024 08:40:57 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BAV6KZch; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439162; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H/kyF9mn9oESktxWYdr3s8d2VJ5iyYEZZY3E5BXrvv8=; b=8IsHVunURBb3t17PU5vDa3NeWmoEGwgE1GNyhO69dnWcti20jMoxdxoOwwMkW29MaRDcec w636S88lnkHynmQgOLTXTeXXUzGtvP/7J6/6gcL3IJrc4jQ6V1UQwpLYT1UmS3EoI9wVad b4b6d4ZC83ukEqKPNNnJmlxC6ZrHkcg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439162; a=rsa-sha256; cv=none; b=vhcI2wBpbHVbBxcQa0jBMJ4VQ7EsHncp73lqa774Yf2lAQHnBDZC7loXuk7tQFo98e+hgB bdumYMkoTFpuzJV76oMmOImBG7spRnoYrqn7DDq4PFoQoeFI6FXpp97lTvYfL7t1f+yu5L BZC4qf3JrhZDUjJ6MLobnZ8JrGyZzwM= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BAV6KZch; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-206aee4073cso6770365ad.1 for ; Wed, 04 Sep 2024 01:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439256; x=1726044056; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=H/kyF9mn9oESktxWYdr3s8d2VJ5iyYEZZY3E5BXrvv8=; b=BAV6KZchmI72kPqAFdGm+CTB87yfHpb+3u2UmRVnRHVnmiZe7rTXaZmFk1ChansKTL BhJEZw8jZUyPf77Y7UxoejKu76CX2NXA5+vaA1g9RpLe4sp8qObNsYkWQqBwdMif1mn5 AjO/V+K/pa54HkoQN5zSJv3oqj/m6B1MQVnZhJRGEIXsduP5hbZY43pkfGUhZ2yh3ZCm hQIxfG80r9K8W1yI75uZHlQksIkzjmiD5kBQE2PgCfsNeuKZlOf8nfskiQXjS7Vs002m 462/eh90utAybc1qc2X83bwyawfjMC4zoJwvp/tXR1fPFYApHE4Kj7iEFBdQwSklm6YL FSsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439256; x=1726044056; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H/kyF9mn9oESktxWYdr3s8d2VJ5iyYEZZY3E5BXrvv8=; b=HTA5wo8U3YigXYsgJ7TPHQMMK9wCR/N2gn0Hq+qfxyhMLULQ3Xo5QGPon1J37KQiNH wfVEnyuOfR0MvqEF96BdZ5sIqeZxJj+3PLJOvr9ddio4WQxTN+QCoWUJBZ4qYt6PXo02 jymyBkOi6SacpylaoAtmLMXWcoqV1tjFNcnvMPwtmB+3VQQ2x7+VJBb9xD4o72ME+VOK 20PSsSTvd2150SzG25XbxY9BRjS6mZVEX78LTugZHXq/nXJYVET5rWUo4rdDGoBLHIBZ uzPUw6G4Ce1NM6+dSDekHmDJRH9SIrvA8XYCU7RJVSf6iceHSifYlpDKTQhBT9mL1chP eYzg== X-Forwarded-Encrypted: i=1; AJvYcCWL0Oh/SlXyL7Okik3hm4jaEEp8QwYfSi/qJvA+46bnkeb6UP1LXrGyhgd9ZeJ9JIwTlLpQcKdufA==@kvack.org X-Gm-Message-State: AOJu0YyQ8J1GUIes+MN7hQ2J3HJAuPTkJCL8hZzQkBLEmc2c1kP9OnAU 4sWwhgHhFo0/FQLZy5P0350DIOWDotylV2Foo4ArxLKiy2n0hkdNO0qnowBJ4hI= X-Google-Smtp-Source: AGHT+IEEdhBdCGQBBk6sF5V8bi+PhpxKTNotIZgwSj7kMgQxLZV/04loActQGvyTL5xDqZZl1Kbj6A== X-Received: by 2002:a17:902:f547:b0:206:b618:1d7f with SMTP id d9443c01a7336-206b61824c9mr14802495ad.37.1725439256381; Wed, 04 Sep 2024 01:40:56 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.40.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:40:56 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 02/14] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:10 +0800 Message-Id: <20240904084022.32728-3-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B5F9540008 X-Stat-Signature: oq1f4guhmcgq3rzs9zpzq3amw8qprsf3 X-HE-Tag: 1725439257-479011 X-HE-Meta: U2FsdGVkX18ZkgYx6HYxL8XV0rXAGTT86fBgKdS4oprnhftYdC2tggPFmtX6/pBXU0kRXWrURzGkvtj0XYJ6dpxST92T/F7d8/a05K2BJq1dugUmahLgyc4UgZw7tSo9yNrKgNkxcCeDUQNE1E4d624Gma/saESPIULp2OxBiDNpByWI35aLzTRFQe60w4T3xFrM32x3ZBCuiDiUEcoVLBURX0zUdGS/zdg5G7+fCoFYJY4LInMz4iIGXLBNK8e4v9hFpJ0fdw1R1Xw37yhu5lWgynvJ2if+BJu0X3l0OvtqIRsdKepAdC9n6WdqhNnW9cDqcMvNuueLJAt9eGac/sb2ZtHViRWufgyFeZb/+focjIMaKQm9BroJ/jnJkHSG0PQ7LRKSlHqpTBZCWqz5oYybvhYCQRXkToOhl3zyxiMs/UEoWGeSEh+ZRpZ1qScfAx3nzwR6FxGPo0GT/99jsYeiv7Edwrn6R47oQb4SboY7P06eoD1XpfAyPWaFw2MqzSZ2XTqQ2+Dw4McnVtoE2Z9MPOzGCi0Pzmo+kP8vlvsjiLd2lpaWXWJMWTOxdU7jjWnPtaU5vo3RS/fECtMBvhtO9hYWoQsGVjnK6Rv+Axv75GsSGzxGIAVi7J9xqywSEP+MYO/QKLMBfGu2iGn1bNNhX+1SeYQxd8OpbctM5V1zvs6cSEeRgP87oGyad90oj+tS4FqqSqXR3YYw04ackmF+2IIOO+uATRCnV/h3MkkrNNRokhUBX9SllWYyNavIfZVAuPI+5k48zM18L0HscKO+yQxr6+eKWRPgfVUUsDr2Jg83DcJy5/gBwHN5am1mOd9u6s0n27YSUTLboEsS3oIAtqwNXiEBTvM05ilPPCJ/+pZuz0X4018yRFF/HXBB9Ye4FYkxzaskKWkRCi9aMuf6gNSKejx7OFOcT3aTdJtuZPom4JIkEkuRZ3UiAFcqrspZlgQZSpBgq1D+huj ZJCC6bXh 5snP096uv30av0GBD40MHVk90pIBXb+hemI5/kHx4S16FVw/UL7zIO9IUBQjhAtHcJFTa1A7rRFFbZWSU23AfuyWuUbvc91jhEGdCdapGWjWRDK07Q5rad12HyPMq9uEp6yO/DkbJtkSy/N/MGN4genqINbYrV6tNe16U9ZxnCXFKrU/py9kJFaO8yyWHJ/JALpoaeDy2tlbIMBvEUaO5VVlEeswHjOi6NBrLaFePtD1s8+qcROJ9RG+kyBtcQFi9klkg1JiZI1g5vGkBRWWXClrtIN3qTgy7aXvoVu+p8d1Gf77P/ZUxUQh8lFUP20FTt86fqpn2HU8V5m8s90/KiIKfND3WDkqP00o16vlQ0qk4h4y95Jwi0bo0jropxtm1Wvqngi5u+xs9vRnn3U+T5I65jW4ys09fiRaD0OtfAHiEWZpgCYpAAEMpL6pGB9OZmEwJ7cGOflWPs8A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In do_adjust_pte(), we may modify the pte entry. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/arm/mm/fault-armv.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..de6c7d8a2ddfc 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -94,6 +94,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,16 +113,22 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; do_pte_lock(ptl); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + do_pte_unlock(ptl); + pte_unmap(pte); + goto again; + } ret = do_adjust_pte(vma, address, pfn, pte); From patchwork Wed Sep 4 08:40:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 613DCCD4847 for ; Wed, 4 Sep 2024 08:41:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF04E8D0228; Wed, 4 Sep 2024 04:41:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D78768D0238; Wed, 4 Sep 2024 04:41:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCE8E8D0228; Wed, 4 Sep 2024 04:41:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9984F6B029A for ; Wed, 4 Sep 2024 04:41:06 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5BFD240E52 for ; Wed, 4 Sep 2024 08:41:06 +0000 (UTC) X-FDA: 82526410932.23.648EBCB Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf28.hostedemail.com (Postfix) with ESMTP id 826B3C0015 for ; Wed, 4 Sep 2024 08:41:04 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=G02ae3vd; spf=pass (imf28.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=hYFQSu0zD1rsHNKIHN+pdmm8DrPk7x0gT0WxZZMiKBIY+x+61ldwiLqw1VKyqRDlPeyf3X k93hf5MCdV5HE//iqKNJe8lL7pJvRJPnI6qx56sRAUdSrksAVyaRE7m6btIDvOxBdEpVi9 t063ow/E78TuNdzJFroSr1IvxAY9VHA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439170; a=rsa-sha256; cv=none; b=uCMvhlAZyOzd2xki4+LfSbvFcZYA5m00clGLToy/hNYK4c1KSyV8GzTKQ6i0kCXpelxjv+ wkZyjuQePZDwz0UYUhjaYeej0VPdZsfAIji0vyFsj/yY9hfTSLG6eMH5dFEhMEPeJAy2Uf 1YCit1n0aYRPqghTv2EPJ5JYiiySvP8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=G02ae3vd; spf=pass (imf28.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2057c6c57b5so2890305ad.1 for ; Wed, 04 Sep 2024 01:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439263; x=1726044063; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=G02ae3vd8w9ZXmj0HTwnYzuatCDDiGkmnbS2Y9xYI2p4YqMfCxIBDMJTPCDWmo7YK7 G5VHJnMk49TuNUhZL/eJRfqf9F+jviXCKb8EIZhO/jBGG25lVTQ1rILxbme179ofXFMq oFEE02ndi5NgKJpKlYmaj6PsjxETu02JJIyfYMEkbvKaiLh3ZVFrP5DcpHN16POVVhaJ JzW1Ow7NSpH6jP/ql8PX2WEMwsycoffb4UlbDIUZreXWKk8bFKcxXHIRVXPj+pwNLy5Z M4RElofhvIQsTFFK3lUbP+E5qjwSMf0DUgrHsZ4VR9FLWA9Cy6Ndds+1YVxF/wdVtPgU RezQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439263; x=1726044063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=Ac2yKWMhijvccXYyUnncJZ7T/NMYRE7aTk7cCRVkcp8oJSjyhpy5TNMOGzU6QDP/CR Jb5yJQz3cnQCIuE8oLmM3OdQ7if6Bap0QPD0A5JCGZC2eT7V4u89HlghoWqXaXoYkOmT 9m/o2bjfP9NE6elwjbf22W/2PGrEjgfrWCUmYHot11oRM8sbSgn6oTvGLr78LKi5yO1/ /bvShVx06AsA93aYRrSH2LXxFGjgW6iq20xezYJaBT/dcWYoKt73csPfwKtYgpLr43oN DxOE/Vr0+f0MwZxHnIzCq42Tv8JmcS5eZpHj3Tpbp5Uo+ryvYOwoKooOH5dKHGDhoTbL xsUA== X-Forwarded-Encrypted: i=1; AJvYcCXsJxQ4MHb2y9C3gnZJXhpLT+POsKMGrM8vdHbIY3oB620FWglY7UGVNprGvfI1xTxysa8YhYcSSw==@kvack.org X-Gm-Message-State: AOJu0YzAEf75gYsBvqWFHT4hypReoS4zZoBPyulKD4SdDxTPRsYewfFg ULm3DUY4VxGIpaQnd7TXLmZWFS7ceZ5hW4xq1RLjpq2itgI9VhvGFhOHM4sM9P8= X-Google-Smtp-Source: AGHT+IFosxGKKraD23O7QyExDWz973MicB0gfFUEF3j/RrDnY2sLd35727GFsppJnEl62/xkyI7W0A== X-Received: by 2002:a17:902:c412:b0:204:e471:8f06 with SMTP id d9443c01a7336-206b83566dcmr20294065ad.17.1725439263160; Wed, 04 Sep 2024 01:41:03 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:02 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 03/14] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Wed, 4 Sep 2024 16:40:11 +0800 Message-Id: <20240904084022.32728-4-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 826B3C0015 X-Stat-Signature: 8icukmcps778rmbopyj7adp8f3pit5ab X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1725439264-917980 X-HE-Meta: U2FsdGVkX1+uo0vhupZ3RRyVNEzdC6wWPYrgJUGHDND4kd/6uONC+nUsodFVnivnTWvHPDkQUUR2IO3XrSxerjHLvJJrVCwG7EWfGLAQyTtcl22fMI01J9fm5093sSTLtrOza5DDMjNLgyFlj6y7h5aSXcy7ZTVPN+mCzMwN4msX30XbGzX3IQf6742tiUVp4p7KZQeKWkivWrpjaS6nopK2D5leUMbA7zDnw8gaXVBt90buLvdJQougS1hJo6v1jzDBA57YRRQ4Pz6LSmBpjCk/AjYSl3YQ+brE5MlzOcKkHvW/qj9YMzZpW0iMztYJ1wa6zpo4rOJtuYfFlXjBjFc/CSlW7aiYXATwkeVoUWJTtn2+EpIKtkRt7N1Ss33EYFJyZnh1n4oPzpxXEAuqcDpwE66XwaRY0e6YoO1hDQ4fpTHE3b3H8e73Noi9rpXGIVB7BHRjigZSpUrOhBeMPKLcmUVokwmiN4hPNkoGMwNaaBTCSe4Bovr5jNMl4Njx3iXJMT9EQ+dsoVOaMUU/ZwqP/JBC4w7hnmp4zS647cWMR9GtM3qVulx959cVzkif0EmXVYi09obOvZGDiI/DLRdo5X8JUjIBKph+tdmvqqqP6bIC7ntISsvWet4AACVIp6MmgcyxZ2inHPdi+rnNlNnxx8S0xh1jeklwuKnCv0oArq3rMC3Ra5Q0gVr/n3nviBcHYSotaD1+R+WBCw+0gFiSKtWVioDPGNuA1vafhTK4lUaYts2YadUHZYNOvkggrlsxD6TmmDgtnI2K/7JQIvWVZ8GPkviStgZKC0K0k8IRFAofeRSk9oDWMMG2s+8GAQ36JbVTjhiFb+9EmlDWIBuD1zQggur36lQLw/MmdKGhPsr0v18JIqbpPUHfRd0N08Eh2mbgkrHYH68+AkSv/AMRjDHA94dGa/bxoUioFbM0MwpFKVcphOlhdqwkQHHDjZLwU5Mn/ek6BzmDjJd Pyz3MAsc gAjMSQ6g6b1HSF7awq65qRTSH5Vehsm8gfuDibwMZVvFGUK4BspZex0BmAMprSGsLHyrlKY99MvblanPty4W+JlsEiLXm7GuZGQ7iGSvAZWXk67NiWa19cZmhvsOZafJ2FUBuUXXQbDsVop8V8ilhU79heYmcAzFgsIEm2s+YW7YXUbtMXgRf8i3q6WO4Ef0XlEwkoLCvV+sGkA52w+sSErwuNPhiC6ePmpg5mPf6HRFTtF8EcoTCoNF/8Mn59xCkzNAASwryZgjrINvxT4cknI1NDR0YL+mmiysz6uYeTDb7EB6naSB/+qUnCqdcc+uxYvwF5q7jZOPljOPAwFSyZtlk5w0mWq5RBGvThX5vVIrNRPfnc/zA8eIyd3+M4/xicLKTWCbKSsHukfqe7a9LUNGZibY44Jq4rn9gHoKlDNWb5mSH5MJmqKAspfSqdz2k9+EHPc63Kab1Ul8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Wed Sep 4 08:40:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC007CD3431 for ; Wed, 4 Sep 2024 08:41:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C8056B02AA; Wed, 4 Sep 2024 04:41:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 451E16B02AC; Wed, 4 Sep 2024 04:41:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A5056B02B0; Wed, 4 Sep 2024 04:41:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 00FB76B02AA for ; Wed, 4 Sep 2024 04:41:12 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 99E841A0DCF for ; Wed, 4 Sep 2024 08:41:12 +0000 (UTC) X-FDA: 82526411184.04.8487500 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf05.hostedemail.com (Postfix) with ESMTP id C643A100008 for ; Wed, 4 Sep 2024 08:41:10 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=XcfsyuLv; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439143; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Mwcb7WVpeL59BmxGU8FtBVb/FvaDY4mjrVacU9l97yw=; b=xOFy4kp/BQrRnZ5FZMKnNtjKiG65tS74S6SW2W6+C8+3d2QVQIibwLr4Zb2PEGuuoXWx9m aP7/K4vckwyJ5qTa9MlRdCG/1jRumEUKp1spg+3rBu22aB2OaQP8Zt2zXExY5xnuyhl2gU ptxj8z0SPBLoSN2Q2WoxqbbmWeKu33U= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=XcfsyuLv; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439143; a=rsa-sha256; cv=none; b=5E0u/1A3uja2Xb5K+oXmAZHUcJvJTWVW6AV4EVqCiVbnfN9rZHBRi1N974c80osHUUT8Dl tOIKF6zaR2Nr405mKjVGnuOmAEl3OT2HZ5gxs36Id4dCfoQMbHYfNQ5nK3b8syFv4ixWRC 9hOrBM4Rn2yVS+EnP7LMGTrNV+6G1V0= Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2057c6c57b5so2890755ad.1 for ; Wed, 04 Sep 2024 01:41:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439269; x=1726044069; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Mwcb7WVpeL59BmxGU8FtBVb/FvaDY4mjrVacU9l97yw=; b=XcfsyuLvWUcCR3DfP1JofmvjM7DywAIjEO5qU6+Q6xNkQCrSMQNewDbcqdhIQ7PICc dRSnrNsS7fvGmJ2hiRIxGWFTpVSZmsei+v8r6fu70w8Wr6FyEpRI7ISU61g2YOW0NXof ULkm9G6zj0/Go5FLkqj6ctgy7ufcCpYPKn2RMx5n9OQdxs67IUj6SwscqDl2+SW1/oHK MBLfEaMPj68thnRp7v1IdJ2t56G+yB46dkUFWVUeG65XMSsi7zpMA4+U2IvjJBbHU9f7 xzBV29k8kuDpyzxUXA5nvciW0MQrHncyOwBxyj1Cb/zi6hrGd+ukz+GgG6unMRotHJhw 2zsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439269; x=1726044069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mwcb7WVpeL59BmxGU8FtBVb/FvaDY4mjrVacU9l97yw=; b=w6S8CHVLwbJ2Vun9sS6jXJBwQbVT0HmbY9+HllV8n7LexPo1J4L8kUij+F8o0owtJP DqAMSmZhj8KAZTyaEh9VxPmhTt4+WIfmx37HP4uW1y10z6qimthOxBIwRjnVOPo27YpY UodAgXRnaQph+MSoEdqPvAGdyq1o+CKqVexFiMV0wj0q+5vLWZf6d8f+SkcL3NvqTYgg W8J2Ew8EoC0okcmX+3L8wt2rl8Z0hLVshQjuPKaJQFBT6zw+B2poAe9mixIA9CRi9Y7H 7yjpz2/dKcD83S5E8yll+o6NPxYrvYY2gCygeuQ0q8wDlZLzwZvIDDBRUBOJKby+hOtl TegQ== X-Forwarded-Encrypted: i=1; AJvYcCVnESz8rOWKuKDIAkXaLmPRV8j8W0WcNoJsLOkcW9ai5n0nMBZ9B0vtCQBddfsvE9Ir1lRlCzOqwQ==@kvack.org X-Gm-Message-State: AOJu0YxJixbgOr9eRDv7f4HDRAc4LKT+wrI1eDPKuSMTygFRvax/9GCn dt3ge5bXE2tZuBNZZ4depUBjXGESwEzwHguUd6HUjMhB5U1/CM1cfKfaYyAq72g= X-Google-Smtp-Source: AGHT+IHVi0h+YpQWstt2F+C2N++FuREpmzHqlVlqHX+adh/zaJT2MbahvdDpXAnk4gZu2G3FJSfhgA== X-Received: by 2002:a17:902:f60b:b0:206:9dfb:3e9e with SMTP id d9443c01a7336-206b832146bmr25150715ad.10.1725439269514; Wed, 04 Sep 2024 01:41:09 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:09 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 04/14] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Wed, 4 Sep 2024 16:40:12 +0800 Message-Id: <20240904084022.32728-5-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C643A100008 X-Stat-Signature: 1t4sxemht7khuadbwbq59f13csemhs1r X-Rspam-User: X-HE-Tag: 1725439270-126385 X-HE-Meta: U2FsdGVkX1/fJvKxfp7Y5RB/Ty0IimRf1j3uihjthfwfMkMJ8wu1kxmIcUK1AMT4FhiIhzBdPO6lRyvCT1o5f4/lSeJus5CD6WpuSmnVTR6dDcnTSfoqPQDlCh9oKIaw886MZUvBgKtddBSTQ45juyaO+fYqZIDnnglbAFbU2BAZDL5xgsInxYHbQUHw0GLiaVOV85H0QumjhejMHbl76ZBWdoGOZfNCymul6FptYO7jKtp9i/thnO26tjM9aJvWjnn5cETFEcH/0IZwyJdrEhKNhgtLLrUkhw7U/UgWAW2lhyyiZtahYKei+yhlTG1xHwNMZI9Ev+ZRoiAnqMLs9GO1wkla0O7z94qrrhp927Zz+1weowYPkveE0VCNDIdhSakJzhOAjqwRKRbRiygPcTS4YfkjGKoMaJaTC9XXOfjtYyKjkjYrT/4PEp3lAEEvNAAIZMnr9rTjEIhSJZ14BC4iqISj3i0u6ilsIY8C5h0pC9ntQIsFhJPQm6u1QRU61x/lm0CPiSSYK8ZFsjg/JJyZK2X2LnU2ggiS7aV/4ZGq77A5rhTcZxjIMEqJ1dXxa/RU6Ho5qMsVSh7i2e9+lqFYh6hnFlgvZjNhNekRzI1irKMcHc4SKzBRI7Tb2yNg6dzZW6/XXAekTpSJ6bRhmr6upWrb+8OtWB9G7gl2on5i2a1X+NyN7SvOEDUIxx8ps92VQHaG0U8L55MfVPm2FUtORwCNKnWHCoQzndZp3n3jzYn5ZhyA/Ho/S5EawFjgWvVIi3DhTxyDiEU7SJ2hAwc8SydUaafdbCtVO6NKlaHq83fp5UuOzkX28bnA4xZDTM0fcwIajzEEh82H1+Y2fuIDKjw50qOrUQ3E9kc7p5PO7HP/lWOiJBhvYh6Q2SnbiPJ8UCUSTOX0iM8Dayu/MdjlEGeNajNp7CuFPb2E6hmZkXcnUcWY6BHnIZQRitvb7262Nuzsjc1Cb5jlBtj gUlPjuHA J1Utofmnl4uH48xb/yXPZEZ61doF9+xQCnoIXzvo4rMg/DR9R4MvqcnVdO5ROxD7sTwJgeSyHY2oa85KZc92NIJp9/QAtCuP0SvZYqoOHAdKWX5YoE/1GgyERz7DTUQNhrvzAa5rDiUeCXCD3c0UtbZQJkL9Fr2QhJwePpfHqiVo8wcNfl/nLqsApQTS1URldGj5AMUdN9+AfQTMwJ60sC+e4SfCjLe+s3x1TeJ88nUkEODIIAW7TsFvN4RkpfJPaxsgusYzk8VBEOO2agEsT6hyYnZzMDiNBqepSdkY+lDVm/5ic5/bpmj06Rb+ApdguIPQrMARtAcyEBZ5qRjDaHg+ov8RiyZC0V79jj1SMFoXPeiX2gDi4re+M9XlMXf2sCPIhmYkcAaPxm6PpTAeIUC53Zn3BeCI8uvps/Rud2kwiIkQhUDefGCiN1fBWOqX+peB+avdXvPh9oEA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 20346df53df3b..216405ba497ea 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3251,8 +3251,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Wed Sep 4 08:40:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8004CCA0ED3 for ; Wed, 4 Sep 2024 08:41:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A9DF8D0238; Wed, 4 Sep 2024 04:41:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 032CD6B02BB; Wed, 4 Sep 2024 04:41:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC7808D0238; Wed, 4 Sep 2024 04:41:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B3D826B02BA for ; Wed, 4 Sep 2024 04:41:19 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 712B6120E24 for ; Wed, 4 Sep 2024 08:41:19 +0000 (UTC) X-FDA: 82526411478.15.C200A7E Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by imf10.hostedemail.com (Postfix) with ESMTP id 99FF8C0008 for ; Wed, 4 Sep 2024 08:41:17 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WPeXY1mh; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439171; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=1YeM5kHJFa1AZZRHBSnh5o5C+SsIPAnH1bO7WEMZ52g0KgM7Ab8yYnecOHBb9O72NHd0Vr JimyWbZSmnUjtkaUSNZlFBUVWdFCwf9KrSp07VmVHr+JTktiJTuxwSE62oLqA9cA37F+SH fOXoHGSIHBtQUUTuwSKC/5gPAB1IjVU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439171; a=rsa-sha256; cv=none; b=J2zXI+cTYaOoRt0vbmFOXhhD3yd1yzWm9J1RFReam0vyYppsJWIrP7l3Q9TAG7O0DdnMwg GcdpwKnaN2mzZsCyHKmOmKPlNTM0Ua58fqhMEPXNSncCKycENmasnMisCrbY1s/SNA12Gx xUy+Zr63R3TjlrO0vOFU9iV0lB8AzqM= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=WPeXY1mh; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-7c1324be8easo354984a12.1 for ; Wed, 04 Sep 2024 01:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439276; x=1726044076; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=WPeXY1mh3sj6DOqJafnntfjvhqaDxxDRN+7MztkbazIW7srb5qiMekPdZMUBB9tGGW atUTzx9/rc4ZHts1t9hkRrYUsMKCFx1HkcJb8k1CgQJl9nGorsDArGTzVojSYkkxGibo fQ7TRx6M0luZucf1eD5tIkxqfADrnplO3QJf/o+lzgAlqWwfbuZvib4T/zE8dQmgu8oc 8XgWDT3zwKAO4lEgp2pBGLhct6w8Dm+qprZodlhtBfLEGB1vNmvXOyoA5YvMTVHdc9fP 6jx+vCfGClaDYuXzhvTsuffVElbKWnMmmcr+DRKoon34vXVY11lLiMW9d5f5mbl+Os+x 4RzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439276; x=1726044076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=pdfJeFD0m7Id0CeEpjRxoWGY0Asiof3lGnrB2jqd3r/p1CKvR3GRDBbO6b3AluOYyB PQ5C+ef5eLqzdJfPOO6WWhJL5sCLU7WfGC2t3Zcei4QQ3sZ5yrqjoEXbk3/HzXnm4crc mnqFWnQC4A5z0hd57IRbwkvGDA3CV0GmZ/KHH0cdW5/fN3hVOyfj5rOinAuNKj3DG6Ma bka0KlnCyvHDDoJX2gvfabbKSHxTZzcCdih2Cls1gIvo3oCmffgWyDl2INznmtWZvFdZ 22NZnz2muMengR0F+AaJ0Qn6JIL+/UykF/qS4zWm72HrCXoQQzkUNkq6H3QFOc3J/MP6 tUAg== X-Forwarded-Encrypted: i=1; AJvYcCX1H8vh64KG53oozztLWXjJZGdSk8amFmftPhdJzSuFOHVoOZ9Yb0avgye5IGzuGvWaa0cxRikz6g==@kvack.org X-Gm-Message-State: AOJu0YyZdz5XBLDBwbyRl9/e7XTKFcTQestAHpFKQt2uQ90+R5fYXPI2 brT7XBEGCognxBynPcZOW5Pjur/v3DkJMi6Gj6EfdjiJmFuxFXUBUjwuPbucIaw= X-Google-Smtp-Source: AGHT+IEy+qEnEqpj3sReiKH3tPFUkwjqR7spuVkWXeQJodyHWsfCM0cenE/z+pCdCppofQTZA+ekKQ== X-Received: by 2002:a17:903:283:b0:205:76c1:3742 with SMTP id d9443c01a7336-206b7d00d3amr22901025ad.3.1725439276249; Wed, 04 Sep 2024 01:41:16 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:15 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 05/14] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Wed, 4 Sep 2024 16:40:13 +0800 Message-Id: <20240904084022.32728-6-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 99FF8C0008 X-Stat-Signature: gua8skenxswxuts6mw6cahhe5b9rm6zf X-Rspam-User: X-HE-Tag: 1725439277-294836 X-HE-Meta: U2FsdGVkX19Qyu1UhqZMm309R2GiKDmUBYaggL9v+Gz3kjA7bDg6LPfpoyUFUG3dRACFpBkvQeKb0iy7EYDstxRIy2hmnkwwzuUBO85rlVwnBZgBXQTPsjEvdimS8ec3FMd1ZOFYmAPIPAWdPVl0FEQfIViiF9U2ugR7ZnyiRbeJtDay1LKcQJKbBDrAN8knbjIWAk62VDfK4rdRIG+Txr4rgAfPaU5qRP5uW1vFyGrz7KLZEr4yrOOOE+96V6f/9jmsoB5y+hGknYJ30q2a6JnOTBL7yMBVj5xmwz7rlUZHiK4sMcLjSXbmCHB8cx8d5T3iGRXrvpQqRsk1+0vpWp0lsasT8q35s/rTfzRmM5bNQ6Pd/0jLQMMoFrxTSF8xkF3iQu1HovswSscKchbLUR6taY2prpUe4kzD4/hUierfurdPwOkpA3Q0kneqfDd2qbBE5sj2vp3OK6NeGCdMmgCwvkyWn2BhCsxgWCZDud7zXXmJil8z0ymEJ6jmS4d5huHd4lMYs1XEpauVXk30+557Vnx6Y4LO+lJT8LMd5VR028f+XOZQJfYyczHDKLEXOLM7WoVIqD/jNZI2aQqzWVTh8CYsNfZb0Rl69h/mgqV5ZX+q+G/VglrDohjROgfcM2w9ck7UAoRBNeH6PM8YpJsI/5h3z1PFUSRUZVkbAG28IE/IAwKyAdNv05uABzLpiJFg6zRDtyWHfDPTauzeZy0+7GX3oicbCVlyGZ+70n2jcGSAjl4glm3JdkGslXRa0JuoPGNYEwq0WEPYKZzSu8HJM8vxlvSZl3PLeCi4Sbxe2J/s+ZyqnakzDT5HVpKuYpugV5tT+6volyItDSHznXWOQ60/v9PjgmxroDQWnOmIiPIkWdkRQhXMXabf9da7m7XqLxbROZMTF9sP7mDeHSske7WSyiYPw/dqOau2yZOOqIQnIbEwxKQgwXDv3IUU5cnGlixbJe5jc2elyAe R2fEQVcC ak6JUVCkpA74Mrw86JDkZA5KIlL+fFnEqYUCyuncy3fJTXmnutctDL0OowQ1kKqrvg/k4qpZBN5nwHGyBrWpZ/PIzFLbr6/rcru6GXD0lLcjBEABev3D/I1EBJhAClQRdWTNs5Jzv8O0ve4SBuuaqYxNNgjfbbswOPFb+/kpMqrW9+BMHcOcixBRpuB1IC4TyJkwMhHGPRQsjc7WhpJ+Adzb86sxbeqCgWzRXnNTnG5TsnCrFAdTqPewo2lnV/c/hGvR+jrZKYqP7I2aAHx46NNErizw9zuof/hUSWg7ByInZmzyuWPLxmmkemhyVY65IsVnmxCXLvgKmp0LzBWBvLGTZZzzVo4UQpD3nDOq0ywhsdzh3rts6LMxkNDsHgopu3RMC2lnj9mhG2QFZke0qz40fmQ5UILWdU4hM+4VIKUqRdcH3Cj/tZKY+Dyhb8HffybHz/TMV4wq1pPY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff6..6498721d4783a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Wed Sep 4 08:40:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 863F2CD3431 for ; Wed, 4 Sep 2024 08:41:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20A7B6B00D4; Wed, 4 Sep 2024 04:41:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 194226B010D; Wed, 4 Sep 2024 04:41:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED9A88D023A; Wed, 4 Sep 2024 04:41:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CA1DA8D0239 for ; Wed, 4 Sep 2024 04:41:25 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 940D6A0D99 for ; Wed, 4 Sep 2024 08:41:25 +0000 (UTC) X-FDA: 82526411730.28.807F05B Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf09.hostedemail.com (Postfix) with ESMTP id B35A9140007 for ; Wed, 4 Sep 2024 08:41:23 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IRn1vXPd; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439259; a=rsa-sha256; cv=none; b=MssZxNvCSqpGW/+0Zl7Ie5HjxVQQJzjfJFbSdpAXAD37lnLMWlVyvBsm4Y5krnMzmIY0r4 b7Y8niryC9Q2tklyBW09Af87aAqBMlh7ft8c0GkVm30dTVwTS3l77B3YKQjt6+4CA4u6sh jKjtwxwBMLUo5y5KoyC5qieycji9QQk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IRn1vXPd; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439259; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pGIEfQHzsThgJvUA20OC+gjd0SFgF9+h9gc5ip8dlng=; b=CTeZ0lEmScXl8BHQXH3U/XodnKr+RuF/PvFFuXSqXVfm2KLTQWa69BsKxdVMvOvd02SBYr t7EEqb6tJ0QBh9vUvpWv50YkfpszoKAcast+KBuADtT1ODA5BUmZfmDOtZ7048SW7bBuyD dMiToVMruvLG0V0SQRKOeUzFLQlqQ7s= Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-7cd9cfe4748so4059127a12.2 for ; Wed, 04 Sep 2024 01:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439282; x=1726044082; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pGIEfQHzsThgJvUA20OC+gjd0SFgF9+h9gc5ip8dlng=; b=IRn1vXPde4dKciOvXjUjpRHyNbvoGRqb4a8HghL5hxv9JMvydDfKS78AYsMtnoLZhn xMvONTEPQMzGdLU+Twcw9UxJC1zxrz+az31NmFllgigXEoazNhC4u3PWlQPSzd1P6a2v gN9/lFUK8pIW5vDw6JD0ehaA4Tgs7DHrwvTBoBXBKCskWs/BoGhvLmO+oXnVzzGzj6Cr FyoQ4P2QXMC1vqbcNHUGIyG5XpZm2uNmVATA+QrLMRGUdUftK5d1qP9YyX+XOtB4SBC8 T3edEPuQtibg34Xj7CdDg/PL+C8xuozkrXOW5h4TPiAfZJ7ry9OTYMinVvjpqQYUGJrN xP9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439282; x=1726044082; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pGIEfQHzsThgJvUA20OC+gjd0SFgF9+h9gc5ip8dlng=; b=ZDxafyE2BF7jA2Gw1YOBkt5FCsmOC2SFjiztHEcdBPpiReDGDoxAOktS+iFzRxyYd3 pVMLLZwrUENfzTLc7Qc9686pjYThpmFzxBQ78XgcbIfaSQCzrvuCfEzeteJfvoEUW9rY 402mgL0hFNhTX79qtf7W8yzywkq+brdx+L4kA69plUfatGBR8gd9xuYdGtIuqbQ6HY1e tieWvNog4AH7GApiR+8qQvEHxeVQudf3ztFXy/8Hd33u1p6Qn+1mqgonzaJnoHMU2BZu xemU3JYDqRB2S5vwEBhnz9m0RWoSG0l4jhA7HezolXUTkfPpBYbUT8KJzLqeHN4AtYwa SR6A== X-Forwarded-Encrypted: i=1; AJvYcCXAwLSai28p+ROJIGM0BgAks+r+lEiS9QOJKCq4F93UeKdN6xmAf1sPJ+D6t66bDTluJBqOVBpHNQ==@kvack.org X-Gm-Message-State: AOJu0YyiGUXhUKqlziMMrozHnxZ55RIlQfOK7fl59hzf+2UsQ350XXiY 79iCZhw97gDFaaKbE93QV7Kng+kiqithsLSOePf7FUFgSzn39nat7R7L54Ol6xM= X-Google-Smtp-Source: AGHT+IGQLG6wPNgDOgfzFUGIly89eF+GLm9izjxQGbRYP1g8ZSKlFJK5pYdbKQOnPMpn9+eqCP/7EQ== X-Received: by 2002:a05:6a21:2d86:b0:1c8:b65b:3db8 with SMTP id adf61e73a8af0-1ced607378emr11775108637.9.1725439282360; Wed, 04 Sep 2024 01:41:22 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:22 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 06/14] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:14 +0800 Message-Id: <20240904084022.32728-7-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: B35A9140007 X-Rspamd-Server: rspam01 X-Stat-Signature: we19dw8gg5au91yp1qmhi61axemb6mb3 X-HE-Tag: 1725439283-613729 X-HE-Meta: U2FsdGVkX1+6x+cXC+WcXa7qyuqWnpirs2cZu8zmTafyXfLPeKChU7xjBBpQNg4kIXIIUZDFRYo0L11n7o+8xw0GUlWjhZ0L6qXQWJCK4RJcWQ2dtyTQMMSmiBtGM7IyRWCqAkiP5VLwDSLLZp2X0A9C2xKComJ0MSLx9HbIlZKQiHeT2yXGykNpahhjuysJZyzNFIfR0XCqOUWeMhtL/KWL5MxIrnRzy0jJ3ArZsJurHhGHLeK157MKddDnNABfCfnG9wpJ0ZvcPgB2a/qTnahYSijl6E28arOF0yy8ni8W56q7yfP1msz68GV7v+QqT1wGraldmfVgzkIREWCcxXwVEd/o+IKvoK8jrZGQSs4CzMXRzi3+AElL1WWdfP464wkXsnPnDR/U4NV5WpfZJcoaqGPMqXhfB3vRt4uz87YPu8bMstxBias8/OSywT2+zyl4M8H4d2Mczhktt2TqtYacoVUnpLdsBUa6f9XRVpqSVbqAFYPOWnXX8tZcTySfcdTOl1yEvKlYybLH1k2TUZEJgYc+tItd0YkqhyO7xEr/syrA+rvj5obRIRpTUvXM4AyL3YBJYM3Z6a1bVdDHZIXDVca01JwGq8ihx4w1UdPNoWNL0q9YewajvvDsOV2r6agD+Z6wuLFGTrAbxUrnr/LYnAKmdEM8dDVL/lYOvH32lSnKvXLMsyth2Sud3SZ4ZnUPp1DBRyR2LwPSMGXdZA5A9/zVP93+mdLiuT0qZpnnLjUy0T9VYnoB08UJq6EQw8Lxn9/BSbdyRqXImt+87NxmVdBtCIxk3zpyeGOehOOhsqMo1omrTv4nhSMqyDFb97EiVgjOhAYWY2kUVoDef3xTPVVDaI4JW2vbfOxwOgTqkrYzl9A0zOquz9kBiAiogTmoYVnh4FYSmXOurb6SHWqeBel9OSh5yCm1z2RNhvR7gEw+YPzRq83kqW3nr/tOjuu4wJoPvOTn3Oo+w3S 6GbwctdG oJ+JUkWCmgfDtMlK3e4JDuohqgHfu7uks7TaNDBXv3owQ0Vlovir/FF2eRQ0rjmzxhuRM2oAcSU4L7saU16C+c0lK1WR75D70bc4pxQUp57yvgUxjiC7JOW2uVcDYUfTYWCgv7ilb4RLNcDt2jo9iteFGprcmK3LQsm7KHE7IDuiOEJ6IxK637AMYnrC2y1v8YDxbRKayrPd9rNsTXxcI4Sy1feMSuzDhd7h14+Lc9Wv8eh/UEBZecaBCjRmWQeQ4GjPKjMtp1vgpgSozznc4aesHmMkeIJoIpYcxb0YaKTTcX9jHPugW0zrSw0SDUU6E1nEAMHo8x6sBusFa3oIOVjXDPV0v3Eaj2TUehly/ierFfcde3Wo/626WhMw1sL1RFTd/7RMASNKdqlr7sgeoFIIl+5hSr9QuLcaBAT/7bi2FdQCel5/A0O1yKIzmWRS3xIyvLItg1kRIuUY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 42674c0748cba..06674f94b7a4e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5498,14 +5498,24 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte may be + * modified, but since we will use pte_same() to detect the + * change of the !pte_none() entry, there is no need to recheck + * the pmdval. Here we chooes to pass a dummy variable instead + * of NULL, which helps new user think about why this place is + * special. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Wed Sep 4 08:40:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 613F9CD3431 for ; Wed, 4 Sep 2024 08:41:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCA118D023A; Wed, 4 Sep 2024 04:41:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D53D08D0239; Wed, 4 Sep 2024 04:41:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCD168D023A; Wed, 4 Sep 2024 04:41:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 913CA8D0239 for ; Wed, 4 Sep 2024 04:41:32 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 142931A0E7C for ; Wed, 4 Sep 2024 08:41:32 +0000 (UTC) X-FDA: 82526412024.10.0E33C8A Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf22.hostedemail.com (Postfix) with ESMTP id 380B6C0014 for ; Wed, 4 Sep 2024 08:41:29 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rzf6qdHT; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439163; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jJ720IHYb4PoZSjS34HAH7fD9gF/j7WA/1t49m6Kiwg=; b=YN/n6HyTYKwXUtcbBQl/UJU/3ZvRpMw/SaHJGDTOxfz3wf3FlTycFlVhnHsiFvhi5wJffd kZxsjw0hDsixF0YvexgzNttgAkANJ7taK0ihchXjxUrLlsX+zdqXmx45YSS7uj8C8Cp4do KbRDog0rJpsMkP3ZTncFyIObewBCHr4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Rzf6qdHT; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439163; a=rsa-sha256; cv=none; b=rnHSYQz/T4ioo+XDkOqO1r7k2/pDlb1vx1FaOkwt6oZBBQYxl1j+aDrv/KkABI01XiLVHh 5zCARbB8oVf5nD5eopx6ezpOq1kEeIaQl3349ej+lBPSwdGe0s+VI6hD+H0eKIQwED7Ub4 UJCQ8siREyeB14qOomoQXZvYiyCRV+k= Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7d1fa104851so3590829a12.3 for ; Wed, 04 Sep 2024 01:41:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439289; x=1726044089; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jJ720IHYb4PoZSjS34HAH7fD9gF/j7WA/1t49m6Kiwg=; b=Rzf6qdHTYeQTpgEMg919Sm1Knp7bH/z9x35F75fzjpFX5AAMh+UMzq7gYyP5eM87bi 8OM2Iv5ywlXjNBL77KEwkIY0wS/FILb0bHUL/F9r2NrUaWddE5WSJNJhlTyqa0P+uEDM cRBeLjfXiGCKgXhN0skS8VB40Xxu+cmWVz2lmR4qoDCUw01HNlUvIGb9qqUtamm1RWMZ FaorFytjYfTJMPO5JiU2W+Dj8mLBWZrjdI0nRPfWRXw8rkuSeyci8c0QzJ/zevXORgQ4 aR0kPH9VowiyTJPIqv71yMi/JWcuZOcq/wpW1aSv6iKGLB/S876JgZP0oTcZct2+vdla bmrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439289; x=1726044089; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jJ720IHYb4PoZSjS34HAH7fD9gF/j7WA/1t49m6Kiwg=; b=MkiWcYtNHjdS3HqHexJC3G+/KvXCCfxqX1Ecee0EXhH2wTLpRhdH78973NUxJYq/il uUI7zo2pCj7ePDn5JIZC3HhlJRVSDW7uLFCNe4o0xIK3I6ADaTueWkLt9IqE35z/HABA kse94vhlGgheZ41tKZZoIckgUUfuabXwmssoqjDknq1ZGMHHS3gRoql5dxyFE1XHdWVt uZch4/QBwC3kRZ+/kyX1XXrp6WGX7dpW3v4pZwp/yDRaLp0JJwD+9NOHr+DPaHxY5Lm/ o+rfQ8EqX75Ilsqxy7BqvMWP0nLVnRepjQJrIfYlXfpf1zLTt1VLIw69dG0hbYcKEJjg ljwA== X-Forwarded-Encrypted: i=1; AJvYcCU22AIJvx/uOyR36Ytvk0XoGpI6KBJpQLfgXdtsfMbsLEaxztpvejhgSvr5Wf5OeeB+Oarq92iZBQ==@kvack.org X-Gm-Message-State: AOJu0YxQo8AjL3xxwwYcxjgxZbk2zXveJ/iwIEWMLt7WHFrbg2yPLq8z c6FPfKzl0DFERNg2r0Mia+GeP7z8++kWcci8VnkEKZHLAeMnvOY5lceAb0GiYWg= X-Google-Smtp-Source: AGHT+IHVhSodGEWLJbqfntGARXqFTBke8IrjtWuvc6P4tKsUnW/m6YB1esVTiG0+MVp8pwa1UVDERA== X-Received: by 2002:a05:6a21:330b:b0:1c6:edfb:431f with SMTP id adf61e73a8af0-1cecf788518mr14875554637.44.1725439288977; Wed, 04 Sep 2024 01:41:28 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:28 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 07/14] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:15 +0800 Message-Id: <20240904084022.32728-8-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 380B6C0014 X-Stat-Signature: 6drfw9osybtabrngd41o4xetkndb413g X-Rspam-User: X-HE-Tag: 1725439289-60144 X-HE-Meta: U2FsdGVkX1/TQqWkHfUzDBNuiDHEy6/7QHnk1sLe1cHKcgcBl81a+YMsfGuKajVYnj9OblN67bF9vmbRinOFp8PoZ3W+RaHMd/b7Ggk5fncNQGSMEDI0yIiYEsUict219PckYoUlqipfyKSw37+V+bLeQlaVi+6GnTWrUbHANpzY5eYoN31Kj7oqrxHVpQdSYth24uW2QjMSfez0UNhCHpoU12Lecuo7CWyEnvyCNh1R6RagsytvRaHEJwgAI6ME1I7qt/9Tw6lfTv6g03vxmFZko1gCSrCFfS3k24oSNmkTQqdS/j5dNky3fIkvED7JXrJH4l3Gmqh8AmoLz6bemoHXW1aEeelhXLfUdWnZiECSgVPF61IwgkIMy4U/Mi4jpYRcI3+XyMWBsrnP4SS7qevt4Rw1EjasvHkRlfQsJMKxG0F8DpYVid4I7VUXryo2lyiF4+DKwfWHHx0YFkxmsvp+jdepuYkCnynHt1GZ8sGwf4SM0BF4S/RrzRTwCK83Tz/SGsBQqYHiAld0qPiNTOGHxqPdl5nyDcvYZHq7RbVc5uGWg88DH24sKEl2FdyirJ8/yUHnhH9WefunMEdMGGYzn5BNQr2pu0BHCA/cGwlVo4NwnVgVnQ/PsuksAF5QiiaR4XI471LMJq/fupUHL6yPI6nsrm1mySIulNbD/6f7M5dXn90cd+D9kI4bhQsMc0kl2h6mKIdRSd25HYIEhLBELfWW5MnMHyKTgYtVZiZtictpR6uOSybA4iT9s1fsdgKth1rIDkR+2dVvNjfMKd5WsGH/FO323huAP62bpFUNSFIkz91zaW1NvJnckWvGy2Efl3uRNnItWC3YM2f5fQ5DJEGCo3/FUfniS2y+QU7T9CqCF4fsBLM9W63/XUI0G7/bU5DCUXArlLTFuxlIclEpJ4DR0gxLxwZ6huyfL0qvqLBw5IZ691TpQGW3KCFW5aXy3eJT8/R5UjOo7Xc U4AFLTzI IZrO8z6UDntWRvItBlr0b7FZBCd4/OtV1c32XUQRq8KfCsq66JT4sf9NOHL2SvsgiJmUS99qKqEs779BkMMb1u0VNzjL9kid1euvC3ADx/kX1UNe9nl6qF2dkqAq3Le71C/GJ8WEWTSq1HidsK6x9ni2U1bqix9Zqa7YjZhlzjBuR9rH1GhMG1aYhYADNxy5VvL4Be41QMDZWk6bxng/6ViGXSMMeaXCHpP6KtvboGTH8wIOgHNvhnSFlfyRMacalbszhFAOx0w5fM5rRxhnjAKcc/iHE8fpbUd22aWLnDvm/uCW/xWatCauZ9oH0rP25q84rEWshoj40L5E6NjcBqSe7njoPnDTRX0Lcvts3g7pVkERHuuU+3n3UaPgsXWje2R+Kktc3WGwvAvwvhK7UxcVnhO+0see0ywNXFsf1ADavIQ0Fcv9yzW0srQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. For the case where the ptl is released first and then the pml is acquired, the PTE page may have been freed, so we must do pmd_same() check before reacquiring the ptl. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6498721d4783a..a117d35f33aee 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1605,7 +1605,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1613,6 +1613,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1658,6 +1661,16 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); + /* + * We called pte_unmap() and release the ptl before acquiring + * the pml, which means we left the RCU critical section, so the + * PTE page may have been freed, so we must do pmd_same() check + * before reacquiring the ptl. + */ + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + spin_unlock(pml); + goto pmd_change; + } if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); } @@ -1689,6 +1702,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, pte_unmap_unlock(start_pte, ptl); if (pml && pml != ptl) spin_unlock(pml); +pmd_change: if (notified) mmu_notifier_invalidate_range_end(&range); drop_folio: From patchwork Wed Sep 4 08:40:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F118ECD3431 for ; Wed, 4 Sep 2024 08:41:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FC368D023B; Wed, 4 Sep 2024 04:41:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 787048D0239; Wed, 4 Sep 2024 04:41:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FF408D023B; Wed, 4 Sep 2024 04:41:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3573C8D0239 for ; Wed, 4 Sep 2024 04:41:39 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E2FA3120E5D for ; Wed, 4 Sep 2024 08:41:38 +0000 (UTC) X-FDA: 82526412276.15.A95DDF2 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf07.hostedemail.com (Postfix) with ESMTP id 14FF540011 for ; Wed, 4 Sep 2024 08:41:36 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bukH+c+T; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439273; a=rsa-sha256; cv=none; b=BihjQLgmy5Oz+SnL8R2CM+3MJHz1Tz4aW1qf46egRLXFbjQ1J6lPu8nfzpdTBvNPvgGSvi 4AwYaU9wle4lbJX0MUo+Cy+6N8CFwSlvj7l6Ze9k7EqHM64cNXB2PUY5Nzvzua4mHrj+9h KqFCYluTFzUZ1n2d2BpRgT5CApwwj5Y= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bukH+c+T; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439273; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ffoF6/h1+uprLqHi9escfobEZTHc+3j5UpPJif6C52k=; b=grfZvLB3FfHJivQSNr3abiLbHePUWS8wSvnpXwy/CbaBJSmG/Y8sLdj0Oy0u5cyl4CSu6D TOEktCmdai0CBxh0jZKCgo+OWZBpGCV83LITwGv3i/e7CTfbtorveABr6dCVZNqB13mGrW //xDmEBiTvaEemtGC5m0GkAO1x6ibc0= Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-7d4ed6158bcso1440729a12.1 for ; Wed, 04 Sep 2024 01:41:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439296; x=1726044096; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ffoF6/h1+uprLqHi9escfobEZTHc+3j5UpPJif6C52k=; b=bukH+c+T/nx2mTuA4ed64O1uXJDLjUBtrkDPu0/SvWEhyR+rgdPdrSbKV7Khh0GAte COHpoJjzIssydrF7SpeK+SBZznHyrhoQfeJGzkJCwU14hj5Xw5hGuUL3/F9EJxE8d3w6 A8zyBUchgZ74kLtX3Q2BsK6kIyO1YEJdA95sRisFz/NT6JI16/18Sz7u7rCou+qRm2Dn 0+7fitYroKNcJknDftuIBtlx8uysIK3sZOogZdcsc7PavBVfO0rWb75/QJkOK6AV70EA 9XmxAWIwX6MZftXbK6b/fsxACkqmzZE6ZyLB69+pogdYm8AGONurJN7rqKrmDhrsElnQ sApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439296; x=1726044096; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ffoF6/h1+uprLqHi9escfobEZTHc+3j5UpPJif6C52k=; b=ao3j3hN0GGUqz2DYcnj7NzKxIZB0EHpPbKi0EX/gg2zyoE2xueYvJGNC2XOiVg0UMU eA5OmhiCMS3B5EH7OfiIZ10Bh1tOQJcUIocI8/cxtJu5bXY+bTaLMNHGrJVfgs5k4xKh mFvC6OGJ6Gyq6HNuRP3Lu4hI4kHP/mIjLaAwgfm1fHZ3pDYBCvdkN9gdNIB0embI5+3G QXE3jHFL6aQADv1bKIkruO0AsSrSa87XB6kVQ+Lhf6YHbjH3lyQCw6c/FCHErwcIw6AW tcJfXIq+qg6LvDt35X2HD2NF0puR/v+0bLgXotC7M5q9c6jq+oaMbB1151uShgEoa0+4 +nZg== X-Forwarded-Encrypted: i=1; AJvYcCUUnfUh87cDDbFnM6KDTIY6B+GMW7dTNRwyrMXy7LWGJcTOHLYBmpHA9mMx5Tbv7x3QFh+c08lzAg==@kvack.org X-Gm-Message-State: AOJu0Yzp0TyPOG8csQEctlmN1bXHCAW8ZjyyE6OQA6wh1HtPT/WixgVK lCxYKiQFnZD1Ic7sNuFBt0WTmM6PhMiiBsVh8IK3xX/1hlhMd2HqikFFJkW1Izs= X-Google-Smtp-Source: AGHT+IHnX4UPdOrG4UbMFan2fo99NWoUyUAqXexLL4zg80mEgnx1zTpF+DDeMcB5gD3OIXAOIo10hg== X-Received: by 2002:a05:6a20:b40a:b0:1ce:f77a:67d4 with SMTP id adf61e73a8af0-1cef77a69e6mr6594806637.6.1725439295802; Wed, 04 Sep 2024 01:41:35 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:35 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 08/14] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:16 +0800 Message-Id: <20240904084022.32728-9-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 14FF540011 X-Rspamd-Server: rspam01 X-Stat-Signature: d145a4iqgz99d4wauuh5gaduebg37rgo X-HE-Tag: 1725439296-589862 X-HE-Meta: U2FsdGVkX1/yCMR36eycI7Gvf4FvivCAcsLJcsaVHv5/urpddwyUZ+U/J+ZtTQV61O9+sBkE2PvjxU5VAHR6o2CgjVm0VTyAnPio88yCPTz6it5BoD7fcfQpj2f6ZwBYS2aOyrebgMWCAhyRzcuCi337PlB6UgmZzI8KBdfoB8O1a78FMpvWvPH4GriRAPaKM/LsjlR20iIi7Erkm7MGAf0kKLji1j9bEZFeJEwi/LfKkc4BSGwE+ZtJk95xzQ3cpPqtFRmN9P8/4IR8gA69WjMidqUb6c5gY/qOqqtbrI+qjHRyWO4Hca/hjyS+nSKIOqwd0mm9iv8NOVMEbYtB6CvDy0syiPCSMTdnm3tOr5gHiIjczlgG8SBSt8t71LXYauQJB8kmmUYcRgSFFGBE7WyDTgdvMC7raOL0vET+HkX4vz8H0iT2gHgK9DMdK4RKL4x7pUUGV7TWVdoKoc1tpEFQKoOjHZEfh/8XASRp/KaAURxaOdd0Qc3JD0kzza66Q2ak0PjrXOrzpjtFXg1Dnw+nN6wgUDdgaZQV7YOqqHObJF2avJlZ2dLfaMraGnONabjaBIVNmBo2hiK2AVvpHLPisZi9Bn6ImqAmpfNSvqXvPUiFODt7fv+UiyeVLhHuKilQnZ4zOV5okK03ruJY/UhTCZtRlogeX/FhLlOiftWVlND+iWBB1BZaX2eSSHxvCGC+xysLZYBXx+ttbTy641lExjzRef4Ttx0+aXElWL5IDLtH+Sb3ir4NPp01Zui/0pEaC81ItD0GyM3cnPs3SnfgLHMZzJQtmwsiXZkt8QJEtDQyUl3usHcRTDEYUvYJRuhJCtRKn0JWpElKGAuu5XcIRlz5PCAoQve3+MhmoI73r56zKtYj5PzVW1XXisfWWTln4k6X091Xli0P5+01DnDe7ykhvp/Huk+8SL9MGtC8DVS0xYLTwxlg803CzwPguSJ2CiSqhpBol+yn1kG uDgiPJDI K0SNkEpYsvjYybjO4bMRk+jwaQ1dqI5XcgKacFDVxlbQ+TxetUJ5VZq3x8SLKBO37XhCMB0/ebH3KrZ3oWAk3PesKryM3S0HP2hZr+8FeTfqJKERGb3j7cup1BGN7kg3fBRAVuosT6CDGXgkEeqa57yfNO/s2eUmTJ0GJaTtW0djUZ3z+2tqJ6/xSNLJf+HFz1IErY0rp5iiTCUwIcJ7SdA8qnYPCR8QLJC/LIOotIVuCzeZrUK+Jvn2nyOSw6rvdWZT1AbgtzbAa1rwZu8wurk3j71ZFcMk8TKmZLRWlZ6poQiSsr+bQS82TmmyzegNoDIBWZDm4bqzBrnukGpwbZESVdUuWTLATAqtL9ikVSUDcxdKOMA+bGBvsuHIRH1+aTeqxqn2hZC9W5Odv3I3udlfmgqXCfldiV9QMkkUMxb0iueg2vrLUnE65Zf2IF1AUqlFt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we may free the PTE page in retract_page_tables() without holding the read lock of mmap_lock, so we still need to get pmdval and do pmd_same() check after the ptl is held. Signed-off-by: Qi Zheng --- Hi Muchun, since the code has changed, I dropped your Reviewed-by tag here. mm/memory.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 06674f94b7a4e..47974cc4bd7f2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1082,6 +1082,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1107,13 +1108,28 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * Since we may free the PTE page in retract_page_tables() without + * holding the read lock of mmap_lock, so we still need to do a + * pmd_same() check after holding the PTL. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ goto out; } spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(src_pmd)))) { + pte_unmap_unlock(src_pte, src_ptl); + pte_unmap_unlock(dst_pte, dst_ptl); + /* ret == 0 */ + goto out; + } + orig_src_pte = src_pte; orig_dst_pte = dst_pte; arch_enter_lazy_mmu_mode(); From patchwork Wed Sep 4 08:40:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84075CA0ED3 for ; Wed, 4 Sep 2024 08:41:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1946F8D023C; Wed, 4 Sep 2024 04:41:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 11D128D0239; Wed, 4 Sep 2024 04:41:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED8B08D023C; Wed, 4 Sep 2024 04:41:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CA6CE8D0239 for ; Wed, 4 Sep 2024 04:41:45 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 86A3D80EAD for ; Wed, 4 Sep 2024 08:41:45 +0000 (UTC) X-FDA: 82526412570.13.8E1F17E Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf07.hostedemail.com (Postfix) with ESMTP id B83FB40011 for ; Wed, 4 Sep 2024 08:41:43 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H25gz+Ps; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439176; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=6B8znnXl1a6pGCe77hOJy7wDF86eazWgtv16VUMJpF+LR3Wt9fg3dwyjCcEu9teqE2YI0f fouMnkbrM8hFUs7zY8wdQxkoIh6k+g8ZZfNhV19uQCeB52RKwoXWlhIl7G7Z0kpuo7nqRt t8+k1ahXBmF7FyxOWWWi6FABoDFxJ7E= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H25gz+Ps; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439176; a=rsa-sha256; cv=none; b=v2I/ABaY0ib3yTEj48sYnZ6DAmGjedlN+f9VshZ24U5AaIq73V1RwsFb2YJUoTyt1Fvo+F cjB0Sqto5b5eJri/pInmAqVnb3CQUHsL4TgSZJm3g7OgvsVSgmLhsPtLR+QIJCYpAu3+uC NEOdAjS2+S4V6/cV1PtB1a5n1LStOuE= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2059112f0a7so23538685ad.3 for ; Wed, 04 Sep 2024 01:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439302; x=1726044102; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=H25gz+Ps0S6SUSJ9vcfZMbqrPPVbwe2PXrorY61rfbRwMkzYD3OiMGI2jE7XigRLK1 W2q+cm6u/wItxa9Uf3jMWGkrOCtHmPGbKpA4TCMw/O9RkZAQsji4hqKFZxZPBMdd6x2H 3DtmWvW/W1echXtMXA70+AGyUj9Yz/xSrRtDAlCRqCs2p6TELxGj1/K9x0duZ1jNQVE1 /+PSIRcSMX55gvNipvwcLRoMIul2+kSmzu3fIv14r8U3+B6bn9JdUpL0+p19YIstuP8C sj/cCKMf9zYpIZOSz8ht/x0F597XtxPVO6zjLfbWJYIWO5PrsHbhdbTU6At1JS3Zu7RW Af5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439302; x=1726044102; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=nPutc4PFD5T4ZT5G0hTEOCUCx7JVwnJ8Ioy92QuDhpRsM32yo3slvpHGqWNGktj28w BM2PDNYxK3Ddc+4Hu5bV5+gaxyje5cTSXqMSJjvsAF09Ab/mHpzXiy3Cfc/RYCiUlasj lVv+fOVo1eaWfehmhkBKqYYwj7fEbCTaFoPck0UI6NOqPJKsUqetCTelFAIjqXsWTP7V VWqUYXlwRIsd0XivW/hzXH03yGYJTtycTDVjrbE/0q4Bgq0a5i9C8u9f1pTiNA8Um7vR fFTLT2I62hJmCEMdwzIRoVY+lFZkkEcdorhcQs4qY25VPb29pn41GEnOYczY77A4ieHd wREg== X-Forwarded-Encrypted: i=1; AJvYcCVvHoHZV+WrM+X0cYv8k1w9NvfoVOdTyZ3vF0smPfxZsq3jqSApFKYozGzUlWoSdVbEA/ny56G10Q==@kvack.org X-Gm-Message-State: AOJu0YxW/k7oLqnoPZu7jTpzlzBjuzCB25g4PNsg8Iy/cZ3xTz+wMZEg kt29ef2fsEG664Ul2jnwAXOmGrTHxxyZ459mKW+FylXw092wuJijsDCDVB6RIh4= X-Google-Smtp-Source: AGHT+IGlDE+bZkJMe3LtnR3dMefm4RFnOsn/qpiRzdld3O/wCPCkO6r/LjjSl6iGWtDIKmNPthI/eg== X-Received: by 2002:a17:902:b68b:b0:202:4a24:ee with SMTP id d9443c01a7336-2050c45e710mr147539585ad.55.1725439302519; Wed, 04 Sep 2024 01:41:42 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:42 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 09/14] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:17 +0800 Message-Id: <20240904084022.32728-10-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B83FB40011 X-Stat-Signature: zqu3x5h9afy8pgyhydshop8b9tqn1rxh X-Rspam-User: X-HE-Tag: 1725439303-537800 X-HE-Meta: U2FsdGVkX19nlvR2HfwUGGtViMFQ2CiKqc13xjBILUAY8x2UU/BqH/XPGOroGcRmH6G2kJUOWvC/821zhc+Aij6KL8JxDKVG34VFcePP55HQ961YWNvbGGotPlkgA+uL44Nz+K2mScYhvKIkP4oZEhnnuAYnrpXaSfDQIxS3Wi9xZKVDyX5WUz52nO4JnYBwhW47W3zpEqw9rFzCQFOGagTKUIMM98yDwHOUReqzTyja9TE4Eq15Vjhdav0fhdHjf4nybDIXDxhQmO9D6WUv/G/QJZWGasJ1z0CkJmpIJFIZb3pWmH/UnKKzgcoW3iDdiqfZViQbGk5i0wSyPjEOQs31+CqA1nYz0mk1VViLpA8Ll+eK6nsK0ww2aXypcrFEiCcE19RRR0JzRY1rr48h7KpkvNmh31Kpy2jSy71zQfoMRqSM2YM+XRyqWrbFae08qbBN2q/WSEKAuFi/fU6NAcMTFQzELCrstaIWRK33zQjH4/WPjOQqYdF3aVtOhrbWlwDWHVY6d5Dak7XxKUzL7cQpk7q9akYvIJC4j6o0xP+Ds0GBjNvC5iZhkHDv4yYw35fP1HrCO+yOd2NPiYungEMz/b+Ufw5Kh2wP4kjry+7qZ8zWruqszqZ7BeiG50MPI63+6FzoRHkk2OcGHlL5xJBRFT2zr7Fgfoxvu641TYxxTy8APsRGvTB4XSPRqspgxSCjENaeI+w1ntDhouH8x8RAiC8iArUJ9K1ZnZQVxSLmf7M4aC44nc4SMcVz44Uixb5apOdgbAR67RzDQJA7XJUrPULJhfV2nwpCWEZ6COqqgti0QzFf+O964nGIMhY/vwtRIhqFh+Jj8+N2eClWewB9nOoPFGlwTSA78AzkuAlnDm6TxtwIW7qizNLDqQG8caOKZaXG0KYTho7RVdM7HI9Yj3ynAom1d06NRxEQUGpjwirq0eKGhM9WsPp9cboVD4Vq1YG4PlXVDH9iJQB U6+0Wh92 wzZUgen1QZy+1KJ0JaDMxa4SxBuuz0Y1RfpysC+yiAP03zP4C3EMr3Dfm2uIOMt0c9O046SRlQ0trcd9sPNXkGRfuTXBGavY7N3ilejrPIk/C9sq4nwPZUaSiNm04671uR4QlM90dGFH007pl206Snssyy9uMd4d+/oGJuogekJdLrsturxVTPJQCXLere7TwXwylz2zaxzME396gqn1l2h0Y5kU9x4KiIAwpv3zLmXQ1KS6GA9ReyoOUtSZsBiF1n2tHrxx4vECNznVZkPAg/BApJknS35NhTD/wQgpd+d+zC+QkgUEaokOiCwJcREo1c5V5Z3fGHW7FYhXPPqFcKWVGN4Alw16ZM5+MpaRX+nhZBcaoZ1aJvw8v0JIQGziQ2N59uRll1y/HLRdpXbRqXle8wFfDohobuj6jGWRhn+osb7U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we may free the PTE page in retract_page_tables() without holding the read lock of mmap_lock, so we still need to do a pmd_same() check after holding the PTL. Signed-off-by: Qi Zheng --- mm/mremap.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..16e54151395ad 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t pmdval; int err = 0; /* @@ -175,14 +176,29 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Since we may free the PTE page in retract_page_tables() without + * holding the read lock of mmap_lock, so we still need to do a + * pmd_same() check after holding the PTL. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; goto out; } - if (new_ptl != old_ptl) + if (new_ptl != old_ptl) { spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(new_pmd)))) { + pte_unmap_unlock(new_pte, new_ptl); + pte_unmap_unlock(old_pte, old_ptl); + err = -EAGAIN; + goto out; + } + } + flush_tlb_batched_pending(vma->vm_mm); arch_enter_lazy_mmu_mode(); From patchwork Wed Sep 4 08:40:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A0B1CA0ED3 for ; Wed, 4 Sep 2024 08:41:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2A596B02C5; Wed, 4 Sep 2024 04:41:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB1CC8D0239; Wed, 4 Sep 2024 04:41:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2C8D6B02C8; Wed, 4 Sep 2024 04:41:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AF5586B02C5 for ; Wed, 4 Sep 2024 04:41:52 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6E2BAA0E01 for ; Wed, 4 Sep 2024 08:41:52 +0000 (UTC) X-FDA: 82526412864.15.646BE0B Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 93E268000C for ; Wed, 4 Sep 2024 08:41:50 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VMo8FuaS; spf=pass (imf02.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439262; a=rsa-sha256; cv=none; b=lxc6i6aLGd8YBaDrY65YKzVVcak8gE14eMEzQ9c3Rca3GDOeIefWMLNBxiSSbcDnqQ48dG YKG03/1qhMB6039Ub5t8c4wLxdyqSfvGWz4HW69i/I66Hz4b2y2TwVkuYqNI730J3g+53B Bx+RYZ2GCmgLsxVBlSBQg/pDfvR7Jp0= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VMo8FuaS; spf=pass (imf02.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DklgHNcyYpWTok/ex913aKZBAOiPUnESXBkqa0TTMVk=; b=T4GXdcLqrZLz9MqB4fN1c2dE2vMdvyI+U6IHXPKKRWlZfDEOFsV+xFKT42WkK35bYr5htt TEOar8cMKcle/kMbs4Bw1FNuxuBHEzFOOMfZFyXdKYK79evr0SmXCvwKJ/f38niCK6GhOT /z/EfijOovtsxkjVRJl/PHxykI0IU1A= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2059204f448so20046305ad.0 for ; Wed, 04 Sep 2024 01:41:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439309; x=1726044109; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DklgHNcyYpWTok/ex913aKZBAOiPUnESXBkqa0TTMVk=; b=VMo8FuaSklXP7Xuhzq1GicQ6c8sGeWTo/xQW3CIvlyzw9SEWaHo0KcfDMD2ZbS4Y7J 8EhtDux1N3M55zXamH5K+DGE41xYDNyGLAdKgcHpO6foDdTGjlnqlkqmUTCgm9BPchm5 6JHHoKzzEuaBnlgDqk27bT8H+RwJvhjUxc9qtytD3r4unka4KxERdo7acY9s0ae2UJ3s UnkJwEVoJb3qzB6vD8xoh6MUJdZlmvwyef0GUC3VWtte4GxFZUFEDuohkrBDm1+cuPxg QHPECMniWUB0M2xiD+psHAmEE/YOd/M+FlUZlLQHdXURAJwostCpFiGezFXEYTNh45hb wytA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439309; x=1726044109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DklgHNcyYpWTok/ex913aKZBAOiPUnESXBkqa0TTMVk=; b=o6v3CblAbkCAHBDI15qQPGthSIE0Fo5drvek1rh7qgNFhNtjL2NuO9tRldoBnF7y/t ihWd83NOSY61MLKezuconVC+OBu94EiiD48joaeWcZIySSwh9/rrEAQPIJudNoqybQgY QS1RYK/JtwK0MAVnw9It322JAVx26jIX/JCkF/oZzJOtfPriNJbgqvAjqtlXcd/QK2q7 7IrORudpUz4v3zoSczULdQMcKhghByCTqaQutyw30YCeE6kJwifwcJF3SUTruUDeo/Vq 2YoB92suEHvGZS0wbgVjc+PcoGy5taB5e/lSjlwUMSdRFNPebPDZP0FlaSyyn6dyD9Y4 3FKA== X-Forwarded-Encrypted: i=1; AJvYcCVif1I/r+pLob1UYAJRmcZDNhLMyEul9S8JLbWyBHvFg6G8fSiF1TEbXofycBs5izQvrOiKBCgEUw==@kvack.org X-Gm-Message-State: AOJu0YxJAuoHrU90i4njYJiIxMKS87MsG7lfApKIjAin/Au3M8UZkiYl dYMo9AinOW5lhdMbvXvYB8mcFnk4+m/HHMiyq8LxOtMYGcbkCPDovZDrf/Q5kr8= X-Google-Smtp-Source: AGHT+IErn+gfx0jNqN8oYReDzi5ALFGdG/LuObb3FXvUpU+trUkKEzs/YdM2MbdzLHq+FDgFhXw6+A== X-Received: by 2002:a17:902:d50a:b0:205:6c25:fe12 with SMTP id d9443c01a7336-2056c25ffb5mr96453155ad.34.1725439309345; Wed, 04 Sep 2024 01:41:49 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:49 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 10/14] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:18 +0800 Message-Id: <20240904084022.32728-11-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: z5duycmh3fnmof6fhmaxtoz1rwj6bqif X-Rspamd-Queue-Id: 93E268000C X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725439310-203101 X-HE-Meta: U2FsdGVkX1/CSyuSGFnoOgIwOutiXxHCFBHdqNAIF2fd5uXmICk88oDPt/a8hRuURhK5SMnz80fr+RRWw8j0f1U+wx0sxTLlD7/ND1nGTQxjYhRXkRHKca5rW2wp/WLbZ/2MNuSKzUBb369Yf3az9MSJP7YlqFZ5bBDUIdAIC0Bgu+H2JQRv0Nlck/chAvtqoF/10vu15D0nEmxDjXzNqau7W4rPq19HX3VsFCUpW2CHMAG+lNcpxgQucJkMH1uQw/VfBcmNZrBtzOR/9KAnq9KoWjCpubfIrJTyT/oKWUwFiIcmBj/5MHuLv/gEsNhS9cOjMdZ69uHSjHxuZIyt7RdD14yVYVf0TUISFJbDFW9MgOyY5J590JaybGSv0SF7OvOCpr87o+phg6W1U7qASAzua1+9yodFZdw+NF1IbBBY29erW2U8bpkBu3GI3U0E8lNiAW3ExLvtQ/ZNlbKCUkT4sVf466f3IS9ivJj0TJeOqdxHcbhjyMS/T4EIpQWNywWhiLJdbrnekwmDg7dXGXdpaQgeOxxV0NkEta8oymB31ZzcEFbJXBt/bOmgJpnyg3Hu3gWzxc9Mr8sS/sFOR3aEv1mqlEXM0mpPvFhuojbHofJGpM1wfw+cboofqTWXSiwM6cr+Hf/b8rlE0P3bCwi1Hre61Y4oY/AK8BibTfNi6AT7J75IbAvcQtFOQ1nQWvxruQ+VfnSG5o33h09dy41911BIMAdpj1x9Eo3VRUoPCPM94209LU3Zv4TXlHYiRiYr/JbuFjDINMtreDkWIENQxIKQiL7abT+EHi20y+Zb99N9TTqN6oas2C+ldEncTsX/itPmCNGKP4bZrdtd+RUIWO4FeneK0TdijYzA6WvKcS80JgQLJQWHnzW/2HP29OcW9g0JBWd9sQYErFGgb6ZYAOzJDI2c9YH6lGDlIb5+jCdg1i3/T0c8w/JTHb7/zQN9Amd0U7I1aV7S387 E16Db8Lv BSNaU8vm2/l5wu5x0bjTXSwD/FgwfgueWum9tYixrG9grvzrKQy3Wbl4Zggc/ZTjl59hMTjMGZN1qYoSeTJXDfkz/GfjNe9UFDRinNisZh7TV6Ij/4rEj9DSpX0KVJ+nbLA8pp3Fc9QYCib1ueFVkA4PxKa6Pm6X5z8rMtImbXnqSfjkWlVCz7YwCmc48RUjd/Yz2OZ2sPtAqBjOXkMOe7OV2o7Tz1eosKGNJ6qQmDb8mq0dNkyDrAfHkbgh2GF4/LubCnraWwJpBzdXXK8yoEXiwxhYa27Vxc8Ns1p5gbhCu2PBMHYZv3LD3FjIdvYqnsFUfRgIVojmLy90BA88QUrXBkVysq17oKt5ovrKhMFo0XOtnjxkeUpowLZ+2yQhWeEbetRGlJtyiYZeA+HQdXe6aQt2MBOWFbQFUt7eJmwYaOrEUDCmMZRk+7A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng --- mm/page_vma_mapped.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..f1d73fd448708 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,9 +13,11 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; + pmd_t pmdval; if (pvmw->flags & PVMW_SYNC) { /* Use the stricter lookup */ @@ -25,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,10 +35,11 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, &pmdval, ptlp); if (!pvmw->pte) return false; + *pmdvalp = pmdval; ptent = ptep_get(pvmw->pte); @@ -69,6 +73,12 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } pvmw->ptl = *ptlp; spin_lock(pvmw->ptl); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pvmw->pmd)))) { + spin_unlock(pvmw->ptl); + goto again; + } + return true; } @@ -278,7 +288,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -307,6 +317,12 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!pvmw->ptl) { pvmw->ptl = ptl; spin_lock(pvmw->ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, pvmw->ptl); + pvmw->ptl = NULL; + pvmw->pte = NULL; + goto restart; + } } goto this_pte; } while (pvmw->address < end); From patchwork Wed Sep 4 08:40:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72215CA0ED3 for ; Wed, 4 Sep 2024 08:42:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0356D6B02E8; Wed, 4 Sep 2024 04:42:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFFA06B02EA; Wed, 4 Sep 2024 04:41:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D53986B02EB; Wed, 4 Sep 2024 04:41:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B191E6B02E8 for ; Wed, 4 Sep 2024 04:41:59 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5A9F5160DA3 for ; Wed, 4 Sep 2024 08:41:59 +0000 (UTC) X-FDA: 82526413158.27.C2213FA Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf03.hostedemail.com (Postfix) with ESMTP id 7485220006 for ; Wed, 4 Sep 2024 08:41:57 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=R8W6gVyK; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439221; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4PxEMdLIb7w0CeH690VB/ed9+hRnYGoSsqaGDg4GBd4=; b=VYsyGsWZPq3g04zeMccqe7+sfhtidBqb53X4Akjuy5mH9wVKDZt40GC3B3QtT1SKiJ+m3A 2BvjA3V4zP3TlGhAWOPQwSmEwri8sO8iwHEcdSFJ8V/pamwsgzxmLLXAorPcgXrZCAvfei ehZ5j1cFLves0NaTv1cqDxjhhaQsSw0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439221; a=rsa-sha256; cv=none; b=fX1TTuW0WN31wrhhujWohcrDBEuL9cfuE5uNwIeVOPSWkJbTpXF8w5MNVz45gVX83eY5IE tfhHrvgR97DT4Ie+bbF5jPTfU58xvUz30kNqkRqtWp7CaijxZ/XukKExOS9yoCzJfVie9k pQUhkQpW2Ybj7TAoYEzpt1D19wy5fdE= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=R8W6gVyK; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2053f6b8201so31856805ad.2 for ; Wed, 04 Sep 2024 01:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439316; x=1726044116; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4PxEMdLIb7w0CeH690VB/ed9+hRnYGoSsqaGDg4GBd4=; b=R8W6gVyKImpqSXqQfPAJxyIrcy3rdy5qd2v+DluMAvftwTozl1jgFNy78q1JmCE7tn reaUSrj3pyD9W/5mbgaXyeo+crmF3fi65qMpE3wCfjIckz3O1rrQw+ZG0qU9bYQm0Iyc z0oQpXvAcPx57LIR9M/OUag5mQ5w8lP0sw7hhtXAjQ7p4jRiv4+f8OMeCmzeVkeHXZl/ fLmVVUbpXHFbHFPwKw7bSLEh5a49cSD/PzUwprfJmlneHL6+ChGoyS300ytioXuZ/UIM LEMr35SPvtnuAQIlxLrnhqKXQe2ebiD9dzfvXb23apgJjDb9qJMrG5WAdtTN194uGj5x IaUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439316; x=1726044116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4PxEMdLIb7w0CeH690VB/ed9+hRnYGoSsqaGDg4GBd4=; b=fxF0HysEnWynvZhYR6EtZ8h5Rr+M2XAmvqMTDQjn9lKw+cMWUlhy2v5zcGzCbd7dXp VsRaK6gG7vJifgBF8lyLDn2MJAbrbSbRx7/zsQyUHXwewdxE9emWYAuT0lpauBAoEoQz G1dtlr/COVZoJhxk9f9PQj6dgA049IjcDbhLfaVhC+2OnXmzTkAne9lb6et6tFuTgQkf de18wkWObkGnuF0H0kQJR52ot8Vl6df4Q7DiQy2D2++aJtQMclbAgKkbAitj1+D4w2wm 5C1pmjawU5uQU8swrsRDumXRfdfKOQnSlOHgVf/ZkzNsVN9DCYIWFSZiCVzKklYHVa66 pCTw== X-Forwarded-Encrypted: i=1; AJvYcCU9GuXhuceJGAF+sgb1Hnc5WUXDkp/SiBEN52FLT5PZWsQmaK9HDQr+A0kTPo75dmDr7CPwTger0Q==@kvack.org X-Gm-Message-State: AOJu0YxJLckkH5iEAfVyS1Pw6jNmbG2v091dxv/zrIniVbzOY9RPzi0N M/M7wb+2lPMbvH66cAqfIWCQrRlp0u6FlGeNRVOJOw7Gt5j2rA3ExIMysWSbHcQ= X-Google-Smtp-Source: AGHT+IHUb0AH81Bf+t9sl3x8o3nWMR3fxDcIYnthyKOt2Ee/pQCOOrlcmsn4X1RsHo/HQ+Mvmo6mhg== X-Received: by 2002:a17:902:f94c:b0:205:4273:7d65 with SMTP id d9443c01a7336-20546131866mr119665875ad.21.1725439316150; Wed, 04 Sep 2024 01:41:56 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:55 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 11/14] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:19 +0800 Message-Id: <20240904084022.32728-12-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: kb5e54w3caqcz5f6jm8x5k9jshfm6bk8 X-Rspamd-Queue-Id: 7485220006 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725439317-227487 X-HE-Meta: U2FsdGVkX18TLk+cMKtMJ7bwXl0T4M/jLxO1iDaJH2pSh64393br0knKgtfuyhEltJDJR5EJwyZhcyI/z30neuF4Y7IBHO4VOCgmzeZicRXHZMtdM+EWroGDk9apsUTtuRnm23MEEfntBL8QDH0LecTUqh6eE0ssusq2L3AgPUt47hEU4kSjJfhOGmwXJ48x+WwRm5QCGOFQsG7EW6vG67wdO4lDb7HvUVMQ7b0AT+GiQ5CkjucsbjLM4uV5x//SjrcrYW4z4q1M+sw1h8SjoWA4RUZ1yxzZbsS1CCq/bMgG3NcS1YqnOJc0nBuOofzsS+CUx26B5vrLUm7Sn5cUXIFZi7zzW0FOohhqn5sNxg3wpCcHwmI9axIGYNhG11hugUbKaKEJDbq+KOzn+W6QWQaF9lBzZitLnep0i/KenmklLOf0XDn7RIwjHxQZXc94Vf+nKP+8X6uNLa9m+/6LHTx79R8b7pOlKwtAuxQOCEadT6/j9tWN/5/CYAjd6SL7mrHDZE+ltjDGGtgxbP+GXd5KBZJpTDkuXkPmT/2vAUccb9PYa7Rq8BhdIFeiJK3r6Im0CIRB/fz370uTZWTfLP8exJnyWcUD20NCavAJyY0D137hVPbbGpsQRwWfVpi1rbO6mIcQaaP1DgXq+IK27VuEZ+ISpp0CEL2IqjMgGG3XKEy8+O9BGlK6FutcEbl9zStYuK0KsjMkdkWAAe2Ae88i6UbBENOQzn6LrJ5OdgZOd5DXME6NBwbUaANy+oaQX91mVVhIPHrUHsJRkuyCgp1QaMTqylX/aX14H++7Ga1UfQhUif9Sp4bL9YYM1m5entejx1UKiD8tRTuXTNaQkatK37obbmFWJcc8J12IoDlAz/t8Pq0rWK2JFzHT8jLSIUkuXIbfc0fkaMUwlVmNYwsYc6RtSRlj5MOpF9g2rWix7ZEwIkdwHFfW4ZrXE/RHPV4teWlp27M0Wau7ERl IyHKh/hb +kGw2jr1TDdGnNOwJIjpCzcyF3TE2Rzz6nV/87XUfL9uESYDQ5z1+rvhQX6C/t8MiHHmdLRr+XvtGTqync0b+XZSfgiK+CPAXSWA6ZaPwezbEpAnF/a3XGC1b7HmoHXhFopD4G1FQ2uSIlngZABHAU8E7icm1KwuguCHT+yVQ+tCDjXIlS/+nqpIWiYtz1Liq70jUYlAmNhuR6Zd4AU/WAwQ78/JW9/6nh6aU+hjktOwvT30X6N31IsqhPuVNjzMhMP0PkSOKJU4tO2NypAUq8DWHtfAeaqOl4TQmGfnlykGz5DML2n0Q83W6tLWXJt+Du9sek0cnUhRvOyTee6PeZWHycLjrgtXjrClCSqghQcUU8IiLMfUULvva8JGXxLgiusu6oG3CunA2ekJz+s6/lkwuVEpqLV60lqfj5/+CKWK1dg2Q82lu1GQL6Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we already do the pte_same() check, there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ce13c40626472..48b87c62fc3dd 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Wed Sep 4 08:40:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A846CD4847 for ; Wed, 4 Sep 2024 08:42:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7BB38D023D; Wed, 4 Sep 2024 04:42:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D04088D0239; Wed, 4 Sep 2024 04:42:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B7E5D8D023D; Wed, 4 Sep 2024 04:42:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 943758D0239 for ; Wed, 4 Sep 2024 04:42:06 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 37CF81A0E91 for ; Wed, 4 Sep 2024 08:42:06 +0000 (UTC) X-FDA: 82526413452.30.5BEA6B2 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf10.hostedemail.com (Postfix) with ESMTP id 56D6EC0007 for ; Wed, 4 Sep 2024 08:42:04 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VYLAj4a9; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439276; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y4jsyz9eZnVjuIeNrv9jm5llMD04VewCinfUm/XXhZs=; b=gYVHJ2zswk2RH9iLBjB2XP/0gxSMuzTbBTAQ6JAZnlVmKBsfWesCnkPMP6Yi3TR3Jp7ksz 8GJKuCAtAmWFy1Im8UKTywKcNd4AZ6GE5IvY3FdIsu5jlkwIrydPTAYq3/sTjYZUwadzQc M2TK1FLkAEvZGWYkZOwASpmMUAIgbEk= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VYLAj4a9; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439276; a=rsa-sha256; cv=none; b=CynTriYgRSUxoXOKFQZRkdA0rKl5CyZLjg/pbxv3fVPDho+5op1DzbcGAZ3YPlsbMhwwkV 23RxbY2IZrNNoDPYMJOQjgHE7uFVQpWHbRNoh5svVuxvl86WzeZ+v0EmTA07Grh9+xeLVW 7bfQdL476yYat95W9v+lWRFogRyFvrg= Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-7cf5e179b68so344693a12.1 for ; Wed, 04 Sep 2024 01:42:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439323; x=1726044123; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y4jsyz9eZnVjuIeNrv9jm5llMD04VewCinfUm/XXhZs=; b=VYLAj4a9tTguWAynEJHqFDDCt53iqUFsBS4D+5LaOOagpYhtbETbUnYUf0fYjQv1hu K30RqWC312v167gqji6y1pNKDfXjbZFPbFtpWcSJr9vyvDtfq0ufAEmL3meHCzkx28Ww xL4Yo5+A+IqZIiAd9Wqhypn0Y2zOi9Ky9psWYi7YtKPnJjy/uQpx9vrUKQ+mlwRgmhFL D+2UodjBeq1SA4WBEDJoht+1w6yXjNnSxyPknOQuF+AcnbzPsny0IW/tf4iwp3HwIlNF 9tEi0lOycxq3kHbzAqKgu8hXU9qKkIG0/lXZNWm8Vz1LDmkz6TfA86bvVsAwf2A8/LfH K01A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439323; x=1726044123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y4jsyz9eZnVjuIeNrv9jm5llMD04VewCinfUm/XXhZs=; b=E53u7GMIqyI0pSvAvxs82nlGCPIwFI8uArbxUqtu6fVRtD4Uu9r327FmNDOhw+0dqK LQjGJOVKmDzYlMjEFNgurerAT+r037JW/7EIzZy68IibdwNZ0sUdQCGkoE/eVFbW10TQ d/Czb3M0RfBSdsqccIjhTYzJvZqNXEDcOTruPJzVjjtUvsSYHcpwW/4+Qonqa01Kb4n0 HyLg1KMwhoinVeQ8E+ObT2nlDocXMMeoyIWsdW1KH0nPEoF/7VT44MzKQGMSDE/8Qb5I vGKWb8kKa3c/QfIT7taOsBzxRRvmyR45m5chRgU3piKJ4fXLcLPg+0uW4bULhmcNa4gJ Kt3w== X-Forwarded-Encrypted: i=1; AJvYcCWb5izhyXzc4jfRcYfa/3ZbGdjs1hC5dMBeQahWjytz/j5JoJfMQ8vcc35FyIZJ/iyRW26yFeTO4g==@kvack.org X-Gm-Message-State: AOJu0Yxd8Y7ps/P8C7PdxiUTOH7b9LZ0wXskRyM5AP+WNGHrBV8D0NBv aitKLWKB2zMZKb65NqNHPpSULkEjJVPnV57h6kbHUOZL5tR32DEeDC8LOAHD2Jw= X-Google-Smtp-Source: AGHT+IGXU/qy5PPe2L4ehUmGkYFpO+LVdbh/09mdfTo5RpVJKjaH3T6TEVRYBI0feUcEJkjAodGgeA== X-Received: by 2002:a17:902:e545:b0:1fd:6ca4:f987 with SMTP id d9443c01a7336-206b8341fdfmr24676655ad.15.1725439323059; Wed, 04 Sep 2024 01:42:03 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:42:02 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 12/14] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:20 +0800 Message-Id: <20240904084022.32728-13-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 56D6EC0007 X-Stat-Signature: i3wyqkth8n9gimffg8gfbw5kqbwbio9x X-HE-Tag: 1725439324-503328 X-HE-Meta: U2FsdGVkX19nx5Vyy+cZc+bgmVQK42FAojCJItt9fF5fdGbAfroPmxgTCmDR8poT0cQaYcanbNi6ZJhlc6/MqSTeMzssLE+mlUvLE4XmRjkHDM9aOsISo9cV8OKQlJWd48joDmSZ7n2JNFT6HPmxARA9ODjh1VUHXkRn/CLl6V4zEaulkqT1txddXCQpGrrKjs8Syvuorx3GGoJryOdH39sf2bTUaYktOLwoq1M8yPjLEtcPwQt/hASzMk41aROTLQot4JyyE231xqUZNZD2sanjS7RYnjP5BgMK+GnV3ZfXZj6LCTuuI/02FEIe492VO9YjTfxevRwF9fGgqggsF7fDLAxBtbK8wYqaQWnGDWFc35WJyfYD3Gh/tTxbxLNydXvBtUT9pPJR+OKOoRT+7qNYRi68tMpdxMMagJiKXeGAXjbpMB8U+R5Za3a13uxiemkBODl8MuM+q8IxiNfXMVWSVaTfc1kcveoBO17zCQrj0tfT5InQ8BRv4zdQ8VZpU8WZuAvHAd/Tmy5l+e/CjUham9c+Dz8MJNCjh3zyPIhSoF657ptsTN3iCOrfbIzylRkEwHbub943fgmAm+A3gK2kqb+Ce6/f4pR1hXa7gbPpQ22ZyGPvjUAxSlLNulyuDUQNTdN98rzOWYRYGp9dDpXe89BPsSvS3tb6e+OjfIMKR9wAUsTEeAbXZrg4PWf0UxL89CBx7xiSeQowmL+lhLYFI4Wi/WuGMvi+agMOvzVnuod/SJ3cgOrqZW/0I+34MiqaODKDEueQJsQbCz21xKkXnOlobumYp2VaXMqP8zgxQch4CRrQbWU8g88wYYgCK8sBP0wr3hkdCB75w1wastYY5c1gW7RtskhLiuPJmrg/rKLu/HZlnh5Z4jRLJYkyjgSBVoh77/wEwTYVZSBQ5l6D89luQPlRBFw3vHDE9Xvm/lAK5gUqa7FqjcXGgca8PaiY6rFM9Xl02PDPX6O TR2cl9FG ky3Vm9KzuANdK77KNH/xQo6MJBd8j2lB1yPpJpe5LXTEarg5z9GF+OAt9GR2A1RobOayX59HmAiuHc2S3PmknxjGVy0m8l6U1TmgEgAQQXmzId+peV+z8++mGPqjJSBopTSyA5bhj5ota9bY8b3A/ZfX0T8rn3TCBRYdRFUUWqQZB9i/plYXWBS72YJIWN6zyhq4u44Elq6QZSLsR9JZObE3S0GSVARwgPGnQVn7ARSMA6SG03uZq6dkM8oSOdbQxKTkAKC1y+EP/yie5f5S0ty4yL79DjFYWN32mSLutNqe5hi2sze+EZTwwqLdr/ceKwTPNdMrplUAS90bUaJmT8MtykM3O7nhWBZkw3BowJyuRNZd5rd7DQJTYMkItN7VD/yuWiBSWa2WNfS7hdUY1nTFMVu0fLhlhx4GMyGxLeeL8l1Yfdw7P12vugg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index a9b6a8196f958..36b84e46cd7b5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3375,8 +3375,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3384,6 +3386,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Wed Sep 4 08:40:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF9CDCD37B6 for ; Wed, 4 Sep 2024 08:42:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73B248D023E; Wed, 4 Sep 2024 04:42:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C3008D0239; Wed, 4 Sep 2024 04:42:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53CCD8D023E; Wed, 4 Sep 2024 04:42:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2EC788D0239 for ; Wed, 4 Sep 2024 04:42:13 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E0C52140EBF for ; Wed, 4 Sep 2024 08:42:12 +0000 (UTC) X-FDA: 82526413704.28.3D19A42 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 01163100006 for ; Wed, 4 Sep 2024 08:42:10 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=AMPk75Sg; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TUS6V5hps9BhpBteFi/blbp/e+0aHtnhPyo5Zqj4Bag=; b=bnevHeUffScUwAMte0UBy7aEbfuTAJgSYhDE55fnOjKBcXXxHLzWIPCE8qArlwCzVoSv6r lbBJmsIoaxB8VCOfj6GElws6JOgJMs7xWX/kXZIfwMS1zAOXwMpYm250nYqCAAJ5bSNEba pSgv2hSY0LyfClkN2U1b7IfgsJo0QgI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439235; a=rsa-sha256; cv=none; b=lRMssAmbkXV3lKUrXNnUGhmrv2tkc7YBKAX83DhxhGyC5zOtGylIwD99mOgsRInL8KazRi PV8qBk+6Yd4h6BhUqsoPJiON8N/oinz9Ix4wWx0UK9d0QSmTAL/l1h/LORma6TGsZonDJF X7yPfUEMoksBCPS8/576u3QnVMgPlyc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=AMPk75Sg; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-206b9455460so3343435ad.0 for ; Wed, 04 Sep 2024 01:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439330; x=1726044130; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TUS6V5hps9BhpBteFi/blbp/e+0aHtnhPyo5Zqj4Bag=; b=AMPk75SgWNsmA/SqfqK0CuwI7ACRORWOtPdkJRiLWVABtYcB77qB/Pmdi3amFdukDt c8VtyY4XjvB2iKxkC5+2MjaDmk/IUzBjM8GyjcIse1ibDfOKDGnCX3PNWCGgo5vyehRL qN0I4JEEiHSNSVjBasivFESZQzudhsntpzMpiJRFyUkO3wmLLfRvrltSeBCuOTnYQY/Q lYRDOb6KtAb1YyLPHiZbtVekaOOaKUDNiLElNZoIChBneZLsaQxyBWXoC1lNCOjgt6jd nFdTpf/Q3xuMUQQ5fKCLHN18dqbvWFTNshtwouZnNT0xyXqc72TGsu8bgZfjhtDykbqy 1dRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439330; x=1726044130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TUS6V5hps9BhpBteFi/blbp/e+0aHtnhPyo5Zqj4Bag=; b=PN8lUGUke00613w4YuWgpvPoYJHrHcrUGWlVJNZtL/doW8rQ6Pt14z15dbyB+RaJO7 2pddmPBhZBXVrl2BTFTtk63gIul+5bjWPEIHHgJ2KAVW21ZUClzAbD7hCq4iqlMSl3Ox ebIDy3ZQb6VYVPSTY+SZxBlXRnVUhTc2MfuKW8bCCMGWw4qOVdp5dARdZQmty0OGB6U+ YjuGhJJVtr51N4tc2xWRLPcJBaCf+debgO2YZBkDelRha/8zXWxkub3RVuuzKkGg3fKk O/YQlciNCANoOMgLLnMyJiabQUKm1u4rrF2kRqlHFzdYq/UXlehqE1jb7LUxe+eZmhXp X2ZQ== X-Forwarded-Encrypted: i=1; AJvYcCVjxOTuVzCdV3dxA97nUbG2ia/BtS4EAFh2vS1c88xWvAs3MJK8Zj6aq7tCwsoYn0f30xmNWddipA==@kvack.org X-Gm-Message-State: AOJu0YzulU+rIAJQaZnARhgUNCZ5TPhB5Zf+WmaRzfyKrJzQAJd2ixCR ZEmQ1kn9Miw+1PPSmW/6kyqHjzgyYoplCprJtmKRj7F8cF0vQYd3+r737/J9jLY= X-Google-Smtp-Source: AGHT+IHV13v46mfi8eJv03VSK3v4h4kaXbg2iDfLcHBwpuweJiwY6TIZiDRC9qw17xCxMuqJ20qctg== X-Received: by 2002:a17:902:ecc9:b0:1fb:7654:4a40 with SMTP id d9443c01a7336-206b833e006mr24427095ad.14.1725439329649; Wed, 04 Sep 2024 01:42:09 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:42:09 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 13/14] mm: pgtable: remove pte_offset_map_nolock() Date: Wed, 4 Sep 2024 16:40:21 +0800 Message-Id: <20240904084022.32728-14-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: fdb7q3mt96usywpzf6t5os7ut9uqr78k X-Rspamd-Queue-Id: 01163100006 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725439330-369858 X-HE-Meta: U2FsdGVkX1+8CNlj++f1yYZ5DwNwBq25nUjvEnQ6XNtuv0GW59pEZygIWiQ4E8Hpa71fLoAIsi1B2sXJSzcqBRHqQkJTK11eUtrWR90QDaLdUoibcVaNjiS/o7/RAgHIcEPeEcRnVZFhBzI3LbEL4oyM1hHhzz3s34JdMGKPYLYkOLbMWSSUY5weiYsxvxdOybLD1k61pEaiYHg/0shaY8B9Ej9Z2phRs+s8coLwKeEChxnpvw/Ey1t9QlrFtFfU+dxeq/lfGLb/myusD4x0pPwHvImg4sk/uuH1YFV2+rFxGEL5sFS5yOegsFZOlJqT1s1Nd2phOl7VVXPep7oe3DgF3jowHrDNeLEVjel2iGLQ9nDTRb6FX8TSVkxx9jy2M0//SQedWERGSZ9kpyg+y4SCC8pjte4fhnqIDGJuYf3FY8poPXo1D2oeQzLiIU8cySeacBtyT7brVeqgt70qjZQYr6q3fOlcBf+aNsUndVM6HIQrTYwJOm2e6dBds1i7kkPcWdVY815eCJ2dfGEv9V7RDz9fsr7YD1c2JfKBpOE9dXA1xcqVPTEhngkHKALOYgxeikXiLiF6Xneq+aCzgyhpi3cm5ueQ0FYNCLfP9oqK5SU/0fVBOhgf7xa+E6OvLfkxKMa+6y47yRyXrGM6KrcRvkG5flyUXikjk2a5oIDGOqarTSiuklILS2mD8LF3hJWrKRD0YLCo78MYGJ92Z1yCqL7dvXuS0ED8Sj+rke2gwamrCGDh6TFaPYqP2EYPF1FSw92BkeJ21IRSSQl9Oc32HJM+abT2twfPczkQ5Tzxcp3IR000XSfhv+BuGqYfaVzKwvCV/u/3VA6aRfRSAV3FQHoyF1Y3a0pFeWOsylzgMGIYbMC/egFv4IuWIJMRLCaztxPmxyoklW1dhw43lwTSD9kNivr30TMZzrSKVGgZYtkpbWb/plM8YXRbkolbjebO5YGKjwxUhq2XACK hJAT0yOC yR04adYn0X2088BE9b2qn+Yw6VTI9CPeQijVssxt5NkYajuPeRN+QLY28d60SgClPlmzg0UUisoiDy+3GwqNQeAHmU3j9b8BvjGxxTq2FmLkLXT92W5hb8qD7OWGkZc8vUm//xLTWz/jxLQ+hqT9RzQnu2Zi/sZJ5VLi4og8pFyrCzbCRH75ZB7Qq4qbti6T9qZK7Eac9ujQ/Z5/BVfK97bvTdSuyct8CpNdwie2i0hpQOTOvhvsoVBNDkyJ5iYLVPg+t5TQhViVnaG3fw4kzLfFNvHHRZXSKdT4okSFMXg2nFdMwlY3l0rkOBYEAC6uhacQokuOJPngl+lC6pl1GjputwLgc9SSRPGRE4M7KvT1ejxGHH/E5CShpq5xwTqG5M8ndEeZtTVtsSVZF24YpNt20dHfCTaNusOFXrQoQ4a8cGzdfbDzKtIyKXQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1fde9242231c9..5b5a774902bd6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3004,8 +3004,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 262b7065a5a2e..c68aa655b7872 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -374,15 +362,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); * but when successful, it also outputs a pointer to the spinlock in ptlp - as * pte_offset_map_lock() does, but in this case without locking it. This helps From patchwork Wed Sep 4 08:40:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40876CA0ED3 for ; Wed, 4 Sep 2024 08:42:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCF3E8D023F; Wed, 4 Sep 2024 04:42:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B57048D0239; Wed, 4 Sep 2024 04:42:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9833C8D023F; Wed, 4 Sep 2024 04:42:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 72EBE8D0239 for ; Wed, 4 Sep 2024 04:42:19 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3EDE3120E4B for ; Wed, 4 Sep 2024 08:42:19 +0000 (UTC) X-FDA: 82526413998.19.2859515 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf15.hostedemail.com (Postfix) with ESMTP id 651A6A0002 for ; Wed, 4 Sep 2024 08:42:17 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="Ke1/djqb"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439230; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ylbiZHnndQyBxRrEPbsYoqCd5Ue19pmhRhZr0okk+70=; b=N+SM8ZZrC00b9qrzUVOn+3K3Oi5G34K07sklmT6UR/Kd3m4z3VNVlEaFK0e65A63HzQM1l WKQ78dR9ExU+2vIKVtRjlB1Q0YixSu0Xupl/9JRLUuLs3OEeWZYqQwhhPE/+9fnqtkSaiN bH9RJ+JeFbjPGTjB1hh1KXCRDJxw6qA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439230; a=rsa-sha256; cv=none; b=E8K0e7iCORrMCHlskQkdRIeinXgLF7kAJVNbE9Etlsy3YqwNbiXvEw9rzRjCmT5DaMpu4Z cjoiGOCsN7upG61SCB0FG5CUYOfhRErmcC7nLdK6EV+jtKgKmLGhz6rFif164uFWRld4MN 7l3NaVo9ZRHMUqAepUVkf2dTH9gRg9E= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="Ke1/djqb"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2053525bd90so32717365ad.0 for ; Wed, 04 Sep 2024 01:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439336; x=1726044136; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ylbiZHnndQyBxRrEPbsYoqCd5Ue19pmhRhZr0okk+70=; b=Ke1/djqbHbl11bwC/WM51pOHXtgtMTcIBMNFMLa9tF2eFgMkWnWbYBmocy835A14kb ZEQSCHd8sq0qaPHehzMZozxz7qVpdKi3jAoA9qUghBpV5rBp0oiLqYv5lVUmi2V2Z/1k CEy3iHkwK+T2CKImFzZmAnckxzFZ+Jid24IwJ396iz8yVj/4lZ2J+oLltV2uhV+sZSiY PicjM35zGdalIr8TyqNl93gUpWoOnnKD5aK54rb89g3JhiAI/7KNn1C17tvhpD0cFXNZ mGaXj21wdSMpAMJiy39SwDfnWBLJtfID/fxRrHf//v1VPGMOTLcL0vg3Lm1IhfxZrH4u GQRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439336; x=1726044136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ylbiZHnndQyBxRrEPbsYoqCd5Ue19pmhRhZr0okk+70=; b=XPW9mTAs6A+9Z7o5qONmP4uMIT6+To2oVKOpqB6X6Gyn3zxzjEwjgBDXf0KsSm/cQZ LYD9EKbdWCxNxnMwv4tf42OfJZJ/dIcde5IfnyaAP/JPQey8bGqaxe4AJrbdLJ3VufEP X7LsH/rsP41sIJBuOUUmGRnDYmeYtZQ2WXtol0i3FiFJFDeZwn/MsIEtqQX1rCO7yzgU qwqVTPvOca0vTB5CJI7GTlDRIXeFGO3JbZuOLRnS1rrDR0nCsOZcO+6D5RoInk9w4f0t 1UOVPnroRntzH5YWjbGmFUa3WzmuqUQchSRzv3b8WBlZrIXDkIZJvfS+ZDNyvjMvMy8R Zs7w== X-Forwarded-Encrypted: i=1; AJvYcCXeQL8S6RHVgXi3xcV/JcSyFf9devi0ByXS13Y5bLrkXrQ3uGUJ4z/gxDiNw87YHM/nhcLzNvNafw==@kvack.org X-Gm-Message-State: AOJu0YwQ50EabcaL53iO87x8xsDECTHfMVjwgGj0AMR3MqdGlmQygyFC EAB8EHWHe0XRkxb6Pci1K5vT2ADmTPdtEpoy8Rcc8D5scDI0xVCYVFYqJLHEvms= X-Google-Smtp-Source: AGHT+IGRRbNTDVHBE5FEDByUohEYiQ4MupDiF5X1fYOKyMaX8J1f74Xy7lHxJ4SbBUityp3jftUwsg== X-Received: by 2002:a17:902:e848:b0:206:c43f:7896 with SMTP id d9443c01a7336-206c43f8dd5mr2410565ad.21.1725439336115; Wed, 04 Sep 2024 01:42:16 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.42.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:42:15 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 14/14] mm: khugepaged: retract_page_tables() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:22 +0800 Message-Id: <20240904084022.32728-15-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 651A6A0002 X-Stat-Signature: on1wrjo8fia9atg4ay3u843a9pmy31fg X-Rspam-User: X-HE-Tag: 1725439337-817230 X-HE-Meta: U2FsdGVkX1+CgAUTFQhqpk9eAwJiBzGbjFYwia0OX0bKMo2CeOsfZoupIb1EV2dZ0Yy8WZ9T8ZjEsNMdMhAQ1JXFFzMpm9TCtt40Bf+NVWCk7RIwgG7rsX//e0qR1TeLDcAQQn5jm0UyRStjMVMxYRb858Erg9nK+TtuZpXerKU1mE3JTdPdAIOk+3OyRW2PB9hocPmGF8eQriHiHgzP86wD5oaXdsxMZybyn1HD4HjidQFjFgCxw6sAe74uSzBPj0AYch4zVptKfdSTfRnWSQ3TdDl6oioGkP6r/AdQaMVSL1cXnfVoxK/XvpurYGURPbp+Q7S/FJbe+Cjxj0xWC5C1DjvCGiCLI6mwtoF7dQNnkN7/5a2xaxLmCvkt5Qo09h9v/pLCe1EPqp7NhEm6+MdR3d8twykS9/ig0wF5HHqrI5YkEg8rPSSZE56lAq3is3GinqLL+470LeTxtFlAe7guZ3p2O+oOgOmWmbxbZuGlD3prn1bIY3950iMr0gbA/BHo/gLv88iw2xPK0aVD5jF7z0eFpSb0ZencEo26eFShRQLVfhgPMsL3VBdEk9qV+zA0AzSpv+9+a1JQR5WtrG6NTSzhrLKCiyx+Cv8tEN1jWWE/9Z9fkCSV2pTSjLjFpEdny79O1LVGb/6hZfmO2aGUX3x0gXYlZ1iqiQm05UnGqQiZI5gvSDHjfJbeeRh1//7nIVWwOrdwSY+WG3fdZTEWzS4SnB574OTUdwhRIMCzhYqpJRK9VAaS2+I1ohdDGAYCYLP90geJidOi8Q3CG5wv3OV14m1onKkuN50u/oe/DoZkMhDCx3X375qgq88HWzjjuiGj/LG/CVVOSjwbOE4PvQAEpiGPE5iQ9mwi8l6hnLKHmymnG8NXTVUZlu0RfkD+QIWuOqQ15vCN02yujF18JUpSCOaacGWWwUTW7nDT/HyPfKLqA2byi7tstHqk/2bnASiRoRTJmtK90oS 9u1cgFtA oZo9vVOxqDGtWhvaNjPSxrRLGLVdMSi5GsAS1SmVJZ4TTMmclpuMnkC/uaQJufSP0q3C+6kEN3LANxxbsmYyCwL/qskLW/pi9NvgmD6Ht6IfyMgR3IRM0oMMXoUP3XL73ATgP1Ode5sCk48+l+rlvkJL/Qc7TIT/XOpUc62ER+4wTgOZE0P7Ji+vkfM750XbOe/pp/Mg7x01wxxa8t+cyADqasTgJZjqCI6g/Sdqo9SLoA1SWyA6F8fcNdgteq2WtKBFA7mzX4tK1Y8RqlwrF14T5GbN+i+gEMezaIt3SFixT4K9idZmX2TFpFxBmPn4Ogkj3DI/TZnhcDaxc7T4zKijmCeB902dFUyQd1LDdSEzzv3jxkZjEXXbekTe7/VWDMId8RUneZHsj/md5Bq0l1UaN1rXlrz/1JYwcrLf2GSfMSQWicjoaqI12OA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In retract_page_tables(), we may modify the pmd entry after acquiring the pml and ptl, so we should also check whether the pmd entry is stable. Using pte_offset_map_rw_nolock() + pmd_same() to do it. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a117d35f33aee..318cc3eefb040 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1724,6 +1724,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) spinlock_t *pml; spinlock_t *ptl; bool skipped_uffd = false; + pte_t *pte; /* * Check vma->anon_vma to exclude MAP_PRIVATE mappings that @@ -1759,11 +1760,25 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) addr, addr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); + pte = pte_offset_map_rw_nolock(mm, pmd, addr, &pgt_pmd, &ptl); + if (!pte) { + mmu_notifier_invalidate_range_end(&range); + continue; + } + pml = pmd_lock(mm, pmd); - ptl = pte_lockptr(mm, pmd); if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + if (ptl != pml) + spin_unlock(pml); + mmu_notifier_invalidate_range_end(&range); + continue; + } + pte_unmap(pte); + /* * Huge page lock is still held, so normally the page table * must remain empty; and we have already skipped anon_vma