From patchwork Thu Sep 26 06:46:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8235FCCF9E9 for ; Thu, 26 Sep 2024 06:47:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 17D7A6B009A; Thu, 26 Sep 2024 02:47:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12B476B009C; Thu, 26 Sep 2024 02:47:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0FB76B009D; Thu, 26 Sep 2024 02:47:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CBE7A6B009A for ; Thu, 26 Sep 2024 02:47:00 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 550B3ACD51 for ; Thu, 26 Sep 2024 06:47:00 +0000 (UTC) X-FDA: 82605957000.07.CBB0822 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf21.hostedemail.com (Postfix) with ESMTP id 7D7021C0013 for ; Thu, 26 Sep 2024 06:46:58 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="j7O/5Rzl"; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; b=3f+wqMcxrOu9+3jN6XtYnvU+qUd0+KAV1DzN4ixDAi2HMZq2JySuH4xZY8yw8TLIHPuhsW fuboN0p5/objhd3B6QbW5x6jsQ33levw1xWBGglyFqaHCraJCICiX5uAn+Snyv+a9noG9V C28EHSvTkpLgSYA1J5b7xtnjNB5elSM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="j7O/5Rzl"; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333058; a=rsa-sha256; cv=none; b=6dpRMP8URJTKs6oBemCGID0TmsYf9PRJi+pyntYyVscxfcZPJySk1eYJv82hjHElVGLH9W e7dcy/DhHLOpI6zvtXxkVgYOIxqdBIyyZfSJ32FmBV2X1mfjklN1xCkGxB3vcuHjDL9Nk6 TmWZGJkB3a0d5NJ/eyVVZSHcA+OFT/4= Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-7a9ac0092d9so66625785a.1 for ; Wed, 25 Sep 2024 23:46:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333216; x=1727938016; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; b=j7O/5RzluhXq+8xCf7/ucGPRUf1QevnOVBjLKqyIHsrJD3C1a161ykDrXedjt0ZycD dl18G4Ty+fYGpB6aVqnDIH5G+5hCtrkz8FZvjZYbkRt8wro6s/VIj6GqWIWddwe8etYl corYh2xrELjLK91KprnN+tvPMrmDs7cMZfvsR/fOzafvORfDZKjs0wZVZ2tR7Rm8A9hM PSGyhPLP91a/bD1ejtDRg9uBV8IptfPrQ+eh3ey54Zd8vtNqiAkrSBZyNqrdKjXS1abu ajhWqYfAGmkRQUCiGvroubJWa8+AeENTLu2KTWj1Q30XSa/fUPPuK64qTniXvXbT2Vhd SXQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333216; x=1727938016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; b=IEFfYBpw799/sgYmshoi0HqT2DCHEo/Sp9wKVzpG0ZSR5XBRIVDBMP2j2PfJ3x9A0b Pl7rZ2vthe1RhPK1a4zlZQP0arPSbLSzbrCxXKQjzQc+bnjJ6d+vB9vHF771+RqErwoP 1pg54N9WK2j9wolMjgKJj8mPhVILy7Fw4+rxRcI5VmS/FbQGsMzCIv6eYp+180u1mL2b Z/8RPK7Q9xCORn7+387slU2cS2BNK4f62hB7c1GOasML2rElGGrJEH9x4nE0WCM9CCkd g1+0J2KdVeOjRTysFTXjrvpRchD8tzvZ+9t1p9Ht7KdEPF6EtfgUt6fO2y7obbWalLfz 9r2Q== X-Forwarded-Encrypted: i=1; AJvYcCVncFDCAgXj6vPig6b5f+1bsC1TnEsx4iGXlFaEFzOAplUi5QXcVcq4Lq4RS9+zCrMfT6ze7XsVSA==@kvack.org X-Gm-Message-State: AOJu0YyS0JVpthSyrspSJmLGdrZf58jc7ofCYqQvJ5xOyTD/3y3usf0r MSB+LtvumrGHB6vbi4QkTR8cHpXzXUvRbfb2qxKxdKZHhZsP0uw/GzuTFVmN0Go= X-Google-Smtp-Source: AGHT+IFdBHDg/pcI8fYkiDBKrKz5/bcsNyjF/Vh4ia+kT/ICQCu69/2acL+Eh3ZQbIhYSZdsv978Jw== X-Received: by 2002:a05:622a:20b:b0:458:4b6b:ff16 with SMTP id d75a77b69052e-45b5def4888mr86143491cf.32.1727333215740; Wed, 25 Sep 2024 23:46:55 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.46.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:46:55 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 01/13] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Thu, 26 Sep 2024 14:46:14 +0800 Message-Id: <5aeecfa131600a454b1f3a038a1a54282ca3b856.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 7D7021C0013 X-Stat-Signature: dshucc81n61dpzw1h67mf4fa75pcufoa X-Rspam-User: X-HE-Tag: 1727333218-72159 X-HE-Meta: U2FsdGVkX19se0hqBMvl5MxO6/Zlvy/qM26IIgtw/9gR1ztMPM0r5eq16dVHsi6QbpnWTtEvz1mUHSJfe9NvnRS+CgXfmepLfxvTFCZMunvG1iGARwBSK8mdArrgqbo8NM5E8z2SkPxGWp5jJ4fJt6EOLsaCtiyUNSf7RtJ2/tNdF2FHUnlgNS2+JdzY2hu4ANWX3cpXccbK5dq68pSy/6O7mB3onDJ+Za/NG+KJMaalcUUvrPYSAxzecks07uLaYTJNmDetL80ySLE7Fr2GX8HjHQQx9Z2J+H+MFFarQQW2li1cQu3jIOqY7H5jcrcwDSHwXf+bWbqCRrOoXckqR9IABxGmRPqp3j5m+y+xNRrrCkpAPrNYFIHjadeZ8LIsgYVWgqcyabuCHB7lPaMT1u4N0z/VIpdOdWCwS26FWtkslCsTDMj9QHTlshehnDckB3AaZhfbS5Prl9kEOZluXjpDWBIzBvSxjaGpqZrcKPmIf/L0LADGDwDYZS6C4nm9fME+diUxRLbevIsO1g/Ffr6uDX3QCMNms5vmxLxXUcMTdRZ3E6nKQZgmI2ReIU5no9P1gBVek7bOR8HtZWRbYfGlmoW1YIR3ZS7f/JoZcFbKpgiqOkPOXOXaRMhHlweVFE9hX/+p5TOcZHdpmaIbeaJxCwUAwRyPFhdESmhyrtBq91eFpv5HriEcVaoueYH5IOJaIZgNIMDhqmf1Wj+Jexpz4Vlf1FOW/QeFSoxCdqu9deA0VOvQ7LGDx4PPtGQHfinuMMUlQa/4a3WgH5hZcA4+BAZiY5xWhVLEkt/Qce4y77ZMVQOqj5OLqCQ0Hd8gjZxuVuQgH+j/vSvKxdZSyYAQ9qhQHUgtHlef4IIHsU2Rmh4tkEssa3Zjf14NxBgsXe96OEqx/4A4bBGhw99agD3AZoRzc+oSM1q/kFgvPRIQWsDRr4pzVkzuI6qGLmXxaM445BYxM2s9GDDCNoF wdk2p6u6 7hV0Zn/9qIqmJA2XGtYp05VA2E0AKWSFo64vLR086Yj6C0VRE2NJ1uwnYuDONhUQBP+3dLa/2fTKk0JMF5l/K0ha1QNyoTz6OHb5FcRiwzml1mPKySjhgGqSylVWqDDiZPGNVtAeNX7aO5b+/hDqdQYVO2c2j9r+If1LAOh62nZ0JZWuqTClp0mY9Ws7P2dWLhbwX/KLBwZ8bxmje4ZGVwg0ps7pUqFkXe3aWWNXKbp30FACFzT/Vq+iZGJAxj4BcSvfH87IOVJljyo5RSOiDzQp+QROzhxUHsorhuM9m3CwJ7rVrZsqK3a6dwWdprG4bpR9q5xBgtC8S0Kt9s6cVQRBFxDVB28tGTMvjJ0lgOlhU2Jwfzr/HjFaG6v7UnpR+3LM6BEk3uedIZhCujO9UdWMWgK4NHuhpatUyVTJY6L2MLh3700vBu9IPgVpTbkc/BF2nTrpe8msr72c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. It is applicable for may-write cases where any modification operations to the page table may happen after the corresponding spinlock is held afterwards. But the users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will be read-only/read-write protected. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 7 ++++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 48 ++++++++++++++++++++++ 3 files changed, 60 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index e9077ab169723..46828b9a74f2c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3017,6 +3017,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..daa08b91ab6b2 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,31 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pte_t *pte; + + VM_WARN_ON_ONCE(!pmdvalp); + pte = __pte_offset_map(pmd, addr, pmdvalp); + if (likely(pte)) + *ptlp = pte_lockptr(mm, pmdvalp); + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +381,29 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_ro_nolock() provides the correct spinlock + * pointer for the page table that it returns. Even after grabbing the spinlock, + * we might be looking either at a page table that is still mapped or one that + * was unmapped and is about to get freed. But for R/O access this is sufficient. + * So it is only applicable for read-only cases where any modification operations + * to the page table are not allowed even if the corresponding spinlock is held + * afterwards. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the pdmval. + * It is applicable for may-write cases where any modification operations to the + * page table may happen after the corresponding spinlock is held afterwards. + * But the users should make sure the page table is stable like checking pte_same() + * or checking pmd_same() by using the output pmdval before performing the write + * operations. + * + * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will + * be read-only/read-write protected. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Thu Sep 26 06:46:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBDADCCF9E9 for ; Thu, 26 Sep 2024 06:47:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 329726B009D; Thu, 26 Sep 2024 02:47:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BCE96B009E; Thu, 26 Sep 2024 02:47:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 152626B009F; Thu, 26 Sep 2024 02:47:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E9FAB6B009D for ; Thu, 26 Sep 2024 02:47:06 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 69787ACD4F for ; Thu, 26 Sep 2024 06:47:06 +0000 (UTC) X-FDA: 82605957252.16.E667167 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf10.hostedemail.com (Postfix) with ESMTP id A20D8C000A for ; Thu, 26 Sep 2024 06:47:04 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cJlrRvn+; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333103; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=8f/blPnicl1DWwkqvielr9zVAu6Rj/7S669FwmcobI39JJIT13LGFwHgnfWi0lodCfTWt9 kglNMdRzabunJdDoBz7hIBq5J2rdEL6FoDAPlQui9QWQ+hUgo6WiFRbthzi59s6rwBdBQ6 kiYprvYnaZfHa7ckDZ3smzafWQghLnw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333103; a=rsa-sha256; cv=none; b=CXF8/ugxhSrP+U6HQZS259+clVcQ6c0d7DsC9uAAb2nxkwRMZuewyFrXtgFWwsyoJrxFcS SWlXFopoLz5IWtB+koLru2sLbStswniwkH9XMKWmlXHtnHlxTHN0i1x2o2jkOK4oUrn54P EoV0gKUU/kjVyA6r1ux1RkdXqVpSYCE= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cJlrRvn+; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-4583068795eso4781511cf.1 for ; Wed, 25 Sep 2024 23:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333223; x=1727938023; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=cJlrRvn+RlYYQeCABlWexNNqEVlOV/w8CtznFU2dtkS6BNpNH4jwI4bPecITqMNvLP ptKbVDvHmTDyLheRi9ucgm3XVzwQpq9etjcqXbtoVxBpByH+IC79KlrNoKKzf57mmhS0 03aci9MCTPlEcbEKzlVoOI2rmw23/Frv56qeELe3/BRRx0J/N4pMbpWqd0c91h6z53gY AIGLrqU+vCtYo32i4TUtuWtP87HyxYqd8VMBNumedP08aomnhuDUR0eCjEfqHF8klPMM pJI+N/FFSjjMugVllogKCbrZ1Ogf+jyzdrDMjaSHrpak90SPh8q6jORkGMQyPYSUPs3i fIjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333223; x=1727938023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=bK6wsqLu4LvSMMv26IPoSO/4BhhCJXpGfdp8FGjJCilzXH1MnYYTFwMqIsYlIzikFd c1zZ8xdeAJeyjcntJ+mPW6IPdjdpLb7WaEgOPl0sJRJBvsF9Vr+b9grarxoVpmmtGPbG XKDUxmnZvbYuLUgJ/Yw60V8ILoC+aaLpnjADcQmPD7r/e7Yy4QVR3/+rHaM2wAzk0+k7 jLe5e7AYPFA5WuHzXCdJ4suGkxUac7gzhKvFdFYnmvV2wVzMtvvfYDzL//lUDzIdF/Jf MFO1993mTCLFip7E6O0rXm0+V7aY4VskexGuAXeapOD4gxUJs1k5MeGPJbmyO4/Aexif yRnQ== X-Forwarded-Encrypted: i=1; AJvYcCVNs9eAheYV4sxYBA/rIPhDZcPfQMLUrwEaczhGw/W9S71OQdZy/rNH3/785vFuvi/HgIhvYbgrMQ==@kvack.org X-Gm-Message-State: AOJu0YzQd8+RAcpYDp/7F6WwmvfUEVCwq5z7/CKZOA6U5mJvGWs64hOx 0q83YsPhrjUChPwPHDlZY3xFoSmDic9J72b+L5CBF1iCgkljIKS0mvJ4ZrKkx/A= X-Google-Smtp-Source: AGHT+IHKcdRFC9q2GD6595fUPe4NgxH7f7bGHM/1jVFUOYjtanMZU2hJPsIx01XdKDMweQg9NQ43+w== X-Received: by 2002:a05:622a:105:b0:458:4c86:4564 with SMTP id d75a77b69052e-45b5e0952b0mr82475131cf.37.1727333223424; Wed, 25 Sep 2024 23:47:03 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:03 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 02/13] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:15 +0800 Message-Id: <42559e042eb6fc3129a40f710d671712030646b4.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: qsm4uqz1d9pw9r171mujufqjj481bjso X-Rspamd-Queue-Id: A20D8C000A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1727333224-524939 X-HE-Meta: U2FsdGVkX1+e0rJZ7icgAeN006bMEUx66l/g6NKlXEpOHdyPg8kPGqlHNRM7oDaRrjBV6XIethmmM9+KPsTHs0r0fHOLlEYn/kjs1Z9jmxF5HxGyfvXkLyXwXc/4Y0g7FdecreJsC0suxETy1WW/cdpjzy591sdB9praBJTbu8g6Wfo7KcBg7Ah8ZZQB1oc9L2QYqWK2Ds9evqRUfrPywlXr/yEmYY5MhWSH47WKXOQPJE9Nyw5U7GgFXIBwlNrWmLJerfklV9UuYCni8aXcGll/KW6q+CXTF14agrNv44eY0aYutb9XFsyKvIl2KCX+YSkVFC7EkPT/TFSqE7WzEr1d9zL1QlPdeaRw8IsVm30vNZnF4NDWRNwFPor/TYouE1kwgXpnO18BvJMmwLh4K0dyLd6rrCzf1OU7wazREoF+oWUQut647yPcWsEwEn6ccSwPiCHK/gl20shfNi/C14UlUXoj97GoTltWrpZ1AfZLK7fG1fIlaOVQovMgPwcb7DkKnZ4VymEkQ1Hhysvr/ksvkTWykDDRd9c2DRDoCPIxqU4TDe3bB+MlPGWsiLD3Wu5Mtwow8Gm+2v/WAt9qnHz30T5f4TOUO0kaW4rYXibpk2zxRH9mTbuC/0w73U5DyEip5mxokPHPEktzmVFsWKcL04hB2CK6R1KratSglak2TrvhKIlu/3+WjcKt7aUDYtPU/4xSS5FQunKw6zhu6KIFc8ypfSQVGj8qRW5y9HX8aBmfzPmbgsmLBh0FQXlmhK9XianskcOv/0D2c/k5q4tgEpGFg9vSJzKMe6xARIZu6vb/W9IdLz5ozdgYayeSwyJdn4zLU1Fp7TlECqkNpbpyxYg1OIvam10nwL1Kpq0XQmVfUmUASfm1QY5RQArJLCLE4/l+a3+ml7PWoMHIet4tdidfxZ4sTsCRHamoyzYOwKevefTjDviConV8SBZg1hypahvILJ228VFB63q GTqYiNuI 8UNqS9E26UJf/Z4qLjGV7fR9KwfT72nUzDjVsi19zWmqatsPdsmn+RIK8u0mQyzuDpwMXxPdtjWHakfiL9t0VRKJQm9LZcqlhSURSnfl/98TTJRKA5SDHVVarYyCibs+jdkCKWdA28qFBMRlDg698uk5xrLzGG+2i9hILA5QxsF9ljxr6bYf5EY3ac7AqsXpeZLuoDGQ6fI0aTJNdF+oVrzsuTPcQZT9ANrEZt65Ps6B8lOKRKMz++kHJn2cgekwyjJBt8Uby08D9Ot5J3JI/K5gRD9gp3Z4s7dadbGz/+fptwoZfieXR6ts64nt6yb/OXrxTpYDSOLqgB+TT1IXgrc57xPdmKV+KZGJMq+FS4TwN80Aw1n74qFlwHVwYfhIn5GEeKARo4Nr+2nhxV7hVor+BulyW7Bek2D+qOpAYWp1zGT19AYSmXWFj/Tq5N02sUSpVqn1PZBfHK0U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Thu Sep 26 06:46:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A66E5CCFA07 for ; Thu, 26 Sep 2024 06:47:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C5566B009F; Thu, 26 Sep 2024 02:47:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 374DF6B00A0; Thu, 26 Sep 2024 02:47:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23C886B00A2; Thu, 26 Sep 2024 02:47:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 064356B009F for ; Thu, 26 Sep 2024 02:47:14 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id ADED4C08A4 for ; Thu, 26 Sep 2024 06:47:13 +0000 (UTC) X-FDA: 82605957546.23.F7052F4 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf12.hostedemail.com (Postfix) with ESMTP id E3ADD40006 for ; Thu, 26 Sep 2024 06:47:11 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="gZyaG/5T"; spf=pass (imf12.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333111; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=nz2fzJYM5fa0oV6pplVSrDSDWLLUzUFqPVNngv/irNwUW2Jk/uD44gpD8q1fqEMudu87FS LwmoGWzYHMRefDK4cdJYPv+jJfVtsAU7kG7afQJT4icAriK6Bfr1BTZymPrlCcGy9pdFHx 8w4P0cRKMBEKLvHWLgyzav5AJQu26b8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333111; a=rsa-sha256; cv=none; b=yZ51/AcWEWc3h8XOZ6eP7yQYeSf3/V1IQFI+TN25ns2bX77edtwRygM5eNPiNuKb85isWu +6+zOQZP5hBfGT/VE9Socl8ObrmUYRlCLvA7FkiRS0UgIxmEAsiTXSeVjv+6k1Fbi1dwa8 cuskBGwjTqbLCn4r2/E6K2k5I+/Dsa4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="gZyaG/5T"; spf=pass (imf12.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-7a9ab8e5f96so44559885a.1 for ; Wed, 25 Sep 2024 23:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333231; x=1727938031; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=gZyaG/5TslyB/h8enNEvF89PJ4XEKlGgdUR10C6Pr1LEo0ZLbZuJA727qrVMpeH8fv KeVLnHKliUTfDAFDYnaYhzgHU7auVHcHwdpXr9qUl+4Z+PbWdglQrB6LxivRL9NqbJvv fO87hGr6mL1RcA1WiGuYohOuJ2ZbGwF/jiF9v7qdRpvDoNY3E83ZlkrMT8KY5vgFheTO lPl3FNC/B+VV8TENrae+A5IrDdTk+fSKuDwAeqqHDSLOvPZ+gJFareYB0zOyg8kR1bNL 2hqBORCYR4boZif15+bpJzyiB9LDoEdVkdH9g6dRGa0yaGwRxTRQ2ORYQN6LMMYZWdJF pakg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333231; x=1727938031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=WO50JeGnnpl5U0IYJI9BV3CAtnFyUOpmcnWZVKDMls2UzjC1ohZOt8a1qsGuGqPRa/ lhyb1hEA5NjZAD1MA8pNwD6/z+m5XOqpXNESCPDp462pmYhbEiDmaj+on38iKdl/oq9p GFUBJqL5KOn8zypBTfKsA7n5lMgnhU7L8AWtdKyMPSUnUD9MnplLPwugHLO2EnMlbuIG 3VbE5EPL1mBIOyGWKkmw/eSzXHc8LqCu5vF6Q79C7dNntzqV4H/UpPWVeERzqTyN2nRQ /CQHJgdhxvJ5x1tPN/wCjA1owyGIwNPczPRaHQbF4dTeaPcK3Q34cv7x9YSpVMkJemIy 63ZA== X-Forwarded-Encrypted: i=1; AJvYcCU3AoIWpfpTyWQyHAe9GxxXL3cUdSk7Az0I4oCPFELrx3PVvamrT4seOLhZJBsWL6el22n0IERkSg==@kvack.org X-Gm-Message-State: AOJu0YxwxyjSgvxu6JsLKLRsN3c21mP7IXx2xpuEdabTZC8O40qkt9gC +OIV2duOtwjeOFHSqy7K6qEllLOuDymzj2rB9CFzAiSdVRPVm+1k7zUpurjLS60= X-Google-Smtp-Source: AGHT+IF3jq8XAx4d8MvdG//ydygdKBfb8RG21eWCHGRQrOC65QRa0E/gKmvca15+75cy4x5ki2/uwA== X-Received: by 2002:ac8:5914:0:b0:458:fb8:9dc3 with SMTP id d75a77b69052e-45b5def1dadmr61649591cf.25.1727333230912; Wed, 25 Sep 2024 23:47:10 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:10 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 03/13] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:16 +0800 Message-Id: <9f7cbbaa772385ced1b8931b67a8b9d246c9b82d.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E3ADD40006 X-Stat-Signature: q8z8dkr4fbwicwdq39wooi9wq7s6mwq3 X-HE-Tag: 1727333231-707213 X-HE-Meta: U2FsdGVkX1+CY0+xKC09UFYsVazsP3K5pxx5/VUH2beP/zLllomeKfllbFTzsHhXr0jBoW4QvqnSBSxVMSrBfztaLKz547kdA8XrdBcDN7ct/sgsRvA5mlF53oWGoXj83tshZLRMHPcuYKY09G0hv8yItKd/vBi+KWXwZgwMh2WRF4VilMM9HC7liFOSHMO1qo63W+dZ1bEskqrLHUQhBDHeTC889cWFrhKh4leQJRpKmmAlRO0uvW4wZa15IBdc/9/KGqvfkux7R5TkZX+kIPrbr20seuUeJeTmGp3Lw8SUY34wFj7bcRkO9NVe5U3zKMbycyy9go1npYJgUToecllo4O84FBWM7NHIuJlQtd/W5TsZf/ropvt4bKvcKkS+6CNu82ZZQR0eCOfdSOW2dGpqWtj4rfYLoZ1fzCSYjXkzY+wbKkeZB/q5r9eq0aEhL7D+9cncv1mT7AyP1rfcT7SmmSkH9AYOS11fi6d9ALl6IDhOiSZQ8Wgv2jG6jwebTGumBqNsf36D/Ue9NXHpyxG1okc6Ok63dU5Nt3cDyX/gGQUKLD4VxuMLtQssma61oPv6Q4JFvExLuiZHkudWq2R/ijO3F7JVb65J7lyq3pdyp1wIOX7tlfQwXZYsCCdg3ZIbZ8XfUw3o0VPAJcNItiFIByPo5mcaaVP2KSigQdVEQmUzi2KWl2zgjH9H7FSY9mwfWRZRsbsUr1zoA8n46CrqoWtn+OVyDPwmtQd7Tfks6nMKgpLsbEC+vfuleCYPv7k7ety0jvIhBYKQsGYl4YjUHz2XsG9oNlA96xDQZnrZGttAotLj3lb4gcakAIRDbO6KIxwMzmIX5eb8p9Rl5ynIw6YJtkvHN+/7YxZJqpleZf6gzV9XDmuWXvCL+Qu2Cd/V8bL77JBad6FnJrvNRXF8Gm/xb1y7JwDIMsOiGNir7EqX3SwsJiIvW0qt1sSXVtq71rgbQqGmWtdHI/a z7Cu0JOh 0MO8gVrRJdpS16PPsktLKCdW9MmFwDr39NfHR5m/UwUc+nFGZHP8N9tdG0kqovHZr2ycpej4K+2tdaAAKuHMFO6WojZ0+3hZ08fa9cUq5Q4qhH3fhya4CCInHgqdRRoUQGSe8gY2i6VoqiHZsAJQt6onNNkY0E9J68zahm1Bk4NP2u++EnXPHd692lkRgb4qHTGDK6CjPZt2FnnUOA7vFAyvF7e2YWZIAG3X2tfHSKhtEOuYInSipKl2OmCKVpOHn1ySZREwZlF0Px+AwbOJyLag1707DTCacXv6YreKWnDVIJPSL44S0YHw+LIqiPOdc1jML0FCS+lreZPpxVoo0ylK9w10vo4lrhfoTUKgF/NWaVxn+fSd31Ewj/I8+zYnIeKGU0ht231UFdK5nDpCGnr0UY4FrBg3Khe3XThj7DFqq4D1Sh0ZKmD0+0zMYO/eq5wQLNBN9tUBrCuM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3e46ca45e13dc..6c6ff8722550f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3271,8 +3271,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Thu Sep 26 06:46:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4927BCCFA07 for ; Thu, 26 Sep 2024 06:47:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DAFE66B00A2; Thu, 26 Sep 2024 02:47:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D5FBA6B00A3; Thu, 26 Sep 2024 02:47:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C27E66B00A4; Thu, 26 Sep 2024 02:47:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A423E6B00A2 for ; Thu, 26 Sep 2024 02:47:21 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 66222ACC75 for ; Thu, 26 Sep 2024 06:47:21 +0000 (UTC) X-FDA: 82605957882.23.737E442 Received: from mail-vs1-f52.google.com (mail-vs1-f52.google.com [209.85.217.52]) by imf11.hostedemail.com (Postfix) with ESMTP id 9A57440006 for ; Thu, 26 Sep 2024 06:47:19 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="AbAcDGf/"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf11.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.217.52 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333150; a=rsa-sha256; cv=none; b=e9GVuGLHq2cjQTWHEDjevY3zb1ro0fqa56J4R6p72q7uf6oWQUHekhhCBEmplxEzS3IXfA 3ag+NEQ7K1hLOccUQY/UVlj81RLv5S/ox9j10XWtdS327x4SHSHxKS/c+3qVoFbekJAxL+ ppRoLWkzuWprukF8efPhQ/inspWibAM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="AbAcDGf/"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf11.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.217.52 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333150; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=nsDW413I2TjzHPmDod75TPpB0ur2CDf9l9QaFP+mqw93x23QfUmg9t+Bmqncvm8bWL6HLy 0vCFeMs/jrhYDfb+nHgC+8XCGqm9CN+Q6ElXqyZW2uIhKkbs97W9gMYDd+yY9SZ22F4VmF OxHGgbvKI9nZjns0YJfMYqWbW3leVFM= Received: by mail-vs1-f52.google.com with SMTP id ada2fe7eead31-49bd3bf3b4cso324011137.3 for ; Wed, 25 Sep 2024 23:47:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333239; x=1727938039; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=AbAcDGf/KQdpRRkZIvFV8+DyBw3o8pYXau4BSXfsg+PcFQniLJQR4WjDCw5WuN8ZzR 2e9GRKOeDQqu7B+1CepXTLsLSUwGXlyVSaXTZsOBvd9jNgZIvqSOcbzzJjIy39UYeipF bPRcoFBdfFtZzbyKHzI3io0p5mZN2JSbMiiPILmMkNGqSI6QRKoqeIkcz3EF3u3VSDW1 /SHMnGAk9XrCT4teSs0Lb/c1nAxViCKCIZWPjA7ebIqKjSZLZzYqpBNAvhqeXDTJfRbo WXcTtOHMAaU7sX9y/MLbxKMWLvI6pZA77vLsfnYt7TMs92xZFyPm2rcI6XcX5SddyHdR noTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333239; x=1727938039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=qSqyNSdjfo0xdzvsZbcXkuD05w4E3CnXSDWFD5Pld9+y38uDZEEOCDtDKOYy6re2oN 0Gj7Smbz0NUN0dWwvWCbeZSjecrloJhHTVEusLxB23mwFC/CvvoJd5znKUIe0rtTRFcC qUVSnZWweQknyfksWSk3ciYQaVnbmtJ0FGfjv9OqpNobz2haF3no3SKl8Znpm4w3skfo p6Thl3O/O1uglzDiDNVEQalN1kiBa/P0iHXnOoeKezkeRdmOrfgHc266Ef1HzDwT95mK tZDr0ntRQksOj3noLKrMZp+MipRJ7IhC6z6K7KBhHj8yaVD+XlFNT37j+loTzi0QGM80 6cYg== X-Forwarded-Encrypted: i=1; AJvYcCWYs9p0FFEYk4qkgpgYuQmx4eEfnJ5amWgD3IKNFbdiVWra0QzDq9M/Efcg0P86fNeJqdIk1rD3Kw==@kvack.org X-Gm-Message-State: AOJu0Yw+tCrvqlv7259W6RCQEF/jVlAHyphLwFJ3ODH66Af9Sr/qAOVw xiHqeeepL+NqSOFBoEi0orWSZYKWXY3tlmQiPF9BmQa206sfXqG0PL3J6xmHI7o= X-Google-Smtp-Source: AGHT+IHnBlRBHwKCxjYpHBiGYKwBIoiWGnfAohABSGODP2aU4XU9V+T3qIbiD3kwEf9HDK6sMKBjlg== X-Received: by 2002:a05:6102:6c3:b0:4a0:1cdb:3eb4 with SMTP id ada2fe7eead31-4a15dd1a11amr7011565137.20.1727333238630; Wed, 25 Sep 2024 23:47:18 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:18 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 04/13] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:17 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 9A57440006 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: omhu3dggai5bg8tedbx4edwcc5rtxntm X-HE-Tag: 1727333239-518054 X-HE-Meta: U2FsdGVkX18PBTlgmOd30juuYADWyHxSdjzEIKvBdeNaLcpsy3Y6azv3oVVKCnW2ba5UiW3cPMg/+1/+AN5e/ItD5WJ5vHiVn3M+Bi905Og76hkmKNVfnzbKnasmXyA2EVSB7vr/FV1k6u2IMEoIuk4hrqX8B1hFWPFoXlXuo3v1re4I3Lq6J6a/ghN7kbWEWctAg62Ft7W9gkchWrTSe5DdRokaW1jV2/j1k1m21pwRz4CjF6Rna4l19KxU8MrJflE/BaJFEy/NvhUY9FRZi+nqg8i9t0n92D9AnE52BK5wUBEGRGPgHcMG/m17ouWLNiaz34UbTYzqnN6lGvY+9Xf9KS/mz8f4Gw+WstGnTGlfC8BYD21Om8u5E7/9yG2U+RUgpo+nd+TEUnTDsCBPt8gO1xq6heqkib0+e8nfH0LTVN5XPx8KU+QBUbndgBQe6suD3t3vcchCfXkZ7fw5/2oj5dx4mK/rTCKc/HYPocbUNtdS7+uZhWqv6wE0cwrIsdEaB/NYG0deUv8IdUCNs55Qo1a/NSZ0GEZVN6Oa618Zq81niOVPOgD5dIZO4PmspQXjmg+Lz0QA8IPLwU8DKp4mH1cGOVNY+b34/RFUfJcuTj2VLylR7LRu6Is+uWMMEHs9+4/HL3KvZXtE5sOV7T6su4sZ710wvAfxqBcJAOkANzTkU6tJ+rH4DsGaKARSnzubWlcf4TR+MfJniTaKgcG9G+4tUqWzoDZ7aqA38VkjqUmOnRNVgQVVRAbRvN7LX6C/AYrKEl1H1fj/1iIUpC2sgwGS/4qPtf+7X7lAcGHXZPsQUCCKla2oW7DUtLiKnqwqIBgLc3IwD/dcsGSmU72NHjESEfy6OT/64ONXSVuYxQIdyGDQzVck5RBq5GvgljwYgpNzVhGZyWXlewWTwaM71Y1oX61erT6hLAwE1HFcYYobpeCayHhv7EMoFNq44CB1uY9kaQ0r7ufNlmz HjVVzm+Q FnvhJ3RJydfcYNAv59Lu2hOcyMVEpUWFhGxpZvNq0R0TvME5ZsWAFs81HYowtbYXYErUB1vZ+UFjqrSenQP8V2Gpf0/Al9kmg7EfOhqIJgkAdwqjasOgqPvUZ8dejUXJkbuuG/MQoUrWFj+zgxH7w+UGx4l18uS9tzT85xujUd9KggPObIDJfbPj0YFn0sl2DHtAEo2yXrWghu1bAzz0UYyqTJSZUWS3igPPz4gp+AvkYyY8wr77J0/AZxCB1nOorrbNtb1xc5o6tU08xAmdTkPH/wpDeC0NMLbZuqOAP3X5TC7narc26/fxzbGVZJhvhSk1uLb8npyMce3T4SL0ph0Sow3Z7QyhSrbBGeVQ+2wMylJLecUAJqQ2WWWRIXW6BrKsT59TWAEa5SYRYl82TdxaJFrNE6kx1salEfulwOdyFBRe7UqKq1cY1QazAEJNyLq3ByRUkSspq0LE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff6..6498721d4783a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Thu Sep 26 06:46:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B276CCFA07 for ; Thu, 26 Sep 2024 06:47:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C4CA6B00A4; Thu, 26 Sep 2024 02:47:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 375A36B00A5; Thu, 26 Sep 2024 02:47:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23E616B00A6; Thu, 26 Sep 2024 02:47:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 06F186B00A4 for ; Thu, 26 Sep 2024 02:47:29 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A8ACE1A1530 for ; Thu, 26 Sep 2024 06:47:28 +0000 (UTC) X-FDA: 82605958176.06.1C595B7 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf29.hostedemail.com (Postfix) with ESMTP id DE040120003 for ; Thu, 26 Sep 2024 06:47:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=izWquh72; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; b=yZCSM1RKIdAEAirkb4/RI1U7miXr14nOHzmXppF3fPW8EB031dhnKAQegZeELhmrFNqIfg 63Foq7ORa2aanZzDFf4AiYaZpez/uXi/IjAym/oaOc9eKZgGyQekD8s650oIc+XK6Xuzhu Pd3kdrBvQTJ1vaPSOPDZRSYzFSmATEk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333126; a=rsa-sha256; cv=none; b=kBPvQ0jxyjrtcqLKdyWLarFO/QX679hlQOiQoByfV3LkCXiOZ56B/ArPK4+KYeeCpLWOaO zJ1vFPBG9IWbkDXMJHWDax6qT90uJA3uzHHylVWtJ3CRdfKUMFTR+lyEB1sFvCneWnaztu fK2VyMZinQjJ+coY2nRHjYPwlapSae4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=izWquh72; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-45832b2784bso4621241cf.3 for ; Wed, 25 Sep 2024 23:47:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333246; x=1727938046; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; b=izWquh72CM87Sz8nYSgXWo8OqJvCz7axyLXE7TC+SZIDfmHffEfLeXGkWobWPXWyXK HIhebODT+ivdmVkEMGUQAPXIsl1hEGH28ZaaooONvadCa1VF2j8wNxSrRvv1mo1MkkyZ tUAbBfvxdDmNw7qumWTAz5rQXm88dZpqG9YGShbn5P6DUtUzzFA5vLOsGupeE5bil7Mm SrMW1MUYgPZ/lqbRDhijGjW32Rn+Rr9w8a4MOeVI6hj81W/0YpWn9cFGmQf1FieUJSP3 bKFCWFPMrSqcO/UIhy5fQvpYt0aNKagmd3h5zuF5yvVyQ73gxO/xZEcR2P9b3NiDgWnR ei3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333246; x=1727938046; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; b=vzdAprJP6BtriJHkoQHHFBQFf/9D4aGLXjcH+wAdU3UcYlIA9ZeH8vOP4f8P3KnmJT vSFR9hILds9imeQRJZVMlK89EyxxFhhlvqbnSkubVVeKnMzi7t2OuNeYd2bfiGuiWVE7 H6Fo4L7akH5G6d69yzB0AwcLKmA+0MDkGhT0azPk9GdWV9E2qRRVUeL4wxBUK+2sJn93 Ct/CLO342xDxbeqsaQ87OMiecrFCxFKGHAWM+fyhCKWOTLGWdB4m3CVc01hI6Im0WWy/ OSfPd6JZCbe/j1EVJLTp2GWMRaNoMGdjSqFOLXCU5U2lzCoIapVNdnRjINwxy1v4mx5n jSgg== X-Forwarded-Encrypted: i=1; AJvYcCVl84Jp4VdjchZ9bYfdFRSwC9SrMmUc6sK0vVrFjixY2/jCm+N1gICg1vr7kwB/nuV5TpzAFjmALw==@kvack.org X-Gm-Message-State: AOJu0YyNrO3UopKE5QGLMgVSqk2o8kzR1p7EGhhAdMhlebQx9yDxY1Ge 4s668Sc0m6mf4JfNEfKONsQ9UlCUHI7HFxfe7G83X3Yy0gM9R9iifysRQczxATg= X-Google-Smtp-Source: AGHT+IGInGARMP6O0lOtN8I02J26ctBRW9WAU/Ifxw0RvFOYg2EOnEie2juwEDAOYLUuf+d4v6O7qQ== X-Received: by 2002:ac8:5885:0:b0:456:94fe:dcdf with SMTP id d75a77b69052e-45b5def4674mr66807991cf.35.1727333246109; Wed, 25 Sep 2024 23:47:26 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:25 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 05/13] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:18 +0800 Message-Id: <0eaf6b69aeb2fe35092a633fed12537efe645303.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DE040120003 X-Stat-Signature: o6j49spntptu1dwktp6rh59rrufsrgwk X-HE-Tag: 1727333246-979060 X-HE-Meta: U2FsdGVkX19QauVTH3ciQollVlXVT6cJyeML/Oxp4BEI8EfTKCgG9XEljI07eQl13NJskvVffWrtilYzbUWuROJCH1vLFhr5aXrmJyXdB/9VwF/X4PXqgJEUl+5eofK7LVSDKNSYsw4gGrJLicnrsf8G3DVgJg2gEwOVdRj6Xiv5B5+PXRkxqhFfYgWZjd6x8c25Hyn0v/tphXp2eawN5nwEjnfOZUYScnRfb86BpoCQ7CjRRdfeIy292Wl8jP2DdtCxOEWSSFrFHtJmRFa3Y1mGt4mEN00p83ATJqiiFULh/wp0AadSwRjBt0GdpEg9U8wapH160jRh2fPYeO8sJPsi2OsTCe+kRCnumND/gB93/kaBpgQUA5rY59IhTNuCrO+CUyT2rb3Y4D9WwQIqVsJ23B+Ja0tsCfdeUnxhpJmoLSmSUFfPWn2mX8o7urVvF3AnU8KzrO7yUSbaLPEjXeUBD0gpEARonhrSJtGkUUYohjlXRPDeqVWPwlommbLlaAUWQ4nJ23yNBRqH01mbRnWuxenvA4rtPwo9FtGAHOVFE7dHkK9V6ND5Y499MvmfHSBvUZUu5q1w/S/xNikG5uur24Mvknn38qiWMtom3p5tafetC9m8UD4KoU5/8qDzseUJMQv9znnQblxhpjQjBbEvPR/zNonSwiOmQE8h96Zy4NG/B2ZPPvwm6tDezmOLmfUg975/hJ+bCE8tcqjFOP41lilvtB9uVqDcOlx1H3sGBKzSrAr7HwZ/TEsOewwVfmqC3oP5pn62IM7OkBXD2XJMUeJIuTmhujrdD9/TyWckEgUIdN8zpHcHFITvZNjW1eP+MXOTYOqwM2tRoYJgLSRZCDeKgHSOoihsr5xOomslNXkXvCzXMwb01izzhXHrfiVDWlhMgowMcluVQ3Jw2Eo2P0cZdNvjUnwaV8ifbXO9g2dtDT8Jl1haW8NEUVcIDzE0CTVBe4GvHocnc/5 /XCQmN/a aCP63rq7tCzUHva0VFA3LffH0KdQ2HAvlNVyeFne4P5j/tlusFTsNrTwkM+SjXkXvaWkHMEpqXp3R5CnnCQqG69AiNJjzxwHaMMAwxXUmJQdq6R40DK1ruFYyvzFtGb5rMFid8CqtrBCV7Iv9T8H+zGE1Qpe+yIj4CevcdM5nlaKgYOvLSVbSXZGITnyKRV8Mq4z79myOT6nl2PC63HH7Ijt5IkwpsfKdk3CJ0ure73O6/mFHwheRjs5JvvhlbERTKyLIz0ECPTP2b+Dd+ezeEwzxtu+GG1MhHmiDDNEa12IIHKOZHg+Z5rMkDl7oG8beXjyZHpzkG9XXPcpZMCbZ1UvaA7OABAVt2EULQ+W4/UPf84D1dZ+dtmsW0fcEREkvc/c+wCAmWzoEtdL+91wQuxidEnN7GEvfJWigLtwaJ2F2gFIL/lN2E1WzuPseFlKEcYjkCK3pZzlYjf4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In do_adjust_pte(), we may modify the pte entry. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. All callers of update_mmu_cache_range() hold the vmf->ptl, so we can determined whether split PTE locks is being used by doing the following, just as we do elsewhere in the kernel. ptl != vmf->ptl And then we can delete the do_pte_lock() and do_pte_unlock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/arm/mm/fault-armv.c | 53 +++++++++++++++++----------------------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..2bec87c3327d2 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -61,32 +61,8 @@ static int do_adjust_pte(struct vm_area_struct *vma, unsigned long address, return ret; } -#if defined(CONFIG_SPLIT_PTE_PTLOCKS) -/* - * If we are using split PTE locks, then we need to take the page - * lock here. Otherwise we are using shared mm->page_table_lock - * which is already locked, thus cannot take it. - */ -static inline void do_pte_lock(spinlock_t *ptl) -{ - /* - * Use nested version here to indicate that we are already - * holding one similar spinlock. - */ - spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); -} - -static inline void do_pte_unlock(spinlock_t *ptl) -{ - spin_unlock(ptl); -} -#else /* !defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void do_pte_lock(spinlock_t *ptl) {} -static inline void do_pte_unlock(spinlock_t *ptl) {} -#endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ - static int adjust_pte(struct vm_area_struct *vma, unsigned long address, - unsigned long pfn) + unsigned long pfn, struct vm_fault *vmf) { spinlock_t *ptl; pgd_t *pgd; @@ -94,6 +70,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,20 +89,33 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; - do_pte_lock(ptl); + /* + * If we are using split PTE locks, then we need to take the page + * lock here. Otherwise we are using shared mm->page_table_lock + * which is already locked, thus cannot take it. + */ + if (ptl != vmf->ptl) { + spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + goto again; + } + } ret = do_adjust_pte(vma, address, pfn, pte); - do_pte_unlock(ptl); + if (ptl != vmf->ptl) + spin_unlock(ptl); pte_unmap(pte); return ret; @@ -133,7 +123,8 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, static void make_coherent(struct address_space *mapping, struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned long pfn) + unsigned long addr, pte_t *ptep, unsigned long pfn, + struct vm_fault *vmf) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *mpnt; @@ -160,7 +151,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma, if (!(mpnt->vm_flags & VM_MAYSHARE)) continue; offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT; - aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn); + aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn, vmf); } flush_dcache_mmap_unlock(mapping); if (aliases) @@ -203,7 +194,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma, __flush_dcache_folio(mapping, folio); if (mapping) { if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); + make_coherent(mapping, vma, addr, ptep, pfn, vmf); else if (vma->vm_flags & VM_EXEC) __flush_icache_all(); } From patchwork Thu Sep 26 06:46:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5DD8CCF9E9 for ; Thu, 26 Sep 2024 06:47:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 54BC96B00A6; Thu, 26 Sep 2024 02:47:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FBFE6B00A8; Thu, 26 Sep 2024 02:47:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C3316B00AA; Thu, 26 Sep 2024 02:47:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1C14F6B00A6 for ; Thu, 26 Sep 2024 02:47:37 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DC71D4083B for ; Thu, 26 Sep 2024 06:47:36 +0000 (UTC) X-FDA: 82605958512.08.DA749C7 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 1098B120003 for ; Thu, 26 Sep 2024 06:47:34 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UKJaeZOM; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333135; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=FGfpWKdmzK0jnK1PWirIO4gdjX69d4dGsCK5egCJil8WVCRBVtzdsbGCZ9QMEfkngIYrZG hZAxTEkW8s+z2ssWbz57+XpybS9rBd0EHfXnngxdlFurZjuPMiMmIw6jTL2VlnllEAgiY7 ajft3GpoUy4r2L0LjQK0EbGFW4ol0ME= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333135; a=rsa-sha256; cv=none; b=NK99ECrm/MWrSHkshYmO+i0l0nvhiHSEr49w5iJe7QP0cRo4YIF3Rtz+CECppljlX9V9ce qKokKmzQtEy2FGBj0QBSjkAuDSXGEmKxgwoDipatRIa12NC8EsbF0k+Y8fsE/+Kmr0KfE+ ch2708gfeZvEPEoNXzpj5gjo+35xaeM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=UKJaeZOM; spf=pass (imf29.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-45aeef3ce92so3444551cf.2 for ; Wed, 25 Sep 2024 23:47:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333254; x=1727938054; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=UKJaeZOMGdsQeZVBP2kjkdS1qsw4Bc9pDJIM92X0xBW9mrv/EE0M+7TWzDlwggcVeQ ZcNekqKwSlhKdjcC4QFuAZBGzhb3CfIu/ILNZ7acYb9AzC4qd31lfzMc0N0FKQTWCkcE tYx0hAFf+LksEleK5gkLlPRYn7QW6pcoZBRkk+xXe/yalJaQ8EqttSIBhHF6XxIe9M/L Ns/+mVVJ1B6msW0fqvsOswx98IiBWQB61SRG+A7vtI3FiFGMAegkOL6c3Bk0DgdmsXEd V5oz/es14AWJ0Xk5peZRwXvMhfyeSF4oROkMCjLtvSBO32zcz2wkGx8I3AxXwe6UgbPw 9JQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333254; x=1727938054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=lcyF0omaPySjXxqNisMPkmsCvAcVMTsYr0NDmjp0NBaHoeDdiCcRy82IA/aDgjyuKn zT3YrQxM2sfNocPwJfm45Ggm/WNjXl2HQd9hpgr0a7foGNmMPcQfoKncxnAe4+XZK6/1 LwsTKVGgMNN3L/Hw/IB8xPCVfFGppkj0XMMUty6Dz2SfNKj75gGTi4QAZ8NZCqH9DgGU ghD51cK0E1rHlaMUS0FPZbuSxpVwGH6C1OE29GowX0xIamdw7nOLO+u0CyJZw2+yVGVQ ckn3SXoBR2FM0Av+oSjUINe7yqc2oXPxkGpk5zXMR3irSqIHKrhXeAir46Oq4yOfKNdy 2kHQ== X-Forwarded-Encrypted: i=1; AJvYcCUh7LypOtNH06zkZIYtx4pXcwEL1RZxeUaWAcasREj/DqWC18A5npN5RBHyvUs8NvYpeWjMObKdkw==@kvack.org X-Gm-Message-State: AOJu0YxWh32pr56OytIFmul+PGyNpez857veAm65n01/zZYbQ+5mFIFu BdjicbKKsYRFVRJGP15wh/TOHbot67lDoeju88GpQDP7Xvo88JYZUimRe1458Ek= X-Google-Smtp-Source: AGHT+IEkNfjkqcOSOf8WFqyh7ycpXwLGwDHOuzbRwWfPgE4JYY85w8LbZC4JktZjOBGqNKfXWcA2mg== X-Received: by 2002:ac8:5a43:0:b0:458:3824:a083 with SMTP id d75a77b69052e-45b5def43dcmr69885101cf.32.1727333254117; Wed, 25 Sep 2024 23:47:34 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:33 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 06/13] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:19 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 1098B120003 X-Stat-Signature: 6oim55e49hif4bad3npb3jkoz6xbjrsr X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727333254-528989 X-HE-Meta: U2FsdGVkX1+u5WN1FGG94iWk4SbcimNHh3GmFsuAAE/oVpE6MjutYJDuMeZmsCXAcZfRePyAimZ/PWJUSMtU5kWV33zHchSL/DEEYn2b76NdDpapyP87YoqdcjwGc7rNBEJOF3woISeZhudXvm/8C1sViE4hPOLAtw28m4M+WgFvLMX97MqTpnT10FOAawMyvVxIsVzbc9+yD+9w3IAPh7bRmK5d0wIUDTfCjHYOkDrZeSIVKLoyWBs0On2qfxq0WC0POjqM10npoX/Ka2FWAxlwi+HuQPuDUI0AiUZlRBtBYew/1OJn7RZ6iKVvjmMGYeOz51w8n5V+CFS02tbQ/+AKtY8PJMP1YGvnycFgRx1Hot7Yu7f4hcNLuI0yNvzBZ8lnGulj85SMK/yGozzzR15ryAT11b4yhU2GsPAvRdROUlEv7hRrR//k/CV7dO+Tf1YrtY0zGVY7KQ4GQ5h+j3SkUT51QEi6I1tq3XNNPufGDPRksQF14WvwUSVlnbUE5166vxjhyOaMcIWHOHo9QgdJCOpz24GpRixYIYBP8ZCpHwvx1Z1pd6WbvDZ/5F9QLv6NNuj0X6JM1yFqdm0shCUzGRGJJNBVbC36AN9gPJCgIlCYhqq21BQ0WLDfROzM2tLDjwbZvS4mvstDf6gGa/RDejuf+ZueZSuJ5qrxAJMrfFsyrwQ+4iZkvcHf6stOiZ/rqkL0uypjK0+FExNrm+b08LbKF5r2ZaXFfrRPOemRUT1DS5geLST8grH6SUofdU7yxYX095opQPhNIPd6LCPdORD4nUBpxzy6VZhjs5vYwUhbJmSX98X3AXaFs5Ic/rYRK1iUX41EfF78sNVlvLbzdjLoUkOVYchVrCFdIBZxjG6MtolIxlzwjM0RjDj9qRONbf5kfb+cYQ37QKToswEiKA1TBe7U9jzASrrgJpOkFYCjoDdUVG/JBEGxAUFXwzeiATM+iTYfX96Kvqr gcYvo3pj PPf6tLx/Ni8uofJvDH7AVDcYfRuslYiRj7LlCupLcZ4JGR1KShBenUKgFKorAyS/g7bxyKgTFw8Y/7uAOroHSOeBLZ44RnvZMQM7CQXAol3MllOsGn9Q2f1YpcKT6rn3GGWuhjghPZiaiqWI1CeHEbJJfO2HHSX6xwhgCAvUYBpBeeAssaPaTaz2w95q3nPUNS+KIYzDZTTLpwGQ9Uvr+y5CBQ2aL6tIs0lxsLJo/er32dYx55hl8LmhhoXtroEd+GONhPdTMZ1AV0uyUW0iN4Yf/ncyAzOfdia072rG3PgtjUDrGw0VXP8EiUNQbTFkEZxMv2D/Nl9rZsjj3EjbwwIBDAq0UBF2f7IFri7s1GSjn+6ZuefUrUjB0HaPSan5J8vWvPl/OwzanpBWdoDAwU4AxofH9cRkylLG7v1olW3uZvtWbxoYhh7tkhlW9ZclRFy/CdcaL3JP9kjg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a245d28787034..6432b636d1ba7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5750,14 +5750,24 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte may be + * modified, but since we will use pte_same() to detect the + * change of the !pte_none() entry, there is no need to recheck + * the pmdval. Here we chooes to pass a dummy variable instead + * of NULL, which helps new user think about why this place is + * special. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Thu Sep 26 06:46:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC639CCFA07 for ; Thu, 26 Sep 2024 06:47:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D66A6B00AA; Thu, 26 Sep 2024 02:47:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 486EC6B00AB; Thu, 26 Sep 2024 02:47:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34E866B00AC; Thu, 26 Sep 2024 02:47:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 144456B00AA for ; Thu, 26 Sep 2024 02:47:45 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8AFA3C08C6 for ; Thu, 26 Sep 2024 06:47:44 +0000 (UTC) X-FDA: 82605958848.02.06B9075 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf01.hostedemail.com (Postfix) with ESMTP id BFF024000A for ; Thu, 26 Sep 2024 06:47:42 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BL6Gv5VW; spf=pass (imf01.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333226; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; b=EQNY8yTZOcTDHgwTKoRBlcBws4lmNKQuKYflql/gRaRXJzTmD2GmlM5tnmnWZQdq0plfU7 99zwsA+dws+c5sIHcOaT0Bzo1DQezIr+/yXom1pfZT3YJ6nEpqFE/najE+cbOagx6AQEIZ csiu7yraIf02ZNTLOwJQQbYP7RgaqSo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=BL6Gv5VW; spf=pass (imf01.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333226; a=rsa-sha256; cv=none; b=qZJVs6YHQBYkj2Jdz7/G/kzWPrFNYx3+a/f8jzHKZ14nZUWxIrkfAwBcq9CQ+pdB7zS/OR H3HzGoz2H7mL42dacHN8kn4DZnUgWdm8nqMheLcpR8a3aorFZ0uCVo1gK6LU5Qak7cnu+W KoIdFqwvMb/HYGGs6J8jKieZiRKrt+Q= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-458320b30e7so3909191cf.0 for ; Wed, 25 Sep 2024 23:47:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333262; x=1727938062; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; b=BL6Gv5VWUNAD5G5nxV2rlGzSJGnEl2drmvi0VDgz0CrH20QRKbPj8gMp6Y3YgufJoG 3dGmCNS5LMZSJvPUnUX4CgUNGlbp8oX0DXLG7AfiFTxhpX6iSuuMB4Jc2JX6kG72S1d8 TAV2DPz4JOUvMHc4KTaXNWU4cGaMNcr9ObVUn/OgANn81Nea6rT366ydpXKNTLvWqbRK 0mO9ilxogKM9g3hSmBqN7YVTmBs8NFoNmT4w1dfImly1L8JZLevioWmcPKyyV6OTT9qe f7Hu9xl6A9Vdnj2usgfqMlnrvmCR8JRipe2/QC5rj9VE8B6Rwe4xuPqmeX7o003YZ4ye edVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333262; x=1727938062; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; b=qIYvhvWB0Mnmfj3d2tYFfsKEuh/qR7mpOzIA4Yr608QhALHCorJcknUKRv9utw+luc eHeajtg2KqnoB24FCLNG3SuyseWN91pNihF6/QVcQqnp3327lbytC0sl/HTT+cNRhhm9 esclQ34ICi91SL1b00j949WkQ0FMqdj5Qwwzaxkz1S0x/q/RRGTt4VtGJY4OAAmChJ6g 0qOomkPqpPDe3vWEASaUR+HjVJLtpaK6d9DwNyMijdqT12/hGvg6I4GcR7uoyK/0tVn/ ILIuSVOAJfNFSleBFknpPw60kQS7+uWwUb/lVbI/N6nsBN6vQum6Fj0P36JLCaogBI9m l2og== X-Forwarded-Encrypted: i=1; AJvYcCXd5OsOoRvbaZcK76HW6Uu4McGULaqZywVfsCXO/9yv/0TTWvhdcMKpGiIrqCBnL71nkbtHT+hf4Q==@kvack.org X-Gm-Message-State: AOJu0YzlxGOYfVB2KP5B1pPLiUeLFq7x0RPyNdauk+l1/m6kf8EZ+tjE jzId91pL7vHAvq35DtHzkcUblvall0SSQNsqDbgjv+MB+7Fwbtv46ClzF+88BRmWFSPUQ/MpjME 8 X-Google-Smtp-Source: AGHT+IFPPE2WMfZ0ye6LGa3/R8zXQ/6KjVtS14uw6772ox/YEy7cH2DCZDAwEci91jHXPd4jVDRFUg== X-Received: by 2002:a05:622a:1493:b0:457:c747:cf3c with SMTP id d75a77b69052e-45b5e0534ebmr99370671cf.57.1727333261763; Wed, 25 Sep 2024 23:47:41 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:41 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 07/13] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:20 +0800 Message-Id: <055e42db68da00ac8ecab94bd2633c7cd965eb1c.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: mi8k8yofp766knua7npkrpmh31mar4fi X-Rspamd-Queue-Id: BFF024000A X-Rspamd-Server: rspam11 X-HE-Tag: 1727333262-23894 X-HE-Meta: U2FsdGVkX18fPxGeZgOuZacLH9JKk+YWv6PlqVbhcu1sMyfnBkPn4cyC9Cs6gMklZn7+jNryftnxgQPfn9TZTXhDwrWBPXWM7Ugf4QeFc0G+hxHKAiKjPY0s/0SkVOe7UCrIv3bCZE0Q27VFs40jrCBBXVejgGlb5TwY8k4qXLSOAd0Kr74C7MkfQttzLDVIB+/LO+5GsDzPMciCKfN9z/HIHHPUryA3Yh0jEiEV81KAJK5JnHh8mOUJqIMvn5SKxXOm2qsnmEg/gv0VTOZn9tjU70JG+httyJNhq+VhTHB5nhNVVSbZpdCKp7tTsUovc23Vh8cxqbNsJu4DYu9RqYAigBwNo90na+rI8oc75O61/JxcN7HmBR+4lvU3+K2XGqDgNwJDJQEeX50RPSJu2+X/f1p3JoVNXxNW2amail9pv/Y/yF2LUCC8FVdTzkJ8tGfDfAY/UIr1Ht1IdIQatJVG/Goupm9mtcS3H310Ej74t9i7arvRuc0rHMKRr89ImPKlvuONL5iDFyv/f97xgOBekmy9HWPODpOBFG3y+Mrz96X60o3jPgcES3khvGcTu5ayQrRaLcmBYCLimvYRAvTZ5SH42LdXZ61AlRWhuELFwO9sd3LmN5gqB2rF8DwRk3ztnTTCf1kVdh8gKR7Q+wlL8hOfA1Cvc9lxF1GrSR2iKpN9RCyCResUfZjn1i6YuWdC5h39UQADaLVD356ZtNucud83BELW+w6HFjn1kr5h0ihwzlH/qpuvrI2nzz87uHY7y8swgQeY5SvDojerprkb1dcGsa2PzCU0j6GnxPW2AzAjmYMjtr0Y/0BtswwvpMVw/aWvDe6C4lenJTdRNdSDlzU22+R/2Hx2Kjq7IZ9Zo9HUV6I+7dkwGrULPaNaVjUpZYV1RKFOwNMrfUpwh8eYGM+hHPJR0JAlchWZKTnUEgU/ZJtq95dZx6R3X0MiyXGRdx7Cz7mBjWG04/n ve4Q9oJu ePagtuxvvuz330W4dmaxndnBANly+SS1+4TdCAHcjAF//ec0cJp4wG8kjR2lMjarWFAhlJOrjgeTtW9jyvjUG6QxfytwBbHK19PGvoBU8rzXvIsJaoKDWq/fP3D5OyQ+96G4fIgZ5kuwwm98Lku+yUHoGuseiC385CAerMu1JSJgtYZ7FM1PZZIm3neFisvx3YkznPLHC03SX4M8c5zjJW5JAJGr75nfqA+2FGvY8ulHvqehyAQx/fNz/XiOmSLnESpG15RvbB8q4h2cdQCegJh7MDC5Y07A6/0rj8QtTn7VfjYq6zfdvKKwltdjqL2Sihn/26BocGzfbb8/Z6LU8i7q526xTlZ2IsgYLjVlsVOpi1/5TVo2f3g7vh02sMYS+G+XBtEATJkRD+BQYh0+Jx4IWkuUyDPXfDLS35RlXzVV6mOw8yu51Cmu1dg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/khugepaged.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6498721d4783a..854577f39957d 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1605,7 +1605,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1613,6 +1613,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1645,7 +1648,6 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, nr_ptes++; } - pte_unmap(start_pte); if (!pml) spin_unlock(ptl); @@ -1658,14 +1660,19 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); - if (ptl != pml) + if (ptl != pml) { spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + flush_tlb_mm(mm); + goto unlock; + } + } } pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd); pmdp_get_lockless_sync(); + pte_unmap_unlock(start_pte, ptl); if (ptl != pml) - spin_unlock(ptl); - spin_unlock(pml); + spin_unlock(pml); mmu_notifier_invalidate_range_end(&range); @@ -1685,6 +1692,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, folio_ref_sub(folio, nr_ptes); add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } +unlock: if (start_pte) pte_unmap_unlock(start_pte, ptl); if (pml && pml != ptl) From patchwork Thu Sep 26 06:46:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A771CCFA07 for ; Thu, 26 Sep 2024 06:47:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ACD146B00AC; Thu, 26 Sep 2024 02:47:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7A876B00AD; Thu, 26 Sep 2024 02:47:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91C366B00AE; Thu, 26 Sep 2024 02:47:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6A86C6B00AC for ; Thu, 26 Sep 2024 02:47:52 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 208431A0681 for ; Thu, 26 Sep 2024 06:47:52 +0000 (UTC) X-FDA: 82605959184.16.10EF040 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf06.hostedemail.com (Postfix) with ESMTP id 42B41180004 for ; Thu, 26 Sep 2024 06:47:50 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="HTA/+byd"; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333150; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; b=6s/moLFSzAW9OC8pfrnd9xYX3p9TCJc4xdY5T1nwafQVVahlVle0UThVqohaAPVuPrS+FQ t/v29Z8ZWcyLm92f6tw5igkrkvGQpjjH6eXn6ZhWzvGN0wKGxss8fRJopcb6q1zDzH1KPH CKdBc7awZBDcs36pVJqa8QEaRe2moe8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333150; a=rsa-sha256; cv=none; b=ozXAJZyhXhFe95MNq0MdKjEkncDv9XNcY0UKUD8apnTWC1ASjl+mwoZIfET13PZZOTSzow yD6x9oMoD3n6V4UBRLmNgDRFMBV3ix1JPr3vlZ9LlMpw9ze1Ne20j7UytMAUglZ7VR1toX xvBH3WJagvXlRv5GUEhYQSD6IGTyctI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="HTA/+byd"; spf=pass (imf06.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-7a99de9beb2so44969385a.3 for ; Wed, 25 Sep 2024 23:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333269; x=1727938069; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; b=HTA/+byd4aT4hP4ivk+aM5+GkOiGlu9K4Y+qfn1SEW6SDjh79qNnaaRdCalgsxYQZb GzzoHiiBddUGHrLDTyLT01OboU0B/QmyR3k8taWiUZocNO1JVVRkmabR1Mb7GDJytAC+ gggzMYT+ouaVcQuufEBprHfED2sXFW7ew6jh0Q9NkzlI8/Acw0cy6KE3k6nD4i11g1Ez tIJiL8hVEooiZ8PpkMbRJt+l0aqMSwfm/x4736cToc8VGz4JB5G9EKOB2SeOwMwnnd2v xq5RGyPQoL/k0auZizZcyKiqp8h97WbR0F51GCSAoMMiCjNhIaLSHNtiUCKVB/nqW1LX I/bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333269; x=1727938069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; b=sCJA5S9XqNfvsTmvZ7dDkWQ50gCqWrA+lLdFOnO/Yoj1KA/v003MQ960yoAwkSsIMI GTo1HRAh3AugIqC1Sn9JU7dfZzro456qAlhDLWGzYZdzJA2yx/UeNbGjUiXkXsVBtgRL 0KgUdS0jP+kAWP+PxFNKIHlflReVfBIfJ7POJEglXkiUVm8NjTqsAs3HWmyPKBUH2Yke 38zO4HvpqL4GTQAAuRI7eIclUhF8MpdyTy+NtQuYemWjUf9C5FiwRtUiLr288LJHqyep IG4Z+3ueBF9s+HN93S4ZFmmZlRl1JZMkxQikMHJ6kVbEDKWWEn2gGgMS8/b0Tw1w9/4A vd4Q== X-Forwarded-Encrypted: i=1; AJvYcCWW2nbtyVdmkLLzYx015zzIFGPTaaoYdVeTNrBYJXLt8gajz6rwhCuZOHy3/f846QITYV1n+1aakg==@kvack.org X-Gm-Message-State: AOJu0YzIsW+v97Oh+c43KeHJwd3Q5Sn0PbRAIuZ9oSioW6dOClXj6LBl 9Y3BAQOZY7hLYs0+OH6ItPBEEqwpSUqlqUC280YqU8YYp+y8B4bgdtQ0ii7y0uE= X-Google-Smtp-Source: AGHT+IGf6yhFgM7V9dSkmNMD310kmtT4NpPr6tml7YSLsMbfjuBNhYOZWImI0+hdwHwJZd31DH/Ogg== X-Received: by 2002:a05:620a:4009:b0:7a7:f18a:e46f with SMTP id af79cd13be357-7ace744e486mr712325485a.43.1727333269291; Wed, 25 Sep 2024 23:47:49 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:48 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 08/13] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:21 +0800 Message-Id: <9166f6fad806efbca72e318ab6f0f8af458056a9.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 42B41180004 X-Stat-Signature: 1jkbktpfzn483q7ryd7xcroq7jusbttb X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727333270-619511 X-HE-Meta: U2FsdGVkX19+184xlCAdQZGxZjN0G3RZo/wgaQ/lGgYSSV+lOAesv/Pz6D4VHmMZB4GRV95HzgCa4+zqx6KGCs0rZ+brUIYNFvjIODf4u+8/AViLXvFiJtaOzp/GIwY11kgxzYJcKfKxy1pVBS8ehiXZacBBsMvSzj7iO7Bj/pVLb4JbsPFD6kyLZutPTIu5dJOClEfdpI5N7waMMXumcP14wjL0J/c6kIoygI44eEDFIefuRaWWyXshU+MD+iUJtAWM1jz6HFpYovDqTnkB+ZgjgCMf3fDft8nlSLHj4cUXchpHN+6Q++TWkxK+Yr8n452rudciY+AyNe2UyLrv/Qe9QmZGe2+oAD+Ghy0v0uWMieC1AB1iCOYsYP2rf4JwfvtsqlvI01mcwQq26vr3D1vNyNzaVervh/IqjrKoIM7ZyW8TOixx2x3t5cwJD6L80r4Pc0HX6Edyqu+AlU6l4oCPGrtUkwI4FrjZySIIeNP4U+v+hmJO8kRnV1W57axK4GwAOj7rv99Tdtl+iS/Dpd6cKLxxs84LcHoX3cmTQLsqxInEFRnnUUF16LMJzadXDYTqwBKVSPuL+Kv++9INM6Ykau+LUUX8cZKGumwzWCu1fNcj90bASkRKZOS2FPNZO3y1IEcQL51OTP3IGppisUZe/joZkt3ZCKmUg67MhI5cdujre8L9eKZD08dSd4iROG2Ac9QUhfF/d8VELhxepiVdGalNDFPUjUNY4AQ5V0jsmbDI4t8A8B8ZMuL4RKEn3LCHyysTPZw2khUQWUMr8uQlzGhZMxZLdPkkNuu5itoleYdT6Ko+qkcxQyZcqaxPr5l2pntoLQpGjHtbTsIy3LOwslC2m3vus6afVcBjbxNvOg3RZZDQ5BonOc0vxQTVUGop9Pt9nLtALd5qjxA1x43jLBnTKo+vcCnAdh1VlaMOGeMwV2P6BtHWX1VdV6+XsLdTqFGtowAcs0F/sQN ubRgTcpG L5ChcMu6XVdW2XsJ4JNR2o3iLLEvuH71M5I+FAy4R6FEgYu7sCA6TgBNQNtzxlmF02mM3jjIj4uLWwuQT8xTfabcM/ScO337LLxGJjdhcCgf3YjvSE0OJnf4iBOE3KMX7lv5UH3BeMZjtroCpL8shD/lBLiDr1oSFqRx7MkZIy8grtlnT2Ur+9zqZzX17DR8BM/nmZQqMMhm3X87T/EoPefDjjOo2tFleVqWpdfj7mL8Mww4h08zcQxMbsrzUON6R3J6WjosdQpZJx0uqP75pJ7QqSW8qoVo64PUxYuo10H6ICpQOqJjmdx9vEQUXQ+RhsXbqCRkNiiD0ziJtTztZjuIvnGKIT4XcMIv9K0bIkIu+xXn/c+PSVQn2Yip5FrWAWPWi42z97Z3up1yRKj34WBQRpz4TGFtF7Ipa1TNwAas2mOAWajJbWwVJRA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we already hold the exclusive mmap_lock, and the copy_pte_range() and retract_page_tables() are using vma->anon_vma to be exclusive, so the PTE page is stable, there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/memory.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6432b636d1ba7..c19cf14e1c565 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1086,6 +1086,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t dummy_pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1111,7 +1112,15 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * We already hold the exclusive mmap_lock, the copy_pte_range() and + * retract_page_tables() are using vma->anon_vma to be exclusive, so + * the PTE page is stable, and there is no need to get pmdval and do + * pmd_same() check. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &dummy_pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Thu Sep 26 06:46:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 573E6CCF9E9 for ; Thu, 26 Sep 2024 06:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E00156B00AE; Thu, 26 Sep 2024 02:47:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DAEEB6B00B1; Thu, 26 Sep 2024 02:47:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C50206B00B3; Thu, 26 Sep 2024 02:47:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9FBB26B00AE for ; Thu, 26 Sep 2024 02:47:59 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 66AC4AADB6 for ; Thu, 26 Sep 2024 06:47:59 +0000 (UTC) X-FDA: 82605959478.19.CF8CB29 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf30.hostedemail.com (Postfix) with ESMTP id 97C9380012 for ; Thu, 26 Sep 2024 06:47:57 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=M5M1oAky; spf=pass (imf30.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; b=XYldgDQB4J26Gg05VbVPwS1/jasnpaZ5FUQeoMRVHm5r0ApIpWAh+1Glz0JxIhJ1bdJ1rq Zh/1iMKf7eEQXjgiSVZVJQQjhXjNTtX139b4i3Z94YlXNuZY1oiY1+Dh0ZC247RG5o+Ihi WD+N7IvuvmBpJK68AcRc7V5LbelF9vg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333156; a=rsa-sha256; cv=none; b=NWWMiI9mTpeHuIjzjGFjlusA+QW0+VTT6VIc30Iab/q2ppzhO2aoE1XnJSgoKexNnnvHPz F7kPjycX6SoxRZYqbJS5Nz1fQyDTCPRwwxUXUdbuKK6FIoQ6fdrudwiGZpYOlz9aCMnM5/ 6MTmYf9FIlPwrSiYvXTTN0bqn2piCfs= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=M5M1oAky; spf=pass (imf30.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-7a9ad15d11bso57929885a.0 for ; Wed, 25 Sep 2024 23:47:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333277; x=1727938077; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; b=M5M1oAkyela4Wd0MwTYtXzlmYv+diAv0zP46OcMVVOAOuD0DnSAJ5EXnqNTfVWR79q 7zfvOT+O4ysz0eN+zYPG1bslYuAQE6mK6uKNEP/771kHaf3WaRRFrJoLQbT09H6WvuKW +r/k09aMT4049po5MymgjCAyvyU3a+ezZn/i1xIXuC5F+andT3ptO7ZyQta8Qh9+66wS rg1vXtXhV0arKZSSaZ5jiikn9FoKkww5x8ABhBPGSOZa1+VpS8IfWt1FLxkQCY8o8EH4 xdHGcOGm8bskREM3WmHxVWrdDf8ygMNn+S2C306T5Uf63kMdu+eXFT87FMnnzSodkYGo suQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333277; x=1727938077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; b=ddhUnde37vhwcr8woZn7MdFbqG4wzJOLtVFzR9cc5OJfyTfmU1TBhMkOMJ0Ruj9WXC ariEoOAts4hzLpVJEdYUpHOk3OSNzRC2InL3ItFJ+nBOpi9caepAJhKrmG7V6Gmx0laC iBzv6iVtyGJZaIKWgdkJ3tANWL5l3m0vMBjdBoCN5t08c7hD7kRe7Pwb9wk6Q3AD5H+u ZPPjjbSOeIkPBranDtVS3adPH3XwxdcUmiqCJMh+c3SJ/8EvPHoJ6G4FkVpfJ7pkV8mf COIFkjHjmk8FT7S6r0vDaZxVlHecv+f5V5blh3A8+bn1ki15FSfDRS7oOM7DfeRZYHAu eKaQ== X-Forwarded-Encrypted: i=1; AJvYcCU8mdYh9WRlsFxj/32oYQ/0XanrovHp0qYRt6FaaiutY9dapkGTENYYuJkroY6Lky/WAmktb20kNQ==@kvack.org X-Gm-Message-State: AOJu0YzZw1rvULIrm/+WZ/VT0SSuea0w0W3YXHmUSsTFMKWl/KTxrwEE x4CLjBH4WwvXvgp0Eyfba+8rIgCKoyjF6iu4JNENVN5svu1uDcOLvG7hGtQli3s= X-Google-Smtp-Source: AGHT+IGbo2MSjNAvLC3c2ApJAOpwKD09S+YUwBzMfKcY13i9YN/E+PXjNs0XRR1eOHqqnOUdrqUWyg== X-Received: by 2002:ac8:7d42:0:b0:458:2ce1:26f8 with SMTP id d75a77b69052e-45b5e045ea9mr77074431cf.60.1727333276776; Wed, 25 Sep 2024 23:47:56 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:56 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 09/13] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:22 +0800 Message-Id: <9d582a09dbcf12e562ac5fe0ba05e9248a58f5e0.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 97C9380012 X-Stat-Signature: dgejykeok56j6o8fneer34pp3sjjtcfa X-HE-Tag: 1727333277-41087 X-HE-Meta: U2FsdGVkX1/AVwuBsb/Ap2Y7e7GdrKYhYY8XjBSwlvfIlBPUWkGuNvsO9/06lTUKo9e7y3wGtZbifOp3ZVVG7i93qWCL+VCgPjAqlxftLVzajM64E7yIsOCy9sXEkSpNn8msSnQiBfYzYYVb/4UUqWyo+A8YMAynTLFyo+r6ME48dUKLq0Nf1O0/Tzjb5hGipHxL1GQhvf8n41bjFRV0Fn3uwmw8L1Flc1GhcUWvguuIRb8ce62Ykci6Y4ndcuYea4eA/T4F/UbkAdTGxmIVyT/iSzu/1wyQk8HR40GKpYUF8YrCRqussnJxB5sL0omOsB6F+3WvPy+y2SxcAqrKFoGgEoS69r5HO8BTobaX4hLtOZeaIa9o5bFZDdmL52I9y+UIwnAFZB/42gzBpN4ICdZLc5QbhYvLh14AcuoU6KCoWwSIgITwKoH4WO/Wm2ajUt+Jf/V6ovmG94FDvG1w8ChrI+JKQ1Ul6csuvTKCfLwI29/EeJg9X0JYTHav37xlqT7//wbmaxwRwxxtrVrJ5Bfp+1V/u2BUG36rZ5rwu+64spQIfvhk39VdlgSRwu+kdGmzkc7zFIaC+hFTFxcMfuuFPvUclwzNWBz08E3J8UkNGTjzN+v5OeFFH0ed8EOJ9hlis0AZLiuDa2MFB/+vD4Zaquhbtbema1FOVB3rya1BR9r8xqRQEKxdyxBgfA7D32np/z7Yd2gYhqswiJ8+mURBYwVsBNt50Lvlq2hrPQTLjpBhFWh+U9whzNY3a70N689ZLAKPtDUQjIKVXZjBZ86tQUjgWmq8grBPqOGJPn/zSF1gfSDSlIcjZ/+/z6HCvdNeWY8UbColxUrkL3Mb+kZxUoxGEb/iG3o8hBVfraTf1hRAWgv0PkoOYe9QNER0i7jE/hr7VwjkYkF0vfxuuuy2XYpZXysHFYt0XfeCgHcXXM8+Sv4EQVKxsD0qE99TyOvreJyz1JRC0495zEJ NZA39gxF Ac+uY6w5iju5boDDHfmYRC1oe4PsjFzfIZSgAaW8VstBQ8TnSOJyNbgTxgIUGpRA3Ic3YO3l0U1v+zwgknsXuX1aUHsqQyw/Gk9bDtr0BRowLEAvK1RmHRZhl0qNHOS2PoODLfmFdNBENcAhBmEBuW71hxQSe+oOeB6SfYl8ExoOGTQRRFK25RSEC9c5R1PIYw2uGpXke0puMKmldVUK679imhT24onYBQfNYRvn/2SkULNE563ZbYlZVLROVifRk+E5QG2qgLostrOOEVWtfAblDvGqb2rbEwLey43bVwJPY9Uj2qQNXW+Zltjuv54O+2LfB4VXjI/a39ATL8m8i3Yv5TYyjHWUEdIADNe6mPs/5rTjnGpnstHg22Qh//6IzX1ewNHbniYghICb7Hj4vvfQ2AxGjPsoieCnEsTLHeJhSroc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Now new_pte is none, so hpage_collapse_scan_file() path can not find this by traversing file->f_mapping, so there is no concurrency with retract_page_tables(). In addition, we already hold the exclusive mmap_lock, so this new_pte page is stable, so there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..9dffd4a5b4d18 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t dummy_pmdval; int err = 0; /* @@ -175,7 +176,15 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Now new_pte is none, so hpage_collapse_scan_file() path can not find + * this by traversing file->f_mapping, so there is no concurrency with + * retract_page_tables(). In addition, we already hold the exclusive + * mmap_lock, so this new_pte page is stable, so there is no need to get + * pmdval and do pmd_same() check. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Thu Sep 26 06:46:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD216CCF9E9 for ; Thu, 26 Sep 2024 06:48:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7494A6B00B3; Thu, 26 Sep 2024 02:48:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6FA156B00B4; Thu, 26 Sep 2024 02:48:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 599BF6B00B5; Thu, 26 Sep 2024 02:48:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 38A876B00B3 for ; Thu, 26 Sep 2024 02:48:07 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E1B04413FE for ; Thu, 26 Sep 2024 06:48:06 +0000 (UTC) X-FDA: 82605959772.03.766EBA6 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf26.hostedemail.com (Postfix) with ESMTP id 290C8140005 for ; Thu, 26 Sep 2024 06:48:05 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bCaJwtqK; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333224; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; b=4Hlo8wn56YX1mBnNa7AUDEipPreNG15vJjW+V/KFiDFN9WVdotrV2I+kXmSFtsfJx8TJJC 9Q2MSpKMguK6O1A+7UeckmUNCHMd3ZGJG8v5Nob6GjEIhT6oUboLNTVwAj5+jrPyQZGxFQ 1xwa6gA9nKj+UUR277HrpDuWzRzhB2w= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bCaJwtqK; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333224; a=rsa-sha256; cv=none; b=ipQoYY0ZL8xtbbMM8suOoBnyWdcbfxTlIyjUddGaaau+MuBnMPfldas6zKp4BP7Gq7k/JO xP6qCSj3DIhCMoWMv1pdcAE6wOikdggJHnVpNdM6Iq5wAR9M8J4MnTEK/DpZaKWE15BN+e 1UHOy3eI+W16DBKnUH8Q8HfMicvwBsQ= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-4581ec0e00eso4039641cf.3 for ; Wed, 25 Sep 2024 23:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333284; x=1727938084; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; b=bCaJwtqKOe6KkKkgl46ElXba2GFeJfcj6d3n59iTOUcP9w31YqIXCA5cnTyzE3keRm busEVGko5+4T/6ecCZ2Tk/mOIWIRAeQiKSzc+kUJxZTUjmaMfC4PAB0aDfCLxyuUMOhm nAsykNEE5jQJSAGscOnPKb9MxrdIeKFkhhsf14BtTIJV+SGzqTI1EleVFo+P3yK1cqyy 4ipSHUcucPNjuTijvUYpUBCk9/aPI7ROkbC7+8Q98gfeW+45yf6pEyS/ItaWsqOE3rqh ofv90YIBJXF434WwLqkBMndGO9ijXojK1sEr40a5MRqUtwnu2Virtr+p3D8IiWHZcrtV 8C0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333284; x=1727938084; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; b=wKYZYEx64vTG4RJzWf6r4ZEeKKNh07yfoM2XhxTmkEFbpp8c1qCNkY4gx1oAvwTcPJ FBUJaI5bIxmIRIxCoU4XIGAaYfLG8okkNyKoEY4usF8sjdmZ8Qv8f5Rd5szow/SmH8FU MY5BTUP0aCTtryOh9jDCm1BnUV7GZUtL9EcvycjkJY7xJrFs/cIQHRavDh0TrjFaK9Tz FCtskYUFNrybR0I359IYAB61THI9efaoOZmD3PWHdX8Kc0b7DBgmIbGAIz+PAktgRg7z 0tOPWkz25rDQb0Hw6typWgrJMGY7j/YXOQvkoPV51vbHXUiqxsZ6lr5yi6zsJk2EsS82 4uuA== X-Forwarded-Encrypted: i=1; AJvYcCVPhltq7cIRHuhQk7tV+n/0t8u9s9TgSsinQMFVjOxvlGn8vLhpJ3fIlOs8jx8+Ez6+IceiRv5fSA==@kvack.org X-Gm-Message-State: AOJu0Yz8CYhmvxcMRV+gxYVoy8POya/HNKHlCxtlro6FIMzDqzjYXt5d +KYktHiBYIs3SqoNzn5wQkT35vBZT0M2ECwMt8oi2qjDDXpYnYL2tyaptHniirw= X-Google-Smtp-Source: AGHT+IGAlBKd7aKCt6XxIcAXM/Sjs0v2+e+7/droKiEQYGe4LhoKPN6bWAepHInU8bkNuwlluGjQYA== X-Received: by 2002:ac8:7d4c:0:b0:458:4aec:2749 with SMTP id d75a77b69052e-45b5e045e56mr87889061cf.57.1727333284339; Wed, 25 Sep 2024 23:48:04 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:03 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 10/13] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:23 +0800 Message-Id: <2620a48f34c9f19864ab0169cdbf253d31a8fcaa.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 290C8140005 X-Stat-Signature: xjwumkgipfp3t15fzc8qbqz3imcir44h X-HE-Tag: 1727333284-914021 X-HE-Meta: U2FsdGVkX1/CwLnv6E5vSZ7gsLfjzQAvTY5hDV410ThkWhKrLlEDNE5odJR0XvuvTvJE7e6WMXWWZQX/xB7U9s87kUiO3llo6xBsRCpSIAa3BptnJwwkm+3go5HZkWK7bVSmulwobyZWFRO2G9y71t1pwAq3g1iWCWsexJKnxgRFQ0JxC1rotQPAMPvePhYnCQwEK6BCRmO1+5YC8cwdAOz/d9hL2/LWn4iDfa4cCqcbmHO04tLcHp6pz029AL4bmTv12yTIMwCDwweG+tXrMZyJHDt21oye7bXuKcIV2Tk4RVH4k/emJYymI8+L6PiI8pmUamYGSG1oCWr10IDfTOZ9I0Vs1oRnejY3Jg9wjykv519lt6YwC/KMaH7ZP77UWH96kjljaS7EoaZEp7m/r/GRsLlrxg9ke0KDsylWCctlY5BwnP+mQuzxsXJfFS/qYMFSRV64sXTvhKsGI8HZPCCu/rYnc9Ip+j6nRESYLQjfyBL94Uyo0frgJdFqwN6Gr9Bf8ABHGq0o5D6G16nZrX+Wqvkmm/7fYPUk9eGElTp1Z1DtyfPXHnR0ZrfBLqoY6F0e81V0yx61tRVvKMHYZnIcKstrQHlk0726ADqNsGbySDC1uGpzNZcdPjOKNHOyfRQWAeik2uWfCpgqOZ9ecycK5DtUoCDx60KttRYh03Br6dn+CpGMAkhdHF6abRIO8VPGvXLT0M2eC1zG38FpECRXTuLOcxZ6RwEOao2TR7QID/a5XpBfgcL/NviB9/uL2woFTwb2L2DexwJpJKRleyGCxXc1NI8JVxekbsCyRwH0c4RkQVo+OqdL/DOEs1ohf/mElamztNV9qcNEWvIchlqFmAyHzCld37DRZcrB6BRuQEVLC9hsof0TyGC7ReqM1vujWV2XzZOfrlXQR91e2V+NoKNS+C02hQ6CnbkfOct4QJ1Ml9WlJKOa0xdZmZoyAknWaCje/oNaReg5wax 9RVIGna0 4SWOHUMN/UF/lUk4RAZoE9KWaWUIfG/zayUJXpNdpsHj2NZd4VxqN5KzfbNw6fjivTsVmSS4JofQWypwh9UYJtJHKINRj/RfGDTagaL9JsH4FcGFmpZ82ImvL+HWRwmDcQagy3BieTFwnylQaiFNO5sjozJKE9o0obNgQni0QVhuPxSQixMpktuYl73Pe4b4MzjnD2GY27klX/TQ5whTq20AT5cK61m0oArmrsKg0yZKrNTU6Ero8Kp76agSdYwHmvUXBRLmZetTEIS/cAfqHc6dAo8099y4rcZQ0LHWtCUmpFQ+ddswEv1KIHwEOC17OH0+MjNpXvlmjc0ZDntWowr6hg1PvTRdIVcadHstu9yIJnT46hNaJUuOcKc4FjgIdrCp+nJtLD/nqdb0GyLzllZMCkjMV5ivwAPPZDWdMq6LCPquxEdXk8j980g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/page_vma_mapped.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..ab1671e71cb2d 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,7 +13,8 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; @@ -25,6 +26,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,8 +34,8 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, pmdvalp, ptlp); if (!pvmw->pte) return false; @@ -67,8 +69,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } else if (!pte_present(ptent)) { return false; } + spin_lock(*ptlp); + if (unlikely(!pmd_same(*pmdvalp, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, *ptlp); + goto again; + } pvmw->ptl = *ptlp; - spin_lock(pvmw->ptl); + return true; } @@ -278,7 +285,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -305,8 +312,13 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) } while (pte_none(ptep_get(pvmw->pte))); if (!pvmw->ptl) { + spin_lock(ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, ptl); + pvmw->pte = NULL; + goto restart; + } pvmw->ptl = ptl; - spin_lock(pvmw->ptl); } goto this_pte; } while (pvmw->address < end); From patchwork Thu Sep 26 06:46:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99C0FCCF9E9 for ; Thu, 26 Sep 2024 06:48:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28F246B00B5; Thu, 26 Sep 2024 02:48:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23F4F6B00B6; Thu, 26 Sep 2024 02:48:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DED76B00B7; Thu, 26 Sep 2024 02:48:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E3CC66B00B5 for ; Thu, 26 Sep 2024 02:48:14 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9FB5C12020D for ; Thu, 26 Sep 2024 06:48:14 +0000 (UTC) X-FDA: 82605960108.09.28E75DD Received: from mail-vs1-f50.google.com (mail-vs1-f50.google.com [209.85.217.50]) by imf09.hostedemail.com (Postfix) with ESMTP id CB6EA14000F for ; Thu, 26 Sep 2024 06:48:12 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="jE7cH/IL"; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.217.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=aXTwut+dzNyHtytsuWFcXJ2Z1C9apqorQFsSnetqX1dblv+93t/qlZ0Nc84LYrUQ1PpBUy gJdt9hYiBinZjn1Ffa6vXJeCwZ6WHHYlrMpKHjJRURdX1Lrpt9BrfXHrQzK43997/yYFie mmImM6q08VoBH60moUEriRmpQgQ6GSk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="jE7cH/IL"; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.217.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333256; a=rsa-sha256; cv=none; b=6R7sD4cZvlhjPAdy9vNYiEH604ubXvKgm+PNgeKCS8dsMRX+AHsBE1NfSQzF4eDU948y9C AHrwKyNdSK63Z/X+O0BRp31Qpsdgr2jIVhhRsPBFOqlZvRnCt0xZkgbZTCY4mstgUpqHym 0OVkM+Xuxgl8uwTcXxH5RrC/C/kkTQk= Received: by mail-vs1-f50.google.com with SMTP id ada2fe7eead31-49bc44e52d0so278594137.1 for ; Wed, 25 Sep 2024 23:48:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333292; x=1727938092; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=jE7cH/ILLfiwyXsMSzDw5cKQkPnIVIQogcC99lhXoEhXUS3YrN3Y+9wTJK+RpEjNMc QYVIUJA4m5azeKrI6diky0WvqXPS7r4bmFmCi+HO9dLGF4S0tm3139RZfc/RAw8zL9h3 x8XTNdEOy5KbpV0z84Eph74dsPdZ0ImwyCecoItCCBievB0L5bHV8xjCIMVxyNVwr5Jd GU34ZL5jcwcNb2BhAjLFqVsy+y0yO8gfBMsZ557fIsuJrVh217JipoJ4R0PiBQUnDVEH X2D9eKLaFOl/mdty2PUxPWOxTAKBb+A4IgfPWWVmvkzFrAIqbTJxXona4MUwDmGSekaW rizQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333292; x=1727938092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=sO603y8NLdWTDi9ba6s3anAek3DkXLoTktqDM6eFh51D7K8BFwR8/+TY8erfZSfKCV L1EJsqywrPBPd4lPiYXcs66CEMQZpZRo7Y+lQzzjr5f7JuyAFMy2bZ+WGlLotSoztIQS iZwdGmo0xifoHpEJrSYF3zjjyZv7lbebGfSL+k5LeFkGE0VV5uARQqT26Ija8rRTgMbD vDycYkMzsfTFRwFwk0g5ab/03EcXH7QXydIRt4pJoZRHk9Z7y4WlECMxXKZnarD35AtH M0ITunzL4sZyJsFo0pbn/dagcMM1bHzlxUSGTAQMkvxgFGz3YsMVpqTT6NPGlAxy4eTf 60dQ== X-Forwarded-Encrypted: i=1; AJvYcCX/l9qa0NZch+E/qrlNSCj73hH+bBhQ55E/rgql59bpt6cen+yBuuT1sU/F6x4rIlofff4qGTMZrA==@kvack.org X-Gm-Message-State: AOJu0Yx4y7kO752AZ9qRzFbha5kRl9SIlMgyZMQZTEcnlDTP9caE4SyZ iD7psFKT3AehmwaR6JSEnttV6Z6VxN41ctOprSQrk2boGUEzZeG+fMxKbsftDro= X-Google-Smtp-Source: AGHT+IEjiufoSG+9f7J9DudEHL9YuIq0UaJx7YZ52pOk+qHYx73DXmea6YZQnzcRnEEhuuY+i+vKHw== X-Received: by 2002:a05:6102:e06:b0:493:b055:bf56 with SMTP id ada2fe7eead31-4a15dc39965mr6802341137.2.1727333291940; Wed, 25 Sep 2024 23:48:11 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:11 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 11/13] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:24 +0800 Message-Id: <1530e8fdbfc72eacf3b095babe139ce3d715600a.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: ywz9jaf8okar1rksih4zripdnweqyeae X-Rspamd-Queue-Id: CB6EA14000F X-Rspamd-Server: rspam11 X-HE-Tag: 1727333292-62064 X-HE-Meta: U2FsdGVkX1+XqEraoeTVfl/PN68Qt9SWyBv1wXN4j5PNrKGoJi8GfRLRCiQGSWRY3tDJDrShUAUQvA1dQ6x6C1vv1ul6CKMPVJQ00hGLi2luGM47yo5GvyZU07bpKdCBbQGVq2EyRedaStynCkXb2xVPfbwMX8yLn3VJP7it/h7ajcewc9FCOnNwjfC9Hp7WXF23sYSr8AZ4bt1eE++fX48JjkL5cZH6CnylJwUBTFdkdAmTjc9u1q3dE+gVtoC3fXNtWZQuz0NFCJyqpi7COL7Bs+kKWw9cI+RGdAfjU4YPWPIFpSpLFDG9s4HCwJkg5FfDVUVG6iqCXfG0dscUCvycothiOToF01NqUVgwuzU+dR9yHuXgQ7T4GhRD0mR80LJ/r41UFhk+mN3FB/8++7JfbH/AAs8Y+vVC0cRvxJ2gAYaZiBaaRAf7lGb42wC/dHUPIwFlfpmmrhbnNsleiDUs3VA4VRSRU5uMRXK5BDfLORW2yHD/rnBtZ1ZmoaOrEq7QBNDmV+PxMvxn/e8/pO71yk9BiH0enu9cUv/ETajs27h1aWqU9pOG6s3M4GN445hkIR8pdfTUSDMC3v8NogstJceN7mRgauwJKtykerLUOG+IAGjKgRN7bek5yV/EqQBjX+SBJR+URKDh8iqCr8lZ64QI/hDKpooZAyv6k1mQykpFqSzaVqsMEKGRYcvdFZAmdxl87qRdOf9r2DcBiCOUH+u4f2uu2F/j77tzLjVizrPXJDhev5x2Pfzx9Xyml6hnTqSVUOw2kF5oLsH62zkxw21P6moebpMfrjoeSpxla1T7lH2Kb+Vlegu0alU4og4W7KMg8amHeSPJH33c8ft+JiAj2QYVJE5ll1W2huu9n0A3B00JhzS+lNE3h6qZoChv2h6pUq+n3R+vRqLuhj9k3jhvpUHJ2iblMW1p+CA8DTgrdXJekf7McxF61nNwxxEQid2Ze4i++uAn8D3 dckxGE9o BrsvNHnue2DxFsXvalsZOMQ1MwMMa5KXDluU839AJzUBDNMgzn/xoSe98W24Dx9rBvCb9zmcnw/KNHSpicPai+2YDxceTQtxEndYmeIBNKE0IT/YeIhrTJpl6PhSfi5xMhv3JxHl6r0cdD6ofz4FjMLilg+nfEqY22EQNX0VCYlluFm0eo9ZnxqhUVAIaiY0fZMIBrXdUhvz6M4srZHrIaeHcpvKyzVF1olwzx1VYW4zsUJv/q+j7hVicoLFAWKuPgp1qP0rzLau4rJvLqhmAv0SiOPI0HSNwRCU7zlMT3Fcf938UgDJRSNkg6oStWevuLSzYWhOFpUzn1FqcGZP5oqBNUPZ5gPdrkbhg9Bf8YRMce/TIf/fU7KBWZzurKCeiZRF1pYmOoTdlE2unF67osqwZpyhXr96M/HHwZj+iWZo9rwQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will use pte_same() to detect the change of the pte entry, there is no need to get pmdval, so just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ce13c40626472..48b87c62fc3dd 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Thu Sep 26 06:46:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52AD8CCFA07 for ; Thu, 26 Sep 2024 06:48:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D63E26B00B7; Thu, 26 Sep 2024 02:48:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D13586B00B8; Thu, 26 Sep 2024 02:48:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB40C6B00BA; Thu, 26 Sep 2024 02:48:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9B4356B00B7 for ; Thu, 26 Sep 2024 02:48:22 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5B2F41C6B9E for ; Thu, 26 Sep 2024 06:48:22 +0000 (UTC) X-FDA: 82605960444.29.6E25C14 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf04.hostedemail.com (Postfix) with ESMTP id 8AA224000D for ; Thu, 26 Sep 2024 06:48:20 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PYNhF9Lp; spf=pass (imf04.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333239; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; b=PtZCn/LxEE2ofpEZ8uOCEyoOxEnHh2d3PSy8Rp2RIGsRI8ATGn19BI9jWrmlmnzUrOEmvv L6U3Kp5GuIv2HyT0BejnA4xA1SvKfz82kNQzU8Y6wiNc1fviQkZd+GugTwf1jnrTF1oyWa dxP9favonTnSO1Opp8/ln75taBFZ6wQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PYNhF9Lp; spf=pass (imf04.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333239; a=rsa-sha256; cv=none; b=ExpZs4RZ09haZ3Gwlvc23kYg/0I4yG4JYT8XiKBkhINWZsW5EeRXmB1cu07tjtfvEreiBd p8rZG2dkGs64h3JGy2yPpVO3wsTOTaqqS497I5HtPRz2kRXLxmhO+HmrGSA2ui6COI6qDq aLUvod3ems1tZxXwiGYnqxjJcfCBOJA= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-4582fb3822eso4032111cf.1 for ; Wed, 25 Sep 2024 23:48:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333300; x=1727938100; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; b=PYNhF9LpQ/pdM20PBJAVk7fTA1EHXeaYvEi26PBHRDATn954o635Q1AjlwbJZOg7Gu gKDa8VY3EHmuw4In/5oitn9tUqAfxdjEkJ9gKG7qsIC9GPAZ5W4NZT9gWwKVTGRxgoEe PlfUck/C6XEnITq6c6xmIqKOBKSLx3GAVL0SfyMZUYEhdOOHomukKJV41BmGiolc2OzL PWdBlQmr9PgdX4GKkf2U4kCWMXTJOgpRHzZmnurh38GHzXnwx4e4fQ5yRGGVnyJOmIbH 2yhyVhQ2cor8062ajEnQCuxxUH4tg7r1+MWXAyXsNbeb9tMxJcuGg6Fk9LFXwXIDJ69c cegQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333300; x=1727938100; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; b=Sj88/W4tJ2kzbh2WLS86X4jS0W6QVbBZ7kyegk970yEaMrOt3MelhJlkjKNSYzZLLE jkXRnw499O1E8JqNAChTCUKtXJf0lAxzMkpgQA0R+LltGdH9KjtHgBmiDaqBVDNqeew4 03MriVx9gNgEFkd2fb/zIacDnAS/UX2DYO39+7GrC/MEvNmSIdMBZIwzUcrrGgN6xSAx fp82dineHdY2ERPZjKSDVgosP2h9Q7cnjdsbjcGLK9yYlnTwX+HXNZ7LwK+MYDHlpefI gMJugIWA4scralM2hVhMW3lqYsOmLLV3e7eJzE/dhet9celm5rb5NFFBTOVAAWjhBCk+ 8ECA== X-Forwarded-Encrypted: i=1; AJvYcCXzg1t8V+90XWh+5lALjOsIN+5Ky4EcSrgUpROKQA1faQG0Wh5bIF5D38bA+0pR3+fXynkaUbqwCw==@kvack.org X-Gm-Message-State: AOJu0YzaO+s5a9drFvVEqqpUN2IvY7JxRx0wPRjiH52hS4NF86IKR1+I CjjOOOzK+gufI3kmPDciMJgakNIyox6+IQX2nAYhQZSGurhYp1W4o9uvS28p4sU= X-Google-Smtp-Source: AGHT+IH0PykjARYOlhRdYD8Ei00ntSXkZvlp+w/DvL/wK29HFMzX/PK1qWz6iq3HMQhndU3PBLr5iA== X-Received: by 2002:a05:622a:51:b0:458:3e20:65d3 with SMTP id d75a77b69052e-45b5e06627amr81705351cf.60.1727333299714; Wed, 25 Sep 2024 23:48:19 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:19 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 12/13] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:25 +0800 Message-Id: <7e9c194a5efacc9609cfd31abb9c7df88b53b530.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 8AA224000D X-Stat-Signature: 4cu1fkjumaepo6j5hpk7gwhojkm6d44f X-HE-Tag: 1727333300-333487 X-HE-Meta: U2FsdGVkX1868Tn5Dt8e/vpbQTnysbJDU8sby4ebGhiTqP96nNwubIxl5BZRUm8FIzPrt5hOGJgfaOYHnSqyH36a1dK4rHm8SIITZPGB6W5kgTjWJyeeNLaPOcnn3mwx9LEVf1Nt7/2FwqnZz+c+ZEC3Ev4L1zA+bKCqgs2JEnLY32Tk/A1rc20ZATJhOokQKnkisvW8utx0zsUF5tqPtv0+s8NPYs9UEXkI8002HCch4L5/B2RtDFt6KMtYAh6YcZs6NMcRTs/A2cdGaQTp5Fspv4jXKFvVQCwoF4ixDjNFeB8xWgGrn0X7PSTyz+5yjTThLEcX144S7E2wGcBpWkmZdh1gbcj7iECyRZ6+dHVC3yjLtQHJEZ+w0tNxcn/0jRdr1vXplbjt6gkqNcyFYhXBbY8O/s0gqaOXH5rHl3Ts2QusnnkBlE38u6bHQJwXNXuR1zyvTCeV6EbzDBrnQ1NqFldx6a1WOx5/0hoQVSXpxqNYkPWwzII4rK9TrkRGfDUmUf8Lvds5m2LRXnyQTAx7f/uc0PTdDvzAX6DEK7GK7JVsGJpwctIQ6dPyGCc0DUNJNkzsF7VSkhXs527az5fDNAye6f8P6qcn+pfENgDAtDmQZ2Cp7xONlGT1YOM1OfYz9WgatxAU1MlAWypxKynJ7jGceZ7ZRKveGYuD72EDcl80pzn9D8wDQjBoST0dVEdBqkpX0rnVgjLGyIswYoI4lYQ0l3hkyxLFF32wm0/1iDa7OetvBs/vWuhFdUXfLxhmyBU4erev8IjlXOxIwHOzH0cWMXJmlmW4lwDyVoCoZKw75swty98njai+ebT025MCO+2tq4tIu0ZtPKADtiw3YMs5c3RLrkNkypmFIonND6KjvVG1pHx7qELNKo4cMB9FTVvGO1FMENnWybSvT4sCg5y4ldVfKbM/e9nuQnLVdxGA51zcHlchB2mLR1x9gQZ4LZlMJjnGTBynlLR k38ovmrM R8qyXP6jT0BLd6Z3wWT0zAweC6JnxjES2CB2+dDH2b7SGQA6qJqyn8B9+Zwy2brxDS2KXa46WgmAPuNCGHZMyCoSO3onSiSftvIDzp8CGZ8Rtvg0ao0e1RBOnpO9h3/kOtb7AiN54FbpeiP1F7JYBDs96n9QiHdPJGg0jBmb3+selxH4+Fbn9piNSPG2LvIO5oWwsoNZDpGfF2KRgLDuAKWQnpBhI7QosgXihP3ilVpYpHQ6bZQOyMc83EgLnOAyqBj99LqokyWxn+xdxSavBZJB7LywdhTyQUyewpRFKqtoO53WC/v8tA3z99HE2UENa3WGZpYCrAA2uyQXcljrotgf2xgKo3uJG17qzEO4TN5KvRyV3hJJfb9covBTeItJf259wiAQkxW8UyhNsWEo9WrQgnVPPhwL3VX7m9jVU89V4pndoQo6VuZzK5fjnNXAFC54pzOj9AIQnHzA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 749cdc110c745..bdca94e663bc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3375,8 +3375,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3384,6 +3386,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Thu Sep 26 06:46:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13812869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3586ACCFA15 for ; Thu, 26 Sep 2024 06:48:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BDF056B008A; Thu, 26 Sep 2024 02:48:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8E816B008C; Thu, 26 Sep 2024 02:48:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2DAD6B0093; Thu, 26 Sep 2024 02:48:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 849776B008A for ; Thu, 26 Sep 2024 02:48:30 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EECD881188 for ; Thu, 26 Sep 2024 06:48:29 +0000 (UTC) X-FDA: 82605960738.14.3FC267D Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf17.hostedemail.com (Postfix) with ESMTP id 312EC40008 for ; Thu, 26 Sep 2024 06:48:27 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="BxOfjuV/"; spf=pass (imf17.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727333147; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; b=SqTQg4zsswiTanZjoBVrIrKhuuCAFzlFjCUx/gRXKSATDtr+y4/q62FguqiUhzReqGr584 6CAOLsC2v4wTuJHjt++EEIhRrZFa/CPSyJUISK6W16fFcQCHedv2rC/JPnNRlI+pGudXNX u3Ina7Xno2LtY2XYFfvf3VTpDqD4gr4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="BxOfjuV/"; spf=pass (imf17.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727333147; a=rsa-sha256; cv=none; b=zypSXIjCwHHI7i2m6ogUdsA31lD21eIghEApXk5pokFGEioCxdmoxiqL1E3FV1qCrUR2cZ 4K1I0w/4eZIjLNoC6ozHt45UNiD4DVP4MaJzNC2QQz7sMLp/OM6GDre6tND0WslzfExAtW fannycBvqtaGreLAN5fHaudYeYnMeog= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-4581d15c3e3so4777661cf.0 for ; Wed, 25 Sep 2024 23:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333307; x=1727938107; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; b=BxOfjuV/f7oswGPeB1fvFPgKEhZi35teCgZqZ6ggbG/IBw1HelIyCZtokab2WK0+5F 8j5aI5w+pkry98HiAOpa3QyUOA8sdct+njbpjO+8QqNbyGmreZlzKVsbQwoIVbYMtyEB xOlU+CK0u/JLRi4ZcrcleKVjdQjZZSMlckCXs1V460AdQwdGq37De9whkvZATGpjFvOu yE+W3TOBj/1qyo1Z05ygHvW4N6AzQKi1P8lnH3cFw1PQtXq5gbooKpj8DQs9FrxZHX0j o/J//NrQk80FT8xSrZDn0dP9OoUwdyl7DafAOitLHjhRoyI26GhraC+sMrLTqSDCKxSx GIRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333307; x=1727938107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; b=NPWAGdeygclBP1vSaC33b9KnobEy4QgvzWfeRprfg84026xzlh+jyJpKqI/45r1JxF D9OlpauucWT3kJaZNFESNbFq+FjYRuK1Rb+NEoL8cUaahuGNTN6eDmHKD99hYhylk9ei LcrRAFc++kBD7pex3VmSLwM22z8tXulmVqsw2+B+qF11V7AJphFr1NuoDSYJPRvmb1Zu A9eMbAFDDznpZnmPj4sGR9N3o1/quQgc06fzldLhKC1UVtGDZKfvXPRRHmvmj0i+ZRyt vSQkAw2dr+Nwdc3isr34SKY2OI1r+BAcN898a+j8OU/MCdRGyiniXzYssvd7KcvtFacQ GDCg== X-Forwarded-Encrypted: i=1; AJvYcCVEVa1JeQMcmnjqC43w+1GevsgbqrqZPdeFiUspWC7AFIt36MPr4uyD16gDjYEEo/nyw18DKszSkw==@kvack.org X-Gm-Message-State: AOJu0YwYakoqGZAxcRuiYXHSnN1ImLkQQF08Ta+bi9e+R/qIE5jPE8lo IhPDWv7rf2U3Qtd96GnaQU7Mvo6jrKr0KIE2jpmy1339GoRJERoF95xEUpQbJDQ= X-Google-Smtp-Source: AGHT+IHnYRThLjQtuqQ25g++f96or6jGXpOGKBOHjBYtOEzyfC2Zv3m6NDV/6g7eYEpePNjEvQY05w== X-Received: by 2002:a05:622a:2996:b0:458:37c6:46fa with SMTP id d75a77b69052e-45b5def2036mr66535021cf.28.1727333307294; Wed, 25 Sep 2024 23:48:27 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:26 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 13/13] mm: pgtable: remove pte_offset_map_nolock() Date: Thu, 26 Sep 2024 14:46:26 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 312EC40008 X-Stat-Signature: irqsfxhyoxa65y8t9sg8thk5gxmsmytk X-Rspam-User: X-HE-Tag: 1727333307-205162 X-HE-Meta: U2FsdGVkX18+JBLx+/iWl/4KfmVsEBfylG+xpNVydVc3BXaO196qih8ig+wXZEsrecRLXSUklpsD4m1z+gxj8KlvUJJg67ySUGoG0W72OpErjYGFNZMPXgyxrVyDt+i2q4eCiYb4SLqAdcaQlJ2hY3iWoyJKD1lSKPXYCsfFHukbE6N+qSUbW9Aoc1A7hWVAoFA7cS2FZcskernXCV03aGyapTanWY+96fcXosIF7VcgJMvniD3A+sWr5QXLOqbIfGi/dwnqkf5FHLfiIlzTXDI7cVG7ctrw2Q4lXiat5ikr/i09tnBkaXq1V6jdR605uzV58kHnoveNtjoeQPtztI9+xs7GhVcTDT5aFSN0EO4JnOWhuDxpdCi1uUks/HUeTxQwftijtBIebg54Voj3B/ZOz0+75ATyCQVnGmLyVAip/ZD57KqQRtwgNBqamFiMpV8eVbEcuKExM9TAF7NokoDh9kA2iow7fZoitzRvORUD5unINoknyCCWyJwgV8XeeFcQkMZHfb8naC12we8fM51RuvEu/+zW92DTTTXtIGrfWZgavSrAXcsGnuLNu/SednKpQ2QMHwGZtubtkt2Tgrl/fbvxV/hJkJFjcRpoedGB3bhCXW9GKLH4b++qSTHfVYBots/QT/cauRs0vRgIXpSaPrUiIoBIcbAvOI6my5wvtN1GZFfo9MRbt5KBzGEIO8N4HDiv8ZBZ7OG40WPlSkQbBWVSZUMOdg1veFKAFV1JORPJH7bR/AChWbc1xWBzRWe8Uys4iRz4wicLlnuS6BfNo+isVmtkoWjKe2jHZHeUR72BDugRkRRVLhz+WvjP3Oy7MK9KRfEY+4NXl4GJS85Sw5CvJeNW3JFq6Bi8fhEUtKk7nhibW0xnIQaX4TGcpvrkzSuchtjR8gExSwgi6L+sbQLeFIBISoaW6FRK22CKEbakY4YOYElFExPgD9bBb0g7SCZXdONpefvknjl nzEBPkNj C5rX3ibHolAyajayowALJLy/sDZS6aoLHUBD+rL9MP2onJPupu04ATVPWpH7DzaBTl1r9sVB9atHVL6qTVnPwdmdXL9W6M5NXCYdQ8mS3aTGJEM4KhmGu+jDCVXiKDrA0KLPUomnQVAFbyU2N7x0Svmj3tQKOvEX2LXm2ar9a3lVG6+MgwzJxBC6aJ/0LoO+dHtQ1a2TFSO0ClPgfU2PwsUxlbUFXjWSB2OLyHHh4hezjvcbu1ZfQbee4igo+nA4Zr/7FoL0+WQXafrVJ0rojZRFfdearwec9kxc+Qan1S19XvARdwYxpFRCAxxvbK32qZvyqlmALMc3EuTMqlhs0yHcoh4HqfbUuwm3w8vD1OG7XhUD6aAAyIxGLXStd7yodvAmY9KyyoS5F24kxctPv3Sjt4r73OtunQtoHOFQqnY+ZYO39X0r6PWDhAz9ymA/BURWfPAlBH5jozNM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 46828b9a74f2c..60a55b28cf679 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,8 +3015,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index daa08b91ab6b2..5297dcc38c37a 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -372,15 +360,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); * but when successful, it also outputs a pointer to the spinlock in ptlp - as * pte_offset_map_lock() does, but in this case without locking it. This helps