From patchwork Wed Sep 4 08:40:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8FBDCD3431 for ; Wed, 4 Sep 2024 08:40:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3678B6B01F0; Wed, 4 Sep 2024 04:40:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F0D56B01F2; Wed, 4 Sep 2024 04:40:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16A966B01F6; Wed, 4 Sep 2024 04:40:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EABF06B01F0 for ; Wed, 4 Sep 2024 04:40:48 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8BE2B40778 for ; Wed, 4 Sep 2024 08:40:48 +0000 (UTC) X-FDA: 82526410176.21.84937F5 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf05.hostedemail.com (Postfix) with ESMTP id 3D706100009 for ; Wed, 4 Sep 2024 08:40:44 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=DF0bF21n; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=PnxZ3keyEwgQ9RkNvwyvGwt3DYekuRW0HVUJKFS+J3s=; b=5g1WNQszJ4ZHKXh0RMUNg/7m9I1Jhx4dM1QL7l9qAuuf2DhTAlRBsfdEYjtgqyxzrHygrt XT0dWlBQKtAUeQgYKgAGP2Q4JwG1rXyQuRYNsIx0ZPVmRJO/7I8JmgYwYkW4x5gg0NCO26 nsPgsmMBzpO06+6ypYGsS0MC3vkakms= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=DF0bF21n; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439170; a=rsa-sha256; cv=none; b=bYc7ALr10CmBW77hfmcZk1NqC789xF1H6bWYcHdQEQfywnu3+0mSzLIz7PYUDMpscKa7ZB +Uz9TLhUBihphMqkTJ3l/MHzOttXi9iPLV22ExlHqOHnHbGOpxusiDg/LH95xUrd+gFELi vGZ1Wz71+NdEaT4fl4gDo3h/nx2fMX4= Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-7176645e4daso1893354b3a.3 for ; Wed, 04 Sep 2024 01:40:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439243; x=1726044043; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=PnxZ3keyEwgQ9RkNvwyvGwt3DYekuRW0HVUJKFS+J3s=; b=DF0bF21n9E0+4poYRoNokgkjNE3S8gAYJFbWzFQdEKh7gOjIFivhHpWwtt35BF94Oc IJO6lur6zB7E7wQQpJjUAI+meMKoMf39LyKOsq6atfbXKTIqYCxlebQbFkKOykNHEzRB or6DObLbWsdvW0i/3c5djaLdor/tDQbHpxBUcYlZDJEE3o1B/5VzHl6oxbjACotcsOmM OBehFn3MSuHozkCMj7hSEID1AvFn4vxjxdbed9B85hsJyhn6nIrlW2ArJ5N206mpskJ1 C+TQWh/M1fF/ajjtvHbwMWmpB05AMIQg1h1ALyIvRqpWWeSkvbF4yljri20UDFp09pUN pLlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439243; x=1726044043; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PnxZ3keyEwgQ9RkNvwyvGwt3DYekuRW0HVUJKFS+J3s=; b=H0TYUnw1h3MUD/bfFuSs7k0p39hSsXFZl1K6OnooxM0DpCs1zw0jyown4W3OrzhRkI J4hphJNljv47PWOnEphSNw0dtd2B1tlREaFoH82V0i8Wsd6F4IDBLDHRw5jxu/xVWvqD AiuLxj0SRw0dogtpMUYtACk5L/yYtoBfgDwtAsckUKHD1Ovc5Y1S1r4YlpfzQj5ngE0b GsJXo0dEP8Z4jnxXxfEQNi7N3t2S2OcTNVGRdsae0aAG/wZBRRVDpYoq4P5NqQ6lIoLe ZGQ8brFz0k7e4NJ++kleGFPR9WlVp9+u9QS6ecQ68L9pNXvBx7dQlWjLk+k83k+WPCji h2oA== X-Forwarded-Encrypted: i=1; AJvYcCX7P2a6cBMccLCgSw5hR0icYEB0KD5gBlnz0WDZjUWKSsU+wXat3EGCUr9RZAForKiTGsd+Ber99A==@kvack.org X-Gm-Message-State: AOJu0YzoMpno0vrn3JEdf+BrH81bPHq7H2Imzarwwy4B6ysu5Jocq8Gk r8JK3168VzGhEODFMo+YOGRvAMoW+s1I3RBbHj2GEQJI2DLf2j28C8A88DQj62M= X-Google-Smtp-Source: AGHT+IG+iEKF7FtSjHKinQSwDvpK7gnjP4wMOie8wNTxXeNBeSlHHCqUvqbL69VXmYEDlh1n7ZOmCw== X-Received: by 2002:a05:6a20:d503:b0:1c0:f2d9:a452 with SMTP id adf61e73a8af0-1cece4fdcacmr13163272637.13.1725439243372; Wed, 04 Sep 2024 01:40:43 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.40.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:40:42 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 00/14] introduce pte_offset_map_{ro|rw}_nolock() Date: Wed, 4 Sep 2024 16:40:08 +0800 Message-Id: <20240904084022.32728-1-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-Stat-Signature: s1hdyzeieg5aizrjep15aujye3w4fai8 X-Rspam-User: X-Rspamd-Queue-Id: 3D706100009 X-Rspamd-Server: rspam02 X-HE-Tag: 1725439244-702626 X-HE-Meta: U2FsdGVkX1/tGtUHPOUBTc/l0MlA3jC/KlrOlKfJ4gUjspjvELe8Stotn974uhe7neh+aBmL1j2/rxJaY7/txChu3MhJILhgf4a+zi7DE0d31WsGLeTn1wr6qQx/tges/efe4N3KYm6ewN/Nr4gVd54RocV/x5J9BodLpo+/l0ZuUyM0MHFH1IlFtxcHL/oW2r0l7yDUwQk7QlZouoo/kOJK9ekAKB0GqFC753f5ib3ERH174IVKHef1Sibb5kotfmaM5gR+7sDOI1ncDtTONMIGig6T1ut+ZPMSwsA5gXa9LZctYcZYChgnhwJRKXCSNBc3udw5xM42jrZUDdlfJ+a6JZaEelaelBrzvjtYVKP/TwsFlMWUFjQadkrBe9t+Q93XjSNLGGWI2poZIZmbo4aWbzn8P/5nt/RsIT8lm8yNw/BrRSzCoyiV0KlJJ7zSSXRB8Hrxp+pPbxm+6R8dQ7hTm9zSqH1MIHN1AGDf6zAtsN+v3EjG6rTwo6Bp/0WXS7iCkLci3fb8eNrHYfKACC6/YMPbU8hA8O7oc6v9k287lQ4YQLFStEh/w6OzsE6cuue3Nl7y+IDWel7oCSxv5xl64I79c8jVJUDT6luO/foHHUekqK6anG1i2+rlm+1lmm1FVXm0FgvXi3yRS2UirTxqBD8HxcvxVxeSM8AgY4SsIVOW4ynj8UbQtWEjlKYu5Fhtu0yrP10kI25tDz0h3T6D4ET/Nr+ywKZGvtdU2zCJsZ12fCnJu6OgsL+7MkzDTcg41FkYabbP6bW4xzJZLJwaGRJqrAv5s23ARbDVHwABkGpHQlYWUdbfeFwOevoQPWT7ng/9uLqkJD4FmRsGRqVeQPeaOQv4lvJOQGdnzbgl3hhfJX2g2D8aPGAbO4CWDDFzKdo2A56oY/HwYi2hdPMKiu9eTHBAd1iJL+4dBYUSxfFNQofa1nOwlsq4+UHeM3jbRMf1IkGgXNELw6/ BZlGXLvI +w0Mt/L9r0IeoDPiqtDYWUUDtGhWmyljJfTF8YHPTsMcxHS6uAvipwtQDANpnN8Bc6k+2uvlNTeoRNJR+/HXLE+PWSYxqw0T0VaQiB+s+6oEZS4g8SMtpgsBGvV/2h5Y/I9BNTiWdQBRnAQgY0VkZ7KmJZ39D11Zgpfg72ijE3DPErTrFMOdLXYVTLAiLBhPf+vcjmPzfPyLUR9aGVuiy9qMUxSpixdeQ5qbTnh7nu5D6zUPHyEqkg9wO7WC2sTY0DGZvXZQTv/AoDg/XBNdz5nRMVd7rUDI5vFjJyoe7pGgXOFsl/9WQ7DbrxBKPZhz3Yg2b3mc1clo0FJrHb0fsNxypxZ+RIVya+rAvCs2l+ux0rqKiCTh/L8tfw3JB8qNoVe8PodxjseXEyel7Cgd9+NS3Sg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Changes in v3: - change to use VM_WARN_ON_ONCE() instead of BUG_ON() in pte_offset_map_rw_nolock() (David Hildenbrand) - modify the comment above the pte_offset_map_lock() in [PATCH v2 01/14] (David Hildenbrand and Muchun Song) - modify the comment above the pte_offset_map_rw_nolock() in [PATCH v2 06/14] (David Hildenbrand and Muchun Song) - also perform a pmd_same() check in [PATCH v2 08/14] and [PATCH v2 09/14] (since we may free the PTE page in retract_page_tables() without holding the read lock of mmap_lock) - collect the Acked-bys and Reviewed-bys - rebase onto the next-20240904 Changes in v2: - rename pte_offset_map_{readonly|maywrite}_nolock() to pte_offset_map_{ro|rw}_nolock() (LEROY Christophe) - make pte_offset_map_rw_nolock() not accept NULL parameters (David Hildenbrand) - rebase onto the next-20240822 Hi all, As proposed by David Hildenbrand [1], this series introduces the following two new helper functions to replace pte_offset_map_nolock(). 1. pte_offset_map_ro_nolock() 2. pte_offset_map_rw_nolock() As the name suggests, pte_offset_map_ro_nolock() is used for read-only case. In this case, only read-only operations will be performed on PTE page after the PTL is held. The RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. Therefore pte_offset_map_ro_nolock() is just a renamed version of pte_offset_map_nolock(). pte_offset_map_rw_nolock() is used for may-write case. In this case, the pte or pmd entry may be modified after the PTL is held, so we need to ensure that the pmd entry has not been modified concurrently. So in addition to the name change, it also outputs the pmdval when successful. The users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. This series will convert all pte_offset_map_nolock() into the above two helper functions one by one, and finally completely delete it. This also a preparation for reclaiming the empty user PTE page table pages. This series is based on the next-20240904. Comments and suggestions are welcome! Thanks, Qi Qi Zheng (14): mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() arm: adjust_pte() use pte_offset_map_rw_nolock() powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() mm: handle_pte_fault() use pte_offset_map_rw_nolock() mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() mm: copy_pte_range() use pte_offset_map_rw_nolock() mm: mremap: move_ptes() use pte_offset_map_rw_nolock() mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() mm: pgtable: remove pte_offset_map_nolock() mm: khugepaged: retract_page_tables() use pte_offset_map_rw_nolock() Documentation/mm/split_page_table_lock.rst | 6 ++- arch/arm/mm/fault-armv.c | 9 ++++- arch/powerpc/mm/pgtable.c | 2 +- include/linux/mm.h | 7 +++- mm/filemap.c | 4 +- mm/khugepaged.c | 39 ++++++++++++++++++-- mm/memory.c | 32 ++++++++++++++-- mm/mremap.c | 20 +++++++++- mm/page_vma_mapped.c | 24 ++++++++++-- mm/pgtable-generic.c | 43 ++++++++++++++++++---- mm/userfaultfd.c | 15 ++++++-- mm/vmscan.c | 9 ++++- 12 files changed, 180 insertions(+), 30 deletions(-)