From patchwork Tue Sep 24 06:09:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13810123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14697CF9C71 for ; Tue, 24 Sep 2024 06:10:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 615306B0082; Tue, 24 Sep 2024 02:10:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59E0A6B0083; Tue, 24 Sep 2024 02:10:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 417DC6B0085; Tue, 24 Sep 2024 02:10:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 21A116B0082 for ; Tue, 24 Sep 2024 02:10:33 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 70FCE81823 for ; Tue, 24 Sep 2024 06:10:32 +0000 (UTC) X-FDA: 82598607504.25.D0ACFA5 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf12.hostedemail.com (Postfix) with ESMTP id C019E4000B for ; Tue, 24 Sep 2024 06:10:29 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="T/GF8UQQ"; spf=pass (imf12.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727158111; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=FZ+6J9fpvc8LtEIEryLlJSsbAdd45v7edjcT5rDHpkU=; b=wpUxZ0togLK6iDmwwwy5KzZ7h4QRD0OY+w2suDY4zgtsZVd2WICGj8FPASUtiD4KbeITwp 22Ur4JgVrKU94of4DC7kc97srJ6MKiBXSppNIj9RLljbtJlE2PIPdyYlFpHotl+5k9wDzU YzbXYUrNhSrpzyDu/CxLZjDccJI5LQM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727158111; a=rsa-sha256; cv=none; b=SK3ULF13/vunTiFzFbsytvPv3oS/PB9uSyh/GIGhb2UDqrprMfrg7Kg2U8m54doz+GlJp4 SIuKp/ar5568Ckx/qAxMX6gIdAgHaxtIxDYyGYj3mayEmJ30EEsilC1hFSsdRwZJ5iCn6i pxnvhnWLAFde6GwKlyPSE5NGNvvrINY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="T/GF8UQQ"; spf=pass (imf12.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-20536dcc6e9so32521045ad.2 for ; Mon, 23 Sep 2024 23:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158228; x=1727763028; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=FZ+6J9fpvc8LtEIEryLlJSsbAdd45v7edjcT5rDHpkU=; b=T/GF8UQQx3gmV7T2vQxaZnE65fP6Ol7u7GIkRQREce43y5+a3gwRUDPlusEzHtMdv6 x35XRW10RyGMUvhAeMB5DUdk3ALB+QwxhPGhFDZkVqsKMH6/tk5B/aFV7i7l6gRFTmBE HxL3zKJa7hAKCeUlVvvJdVePWQ8t6h32gARTPN2bb3PHUB1VqcyQXiTKgQbgyyOeNi9s nLvr7AKPcqWuQtZrgN66VlHJShaUQJ+rY3iC36pC6DffiAUxH9blfQJPzcGxBqiVnskZ sYPe9iDKzSpdt2GcxPdgpMjhI2gvOxzrLeUhD/hFZiU5fuFPkb0LGe+rJF3Mp3qId4he TAVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158228; x=1727763028; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FZ+6J9fpvc8LtEIEryLlJSsbAdd45v7edjcT5rDHpkU=; b=thDU44CloxFacaDxHHqrcaq1mhsC2dbjtOp8U9QW7JH34r/C0rxadn4xR+f0ae52F4 TOr8mvg9FN5kaDmkmlhF/dlopiVT0bc+p7LP9gRgBNg5BCaJqnuM3j+3xinnRm1aqNa+ zUhrD+tvkcjyqYlcrfErkvwt74uFJu6nhaI/TWghb35art0g4/d/zMLuYCDpYtCsN1Nm zDv2ePd635vG+gRFxkQXBMHr8H8Y0yd05hrap+p1zKmt/IIR0d6zgYa+NU6NB5fJ9y97 GBRrc6Cu/0Rr2gXMByma3SgRigwhwmxrRk6u24KNiUjcw4aNzRrNfJ7OZuKa3ZdhoO1H JYDQ== X-Forwarded-Encrypted: i=1; AJvYcCUFT5ptDCo71HqSbkN+H71IqH8lvxVWV6Ao2tRLV54F+/6XDBU5/iO8f7ozvES7lKIkMSWh1+eAXg==@kvack.org X-Gm-Message-State: AOJu0Yx6393UujITPgp6mdxJD44dEdLEfpGwqO64Rbc4FgD5XN6OUUYr L1NGAc+jnv6pKp6JC5hExogbzWIybPcflTqyNfgiFZCe8LfPGbkV0DeLKpTNMD0= X-Google-Smtp-Source: AGHT+IE3JgWdRJx6WCYXen89wPhryGlCmdGWyOg8hF+XH/7tbK6pMmGNNEBSXyybrad6ihXB6X7r6g== X-Received: by 2002:a17:903:22c7:b0:206:a1ea:f4fe with SMTP id d9443c01a7336-208d832a9c8mr174328585ad.10.1727158228146; Mon, 23 Sep 2024 23:10:28 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:27 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 00/13] introduce pte_offset_map_{ro|rw}_nolock() Date: Tue, 24 Sep 2024 14:09:52 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-Stat-Signature: podmxrijgbqom7etutxu4ik937edoyqb X-Rspamd-Queue-Id: C019E4000B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1727158229-491745 X-HE-Meta: U2FsdGVkX18Psz6Bd3TyJU102wd3hmlN2QSeHP3/ryODlbj+VQALelMILvwunGMKqen2ydfEzM7L9wNjt4UVvjYLSy2CWVDXs+0e9fCkPDSaPiWgdpj5+r4oV876CJfjxSJB0au1GA5pQQ8N4JtKaeSC6FkAA09vlFjqfDxcihDV8hPiBIAcmNxYjIkHVLEMMZd8Vjyc3JFP2CoUWYo7kzXa4g3eWiSQly9GmJ/75+QfdQ7F7uKxW7b85y1vYh1Jvi4prghkd7oG+jkJKOpMzE2zAD9DS+Yic4jxLZ7Kt+1iR/7jVlqknHEgeem67FzJr5qgfYp8a1hsDiCqdQAh9kGdKWjz98j5loPIVbF660z0Lf9Bt6/KJa6CDJYibigbWeoQ8JRBfEK1Lt1z5/Bbj+iFTT/8Dtm66phaIg0i+2aFEK3naI9EzoyxV4uJTzbFowm82owYwYsct87oNPxD4V5Z3xb1sWzM05v1FyR+1q/pB6mYylePYh57HOgWMqPJt7+t5B5FUnoVt5ROlE0SPTJbeKWX5EGedFGG3u78NQg1aUM6v55Ev07D53Ro9sr1ZVlykhWp740qgLmMTpwFW7slMxDheaMF3MgM1h7NbNWveIlz+HkzgrrU7fFNL2PnY4VzCWRKJ2MBAVb7B/JxHT1+FhqykItU47eqyIabUmYArAvUI3Oliad2dQKyoK13iEmjHuXBHdImruofEDd4nSjoMCqGEld0ymrn98wzEJDstfQWzEl4CCMWfyK2TMfgjOAReaj6E2OdESw/1nZOhTH1iPDq0C935g/Y29nsKF4PHsfApPnmoIGpKAhOWXacSmvX7/gykDY0nE2VGUosaktGYpY39yXk+6M11V8FkMJDB95dIDVsLKSkOK7BS+48XiHb5Huk7Nyh5TBxpcenQJ3h2VcPA9bVhKOZZh7SvKepZWXjoMBmo0pf12y8Wjq/bO0hoTS2vDD2WpSjGnb QUEUaFXC VbTClavw6RqW4FcTXq5qxnEYiDpf9oJtRx87hD1g1gQCV44u9DIAhhq886kJSpLq+tfuhvzRWeQ58WpU12/Eib4GGGOc04G5r6aYBDUu1CeBAo6EaApdmNe0kdCPFTduq28IMmgo3OsuQ7eO2P7FxXV8XBtDfYB/+BgMnPZu/5bZxhXSws/5HjqH+tR+R8vfObl3W9HYJx+B4gBP5ht0c5eQmFfUVnZqmUTOTYB2kRPZgiEnZ+8uzE/LF9P5YurmDSPBSM1wyCwy8OPCJCxyl5GJFvZWRvsSUwq0K9b9cxSLsM623aasyipUw8AVkeKm5wdVnaZxsLagX2cHkuq4H9QRgoqlmmHjZMLssLB6SIvVN5win7jPL2rxiGXWfqalLMuGrptsy+QbgKXdw/F9udv0qnw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Changes in v4: - arm: adjust_pte() use pte_offset_map_rw_nolock() (use ptl != vmf->ptl to check if we are using split PTE locks) mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() (move the pte_unmap() backward) mm: copy_pte_range() use pte_offset_map_rw_nolock() (remove pmd_same() check) mm: mremap: move_ptes() use pte_offset_map_rw_nolock() (remove pmd_same() check) mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() (move the assignment to pvmw->ptl backward) - remove [PATCH v3 14/14] (will be sent as a separate patch) - reorder patches - collect the Reviewed-bys - rebase onto the next-20240923 Changes in v3: - change to use VM_WARN_ON_ONCE() instead of BUG_ON() in pte_offset_map_rw_nolock() (David Hildenbrand) - modify the comment above the pte_offset_map_lock() in [PATCH v2 01/14] (David Hildenbrand and Muchun Song) - modify the comment above the pte_offset_map_rw_nolock() in [PATCH v2 06/14] (David Hildenbrand and Muchun Song) - also perform a pmd_same() check in [PATCH v2 08/14] and [PATCH v2 09/14] (since we may free the PTE page in retract_page_tables() without holding the read lock of mmap_lock) - collect the Acked-bys and Reviewed-bys - rebase onto the next-20240904 Changes in v2: - rename pte_offset_map_{readonly|maywrite}_nolock() to pte_offset_map_{ro|rw}_nolock() (LEROY Christophe) - make pte_offset_map_rw_nolock() not accept NULL parameters (David Hildenbrand) - rebase onto the next-20240822 Hi all, As proposed by David Hildenbrand [1], this series introduces the following two new helper functions to replace pte_offset_map_nolock(). 1. pte_offset_map_ro_nolock() 2. pte_offset_map_rw_nolock() As the name suggests, pte_offset_map_ro_nolock() is used for read-only case. In this case, only read-only operations will be performed on PTE page after the PTL is held. The RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. Therefore pte_offset_map_ro_nolock() is just a renamed version of pte_offset_map_nolock(). pte_offset_map_rw_nolock() is used for may-write case. In this case, the pte or pmd entry may be modified after the PTL is held, so we need to ensure that the pmd entry has not been modified concurrently. So in addition to the name change, it also outputs the pmdval when successful. The users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. This series will convert all pte_offset_map_nolock() into the above two helper functions one by one, and finally completely delete it. This also a preparation for reclaiming the empty user PTE page table pages. This series is based on the next-20240923. Comments and suggestions are welcome! Thanks, Qi Qi Zheng (13): mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() arm: adjust_pte() use pte_offset_map_rw_nolock() mm: handle_pte_fault() use pte_offset_map_rw_nolock() mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() mm: copy_pte_range() use pte_offset_map_rw_nolock() mm: mremap: move_ptes() use pte_offset_map_rw_nolock() mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() mm: pgtable: remove pte_offset_map_nolock() Documentation/mm/split_page_table_lock.rst | 6 ++- arch/arm/mm/fault-armv.c | 53 +++++++++------------- arch/powerpc/mm/pgtable.c | 2 +- include/linux/mm.h | 7 ++- mm/filemap.c | 4 +- mm/khugepaged.c | 20 ++++++-- mm/memory.c | 25 ++++++++-- mm/mremap.c | 11 ++++- mm/page_vma_mapped.c | 25 ++++++++-- mm/pgtable-generic.c | 43 +++++++++++++++--- mm/userfaultfd.c | 15 ++++-- mm/vmscan.c | 9 +++- 12 files changed, 159 insertions(+), 61 deletions(-)