From patchwork Wed Jul 12 04:46:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13309573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FF54EB64D9 for ; Wed, 12 Jul 2023 04:46:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A52506B0078; Wed, 12 Jul 2023 00:46:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D8458D0002; Wed, 12 Jul 2023 00:46:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A22B8D0001; Wed, 12 Jul 2023 00:46:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7D1786B0078 for ; Wed, 12 Jul 2023 00:46:32 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 488581A0509 for ; Wed, 12 Jul 2023 04:46:32 +0000 (UTC) X-FDA: 81001723824.15.8FDFFB0 Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) by imf28.hostedemail.com (Postfix) with ESMTP id 59484C0013 for ; Wed, 12 Jul 2023 04:46:30 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="J1OyC/Pz"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.128.171 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689137190; a=rsa-sha256; cv=none; b=suKm1PJC/IJfrgN1uPaKVY9i4GDpKm52E5wyvHLH/BL/Se6FwzCggPI1o6iBCIg6CNLToZ tAYEFwarHmmHJc9IjKr0h+k5CpPVcxX2prkFYoecHW38VhK76l5k/yIhb7isvStDzbyxPj Fh7Yd1RbzXs19qrOY1zOvd5N/Jb1V6I= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="J1OyC/Pz"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.128.171 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689137190; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4LOZtvGEF5g0eQvRdR0fiwT6tMgm62bdW0dz/Hhyqlk=; b=c66fxRa8aMGvuw2BIyj5sgn+ntnqHPLfnQ3cMYlxLW6Nkx02TzjQpzCi+cizO/5JAnmkEk 9/WJpqzvXFxNbDPqTE4EULDgJufyMYkoKQqvuHpola1PQnD3h1mcN13q4kPoL4TzXIHKeY cbSN7rVw1TaMb6nY9SCgfgh9h4cGqLw= Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-579ed2829a8so68751727b3.1 for ; Tue, 11 Jul 2023 21:46:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689137189; x=1691729189; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=4LOZtvGEF5g0eQvRdR0fiwT6tMgm62bdW0dz/Hhyqlk=; b=J1OyC/PzRxAcAnxReOGyZoimdKHV4sne6EV+ZF5GnSzIGkj49DIxf8CJZXSiYaLo8O rQDVnXnu0y21FRcOCZP0OeEhb8w1Ae51u8RxjVwTlnPwlHyRZu5pHWxjcdG3sLbNOkgm l9DRu2V+r91JT6sg8uxZ0xdueMEWKHVeYHw/ucTn/rvDkwQeKDUMIXdd66/4pn+ZPDqc bbexzf/2xXArSFmFzEjH5dtsWbOgASWzeyhuG3Z+bgzkYYJreHWjygSN0MaZVhpwfXUc /vz1Ww9YOcG9dsJAekHocNmUdgaeY/Jq/K6dI5tFUgzSXd3eubGhE9SWhC2zJPkylOJH UPbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689137189; x=1691729189; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4LOZtvGEF5g0eQvRdR0fiwT6tMgm62bdW0dz/Hhyqlk=; b=ZVacCtlxHFm2GPN+2sFr1Kxo9rAasklhIpppaW60ThZ16+eFSTnxx6CiaQcLfSnGGK 6667+m5PXCgm9MxiRtwY8itzx+oFDwLrTrPNvHspKPtApqPx0tdCbvvmWr3hCxDDDt5a hNHa88osHj8+1+o/P7/tcDGL/shnKCUIskEg9hE8gpeYYOCs4OnctGLWmTdZBmUThzFG i3nipN86spIlHljyxa1eVPwkC2OTXFqfT5DzHFO2ihDp9WAYgWhmcrtJixqFoGJT2Kr2 JE6ctaoiLFwYX192IIpK7vFBHU5n8WSR6X+RdBdFO26+mjRmDYYI/+djR4G3EbvMqKmq SGug== X-Gm-Message-State: ABy/qLZm6LcIntLm3dN7IciO26FM2h6l7gOY5spIygEJ0imgRnQV/ZUw gMk8bwypdR4wSKi6V+OImtJfZw== X-Google-Smtp-Source: APBJJlGajCIPfwdsjGp7sM9eUSR1neGCrgzZWT479LGOvJujy6hRuJqugeEnCaM3mBFakTQjRwJQ+w== X-Received: by 2002:a81:5dd6:0:b0:562:16d7:e6eb with SMTP id r205-20020a815dd6000000b0056216d7e6ebmr17894738ywb.40.1689137189022; Tue, 11 Jul 2023 21:46:29 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id j126-20020a0df984000000b005772e9388cdsm969335ywf.62.2023.07.11.21.46.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jul 2023 21:46:28 -0700 (PDT) Date: Tue, 11 Jul 2023 21:46:23 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Gerald Schaefer , Vasily Gorbik , Jann Horn , Vishal Moola , Vlastimil Babka , Zi Yan , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 13/13] mm/pgtable: notes on pte_offset_map[_lock]() In-Reply-To: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> Message-ID: References: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 59484C0013 X-Stat-Signature: 1n8p79xdnncn13sgzgwtrrk9a4tehp9e X-HE-Tag: 1689137190-186703 X-HE-Meta: U2FsdGVkX1+MPG7+8b2W/auCaO+3K+dBbW5z5yJUQNdpEl61u9KDPrGt9sx6YUM7owGRHuRlGkBWRrhyOYiATAUIit+3Mnj8jZHc3LJdz3YfwLT1/Um7eshQtTE0jAGJRcCF3u8e+g8YjgCyXpd9BP9UyRbvusjSvcSa6O6tmmvyyJPzrFH7OdGEWecCmqoKwHpYDat2ljhYPoUW5EDisUF6qbi8P5DYCcw2eB7/watLUlarl4kQtqySsIyRnDFvpUroNxqCcbSeuP8xkWOD7GdYc2jVNIdx3mmzQnFxISB8OsA1tG7c8AJQ52HqD9rLLgokPRoTX0qWe5awhib/npe0uP28YixKkpcvB2R2m0tqErzUbVfOssw0iA0IZ0SpRuS47H0F3RPUqKHOhnkb8gONY+SzV/Ptm6bazzFYv2pgD26OsqQGeguHsIt4AzbUSsUJr+nBysgZ943lYW6RN5DDRX8TKIPY9yRp9uJdEuwj6lEWuLoRXj/36IHxr6z+mao8+3Zoih+UKezl9mVfLMXpZny0RSYqdF4wSxwqPFwEu3yEJaKHteWUHnFWEVQeQhrO3bdu8BUurJnVpPeH+U5+f1tqAPZpdGtfU9VCSTHM9C8U6sKDc9fJipfac53OD3lBptluqvr6mDCzxnQ4HrCpwQExH3Nkw22Qr6TyBhsr4Kk/bcq4y0q0x1iBCDD6kN2IuLYOTFEhIxzAtBDInYZPBt5QZUjNdMsNJ4rwL4PBTy+SpIwPEMdjAY7iqMz3GNy4MniO6vrtZa6O8drQJvZkjeA9Q6/QGDln84lgk+xm8GMdimjhidwkgFX4B4QlxeFtCaeIczBzI9DvtjdIVm3x3KcRNzk4qFzcm1/MgxSC22tPs2pmpg7sVrsVbFJL5J6IiQ+4RbpDr0woESEulkoCtq74VlbIFXu4adfYISebUeaPA0tsv2Pm3nMccUpIxGpCA9XFMMSDLjhHNqo lYrN96IN YfWHO0Y7bqpAmAIH/a5HL+T+HFHG3WnOLzUnGyd7TS97ktMPpmp6TgVDXfa7XwD6GsPvELukrBGxZZCI5euWhI5OeXHZFqJzBPJ9uO6D1sqcxTxu901paz4tgM3AUzjQbWIiz5Aj6i2JETDsnKsSZPinGgZi/QtxJGkFq65X7aX9eMcmeiekKxerLPyE88gDP2nYcCdTLcCQRanUcTaoTvPnvI9124seLtPj1tc5gWxgaO+bxFTJAijDg3NuehCeYFGHMno3h5Q70pQErYh69qrHB8arawlwUAyOLxTk7FKS4asXncdWqwqoYMl9YLRQQZvSCEbcZCTFexTnqu2Oh5ytck+d0NonUCIUskuUeInZR6+A+lqQ4SVDFLmEE0ktqQYG8ih/3l9OWNGkBNE39jYdP2nv8ue5kplyFwkO/e/2O0D4JgIUNFlBp7AOU4SNf6Bp616X1vSMdZkcR0fumOk4E2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a block of comments on pte_offset_map_lock(), pte_offset_map() and pte_offset_map_nolock() to mm/pgtable-generic.c, to help explain them. Signed-off-by: Hugh Dickins --- mm/pgtable-generic.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index fa9d4d084291..4fcd959dcc4d 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -315,6 +315,50 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +/* + * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation + * __pte_offset_map_lock() below, is usually called with the pmd pointer for + * addr, reached by walking down the mm's pgd, p4d, pud for addr: either while + * holding mmap_lock or vma lock for read or for write; or in truncate or rmap + * context, while holding file's i_mmap_lock or anon_vma lock for read (or for + * write). In a few cases, it may be used with pmd pointing to a pmd_t already + * copied to or constructed on the stack. + * + * When successful, it returns the pte pointer for addr, with its page table + * kmapped if necessary (when CONFIG_HIGHPTE), and locked against concurrent + * modification by software, with a pointer to that spinlock in ptlp (in some + * configs mm->page_table_lock, in SPLIT_PTLOCK configs a spinlock in table's + * struct page). pte_unmap_unlock(pte, ptl) to unlock and unmap afterwards. + * + * But it is unsuccessful, returning NULL with *ptlp unchanged, if there is no + * page table at *pmd: if, for example, the page table has just been removed, + * or replaced by the huge pmd of a THP. (When successful, *pmd is rechecked + * after acquiring the ptlock, and retried internally if it changed: so that a + * page table can be safely removed or replaced by THP while holding its lock.) + * + * pte_offset_map(pmd, addr), and its internal helper __pte_offset_map() above, + * just returns the pte pointer for addr, its page table kmapped if necessary; + * or NULL if there is no page table at *pmd. It does not attempt to lock the + * page table, so cannot normally be used when the page table is to be updated, + * or when entries read must be stable. But it does take rcu_read_lock(): so + * that even when page table is racily removed, it remains a valid though empty + * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s + * afterwards. + * + * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock + * pointer for the page table that it returns. In principle, the caller should + * recheck *pmd once the lock is taken; in practice, no callsite needs that - + * either the mmap_lock for write, or pte_same() check on contents, is enough. + * + * Note that free_pgtables(), used after unmapping detached vmas, or when + * exiting the whole mm, does not take page table lock before freeing a page + * table, and may not use RCU at all: "outsiders" like khugepaged should avoid + * pte_offset_map() and co once the vma is detached from mm or mm_users is zero. + */ pte_t *__pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) {