From patchwork Wed Aug 21 08:18:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770984 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5253EC531DC for ; Wed, 21 Aug 2024 08:19:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAD786B00BF; Wed, 21 Aug 2024 04:19:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5D2E6B00C0; Wed, 21 Aug 2024 04:19:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFE2D6B00C1; Wed, 21 Aug 2024 04:19:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 903506B00BF for ; Wed, 21 Aug 2024 04:19:36 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 45DAC809EF for ; Wed, 21 Aug 2024 08:19:36 +0000 (UTC) X-FDA: 82475553552.17.7871E98 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf03.hostedemail.com (Postfix) with ESMTP id 5E57C20006 for ; Wed, 21 Aug 2024 08:19:34 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=c5QWJmhS; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228268; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VgE/WG98IUTB5tF1fVgrlRXpW1wPHqZ1M9xizEuPG+g=; b=Qm4UMX3tRMW4JhMPoqPc4+QN6OyJ72IxJhPbUlox3VS8ix5WtxsBF1RK+FIUv9UKOvpbo7 nI6tbsocrOShZTpZX5LNe9v5jHlrc1ukxeXnVmxNgHgHXugsx2TThtiPaTpZJGDCHGBEXv kwApOT8l0JbFcDwrxYL5AxMxqmE27lo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=c5QWJmhS; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228268; a=rsa-sha256; cv=none; b=zWzcod7OGSI/EJxVjg8XusEVi6ZYbFG4VaJwUo3QO9T9Buo2i7M0a1f0NL9rKOFuHl4f2Y aYm8aqihvCRygKrsM88pAFbQEXWIvzwNA78BobEkzL/0v3UPJzl7/bmt5WChMXdjqoYggf u9oNkHjN3+9n9+jgv0TklDGHjqVhGu8= Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-7141feed424so768680b3a.2 for ; Wed, 21 Aug 2024 01:19:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228373; x=1724833173; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VgE/WG98IUTB5tF1fVgrlRXpW1wPHqZ1M9xizEuPG+g=; b=c5QWJmhSwy4y5T2pcmfA+a37d6fFAuvVUYdJ7Kk0qDgmxmqVOmAPdLp59VDUtAClt5 1MX5zYnXBUgLcwB2u4187d+vVm0jn4w8fPdxAJMjxm+xv8NsO96NmRmxfMnwJaL4cBM2 DLKpfI4d5SBTY87/HjIz35tDEfQp0rbFOTKd1RGfZoofHkYVeUalT5wyR+5O+FLBt2SY rvWqo4MlOFMsc7HupdUG34fj8Sb0V9PXBHVfQHNleBZbHWcmr1MWpfcHtyrdGWTsz4JQ aLk0XB5Q3zvoEnn60iTa1o6FpYS7YTDRWBWEB7gi1Qzpuc+XVxzgJJFz1+BtLdG5zPRn Avxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228373; x=1724833173; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VgE/WG98IUTB5tF1fVgrlRXpW1wPHqZ1M9xizEuPG+g=; b=LDQj6jaznaSnOqLdQNi0575b2678d4Ao6IabBQxIc80ndayd2gpUgsxYkMGENAyapQ T4ZCiIl9xBWegAXgFWx1/0c8GlsJ/SzUhnVJ4Id1l/zflHZepidiL1rFTEWdsR36lZqB uzxGUce2ZeHa/HaQlCzi5ZyF6pkfjWv23onZXAVQtsmyfx++u7nMgXLhR12cBtzKDKcM kia23qkqnJhDBAwos6KMJK0lqXUZkri2QNUDMDd9I/EO7xNCDZZ4xyUTGlNDK9YziPXz iHl2r8K6Q0kTr/7rdXm82KuoYdMOQi8NyE02zq7FdtAmmdiqrjeTI5jZv/vMdyW8LPxA YP3w== X-Forwarded-Encrypted: i=1; AJvYcCWa5O/RGV4j5ml/6U6n5QQC5B5ohIosddgrLbcA0WKi5li2MdMbsuGkfx8X3SMCdd1V1Z6O83Ta+g==@kvack.org X-Gm-Message-State: AOJu0YyAENGGnRFOAO41yvls+vX6o0xbMbiMA5whttHdTEhue0k8Fd2W +z9ThzXrX7cOf0mW1CQoBzaf4xuMEMGsnVBQu5yIiEnGyuqFJb/mdnwXblMp4bM= X-Google-Smtp-Source: AGHT+IF11KqU9EwWDBxqOuV+x8Or0DtjlDyZMn8HSaNUPAUvyVetfGjOkbkYFHx43yYUuJqRZJ73CA== X-Received: by 2002:a05:6a21:2986:b0:1c0:e49a:6900 with SMTP id adf61e73a8af0-1cad80ef085mr1849540637.7.1724228372801; Wed, 21 Aug 2024 01:19:32 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.19.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:19:32 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 01/14] mm: pgtable: introduce pte_offset_map_{readonly|maywrite}_nolock() Date: Wed, 21 Aug 2024 16:18:44 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5E57C20006 X-Stat-Signature: hoc4ug5njziuu4jbz9hajycwgosx7des X-Rspam-User: X-HE-Tag: 1724228374-685525 X-HE-Meta: U2FsdGVkX19ebyhSckfBW0C3QMaZBnyWVmW4IW0TbyhcZFfi75uowiN0nvedx04Jfqk19ld/sQR5IN1z397QSftHoEw0NHF/fp4qcG2GEvAwP7xzQHcmczeXXcsu5d8en7sdMpHiqXx/IKakqKPRSWDscsvNITbtEV5N1FxIEsYMU1rztssnTs0YuLYtP1shgLcP8uEGoC2TjMYVMF8uBsz3XEt1x33Ea0SnI+5yZj/M70tbTylHnkR0vo9J91ZjA+ajT3OwBr0JX36mwZ6o7qJWA3ZQGz4k47J/xn4faN5iksp7t2zlONmdfVzfBLvT7W49tCN3akevLDNCNWBvvZNJwfZJCH2C/wes4wxRFzUl+0FV4fEOncs+VNtaBKtMs2CfY1C75nAglIbm/86oyPN9XKMC2TcP4DXn5v9XOnrwRYRTtypz9oFjWhtjWS9E2sWjJ7/9t2ZWOxsdPdzIXwY8RwRBXAXF2baRuEBmqK/tZ2SW5V515j3fdmjyFycfFa/0TDKMbGP2taXvWRKcN4PaSloshCYCcBZq8k2KuGbFA1qiYL1+gcE8Gz+rUy/Jn7Gg+cpXVeqi09LPcBeH23sG45yIS1M11aL/a4RvgHAdB9eFCdU+DQpwMN4kC0WPu1vYjfaJifEKHbAy1lsLMBTZyI+P6y08hWWymeaIjPsgjsCnSGftUNmnNR4KxPncR9X12siHRpM+eYGYemiWAfXlBHblNASqPNYILvsja1DtXin0+r0y53JWfwU0at4kH4qQsVMz3r4rO3omKh4w63aG8hjHB9SFXE8ran1ZkJATNlhA58bDvAYWqf1Tvm7Bj1ub7ch1DPFQp37qWLCsNaLfiZYKwAd4CmBxfqy9tzkQo8PJFzlxC99xFSzrWhfaOi8fjW0gl7hRTtwUUwstjZonTIqCbx56Y/e3WaFKaAr06iFwfKl76++OLiuoHwwf/95aL0Zmpui4Iy1/QOf 7comXfIb DwDrDVgqTliGksHQ5Vb09CoQKvoHrXbDbVdwaHMwVJ5a5D/IvOjq34B4jwdi590ODgt4MYJ+YXuTAGgmOG3bwp9Lk1VEvhlGNsTUi9tPRZlq90MDJn6AW+b6SYCvX7/QAA5jvnvnG6FfY16kuHMCK2PSjslmzIoTwhIFHBSj0Cx5h5qndeJK4CssNxSBR2YYrAJnU9pg496YZgn94Fe2RQjDocXNLzWyYYmsl+947bECl9B7m4ekkNE29xGtL2cgVy5lrHvbgGpRcyRdHXqIEsQwIH29QEagWeUypZa5+a8/nZgDRzXPM+72vI+SWNZge0KOqc2STNihpy3L4TaSkGsob3KSl5ZTllcEGmtk3gI4a/qpEXdMrX2e9h4LtWzS3avPpiEYkA9VeZPTdjUY+/bhkWFz0Mfcm1jotRY+zRuW1i/jySSp5gjhx9uHWHydcapNX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_readonly_nolock(). For 2), in addition to changing the name to pte_offset_map_maywrite_nolock(), it also outputs the pmdval when successful. This can help the caller recheck *pmd once the PTL is taken. In some cases we can pass NULL to pmdvalp: either the mmap_lock for write, or pte_same() check on contents, is also enough to ensure that the pmd entry is stable. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng --- Documentation/mm/split_page_table_lock.rst | 7 ++++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 43 ++++++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..f54f717ae8bdf 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_readonly_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_maywrite_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index 00501f85f45f0..1fe0ceabcaf39 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2954,6 +2954,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_readonly_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_maywrite_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..29d1fd6fd2963 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,33 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_readonly_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_maywrite_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + if (pmdvalp) + *pmdvalp = pmdval; + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +383,22 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_readonly_nolock(mm, pmd, addr, ptlp), above, is like + * pte_offset_map(); but when successful, it also outputs a pointer to the + * spinlock in ptlp - as pte_offset_map_lock() does, but in this case without + * locking it. This helps the caller to avoid a later pte_lockptr(mm, *pmd), + * which might by that time act on a changed *pmd: pte_offset_map_readonly_nolock() + * provides the correct spinlock pointer for the page table that it returns. + * For readonly case, the caller does not need to recheck *pmd after the lock is + * taken, because the RCU lock will ensure that the PTE page will not be freed. + * + * pte_offset_map_maywrite_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_readonly_nolock(); but when successful, it also outputs the + * pdmval. For cases where pte or pmd entries may be modified, that is, maywrite + * case, this can help the caller recheck *pmd once the lock is taken. In some + * cases we can pass NULL to pmdvalp: either the mmap_lock for write, or + * pte_same() check on contents, is also enough. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Wed Aug 21 08:18:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01195C52D7C for ; Wed, 21 Aug 2024 08:19:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 694816B00C1; Wed, 21 Aug 2024 04:19:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 642CB6B00C2; Wed, 21 Aug 2024 04:19:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E4886B00C3; Wed, 21 Aug 2024 04:19:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2EA9C6B00C1 for ; Wed, 21 Aug 2024 04:19:43 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C67CDC0855 for ; Wed, 21 Aug 2024 08:19:42 +0000 (UTC) X-FDA: 82475553804.04.6D81D9B Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf10.hostedemail.com (Postfix) with ESMTP id D57B6C002C for ; Wed, 21 Aug 2024 08:19:40 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=LgLLmggg; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=arLLKSkNLaMIjiTVBc0cMKPJV4f+y4Msx8WJt/66k5c=; b=AaA3yh8fyKBHkKo0PPdq67zoYYadtcbqvQ75BGBnm1Vo6WUcMQ7GF/HvCBwfFOGjEu9yZt 54x5rcnE44Sco5QQOBymia3wZ9h4ELq0nxSXZ46uYJet8dqGaWO4S1hlOqtW0LkimuyAdi MgEpH3/T1qn7EEzWKTXP0KViplyTWQc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228302; a=rsa-sha256; cv=none; b=EDP7jO6DTSX0d46RLbOMAL4WEgHuIaOf8pWBCLr5kz89w2BGi1Vzz0iu34FVdqx0svosO4 BX/HxeUWNc1MI8GLlvbYf0zYqNt+JLAa46qhsI9iHQEoBO2qDLpKYMrdcCbmMv7X5UVwtN vlXpPkPhfm4AFVr3xows1a7/R1noLfw= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=LgLLmggg; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7bcf8077742so4723020a12.0 for ; Wed, 21 Aug 2024 01:19:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228379; x=1724833179; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=arLLKSkNLaMIjiTVBc0cMKPJV4f+y4Msx8WJt/66k5c=; b=LgLLmggg+bMyiFAaytP1QABJscooNxFzOuvtEcAVsokhKQFOj9iVXDS9TvaqBR/diO 8rzU12VeN20OBnXhYm0oUiKwSztN3UyW6AqcubY2PW92t1y3fzmMW8uiZ1MBhHYsVqKv t6TX8a7oI+zytifvA1GoJQx0vC6rgGrGPzBxopPotgUaR+QlQBjiVzsEuzqgwYzPHFkN g2GDwwuFMekD9DYs1pYCbiQv4m3+HDuT9D+2ZnaOeF7tZ9jhAUTU3jjgR6azahfk/LYr 4RTFfpINM6kSZlbFe09YSJAabdBfw6m7Cmi5Ut4QyVwJsJ317P0vqtBV8Ljnbblwg6Bf UK3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228379; x=1724833179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=arLLKSkNLaMIjiTVBc0cMKPJV4f+y4Msx8WJt/66k5c=; b=aVo6CkFXdMools94EKB1WQUPCEd1Eg6g77s9y2gILlwEA2H9vrp/9AZELNLg06KW25 WdYUnwai6A3vm1rDm71VuU0NkMc0NoYD9Qza9R2KMvlIXxOyljzBKgCqAPMnU8QGIVRb EG/mV6+rK6ytPm2O2To4An3ze/1x8sW5FMW4WTcAatmWLCQbt/vUJR/VVaKWbvGblyo4 Z+ocxbGMpI7pCsIyU0LCUOv/sTX6aox8ZueRueo524+YX5tm4Qlrk5JodK58z2dyoyfR n5L/VgUufion4FGVvDGfuuhUcvI6Z9k/OmYnBBBaezrN0x2pry1EHyjvawaSzGxeleuB rBTw== X-Forwarded-Encrypted: i=1; AJvYcCUfkeEDVqQiLpaRn9PblYCYAQGlsuIHdTtoLizxCln6Ugxys5ILhTs30TLm0r3hVMYVSme6kJpybg==@kvack.org X-Gm-Message-State: AOJu0YyuXSKKZ8MjMfJilchAWqhIXp6ozwV/ETYh+14gZAlWpMvhnFPy SBwHCstf2QBay87n6XEvr7umCqEI+bulFPa0QKHFDlw+HbGfLly0/7UbUg391mM= X-Google-Smtp-Source: AGHT+IEvHWRxH/KV5glvGK0Xiv2dJ13pnLs9scNXFJwmOSFepkjBPO6rW4EjBZpYY7GPQ4yq94i7jA== X-Received: by 2002:a05:6a21:168e:b0:1c4:d05c:a967 with SMTP id adf61e73a8af0-1cada179499mr1734402637.51.1724228379371; Wed, 21 Aug 2024 01:19:39 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.19.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:19:38 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 02/14] arm: adjust_pte() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:45 +0800 Message-Id: <3d151aaf876b93a97fd3f070780345a3bf572123.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: D57B6C002C X-Stat-Signature: 9nfdwrcgb9k6mu1whbcdii84sewi376e X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724228380-946448 X-HE-Meta: U2FsdGVkX1/PI99iFR07iiTfyNqWBqaNEysSlN6toAd5ZppVYOo/MU2rFpuwMUOrzEvJjKi8Q7QxZRIvIDqKlcVuSiuvU89HwUGI1FjuneDOHS0wf9HhYeE0u1ouN5MKx6rEESCYSaD4tM7UzQh9VXysq5NpFu/BDdUQBgzMkQ69LxOlTc1lWWnHxRnylSo9nA3aSZP9qIVNtTSOUy8kKmR/QIluroaBiMllEf8ywbtHjTq4qHEdPGg34cEmM7Zn090JC+BAJjMVLHBO4vtlg2nUChXrdH9P7uJb919FMFHhj6B7pICxNxaLEU6OhxVnIVHiPsQu/50Rf3pHc6Cq8zCL6aa9SReWZPPmEqjkKfxEANEZ0bNZ1IcxeJufUedZBPM8C4VtlvqXwDEWo5ds3jcRswj50mF5wquMP6gl01YRot0LpMRBzlJ1fDgOKn0+HykuEy6oese8jOQNm7GSifuRvTp1xibJiaQq+BHb0AXSI32hRwEWSsCK0HVe/KHHjHSbiLd3YDAslrbMhYO7E/QpeZIkueOT6PVZw/rVxzAVLYEQdPQgsVjoqEtAeTuE7cnEDS/uxivFiMi7YEPa64y7Z32SxDjv5qoOv4vnV6c7yC9RmbyuWSxYXNSBJBmLdHZUtIut4R+/BuAH+ZZffdMZHhJWtOLl2p9YHzPx9s6mat0ShKOsTx+1gZ0SZYto2q7iZE6R8if8uXzVq8trJh5mfbblV4KaB4ry55/P+wQVU68oOJrRmY7CtajeEd+oiOXhPFcwdNiVd05ZU2d3rdC++4BzsOtISStdOruhDdI7sUZqGCFItNJWPN32RSvPXUt9eyTxl18Q1C0xLeisIQlvzJ3FWiu+VzXq4Jwvh2SAsRDAMM/VWK1LlaG9DVGSMd0Jet3sZZjDik01edhWBMsKHjlmhSXptTGVNM+xTF5exmbXNyNUqHjj4BtvFQUJybiW03hZwz0xoZP0t2c HJkZgSKY trB9EB6u+H5xZ00FXdwkVcJoqT6Lv1AIIby11ae5jLVxF33mgfnW3RjBCbRmp+AkSUL9PQOeQZ1qRsgvqYwQNU3F4B2FriBMm+HxmZ6YomV3taF6ZByqWnRLuqdepFeAY02J7rWeYrftVkX1b8qd8EsRR2tj/0RCNFZjDP9tzPPGlxQr9jFk3dtYcmVAvrMWSljIPiOeSe7uq4Mh/Wz97Fy9BUzEZaMk4GYyWjABVESGumTOFt2CnCXfB3/ZjQS42kdJGnTLQK9zTjt+5C5L3ue43W/EGzjdiLXaqXfXx+Bb/LbrUzn7FHEe+iSvVJIDq1ZqpuQZgjD3FmuLzH4en1NE2V0vcExzHbJlBrTqsoGU+VDGg+bZTI35tLHFS3W76ykeIu9HtHeDNSSF2YKSASuqXjruST0IEuFY9MmLOYW4QI+ntByrWsflA4sWxhJ81krZz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In do_adjust_pte(), we may modify the pte entry. At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the PTL held. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_maywrite_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. Signed-off-by: Qi Zheng --- arch/arm/mm/fault-armv.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..5371920ec0550 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -94,6 +94,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,16 +113,22 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_maywrite_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; do_pte_lock(ptl); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + do_pte_unlock(ptl); + pte_unmap(pte); + goto again; + } ret = do_adjust_pte(vma, address, pfn, pte); From patchwork Wed Aug 21 08:18:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770986 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC146C52D7C for ; Wed, 21 Aug 2024 08:19:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E5966B00C3; Wed, 21 Aug 2024 04:19:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 495066B00C4; Wed, 21 Aug 2024 04:19:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 335D66B00C5; Wed, 21 Aug 2024 04:19:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 13A006B00C3 for ; Wed, 21 Aug 2024 04:19:49 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CA26E160AA3 for ; Wed, 21 Aug 2024 08:19:48 +0000 (UTC) X-FDA: 82475554056.04.570B7FE Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf24.hostedemail.com (Postfix) with ESMTP id EF6A818001A for ; Wed, 21 Aug 2024 08:19:46 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=kCU4OnJB; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WkLNXpSDw35lj4Q8QTsPG+BhSNUK5Ar50Dqo0ISPkHo=; b=EAU4pGu3darsKOOU/0lYsgyQtiVT3ayYZL39yEVwUwglqWNcBihr6jf0/gg0Bj9ipJOzUr 4vBZAZKwN5a22gLMWJqzSVRGonPROFDQigzaivMmbdAmbAzRZ/Hss+w4piGCrL8u4fwF8i /jejtoCUICNqe50ub2cTsbmRi3yfcug= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228308; a=rsa-sha256; cv=none; b=SdH5F+zmC5Fd2yNZoVq/HiMJGB4LdQp4Z0EonWkqiZtPNvehUzc/2wKG3yK2fjY78Pq14U emlMTeOU1oTedz4+6Xjxgi5MlXph0g1EmR1xQb4/lK++9HJWpo23NYa/+K1z4gSlPfM24K ugAY2gI4PJZtMPkM0IxgDYGZZSSOeJs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=kCU4OnJB; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2d3bc043e81so4980462a91.2 for ; Wed, 21 Aug 2024 01:19:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228386; x=1724833186; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WkLNXpSDw35lj4Q8QTsPG+BhSNUK5Ar50Dqo0ISPkHo=; b=kCU4OnJB4HJOQitaVCa+dTBJSXLku1hrTscKDwbRzijSx/1g4SxA0rnrKqckF7OhW0 ouCjOPBQ8RGw5MkkI67FV4eWSRhO91QIIOw+U85nTSsslfbV1mJSB4l1+9NB88WQI/d0 tj6XP5P8SFQCttYTy/Brl7QTlG7iCMbA/TzM2QRgDg2xfZoIoYwvOfBO7QBj+2s53U8Q ockMK/5rgBUR8XircGpvPOCNPE+w2itMDMFc9YblWk3ghl4tP63XlpCjgET8CPSuSX11 cj3BPqFRhikOT7lgKhQBwu1Lp70kEcwxshKlKaLhriLtDrf7KLpqD+EVhvQGm2hfeWEz ZyGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228386; x=1724833186; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WkLNXpSDw35lj4Q8QTsPG+BhSNUK5Ar50Dqo0ISPkHo=; b=tSRDqVrQJ0s2dJNQbczlsKRLfcipGN2+rgqjP+POSjyLD+voiS8o7jhq9BJeqMhsZJ aIB8OshSbbq8N2EI2PgOq3xlsQ1n1COZc6EnM4Rb2TZTCHFr9bSeq5fj0c0r7ebjsnyv jfrzXKyJazJuu4SCoHjd5afUBhquhpVx29la/6BNpOFDotmju7TRKWZAjytSt/ElD/n0 ZGFbpwOcxtE1v6+CHvgd4Qwxiy2SyankLQAFiuRuj/DOyeiVe2sCDtY/01kW0W9GIULP bkE7Vx9z/WekyO06l7H7QsMSheLlAj/NU7N6LVc0+dFq1g9diJ5uAXVr/31y67FCax79 Dbjg== X-Forwarded-Encrypted: i=1; AJvYcCXFHIRDaHkYeHeK9Uhk0zHTzMbftSsoSab80zkT7CWBqUKXK6W3iUE/owQ+vT929LCipQ/3Q1dtaA==@kvack.org X-Gm-Message-State: AOJu0Ywl0P0b63hI/WMdVTwx961LDwq4rakthOSamAyI7/ryQKGv7L2Y stAPRU4P6UJul53rosVGtZ3fkMHm9ZzacBbW2VFjVTaUfX0XiTpcekIwAEU660g= X-Google-Smtp-Source: AGHT+IFyY3NlNyztu6E70clqH4irJgHSXlPUHA4AtOW6RrWWSbwAKaoUYDBEK6KZMX7nlR++kviHPA== X-Received: by 2002:a17:90a:e7c6:b0:2cf:cc0d:96cc with SMTP id 98e67ed59e1d1-2d5e99c8b75mr1585510a91.9.1724228385781; Wed, 21 Aug 2024 01:19:45 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.19.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:19:45 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 03/14] powerpc: assert_pte_locked() use pte_offset_map_readonly_nolock() Date: Wed, 21 Aug 2024 16:18:46 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: EF6A818001A X-Stat-Signature: 5drd3t6ji4axr16pr96ot4h1wrsrf31n X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724228386-766468 X-HE-Meta: U2FsdGVkX18pqZPkj/C9NZNBVmGgB2OBZHe3bxNxz+jXkhCjEBkcTO+jaIscD//qRs8RLafVuzI218eAuDGubWFJfQwxQ+vgbj7cM42dTiQ5qiCfIQXN29gFRCsnYikhhd7hRIIaz+OFeNeMMxHOLr10pGbUH/zTgckqwNvKhK1FqwVrLTWx2xjuL7uHhNtvAFeTI4qMAlVla3FFzHcdfNpDZqkccUO739fARY4LlVxUz6lgUkB//tc8YX6YIwgUKRazyYhmMDh6+R9ljYGe5TTwbBJF5CmhiS1of4152UtHLxpBKgT7KOXrAVwqyROm+SrGKAFhub26uL1pQ5XtjiZTppaWL8mZ4VPj2hT+ZUT1URwbnlu72uwZYUth01O8ONRp12s0CskmJv0Ml4awY8dhVO2VmkMcVY4y9TRxypQ3uHsAB9eQUvu7/R0ymWgFV86SlWrYzpfS1SFgKKIncWWV3ZykrsjSkueUSxSzlg/n9C3dbFE3bdvGmTocfIz+CkYmiDgshpAbbXqVw6NH3rpb8wyhx+lLrP4R0GPnijFyX4fc+yBHwdPOp3nP5yeq1vA9jsXpJTPoG0DKnKV0Sx97qwgazl4YL1n/yfN7SxmL+NY15TVmKYrguWnV4qC5BX+cUPYtgvL96Mg4clxfpBhodQAIW7sGSLJyL8w0Fx8yfWYMLGBYegJw0TJqdbDIrMPWEyBN64WfxYwtM16fOIWPnmOC17TxdT3qAItuLWNH389blj6DuH25ohv70/Z8WyAOCPEq9wmDuruY/LOli3mA8szkIntho2dZmJPYg2wvOFoDqO3ZVGzJuXWIifEB2IwjM2ojLt8CT5Njzcju2M0Nk6GQzd8To4s+YJ7kcKuCp0lhzpUYbdsenWWP2U9DHJnlCpkGxQKH3IjEmAlhRnCr/O/ZyNUO4poMd4xsK1T8R1/wwiMo2CbjpBm2GGSYDTnqV7tLMR5QUDcvwnk WsrVA7UH ti/1FKWd6L736lkbLDajJ6uApLwASuGS7kj+8358d7SNqY4Yz0vEL2po6c7NFVkWPzSDgj6vQ3p64AJZLcMM1nCRptMI+2qiMqTPKkmVjmBfZBAl18UWdlwnE6vV5zgQMf9UMro7OuA1lGoTZgng6WxrFOpIz6A0RfNe3SNm1dTVdQghT4VAWOeEYjLmNKkR03jNVD5Hk9wqxxidXdEYgiVeXlwcDJ+Gubui4IKJnqd/0A6g2zxSFZyb/JYv+rlqXwAKTTyEHrUXAkruiPzl9RM/KczqLezy16irMCR7U12FoO3cCkmO/CBMgc/nCMYC7atN1R3UQeG8QPeGnmtc/RW+H3XJnKw5jt0Ezzo9vvOFhYX4iZ5lXA+RYAXNb2sZLwuusiEP/7zf85XgyMoP3IMF2Rd2Ov/n5KAFLq6jrIgijTBc6tagZHM3shpC5/KOKJWjr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_readonly_nolock(). Signed-off-by: Qi Zheng --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..ae4d236b5cef7 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_readonly_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Wed Aug 21 08:18:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770987 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF3ACC52D6F for ; Wed, 21 Aug 2024 08:19:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CB3C6B00C5; Wed, 21 Aug 2024 04:19:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77BD26B00C6; Wed, 21 Aug 2024 04:19:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CEB46B00C7; Wed, 21 Aug 2024 04:19:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3E38F6B00C5 for ; Wed, 21 Aug 2024 04:19:56 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B7AA81C4290 for ; Wed, 21 Aug 2024 08:19:55 +0000 (UTC) X-FDA: 82475554350.12.4414C54 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf14.hostedemail.com (Postfix) with ESMTP id DE8C4100007 for ; Wed, 21 Aug 2024 08:19:53 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Yv1o30tZ; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228314; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GHfDnFSOmlgsN5ypR9SiqdvWOZgzK74sPfUFdgGElDM=; b=HMWeu25FnHerws328Y4eGViSV+FwJLdTx5eyvgCTgCHBXA534wAO9a60FVv3YUb+0MyI0+ M3W/mCdBErflwke4sbAobskuQUbfFk5gAkiZPGw8pxdpm+8GdlhuMBZN3LbcFa0//dH2uo fsqMSygmXdVg6CfKC8cnc9g6zShiBxI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228314; a=rsa-sha256; cv=none; b=fRP+K35Svj66c8AeFT4ThDjEdYkt0HJy0T3fchNrv28Xg46weYGRYgmzjBWouqR6wH+h9Z j8dVYyd15IeSlpuHBwzJ4/77Dz7SMIdVIZiIyDFW3tZif7h0yXtdoCKUwuWd0zuxVHze3u Le9e3WT69wTzZ/JVvFvcMqR2e30lWlM= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Yv1o30tZ; spf=pass (imf14.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pg1-f178.google.com with SMTP id 41be03b00d2f7-7c6b4222fe3so3874665a12.3 for ; Wed, 21 Aug 2024 01:19:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228392; x=1724833192; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GHfDnFSOmlgsN5ypR9SiqdvWOZgzK74sPfUFdgGElDM=; b=Yv1o30tZbE9T1wicoe5EZWLHFuUh02WkwwQwWZoqQWo4cWrziWyyK7zi1K3H/rQ+lp yAUFcu9ycEHICbEWqZtCTGGHYLBZb69UWRAC/KORKsgCwfe75TKiyhrqnIAC6T1AsgGr ytWJ7kR1vQAL5f0MAOACuno6tQXjAWM8fKJ8KqPDT/bVoDzh8gozMDWPjeDbJtrS5zz7 MLyG7AKHs3WMQy3ZGU365hTeEkquODq5BkOnrxyQL0WEVrFSFXI9AWjnDAf/Q3bzkaba p58fA3SAEAFjQRUxh6MJBxIafi8mtLYXLUnzjdyn968BT4LKtRhDglM3KMLPhlD5l+xn 4Niw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228392; x=1724833192; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GHfDnFSOmlgsN5ypR9SiqdvWOZgzK74sPfUFdgGElDM=; b=EubcKIB7Zrmx2Qg7Te0T9a4Q5Og/IzrCtYwH29py/T7d07WVP5e1djTlkiA9nLeG+g boCP1KrtlcBbGPNZw4DRWen3k+17D3JPLBxGvGxMOqqztMmVVWZ0PIPMRzLKC2dpzfuE EwKrN6u2MkmDG+uXoPgUveyT99mBKquC/T0zXWWJ5emxYvhSGFwE9VB1htKGV1K+h8Bn XBTkwxXFcBLR6ut2/7Yh840eDtVdi2xz5i1176W2hfrkEuxyB2F905BigL9TG/gMcjKZ +n4/cKleJF1ILyCSkNnh6op/f/Ta+0Jv09lRZCju+NLKElHJASuEKni3SflO+kd6Xa+z VYzw== X-Forwarded-Encrypted: i=1; AJvYcCVH7378JCPym3Stfe1+fZyNzMq81HuoGANSUAJ62785LlRWyHanR0wmUrxJz1NpSTcRa+rQdll8sA==@kvack.org X-Gm-Message-State: AOJu0YxxDVSLp69mZ/1alTuqac6bTzzwF34kZDbr5BSAdsxLcWzltDkM lT8wfLnh4UW7mpc2cPDkDieFy1alFX1V2OcfU4aU/Owy+khmL6kNY9ARJL1SuHw= X-Google-Smtp-Source: AGHT+IEDKdhDrkAt8Xw9HjhxvAn16yLBj0ykk9gzeBgjUCoqqbqdejlnt+3gntiVrRuxkT/2XR+s6w== X-Received: by 2002:a17:90a:d98e:b0:2c2:c149:ca4 with SMTP id 98e67ed59e1d1-2d5eaa9d141mr1321075a91.43.1724228392525; Wed, 21 Aug 2024 01:19:52 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:19:52 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 04/14] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_readonly_nolock() Date: Wed, 21 Aug 2024 16:18:47 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DE8C4100007 X-Stat-Signature: pcjmydinfqtk6sbn4g13u7ogt1sdis34 X-HE-Tag: 1724228393-991981 X-HE-Meta: U2FsdGVkX1/3FUr/LkvRF1B0ApNePtm2s7nOglulUnz056m7e3vpj990jTrTay1btwkSo8fXyXATu56o7fZH7pLUiL7HLoQw1FrFZc5a+RzD7kZ8T6poVQDd20bqYsRvqxpWa5hYiFBNRQAeietw4hXgoHoYHlmHiVxYvY1GqhIlRJgQp0qwmcKo9/rtax2WA73IVO+vlrxQb2KwK1F6kZwdl0MpMi6C9F20m+AzGSfwpRbD9b4z7hYCKEQwBQfBmveP9Lfa+RYNneB1AR03BldaEKxvhGpD62DurEvc7hKsdPkZgA7wrjzPKM62NOabSMEdgtYKAKUEW9xWMjeAAieJ6A+cUo/bysC9wEQR0i5BOrM0HfcMLfcLmC6llNruCAgnnAfbzlvdqx/FbZJE6EKsIKjm9MN/AJWYuiLYIJAQHVEfJjjnBBPBcE3Dv88v3rI30gswH8RogFq38j4ScOMEgakUmnNBPHVhBDyPxwPr+Z9D2B85uFoBpOwTm4fbNjHn5iSr/suQMKWEW0qJkBOzW3IcWuhBVbsViqUhsXk6ZDf2Y4ejGaA/HtRoBM3hJcs9W/TM4PIWRgE3YxWKq9nsR9R4hEIUq2IFwHHLSTc8eu2RzmPLkps4Uvvk3lI6LBEg8dLVcSRPNnyZ7GiqiIxmGteDxeY/sf1YReIT52kXO3Z8g7n0CzzCsEQpLhTd8IYEkdu5DxPEB0iTD4QeUSpl8074XRsw6eB/5otO4nwT2IKKgq0PjsZow1fmjMkarS2JO2ud9KG6fyaPdHAB36sZbt8aZ/5hmdDl3t4rRiEYUHrRIDMXUpTIGPvC3aiZHHulNbvs4XfFzzVT6OkpN+iGlGHigq2CM214Re0OtqmiGF+HUP2RxZKwDpzD5ncnTNRC5wfvxQOCezyN0519TZGQJdQmcpVx7JEWobL/4PllSkwTf5FBy2prvGNDwz7kiH44c1a6XJb195By62v yyU2ARqW 6W5S/r0oGDqgfOqUu633au0yKSKl6mm62vJnTQHWeDgYrJF1F3UMVT5i7PCU3DGuHcztRPa/YIbDiHqsICZLDu3JI0W2NlqKDz+r3J/FiUKvktJwUnVUEmFMjMr2ZCIErTx02HaXK8mln/No1WHUsbE5Q9RBHX7H40lZTXCzN9yd8x2stInErSa02FCG5OeCZ6E8jfnAOLOGV647jTjuwDRrHnB+B04mb7hWTY4QfODkN9Rd8KaQjXuyvb5O+7EU3hqrU7qL6jHVI2jhYMdMMQTupJesXeV2F4hNv2plj5rkF1MvB82FCclziQbElIqUvTU8WKgz0tjLD+B+RYQkQ7mBzird9cASTHjE9ft9GxE2ktxZqrNJPPi0IPR0LNKpRKJ9k1ytzSSStFATmAIBnpX90ZKL6TcvkTDD0Qq8SjvF8A+S+d6n/9pvjUQNbe1MVgJk3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_readonly_nolock(). Signed-off-by: Qi Zheng --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d87c858465962..491eb92d6db1f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3228,8 +3228,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_readonly_nolock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Wed Aug 21 08:18:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770988 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 955B1C52D7C for ; Wed, 21 Aug 2024 08:20:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 24EA96B00C7; Wed, 21 Aug 2024 04:20:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1FED66B00C8; Wed, 21 Aug 2024 04:20:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09F496B00C9; Wed, 21 Aug 2024 04:20:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D96796B00C7 for ; Wed, 21 Aug 2024 04:20:03 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7C777160AB3 for ; Wed, 21 Aug 2024 08:20:03 +0000 (UTC) X-FDA: 82475554686.18.CE5BFC8 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by imf13.hostedemail.com (Postfix) with ESMTP id AE4F720002 for ; Wed, 21 Aug 2024 08:20:01 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JUhqbFn3; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228385; a=rsa-sha256; cv=none; b=J7htfW+9MKmZeUeFUAj0trBiof6mp0kFjxrm8Nkgq9mcICgLsOt8FzOfJdeWUZMPhfOb0W CF4ZESCTBh1rsOPIli+NkHXWZXQ3CIyUQob3Jc3tJRx0s5v2PntP9fzrxvQXZpaD/ARcOH 0wZ6Tr8SHPrh1a6Q6cl4QJmNja0MmgU= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JUhqbFn3; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228385; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AWBymtmrK19rfh+o6mKAIf3qo5pwJHde1CNDuWd4778=; b=THB0ExtQHmX3hgVz2RICO2csqyysCBbQcLa4JnHrjZnid+8ZKuqGZOhiDtQSbTQs+dyE+b DauFXgX1rcg/jH773fF5S8gjFdMEq4XihinhGvKOk4yT8AqVnr2YzDcGQ2xOrttyVm6Pic cdLetNaglCSS/TbYYIGyMF+HsoNoCn0= Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-7c1324be8easo376312a12.1 for ; Wed, 21 Aug 2024 01:20:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228400; x=1724833200; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AWBymtmrK19rfh+o6mKAIf3qo5pwJHde1CNDuWd4778=; b=JUhqbFn3yeP7IrwDsyzskXcZOwlBh0FDgHGsF7v1oghc16P6nKxdOa3Q9yXnvqIJOW tLawfBmu2A9iRdKpjLDYHKV/Hr0fuzh036RIIMyN7tTZUHSG63WCPYHXK0Z5ZQdPCmRZ wO83Lcu4D9OdXqf+q05Wn26DsnTRtuKeecAqgrNvpylo/1hqXI9o5mj3xEQAyEjAE9HP yaAIXAxvvvsL5NqIUXWc3JQrYgoqR2h0jnnFE3RzUZ7wxPbcVGsmCxI57hGDwkbqbemt UydmCwEHrem8FJ1amh7iSTtC9JLqmKmkzuRdvovhgHSniBMgNIO+yTHE37o86qGsQ5p4 UrFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228400; x=1724833200; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AWBymtmrK19rfh+o6mKAIf3qo5pwJHde1CNDuWd4778=; b=u2jd+rW+Toqi/euluK4iiyXOanbrxwsh8kr3KC5fzCrQg0pekZnDs47/r10mO6bHto V5wWi3yLCUqo15m7yqbPWjzdwvNuTAFMKlQLVuP5HRwgzgQsi7v4BHCvE8wqH1gV3fNg EdvYauijyMdpmT+c2Pys/q30Fx/tN9LTRSD0YVoy9S88joo5e+F/yVhEJamgts8B/UE/ eYu+RYwgBu+qSKLE83+iitsubKpXn8geihVueNC4r3otCU2K3wngzi9g6HqZP9+DEmpy Id3Ktot3FY9ZwAs5OL6w8WGBCrqtCSLKjocuXzDRxTLLWSO+0VbrXMJxYF91hZlkAalf 8ULw== X-Forwarded-Encrypted: i=1; AJvYcCUd2+Yc4NT8AuyYc5WKQ6bs+erwlrzp5qwHcF3jKOX2zX+fY6d/3zbGtTyJF27ypXFkF25FjOlaTg==@kvack.org X-Gm-Message-State: AOJu0YxmwPWQu41mjNLkyVNiqQvdYzQfuyJ7NDYNC/c4XAiujUeBKlE/ WzyuZGOUR+lYTc/ZSkaxjI8W/YtEfRX3hATQSSOaWfqc88L3kPaO4CU1tQsMu3E= X-Google-Smtp-Source: AGHT+IETLNjbMtdResL0gCQrMShCwCpeGL/L1zL6e1zJRM+ER4woSJMvk4wtR2Cz+Z4m4+aIJL98tA== X-Received: by 2002:a17:90b:4c8d:b0:2cb:50b8:e59d with SMTP id 98e67ed59e1d1-2d5e99a5e05mr2805975a91.12.1724228400217; Wed, 21 Aug 2024 01:20:00 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.19.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:19:59 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 05/14] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_readonly_nolock() Date: Wed, 21 Aug 2024 16:18:48 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: AE4F720002 X-Rspamd-Server: rspam01 X-Stat-Signature: onozknbitzmyhqi5wdt6ndnuxf9tskam X-HE-Tag: 1724228401-780620 X-HE-Meta: U2FsdGVkX1/mREXQn/di5Rzk5jXoC9ZAbIa40jslwsT2+REq5AIM/tvZZ0eKsuKMhRxmyR76RfiyRb0tcsTAiqY4i9CEJZ1hnogAS6QJZnpnE20G/6i6iLkZJ6y6VdkJYt7OaabjUER0rLTmpOBarGj8l6cxUcADDA0VVp9fGxjtGtPyijAdHpSGMadTn5vsFOpWfC99sHS7KkHftQ1L9cAhl2rwg8wRJL0B9DTMphsiNyfwq7pp954XkR9cl/L733CMsXBCreFhmiY+SjRkEE1s0GzQPtvY6CSYB0JvQ2VGultZ5s0CQRKUARqXe4BZLRjbuTqwtztYF6YK913+JPyajCWCAM6cRNxKs1x7xkHtrd0Rg7CN8/Ux5L8c47vtsiRJI8/Ut/zTvme0yEd82t7dZAyuR6h0GHXfvPA3DQnscOgLx4BHSF98LiFwZyeY4/usDaz5w9jVoy2IF05gDLmLuP4wm7IU8x0pnG2pBAu8Sa8yaQN8+Vv1r7+loyaocEj8v9LAxOWFwhzpbXh281bqek8VYfPZF1RTQ26wrpqlt3hYlMUJcIlBMfQSsYvEo4pLUbSdZqplc7b27TlZGjwTBQXQTgF1w5Is7HhqwElJH0STKDd4SxnE3gTzUGCstBMnJp/Ma5uv9JtikFooA2aWLiLrtdc7+XkS25ZPrzYh3pnJ1JqAmDc1fxmg51Sd+0KMKSlfkw+P6U13/bQs7ExEltnrqcTE7fmmq1N+QLyiFbTewOko2fAcLO7rgrfYg2/wMiuzDgMoqwEQhRBDa6Xcrs2hcKPCEMgC9PMYD1G12txtOX7liFxtQZDC49TFC5cmUk3vS2hOlozEmYrYhT2vWcSabYV/++SPt0v7BEML3o07fi9Ng+ZGZV0YSxQa+6bNAZVAsSK1EtQ/HWivm1Kx3llKjvV//Y4pJAh+sFvcOnzQJUQXLgb5Ai7lBRGUdsAS6zynIJdWqgYyKAa KWrybNk0 elXWrCUlmTPo3FKj4E/LbcKbixlibfHUHNNNGd7COPrFqK5DAnqvSmLT0VDZ8FNRVTsdVf9nYF1PLM9sCwKP+91eEgOOxyh8OOAZr+3a2iz5oeazo5lWhUiCJGD6eIRCuOv48bqPLBtBPt9I5KoziWweL3F2wQNKG0Pc0ixgJEjV7WiyFpX5m8vZ9v0Jzm7scMe86tLU+F+mh0xt0Uq0w3adRWc2/QyKlvzWFhbAFJNk+9xUF8147rlea8qEVH04QjLjTe4Iy/PSlGAmcaftRURNKbGZ8Vgkc9L10ymW63y/zk6fZAM8kLvFEPrzw8FLEml96XMnoeBht9qSgnE8rP4CFZiwtTDBbS8RHHiaxf+sq00b7WggnFEUahsR1lhqFIVsd5ixKL6eGh4xIMSYm2dAWswykywsQxMmKCXBb1yThcTO8FgpZg41Q24FuvGygZv8d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_readonly_nolock(). Signed-off-by: Qi Zheng --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index cdd1d8655a76b..26c083c59f03f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1009,7 +1009,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_readonly_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Wed Aug 21 08:18:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FABBC52D7C for ; Wed, 21 Aug 2024 08:20:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8C176B00C9; Wed, 21 Aug 2024 04:20:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3B9B6B00CA; Wed, 21 Aug 2024 04:20:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B6836B00CB; Wed, 21 Aug 2024 04:20:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 69B4C6B00C9 for ; Wed, 21 Aug 2024 04:20:12 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1FBBBA09FD for ; Wed, 21 Aug 2024 08:20:12 +0000 (UTC) X-FDA: 82475555064.17.C0B5D89 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf18.hostedemail.com (Postfix) with ESMTP id 42D6A1C000F for ; Wed, 21 Aug 2024 08:20:10 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GdfPQOzZ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf18.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.41 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228345; a=rsa-sha256; cv=none; b=4K/YdQYL0mnRDy6uDg8/L+vMY1uQeC1P/zSgAHpmbg+F26Wu0JRfnmaP734UrkEVwI/ADr iqKzlBwoXGNscVBOpMv+G2Credd/KnVTt/Xx6YF//aVw8fkuAU0z7FH39SFYby5crEfH27 pF/8jXfZAJCDx1mQEvSDDTT1YIqhg9k= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GdfPQOzZ; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf18.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.41 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xk5nZGQoU/UHicxrt5T+UXYIL6+4L5PcGNmpQKRqaJs=; b=YdCcCkk6zSbwmbSlY1+JvLK03OLkG8GF6ixb/M3UkNLsmQ9VJNxd7s4xJytEvZ1syhtyCq GLx+8bwXeWNb8E0SF/2LH+L+Nn9saYxh4qVVe+lU9UiCp6woneU0hOHc/t0yFsTIZq3bv1 wmJupctaFiiF/RHF2wV0juQnYB2SBgI= Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-2d5f5d8cc01so117895a91.0 for ; Wed, 21 Aug 2024 01:20:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228409; x=1724833209; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xk5nZGQoU/UHicxrt5T+UXYIL6+4L5PcGNmpQKRqaJs=; b=GdfPQOzZSkCDg8196kkByoENjCFu6aQ7l3aY3UPHnV2qbvpCSe4BSQ+GpjIgfX8EUe y/8T7AzfrNlbr1Hy1N9U5DtwoIQAckrcStaeNVROY1J+8N/3OoxUAXAQzej2eRzdKQKY 18NV+a7PkZbqkSq/anfqbcrkntZLXX+ibQ2FFRUvOeZh8U3LBXy0AdQNYwemmsrU5jjS mqZfuiWDlOlUhDM32d+vgel10qxE/AJREf8K8mEFO8ctPJL0LQGQowqjBW4/P6jFkRvn AWShaNU6oy7Ns0t7W4gzOVtB6Y4R1OABDTmbnGoib4iAYcQmPYGeGWekY+hDweLdx6ZB FA7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228409; x=1724833209; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xk5nZGQoU/UHicxrt5T+UXYIL6+4L5PcGNmpQKRqaJs=; b=sqIEHU4HEp8sfM2dR3KsYz0rnbMROidEhK4H+suzZXe+AwuwxtiQC44lswcgomS6jv X3sQIMGcermi1sE7Atkhz+yoUXJH5gN60dRJU6Bcenu9z37F+K5OkjHoSchbOZveu34C CEoyt/34fpWUQx7xEqtLPO/exTmLxCxS+l5wIXC/GodYBnNyLLe4lfO5I3eJO7wJ2akJ yOnrquO0CxtlCtqjFdNAvQaHqXXripGqdO/gH6zo/CrWNIvsmLl5fK2o8+hdIxBal/QP UvByUGNHlpjpl/Lsw40R+KIWZUSBCDaR14C2krRjwuRlm3jnaqIKfvGhgmajTU/fCVCU Lzjw== X-Forwarded-Encrypted: i=1; AJvYcCXOZh7pNKqnNxfTCMunO5UGO94CTNvd4lE3ITEegUeM9ZWohLe8BPs7FhOA6sgA2kkDx99QxjZICA==@kvack.org X-Gm-Message-State: AOJu0YyLLjr9vDYR80nR1YGVGBTnWne9SQRGZ13AR1CxFT0F1jEIgaBy 14JkSx7k11lAD/PAQELArALezacXYODOdtHQGrt3tWGV8ysWE2hFbpUZRfs97Ao= X-Google-Smtp-Source: AGHT+IEz6hEUMU7QDWWn6q3mNXap3/YJ5XtpoR3UgLNTk8qimJ6ZhRH05YhkApxF5Pk84JIDmuV6pg== X-Received: by 2002:a17:90b:4a10:b0:2cb:5dbb:d394 with SMTP id 98e67ed59e1d1-2d5e995c932mr2717849a91.4.1724228407428; Wed, 21 Aug 2024 01:20:07 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:07 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 06/14] mm: handle_pte_fault() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:49 +0800 Message-Id: <239432a0bc56464e58a6baf3622fdc72526c8d57.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 42D6A1C000F X-Stat-Signature: po7cjwrskxf4yytojusxmrbb11nrkpsw X-Rspam-User: X-HE-Tag: 1724228410-819239 X-HE-Meta: U2FsdGVkX19ziTKsT7BeZvATQSAGfGk3mNUpc9sjQqmYpg3A7vJDkktwYN14Lajhn1+Jnzh1uT7YsO1Ykv2O3TZOssMqM36KLs3Xtd0VgEyUhqutLzKIVxKIWTdQrlPRonqXJGI9t+0sD+WUFzGfNOKZg3G0dlb+UocS51Zm04qGxcKFinzgUpCNCt8i5z6BLtpB67ZOKp68M29ENhg73N1eB6yOJXJCWRU/OFdBkoISVJKZ3mWVWQRL9HNSOG0SiBczs1MNEsF3W9jWPvmMsARMQ7ABrz42OuUHGeqTphlCCqFl9iMVz/Vi7KZ/xDNLCKPe8fcHIe7kK6iYiJWh6qd81y4LBmX1VeRKniRm9dkCeVEMhL3r7ZfuPzELUBFOmTAAUiFBt5riyH+0qH//bojdnaSU121KtPlHyplxCnE1pWyhERgAIOxD0i9oHaqdVSkjmX5KvUF9fakQ258Wu4v+651FLTi5aKTPPBc3RLjlyRWf4bpWDrkmp39wvp26cIqcLYiM6Wkmo//ZNkbvx30HHh8M6dcagbO9E2TiixrMP8I4imqcmdt343WE7BynT75dl1vEJPi0zrSwh6UL3t41MsiyIopjJX9BGQp0dHMip7pY84SasPgTG7SXkDCcGRw6gNQaxLmSelYactoP3sKLRVjOwI0YpmQSeXANoYZS4IeEfRg0/FAuL9G2eWxDDl6QIWy+w0rFo25/zbD4pHJXt1jgNdi7gMy3chdX9PjI8IqoeuiE5azxJbIDAp713bgfvABtFEGGJdwdzFW5VqvXrfan0xHlBzmYodLjbI2JWu7FQWkR5jWizKBQg0ZiAqJzmJTVj3oM06OZoMf6RwJEpt8Ngcg7y4r/va4KIiunSPQwEx6JwbXer7JrbgaRrj28/ACp2EGM1JBXda4UvUaMpp7rZad+JlDhmKK/LMBnMrxvVduu1jB+IJujLLmSJg6QA0HNMD4IQbz8Oz+ JrMEHsFO hoEm3aFZJGfCTNAObELsy6xqbwaNJXdvSNdIS3XhKv2m1IrKkYCVUoYlfjq48oA9fjPzskda/tEwo+Z9BrjVLFmfNcNx03slY+aLoE6n1cYwkQ7YzGGEVrm8sLo7gSkVDHz3nYQaKwdxU9ePtfw+vH+Cy6LRvPh5qqARJbofxEjQrzPmtwLr7ijzGx0bCArkqjiTXz9Z9wvBOlcbWJR+DKxDo2SHWqqDEfvGBqjC3vEvPbYtyDyEsr0wG2DtVQlZc5W/Yh2cc5B92DlbIPDu3XQUuLAZkPbGnvm2lBv6pS1fiCxgoSrPj8Pp8KIfNPha7HRI5MupifABJjIV9EBxkO6/342PS1uG9JlpKeE52ULscvUNG7W0Cb0BSlSdAPpI2eJk6U+vNt341uXNzNEkteNmoQeOFz2wPB1zEzsytArooY6zG5vKQBvfmlw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_maywrite_nolock(). But since we already do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass NULL to pmdvalp parameter. Signed-off-by: Qi Zheng --- mm/memory.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 93c0c25433d02..d3378e98faf13 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5504,9 +5504,14 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte will be + * modified, but since we will use pte_same() to detect the + * change of the pte entry, there is no need to get pmdval. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_maywrite_nolock(vmf->vma->vm_mm, + vmf->pmd, vmf->address, + NULL, &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Wed Aug 21 08:18:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAA6CC52D7C for ; Wed, 21 Aug 2024 08:20:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6948C6B00CB; Wed, 21 Aug 2024 04:20:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 642E26B00CC; Wed, 21 Aug 2024 04:20:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BDAC6B00CD; Wed, 21 Aug 2024 04:20:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A3C46B00CB for ; Wed, 21 Aug 2024 04:20:18 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 97F0D1409A4 for ; Wed, 21 Aug 2024 08:20:17 +0000 (UTC) X-FDA: 82475555274.30.8200679 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf13.hostedemail.com (Postfix) with ESMTP id C222E2000D for ; Wed, 21 Aug 2024 08:20:15 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ARp+xsnr; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228335; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ATsR6rCpxZFOFORcauHss5xESVmegH/7Abibhpgyw90=; b=kirXcut88OjEz2zGW9+xg1M9W8ljFA6lY9bF9wrEATc55uwJOgcpozqtSZkdW3jqkf9K3r K2hWW6+buWU5h2BReOKF6boeuwlo3/s2VcFg9LBoeEwOD85m88jAfK4FE9eZNDl8o0NqAH dsJa5oGOq/uLt0gPwuK8bozg36B2In8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228335; a=rsa-sha256; cv=none; b=qFauzgrJxTGSCDuUo9En+iPk5g+gKMgsVPt8lkjvWxZzGp/hihT20ikmfzpUl4+7wMWqww hU726gs5+G+pZGOlV034oWOjCCkNph/672GCp63g2X0t/ZL0nFgV16Sk/g4Z2d0p3MYgS5 opzXtO5K1+6b1G840j5YKJyBYsz2fsE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ARp+xsnr; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2d3da054f7cso3951823a91.1 for ; Wed, 21 Aug 2024 01:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228414; x=1724833214; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ATsR6rCpxZFOFORcauHss5xESVmegH/7Abibhpgyw90=; b=ARp+xsnrbLXGzgahhdCk1M27IbMsZm4i3Z03IW2gIvucn9ryCvOASP9xHL9AiRMfU3 mvNp2oObAtFvLC3qBVwMrpVTvczybZeGsTeeEB4tVGD0RXvhWZ4hBGcD+78ez82ZYT63 ZNeNTHuCWwkEa1U+N/gRA/cBPvlw7uvCn6+t5Oodt97eu16suqCXzP5HUtiGBXimtOKJ 4Ha2lNwkEenRhPFT+WTd2HHYyN1VzjzgPg7RfO7xuVAk6FzyybdDWHvGKrav9OrkDtXM uJt9BPQJfoFLTzhC5+SNsOg9WSEAEazQVh8anUrnB7RGA/Y+IcOz9u90tZTV3MSJoLgy KCLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228414; x=1724833214; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ATsR6rCpxZFOFORcauHss5xESVmegH/7Abibhpgyw90=; b=u0fVJZUfxqP/Qbt9K/MFKhUz3sLhCS9sMQ2d3Ys6Rb2JGZXycmy90GBq16Y6/NGAes diPwIdGXBDXlzNbItaadPYos8Rc54QUTYUuHJn++MOxM9rUEjxkQWsFkWYRRVMH7mDRN zjtM6gywMi/yvHsGeD9zU6cqkYnYEKOtXkSV6jSQ8v/BsIlXP7/d0lNMcJUCHjw3WFZs UfQecwYRaPaDxbouIAN7gBkrUhD3inonC74VfHsTTqITYd3M8meCVvbhOFo/tBxO+kY5 /kUVxw84yDtLpWLuahl0niYtKCYaCJ+fru99oQVkEdQIKR6GLcPleJW309HwJuMWahvQ qoDg== X-Forwarded-Encrypted: i=1; AJvYcCV1Jyp9SUvWbj/iDj3Gi4hk8AHHR6J+ZZEK6r4DX1mPH2gftPJCGPP8YZgMGCAzphAyqnQV5RB8+Q==@kvack.org X-Gm-Message-State: AOJu0YybOGtryS5wxFPePiRg9a+ZqjGpSO6+bMEuxpt5ShDoalUOd8C8 mYMR5m1SJR2i2EoQf2UqpsaPLJ6Er+OQCfb+hOm2CFDXu7a5eSpXM3x8F/8pGLM= X-Google-Smtp-Source: AGHT+IEASO/tYfbmKEqODBO5ptoLkIYqajvyb9bjTzXqrTBGFe58odiOJRUvoUKiSb3oI+1Cc8Ov6A== X-Received: by 2002:a17:90b:1c01:b0:2c9:9658:d704 with SMTP id 98e67ed59e1d1-2d5ea4c9ab9mr1366119a91.40.1724228414402; Wed, 21 Aug 2024 01:20:14 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:14 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 07/14] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:50 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: 7q6dabai3aubed4rzs5ckpfy7174xumx X-Rspamd-Queue-Id: C222E2000D X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724228415-14292 X-HE-Meta: U2FsdGVkX19ObH5P47tsQEDjJ7IpPMuVN1kU6YGz4NsQBlUPGt1LtysywXnDepOkEr58ifjuo0AQljgQYdZ6hfXbj4vuseEjuN97kYDReX9QwLpjmW2GnFY7Lzx3M3rdm818NkIvgUK8TfxOq8A0B4ctZaNMz0vh4DcnBu9Jqyt/q0eYNI5eVR3yLar6MwiUdu9rS3YGkT9oxoIT31kT+Ys8Y/SbdGU3yz+ptUnBoP4x70JeSUL4CVx72hfzUYi2xPAfxqKxQWtLfJH0TfzCM8c1eK7CK1pzYEoC4gt5MlIca2RCW1S+mE6C2sIgqYqhVQh3CGlByLiubJxFZlW4i94eKsogSeU+5RYgcRFwabhC/sMrm4mNsG072XR3WhtxsZ4LKbdb30MPCFOXE+glS34F1cVqYRDlOipv5vcPVrysfD5NyzVYmRht/SgdcPcxu3jAVkqN8P1NAJNpILn/uRTVY+pAGbhrdxVac8SkyPHm8Ephm2XO5e5q1fFKfzVOCjTudCW8u4DAc01VpsapQTrC59w7fpz8YY+9gpAY7GhfPfdkQ1/UwfUvYJC76wMpVMG594XIraLIjXSiwjxJWV1P8fnbCetYEQnTSh6cUgI4LdC6Mox7Do/18rHw/76WxlK2wFQFYQrdYJvmkpSG0xUfZcDyEyH1oh3CIrZ5Y/hk5k/N5EAceIU149hj+yjUswbfylAM1nF2H7Q1n5BTN1p7USGQ6LQBH84qa79anE5DA+7WIvOnd+iPPafQfAkNBo3MxAEF+xRwkYuxVS1hsc4NXjiQT946h5wDb6v031e8wFYzwaf9YVT5gnyKhKx2tUD0GQK790UloLRF6c/qsyz0yNnPTrWwOId+D7TRUPsP5tCU+G0EyaNdckjKMTqO3YzRm1NXk4o0yRK4KvMVvZ5dzQgE1gYICcx6kP+2Ab5daalZXI3N7d8khq6LOqGwcK0CHYFzjZjNUDmJsut ZnjLPRrk FEWyw1tPQqwxqcA4lZZndtCG9lYsy+CJQ5FyZJoZPqjGnRT5eFvC93BSfJ1jyLf0Avxf7j5lk6m4dcyqhrjXBzOW1Xy9gp/6Zx801nMj4u+pzoL4qjjuZqn7L2BowZcs3ToZpNAs6e+u2dqcxevDreNDCuh4Xwsuhfmg17cdGW30FiVE3zzfubYSDGw73KpFWJWmLx29dUpM4wXXgfSgAO72Sld0KrMCVSR+pKfH11LkWg8FZnc+gUkBvtwZBysEd6iHbMTiXroBep+/yFzHJcMDAYNeBcY7LRlTuLQ7Db4arjc0+fgKR2ltQc3/WqE5fUQL8Bf4WuOl2u5IpKSQ+9wQyPftXFjrTos/9QB5kwGWyrAkL95psBGO589JG4R0NoSiqrRXMIHHnMUZ64jZ+xa96tAeXEwRwGThQKN5+/XnM5c20ush6D3w+lg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_maywrite_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. For the case where the ptl is released first and then the pml is acquired, the PTE page may have been freed, so we must do pmd_same() check before reacquiring the ptl. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 26c083c59f03f..8fcad0b368a08 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1602,7 +1602,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_maywrite_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1610,6 +1610,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1655,6 +1658,16 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); + /* + * We called pte_unmap() and release the ptl before acquiring + * the pml, which means we left the RCU critical section, so the + * PTE page may have been freed, so we must do pmd_same() check + * before reacquiring the ptl. + */ + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + spin_unlock(pml); + goto pmd_change; + } if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); } @@ -1686,6 +1699,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, pte_unmap_unlock(start_pte, ptl); if (pml && pml != ptl) spin_unlock(pml); +pmd_change: if (notified) mmu_notifier_invalidate_range_end(&range); drop_folio: From patchwork Wed Aug 21 08:18:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 667B9C52D7C for ; Wed, 21 Aug 2024 08:20:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5D3C6B00CD; Wed, 21 Aug 2024 04:20:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0D476B00CE; Wed, 21 Aug 2024 04:20:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C85AA6B00CF; Wed, 21 Aug 2024 04:20:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A741D6B00CD for ; Wed, 21 Aug 2024 04:20:24 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 295C440A67 for ; Wed, 21 Aug 2024 08:20:24 +0000 (UTC) X-FDA: 82475555568.22.7889C8A Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf16.hostedemail.com (Postfix) with ESMTP id 4D09A180018 for ; Wed, 21 Aug 2024 08:20:22 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=DIpIi3Km; spf=pass (imf16.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228382; a=rsa-sha256; cv=none; b=AZkEcOwKv65mlExqO/5QBpJVdpFE+j5IXBjG3x90FqvcIkZ7bi+7SS26st9PPVTyj+f1lE 9oJjLItp4pm5blkkEF7t9lJXJQiiomdLW6dsgpvOSKtzMGcFOfO1wGfYOAdt2eNLZ5VVYW YuAU49j9R6I8ShurHM9Rcax3OagtFbY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=DIpIi3Km; spf=pass (imf16.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228382; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ib2peXsn8Xzv08d3et99PPledeQt2suHPjEB9mw45Nc=; b=gnPKYe7v113wCAJzruOF0EHhL7AwAwyzcWYpVALP8R2gKv2DiOg109nPx0M8W1sDXyHlLV /HLDy78mOkFsOjwPppP2fsMyb5OqJJkk6BExH65j3KmwICeZkfT9ubVgul/p2LC+JfCwQH C+JcxNFl6QwOocV+Tsnj24Ykk5AHITc= Received: by mail-pg1-f180.google.com with SMTP id 41be03b00d2f7-7cd8afc9ff3so221234a12.0 for ; Wed, 21 Aug 2024 01:20:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228421; x=1724833221; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ib2peXsn8Xzv08d3et99PPledeQt2suHPjEB9mw45Nc=; b=DIpIi3KmTTTvjlzxqPDbyah8JhGwEp3bwMN06POXmLMKTTNTQJYoqTwlltnuGWW4o9 472EfzF93R8xqjr/p2BSX3CkwXj1hTkE+EYKTayDpExAH4D9lnCfgK9TlWa+Ne+9CjuR wxXKiWFONPO5sEwBIhHLutr4bR+yRMeDXNFiIisp5LI+bgGyF7awEcGzCDAzbswwcVqE kFZ9VoD836R5vKvX7kFBUhmR6IU+brhUWKNR4TnaEDPBdoiQfX/2tWhoU5rsLXNeV5ZH 2mywZF0Zm6vs7a8hI5CDegQwixM2rsCDpTp2GBwwAktKrT8B/bQeMEk1fCpnKDqiLsTK CbjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228421; x=1724833221; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ib2peXsn8Xzv08d3et99PPledeQt2suHPjEB9mw45Nc=; b=hv+u1ZsXFCkRhIVpUFCWZdZQTEXXtp7WfaOkt7uk5R9yUYZ2vVei7n/fn0iDG8gebF SHNV7WvPW/sp7CKNOcibX6Wzjkn79iYM8RIEXDM1x/uJ2T0u4k+GlzcoCvOd3clVGtBM xAAuCjj1G5VdWdd1+YYX0lacDEkBD4YLvocer3CZARojOBxXwwj+Q0eYNlSlzLqY8N2/ Lbj0cMpsglbP6/3uy1C5Y9RjaIv8VGtPczbdTAwaR8PvYkE6qrREzqPH4pf0Mq1gOueh o0yEXiIt6oJuaMXAHJXIHh/X6atPPBFnDPMT2OJBgbkpA+YU/5a9Ybzt1wL/mSMnlCLW 9Cqw== X-Forwarded-Encrypted: i=1; AJvYcCW9PAJGV/uMrVRtW5H3MGpuoYs8i8CEp939Y/RTb+9s7o+gbel/CyT3nFQxjeOiITM5ZwBKivwrxA==@kvack.org X-Gm-Message-State: AOJu0YxBSu+M7KCuHl4bWBUIP6cAgEs9ACOElyrQB9t3a0+1CmPdSjLp jqmZMVmvIKAkhnxhIUEF412/fnj7Pv/JExjlXzmUVrdvB44+6E8pM9R0qkwKmhQ= X-Google-Smtp-Source: AGHT+IE93wrdhvj1jPWy5zvxTPwSbhlcdosDYnf+NOlDda6ytSjhdKrpXWKVADSrOUal4AF7Aqrd8g== X-Received: by 2002:a17:90b:350a:b0:2d3:c084:9d93 with SMTP id 98e67ed59e1d1-2d5e9a2f0damr2610948a91.14.1724228421082; Wed, 21 Aug 2024 01:20:21 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:20 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 08/14] mm: copy_pte_range() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:51 +0800 Message-Id: <05c311498fc8e7e9b2143c7b5fef6dc624cfc49f.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: yg79zhxzxm57urh6jfbsxmyc6yw1dkin X-Rspamd-Queue-Id: 4D09A180018 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724228422-920983 X-HE-Meta: U2FsdGVkX1/cbYPqHmHQrlATHNKPNJH/6kpMHQSueAoKIEt6XDa+v2WuZvlirp1tiMOILDdzP44J83LbVSp30bmnf5bOuwDLncpAO6aT8VPzFI9BhPShHPtdcgm5BPO2hlB0RrCPdU0tqjAtMZcBA5fK8QHtv8axrW2XjzMyfbhfQDlUJqI6MdYlK+Drar4yK5r4Cp39uUzbHSwXxZ23FMzQC2dUSrDBl8sYzCQu1YlfCXyJfrcyAuaxdrqPzAMnTfV7G+/xIHVW+sDX1GEKYXGXxCTWNcly8Gj4TEuk/ayzGsTuBvatKvpv1b+p1MKVjCrmW44YlFL7uCFDWBqWoiCJ3mdhnP//gThax3WynFrMmzne2lb3pq+I/hrqxQKUGl1x26zKnDPsw2rg00khwn95f0fORlMWkf1mv77+2vM1ZS/cxADb1oJGnHXcavF348XJ5HN6qiH6k15YuN/mOYZN7BPKIHTBaP5naf9OElipQAcC2AGMa0zsUInXm0NcVdMxz7Xbw6BkaSik+YvybXsbTVeBYeAn+IZKoOmKa4i1xrai99qZoZsIabqQt62BRfuDNNt23qs8JP6QpkmOu3WpC3OD49DA3fbj5WEbN21iRzfianMge+sZQ/hAvO+NX5L1fw/E3GqTNjZVQFyaVHEVsqM/Vs0tnUS4HNKVtYPPNGj1Sc+1b5c6zNNt8U2bxyaRCRvAQ6v0VHZuE9/vWdCDz+o5htK5gIQlIje5dLoWsLiYw/MLcAgRoUoRdIuXcfxxeOFsK9c7Sdg7pFi6d1wfQYcE1WfpV3uKgqtBu+jkmeQcHz7TXw9dyrrmmpoY/huTzIoVw03LOOA2pRtzkifqTZ+ama+qY0AaUzFW7UGuU5594uD5AO8I42n+RVJ3gizlKW63cJqUzBKvZ+zNbVHuAoSxOPqY6uW5zet50+Bz4q1KaibuJW+6R8v6JSq2L6L6PP3VuInLfCXzBMZ fXgANgOC QOKT/7ibHPuMTciIz2Riha2AOPha9BV6DM3TPBDXGuDmgeZhM5HOjZErO77xEQLvadyCuyLLqbIKwc1v1ltCDVgiNma3B+0eWPtkV3N8Yq4K04XDlgBUZMkrEYmr+eoT9wRevSgTDVwiWNE65cvLtpUrYBzLyqPHJod8167CitRt0axJ1NgWikJnWqEAyC3GxMtIHEHSLHqg4mw09hG+im5IDVpoI51kXKCExu3IoCFvFsqBKD5dB0l3g8iKSsfrrQuOIJU3XRO852DL32k1IrcynHQNwR5D1gF7BVdCet57a0Ty0pc71CcYEyU/A9B9y3PBZFi+HcilhLy680hzr+bOZtcXJckiR9r6FHmihlbZT96UCRh93i9gPI8e2oYbJzbPPjStF+nbRXFpU4/1vnC6N9R1Z25LK8vyrLdPgHLbAtaMZPlVuiXgGk0GmAa1wo/VW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_maywrite_nolock(). But since we already hold the write lock of mmap_lock, there is no need to get pmdval to do pmd_same() check, just pass NULL to pmdvalp parameter. Signed-off-by: Qi Zheng --- mm/memory.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index d3378e98faf13..3016b3bf0c3b0 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1083,6 +1083,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1108,7 +1109,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + src_pte = pte_offset_map_maywrite_nolock(src_mm, src_pmd, addr, NULL, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Wed Aug 21 08:18:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54096C52D7C for ; Wed, 21 Aug 2024 08:20:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9CE66B00CF; Wed, 21 Aug 2024 04:20:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4A476B00D0; Wed, 21 Aug 2024 04:20:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC3426B00D1; Wed, 21 Aug 2024 04:20:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 99B706B00CF for ; Wed, 21 Aug 2024 04:20:31 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 55ACD160AA4 for ; Wed, 21 Aug 2024 08:20:31 +0000 (UTC) X-FDA: 82475555862.20.D128463 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf18.hostedemail.com (Postfix) with ESMTP id 6A2261C001E for ; Wed, 21 Aug 2024 08:20:29 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=QNTuJmjb; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf18.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228364; a=rsa-sha256; cv=none; b=7YRco554mEdIul6M217B5A3z8JrLICPSvKhU2hPwKI+gScdKG9S8pIGA16MkN/2R8yCWTI DTLyduUhrVd2korgv3emIZn8qjm3hOgAv0XjDwaBPTrXqXh8lRH3vl7/7VPNO1TaZmI4IA 69FBqDunHuhENFo464PTUmHgBYW4XwY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=QNTuJmjb; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf18.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228364; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6wf5h3KMWWmEe6/3TeJpf55VgCdFHsYKNAkaT296AWg=; b=XIgzqTCYbtU+Xmg5zEeExzoddVzt5Y1JZrxvSLjipnCe0LbKsUXIAb6FiwXWUQuXB17eaT WWduxKDGyFZFHBWSt7BMlnPnHlCuFppUxDFwXnujSGoDRcPK8Kf7a+9T3OqUWXUflE4rAk /RtpyNwEo4PRO1N/dzcZQrPkuK8SPNc= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2023dd9b86aso25747925ad.1 for ; Wed, 21 Aug 2024 01:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228428; x=1724833228; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6wf5h3KMWWmEe6/3TeJpf55VgCdFHsYKNAkaT296AWg=; b=QNTuJmjb9Vf20LkYSKSo1UKC3Yidm7QSIUKO/Kc9WE51HUToFRQ0bTYIpLbeKvP79X 7Nd/+HIbYx+WrA3Pf/rSpPMHeru7d6rV6PYgg97cSSiAZA/T3ykcxnNLz3Vzd95Y5nUw yCqSMijUpmWWOnZ2DxnF3YWuPtDXkxye7416CjvYk7in8xDOSpel1vAAeYNjUxUdapnS WrxiG74ajIXRt/qKdE8JXUA1CMW8GVzozJIWCxd6MrDAPzQwoftmJsjias1DadyA90rv kCAXNXSMlvbIPXv/FIZ55ChfEIj3onSvoq46Nl5fEFFhcYyEE14LXWjJHljNI0sbpV8C gImQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228428; x=1724833228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6wf5h3KMWWmEe6/3TeJpf55VgCdFHsYKNAkaT296AWg=; b=hRgeq8Ffhbf1duS5TvOK4O/3gWkaxNNlrAVZcPSSqUtJp0VG4vAxto6+uKiiGzC8bW 6YRVZfLFNk0DQV0Y/0rzfRFzyhQ0NuJJZvyaU6gcmn0QONzTqHQnPze8KS3I43UGXCuT vG2ubN6ZIuX8E62zkzqvjTeiZKLIkz4WQwp1lAgmUwD/KASA2lMGzzLb6HISN3t86aND RYRy7lg2iq5a6qL4Gag+R21lrktjuvVn+QaXTB79x/SC5QOIrOX15Z5dTdVIxExH/Xqf oUjH3XwzIq3BPWDJ/C9Y7NVWOO7oCljqyyrRhplA7165ZI0W4UX49H15nyzRH98+DkOG 2wZA== X-Forwarded-Encrypted: i=1; AJvYcCUOeH8oiL9dcoJ8NhxLfn+JElie4dmUmrXdckdMm5SEI2+ivywo24C9O5DEbBJRaDojNUR4nWIk5A==@kvack.org X-Gm-Message-State: AOJu0YymZRXNa1OqPu0sP88ldtOhFSZBHHNu+Wmr/q/mmp49E8npbUWm 8CQSqJ9n+nXELbd3See3vNcKISUYH/4bOOEcgTqCLbqhvbiUYfA6z0DM9f+YRi8= X-Google-Smtp-Source: AGHT+IGct1lfg7s+Av92JTuFI7wGr/k4q1ybIYiVEVcgmdImld0nRYlRbV5Vh9eNiEYSyV0nSbS/hA== X-Received: by 2002:a17:90a:7893:b0:2c9:635b:7271 with SMTP id 98e67ed59e1d1-2d5e9a24ea2mr1616047a91.21.1724228428100; Wed, 21 Aug 2024 01:20:28 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:27 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 09/14] mm: mremap: move_ptes() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:52 +0800 Message-Id: <89bd1d48db1986ec83dcae228defa748f38238ff.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6A2261C001E X-Stat-Signature: nrjxyx4zniqoo9w19izbh3wjpiin47yq X-Rspam-User: X-HE-Tag: 1724228429-575899 X-HE-Meta: U2FsdGVkX1/COWJ9NZx0+HOl76pM56azPGbE/jvGZAEu0Z8tTf64j3i3RoSGKSfKvjlf9ay+LMfqlWF1cGFR1D+pA/gCrgKHTtsgKDH4wyqPFIMy0FVQDXSEPYJerY774e+MneNqlUIwbComkGespTsNxQWzgDc8SdOvpYVsg539MTyB71lyd5Klm4PRGRQXXBRgBD+F5CUJ3PDxGxfN0Fc9Udl2n4fe3Wo2f+ValkupZWRB59y86CuEGVqgW2EUUI+sKrAWL+sl5T/Evwi35D1KcqFcKQMq8sLO+I91gJDniHf8nKonfTlOyO90GiHMvlgYAfIDNbRdzGxGANUTs3LOmQSM/c30FmXcP47CWOgDndryR7d4ERhl4E5D+AlL89+dDoaUtygQBDTYkcqvuBJbUqWCcot+vIEGnRgyYUsKXK6tp2kd4nV1iiJvZKeusmh99RNZqnan2/z/DsGftfoRY/7yDdrmWC07G82UhU25/iKYIRW4kUM0NGWFtNXZgi69DNk7J2NtF2f0QeRUTvOz8evOjT5DullbxAByTuYEUWrOrhu1DID5xanHQWSZDsGW3Y2JChbglGB4rYPqpPFFJtP7FSKWIYxmiz7o3dfmShXaUrUS4+4oZnxoMo03JfTYr03kgDl+8R7UiwyQfgbMK/FBYU5lujJbTkbpp09gTLqvbLetKzfmr4FaoHCztBGDUM0N3qVa2OEhUKqRrsnlqMTDSnaRz9jRgo6WjbAjxaZwes1oG9C0QdZU7TwVQUWS+RvkeHrDqdtsbEaDD4N5HEeflzbkiVxXDhvre61b6ajjH2wk9Tfe6xjTuHgvx51SVg9nYi8oDZBPrkPFFsnhth0WRQTDkfumQod2egN1vqFK23IgqEnjDw+UyPVa46VAiDl/DvTqRVUM8fMiUEq0fDEBv7dL5C9ry94a5nO8mQ3ZwWr4a1+sqP5omYk99gcKdgEity5xRwEoZTh gFwsi9w0 dW3rw9B6iTagvCzhStn3ga5F0io/ZGNwvGaej053H8DsujZYmszZCEKxrRmWTyiPXIZnWL7yKIPQl9d5kMB4556G52Kz2LU0NjdnUkRmnNUopKGxMtRDpvWDWR9BA2EFYlsyUcX7EQaHC3U/+hSZZ/Q3Cs70KLSCb9bheCB8SAmQq7Nd/90IOOswW+nfEe3DjUeT00KmTS+Fegq/XOIxrQ+fhyXeNVTmYCDIQMlc1OulNYZ2yz593f0C8cotmLnsS8vTq018kQTvDqtoPeHjb585jK1J4s/U9KHSzlEvI/VyrFa/3p4+WffQYu5z/V4w3wZAuD/rD0x+FdSRIFuLE4JTtQyVIfGs1BlB+qrhh3jZOWrye51o8VcO/YXZ686mcV6E1zdsUcNZQwIZf76ee+POz0sr99lB3lqEiQA0AdG/t0nYojEQ7ZvE3Dy7Ayh9KDViS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_maywrite_nolock(). But since we already hold the exclusive mmap_lock, there is no need to get pmdval to do pmd_same() check, just pass NULL to pmdvalp parameter. Signed-off-by: Qi Zheng --- mm/mremap.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index e7ae140fc6409..33a0ccf79c32d 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -175,7 +175,12 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Use the maywrite version to indicate that new_pte will be modified, + * but since we hold the exclusive mmap_lock, there is no need to + * recheck pmd_same() after acquiring the new_ptl. + */ + new_pte = pte_offset_map_maywrite_nolock(mm, new_pmd, new_addr, NULL, &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Wed Aug 21 08:18:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC7CDC52D7C for ; Wed, 21 Aug 2024 08:20:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A72E6B00D1; Wed, 21 Aug 2024 04:20:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 755AE6B00D2; Wed, 21 Aug 2024 04:20:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F80C6B00D3; Wed, 21 Aug 2024 04:20:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3B5886B00D1 for ; Wed, 21 Aug 2024 04:20:38 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id EC4391401A9 for ; Wed, 21 Aug 2024 08:20:37 +0000 (UTC) X-FDA: 82475556114.24.A42B81C Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf26.hostedemail.com (Postfix) with ESMTP id 136A9140027 for ; Wed, 21 Aug 2024 08:20:35 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=QxH4vD5U; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228373; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MvwQXaavnjACpbuRjCPShfvJkJjZwFPa7EtFYmq5Q9U=; b=hKY5FeunuEilAL88ZDbQ6L7UBhGBEzwvx87x8+poBZNrDomBJ00eBK82lzmfVX7BCJucSL 6vO0hkfRf9yxQ1Ejlr/Th82FNfYdt0XyeD5QP/eRcdPaX0fIhG+Q2+emHDslOUxMEE8Oto L5WLFbGge3IpAPk9OXvqt1Q6i5m1m8k= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=QxH4vD5U; spf=pass (imf26.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228373; a=rsa-sha256; cv=none; b=U4geRUhBBhhpDm5VofS5Nxu4G/e5Z0OjP1whJaB5MCrBT+XDQadbp5lv8/9fkLJg6Sv6Y3 DrEjrZC2rslsCIbQ8CI4jgpuRLqsfMC6vRem9IjJGMSJwRMC/RcgjSLm06J3iT14m2BfBX +UvxyqVSKZ8LAmiOFz2T3ugKTYUcngs= Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-7c3e1081804so4237100a12.3 for ; Wed, 21 Aug 2024 01:20:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228435; x=1724833235; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MvwQXaavnjACpbuRjCPShfvJkJjZwFPa7EtFYmq5Q9U=; b=QxH4vD5UjPAXhUpoT2T+Jv16XfUUmrqC6e5N7ogf1maM8rDGs3AEseU/WgcPEmNqxw f3ViU2Ef14EP1t1JElQfbNxXbRL3FhDRmE9sKjdAgKSUcITaRY/eEHtTEcdYVE/H2Ylg gbPx09gyyxN9Z6ottYHQhbfpZP+IPkefRAmKGq3w4jCY+szOtPQUufkqfOmThZfv5RI4 zhkkw//Gwk6yxCo6lJU7UEIHhIM2xEnThzrtSEpVFprpeXY9iIO3V5b1S7x6piMyzKO1 l7buWeDh+l7QxtfbCtQ54yQ3IMKLz5Cf9Q6s4x0FLzf8MGdJDogt7xVh2rChOdk0ckuG XiEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228435; x=1724833235; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MvwQXaavnjACpbuRjCPShfvJkJjZwFPa7EtFYmq5Q9U=; b=ngbw5Pggdep6M5PZdyA5Y2CHyPMaxNXuS/xZefTL8CZaU//AOevrMmlqnYZffHnm12 M3DedxkHUi6asdgdTC1dFAaRMyQp25sBX6wO3I7c3rqGHGRWX2o6Z1m3ZbydCvAhPCgw 6lpAq6xAF3v9OG4TgBqfrG7z+NzGuS48touNgeUPSF7G+PfrErCR8lvs1E+kVrYHUksB 3tPJtFUmwEUv8lZtjOCiL+aXznA5lXyRrPejBBGPUTLH8RrIAzfVxwQcoEZGynbH4naq awIjjiSHe4/jhZpvQInng0jN/58lg8wPmjJa3jhZ2Xs1sveC5sp63QBmm/aqoRpg1HxR qBwg== X-Forwarded-Encrypted: i=1; AJvYcCWYoGj1b+llVT6nMjTORccBDDoPQN5ZPnPy9AuwtFy8oKUd6ChNEPM1tkZ21mhbQF6rSEKWcp/pvA==@kvack.org X-Gm-Message-State: AOJu0YxtwB37wqWIPat3rYQCZrLmwU+2ddlCbu2hHO1sOGW97AIrAjwZ 6yOAmrrebqr9V3i1YHtHPiSk6mvxreQBD9fClsXXQeF5JLM1l84SO0R827a/nd4= X-Google-Smtp-Source: AGHT+IF3P/4155uMvmGYpoowuXrhy2v6aHwudYM9PJFMCLkX9j2rCBHO6+Z7X1wkaLdhxTlExnQxpQ== X-Received: by 2002:a17:90b:3b41:b0:2d3:b5ca:dedf with SMTP id 98e67ed59e1d1-2d5e9a0db5cmr1626393a91.17.1724228434854; Wed, 21 Aug 2024 01:20:34 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:34 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 10/14] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:53 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: gshfpqnoukjm36betdq4xffbnsr5zuws X-Rspam-User: X-Rspamd-Queue-Id: 136A9140027 X-Rspamd-Server: rspam02 X-HE-Tag: 1724228435-685371 X-HE-Meta: U2FsdGVkX1+8nKnQ1bER+NlC3W5J+ei730LPM0TwZduvVaGR4vSn9GtIzS8ISdTyxyooafGD4+rDmMMN8AvlGRhyECYOzuZXrY4ofFc+K7dQ8EXDm8uJFWr6kDrIHcY1jzNOdyRtgPCzhnicyJGttVdINl8teHLAoqkQFSkrXt9ogWWLyFBSYA3TTi0B30v0sEeDUxMQdZGimijSHvrQyUgg22rc60kexe9DCz/8aFL6rPkmXyVkGv5GvymIyEZ+7U82OASww1MqER9CBNQSwQ9F3KhEKVOKHfVuOIDnWm14m6yoGX+8ROWxnURJR0BVByEldtGexXByOpfeUIb39updyvNvT4bJ90kKBmALMqdR9zsaxVfJRM/AdT9sdOJqPvSaBiHu0BOu+30Eg9sDwnSK7bMI36o1PQbz4svSfmWCd9hp3McSAzHTPo14tRnGjy1mp6lTYJWQc23bvAguyQXr5bViK5m5haIGerMzkJcDyOjg7zIKNtX2+9d0oJtl3iYQXAM3B+llOHG+0wHUhEWucjt0Eg6keb9N2zqglbkSwSe5AU316S6ZF4tAnv+1GLd11IMCDTxA/jpqpMJ+73We3w4d0UVPj+iX5obiDzSYyVUK3wzT8vbRzijrkejl2awOGNPmUn5zEwCvVVngnny57xPsPJ7xWamojUyeRvQT6+5S0QMUw2Yb4g9S6q2zbFdwHoU2UIJyydDEYbd1R2rUL0W2BHJmYoFCbR3TI5UT9dsDmKILStVXE1na0CG0Hf3C8lZ+ux2GlWU4eIV15YgnyQHt6j9o8ub/aWKWAMRB0K2jrlWYDc/A3jB+qY1OpGXLb7c+h/bnEBmjPkdDvN0aKfkT/UXqiM5uXWHffr6N4fnzbcaO2Tzl4GWcFua2Ful0CJCrP0rKnkr3z0YEMXceL1nhLXlLsvwJyUNckIT77M6jIBz/KJ4pulhrZwxxT1sRl3pG4BFP7U2cqnJ 7qxy5BDr KqpmgxPpCkOs7p6fedWEaSJaWIjAkBEj2bpxRLbH6rI6VbK+03surzK30ppJ/ytk9p4bezRTYnTh53YMkHwe8sVnEVzYWua2TKXXsTM6uYJELLiCPuuzoU7dn0y0fRygBGhfOTS7smHtvxonKqz3IoTDl0fVKicOnbLAGxuj8Id+XuMeCo8UUZ20pUYgaICeGvTE7lTgO+fQPR49XoEJyVdk4pN5+7SsNqj60SHV4eN/Snx9xadWjVJK1okyN8JrgQrmXkWQ2pPVIEqzBv6DRRtoZ53Axw/O5+9Va8AFnIpA8urE4CyWxTXVj6nA7nRNvZn4dqWrIadOHvOsRiCgNkTlSbzJ8ILOvHPj5NKOex9F31Mjjmptf37cGxYMtkIhgho0Jg0jF5hf0+O/6ttHqlQDmE1FxbOy0oJjN3HDUE0mAjSOuEboeN2adUjxKX23m+ekW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_maywrite_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng --- mm/page_vma_mapped.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..da806f3a5047d 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,9 +13,11 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; + pmd_t pmdval; if (pvmw->flags & PVMW_SYNC) { /* Use the stricter lookup */ @@ -25,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,10 +35,11 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_maywrite_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, &pmdval, ptlp); if (!pvmw->pte) return false; + *pmdvalp = pmdval; ptent = ptep_get(pvmw->pte); @@ -69,6 +73,12 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } pvmw->ptl = *ptlp; spin_lock(pvmw->ptl); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pvmw->pmd)))) { + spin_unlock(pvmw->ptl); + goto again; + } + return true; } @@ -278,7 +288,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -307,6 +317,12 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!pvmw->ptl) { pvmw->ptl = ptl; spin_lock(pvmw->ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, pvmw->ptl); + pvmw->ptl = NULL; + pvmw->pte = NULL; + goto restart; + } } goto this_pte; } while (pvmw->address < end); From patchwork Wed Aug 21 08:18:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1D0BC52D7C for ; Wed, 21 Aug 2024 08:20:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55B596B00D3; Wed, 21 Aug 2024 04:20:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50ED06B00D4; Wed, 21 Aug 2024 04:20:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 369AB6B00D5; Wed, 21 Aug 2024 04:20:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 14A246B00D3 for ; Wed, 21 Aug 2024 04:20:45 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BCBCE1C229F for ; Wed, 21 Aug 2024 08:20:44 +0000 (UTC) X-FDA: 82475556408.11.1E069FF Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf07.hostedemail.com (Postfix) with ESMTP id DFA0540004 for ; Wed, 21 Aug 2024 08:20:42 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ghE5JOPV; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228354; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jOO3xScatAPPPb8tw2FfV0S/NtjYDESd+MYV0EHgGGs=; b=P1eBtkWe/G/itYLAShGYn1CakAw52joJ2XpmC18uTsb3O3soZaTlUI8shSXvsKtimS67je 7Aegxdk5F/L9gr1//2eMmlRGPv0EaZ4/5n0tdjStXo1pYafs3iD2+1bAdsrUP0uafgAxeo TJtoHtVvr2NPGsmxQErY1RsxG33jacU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228354; a=rsa-sha256; cv=none; b=J2KdnNn6CU5EwDPwsKSAu0kXcTGJA+KgUhv7vn2kDxrB64pfgLn/C1RvkhH4O/vklCUTG6 ihA6rfX11PDASqdpweMwXwHDTPLElpIDew5qSWYZUyKAWpV8Hm0KtT387wWGmGy1V9AiXs KnFccX7GWosHasBtHqexo1YEz5/amTY= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ghE5JOPV; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-713f3b4c9f7so2304158b3a.2 for ; Wed, 21 Aug 2024 01:20:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228442; x=1724833242; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jOO3xScatAPPPb8tw2FfV0S/NtjYDESd+MYV0EHgGGs=; b=ghE5JOPVfe51qYDOQ7yFdkh2hMuenpFStFrijxpc0LSGa+aGhTZkrBCSU5VJuOqjfP BgnTgofa9MXnbdbuTTFu3vUn6aInuQvgcgS5yxoFIhvoBakiVd5vAMNoJb0gZMZougUP gnM1JOSb4kyGDWsmDKuoYTKyVGdKWWe0FLbR1Mm6IvRhYUUZIewwPCFRoRq/QFzT9RQI 2UJGZyQC3NR7QUufODq6Z/VDARm8MuRoKcltziCRr7D4U4QpmfHnaFtYdVxRYuMaFOt4 bKALA+EauilScyWyiVVdGuMTKPMbBEPksNK5UGiLS+AeUyMKUgzwOZsvI5/zKbubLEU3 ZEQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228442; x=1724833242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jOO3xScatAPPPb8tw2FfV0S/NtjYDESd+MYV0EHgGGs=; b=FH2WD6Tr2whDzlHjKIlFyBrdh4ulC3SK8eiOyWvgEd7dqqyx3fvP5+5JmYSGNiptvi RnHecirssObC8wGgMWQncxl5Oxsyg8/qqFF90WqfPu6cCr5VWYP/+UhAJnriDPB84RnP 4h2ThSJPTD9K0jqXZgprYhSbSahhLEHwB2z8ruthDnrJnajSAW1k2mbmNsRCbf7GA2vK 0AU31l+p/Vm1svQhbpvcD6NgpcYCy0F/Gt89ff+lD+0EhCTsG4vYBXmNaiW+lXfZKXxc Q4AO02pCd0FcHA5R/9lI6NvSFrLYOsGzrlxS9r4jgDQrEztd/0fgB1CFWXP18L5w+PvJ 0U3w== X-Forwarded-Encrypted: i=1; AJvYcCUCtWsUndAsY/lWjaPrqC9ZeXw8r8wvjmnloeYHt2ErbIza5cctRRlYcUe+D8Uj0JrCbtlvgV9YXA==@kvack.org X-Gm-Message-State: AOJu0YzDOLUDz7zct+0V60u6o5p0ULtRU1XHavTxflOo27IjA/eSPXzo M1AFCCweKyzSebBV3rZOtZ6jmAdCY2hhTgDsArFjcom+fDWwEh8PmerRqzNGUss= X-Google-Smtp-Source: AGHT+IFM+x1tDjPFZswDxs94u/u9nyN6BfWsdj/MLtJd+fBRDvg5mEN2O6omshjTkNyUCHM1AQzwKQ== X-Received: by 2002:a05:6a20:e605:b0:1c3:ce0f:bfb7 with SMTP id adf61e73a8af0-1cada055937mr1798278637.21.1724228441599; Wed, 21 Aug 2024 01:20:41 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:41 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 11/14] mm: userfaultfd: move_pages_pte() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:54 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DFA0540004 X-Stat-Signature: wzn3t7q5p843837zhb54ockc1faorud4 X-Rspam-User: X-HE-Tag: 1724228442-391719 X-HE-Meta: U2FsdGVkX1+RtLuQCg+HYgFgidhSJOxbhV8wanNn1ag4GfMKaiAgDvHTPAuvl90MsIp190pb8G90MQlgT3zfE/FNnw3QSxxObd4JdjmIsF+FJZXrAKE6tZwONaxS6nKZ7p++XpwH+TYZcvmjEjqhZV0A55pTl2qfRoTlCIRFJwxIq4DC4MvUk83jGOxu9r0OtZON+oVsBOAWnmfeY8llzug0d3UfCa+0XtEinDaXP6rfqHxDZdHjYm8zcbX/oJFAYZwiI4nIGUiZMPmpOGGvexOMVuPhXJJAugOlNTRnYciRtdux3VcwU1uCjd6L8R46TRjkoHucrJSMYwjf2xeSYNWFtzwHa2UZILgGimGY6oOAGpz1vV9vWVS2g528XjLKH4yZWmDXbHMa+D4LHbm7Ew25F8JaSqokxGtCLn7zowt0CGlM+YyT8w525angEC7Mms4RqvTwsgE0aCin6YUwgpM3tbQMBa3b35UhODu22hntDMtyikVLLYSE/BYNzGDd2E848MRvwwE0rd/3DIKCyEmdF4da0u8KvUYgBGScN9rsHcl+T1WNh4GUIjG6vIg0gB6xsCrSrEtXTqaTUhjQVRyYUBy8jI8tD6hYrN9za1SrZNnaS5iTkeNVkk7PLHq3neJbbT5ChedXcpfnyt447cuerYnMKMB5Rc1ELJI9hFg6EtjXf3fI54Hh6LEJGRutfZwTcH5l5x36FYxu7mFVeT/k9qhhRScrHwrJkIxboKGstpjkRzpPxeZGRdmlYSTthrcFhxsixidsI+z/Wcaq72NuJ25LJvgqXdEM8zg4X1uFhEkM6he1p/ZjqtZNZpn8KA8hhcRG7jQRK5o0td8t6Y1ap1mpjSVHhVeJ/H/JRE/XHdauPLlMr14+mT1UMZ2pq8OBwMlSouFA4cFx1IElS0GsrxqcSb8a4V+4LkdS1ZDxt3lWfUawqT+MhFl20rtCxwnQ/7gtCCKmUWOPvJ/ bJ112/4l UUl2bPfBjwO5fbkzIAm+85vt3CKY1olxVdm879Z5dG6HvlFgpmDqDy6ThaC84ExTyq1Z8RWpXSuM/n2UIF+L8HpeCrkTfG9xtUWwc+YxfmpUaMYfnV/vYK49t/Hk06gsyZ+n96cf3CVWr9OOGfIfgIdlA1fOKGIpNGI7Esh0G9stxUxNZozpUB7vfy8ROY5/ylygS1bH2SAmWPwFC1ygrWms59ma/dL7luoGOTY1+7berEKHcOBuG5d2bZOvtkZxYMloe5dhOTBsjnR857DX9hy3fmh1QNYLsl0ulAFPZicUw+MqBnJj3JbeJLlJjvjMPc63wL8bbI5xJGHDiqpAsyRLiiPbqsgyDpB8OFPh57zn0YtAugx9+Ai/GLWMWpuV+CJkeZ4GimRRljXYRGq6j9/JiA829KIreBIz62nY1D0+k02OHRG0VLkasuQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000015, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_maywrite_nolock(). But since we already do the pte_same() check, there is no need to get pmdval to do pmd_same() check, just pass NULL to pmdvalp parameter. Signed-off-by: Qi Zheng --- mm/userfaultfd.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 6ef42d9cd482e..310289fad2b32 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1146,7 +1146,13 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval. + */ + dst_pte = pte_offset_map_maywrite_nolock(mm, dst_pmd, dst_addr, NULL, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1160,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_maywrite_nolock(mm, src_pmd, src_addr, NULL, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Wed Aug 21 08:18:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EA88C52D6F for ; Wed, 21 Aug 2024 08:20:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 905956B00D5; Wed, 21 Aug 2024 04:20:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B57D6B00D6; Wed, 21 Aug 2024 04:20:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 756ED6B00D7; Wed, 21 Aug 2024 04:20:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5455D6B00D5 for ; Wed, 21 Aug 2024 04:20:51 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1581E120BAB for ; Wed, 21 Aug 2024 08:20:51 +0000 (UTC) X-FDA: 82475556702.27.19C33DF Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf05.hostedemail.com (Postfix) with ESMTP id 3FBDC10000A for ; Wed, 21 Aug 2024 08:20:49 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JFsIGKtW; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y8nsSUZs0Rgdiv42AW7Io4vKbgKEW01jAKVQTF6mEcw=; b=eg7hFC1jsV36V5XMgjG7AqgF3drRu8bhd0ptOuQyJGzUBCp/45DjZUxlpmsqgcu3JDYk+Y meZLLsdZqx6MJb8feic4+MSie/4sZ67P4EZ21b8cUO9BsqZibgPTXArHhoIDW0MvNkAh/g xNa/1zuaqOfNbU7X25DJMe0cQtcMtXc= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JFsIGKtW; spf=pass (imf05.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228409; a=rsa-sha256; cv=none; b=UvU2KUCV/FUm5fCSrR2w9oQe6M9xxSKB5uGqgyAJ9+D818+FgbbAyQVAFByCcIY6eZIYBR HqcWdej0YiLIwcHF1vMhMTdiMtcEYvO5qSNsOwqguvN0sb86Ej3hFK8SHOOMulxOFqcVUL vaR5quT5OQyqG61U/zFzY+BPPnLOqmA= Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7cd8afc9ff3so221502a12.0 for ; Wed, 21 Aug 2024 01:20:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228448; x=1724833248; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y8nsSUZs0Rgdiv42AW7Io4vKbgKEW01jAKVQTF6mEcw=; b=JFsIGKtWXEdIxquj/1dpz+hdBmTV+9juyi3sCTJe7zdfoDQ5aHTobIfKf1S1zbjbti B3ccDIDN3BeiSAGZGTFq2nDbfkoJUzSuuaXTvJexq5VsquqtdQkAaQt1ktTU4GJcv8LA 6/3X7Z41wjgjXnCnUNUxW06ot1PYxnLP/FV+VC6UmM4VVFT1V5f/F6glA4TejJ/Ob5ks JANWYZGaTc6yVNp5Ljgjs4mK3phQ2Zs/gWpEABaZkdtfwVRFtKs6AFdM0CnJLB3WSysU qLWazWn6TJFJVXL2MfEpMpWnpExfnoCLpIBrVAOUkyDg+1hI1YMQJ1tcAascvkFQ0cIv +G5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228448; x=1724833248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y8nsSUZs0Rgdiv42AW7Io4vKbgKEW01jAKVQTF6mEcw=; b=CRu6LnNftHC8xlsngNk+fb9+QET51R4ljO6RjN/r8boj8XjeJhDRR/QGM+wBGjisJ+ 2dfj5M80HK2yHYML9IDfXfmAlnvDYOyWibWrIpXKnoMJVdyFcS0DZ5FNlb6QDIO0id4C xAowC7YbGoq4ehMiw0a/ADRqvtrPn92iUqfCXtyNn3+zfA9KWzRnBzAJCVeDpVHpMCyk pp2b4bCeqKikO53jfG6LNV5/Izzt7uDTLalS/pW/SNxE/VTvppjGmNVYhm3IiTWtV+JS YiHFTpt257tXcfoo84olU6d/uCVt14HIvNrK4TnQL/B+YJHkZiyiAOrqguE9/AlFnGIT Bsnw== X-Forwarded-Encrypted: i=1; AJvYcCU9Z0b2Ge+ta26MaNeZ5pH/iT6O3O554TRprbqAPgybRKfU92ieMmY5kUWZUB+ATdlsut/UY617Dg==@kvack.org X-Gm-Message-State: AOJu0Yz8SJIOSaQVhSorJHuzqGlME/ioSD80F/vm4mehfaiLvpXAJq4a lpkoEPsw8tWLBk4jAvrYE3tjCztsNsjtoac0pr1J1vbtDntz8WnDavCvbuQ2rHw= X-Google-Smtp-Source: AGHT+IFBw9G+gRGiVw886HN5nxLIybNKSDx7re7KNY7nJCh1bBtNFP6GTlUB4MiVeFJVy0IWyCdYMw== X-Received: by 2002:a17:90a:d987:b0:2cf:fd50:a2d8 with SMTP id 98e67ed59e1d1-2d5e997c3demr2593063a91.7.1724228447873; Wed, 21 Aug 2024 01:20:47 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:47 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 12/14] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:55 +0800 Message-Id: <1bdb1ec4a30acf6b3a376f746c96236328904768.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 3FBDC10000A X-Stat-Signature: 5aakwiktd9zib3nm8rhp98rx4wqr11ny X-HE-Tag: 1724228449-207995 X-HE-Meta: U2FsdGVkX19WwrmPx9oXxaER0kRx/L3xI6fd5vk+DoHgVIFt3dwN9z/fmEw7nNLt43m0nw/Cso7ZA8USRz7XwYLROFDrlis3JeiQLLPkd4uRMNVe3nHUUvqqEiVKipWiJwKMIqm2wrlGKqjAdCriDnSlNwt6zNUXAf9kFyKZucbOoNPD+wna9hlHSVQ08ek0ei6PrWH1nO3JoKGyeKrIOdWE6lXdV72Gcrfc+eh3qYidyp4Ef5f5Del+yhbsMe7YkxSYTFkD+iXyP/dyqQNLUNdOkDtUS8QnY3NpPsNHdOKmZpAcES7e7wO0wuG9D5YAvnALrz5h72t+ZTQDCv7xuUDFwAawJmAMJQLkbHmwW0lBTMHopXASyC9JEdG4xgXkg7LiWGjiTyin1scxGHtK+uRAFbxAXyyqQZf3n3PxNPJBKKBh9cg1TzYYwvu0UO1dPpYXRK52oQCf03cHq9wFrN9Ou7zuXc0ZVpKTE9iURzEoQJJNlWRUvwQAgvm4i3GhhDqeZvA1AlBdLRaRw3Y/cug4HPEKjgPiFLogXnJq98hqlvUbTNsIwhF9+1FoKcdgjw+V5JBc8qmpLUU5ZevO6Rlpyo3fjQ/Xal946DfhIxF9cgFPyuf/qyLWZCYjEcwxaHlEDxHme6f0hC9YMV1vjD741JEPbk6zro1eEzfdyP+rNQU+UhZKWK70EVc6GurONYM0evL1fmhGXkOSdu7saGJpKB8JFthYpJF+aYBc96Z/DFhAuXrhlqH2TMvxX4J5hFXtInj4fuHxyafUeG6FEYp+5p32tlPgneTDVErKSxJZmeLZVy9bkcZm4Y5VTqKSyvwYEeZHOrzeQ+0uoGZC/gY15jm9s6bWh1aFZg4NHcMVoMWM8eG8pFvHQ1orjXLRhhORcaalf0EkHrAcPIPgqsXGmMPs0PrpVOCVKFBQxlji8MvRXiF4eHUGyB128U85Gi3ZVCZeSfZRW0AT2xI VUgXCdYj SCT6h1KmBf5Vlxfr9NiwJVhC3114wD8TFvfJFTtutwFZaumG270LDl8JVsOR12afqxDqPBvAezjFnvOMLUAjgFvBBDKtkpzNizuShwVVfdoyQZ3WevPmVzH/8FTVEL2ewIVyUarGo53I70CgvuECCzx70F14JpEa58h7lM87gM7UOgDY/KoqgUAWmekuoEYMJxlzIR1zndS2aEyfnoaAbL3uS7XYMg2QlLIqdG+WYYpBEiJQedT7xR7FUUcqtwwouSJGUyvLVMsEcu7iaXNTguV0Wi1D//LpS3klMopoxsMyElknUPi67k9evBiqlBzM/nB5wGq7VZMq1vP5zwg8v4adr2vkKFk5vpqChE3baqRYvBsyAfAvlEaguf0+rZtFciAqS4F5D4PBfOKhjVKCCtEVium9fdDDWk6AIpjioU1QurKu4ldrnLH0BNQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_maywrite_nolock(). At this time, the write lock of mmap_lock is not held, and the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 5dc96a843466a..a6620211f138a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3396,8 +3396,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_maywrite_nolock(args->mm, pmd, start & PMD_MASK, + &pmdval, &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3405,6 +3407,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Wed Aug 21 08:18:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E47F1C52D7C for ; Wed, 21 Aug 2024 08:20:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 732F06B00D6; Wed, 21 Aug 2024 04:20:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BADB6B00D8; Wed, 21 Aug 2024 04:20:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 534346B00D9; Wed, 21 Aug 2024 04:20:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 325966B00D6 for ; Wed, 21 Aug 2024 04:20:58 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EF329A0AA9 for ; Wed, 21 Aug 2024 08:20:57 +0000 (UTC) X-FDA: 82475556954.01.F8A5D54 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf09.hostedemail.com (Postfix) with ESMTP id 18ECD140017 for ; Wed, 21 Aug 2024 08:20:55 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ceBmxYvo; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228376; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FpkIwk88smkdA0CmKwnRaaffLUYg3/8r+GLLHrp0y28=; b=g0ZrOIZcQQWtoaiGNVcsZTLvS3AGo3MV/u6j6FIuOgmcLzSgIb17+/6grYuMg4sWgHpnNO FNAKeN+Vn+uc7R2NK7M7kF68F8c4DxRjj1F0V3VwKdgK5fJHf9rqP3Uhmp/RpsG/pr6IJK ltjGf+Um5S8+N5X7zqJua5sepojrhqQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228376; a=rsa-sha256; cv=none; b=59kFz05rER1FaTxIG+9NoasKVBcwIdLXtKrxcvgypG0nvy7hgQVRUw1/q9+lzXLuJVdWH7 uMzGAUC80LX6YgoTIB7xbiW4xL9oDX6H/BN2OZ+ZkgxtOThGmSuclRhSqNRXQTmNRQZabY ORccGH1jV9T1Yl3aFMeK3rtWeFPHqdg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=ceBmxYvo; spf=pass (imf09.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-710e39961f4so4383778b3a.3 for ; Wed, 21 Aug 2024 01:20:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228455; x=1724833255; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FpkIwk88smkdA0CmKwnRaaffLUYg3/8r+GLLHrp0y28=; b=ceBmxYvoJhvLze9MbOpvPPGRMm60AQnhSLve1bnQ9C5NPQcFCt5bcBOnEGd+CnG96/ ngWxvj3GFkr+gC43f+6wXwxS3UpzpcNIue1Y0feND5XZ8BLbb1zr6+LG6Tp55eoKArCK auntXcY2ubiBT57+95SsTzB2KMRjlANoPNGIzWrtx7MJiA5fQyRQ1XEd7ivtNVxlirXC B7EijCpQKyuHjMRxSNgFVsCdnkXw9UZq5o2sjDdXrxIyXLuVvKoDPgFe0RGevYepXzIL lidSPLocfnq9yhRjrMARIXt/BX68tv5tdte6btXQqUy6H/8AabvQK6QBPcYsntx20SPB JP+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228455; x=1724833255; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FpkIwk88smkdA0CmKwnRaaffLUYg3/8r+GLLHrp0y28=; b=Y9ahOb3Dmw0XAZPLerJ3/0NvzoWywZLz657Zh02m+NYSushIQXAn6ofzj/zg0Napgh 027pT2jJ1kgvMYVf0gW7Y9WrXD0WyWI15Kp4eXt0v0Ble1D19jauHw4y36DR/Yz4gObs ZBFaHjtjM7CYWJjT6V2erJ2dZvXrE1C6c6a/DozYMYWLzJe6JHTP8+wKYKfhL7eIM7wz tIi/y8fIHnCbQAGtdB1GrCF7L7xP6ybqSVMl2TMvnoeJJpDc73ENs9Y83Sg6am2UJDFi ByuFX4yJcEn0NXxDsWQEQpXhQnbNKtGhd9u9GT7eTnpsLmwdX6fTq2Zi9hEwiNu9Qj6T lfxg== X-Forwarded-Encrypted: i=1; AJvYcCV+YDdREnJDMIsvCI1exQTh+/EDLrN1PjEqCKA3kLIf7y+Mp7zpZjmRnOkYQyIFx0HlN72+N8KmEA==@kvack.org X-Gm-Message-State: AOJu0YzaHe5fhoyLh5A2rT2pOEFHoKuneXK4+XbWzmDfn1H5lFefL9jt q/7bFTukwgfAOhrr1XtoOghpjSspQIwzngTfoZGyuZVwZEWWLChd9vTe6kTgfXg= X-Google-Smtp-Source: AGHT+IGrVkekNFESU70hIpUuBMPasbPeBaSvK0LFz7NLCJlpTpTqHUrWHKc0kKTsT//aSMihe2hbaA== X-Received: by 2002:a05:6a20:ce47:b0:1c4:a49b:403 with SMTP id adf61e73a8af0-1cad81a4248mr2186429637.46.1724228454750; Wed, 21 Aug 2024 01:20:54 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:20:54 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 13/14] mm: pgtable: remove pte_offset_map_nolock() Date: Wed, 21 Aug 2024 16:18:56 +0800 Message-Id: <129f11e3434c5af898096c1cf3fe378f50bc94b3.1724226076.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: yk94d78rf4ea7n9fbbaypzpqwsc47tg6 X-Rspamd-Queue-Id: 18ECD140017 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724228455-859933 X-HE-Meta: U2FsdGVkX19cSXBcp46tN0sQ4EpQ6BMP6LOqzYcMVHS6b+56RQYRaew/q/eaGKFAopzrGet944PmO0EO5dxvay/rSrfl12sPfnHq7XD5YoQg+VZkf1SGqLjFN8COlXtVFGupRUkotl/HCfhyYhdEO5L4JnYfK5OxJDavQocNvikOugT4o63pp/5MR4gNJuFxkVgek4redkesuaT6PitlLVluwqP80LM3D3xGL8d4pCoB0OR0SC9HGs88PCdY3rzRfDWbBBFGwP3ETZK7CbwhTLV6JZl64scyOUd7g4Q/HAmSEnY6oxx51nEHgCbz/WcPoFctDlG/bKBVvmBJOJaq8W+Jy5IQnpgfxZGgYnFh2fjYmDRyXmWsJ8d/hO5aO8ActLRzwQcoxXxxrXvMWRPVGXxvMj2OCUXlbVXBSjsaW2nL5MIEDQEbx5vPhl8QG2b23WdwKXPIQfbIlFNJLG+vBJIUPowPq5F6KM+flvPAXjKqrYeDsBJLEdmnrXkQXPB6jwCh249PhMjkAODb+6Qm7RvbXHw8+699dTHP5L9jX+BlnWCluZOVRvhRVysS7sGeLOlqPa7laxWOuUm6PWCW0a14OGvps9dBI5fPMPuEEW/Nb8jjd1Zk4Uz8ulLkFipsCgoMasA48CSySwN/XhKhuoZI35VQXnvHINj3ZvT408N6wiLs4t64/CSPNQa9ncjFchJS/7+eZZbPLSI7Xq9qjSImY8nvXp2CU41LHrWrCFsd+YVSsJFXE9nMt6Wk8KVJ8RsPa2+Uv2IVSzg386uP+t/3SVwug5hCjF1ZlvXU4rFTmJSNarOPX9BmqmfEJ8DjDkR8QDmMQxbWTUiHZ3DN4TZRSqj+KsHw23VGs1iGeVg4aKfBouIjsUkOjp8O5yCx0mP0UjfAXpXTZG7hWjff3snA5pLDkLIg5ZH1rjb9GYppSItsDITkzUElq5e1vqK50uOs5jweonu2Kie2eka MW3vQd+F xcSm6YCwF95DR1uuJGkFDstgTJHu2goxJMg6eMDAyXJXvxDA1TTOu1+lKfwcY81VafdySrhud6FIAl2A9sJ4+u7OigEnVHDn736Z36j76//7td7mAL0PioC+cOqN/aLntX7x1p3LJPfVBeKW8bu2uxk9xhZhIz7wRY3CFos+Q8UY9Qp4Q+T2QswWcofgYui3UiJ5tBG2Rkl10BudQmk36lYZI+LcQ+0frM39y7fMwnT1SdeGPBsIdxRY979qaw2PmkEgd6t1TGocca/+I4FzmsQZme2q5q4G1bNWFNE6mYogErFjk4gRDPSOYM8+tXX9aZK2CrWBgrdlVDxNSDzZWgeWYUE5lZGMgDI5MzIFOt6z0VYXtxGs6Lc0TV0eu8zco4DP1xsvobI5oyJEVqPX1LQMDWJSc+IkfgI1M04QjcLip53fKTd7ogg/CCERlUHoY8SY1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index f54f717ae8bdf..596b425fb28e6 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_readonly_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1fe0ceabcaf39..f7c207c3ab701 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2952,8 +2952,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_readonly_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_maywrite_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 29d1fd6fd2963..9ba2e423bdb41 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_readonly_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -374,15 +362,6 @@ pte_t *pte_offset_map_maywrite_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_readonly_nolock(mm, pmd, addr, ptlp), above, is like * pte_offset_map(); but when successful, it also outputs a pointer to the * spinlock in ptlp - as pte_offset_map_lock() does, but in this case without From patchwork Wed Aug 21 08:18:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13770997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 162E3C52D6F for ; Wed, 21 Aug 2024 08:21:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A23A6B00D8; Wed, 21 Aug 2024 04:21:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 951596B00DA; Wed, 21 Aug 2024 04:21:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F2D76B00DB; Wed, 21 Aug 2024 04:21:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6000F6B00D8 for ; Wed, 21 Aug 2024 04:21:04 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 28BF440A01 for ; Wed, 21 Aug 2024 08:21:04 +0000 (UTC) X-FDA: 82475557248.05.355D22B Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf20.hostedemail.com (Postfix) with ESMTP id 513C31C0022 for ; Wed, 21 Aug 2024 08:21:02 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=lGyk0mzY; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724228383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9nMMKQBovMbpHaGn8Lvg/undANxxFw84odzUdkp5IAc=; b=e2P622kJ9gRaesobclLaX9yYk2i7S91mWJd+HFTrgOh65DkOARXSAoMX3B41UdGQoSB1LG H7SNq+Npki3zuxRoad4sHc4dKsV8rP59hOzITJdZidut9wWsZee/CCLtAXIvLJKjinUrtK kDxJEN3lG6askcHqk9wA/dgskP74iHQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724228383; a=rsa-sha256; cv=none; b=OWQq/rUBFR/hBQnqXJvTvPM74FE7j+Uop4ZPX9QDtfFH/5S9IFt2uHrCS5jlw9uQNybK/1 xjUhT8I6kGID/M1uuXkJpz2CHER5A5/I7Sc2CW9s5fiYflpiWuJVT4dMDYvqaFnnU5ar40 4xXrb3Y+sQc4ecGqsOHJPeffeWYwK1E= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=lGyk0mzY; spf=pass (imf20.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2d3d662631aso4423362a91.1 for ; Wed, 21 Aug 2024 01:21:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724228461; x=1724833261; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9nMMKQBovMbpHaGn8Lvg/undANxxFw84odzUdkp5IAc=; b=lGyk0mzYHQPRG3NCU+9CS4JOtwk1TZVW7RkIMZhSfZ2N8sP+SdXn82jUKKZvzyPyKi P0Oiu3jp+hCcpbBz2ILwYk2Tex7XENkq8X+yI/Zsb+2sEjPq6ExBofVyxBsYJdIDB2BS IzgLcC1DjiAb3ckh3q7v7tFyur9iByATwB0udYuSS4FMvDKBHTiw19twoSIcIietExBM r8XyslPSQLkEyVCSNNd9bClzfhb92r3o6skPLwSSUP9YS6k7gAic4Gm8dwxqm5VHfSCN KKmcwYfpNiarEomDWBsNpRmtj4WAUVhc/CbeTszOShzggbG84Vv2SzRqYGaAzweFO6Zm OI0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724228461; x=1724833261; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9nMMKQBovMbpHaGn8Lvg/undANxxFw84odzUdkp5IAc=; b=k+KdizD8x7WVh3cvtla3XDqfqj4Jpy/Lg6M0t/iERAUWRLH1K9zX4YhZAj4T574rGq OmTxwOhnIxHpt6WCBLmT8xh6xFTH5bSf1MIyS8QHieGBc105HHcgCigYfs+vsXwyFL8p CRGv6jOy9VbVTN4IokFeL/NA2if3xxkah9oTc6n16ouQX1QUMhBCAQ4D9hXDSrjyR+gl hp3dvUzI+hMZ9s+AqnootDCUng10H6yJd65M56jVAMLdlomecynaET9pb75QcDuiAnj/ sZhmkFx9+r6t0sQhBEn01Fv7IXFvKpKJ9IPPt5fLqvYxWjBdTnnrq1eJKSEXXOsyi+WB TU5A== X-Forwarded-Encrypted: i=1; AJvYcCUFPgeXqVXUHTUQ5vmew4cQGnow9ZlTNTvQZp9IaV0rnxfZBrmgHCW3/yOOh/hKb70h+7OUbcLR1A==@kvack.org X-Gm-Message-State: AOJu0Yy4pbYVcgy9zAdjG48VnQIa2mTUF2J53Aj0G/Cp2m5JAO33BF60 0RTKTLLB0oOo9WuNL5v+fNOc3tUQzzNt5fbL46e9sSEc40fGAkZ7WxCW7XvBfu4= X-Google-Smtp-Source: AGHT+IGVCW7LuSOfGhGGSqonUfZfQEf1Y51LpMxCXHDPBrQX3ltYUdmnoycYhvy3ErzvacrcFVYd1w== X-Received: by 2002:a17:90b:3851:b0:2d3:d239:24c2 with SMTP id 98e67ed59e1d1-2d5e9b99f57mr1469884a91.19.1724228461120; Wed, 21 Aug 2024 01:21:01 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5eb9049b0sm1091453a91.17.2024.08.21.01.20.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 01:21:00 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH 14/14] mm: khugepaged: retract_page_tables() use pte_offset_map_maywrite_nolock() Date: Wed, 21 Aug 2024 16:18:57 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 513C31C0022 X-Stat-Signature: eitr4xm35hbsphxeso4pg7hsgy6su8rn X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724228462-861508 X-HE-Meta: U2FsdGVkX18LxsIQAsEjp3sHg3s6xwwc4LDCOQknXep2vLKMoCDiIRqKo1mAyuAvuoXzFKqKNXsf3G2qvBr8Gp9lohp7AMwe704zx/oQDg8MZnN3EZAMUCFVFZVIkFSbmWq027I2uvAjBQQoc9Ej6U34cCvwZYZvzsguwxg47/uTKjoquv6LjMXWSJdD2o6W7vIxRR3DJFLMACrgcklwksGFSRa79l/O73iACrqPd8IOLENm9YBfXGpj+kwGuYzPU8sxUqLB75qKhYzKY/vsuiyLjMeeJ7lN4DaEM5zlIiys8dS5mz8BUUrve3Cqv/v5eAbhWLxe63dF0XlYgsUXeBBT8AK7l/8kb7UJ5FNYn9l1uJO5+T+fuA4mFDFVVfmXkxOu/052LyqYIm6HtwV/N1A5xZOFlSH/AzdNDnNnGvB5P+4cqjQqXihfgKmOR0DMLu81MOwq4qFDyty6Cs8PfgYNtC1jwqF5EbcT9rrZeMGWuthHyn3dwkxB/bd93VoSXHHAYRH3IMd5YP/g1xRNZlJ/pqknCvNH5CmBXu6FlofMdaXBJPTEzMNxURuVj4opua5l2bZUpeCjX0hRkWuP0NvCzL4Ymk8ru6vznydqIcymz103fvNljiSClw1rG6sb4o57bLirrYWt4kkKHkRp8GdQsYg3D5qhFJvSqH2MYDeyMNpv8ujmwv9USBnZ8Nj0RyuLULwTSD7U7kg2KWt5jDq3Is0v9odFtm/NMetF6XoNEEqyMrYXNCRh+Rdzk7Oz8d3UyUEGtKntF88HG542J0AG7QcJj6gnMhHYteOwQFSqf/3vFDvuHMLxWRvI/k5dze7kuQ+rolcEppVR3/y5tFNUayZXc8preDps9FqDrIbUWIoN/9pNnxzr70I7AeB6o6xgEr2uyshjXr6ZLoulXP5NLpASEaTfVR2Tz40+jQOK5KBKf/M0jkSWySNs0Vs6gp8rpR673BYyzF368y2 2LMicEbO msYKQjDu2o8zct0ic7pZNtVGZ/tQCgBtBTAhy6qBXEzwA0sprI8V6P6etvBuJ8CnME5H2Utg51MDw6JzNDKDVSMcKosj5JllrlSMHWjtwlz88n49cWRFgPnx71UWWZm+mULjnV+oDbkJTXxbT/HRY7VLdCcVsyHl6T5pQrsKa7oSHElZebU6ua0+krG7Uhb3IFmNKAbNP2PaO7G5DZPrJLOobdPDbA+KvTQF9XsD6pLEPuPbZtgNINMGNQKhTr7YNRf38n9bABdvrDDCS1Dpef4kYKMeKpu0+9qMCW1hhYmkAZGpLFCesznp8+9mk+rs3WoOGAIBxuos68zLG+ecRmIPlEAt7/LwdJ6dNO+D9ZUNM6+sDTqD41BaVQxEzPBVPbjaULxggC//FFwMtSdb86yUwbk/gG4jSMvAr4v8kf+V7PRf57JfuJkYmiPzoOP+WHJ0C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In retract_page_tables(), we may modify the pmd entry after acquiring the pml and ptl, so we should also check whether the pmd entry is stable. Using pte_offset_map_maywrite_nolock() + pmd_same() to do it. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 8fcad0b368a08..821c840b5b593 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1721,6 +1721,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) spinlock_t *pml; spinlock_t *ptl; bool skipped_uffd = false; + pte_t *pte; /* * Check vma->anon_vma to exclude MAP_PRIVATE mappings that @@ -1756,11 +1757,25 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) addr, addr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); + pte = pte_offset_map_maywrite_nolock(mm, pmd, addr, &pgt_pmd, &ptl); + if (!pte) { + mmu_notifier_invalidate_range_end(&range); + continue; + } + pml = pmd_lock(mm, pmd); - ptl = pte_lockptr(mm, pmd); if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + if (ptl != pml) + spin_unlock(pml); + mmu_notifier_invalidate_range_end(&range); + continue; + } + pte_unmap(pte); + /* * Huge page lock is still held, so normally the page table * must remain empty; and we have already skipped anon_vma