From patchwork Wed Sep 4 08:40:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13790097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84075CA0ED3 for ; Wed, 4 Sep 2024 08:41:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1946F8D023C; Wed, 4 Sep 2024 04:41:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 11D128D0239; Wed, 4 Sep 2024 04:41:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED8B08D023C; Wed, 4 Sep 2024 04:41:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CA6CE8D0239 for ; Wed, 4 Sep 2024 04:41:45 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 86A3D80EAD for ; Wed, 4 Sep 2024 08:41:45 +0000 (UTC) X-FDA: 82526412570.13.8E1F17E Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf07.hostedemail.com (Postfix) with ESMTP id B83FB40011 for ; Wed, 4 Sep 2024 08:41:43 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H25gz+Ps; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725439176; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=6B8znnXl1a6pGCe77hOJy7wDF86eazWgtv16VUMJpF+LR3Wt9fg3dwyjCcEu9teqE2YI0f fouMnkbrM8hFUs7zY8wdQxkoIh6k+g8ZZfNhV19uQCeB52RKwoXWlhIl7G7Z0kpuo7nqRt t8+k1ahXBmF7FyxOWWWi6FABoDFxJ7E= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H25gz+Ps; spf=pass (imf07.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725439176; a=rsa-sha256; cv=none; b=v2I/ABaY0ib3yTEj48sYnZ6DAmGjedlN+f9VshZ24U5AaIq73V1RwsFb2YJUoTyt1Fvo+F cjB0Sqto5b5eJri/pInmAqVnb3CQUHsL4TgSZJm3g7OgvsVSgmLhsPtLR+QIJCYpAu3+uC NEOdAjS2+S4V6/cV1PtB1a5n1LStOuE= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2059112f0a7so23538685ad.3 for ; Wed, 04 Sep 2024 01:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1725439302; x=1726044102; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=H25gz+Ps0S6SUSJ9vcfZMbqrPPVbwe2PXrorY61rfbRwMkzYD3OiMGI2jE7XigRLK1 W2q+cm6u/wItxa9Uf3jMWGkrOCtHmPGbKpA4TCMw/O9RkZAQsji4hqKFZxZPBMdd6x2H 3DtmWvW/W1echXtMXA70+AGyUj9Yz/xSrRtDAlCRqCs2p6TELxGj1/K9x0duZ1jNQVE1 /+PSIRcSMX55gvNipvwcLRoMIul2+kSmzu3fIv14r8U3+B6bn9JdUpL0+p19YIstuP8C sj/cCKMf9zYpIZOSz8ht/x0F597XtxPVO6zjLfbWJYIWO5PrsHbhdbTU6At1JS3Zu7RW Af5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725439302; x=1726044102; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vHfOyy55HFvG/XxGZW1ra6nqYg449VLN9cHJI9/IEBs=; b=nPutc4PFD5T4ZT5G0hTEOCUCx7JVwnJ8Ioy92QuDhpRsM32yo3slvpHGqWNGktj28w BM2PDNYxK3Ddc+4Hu5bV5+gaxyje5cTSXqMSJjvsAF09Ab/mHpzXiy3Cfc/RYCiUlasj lVv+fOVo1eaWfehmhkBKqYYwj7fEbCTaFoPck0UI6NOqPJKsUqetCTelFAIjqXsWTP7V VWqUYXlwRIsd0XivW/hzXH03yGYJTtycTDVjrbE/0q4Bgq0a5i9C8u9f1pTiNA8Um7vR fFTLT2I62hJmCEMdwzIRoVY+lFZkkEcdorhcQs4qY25VPb29pn41GEnOYczY77A4ieHd wREg== X-Forwarded-Encrypted: i=1; AJvYcCVvHoHZV+WrM+X0cYv8k1w9NvfoVOdTyZ3vF0smPfxZsq3jqSApFKYozGzUlWoSdVbEA/ny56G10Q==@kvack.org X-Gm-Message-State: AOJu0YxW/k7oLqnoPZu7jTpzlzBjuzCB25g4PNsg8Iy/cZ3xTz+wMZEg kt29ef2fsEG664Ul2jnwAXOmGrTHxxyZ459mKW+FylXw092wuJijsDCDVB6RIh4= X-Google-Smtp-Source: AGHT+IGlDE+bZkJMe3LtnR3dMefm4RFnOsn/qpiRzdld3O/wCPCkO6r/LjjSl6iGWtDIKmNPthI/eg== X-Received: by 2002:a17:902:b68b:b0:202:4a24:ee with SMTP id d9443c01a7336-2050c45e710mr147539585ad.55.1725439302519; Wed, 04 Sep 2024 01:41:42 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.242]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-206ae95a51csm9414045ad.117.2024.09.04.01.41.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 01:41:42 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v3 09/14] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Wed, 4 Sep 2024 16:40:17 +0800 Message-Id: <20240904084022.32728-10-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20240904084022.32728-1-zhengqi.arch@bytedance.com> References: <20240904084022.32728-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B83FB40011 X-Stat-Signature: zqu3x5h9afy8pgyhydshop8b9tqn1rxh X-Rspam-User: X-HE-Tag: 1725439303-537800 X-HE-Meta: U2FsdGVkX19nlvR2HfwUGGtViMFQ2CiKqc13xjBILUAY8x2UU/BqH/XPGOroGcRmH6G2kJUOWvC/821zhc+Aij6KL8JxDKVG34VFcePP55HQ961YWNvbGGotPlkgA+uL44Nz+K2mScYhvKIkP4oZEhnnuAYnrpXaSfDQIxS3Wi9xZKVDyX5WUz52nO4JnYBwhW47W3zpEqw9rFzCQFOGagTKUIMM98yDwHOUReqzTyja9TE4Eq15Vjhdav0fhdHjf4nybDIXDxhQmO9D6WUv/G/QJZWGasJ1z0CkJmpIJFIZb3pWmH/UnKKzgcoW3iDdiqfZViQbGk5i0wSyPjEOQs31+CqA1nYz0mk1VViLpA8Ll+eK6nsK0ww2aXypcrFEiCcE19RRR0JzRY1rr48h7KpkvNmh31Kpy2jSy71zQfoMRqSM2YM+XRyqWrbFae08qbBN2q/WSEKAuFi/fU6NAcMTFQzELCrstaIWRK33zQjH4/WPjOQqYdF3aVtOhrbWlwDWHVY6d5Dak7XxKUzL7cQpk7q9akYvIJC4j6o0xP+Ds0GBjNvC5iZhkHDv4yYw35fP1HrCO+yOd2NPiYungEMz/b+Ufw5Kh2wP4kjry+7qZ8zWruqszqZ7BeiG50MPI63+6FzoRHkk2OcGHlL5xJBRFT2zr7Fgfoxvu641TYxxTy8APsRGvTB4XSPRqspgxSCjENaeI+w1ntDhouH8x8RAiC8iArUJ9K1ZnZQVxSLmf7M4aC44nc4SMcVz44Uixb5apOdgbAR67RzDQJA7XJUrPULJhfV2nwpCWEZ6COqqgti0QzFf+O964nGIMhY/vwtRIhqFh+Jj8+N2eClWewB9nOoPFGlwTSA78AzkuAlnDm6TxtwIW7qizNLDqQG8caOKZaXG0KYTho7RVdM7HI9Yj3ynAom1d06NRxEQUGpjwirq0eKGhM9WsPp9cboVD4Vq1YG4PlXVDH9iJQB U6+0Wh92 wzZUgen1QZy+1KJ0JaDMxa4SxBuuz0Y1RfpysC+yiAP03zP4C3EMr3Dfm2uIOMt0c9O046SRlQ0trcd9sPNXkGRfuTXBGavY7N3ilejrPIk/C9sq4nwPZUaSiNm04671uR4QlM90dGFH007pl206Snssyy9uMd4d+/oGJuogekJdLrsturxVTPJQCXLere7TwXwylz2zaxzME396gqn1l2h0Y5kU9x4KiIAwpv3zLmXQ1KS6GA9ReyoOUtSZsBiF1n2tHrxx4vECNznVZkPAg/BApJknS35NhTD/wQgpd+d+zC+QkgUEaokOiCwJcREo1c5V5Z3fGHW7FYhXPPqFcKWVGN4Alw16ZM5+MpaRX+nhZBcaoZ1aJvw8v0JIQGziQ2N59uRll1y/HLRdpXbRqXle8wFfDohobuj6jGWRhn+osb7U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we may free the PTE page in retract_page_tables() without holding the read lock of mmap_lock, so we still need to do a pmd_same() check after holding the PTL. Signed-off-by: Qi Zheng --- mm/mremap.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..16e54151395ad 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t pmdval; int err = 0; /* @@ -175,14 +176,29 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Since we may free the PTE page in retract_page_tables() without + * holding the read lock of mmap_lock, so we still need to do a + * pmd_same() check after holding the PTL. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; goto out; } - if (new_ptl != old_ptl) + if (new_ptl != old_ptl) { spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(new_pmd)))) { + pte_unmap_unlock(new_pte, new_ptl); + pte_unmap_unlock(old_pte, old_ptl); + err = -EAGAIN; + goto out; + } + } + flush_tlb_batched_pending(vma->vm_mm); arch_enter_lazy_mmu_mode();