From patchwork Mon Jan 6 03:17:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13926914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F37CFE77197 for ; Mon, 6 Jan 2025 03:17:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5EBB76B0082; Sun, 5 Jan 2025 22:17:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 59C2A6B0088; Sun, 5 Jan 2025 22:17:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43C9E6B0089; Sun, 5 Jan 2025 22:17:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 25D4B6B0082 for ; Sun, 5 Jan 2025 22:17:26 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8A69A161363 for ; Mon, 6 Jan 2025 03:17:25 +0000 (UTC) X-FDA: 82975566450.14.CF15294 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf06.hostedemail.com (Postfix) with ESMTP id B0A8B18000E for ; Mon, 6 Jan 2025 03:17:23 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bRhXrcKm; spf=pass (imf06.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736133443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=HH3ED89tsTdnsw+rE9QWfbTCRTEYg4GmdA47cOVplWg=; b=Eyr4Hq5GblQ4DqDj5ot7Lk+rNeV89+75GgLE+lwbs2CNzbQXRrbAR1zDAOSCRmu1rkFPsn o11MR50/Dod2PWaspof88B3jF5gGaSxrHNe0bPrMKssS4cSFcOo7KngO7EDryEfMXDl/VO TTh3HwyfmrfVhuwFSpsrJUrRVhX3+t8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736133443; a=rsa-sha256; cv=none; b=SvOvAC5uj+lHDk2+K3lhFatQEcmznSh3THPDQDGyFwM1bWy5aY3TKrqR1rJQVmHN/AQczF /eI1JOt2y8KbR9HlZgHkywA0iOUyHpU4OGMMCq66PAbvT+vwjoO0+7Dt8jHzP6TPpxMpAw k/oUHKGxj/fAor81l8ojBMGAY4V/DYk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bRhXrcKm; spf=pass (imf06.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2ee50ffcf14so17676466a91.0 for ; Sun, 05 Jan 2025 19:17:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736133442; x=1736738242; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=HH3ED89tsTdnsw+rE9QWfbTCRTEYg4GmdA47cOVplWg=; b=bRhXrcKmU0NuRNXqfeE5WiZqBEDLRZZ8GK/lZs64ShPOl3m/KS9sC1kuus5r7dEACe yeHghpESNxP4XnpT0P0gl3RsSe1/a6HLhbACqDgaPS52LPBodKCG6K3C/KC7mEAcUydy EFLjPLvSwlX6mMFYStlCCIt/DWHmrYvOzUDGg0UZV64ZAOgMAWJYDRaFnV/RPmiyRQmU UYZKFr73GyDpt01NZEw8u7cY/M3KZHriYmt7QL5DiZfYvc933j5Atj8hRyKD3ix7g6Zd zJHLbKqFrKomYUXnQgEWCfIRzwYvflZfwzHze9kGe9ksYBgv1/ymiSHc3KsRtmYL9vdL 6yyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736133442; x=1736738242; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HH3ED89tsTdnsw+rE9QWfbTCRTEYg4GmdA47cOVplWg=; b=kb8DbQ1r5sjMrFttYlzEGd1qPAd/t6xLBhrqZ1ZiKjQUb4YD4VhJni1/WvYF1xKPEa FACvZyCXou181wcBsdf8erZoFn+xAatDEoPX/+IMABcyO0hWFbW9oqNUuxcipP0pMJHr b0hfYeKoTlz6XNLzflULqOUnnDz9CwibOdZX6yoYj9Hb61oQvOMcqmFmexYbMTVCLKlI lPLk+2YwzF5JkzTURJxoFaAhmaj1EUut32IJQSz8eGuJBcsKJmVVuQHLww1infkOP2nZ b0s482fF1qOTUe7vBLVKYlsZFJpE7EbyDbNys4w3Y5SEgVYNWPneNRiqPBii8hCD9DaA UGIg== X-Forwarded-Encrypted: i=1; AJvYcCXaiAaSDHTNnySArfCfJCFeqgnHT8dcxFr+VowDq5znRRDmw5Nw8dl/IJPD9MGLu2uylCJzWKWZRA==@kvack.org X-Gm-Message-State: AOJu0Yzw2O3bzRueloSDcDW3VoAwfCX1BMUCKwEnGbjh4D6NXKAZ23gm cgUwKTfKeqk6Yx4GHUD2WVPPveZxudF+eWA7vS/RFwE9WpRdduK0 X-Gm-Gg: ASbGncsNDCHvEhqKLpzPhXlj+pTZKsT/iVRZnBWM5ki1522iaJmNXuAaTuYy0WfjXU3 gpY1xCeBvR0NoxBrvZxu+A5m0eoWJWjGoKzhZogVD+Jb4oFC1Bu+ymDQqSWuTzqTBPnXK1VcL9J 8TrvqzPwXVr1rhpUTX5qY6+lM08TIIKhhNVjLfC8I7JXnf5+ppGsy5z1SgT5xuI89Dv01VPAlki 02CMl4p1pcFq+CUcbF9l/1g3eFBaAdpewUaQhGKMXPN83vg9PU9aEcxDIJMgea6W83dId5RSB1F 8wkLbz5P X-Google-Smtp-Source: AGHT+IHJsZoBMBkLol8NRyoMUhyLk6Yxfu4EjG4wqDyWD6EwH7i2BeNlGm0Phl+st/gLUAqxALBIPg== X-Received: by 2002:a17:90b:2b83:b0:2ef:9dbc:38d1 with SMTP id 98e67ed59e1d1-2f4435ad5e6mr82526493a91.5.1736133442245; Sun, 05 Jan 2025 19:17:22 -0800 (PST) Received: from Barrys-MBP.hub ([2407:7000:af65:8200:a54c:5ad3:ad27:edb7]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f2ed62cde6sm38471399a91.13.2025.01.05.19.17.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 05 Jan 2025 19:17:21 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org, x86@kernel.org, linux-kernel@vger.kernel.org, ioworker0@gmail.com, david@redhat.com, ryan.roberts@arm.com, zhengtangquan@oppo.com, ying.huang@intel.com, kasong@tencent.com, chrisl@kernel.org, baolin.wang@linux.alibaba.com, Barry Song Subject: [PATCH 0/3] mm: batched unmap lazyfree large folios during reclamation Date: Mon, 6 Jan 2025 16:17:08 +1300 Message-Id: <20250106031711.82855-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: B0A8B18000E X-Stat-Signature: u5wmao7nc3bf579ky4nagefiszim1ty5 X-Rspam-User: X-HE-Tag: 1736133443-848981 X-HE-Meta: U2FsdGVkX1+SJ6ehAqCgmGj0TqpMxHdCNjPWB7KXAKyN4NuNop5qiQkMRcELrceYLROlHFdKhQ+JWfgFkgZnBJwVrsBTLNTfTPsYAJ7Vnrxfcq1PDZLvtI8pORHtQWCoWU0OrJM7jKDInMdv32I32iLvN9z9YQyuHqWVyAk8oMfFWYJRUASKt2hTFXvaksLydDNn3X3shGaJgv2lNSiXPYZeKynF0CfcjD7MHYnpFznNSQI3GBuTMG1HZFvly07f2fIeAWnqs5VrjA1c6eELeRUTeVwPs04+Qns8DHAymEZOmRLQ4LtauScBFTyx+WLcsiOcHkQWIFrl1XkOXv+C6YpHM1Oh+A8UvRV73Ykd/ssHfV6WHN9GDffR1vqYy0LcLbv/wBdKSH3dugc4r+K2y6tUA6CeLK/k5cV7DGIT+VlJH3Ml5nx0H8PAnR7EhaUrp2NrFjTGiU8PIh4S+R046S77d/uQO8vGWr9Zs+FPwvPr/U3+3sqLcdm9vuOZ6AGWqjemahQ5T7rM0X5RMAmyXb3IhVqOV2kCfA1/v2tbi3cHnCKs3zjjYGWss4HGjKJdrh/cVdczS14H2FhIL4SJJzQl32XUq2XCCwBJ2FHsVLrJyOzv6fIPzu7Dgi71BuA1WdkTK1AcON7b2dbseZ+GO97hFkb2GBW6m/e95Zite3odrqnmymM28jnD0HF1Qw4VkF1IE50xFkZwyd9PDPjfHLtpkbroEwL/+s4QgfzD0mMvsFsv88Jyhb020vkx6vPNWBETs9IV1/+xajANKrYXb6rO3b/7U5iMVOPdfxKGP0U8U6ah6qlb7HhGL2B0VmTFRDgQF21v9EdKvB6+YkVVQHpBMSrpY4HN8i8cijq16vm5eaaQr3zmvlMGXrlVB6Hh9pi2gWRNsZ8RzT02yBpylBSRdQkNBWc9Yuh8Bv0WAi3S7jvlsv32gONf+GdLb3dNs6fNs0vM5CDV480+/gP qHIRhOJB WWEtGJ3I7jdzUjj3w/jzFzACA7mqQ1ULl6zozW8w/omrObIP5pnVJnKcboxp46DnFTv/9VmzJlW4nOU4WlIgr7dsUOzP4CwTOFeNSpk2cwDaf7DvRll3C+eaqA1N7IMljC45Wvc0Zu6maK/hQor+ESPKIkR9Ve1vPCGMQ8mdwFPbJ3I04U9o4cJmy+uASoQFWSWZfWiJmANKoBPUwrKCY+6rssnbfjKl2pKI9IFXwViTxdnyvOigtyisboqHbKiOeYA6o/7wOcs99LWZjg4F0pPxNU/PJWYErHYcUrb7tq4tgnWH5D3LV8aP/4kxTpXwdsTLWfQLNCYJhtwbNUB8Z10P92mq0pBcfKlTFgjmBrE1A7jxknY+YbWts4ui6OcU4W+GclzLlA9yFwICjdigvJyidRFgEia4rNDlyM3Ll7tUGrcI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000115, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Commit 735ecdfaf4e80 ("mm/vmscan: avoid splitting lazyfree THP during shrink_folio_list()") prevents the splitting of MADV_FREE'd THP in madvise.c. However, those folios are still added to the deferred_split list in try_to_unmap_one() because we are unmapping PTEs and removing rmap entries one by one. This approach is not only slow but also increases the risk of a race condition where lazyfree folios are incorrectly set back to swapbacked, as a speculative folio_get may occur in the shrinker's callback. This patchset addresses the issue by only marking truly dirty folios as swapbacked as suggested by David and shifting to batched unmapping of the entire folio in try_to_unmap_one(). As a result, we've observed deferred_split dropping to zero and significant performance improvements in memory reclamation. Barry Song (3): mm: set folio swapbacked iff folios are dirty in try_to_unmap_one mm: Support tlbbatch flush for a range of PTEs mm: Support batched unmap for lazyfree large folios during reclamation arch/arm64/include/asm/tlbflush.h | 26 ++++---- arch/arm64/mm/contpte.c | 2 +- arch/x86/include/asm/tlbflush.h | 3 +- mm/rmap.c | 103 ++++++++++++++++++++---------- 4 files changed, 85 insertions(+), 49 deletions(-)