From patchwork Fri May 31 09:19:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13681401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32FE0C27C44 for ; Fri, 31 May 2024 09:20:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D56186B009C; Fri, 31 May 2024 05:20:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1B2E6B009D; Fri, 31 May 2024 05:20:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABC006B009F; Fri, 31 May 2024 05:20:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 80D2F6B009C for ; Fri, 31 May 2024 05:20:19 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3C65C1C226F for ; Fri, 31 May 2024 09:20:19 +0000 (UTC) X-FDA: 82178144958.27.20ED6B8 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf22.hostedemail.com (Postfix) with ESMTP id 65BA6C0016 for ; Fri, 31 May 2024 09:20:17 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717147217; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=zPJ8LPXwHB1Ubve+4Fc/lsX01Hbuj9Tq2nq76J2JNWo=; b=ZETPGRZqgi/Gt6FUS6K9tS/qRA5Kv6WT6lPbUYTCRxtriDYUUp0A52OSBID0DY0iBkVBUu Hff5pbnxjdVRgNonunxLSMMghM8PuT26E09MNSABNdSvCog/0re/ISenGTvUEMlT83ANis M3l1VvnCZbaPFIcmo+J6kuHkpM740fo= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf22.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717147217; a=rsa-sha256; cv=none; b=QfNjly5H4bp7or5thMTBOsnjZgIKS3etI4lqB9C8DThASG+GeuDkeKyaJtkr7ej6dL/C6w 2JaHi5wEXmygAEXyl+3+/UVfmhauJLq/UEQRydpVwVyEiWvplTVjkOXXEgLYEXBZ3a0TCd TTWl5liq7Y4mh5jbIoMSRPwB1jdhWMw= X-AuditID: a67dfc5b-d85ff70000001748-52-6659964cfef5 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v11 04/12] x86/tlb, riscv/tlb, mm/rmap: separate arch_tlbbatch_clear() out of arch_tlbbatch_flush() Date: Fri, 31 May 2024 18:19:53 +0900 Message-Id: <20240531092001.30428-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240531092001.30428-1-byungchul@sk.com> References: <20240531092001.30428-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsXC9ZZnka7PtMg0g1lNnBZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLyrjlsFvfW/Ge1OL9rLavFjqX7mCwuHVjAZHG89wCTxfx7n9ks Nm+aymxxfMpURovfP4CKT86azOIg4PG9tY/FY+esu+weCzaVemxeoeWxeM9LJo9NqzrZPDZ9 msTu8e7cOXaPEzN+s3jMOxno8X7fVTaPrb/sPBqnXmPz+LxJLoAvissmJTUnsyy1SN8ugSvj xukuxoL7fBVTVr9maWCcytPFyMkhIWAi0dD+iA3G3tJ3nBnEZhNQl7hx4yeYLSJgJnGw9Q87 iM0scJdJ4kA/WL2wQLHEnN7tYHEWAVWJG/f+soDYvAKmEuc27WCFmCkvsXrDAbA5nEBzDvy9 wwhiCwHVLPrfC2RzAdW8ZpN4s20CC0SDpMTBFTdYJjDyLmBkWMUolJlXlpuYmWOil1GZl1mh l5yfu4kRGPrLav9E72D8dCH4EKMAB6MSD29ARUSaEGtiWXFl7iFGCQ5mJRHeX+lAId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxG38pThATSE0tSs1NTC1KLYLJMHJxSDYzNRnL8/nuWnzK8qckm dq89+qGT2Yne8kVT90eyvfEWXxzTv3FlsjNL3Gn1u+zbr29Sf/kxPajbUnClote7hrnizOzi aa48x7I7rW2Lr1ty7bfYqpcyY4PbKU71ZV8WZwXVOccUTNFmkmadP2mi8nKjlp3p+YF1cq6X mNR2l7btOiJqWPn2uxJLcUaioRZzUXEiAHlk0Fd5AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrOs9LTLN4MVsG4s569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfHeA0wW 8+99ZrPYvGkqs8XxKVMZLX7/ACo+OWsyi4Ogx/fWPhaPnbPusnss2FTqsXmFlsfiPS+ZPDat 6mTz2PRpErvHu3Pn2D1OzPjN4jHvZKDH+31X2TwWv/jA5LH1l51H49RrbB6fN8kF8Edx2aSk 5mSWpRbp2yVwZdw43cVYcJ+vYsrq1ywNjFN5uhg5OSQETCS29B1nBrHZBNQlbtz4CWaLCJhJ HGz9ww5iMwvcZZI40M8GYgsLFEvM6d0OFmcRUJW4ce8vC4jNK2AqcW7TDlaImfISqzccAJvD CTTnwN87jCC2EFDNov+9jBMYuRYwMqxiFMnMK8tNzMwx1SvOzqjMy6zQS87P3cQIDORltX8m 7mD8ctn9EKMAB6MSD29ARUSaEGtiWXFl7iFGCQ5mJRHeX+lAId6UxMqq1KL8+KLSnNTiQ4zS HCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBkYd80cf9bWsTEMtDzqfUl7MKb3L3bLIIaR8 p/vFqnaDt20ar7w9dky5sEqWR2j2ny1O1/teb/jtm/vq0Du2Lwc/uBQ4q8e5/DrxWOnnxge8 AvdCHjmwh67JrYlfbnY5RVwovEljpWS7d9dh8XoutfmyLqFO3dNY1/0//Il1r8W5aVPzGDst XJRYijMSDbWYi4oTAWUft89gAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 65BA6C0016 X-Stat-Signature: cimbseo8w6cbc9oca7swm6pf1dnohsjo X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1717147217-203413 X-HE-Meta: U2FsdGVkX18OMD/t7S7LVoHCe/ah8nT85+roWkfJAJwMNWEgPjBIQhEYiM5Ni9xo/HaMFM66+o8p/QVU0AQ99Ti9ayk9lqjugDMRzGHZqlsztEnzko1fp2IXm7yzF6PiXSfYjwSZjDo0IZ+o8vZ6dJmvxo6Oi/N0ovk/a0s2+mTFWNywX56XBZSxMMYwfg94l9PhwsJzQQcJZ5sG1uQWhHor76lihClhdN/qci4hwj+5O/3rN/UH5NGSI1sTW69WuoF04GGMIF55IsbhJFDr1V4f7c8pOO9fsSYYU+bZHttjgGaGRpwYBitqozUez4LogwEd/iTL+wrnLvKopT83HnqsNByhcs+P0RBiuCW23PzI3bLOSDEQGdhM1wE8s5Dj1RrTDVJ+JAoO0lIxRW15Gaq5aCIXpkOdWYvchwNNFQSd9xYJ8ohjQEeFWdg0+eKfZRaKUf3Q0RB/OwKi9Q+mJJZUyPCQmWmEY6IrwdPFZM80tWFDcfO+hbuAAD4tCCvpLrBYDx7JITk7rid5W/UtOsT8vwlXikCdA+sJQycgQuGl40lNn6AeF9Uk1GtE1QUpwd8LeR41eW+tnzjavgQ6/vVN6RSc/jPtXbUvSxN+98StVBFKe/ZCGufmgizvkLYa0xdz9JDOACfCruCQ3PNJMq7zGzqOMiZO8uPh3v9BRpMBl7CLoGXHeRrzwuumDnQR8rRzf55cJMu+YUFxl6MtVCMbIEM2/qMcau+VIf/IBPU2IWdKvHJAxBnUk8xgqQmT6iVe2BpTJSVNmcqA8gaI70Kpj4Vo0GXmSRJ7Qp93U/PyyLI2MBxhT+kukNuO0VGVKiVRf0Vv+WQ/nnMMzvrwth124ETCeZsMhlVZjjqVlHEhM+s1iGD10xp34SKowIzydoytV4Nq/uTccDkkRdbnckNKsmD0dDmQEK0i2MnP5TaWO/vDoPvkieontCvlInVmmEAK689xYjtIsRM9aa4 QfGFAFuu +9GD3EqF/Gmzu8kZmHtxUA27XrxEKxsi1v+tu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that requires to avoid redundant tlb flush by manipulating tlb batch's arch data. To achieve that, we need to separate the part clearing the tlb batch's arch data out of arch_tlbbatch_flush(). Signed-off-by: Byungchul Park --- arch/riscv/mm/tlbflush.c | 1 - arch/x86/mm/tlb.c | 2 -- mm/rmap.c | 1 + 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 9b6e86ce3867..36f996af6256 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -201,5 +201,4 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { __flush_tlb_range(&batch->cpumask, FLUSH_TLB_NO_ASID, 0, FLUSH_TLB_MAX_SIZE, PAGE_SIZE); - cpumask_clear(&batch->cpumask); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 44ac64f3a047..24bce69222cd 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1265,8 +1265,6 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) local_irq_enable(); } - cpumask_clear(&batch->cpumask); - put_flush_tlb_info(); put_cpu(); } diff --git a/mm/rmap.c b/mm/rmap.c index 52357d79917c..a65a94aada8d 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -648,6 +648,7 @@ void try_to_unmap_flush(void) return; arch_tlbbatch_flush(&tlb_ubc->arch); + arch_tlbbatch_clear(&tlb_ubc->arch); tlb_ubc->flush_required = false; tlb_ubc->writable = false; }