From patchwork Thu Feb 20 05:20:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13983320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29B81C021AD for ; Thu, 20 Feb 2025 05:20:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37C51280295; Thu, 20 Feb 2025 00:20:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 24EA0280296; Thu, 20 Feb 2025 00:20:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E5EAB280294; Thu, 20 Feb 2025 00:20:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 640C3280293 for ; Thu, 20 Feb 2025 00:20:43 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 01E22A29E4 for ; Thu, 20 Feb 2025 05:20:42 +0000 (UTC) X-FDA: 83139173166.15.646AC17 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf01.hostedemail.com (Postfix) with ESMTP id E31B040008 for ; Thu, 20 Feb 2025 05:20:40 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf01.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740028841; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=G30xGqy1s/coEL/U1j52C+U7PjQw0ciNR8FppuYW4fo=; b=AIC306DlZ5snILwlMMrQWVAGYOcgJF0k5rVE8qK7gS8NUU0/mGIk4TKvvaZXLguMZyaMzb dSFC2SsibgfUmvAzc1kJf008x4OUeh7eC11Vr4s0lMpZQKpZxnaQyUIntNybekP1KQeLSs Id4WpHxxn3iRLzhDjHMLqVFs+6ovaVQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740028841; a=rsa-sha256; cv=none; b=rh0QyKFU73x8oZ8zImsyI9qrRPIM+MC0s7y9gH/6IdTA9I74iTA+4xFso/3rpy7LpP89G0 su+92CRF36Vqmb5gEvBK7yW5RfTawsmRNx8ZEmrxYmuQela+mZFeQsVAueQeTw0/cHL4Mb Cmdnj/xPOCyW40jH4VH/pxw4mnxi55M= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf01.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com X-AuditID: a67dfc5b-3c9ff7000001d7ae-ba-67b6bba6ff58 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [RFC PATCH v12 03/26] riscv/tlb: add APIs manipulating tlb batch's arch data Date: Thu, 20 Feb 2025 14:20:04 +0900 Message-Id: <20250220052027.58847-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250220052027.58847-1-byungchul@sk.com> References: <20250220052027.58847-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsXC9ZZnke6y3dvSDRZ2ilnMWb+GzeLzhn9s Fi82tDNafF3/i9ni6ac+FovLu+awWdxb85/V4vyutawWO5buY7K4dGABk8Xx3gNMFvPvfWaz 2LxpKrPF8SlTGS1+/wAqPjlrMouDgMf31j4Wj52z7rJ7LNhU6rF5hZbH4j0vmTw2repk89j0 aRK7x7tz59g9Tsz4zeIx72Sgx/t9V9k8tv6y82iceo3N4/MmuQC+KC6blNSczLLUIn27BK6M HWvvshRcE6p48vgfUwPjK/4uRk4OCQETiW9rm9lh7GWH7rKC2GwC6hI3bvxkBrFFBMwkDrb+ AathFrjLJHGgnw3EFhYIlbh44jFYDYuAqsSulvdgNq+AqcTZXxNZIWbKS6zecAAszgk058eM XrBeIaCadwsuMXUxcgHVvGeTmP95PxtEg6TEwRU3WCYw8i5gZFjFKJSZV5abmJljopdRmZdZ oZecn7uJERj6y2r/RO9g/HQh+BCjAAejEg/vjNZt6UKsiWXFlbmHGCU4mJVEeNvqt6QL8aYk VlalFuXHF5XmpBYfYpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi4JRqYJQ1XSmtdOvglDaX t5zPrGM6rSYosl3bwZXhXXrfZ2rwM8v9+vxcN3QvzYha+YXpdU2K1LWfmtPv892V5GdRT977 NDpv072Oib/zd9Vkdf93zHxue3/3dskbV+5UPZ1n/S/Ta89mkc08FzleRXW7rtK7uGE7Ewf/ phWRHBuW6DVFnW+wKK0t/a/EUpyRaKjFXFScCAAdaBcNeQIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrLts97Z0g+3HeSzmrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XhuSdZLS7vmsNmcW/Nf1aL87vWslrsWLqPyeLSgQVMFsd7DzBZ zL/3mc1i86apzBbHp0xltPj9A6j45KzJLA6CHt9b+1g8ds66y+6xYFOpx+YVWh6L97xk8ti0 qpPNY9OnSewe786dY/c4MeM3i8e8k4Ee7/ddZfNY/OIDk8fWX3YejVOvsXl83iQXwB/FZZOS mpNZllqkb5fAlbFj7V2WgmtCFU8e/2NqYHzF38XIySEhYCKx7NBdVhCbTUBd4saNn8wgtoiA mcTB1j/sIDazwF0miQP9bCC2sECoxMUTj8FqWARUJXa1vAezeQVMJc7+msgKMVNeYvWGA2Bx TqA5P2b0gvUKAdW8W3CJaQIj1wJGhlWMIpl5ZbmJmTmmesXZGZV5mRV6yfm5mxiBgbys9s/E HYxfLrsfYhTgYFTi4X3weGu6EGtiWXFl7iFGCQ5mJRHetvot6UK8KYmVValF+fFFpTmpxYcY pTlYlMR5vcJTE4QE0hNLUrNTUwtSi2CyTBycUg2Mmf8unnOzMWgPtpv7YNm9NR7ndtyzjxBm c8/cdf0O+6fgDUe0CtYuZuboz4h4eTNl9azm2iSGtbU8d+6+azryIF5NqXPh7kMBmdk9jyN6 3Q+on9C1iov6canheVYog9OKibGBF3+xdcZyzheWfpq1ovml+qLM23wnX+7kTuUqb9z9W3XJ toptSizFGYmGWsxFxYkAIR/GaGACAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Queue-Id: E31B040008 X-Rspamd-Server: rspam07 X-Stat-Signature: aiiebrgfobisaj57dq15xne8mrpqq457 X-HE-Tag: 1740028840-509193 X-HE-Meta: U2FsdGVkX18DgvY4bTzVqeflIbJssxqidK8krBtWHdTctFjovf6jqblduMmtFl2q2NHL+JbHKQ3B5UdOZ2jTc1+58pMXC3ZLQMCSR/fnvspnDe/BJE50+Utp6fuvGf/NQ/MdXWJ+tXDXVrcmYqhD6xpCKNutBlTgMyxjOa6qG8xW+MsFAjglcyuQBS+v4zX2fGhE2nMiumkDAAzGycBtKVPV936VEizzlBrhM7Yyb3TCjz63mPYF7LKtrXbE7DBAQbuGOpxF2i0ykBbJ5ib3g5jXKpTZoEvq5b/eDYBLxG6a9qklwgPHffnj+ctl8ZdRHVOzzrXnBtTwdxujxVReml0+pKX0fc+rXV3owCGs5n1JfmL5zBSU8Yq67m8mRg4d29c1a1Njby5ceS/X95XClLHnsq2dY+NXClueIxUqr8KvDQZsYdNJwjzfS+QtHeVeHyV0p7b5GPdeZdnCYMbUmRhwa5ceFW2lQzlV8GcXRaNOPDqj1WKZKhm5GEEKjZ9Qt9J27ncFNiTm3eZD8RP6XvW/HtjrrXCwZQjlyMF+3fliuXRrBMtqTHSMrlIrXojV+5nPOsl0WRt0TQfhy19W+qZosTCB+thQiVXDHCE9Ppag1B2t8k3uif7E2X8iIEKigGN/lkG0K5o91ibJh+2bcdktgpdu91Fwhu4oluakiYLSCVYGFQrgIQ9ttCti+Lg8DgrgFBE06PT+OKtywvcuLgmWgUIiv0SG+lq1UIPXUsKVdwnGozbG8M9Sgx94I+XZeZJMDKjSH+udKCWwMHED5bjg/0rr3Bpc8saiuricd8ynif2rn80SQKWkyTta0ojQkRZyke8EhKKvGygvxUmP0jUvmGpUddz/RBYWTi44jgV7HWuRAqR2wu7KJhcKciuSyReBWyZaBAXutf9GB+IXArpStVKqqLdtfXLsvoaGz7g30tg4HYG7j6kNwYk0J5dG0sCykah4EaJuZZt2MGM wPjROh/t DIWfrUlVV17zUMrEVTNzVAJKwyA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that needs to recognize read-only tlb entries by separating tlb batch arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. It also optimizes tlb shootdown by skipping CPUs that have already performed tlb flush needed since. To support it, added APIs manipulating arch data for riscv. Signed-off-by: Byungchul Park --- arch/riscv/include/asm/tlbflush.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 72e5599349529..1dc7d30273d59 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_TLBFLUSH_H #include +#include #include #include @@ -65,6 +66,33 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); + +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + +static inline bool arch_tlbbatch_need_fold(struct arch_tlbflush_unmap_batch *batch, + struct mm_struct *mm) +{ + return !cpumask_subset(mm_cpumask(mm), &batch->cpumask); + +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return !cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0)