From patchwork Wed Feb 26 12:01:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13992177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CA87C021B8 for ; Wed, 26 Feb 2025 12:02:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06F37280022; Wed, 26 Feb 2025 07:01:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F413C280020; Wed, 26 Feb 2025 07:01:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA0A6280024; Wed, 26 Feb 2025 07:01:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 835DB280022 for ; Wed, 26 Feb 2025 07:01:50 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BD2241A137F for ; Wed, 26 Feb 2025 12:01:49 +0000 (UTC) X-FDA: 83161956738.19.C17EFAC Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf12.hostedemail.com (Postfix) with ESMTP id E3DDC4000C for ; Wed, 26 Feb 2025 12:01:46 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740571307; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=9TElu9bIdjxoSWdWYcuP0UnxfGdL8F6PWgbYEg9QfNg=; b=laSdL0uA7dWWLcm1+lOmRISLH5QjK3junrO5pMMtNqkItPb3a5ji+KxXQpeg99Rm+FMTef X4qkSuyfRoQeV4vn6ATpsmKgvDfTulUmMB0/Xf1K3+4hBnuaC+SYKYx6bMdWPPr7Tu+oJh naoNXqTRwDF813JHcHQdLh7wObPOucE= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740571307; a=rsa-sha256; cv=none; b=66XQCOB/VuORpReRLXIkr6dpO6keYTLa2DHEWeO2FB4eQfZxtFRbF19+bkCX7EVeOzkLSm 5vfU28RUrhrfINkkZKYBCx+KiTpT+nHG66NU37b5PmV0woXt+cMx62Ou2fNv8rvBrXyF59 dZgjX667Htjak2KY0/zW4hSOJJwmOgQ= X-AuditID: a67dfc5b-3e1ff7000001d7ae-e5-67bf02a658e0 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, rjgolo@gmail.com Subject: [RFC PATCH v12 based on mm-unstable as of Feb 21, 2025 07/25] mm: introduce luf_ugen to be used as a global timestamp Date: Wed, 26 Feb 2025 21:01:14 +0900 Message-Id: <20250226120132.28469-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250226120132.28469-1-byungchul@sk.com> References: <20250226113342.GB1935@system.software.com> <20250226120132.28469-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrCLMWRmVeSWpSXmKPExsXC9ZZnke4ypv3pBqeuqVjMWb+GzeLzhn9s Fl/X/2K2ePqpj8Xi8q45bBb31vxntTi/ay2rxY6l+5gsLh1YwGRxvPcAk8X8e5/ZLDZvmsps cXzKVEaL3z/msDnweXxv7WPx2DnrLrvHgk2lHptXaHlsWtXJ5rHp0yR2j3fnzrF7nJjxm8Xj /b6rbB5bf9l5NE69xubxeZNcAE8Ul01Kak5mWWqRvl0CV0bzqwusBe9EK570nmNsYLwh2MXI ySEhYCJxdf4zdhj71YmfLCA2m4C6xI0bP5lBbBEBM4mDrX+Aarg4mAWWMUnsPdHABpIQFqiT 2LP4FFgzi4CqxNymD2ANvAKmEn9ftrNADJWXWL3hAFicE2jQv92/weqFBJIlWtb/ZgEZKiFw m01i3cKrrBANkhIHV9xgmcDIu4CRYRWjUGZeWW5iZo6JXkZlXmaFXnJ+7iZGYFgvq/0TvYPx 04XgQ4wCHIxKPLwPzuxNF2JNLCuuzD3EKMHBrCTCy5m5J12INyWxsiq1KD++qDQntfgQozQH i5I4r9G38hQhgfTEktTs1NSC1CKYLBMHp1QDI8OV5vXM9kcfzHxh5eYz7bzQx/c1B14UPdHy O1zU5Szb3147U+poxKsk52+nkn4LH5vgYvWZ1WTC2mreHW9NStT2y16WFbBq7b67TWeOo+KT 46lf510o0qtg22V6+8DXzF01pRc+Ti20ObTtQ7v8Ls29LTPyM3bsPpzzp1vTXnoBq9yDiKr5 h5RYijMSDbWYi4oTAWFdUNZnAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNLMWRmVeSWpSXmKPExsXC5WfdrLuMaX+6wemP0hZz1q9hs/i84R+b xdf1v5gtnn7qY7E4PPckq8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/f8xhc+D3+N7ax+Kxc9Zddo8Fm0o9Nq/Q8ti0qpPNY9OnSewe786dY/c4 MeM3i8f7fVfZPBa/+MDksfWXnUfj1GtsHp83yQXwRnHZpKTmZJalFunbJXBlNL+6wFrwTrTi Se85xgbGG4JdjJwcEgImEq9O/GQBsdkE1CVu3PjJDGKLCJhJHGz9w97FyMXBLLCMSWLviQY2 kISwQJ3EnsWn2EFsFgFViblNH8AaeAVMJf6+bGeBGCovsXrDAbA4J9Cgf7t/g9ULCSRLtKz/ zTKBkWsBI8MqRpHMvLLcxMwcU73i7IzKvMwKveT83E2MwCBdVvtn4g7GL5fdDzEKcDAq8fA+ OLM3XYg1say4MvcQowQHs5IIL2fmnnQh3pTEyqrUovz4otKc1OJDjNIcLErivF7hqQlCAumJ JanZqakFqUUwWSYOTqkGxjRprpfy61ZX31/ldHqa5/pFHmt0ktwKO+V/Lbr18tK1umbnHrkk 7V9bNA27uQ+a626vie62fj2rvWnOgTV3L356YvDJtKRlcSP/vlyFDYf3az/l8FFk/d+doKbY IBM298bFD6GlLZv6uCbaS/9KNTzy59e/vcEVf0+pHA9s2V64IcKz4IHZdiWW4oxEQy3mouJE ABhXBHZOAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: E3DDC4000C X-Stat-Signature: dnyjixfwsj98jzsh6t4hsz1ujmba8y1f X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740571306-615163 X-HE-Meta: U2FsdGVkX1/Zk/MpZIoW2v6N17B4x39POklkX+2Fe5CQfythZWRBuGgdrQAGLMz9PNNw7WyP16sVobBdIGbo4lPEU3ig/6g7J4o3SIegqzUFWXiK1LjnHTSEZ7M2QjgUvDKXkHhA9xlk6AkoEglj8KJE9IK9dahVWzCN6d0pm+4TXLLTMlle6zWdrUqAVwkxmqsgTIOqh7YI3Xqb0Tmq7m9oG00eQ8PjMsjtjJVmdk15+Xo19gLWudUeuIvoVtnrDtuRNZb+gtCtpVawxaMEMqPnRQy8L9XMp3tGfjL5x+ZzOON7mlsadOaG6PjGOwUnOmukq1nr9zq8Woxj8FsczckdS6D6agvehlRreWvJ1jltSo0MLHYYyCx8W22f3hS3wq/KUerH/+aYtAvPg+0Ur9A3R6KjMh8vOsj8U5jlv7cpwX8YkLWLCu0voD5yAcqUuYPIuCZptcqrulQpaC95NVFEpdxIrNCUOMW1+lkKiHKf1lPLfFmV2gXBrxaemOB6uO0Rc1FdtRhF7nhKwBdCYB5vZ8GpXYAEYErR+EVB8qXC40xv9AjfytJ6GxeizA/KebYgB8rSXwbcWXy3VDUb+uOcFMbME6lvfghXGqH3QVYLUfXMkIi7aKHYPxu5u5QrVun5VwxN4Db8PumDDYguC8kMdD0EeDFIx1sgcNjcXmnGLwD98BiMWiwBRWzvf9BLrAa+NcqWyhXfVZDxenp330cwYvQkuIRbfdJNM5Sjcb0Fejz9/YLVfqAHvIVd1Bvm4RzUX9KTjXRolxYloFdcP5riDUEFEUvKIadd5bYKxjOHeQmP+UVQSy6ADYZCDtlxXl0fFPwygEbCownF2ZFZ0tuEO0QX6r9rl0YZHHghpu54ow8nj9S4Be563V83qBJOohCHK4apsknqN+z66xlcAJSfQxrsfOrka6r+ft1hgcmcsjXs0OLrdveH3b2NRBLqQed2nBHVZU92pa+e5LK s3BhY41v ciWDLYVLWGHuAhkabLjbkPdvdp8u8qZEpFjJqrw61NhahZDgr/gawsbCew4sMEYbAwPBI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Functionally, no change. This is a preparation for luf mechanism that needs to evaluate the temporal sequence of events to determine whether tlb flush required has been done on each CPU. To achieve that, this patch introduced a generation number, luf_ugen, and a few APIs manipulating the number. It's worth noting the number is designed to wraparound so care must be taken when using it. Signed-off-by: Byungchul Park --- include/linux/mm.h | 34 ++++++++++++++++++++++++++++++++++ mm/rmap.c | 22 ++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index d82feabbe44f8..74a37cb132caa 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4240,4 +4240,38 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); +#if defined(CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH) +/* + * luf_ugen will start with 2 so that 1 can be regarded as a passed one. + */ +#define LUF_UGEN_INIT 2 + +static inline bool ugen_before(unsigned long a, unsigned long b) +{ + /* + * Consider wraparound. + */ + return (long)(a - b) < 0; +} + +static inline unsigned long next_ugen(unsigned long ugen) +{ + if (ugen + 1) + return ugen + 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen + 2; +} + +static inline unsigned long prev_ugen(unsigned long ugen) +{ + if (ugen - 1) + return ugen - 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen - 2; +} +#endif #endif /* _LINUX_MM_H */ diff --git a/mm/rmap.c b/mm/rmap.c index 546b7a6a30a44..8439dbb194c8c 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -634,6 +634,28 @@ struct anon_vma *folio_lock_anon_vma_read(const struct folio *folio, } #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH + +/* + * This generation number is primarily used as a global timestamp to + * determine whether tlb flush required has been done on each CPU. The + * function, ugen_before(), should be used to evaluate the temporal + * sequence of events because the number is designed to wraparound. + */ +static atomic_long_t __maybe_unused luf_ugen = ATOMIC_LONG_INIT(LUF_UGEN_INIT); + +/* + * Don't return invalid luf_ugen, zero. + */ +static unsigned long __maybe_unused new_luf_ugen(void) +{ + unsigned long ugen = atomic_long_inc_return(&luf_ugen); + + if (!ugen) + ugen = atomic_long_inc_return(&luf_ugen); + + return ugen; +} + /* * Flush TLB entries for recently unmapped pages from remote CPUs. It is * important if a PTE was dirty when it was unmapped that it's flushed