From patchwork Thu Feb 20 05:20:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13983326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9A54C021AD for ; Thu, 20 Feb 2025 05:21:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EBE3280299; Thu, 20 Feb 2025 00:20:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 52A6D28029A; Thu, 20 Feb 2025 00:20:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 268B928029E; Thu, 20 Feb 2025 00:20:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BFD8028029B for ; Thu, 20 Feb 2025 00:20:45 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5B5B9120B46 for ; Thu, 20 Feb 2025 05:20:45 +0000 (UTC) X-FDA: 83139173250.10.709DFA2 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf05.hostedemail.com (Postfix) with ESMTP id 3FDAF100005 for ; Thu, 20 Feb 2025 05:20:42 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740028843; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=YViRiZjtPoB9kuQIa3Bm4xJPkLXziufXUzBCUgtihVs=; b=uN6gWB0d0IOBD7nsicKBZUZfGPEeOCTPZsB3gcTP1S+wAleenpau2NNoxYYXOHEiagSa4k v4jfWnfKtCxHXihqAfjw5aqX7cA6qgR1SYIovxIdYV8kbzFaJOHzC0ncUPYTVulzhV9Ica B4TNDvCgxWO+hls7G8uF633RQe2qXpU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740028843; a=rsa-sha256; cv=none; b=2XTBY+5DxV8bu5DZhQOpvxOmnjMD42YxLO1euh/RNGWataXFthCvRzCoZ2TPLpDjuKVc7j tEJrba6cpz8SoYuA4YBl8RYP+p19CtRcnNP/5qZSV6EnEUNy0RRblObkL4RDGRnkyKE+3S wZSLR9JQvpS+IkfhimdniORm9dKpckc= X-AuditID: a67dfc5b-3c9ff7000001d7ae-d0-67b6bba63805 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [RFC PATCH v12 07/26] mm: introduce luf_ugen to be used as a global timestamp Date: Thu, 20 Feb 2025 14:20:08 +0900 Message-Id: <20250220052027.58847-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250220052027.58847-1-byungchul@sk.com> References: <20250220052027.58847-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsXC9ZZnke6y3dvSDdZuU7WYs34Nm8XnDf/Y LF5saGe0+Lr+F7PF0099LBaXd81hs7i35j+rxflda1ktdizdx2Rx6cACJovjvQeYLObf+8xm sXnTVGaL41OmMlr8/gFUfHLWZBYHAY/vrX0sHjtn3WX3WLCp1GPzCi2PxXteMnlsWtXJ5rHp 0yR2j3fnzrF7nJjxm8Vj3slAj/f7rrJ5bP1l59E49Rqbx+dNcgF8UVw2Kak5mWWpRfp2CVwZ PbsvsBXcEKm4PHkHSwPjaYEuRk4OCQETia3zj7DC2AveHWUEsdkE1CVu3PjJDGKLCJhJHGz9 ww5iMwvcZZI40M8GYgsLhElcvX4PrIZFQFXi5px9YDW8AqYSzT/vsUDMlJdYveEAWA0n0Jwf M3rBeoWAat4tuMTUxcgFVPOeTWLvmQnMEA2SEgdX3GCZwMi7gJFhFaNQZl5ZbmJmjoleRmVe ZoVecn7uJkZg6C+r/RO9g/HTheBDjAIcjEo8vDNat6ULsSaWFVfmHmKU4GBWEuFtq9+SLsSb klhZlVqUH19UmpNafIhRmoNFSZzX6Ft5ipBAemJJanZqakFqEUyWiYNTqoHRWkWcZQ/z1K3b Ct17P9uv/y4+12zX8/SHrS8C98anT2o/d4vR8J/mi5USYus2HxSw/TdhhU7lsidJheu3Wz/8 UB+324vZpCvwxNqaSc8VrnL53/Buv7wkr+yRv6nRtq6mp/1JVydeja7+J74rbOMFJff2sKWv 1r88dWjpmowLpz2r9Zf+3SW8SomlOCPRUIu5qDgRAPjD1ol5AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrLts97Z0g47nEhZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLw3JOsFpd3zWGzuLfmP6vF+V1rWS12LN3HZHHpwAImi+O9B5gs 5t/7zGaxedNUZovjU6YyWvz+AVR8ctZkFgdBj++tfSweO2fdZfdYsKnUY/MKLY/Fe14yeWxa 1cnmsenTJHaPd+fOsXucmPGbxWPeyUCP9/uusnksfvGByWPrLzuPxqnX2Dw+b5IL4I/isklJ zcksSy3St0vgyujZfYGt4IZIxeXJO1gaGE8LdDFyckgImEgseHeUEcRmE1CXuHHjJzOILSJg JnGw9Q87iM0scJdJ4kA/G4gtLBAmcfX6PbAaFgFViZtz9oHV8AqYSjT/vMcCMVNeYvWGA2A1 nEBzfszoBesVAqp5t+AS0wRGrgWMDKsYRTLzynITM3NM9YqzMyrzMiv0kvNzNzECA3lZ7Z+J Oxi/XHY/xCjAwajEw/vg8dZ0IdbEsuLK3EOMEhzMSiK8bfVb0oV4UxIrq1KL8uOLSnNSiw8x SnOwKInzeoWnJggJpCeWpGanphakFsFkmTg4pRoYW/d2RHkca+B9vJFh53KuN8mTdZ89zMzL ffNLI+Dor52v2Rhtzm6fuVwy99qCTT6F68QVG9aYGk9+e5wl+lNu2VUJrdN/73efdPK6WZox YYH6w6279jZP63dWlda+EOl/ZorP4Rdv9Mo6D9XcOfz5aLa5x0rlW9opqxhN3YTyJ+bUur7h ZPD1V2Ipzkg01GIuKk4EAAldmBBgAgAA X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 3FDAF100005 X-Stat-Signature: 7gsn4tg99pg5ndyg3b1a1tssojz8b3m3 X-HE-Tag: 1740028842-491042 X-HE-Meta: U2FsdGVkX1+F2yTtUNIVaBeXxI30DmE+8VF0qTCi8NTHSyUvC7rWqlIYtgsDSBFWRhBBPKMqzNAXPN/ZJKg9rIObG5gn+mYNtd6De9Lv6KF/NJqEpOEsjfbNZHQkCRTOPQclX1scMuA+KNGuwO7p14LgJ60x+1qTfjpvJkbSBsmedB14G0ezSyYat6IMyuu8SDssVb3mPWnES46g8xR5Nq39Ol5J2QWGbXzu868htRGMXYITT7S0nNm4cyxqw9OZXEts8/MaRmscnCUk+QPpLkFyPPEWkNZ+0r/zoVFJ8tUGjffXu4MOvpg5ck6J25CsiY8cFDoHd9tdvUn09S2KV/QL3XQKLfWhW6j/IrU3x2DtE0lZW58iAhVKnZQkfCD0xs+FPrt7+toIy067WFHIrplLJlQnteA4DdrO/SeXPSrj7O/N0C8W5erQfm1ZfdCtPfB3nzOokLx223uFcBBrELNbbELGM+mtoRVb1RMikDCqqdpVy1anZPtX0Gm9LUltJBwv4nVSQHHKXBbBUf7YlMH+g2pGH32Z9nVlWwMnV5o0cENe1nLuwf1EjXRrbIoCggKERL+X9XTtyVnqiDafjMeJOEGgdNzLrJc5ScTfre7BVVtv7Yibn/JCu6+759TNNr46MgAxfiYIF6XyizdfK1osh16HijesYvHmLXYmyjq2caLsaMNNqDSI+kzHdmWuhy3G/TVlImG/i5fEYzBK0oPmBAWBklm14pSX8E/U5GHj9cMWWp0EVNfptrv92zo4/6uD9EVpvn0yxxf/3y0rxEDnVWowpure/VfGtuwMgro5xGUTyVAm/KJDWPnnhskdqzzoL1lndhdJc9/B0vpKpSKty8EDznEIxS9P8tsBe2wMUXjPYmyb+vddiROY05IFgm3A3rZtFzz5MnxAoY4/jN44f7ZT0FBv8PzqG6e9Eqc9KedfxUesSf1U/xLwMP5efRq/jGFe0PaYVUoU+W/ a87w/eVa MAFs16Ga5B/60KgbI0EgyyOn4dy3wDnZMaxuRT2xMMCnjDbVHWAEU66T3memOJ5kZCOYV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Functionally, no change. This is a preparation for luf mechanism that needs to evaluate the temporal sequence of events to determine whether tlb flush required has been done on each CPU. To achieve that, this patch introduced a generation number, luf_ugen, and a few APIs manipulating the number. It's worth noting the number is designed to wraparound so care must be taken when using it. Signed-off-by: Byungchul Park --- include/linux/mm.h | 34 ++++++++++++++++++++++++++++++++++ mm/rmap.c | 22 ++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index fecd47239fa99..53a5f1cb21e0d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4161,4 +4161,38 @@ static inline int do_mseal(unsigned long start, size_t len_in, unsigned long fla } #endif +#if defined(CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH) +/* + * luf_ugen will start with 2 so that 1 can be regarded as a passed one. + */ +#define LUF_UGEN_INIT 2 + +static inline bool ugen_before(unsigned long a, unsigned long b) +{ + /* + * Consider wraparound. + */ + return (long)(a - b) < 0; +} + +static inline unsigned long next_ugen(unsigned long ugen) +{ + if (ugen + 1) + return ugen + 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen + 2; +} + +static inline unsigned long prev_ugen(unsigned long ugen) +{ + if (ugen - 1) + return ugen - 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen - 2; +} +#endif #endif /* _LINUX_MM_H */ diff --git a/mm/rmap.c b/mm/rmap.c index 2de01de164ef0..ed345503e4f88 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -634,6 +634,28 @@ struct anon_vma *folio_lock_anon_vma_read(const struct folio *folio, } #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH + +/* + * This generation number is primarily used as a global timestamp to + * determine whether tlb flush required has been done on each CPU. The + * function, ugen_before(), should be used to evaluate the temporal + * sequence of events because the number is designed to wraparound. + */ +static atomic_long_t __maybe_unused luf_ugen = ATOMIC_LONG_INIT(LUF_UGEN_INIT); + +/* + * Don't return invalid luf_ugen, zero. + */ +static unsigned long __maybe_unused new_luf_ugen(void) +{ + unsigned long ugen = atomic_long_inc_return(&luf_ugen); + + if (!ugen) + ugen = atomic_long_inc_return(&luf_ugen); + + return ugen; +} + /* * Flush TLB entries for recently unmapped pages from remote CPUs. It is * important if a PTE was dirty when it was unmapped that it's flushed