From patchwork Thu Jan 11 06:07:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13516787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A124C47077 for ; Thu, 11 Jan 2024 06:08:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AAA9B6B009A; Thu, 11 Jan 2024 01:08:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A5B496B009B; Thu, 11 Jan 2024 01:08:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 923066B009C; Thu, 11 Jan 2024 01:08:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 81ECC6B009A for ; Thu, 11 Jan 2024 01:08:30 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 583E1140613 for ; Thu, 11 Jan 2024 06:08:30 +0000 (UTC) X-FDA: 81666000780.24.045DEC6 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf20.hostedemail.com (Postfix) with ESMTP id 5153F1C004D for ; Thu, 11 Jan 2024 06:08:11 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704953292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=eRD1/8o+qtEOTAKMY7ucioDJJtMI3+WpTL+ASvHX4To=; b=tLq7Qhq1LtK4qdsWcQDK1qNB+uD5hOZ9CjflspFLpAu7lfdvh2jI9cKfX6tsuYWhcdxkGp I77fWfHI7vpxYKKOHpl/CcYew3ixokz5R95v3CbQqdL6d55lEGTWDAEzcjjjN52yorg18x G16axUv0kUqBG9PDZpTdvjU5mBN9EiI= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704953292; a=rsa-sha256; cv=none; b=INypHW4Weha42z6QZ83kB8aJmRXlgyeG7rIXDoIq5LB7TJcZbcv1xomJYH47MM39XZdOKj UGIpmsTyX7lA8RU5j/NH+J0m53Wqw+Rv4LaZEHO4e3lQQhXnINYuuV6p9CLiNA5QWtfd14 H+ed1sC8ulYGDfmrYwhIYQ/GVbawdpk= X-AuditID: a67dfc5b-d6dff70000001748-de-659f85c70830 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, namit@vmware.com, xhao@linux.alibaba.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com Subject: [v5 1/7] x86/tlb: Add APIs manipulating tlb batch's arch data Date: Thu, 11 Jan 2024 15:07:51 +0900 Message-Id: <20240111060757.13563-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240111060757.13563-1-byungchul@sk.com> References: <20240111060757.13563-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsXC9ZZnke6J1vmpBg+usFrMWb+GzeLzhn9s Fi82tDNafF3/i9ni6ac+FovLu+awWdxb85/V4vyutawWO5buY7K4dGABk8X1XQ8ZLY73HmCy 2LxpKrPF7x9AdXOmWFmcnDWZxUHA43trH4vHgk2lHptXaHks3vOSyWPTqk42j02fJrF7vDt3 jt3jxIzfLB47H1p6zDsZ6PF+31U2j62/7Dw+b5LzeDf/LVsAXxSXTUpqTmZZapG+XQJXxu3b L5kKVvFVLPnRztbAOI2ni5GTQ0LARGLby5dMXYwcYPa8TZkgYTYBdYkbN34yg9giAmYSB1v/ sHcxcnEwCzxgkpj7dgUjSEJYwE2i/8M3JhCbRUBVYtHNTlYQm1fAVOLdzR5WiPnyEqs3HAAb xAk06MLhWYwgu4SAah611UGUfGeTeLK0FMKWlDi44gbLBEbeBYwMqxiFMvPKchMzc0z0Mirz Miv0kvNzNzECQ35Z7Z/oHYyfLgQfYhTgYFTi4X2waF6qEGtiWXFl7iFGCQ5mJRFehc9zUoV4 UxIrq1KL8uOLSnNSiw8xSnOwKInzGn0rTxESSE8sSc1OTS1ILYLJMnFwSjUwTrb4OvVb5d9S D6ZpXYaRl+LjdnRMndy4f3VFTuefjZqHeTySHUVfrylIWJGwti3NqKsmLMJ6y7xNaUenur6z 3dhvm7B3TfjV5dv9v/YY9K3ZNLcv+leuuqj5dr6EY5eTGr4GzfNI59/59dLaxd9PHKzv+nT9 4zK+ednHN/h+M7uxNdTxtvM/fyWW4oxEQy3mouJEAAWL0dt1AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFLMWRmVeSWpSXmKPExsXC5WfdrHu8dX6qwdM+C4s569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfVdDxkt jvceYLLYvGkqs8XvH0B1c6ZYWZycNZnFQdDje2sfi8eCTaUem1doeSze85LJY9OqTjaPTZ8m sXu8O3eO3ePEjN8sHjsfWnrMOxno8X7fVTaPxS8+MHls/WXn8XmTnMe7+W/ZAvijuGxSUnMy y1KL9O0SuDJu337JVLCKr2LJj3a2BsZpPF2MHBwSAiYS8zZldjFycrAJqEvcuPGTGcQWETCT ONj6h72LkYuDWeABk8TctysYQRLCAm4S/R++MYHYLAKqEotudrKC2LwCphLvbvaA2RIC8hKr NxwAG8QJNOjC4VmMILuEgGoetdVNYORawMiwilEkM68sNzEzx1SvODujMi+zQi85P3cTIzCE l9X+mbiD8ctl90OMAhyMSjy8Bi/npQqxJpYVV+YeYpTgYFYS4VX4PCdViDclsbIqtSg/vqg0 J7X4EKM0B4uSOK9XeGqCkEB6YklqdmpqQWoRTJaJg1OqgdHRV6F6dvfDWYfPVhxTevCV/ZZT UfEO4XX+Pckit20DZh7cy3HyovC9rDdHZBw1/untvNm9f4rPFY2DQdrTbklzq4c6btu0Ikda IKPsssTEG8edT6wzaCjM3xqdXHTvyaFCVgvzXqeWi9/CRPIrmf85ey7jNVWdkrNucvb8G2Gl D668dZ3esEOJpTgj0VCLuag4EQDnu2KHXQIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 5153F1C004D X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ja1d831zagxup5ho1743z697zyqhru49 X-HE-Tag: 1704953291-328286 X-HE-Meta: U2FsdGVkX19QbKaWMJuu/YhKuzOLbTPfk+5Z5hIAIu2ExbO0INwOCK8n7Ye11JXi9UqWH8eIMZiFoyCPbJZB/ZgLV4pFMz4eOlPIrUa4IoPmOLdeCvOOqOx3+dvdzXMMlTb5HVDSPRXoIum0K9coCjaKiAsTgdLs1aQFstf2IaWfroGsBOSVlHqcIYdmI+FYoSAtMRU2saahOLRG2QjLcnXvgSxspBBcWbpuaiChEwXdy03M7RhSlfQq3qlsAZkuU6iyI5ETesqCYyL2JtUu5b71tk66GwAIFkQIzolUp1ouHvIJMtKZ/LWadyEVh/gaid8WIUQVX8bnCg5n+qLd7lwo9ZMKYqE6lGvV3vQZ14/H9k3BYbc8zU1VWdTwQBvqrhK4HGNSSfajUgjj04eC6d6upAaMsxAZWDHCtR8aizy8fGYESqWrBoM8SZ+4vk0WzCVp9wwRyFt7J5L4xj6iwttJgbQO+HkxXQmVecwSGGVyLGgMd7xtYROXnIGH1lq40vsymj68vgkV3WT08eq2Vrgql2ckpUxhJ1aYoqoYdHp5Lbz0oVQ3Yf89gTrkUCiCHxNVFsEZNxb2l/TCsRg/NBQxSByMKI6iSDh/y9KxEfz0dtF7ukHsa6CQbv2VsWQTZYtg1tgqnQi/JWMwQC7uC+k5U2seWIJrpNhxenqIYnDXqzlq/awS6s1WYyD0ybTR4DynwkLRG1bzrEPJcAlJTJlZMDntrr75CQ6+cT0DSxhAK4Yp9eO0IQkSF94Eri+4Rx+zXi7NmiNnOV2fKNch+lkSjUZ7tl/s6/gD3fSYbKDnqGDR4S3uyO4yMOMQndo8tx5og08VMZg6mM/Kxywt0BQFESEamOrsLfpH+Kv0eyFMyfJYmFwOXOZnRY2EkoCAGALPsYSzXnKMzxAd32AQw61KTf81j3v9MeMIPUIjt/QxeHfn5EqZzVr7A8DWrqm32xMvc+r7Z57jftRGXD3 QGMmGhRw oa2CSrPP2f5Do61/K4L0SitJH5tuoznIVZ5BA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only TLB entries during batched TLB flush by separating tlb batch's arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize CPUs to flush by clearing ones that have already performed TLB flush needed. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..fa7e16dbeb44 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)