From patchwork Mon Jan 22 01:00:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13524730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A9C2C47DD3 for ; Mon, 22 Jan 2024 02:47:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F2836B0071; Sun, 21 Jan 2024 21:47:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 153A06B0072; Sun, 21 Jan 2024 21:47:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0F536B0074; Sun, 21 Jan 2024 21:47:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DE80E6B0071 for ; Sun, 21 Jan 2024 21:47:41 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7E240C0176 for ; Mon, 22 Jan 2024 02:47:41 +0000 (UTC) X-FDA: 81705411522.26.7B226C4 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf04.hostedemail.com (Postfix) with ESMTP id B537240019 for ; Mon, 22 Jan 2024 02:47:39 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf04.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705891660; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=eRD1/8o+qtEOTAKMY7ucioDJJtMI3+WpTL+ASvHX4To=; b=gLbkFrve7zrP9YjpYZ+TinRYCJE0r1e1AdOBGmtj4Ytdelbz+vz7gLnO1vDeNGlHPOOH5e x+bYasdgvTHf7JscZaH0V2PpJGQcWpK3eBUFNsqAVaqyt3tyWctDdfLQnxhGQAFZa274GH lw+7onPzFdMRP+CEpSHRKKdAJku/n18= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf04.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705891660; a=rsa-sha256; cv=none; b=Xjl+tmSj6RkEKZsJCTXW8bZyQV/jKxvraRE6kdP4YPgewdaKJSZItLadORfUbJ0MbOGjoo sTre6Ii67hDw3yxSKXp91VaMppU2v6YGoFrdA234OuqyWkFOjZL7CvG7ug7ubHmoqz37MJ af3gruJNQTjAWm0xIPeSN84Liz88WWM= X-AuditID: a67dfc5b-d85ff70000001748-6d-65adbe43e950 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, namit@vmware.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v7 1/8] x86/tlb: Add APIs manipulating tlb batch's arch data Date: Mon, 22 Jan 2024 10:00:33 +0900 Message-Id: <20240122010040.74346-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240122010040.74346-1-byungchul@sk.com> References: <20240122010040.74346-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPLMWRmVeSWpSXmKPExsXC9ZZnka7zvrWpBgunSVnMWb+GzeLzhn9s Fi82tDNafF3/i9ni6ac+FovLu+awWdxb85/V4vyutawWO5buY7K4dGABk8X1XQ8ZLY73HmCy mH/vM5vF5k1TmS2OT5nKaPH7B1DHyVmTWRwEPb639rF47Jx1l91jwaZSj80rtDwW73nJ5LFp VSebx6ZPk9g93p07x+5xYsZvFo95JwM93u+7yuax9ZedR+PUa2wenzfJebyb/5YtgD+KyyYl NSezLLVI3y6BK+P27ZdMBav4Kpb8aGdrYJzG08XIwSEhYCJxZXNgFyMnmPnnxXt2EJtNQF3i xo2fzCC2iICZxMHWP0BxLg5mgY9MEqu/d7CAJIQFfCQmHm5mApnDIqAq8XmTE0iYV8BU4uDG 14wQM+UlVm84ADaHE2jOlF9dTCC2EFDN3A87GUFmSgi0s0ts7p3PBtEgKXFwxQ2WCYy8CxgZ VjEKZeaV5SZm5pjoZVTmZVboJefnbmIERsOy2j/ROxg/XQg+xCjAwajEw+vAvjZViDWxrLgy 9xCjBAezkggvv+qqVCHelMTKqtSi/Pii0pzU4kOM0hwsSuK8Rt/KU4QE0hNLUrNTUwtSi2Cy TBycUg2Mq44Xf1h3iJtfq3Lpgi+TuS5bvTrVez/0xVxXZn9fs1Mzzv/aUrjt2yStn3X5orzP BL0lxf6aXrm8XVrZQ8pIRqBwm+OlZKtD53yk1Ix05re7vZn+I3FPXNqsac+Vnlgmphrk6C4L te098muTMteG06GvGC9cSrfviLp4uVT27qzDZktdhJ8cU2Ipzkg01GIuKk4EAPEP+0yCAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrOLMWRmVeSWpSXmKPExsXC5WfdrOu8b22qwckZ/BZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLw3JOsFpd3zWGzuLfmP6vF+V1rWS12LN3HZHHpwAImi+u7HjJa HO89wGQx/95nNovNm6YyWxyfMpXR4vcPoI6TsyazOAh5fG/tY/HYOesuu8eCTaUem1doeSze 85LJY9OqTjaPTZ8msXu8O3eO3ePEjN8sHvNOBnq833eVzWPxiw9MHlt/2Xk0Tr3G5vF5k5zH u/lv2QIEorhsUlJzMstSi/TtErgybt9+yVSwiq9iyY92tgbGaTxdjJwcEgImEn9evGcHsdkE 1CVu3PjJDGKLCJhJHGz9AxTn4mAW+Mgksfp7BwtIQljAR2Li4WamLkYODhYBVYnPm5xAwrwC phIHN75mhJgpL7F6wwGwOZxAc6b86mICsYWAauZ+2Mk4gZFrASPDKkaRzLyy3MTMHFO94uyM yrzMCr3k/NxNjMDgXlb7Z+IOxi+X3Q8xCnAwKvHwOrCvTRViTSwrrsw9xCjBwawkwsuvuipV iDclsbIqtSg/vqg0J7X4EKM0B4uSOK9XeGqCkEB6YklqdmpqQWoRTJaJg1OqgTHftnX2xfx3 FenT6ut/s2VKcJ9/P0X5S+rcc488mncab3jINtmTL5bhx+flJ8PMprJJv/2+6e6f40XNjtwH TiRp7hY0PyrlH2Vb6lzZ8G+n87uG50WGi89xLkqT+PEk5VbsYR2ObMHTm12uP57CJz5R+Hp0 X98Bu3C/2eb+b7/P+vq2x3/ltMVKLMUZiYZazEXFiQAKYOYmagIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: B537240019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: kd65e4tbgk886ssj5e456tdawwswwg7b X-HE-Tag: 1705891659-890021 X-HE-Meta: U2FsdGVkX1+PlD4PBvmvvcUweHeU8pdTqsYsXix9iWrNAO14QTJ16CFcrph5WpjWCI7ZQxQst+p+7WmosqYTBKj4HIoKxTSo67bxShmTEl83R6Fb63YvKL6SE/8a7vfHVgxHE3wm1KT8fC7OSKMiJRzcMhyZecPxKlNRoNUMe35BHmCbAVPIgCTwbW2tFbEqaNq9Qaht4v2H9uiNzL5fDkrk0zA34/efkQcBZVyvChsW/YNK1Rq12I3awYaD2ZbnS+cjne9WhtVQznJeyvwYrySgem0JEKLh6Gw9GdYFDJcrV7T+tbeterqI+Ip6kj+is+kGz9mYYJd92yoa1qzY8b9BMNMdsJ+zSrNu4pboNTfPuNa+1FW4+Fd7ogqRKdKqpIlU+bZRYmI7J9++zK51UOIrJLDo5EXrVL+UrCkWxBPAcEI4SPT9clZy7CQNcQQI1bioEemfb5Yr9DkKnqMjXdACy19lFlAE/NuQV2LdBCSAUP1mvIMtb8It38TlqOkS1d4OHGe3HPkq3xEaTAxRmK7zFAUMKGEZiaxBLm9Ob3E2dgbWxb8lyargEcwj6j7qmyxsMNNe8i9pfhyYiqZ7E81bRzGkxtP6SX5d/6FzmvEqJaRyfJK8GtnG5dR91MjKaf42qIy+dafmy8I0Kl7mAn9XGyQ6xm3IRxdwk3h/shNdsYj66nV+AzOQtTczpZK3YcNLvgsCTcZ1664L8dXAJQ7CNTiyRT8Y8BdHPWpEMoLTmQPq44mz0FwSKXsiQgMRSZ9HeKg/6IrXpvRthzWK808nkDa22FMCMe+bVYsvc2YhFxYbirGPhsSZuNLVuO2C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only TLB entries during batched TLB flush by separating tlb batch's arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize CPUs to flush by clearing ones that have already performed TLB flush needed. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..fa7e16dbeb44 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)