From patchwork Fri May 10 06:51:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13660936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A80CC25B75 for ; Fri, 10 May 2024 06:52:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7CA06B0099; Fri, 10 May 2024 02:52:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DBB56B009A; Fri, 10 May 2024 02:52:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82F396B009B; Fri, 10 May 2024 02:52:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6730A6B0099 for ; Fri, 10 May 2024 02:52:26 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1EC748177D for ; Fri, 10 May 2024 06:52:26 +0000 (UTC) X-FDA: 82101567492.23.50BF901 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf02.hostedemail.com (Postfix) with ESMTP id 369B38000A for ; Fri, 10 May 2024 06:52:23 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715323944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=evCmjUh3KdBYN1DfbQBG2ARQXc+DAdC3ZcSVsI5RRA8=; b=oLgdmbG2q0WuDUWfKtkprkaD+GjFVy2xB/t5HuwHk6Z05DI5RzUMkUu63HtMQagZyzkhCo 3Yiaf4expT0P1ElF/JAC16n3WsTGuCfT2d1SPMvchyuaiI48ysoizMIByOsIj+8F77x/0g +lDoBeOyCzrZaAVfmcbE1IQ1wSZk2Vo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715323944; a=rsa-sha256; cv=none; b=thCkSwrzkBLH2Sfc2yx/rURYN1fIjci8viNPwbYrFuasozMZl3jcLuDm83x0nVBe08xYZS /ZqYdX0cmXEue2BU8pDov7ARDRBMMwn8EthfwJ4A4AabhrQMY8bgTAruwHG73zy1eiK2Rj XeHJ/ygCGYa8snUMdedHYdh0upNKYps= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-d3-663dc4210637 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v10 04/12] x86/tlb, riscv/tlb, mm/rmap: separate arch_tlbbatch_clear() out of arch_tlbbatch_flush() Date: Fri, 10 May 2024 15:51:58 +0900 Message-Id: <20240510065206.76078-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240510065206.76078-1-byungchul@sk.com> References: <20240510065206.76078-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsXC9ZZnka7iEds0gzXnFCzmrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAlfG o18T2Avu81WsmbiHtYFxKk8XIyeHhICJxOvGt2ww9q+Wp6wgNpuAusSNGz+ZQWwRATOJg61/ 2EFsZoG7TBIH+sHqhQWKJdZsPQBmswioSlzq7GYCsXkFTCWWNz1ghJgpL7F6wwGwOZxAcz4s mwFWIwRU0/FgIlANF1DNezaJnvPX2SEaJCUOrrjBMoGRdwEjwypGocy8stzEzBwTvYzKvMwK veT83E2MwNBfVvsnegfjpwvBhxgFOBiVeHh3bLZJE2JNLCuuzD3EKMHBrCTCW1VjnSbEm5JY WZValB9fVJqTWnyIUZqDRUmc1+hbeYqQQHpiSWp2ampBahFMlomDU6qB0fvfUYM0u7jVeiuX Vz9Ise7smlok11CxukSPyUt5XnOvldRbdU6rHa7F137uiFg+zV/U+qVv4xYRgzzVdzF6DpcV Q70mnvrQum7Xo0Bu6Tal/6rVP8+23RE5bmbn9zbo1f32n7166oyrvqdn7Frow7LQ3o55qvtn L76Nflxrv+XscJ2zZ1eaEktxRqKhFnNRcSIAet1+x3kCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrKt4xDbN4Ol7EYs569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfHeA0wW 8+99ZrPYvGkqs8XxKVMZLX7/ACo+OWsyi4Ogx/fWPhaPnbPusnss2FTqsXmFlsfiPS+ZPDat 6mTz2PRpErvHu3Pn2D1OzPjN4jHvZKDH+31X2TwWv/jA5LH1l51H49RrbB6fN8kF8Edx2aSk 5mSWpRbp2yVwZTz6NYG94D5fxZqJe1gbGKfydDFyckgImEj8annKCmKzCahL3LjxkxnEFhEw kzjY+ocdxGYWuMskcaCfDcQWFiiWWLP1AJjNIqAqcamzmwnE5hUwlVje9IARYqa8xOoNB8Dm cALN+bBsBliNEFBNx4OJjBMYuRYwMqxiFMnMK8tNzMwx1SvOzqjMy6zQS87P3cQIDORltX8m 7mD8ctn9EKMAB6MSD++OzTZpQqyJZcWVuYcYJTiYlUR4q2qs04R4UxIrq1KL8uOLSnNSiw8x SnOwKInzeoWnJggJpCeWpGanphakFsFkmTg4pRoYrUq/Se5wY/gcFDRDdW5RKZf8FrlQtXUc yZofHI8zzp5mLR3hvE/2nMmeLsUfOzT8H//51+skt/g0pzv/sWOKgkvfFzd8kd7p2/znnGfP NSPrXdzOJdd1wn8ZzS56sJP/hfbjlauebfN8+OWCX/I3gabz88/KTw+7ua9VgrGhvieqP3Ne 1UlXJZbijERDLeai4kQAZoPofWACAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: dp13dtu78whuparzb3z1a1xo8dayr7r4 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 369B38000A X-HE-Tag: 1715323943-523185 X-HE-Meta: U2FsdGVkX1/DQZpBJjHo5zCrBhMBeNYTsRng6hNuPw9rcc0uTGh1qHAd1TbIHkWw5yBzD+upgeTiIanuRi93icRyh9uVVFWk1vkaMpVxT19MaUAzQPZsvvPcrCulWs/5Xqw/W3fNlbMqE4FKChcKFK933fQTf55p1Gt5qWU1KkLFVmaLx5aq/XpAkG9j/hZPswBGgjCG+UZGNHv8nyWe7DSnVkXjvaCH5+Fn9kpfMDvKEp6B0YVOwcriCEIzxTpZev0kqY/o/vtKGL3wN5PcxJpRKIo7hWsyXE9PHKhvQRZUUSznTVtWec+fJ9bSkZaqtksPQ/m9M+rbiXujXVCl2PU6KKhQTd3FjhfhDULcPPYu7ZLHVweMdsX5OIR0Ti19eYpszWKiYsyru04y1HyGrlGr+JdyfDinhopiUGgIU+4VRn+ujCDs/FZw2Yq90pNcMIaCpQiB+Lrn+s9O8Y7Q/xfb+HYbpTWnBz6WMdS0JX6eWa6BrLYCOX73ueRmcRJ1d+D6OrkJumw/ZzTSTe3z97COnDrN1fSHyjvNHBNyGVo9Eg3BbHMjuM2Q2PecGuUzchOsj5YlOu81WqZYs8wrgqHMQiv2jOIOl5PH23zQawg66lRw8Oct66p+Weps9p6yrcB8afcAeSMmUH3nW12XV8nJRKgSsMkFkFxCoc1/XYcMmU4QuJN5rbt+KRHoNgmK6QJIhnfQpLak8TOrUZrFYOpPYkvIpl+IA5ArV4lj5oxWo+Geje2dXFQ+J5LTu/vy2D7qAzQFstGo/BBwJC+JWUcovOc4PmMGjvTnVOMk05zsFV7aRM+3pZwxdh4gSoCdH++nuHmwAFMsZI/VFDetWnEsyla2nAg59pKLyun5IbDzAKoLCRCUAOTbLdxUtr1ayip5fBZuYbBJ4fKk5mURFmHWSQNmPHb0BsftrIiFtDICWf0QIff9G5hI3mF0JICIPOspV1hl++SJBwV47hP l3FDG3cp ztZ3lfFpGXgbjrT+kEs20vGPFdAsUOYQ6Cao7CQGzOtEuH1WTo9uhC3A89vxxYNb4VeJBClSuC9q66zMONAZUP0ZELw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that requires to avoid redundant tlb flush by manipulating tlb batch's arch data. To achieve that, we need to separate the part clearing the tlb batch's arch data out of arch_tlbbatch_flush(). Signed-off-by: Byungchul Park --- arch/riscv/mm/tlbflush.c | 1 - arch/x86/mm/tlb.c | 2 -- mm/rmap.c | 1 + 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 07d743f87b3f..9cbd27148357 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -234,5 +234,4 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { __flush_tlb_range(&batch->cpumask, FLUSH_TLB_NO_ASID, 0, FLUSH_TLB_MAX_SIZE, PAGE_SIZE); - cpumask_clear(&batch->cpumask); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 44ac64f3a047..24bce69222cd 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1265,8 +1265,6 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) local_irq_enable(); } - cpumask_clear(&batch->cpumask); - put_flush_tlb_info(); put_cpu(); } diff --git a/mm/rmap.c b/mm/rmap.c index 2608c40dffad..cf8a99a49aef 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -649,6 +649,7 @@ void try_to_unmap_flush(void) return; arch_tlbbatch_flush(&tlb_ubc->arch); + arch_tlbbatch_clear(&tlb_ubc->arch); tlb_ubc->flush_required = false; tlb_ubc->writable = false; }