From patchwork Thu Jun 13 07:11:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25666C27C79 for ; Thu, 13 Jun 2024 07:32:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wqPl1nP6zyXQjBRrJ4U280wsPUDjyWFy082Q0XgmG7U=; b=d3GVhsv2MCxao+ M+SzkMAclNvPuvoPPDNlVCv+k93uv7LhAsArfQrteYpQctvsAn8iCBrIzEEij9cSztgWa/BILaA6U U34hBDg+JfCSKLWfspQknQJQBr28shLR4e0cRPHSS2rpgwI0CdTrpVpzaXY9wSMQFVv7O770/CeMa nYQUCl3UWbWfBV5rGzsZra4omtb3mqnNQVJtcqPQhnzDh0DvOY2+GiyYViGDcI90z2VnjK4x+vnjZ zLrowMQ6aYwMo/sueU//cMgCJRjLYIo4oZqBD+DDRUU4g1Y88h3ICtt4atzVajOhfjdsPAOqzeyoU gKgVB7Q/oFAGRi6fRhCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHewy-0000000FUMp-1JFa; Thu, 13 Jun 2024 07:32:48 +0000 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHewu-0000000FUM7-1mEx for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:32:46 +0000 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6f4603237e0so519917b3a.0 for ; Thu, 13 Jun 2024 00:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263964; x=1718868764; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=90j1lP2iehIU9E2euRd9xJoPIJsKCYmj2Va6/Ket5KI=; b=kUchnbBDSxKylodBGsRZwOp/eF8yxFjfOftndnKfWJNoNs7nNok586qVwPilnPDq0j FXaNYVAo6qW5HfNA2myCY1201yN6AevzlMI8UA9Q12cyis5E34AIo/cbxHnJoQitFals dKayZN7JXtAMVLnOnIo5Vitzx30LuJwVMG1EXG2czyt4FiAJZQUMYP57jG69DXsiipoG UmccYF4lBY5w1f01OgKmsfrdaYoXJul6+R4mTNv06Ck/mqj7RGhY1o+EQI0GcXAKkvxD ElkJ5U+Gp7EKrBiXDk6Fgsjeg0jnmfZwyD+dSqrne9Cq1u8/RQ6tsPXGYX913LMdmvZg xs/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263964; x=1718868764; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=90j1lP2iehIU9E2euRd9xJoPIJsKCYmj2Va6/Ket5KI=; b=AlSCbdBCS2z3+aTGjB+i5M8wQUcerNf2ogQO6XKbXr5YHjoPB0nRfE9lt7A+mD7SMO t3AtmDRxYIBagw0RmztA/6dTix9G1JvL5dBM/RubT16ad6kBRWFDmj1hVxpuT9ELgo9L YXybvBtGd4mKwultSnFEXFJ+IZOZ+hrIAEV2fj2lOl2xHEC/7UOYkXFPTV+ZaFtPvUoE tZY1E2mAk1Ol9kKgBjvEtMnyF17qirjOWrNhLfeLTWtjDkWy4nRhpRgjr580u6IZRAbl fGkE9neIxivxld8LwOjwj8wftbOczu/w/QUS+SIe/SzerhAubNtiDAgcsiYFTozaLRgn tJBA== X-Forwarded-Encrypted: i=1; AJvYcCXmU3ZsPnY2ujyw28dHjhapEqqi3NSf2S3V+LqeSnH4z401wMXDd6ET8sAXuDXTHPsvNevfXObIdNF+yzCzvZc3uxRVghz45HkvEh51+Dcs X-Gm-Message-State: AOJu0Yz6Ksvw+tYUk6bMP+MaGD+mcAo9VJkQeAbKL5juPQo9zZWoyADp qzIszhug3xhHagpSdb0chdc9QQIU8r0ITu+/ZLbMhpZthGJnyuM//6iln0ikyMo= X-Google-Smtp-Source: AGHT+IGTwQ5EB55toThK3E4Aeh/60oGzOVaQMTb+lJV8UkAi/aBjTebo3WO7lB+gn84w0BlxoTDk4Q== X-Received: by 2002:a05:6a00:2d84:b0:705:a18a:686a with SMTP id d2e1a72fcca58-705c9389332mr2770164b3a.7.1718263963654; Thu, 13 Jun 2024 00:32:43 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:43 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:06 +0800 Subject: [PATCH 1/8] riscv: stacktrace: convert arch_stack_walk() to noinstr MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-1-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003244_513743_66B77461 X-CRM114-Status: GOOD ( 12.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org arch_stack_walk() is called intensively in function_graph when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. As a result, the kernel logs a lot of arch_stack_walk and its sub-functions into the ftrace buffer. However, these functions should not appear on the trace log because they are part of the ftrace itself. This patch references what arm64 does for the smae function. So it further prevent the re-enter kprobe issue, which is also possible on riscv. Related-to: commit 0fbcd8abf337 ("arm64: Prohibit instrumentation on arch_stack_walk()") Fixes: 680341382da5 ("riscv: add CALLER_ADDRx support") Signed-off-by: Andy Chiu Reviewed-by: Alexandre Ghiti --- arch/riscv/kernel/stacktrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c index 528ec7cc9a62..0d3f00eb0bae 100644 --- a/arch/riscv/kernel/stacktrace.c +++ b/arch/riscv/kernel/stacktrace.c @@ -156,7 +156,7 @@ unsigned long __get_wchan(struct task_struct *task) return pc; } -noinline void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, +noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, struct task_struct *task, struct pt_regs *regs) { walk_stackframe(task, regs, consume_entry, cookie); From patchwork Thu Jun 13 07:11:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECB93C27C4F for ; Thu, 13 Jun 2024 07:32:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SXf3O6LY8GMNt9Kcl2kCN8zvfkDwXYK96pWfPs00uHA=; b=Yo9JJQHjjx/I4u 0em6O02su7sBzEg3Mhrw0xPO9EE2oCgboxxCT810tuIlQc8x/KqkszXYXhIpU+ecXBilw7Fub0e3J wJwhhxIOky4OcC9p08YSLS5MMz1x6qh+nuJtCg2C/zfMuC2gggmWoEig7Mvn6aIDvWf/GP0oEocNE I4a2Asv9IdRd02JaupvLb4Rg1n+JitKgKopfoLoZsB63fzWTfTI2BQXgvl7WGi/6w2NxXCCrL15sG J8xBaNbJFxsrYg4U+LZ5JirXbzrG44fWnx2nsBOvMJmql2gzCiabgZZl/VG7n9bNshGwJZaONhhBN 2YRE3O6753lv9GQdTjhw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHex1-0000000FUOE-0Nhx; Thu, 13 Jun 2024 07:32:51 +0000 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHewy-0000000FUMo-3GHt for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:32:50 +0000 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-705959a2dfbso497664b3a.1 for ; Thu, 13 Jun 2024 00:32:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263967; x=1718868767; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=FQsuZ21+9cGptdlB0BPgeEv8WU9oI426BLaNkf2S+w0=; b=SoRytKio3s3yrYzFJhAdxoA39gqTiWTxcBP7mbVhVDKtyQZ4rSgNTrtxgtv022VHmW +F24iPa4BQIMszphDcbJk+SgIEOcmhgPHhcK3ELMWBzwtPgJp/bwajN35j8/bXmmBqZf e68fNKsT8NT6GHtlbHlJSSDRs80uc5KMlUBusY++XXljFe+l1plDQafDFOa7xkJ7cMLm n3rfXpjKLh4KVmkHRYdYKSRqW73Kbc0iRhKArSI984z+KpYmdNVNCLaPKiiVW41jypY3 vuZpMnkJV6XqqEqOqdarQAGh2HMeXkUWiXgO3Obd3yh/4XwDylB98UVacdrrjX2jUhw/ czhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263967; x=1718868767; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FQsuZ21+9cGptdlB0BPgeEv8WU9oI426BLaNkf2S+w0=; b=ootHNlqqIlUk/Je66wKORWf1ew7hoHNFnXGQy4UH/ERdsKxqfpQIcVG/IJx96QUZoX ppwFKgplFErLFCZFv6DDhCd36hOg5Wwx2BUXM/wtfvfloFFuoSJ3TrSzWdaGpKj3vO+Q QbHpjH80k9R9o8UGPbIrY6cY++dKQSsSOxvqlet1bKrbCPr42npJGkBYDjMY/Gb142R0 oEIEvoL9/htgZJTRK15lDnyhFsZ0f4uhhlh/XEenRHoeZoqNOasg2C7bvpSKn6uG1cx+ hz07t3fX7W55DGd25gM8EQAlybP7VlN34tV6PCmQkrPEx6Ropwb3/h7hqSUjthvd1cI9 PosA== X-Forwarded-Encrypted: i=1; AJvYcCW7zI2mkWNcEt5FEn4JSsGWwNCFzhjap4CQ/86PNBwSeN2NE8U7F5dwn1OGwJh23AG0t5jrA+gyjiq4/iUFR94ui72TP5zBZXAM715Wf/a4 X-Gm-Message-State: AOJu0YzU/DRw3EEDAP9Z2RpvE7DmI45YaxumzzQ7fboegls8yuHjSBfv MUemn/LsVSt8n6ZwDGhGn+xUgs6LygSro6gywx+FCtYpD+Wu8MokwbG1XMqZPR0= X-Google-Smtp-Source: AGHT+IGkMW8dM0M3dwKshtF1f8Z+mSNjVPk7wRBBSBrIqzVVkpK0qMV/aHa7vJh7XzX910Gr5qBw7Q== X-Received: by 2002:a62:ab02:0:b0:705:d101:90ff with SMTP id d2e1a72fcca58-705d101919bmr283959b3a.16.1718263967547; Thu, 13 Jun 2024 00:32:47 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:47 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:07 +0800 Subject: [PATCH 2/8] tracing: do not trace kernel_text_address() MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-2-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003248_850681_33F1EE66 X-CRM114-Status: GOOD ( 12.54 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org kernel_text_address() and __kernel_text_address() are called in arch_stack_walk() of riscv. This results in excess amount of un-related traces when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. The situation worsens when function_graph is active, as it calls local_irq_save/restore in each function's entry/exit. This patch adds both functions to notrace, so they won't show up on the trace records. Signed-off-by: Andy Chiu --- kernel/extable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/extable.c b/kernel/extable.c index 71f482581cab..d03fa462fa8b 100644 --- a/kernel/extable.c +++ b/kernel/extable.c @@ -74,7 +74,7 @@ int notrace core_kernel_text(unsigned long addr) return 0; } -int __kernel_text_address(unsigned long addr) +int notrace __kernel_text_address(unsigned long addr) { if (kernel_text_address(addr)) return 1; @@ -91,7 +91,7 @@ int __kernel_text_address(unsigned long addr) return 0; } -int kernel_text_address(unsigned long addr) +int notrace kernel_text_address(unsigned long addr) { bool no_rcu; int ret = 1; From patchwork Thu Jun 13 07:11:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83768C27C4F for ; Thu, 13 Jun 2024 07:32:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9sVDgEINoT1B+0T6v2COOpRmRHsWG01LTPLUyYCl7pY=; b=iszsj2bp+oD3It E2CAq63sQ53mu1Vbn45w3TLZHJK37Tmz329JseDR9ksoNNYMku6DFEex/zA77Ket0pIy98WSlLYIH jBrJLzLqQLAwMLwaImzt5UbnDRQqP6421XeXpCAg2vkQB73paKGhT9EUWyWr1M15J4jCHc6JfOnX3 pnlMoH+nQE+I87/ry8GKWR5udE0ba0TCpQiQ6+S00iGVpfCe70acmdV3Dc89ZgEPR0sat5dJxavAN 5zwY/Aj8PGjcWrsAZteZh4fbiSBc7AxyUHQRddbI+3xjFhMkpesJ1yb30LDCqdzQ9kvUTInUOoIMX T4OGhrnrSAOLrACehWRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHex5-0000000FURg-4C2Y; Thu, 13 Jun 2024 07:32:56 +0000 Received: from mail-oo1-xc2a.google.com ([2607:f8b0:4864:20::c2a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHex3-0000000FUOy-1dor for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:32:55 +0000 Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-5bad66c6e27so294044eaf.3 for ; Thu, 13 Jun 2024 00:32:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263972; x=1718868772; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=KqGOcRG4MSO17L8Yoo0My/BpQLJGME/lNuiKW2jExFw=; b=jbC9gboO7QWY9qR8547jjh+ShUzll5vhObsBDrXaIbrHzIN46fg5fKGiFq7XG/SMjV RI6+uM/niyW6B2jX1f3+pAV7Z6z4wXa1CAtzCqJsLYE7sLllaj0JlVm4WKb84cDG7A44 rKlffSsj+x2iPsfnNl/PgPLnpOlrfz3WcBjjXo31vI0H3cdJgffM3a8ERZAQYuIQ0h5I b1XNCvig2hTBzYp7Z+2XH9wDUC49jvZ+tIpFaw28lKJ5yI1PvT3tc26LkORfDfnt/Ed/ P1WIj9lXTNdNiaIwYORcI2Th0762Jrftr3gnYp0ZsKU9UMDPP5izdzgYSle3LGiEUqcJ FQag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263972; x=1718868772; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KqGOcRG4MSO17L8Yoo0My/BpQLJGME/lNuiKW2jExFw=; b=eENFcivFVAjTjU64a0i0AX2Ua+aqpcytpeHK9lCwHtx7kSUQxE8EQ/IIWLtAIGCKBI mmsgel9493VKx/YjrNh5HVHP+kQz43/YkkYsgv8lqzLbqJ/vFOxYmsLZW5NRK/9R4+TP gD/H/SSh6RpxQcqtmEwMwcjN+bkXjfIzJOVChZ4iDzMxRBw9qAbys+6cut2hwbJW0gNL XXgoWiP0Xgbtjqji0JYKYRfkRSQGE8MsoDylJWBK2WP6xal//CZJ4syhocjg8m4xk9aJ dRPtyCDlu2MO/A2aY/3EAV4agdToICndsxJIb7LXmnERVRrYoc971VWl85PT6NNVaCLh 71wg== X-Forwarded-Encrypted: i=1; AJvYcCUBZDDTSEka6iWQ/0YLZam9SDDf1n6uGWwoh8QyBXL5WpPqvWv+oeo7JoGVGc5K2xhTapT7oZr3YiuYLma2RuXr+da4/4+9QdrqeE6C+xZ9 X-Gm-Message-State: AOJu0Yyet3T2k5HQ2Wh7JEKo4JquUCzxw+y88/pR314QbXG0jSZF4l9I ZOyfng58EJ1vgTJG/EwqQUM9X4PI2JBzbVKVguv3m33tvsnTmGlnZkq1s7I6apHPWqCfqEEFhgr + X-Google-Smtp-Source: AGHT+IH5DuWNaDmtQxAXDaBzm13SBjMPGjRqC4tR8s/HjfQ0ZY47iBUeJz4hi/y6Ws3uKj9ioyrEhA== X-Received: by 2002:a05:6871:b29:b0:254:b3cc:a6da with SMTP id 586e51a60fabf-25514c664afmr4230304fac.29.1718263971977; Thu, 13 Jun 2024 00:32:51 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:51 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:08 +0800 Subject: [PATCH 3/8] riscv: ftrace: support fastcc in Clang for WITH_ARGS MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-3-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Evgenii Shatokhin , Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003253_504913_C61EB3BD X-CRM114-Status: GOOD ( 10.15 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Some caller-saved registers which are not defined as function arguments in the ABI can still be passed as arguments when the kernel is compiled with Clang. As a result, we must save and restore those registers to prevent ftrace from clobbering them. - [1]: https://reviews.llvm.org/D68559 Reported-by: Evgenii Shatokhin Closes: https://lore.kernel.org/linux-riscv/7e7c7914-445d-426d-89a0-59a9199c45b1@yadro.com/ Signed-off-by: Andy Chiu Acked-by: Nathan Chancellor --- arch/riscv/include/asm/ftrace.h | 7 +++++++ arch/riscv/kernel/asm-offsets.c | 7 +++++++ arch/riscv/kernel/mcount-dyn.S | 16 ++++++++++++++-- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 9eb31a7ea0aa..5f81c53dbfd9 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -144,6 +144,13 @@ struct ftrace_regs { unsigned long a5; unsigned long a6; unsigned long a7; +#ifdef CONFIG_CC_IS_CLANG + unsigned long t2; + unsigned long t3; + unsigned long t4; + unsigned long t5; + unsigned long t6; +#endif }; }; }; diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index b09ca5f944f7..db5a26fcc9ae 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -497,6 +497,13 @@ void asm_offsets(void) DEFINE(FREGS_SP, offsetof(struct ftrace_regs, sp)); DEFINE(FREGS_S0, offsetof(struct ftrace_regs, s0)); DEFINE(FREGS_T1, offsetof(struct ftrace_regs, t1)); +#ifdef CONFIG_CC_IS_CLANG + DEFINE(FREGS_T2, offsetof(struct ftrace_regs, t2)); + DEFINE(FREGS_T3, offsetof(struct ftrace_regs, t3)); + DEFINE(FREGS_T4, offsetof(struct ftrace_regs, t4)); + DEFINE(FREGS_T5, offsetof(struct ftrace_regs, t5)); + DEFINE(FREGS_T6, offsetof(struct ftrace_regs, t6)); +#endif DEFINE(FREGS_A0, offsetof(struct ftrace_regs, a0)); DEFINE(FREGS_A1, offsetof(struct ftrace_regs, a1)); DEFINE(FREGS_A2, offsetof(struct ftrace_regs, a2)); diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index 745dd4c4a69c..e988bd26b28b 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -96,7 +96,13 @@ REG_S x8, FREGS_S0(sp) #endif REG_S x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_S x7, FREGS_T2(sp) + REG_S x28, FREGS_T3(sp) + REG_S x29, FREGS_T4(sp) + REG_S x30, FREGS_T5(sp) + REG_S x31, FREGS_T6(sp) +#endif // save the arguments REG_S x10, FREGS_A0(sp) REG_S x11, FREGS_A1(sp) @@ -115,7 +121,13 @@ REG_L x8, FREGS_S0(sp) #endif REG_L x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_L x7, FREGS_T2(sp) + REG_L x28, FREGS_T3(sp) + REG_L x29, FREGS_T4(sp) + REG_L x30, FREGS_T5(sp) + REG_L x31, FREGS_T6(sp) +#endif // restore the arguments REG_L x10, FREGS_A0(sp) REG_L x11, FREGS_A1(sp) From patchwork Thu Jun 13 07:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F4219C27C4F for ; Thu, 13 Jun 2024 07:33:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Pkp0N0MH/5c/Lvv2g55XwSj1W25Z44P/yVAh5FvhfeI=; b=PSYd48tnVFVOgr tum6D951rkhGf1BEo1N3+hXG6yOKSF4CR0mRZKYOrTwRivlcNRW2ZkqeOzhgr4lX3NRoBzAAdkw+W kLZAHhiHHOL1IF1F02YLyuCvhissw4eKr3uQZ8jRyBi3ZeKI5WCAQ8SEvAJR3D32kQYZSfsd2fvzG 4xw8dnuXp1nmn9cIByQFuVoLwAxN91OdUIuzBI3uHuVK9V0qiR458beW8cI/Dz6z7g/C17sXIk9l1 O/fOc8SLgmdbcJc1hUYDIcJGMeEzFMJzoOdq/eqHkgTO9V6QNEQHDERd3F52COZ+NVFB7QhRvpbsU 4o0VMrdQ39YSKAzQsTuQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexA-0000000FUUx-1vzz; Thu, 13 Jun 2024 07:33:00 +0000 Received: from mail-oo1-xc2b.google.com ([2607:f8b0:4864:20::c2b]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHex7-0000000FUS7-1ShA for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:32:59 +0000 Received: by mail-oo1-xc2b.google.com with SMTP id 006d021491bc7-5ba1f60eb96so431525eaf.3 for ; Thu, 13 Jun 2024 00:32:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263976; x=1718868776; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=LTElu2O7G4IRv2D6Es7gQcAecPf3onDHBKt+B4ik+4k=; b=TAxB7fERls9EH3uA55bIclufMClBa0KccTjd4lD4EnknIRn0W4BVFOTfgiWLWcxJui DLpkqxNAeTyRB+G50UhZFKYRe6KkSBfCtzlOcjEUWLY4L9d1d+0snN0UmBxsU68WAoAG WLjh6Gwo8bM26CxLo2Oc94YawL2pLfGg8hYwSjVbzbzYId9fvrf/qseNAcsXWBjo7wvb K8kyTl6XtrrkdKjCZSttxepBufx9elPo63RaK1PdNnHiyZMh+8c7alXWu0/NU/4IYAID Kbv1xYqE3wyBW4YksIJVH0QWL8lg1pCdHrgOZ0cVh16e8b0GhjsapbBPLLN/Rzntlj/h HwfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263976; x=1718868776; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LTElu2O7G4IRv2D6Es7gQcAecPf3onDHBKt+B4ik+4k=; b=OYYZ0dcFbbEDlmxwgef0jLLoTM3bG+LHznvee6huHYuCSkWNzrvY6W9x7XLyWN+Hnc jlZrs1vpxB9cJc5UmldMLZ1r53jc1goeKCVYPUQRoG3JSLrGMraOYwbHmUYIG07MOPJ3 HoM8GYQKO1VJ4sbNVHDxflRJayo59zA6Z9NZtV1yhW5q6XIU2iWUQnTAXfX+upVGtv8x 0pmyG5i+M32OzsKXW9rAee+XTyGxNQCfWVVSgfytX+/Is9Wx77Tal/ii/AI4szvlipO7 4lOEKjiBvC962a9pMv0qNdiYUVBG00xEN+OQvNfsuntDhJ0/PJfw58aW1akoqENrR0/9 TL2w== X-Forwarded-Encrypted: i=1; AJvYcCVa695R0hOk1lqmdzWhncnPfusuDAexG9XhUoC+WQi5yzP0A6J/Gojpg0d2hI1lym+HfGDmJqBfnNRxCs1iwALvHyrTbRB38VGrWhDBf+XN X-Gm-Message-State: AOJu0YxFx2b0YNGyovqlDNkSnbcUPElxjHIumoV+14CAnGdFiV39Ig9x 79AT9pqcYuF9fxEfbJi8JPvklNYnbREFfY1LKKVisBgFgCFvW9nnu8R5hSKlIhiprwkGcEMM5qy T X-Google-Smtp-Source: AGHT+IGdJsIqWxldlAuacUU8l0dK4I8dOhgAshBKtsVS5QR/psuALrZ1MwA3KVg0EXK2RXnVq6OGZQ== X-Received: by 2002:a05:6870:c087:b0:254:a9bd:70b2 with SMTP id 586e51a60fabf-2551501e0bfmr4485943fac.59.1718263975852; Thu, 13 Jun 2024 00:32:55 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:55 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:09 +0800 Subject: [PATCH 4/8] riscv: ftrace: align patchable functions to 4 Byte boundary MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-4-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Evgenii Shatokhin , Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003257_427128_573777D0 X-CRM114-Status: GOOD ( 11.96 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org We are changing ftrace code patching in order to remove dependency from stop_machine() and enable kernel preemption. This requires us to align functions entry at a 4-B align address. However, -falign-functions on older versions of GCC alone was not strong enoungh to align all functions. In fact, cold functions are not aligned after turning on optimizations. We consider this is a bug in GCC and turn off guess-branch-probility as a workaround to align all functions. GCC bug id: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88345 The option -fmin-function-alignment is able to align all functions properly on newer versions of gcc. So, we add a cc-option to test if the toolchain supports it. Suggested-by: Evgenii Shatokhin Signed-off-by: Andy Chiu --- arch/riscv/Kconfig | 1 + arch/riscv/Makefile | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..80b8d48e1e46 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -203,6 +203,7 @@ config CLANG_SUPPORTS_DYNAMIC_FTRACE config GCC_SUPPORTS_DYNAMIC_FTRACE def_bool CC_IS_GCC depends on $(cc-option,-fpatchable-function-entry=8) + depends on $(cc-option,-fmin-function-alignment=4) || !RISCV_ISA_C config HAVE_SHADOW_CALL_STACK def_bool $(cc-option,-fsanitize=shadow-call-stack) diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 06de9d365088..74628ad8dcf8 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -14,8 +14,13 @@ endif ifeq ($(CONFIG_DYNAMIC_FTRACE),y) LDFLAGS_vmlinux += --no-relax KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY +ifeq ($(CONFIG_CC_IS_CLANG),y) + cflags_ftrace_align := -falign-functions=4 +else + cflags_ftrace_align := -fmin-function-alignment=4 +endif ifeq ($(CONFIG_RISCV_ISA_C),y) - CC_FLAGS_FTRACE := -fpatchable-function-entry=4 + CC_FLAGS_FTRACE := -fpatchable-function-entry=4 $(cflags_ftrace_align) else CC_FLAGS_FTRACE := -fpatchable-function-entry=2 endif From patchwork Thu Jun 13 07:11:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56A2DC27C4F for ; Thu, 13 Jun 2024 07:33:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pcnzZEFjfCqeGyPW9NkGrDrJbRlfsmZMVsyeFWZtbxA=; b=mQqED1PYhlRjK2 RH863Xrlm06l0YtM2/kU/21AohmY/YHK3UQUJkjeXJ0eJa4FC7Yz4+zg468bjVPPSCGqXhVoq+G1q z6Z28xXLi41DgTXK+O9RkQh/GJPiyfHIhlF0m5/xBMlwB5DNCXAx80To3Si9NfcF0u85NDLar/o1S kkDGHpdoQ/A14O8zwhtagEjUGr2lbom7E20MSeGJTamXH3k13LcbNYKNdo6KMpfEeMM8vTod3/gP3 BNsYLa9AGAlF97FS/wAVmAdLtq1nYEs7itZ7btcYjRwOuA04Cz/E5FYUXff17AKalPdH1rfTd8a9u wN8AXB3KkYsVZx9qtc6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexE-0000000FUXL-2t0l; Thu, 13 Jun 2024 07:33:04 +0000 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexB-0000000FUUz-1q4l for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:33:03 +0000 Received: by mail-oi1-x233.google.com with SMTP id 5614622812f47-3d1bc6e5f01so351997b6e.0 for ; Thu, 13 Jun 2024 00:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263980; x=1718868780; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xpwccXm6VSga0glYezh9Lcoo9mdtNlokYTZmh5MRn+k=; b=f5C7yPZ2xXIF6xXx7cQrIy2uLyjIiDIgDtES2Z+fnPKxnsZKQ0nZcsSFmL/n4VSnhR w7s4HE3ZvfP7U2I2+WI01zdIEcQ6RpZ74YS0FmteSnNSjoHWrZYyZuxbRqAJfTZ0No7B 7ataJTogw1vgNG6iCjEG06tjqCDsEHwcwFsw2G7+wwwfoPoxxLz8xT4SnB2bUZzjQGNK N1bpM0a6JkNJ7wv7kXNwWMdKqtba6rzX+nSsULwyqaC7kDEPPZP+4ioIm++mOQZLRRGf aD1QmhYRYmidPjk0My8OENv/Zi6nBLa0JBm93ZxT36x1S94v+Sgy1R+BGmrQBfMzCHfg vr7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263980; x=1718868780; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xpwccXm6VSga0glYezh9Lcoo9mdtNlokYTZmh5MRn+k=; b=OCeMDIAji1OeSKuc8u7uIFLg2Q2j2HF8CgTw16bYJkW9s/EEVMMutdmRNXxU90nJIG 32QMWqXUQynzrHnw2SdgKmC8VXFug3cUoc7hYZ2WYGCZthHpOavh/66BiC9Mz8VwR3qj dkSgIMuOmsL7UC4+bTpC9K5uyb/ROhpuhim3+1TrnpfAWN6gTKiradxc5R/Lvi5xb2wB 4K2E/AVyriRY2Ta95Qy7mDPbXFgR3rsl2xVoVm+eq5HtIwi8JGUyQUtJjSkomNE67Z4P nHhJOBLoU9PuSzZvZ/duwrsOQHep4S26qV85SXQD51MBJAEzq+69gtR2EI8mM7xakv09 ieFA== X-Forwarded-Encrypted: i=1; AJvYcCXK/FUgcEvFWopA/wUSb/J1j1edYUcnvZRmqOkyNm6v2bwXcnka05gvmGMyjyYrGT+MH7mqXF7devdeCUwQw0TZ94FnWxrEee+LRYFrLHHr X-Gm-Message-State: AOJu0YzLiXUIAerfKXKbxVXr/q/Wuzm9C/f7k9FF2DYnBvoi9ZKwlnWC 65mPoYmNx+wMgqWM5QV1mRXTJTxJh2TRyghW75YM+YAgGy2cpHGA0YjQDBdZsMw= X-Google-Smtp-Source: AGHT+IEzkbVgn0G1G4t0BzOnoC4MTfTDVP2Z/BlOjvozamyY5nqFEfSfIU6N3LGNo64kZ62JzW5oRQ== X-Received: by 2002:a05:6808:1a24:b0:3d2:4a08:2c99 with SMTP id 5614622812f47-3d24a0832b0mr183168b6e.23.1718263979694; Thu, 13 Jun 2024 00:32:59 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:59 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:10 +0800 Subject: [PATCH 5/8] riscv: ftrace: prepare ftrace for atomic code patching MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-5-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003301_609911_964F9309 X-CRM114-Status: GOOD ( 20.92 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org We use an AUIPC+JALR pair to jump into a ftrace trampoline. Since instruction fetch can break down to 4 byte at a time, it is impossible to update two instructions without a race. In order to mitigate it, we initialize the patchable entry to AUIPC + NOP4. Then, the run-time code patching can change NOP4 to JALR to eable/disable ftrcae from a function. This limits the reach of each ftrace entry to +-2KB displacing from ftrace_caller. Starting from the trampoline, we add a level of indirection for it to reach ftrace caller target. Now, it loads the target address from a memory location, then perform the jump. This enable the kernel to update the target atomically. The ordering of reading/updating the targert address should be guarded by generic ftrace code, where it sends smp_rmb ipi. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/ftrace.h | 4 +++ arch/riscv/kernel/ftrace.c | 80 ++++++++++++++++++++++++++--------------- arch/riscv/kernel/mcount-dyn.S | 9 +++-- 3 files changed, 62 insertions(+), 31 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 5f81c53dbfd9..7199383f8c02 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -81,6 +81,7 @@ struct dyn_arch_ftrace { #define JALR_T0 (0x000282e7) #define AUIPC_T0 (0x00000297) #define NOP4 (0x00000013) +#define JALR_RANGE (JALR_SIGN_MASK - 1) #define to_jalr_t0(offset) \ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0) @@ -118,6 +119,9 @@ do { \ * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here. */ #define MCOUNT_INSN_SIZE 8 +#define MCOUNT_AUIPC_SIZE 4 +#define MCOUNT_JALR_SIZE 4 +#define MCOUNT_NOP4_SIZE 4 #ifndef __ASSEMBLY__ struct dyn_ftrace; diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index 87cbd86576b2..f3b09f2d3ecc 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -64,42 +64,64 @@ static int ftrace_check_current_call(unsigned long hook_pos, return 0; } -static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, - bool enable, bool ra) +static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, bool validate) { unsigned int call[2]; - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int replaced[2]; + + make_call_t0(hook_pos, target, call); - if (ra) - make_call_ra(hook_pos, target, call); - else - make_call_t0(hook_pos, target, call); + if (validate) { + /* + * Read the text we want to modify; + * return must be -EFAULT on read error + */ + if (copy_from_kernel_nofault(replaced, (void *)hook_pos, + MCOUNT_INSN_SIZE)) + return -EFAULT; + + if (replaced[0] != call[0]) { + pr_err("%p: expected (%08x) but got (%08x)\n", + (void *)hook_pos, call[0], replaced[0]); + return -EINVAL; + } + } - /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ - if (patch_insn_write((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) + /* Replace the jalr at once. Return -EPERM on write error. */ + if (patch_insn_write((void *)(hook_pos + MCOUNT_AUIPC_SIZE), call + 1, MCOUNT_JALR_SIZE)) return -EPERM; return 0; } -int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +static int __ftrace_modify_call_site(ftrace_func_t *hook_pos, ftrace_func_t target, bool enable) { - unsigned int call[2]; + ftrace_func_t call = target; + ftrace_func_t nops = &ftrace_stub; - make_call_t0(rec->ip, addr, call); - - if (patch_insn_write((void *)rec->ip, call, MCOUNT_INSN_SIZE)) - return -EPERM; + WRITE_ONCE(*hook_pos, enable ? call : nops); return 0; } +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +{ + unsigned long distance, orig_addr; + + orig_addr = (unsigned long)&ftrace_caller; + distance = addr > orig_addr ? addr - orig_addr : orig_addr - addr; + if (distance > JALR_RANGE) + return -EINVAL; + + return __ftrace_modify_call(rec->ip, addr, false); +} + int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int nops[1] = {NOP4}; - if (patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE)) + if (patch_insn_write((void *)(rec->ip + MCOUNT_AUIPC_SIZE), nops, MCOUNT_NOP4_SIZE)) return -EPERM; return 0; @@ -114,10 +136,14 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, */ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) { + unsigned int nops[2]; int out; + make_call_t0(rec->ip, &ftrace_caller, nops); + nops[1] = NOP4; + mutex_lock(&text_mutex); - out = ftrace_make_nop(mod, rec, MCOUNT_ADDR); + out = patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE); mutex_unlock(&text_mutex); if (!mod) @@ -126,12 +152,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) return out; } +ftrace_func_t ftrace_call_dest = ftrace_stub; int ftrace_update_ftrace_func(ftrace_func_t func) { - int ret = __ftrace_modify_call((unsigned long)&ftrace_call, - (unsigned long)func, true, true); - - return ret; + return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } struct ftrace_modify_param { @@ -185,7 +209,7 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, if (ret) return ret; - return __ftrace_modify_call(caller, addr, true, false); + return __ftrace_modify_call(caller, addr, true); } #endif @@ -220,17 +244,17 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, prepare_ftrace_return(&fregs->ra, ip, fregs->s0); } #else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ -extern void ftrace_graph_call(void); +ftrace_func_t ftrace_graph_call_dest = ftrace_stub; int ftrace_enable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, true, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, true); } int ftrace_disable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, false, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, false); } #endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* CONFIG_DYNAMIC_FTRACE */ diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index e988bd26b28b..bc06e8ab81cf 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -162,7 +162,8 @@ SYM_FUNC_START(ftrace_caller) mv a3, sp SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) #ifdef CONFIG_FUNCTION_GRAPH_TRACER addi a0, sp, ABI_RA @@ -172,7 +173,8 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) mv a2, s0 #endif SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_graph_call_dest + jalr 0(ra) #endif RESTORE_ABI jr t0 @@ -185,7 +187,8 @@ SYM_FUNC_START(ftrace_caller) PREPARE_ARGS SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) RESTORE_ABI_REGS bnez t1, .Ldirect From patchwork Thu Jun 13 07:11:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3916C27C79 for ; Thu, 13 Jun 2024 07:33:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dNWXxEjzW8esYR0OKrwJxp9APhFFWl0OApBfC5/1jGc=; b=juft4iPLYq+loL N/U9q+1WzSLznkQt/YQ1E8KwHY7F0Nutoe8AhFvPpHvTHzbF0oUzeq3yTRJWaMcGGcEy0vjU7OW/w BP1aL1x088EVqeIuJvELr+dzxiBxvataJtwEwsGpaJ4WnrrS4j7kEAoaMECbkpGeLYP8q2CT1Cn9R 37aToocB/7XCqokQ++0wmRsYQF9eDN9O/a60folKilXzQk1BndzvErvwumB4jZ+gdCivqIrXX2Y85 FkRBCLv7+s/O5Ld1zEEnMZJmCDG6KhvDCfGd3U175Tyw086uZ+y3IJsFspO51t2S8mfwBBAUEWtL8 30addTkitl7Z1k3gv/Xg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexJ-0000000FUZq-04kn; Thu, 13 Jun 2024 07:33:09 +0000 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexF-0000000FUXG-0dWf for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:33:06 +0000 Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3d227b1f4f0so332943b6e.3 for ; Thu, 13 Jun 2024 00:33:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263984; x=1718868784; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=wBQdisfSIcfLCxaUXLlDTwMoYdzfD8LhoeZl0/Tj/fA=; b=THq8Q48G8FK3MzohjaUsKMljBTKanTMJOlKnjODpzJBqhAEUnm8I/ee83ZgFlCboNp PeQtD15fwu128R1OmnLyD/6N9x0xENOxuD0ZxtOBZ0IUf3Er7ji8KaFB9SEyVKEFHB4Q DShrU4kltQ8SkOsiWkRZKdXPd7vj6wpTo06T/xuSyidDqHE4lNT46EKPSHUWAKxp/q2s hCKL6tejXzfj7T2EjTT6w8JG8C6N8w8YHVNM4u+EmF/JRKVMIKmPkOxu90VxLQDDDFoM tVQBie1yQg538rh1wfNfJTtuba3NmkYf1bmA1wZb+/e/1WQbqNpcOeFEMFw+R3xqr/HN DeKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263984; x=1718868784; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wBQdisfSIcfLCxaUXLlDTwMoYdzfD8LhoeZl0/Tj/fA=; b=JFLlpJw9zsF+2ABuIqQUn9f6QBf+46VwLnV1cvMZOLAHeUUJBdOX50xgZ0NlJlvAGN D3HdQxZjxOKZXVM+uabIyngGePZn/KI7Ut9zx9oCJJKsuDfLepnowZmYMDgvbiWjjxjl jclHXpjUXZOdVQ5Qpqw6geESl+hMCEYj8D4w0eRWYOPvYLwKsaMJ+bM3Bn3KbkxTb7Ps 6tILw85H4CS6r+gxH8Z/JJwJ2TfInjqkjH+qznUVE0kapGLU5MI9A3fRUfBeveXrhmbq G+GbsvpZoeuZJc/2zxRlA+FL/Ue6vIiYGXFnn63Xoz4eFI632aZwQImkKOvudYdNuGGK PvLg== X-Forwarded-Encrypted: i=1; AJvYcCUAZIEwFJc/rPRoZqpWfktNxaPVihhnDQyDZr0gltyDMsiNRtukLOPCbsjvbu3ja1qbByxPHWXu9k2BQx2e3mMIpeGKBjGVBmfRPLUvFZ/q X-Gm-Message-State: AOJu0YynE3u/+Z1AcUEqkxQjCqs8EtiXv3TeTN+pdGpb7IjHDQc2G4LB Dx1Wz8gYijvpTOW1vLMx5yT7C4H4lCvwtPuwpmyOQl5oh5E8bJwqjmiLrSeHWKg= X-Google-Smtp-Source: AGHT+IHI/57UF4ERYhNsT8IgBFHVZFnNkvkL7rNY4vjoxvdn096FWUD3qDYx3RYqModhDDSYikhCvg== X-Received: by 2002:a05:6870:4412:b0:254:6eb5:ab2f with SMTP id 586e51a60fabf-25514afc016mr4471125fac.6.1718263983791; Thu, 13 Jun 2024 00:33:03 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:03 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:11 +0800 Subject: [PATCH 6/8] riscv: ftrace: do not use stop_machine to update code MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-6-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003305_400316_D49C7D9C X-CRM114-Status: GOOD ( 12.95 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now it is safe to remove dependency from stop_machine() for us to patch code in ftrace. Signed-off-by: Andy Chiu --- arch/riscv/kernel/ftrace.c | 53 ++++------------------------------------------ 1 file changed, 4 insertions(+), 49 deletions(-) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index f3b09f2d3ecc..9a421e151b1d 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -13,23 +13,13 @@ #include #ifdef CONFIG_DYNAMIC_FTRACE -void ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +void arch_ftrace_update_code(int command) { mutex_lock(&text_mutex); - - /* - * The code sequences we use for ftrace can't be patched while the - * kernel is running, so we need to use stop_machine() to modify them - * for now. This doesn't play nice with text_mutex, we use this flag - * to elide the check. - */ - riscv_patch_in_stop_machine = true; -} - -void ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) -{ - riscv_patch_in_stop_machine = false; + command |= FTRACE_MAY_SLEEP; + ftrace_modify_all_code(command); mutex_unlock(&text_mutex); + flush_icache_all(); } static int ftrace_check_current_call(unsigned long hook_pos, @@ -158,41 +148,6 @@ int ftrace_update_ftrace_func(ftrace_func_t func) return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } -struct ftrace_modify_param { - int command; - atomic_t cpu_count; -}; - -static int __ftrace_modify_code(void *data) -{ - struct ftrace_modify_param *param = data; - - if (atomic_inc_return(¶m->cpu_count) == num_online_cpus()) { - ftrace_modify_all_code(param->command); - /* - * Make sure the patching store is effective *before* we - * increment the counter which releases all waiting CPUs - * by using the release variant of atomic increment. The - * release pairs with the call to local_flush_icache_all() - * on the waiting CPU. - */ - atomic_inc_return_release(¶m->cpu_count); - } else { - while (atomic_read(¶m->cpu_count) <= num_online_cpus()) - cpu_relax(); - } - - local_flush_icache_all(); - - return 0; -} - -void arch_ftrace_update_code(int command) -{ - struct ftrace_modify_param param = { command, ATOMIC_INIT(0) }; - - stop_machine(__ftrace_modify_code, ¶m, cpu_online_mask); -} #endif #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS From patchwork Thu Jun 13 07:11:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4B76C27C79 for ; Thu, 13 Jun 2024 07:33:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E31zw1nDWIcdMf8zi68b4EzmEarSC61pYX9RKuAoTP4=; b=eLK6Z5njh4J9hR AnfvdyW5fW7/MSdkX6o7xAymC0wTNDd9cwpzk8l6glxXiUbTxWHc89C6jMvOS3p5HuQyoRJYVQp2m k9ccBOd4FeGECRoZftOWWcMItSK+uxG3udCrA1lWFn1K0yu/F6FQ8OwpCXQgjBgmG+wq8FrO5uzmd Od3CX7JZKgaNzL9G/u2ykmZ7+hO4XZeMNJfc0ROlRKh5eHX8OJQBaHZSR/b0x7EHCtVqeWEmBnoeE Y6j4ky/NJ/zQaB0ncAHFBnRrL0eoUfuDMJvNS5sG9/qCAm8iSs/jCO70yzP0EM5geU53UWDQAF9dt q8AtqgMQnsCm7U1oo8cg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexN-0000000FUd7-1qmK; Thu, 13 Jun 2024 07:33:13 +0000 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexJ-0000000FUZh-2lwC for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:33:11 +0000 Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3d226c5a157so381990b6e.2 for ; Thu, 13 Jun 2024 00:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263988; x=1718868788; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=TeZhGtu+DoZzH37TzHgHW9NZu5mW4WfgIzkfv6QNhDQ=; b=GnbsMc9grot+5Jp0JDhM1cixBhfS+jGsImEXQNS2xfyjmuNXrGu8+hTatSKa8Hn9aD /Ev3GgceWz8NJFOnN9YSqAirLj3OAO/0MOHTLO6+j6FdrWLQlOoq69l5XP7Y9fG3eFwX 0NqxDDoLXoX1nVlhZjWWFRnGc+K8f/4+nKfvG3tLGr9m8wlicZBpPH7TDGsDbr//JuNY OfTb7U4jcz+Y74UO4FJIYvEAXLw8tVpUfTntNupYLFH2JA59lHPBxuPX3JOtuHOt7T4v t/ODigyh/fF7n1CZre1GNSQKKS9jSZZ5/xMCivL6LimzaMav2UWtDbJZP2kKWw38n+5Y Qmeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263988; x=1718868788; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TeZhGtu+DoZzH37TzHgHW9NZu5mW4WfgIzkfv6QNhDQ=; b=ZnnZmAllPkHLnGuUSX8zFh4aQA9+c6pAG6TbttxUJjK3r68/562DAgEmJvUiZquDg2 7ZLaAbFnr5r0QMVbl6PVUUuv4iyYYoo3lVkF8WIOdykqLwMSt0T9fNgx6pcvdwxQaUbh XaCCHhZStClnBIW3XjF8a4HycODng28JdPdJSQxx2gMvdewj924k6RngKQ0dGR5RuBog iub051o0IXXzI4ExNh8+sDQ2qgb612Iq7Ur4aWiHKeXU3seO3FxAj3VN084G1GV9uFw0 joBLTCkr68ihGuf9U9MR9aCwQHkImOy5L0ceK0pQ4kRuDi53s0fLKrIxxKxMaN+T1uBU Yt7g== X-Forwarded-Encrypted: i=1; AJvYcCWiA3JVbs2GWRXnFIC2V3MjzMEiGtW56W1ycuynv6bGcM0ZI22N2hiJnoHU2EIzwH37mq2Hu+Zl+Cs6jgX2als0AwBAh+9/bcdiMqJ1Blx2 X-Gm-Message-State: AOJu0Yw8U15QJGA7cTToCN9U7w8k+NSfqtdXtDq7L3b23Vmfc5MHTeVQ s0ofR/ogz8lwLJH6a26Ml4ZQQ5GOiehqeAhqDgOUQGrqr9TW8jEpjAgU0FDmgqw= X-Google-Smtp-Source: AGHT+IG/QcH0femSCZzF6g5xhDc4dqSZTxWrgAQgosN/9/estM4RRUpCw1U0PcaMW2D86qVbhMaGwA== X-Received: by 2002:a05:6808:2199:b0:3d2:1c8f:be0c with SMTP id 5614622812f47-3d23e045f11mr5371608b6e.30.1718263988155; Thu, 13 Jun 2024 00:33:08 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:07 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:12 +0800 Subject: [PATCH 7/8] riscv: vector: Support calling schedule() for preemptible Vector MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-7-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003309_778314_66457B4C X-CRM114-Status: GOOD ( 14.97 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Each function entry implies a call to ftrace infrastructure. And it may call into schedule in some cases. So, it is possible for preemptible kernel-mode Vector to implicitly call into schedule. Since all V-regs are caller-saved, it is possible to drop all V context when a thread voluntarily call schedule(). Besides, we currently don't pass argument through vector register, so we don't have to save/restore V-regs in ftrace trampoline. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/processor.h | 5 +++++ arch/riscv/include/asm/vector.h | 22 +++++++++++++++++++--- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 68c3432dc6ea..02598e168659 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -95,6 +95,10 @@ struct pt_regs; * Thus, the task does not own preempt_v. Any use of Vector will have to * save preempt_v, if dirty, and fallback to non-preemptible kernel-mode * Vector. + * - bit 29: The thread voluntarily calls schedule() while holding an active + * preempt_v. All preempt_v context should be dropped in such case because + * V-regs are caller-saved. Only sstatus.VS=ON is persisted across a + * schedule() call. * - bit 30: The in-kernel preempt_v context is saved, and requries to be * restored when returning to the context that owns the preempt_v. * - bit 31: The in-kernel preempt_v context is dirty, as signaled by the @@ -109,6 +113,7 @@ struct pt_regs; #define RISCV_PREEMPT_V 0x00000100 #define RISCV_PREEMPT_V_DIRTY 0x80000000 #define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000 +#define RISCV_PREEMPT_V_IN_SCHEDULE 0x20000000 /* CPU-specific state of a task */ struct thread_struct { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 731dcd0ed4de..50693cffbe78 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -75,6 +75,11 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline bool riscv_v_is_on(void) +{ + return !!(csr_read(CSR_SSTATUS) & SR_VS); +} + static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) { asm volatile ( @@ -243,6 +248,11 @@ static inline void __switch_to_vector(struct task_struct *prev, struct pt_regs *regs; if (riscv_preempt_v_started(prev)) { + if (riscv_v_is_on()) { + WARN_ON(prev->thread.riscv_v_flags & RISCV_V_CTX_DEPTH_MASK); + riscv_v_disable(); + prev->thread.riscv_v_flags |= RISCV_PREEMPT_V_IN_SCHEDULE; + } if (riscv_preempt_v_dirty(prev)) { __riscv_v_vstate_save(&prev->thread.kernel_vstate, prev->thread.kernel_vstate.datap); @@ -253,10 +263,16 @@ static inline void __switch_to_vector(struct task_struct *prev, riscv_v_vstate_save(&prev->thread.vstate, regs); } - if (riscv_preempt_v_started(next)) - riscv_preempt_v_set_restore(next); - else + if (riscv_preempt_v_started(next)) { + if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) { + next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE; + riscv_v_enable(); + } else { + riscv_preempt_v_set_restore(next); + } + } else { riscv_v_vstate_set_restore(next, task_pt_regs(next)); + } } void riscv_v_vstate_ctrl_init(struct task_struct *tsk); From patchwork Thu Jun 13 07:11:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8665C27C4F for ; Thu, 13 Jun 2024 07:33:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0OZL4b+DKqqCKPLBx6y+VvjNyUInX/0Ji6F2A/UGjMU=; b=jUP2jnhc9GM0Om nyKG/FVSpLEmYu0SLIpQSvB0blAEDlxN6QkSDSJ2FpvZ+53r0sk67Y+8wJQGMs045vnPkBDhkcWdc lM2egMkHxJAOxGziJX7ouniM5ZEX7ivvwJ11JmyfP1L+LNIq86LFXE6PtJVXcn+4GZUxbInxqtTQr SwrAzBm11SWQzV3trOX1rkFSNkaAg0VpWKU6JbJauT0fYlqnKvpSJKvrIKHKikocp7e7j8x5PNV92 rCLWJ7sdj6xDhwaXA1lQf3gUqLiLmC2YT29SIygJsgLHvMBvlRc0q7+cgcrB6HlejKPJ3v581NzDY 0QqAHZXq6ED9p9dcC1Dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexQ-0000000FUg5-2i55; Thu, 13 Jun 2024 07:33:16 +0000 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHexN-0000000FUcl-2TNy for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 07:33:15 +0000 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-704313fa830so513751b3a.3 for ; Thu, 13 Jun 2024 00:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263992; x=1718868792; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=RlJIkXaRaXYEOo0FtAy4w5B+d0jhj99dMDrjOpWfyi4=; b=XtgteywTdO3XQELyIBELowQHcElUaMuq+jlgevkqoqwhIT18AkXgruu3wim0jLDDAL tit//EXYFkZVgCFyK2HCXvY1bqDVSI+1zRlE+LTr5y3YL0uPSge8J9tZzqlLJfsD6Uf6 zLmvCAHcXmuYPxUggFPhUxHMP/2LV++7ZIWFz5l1Ht3jqmyxDH/bwT7M96iZ3DMoR9T9 6Gmo26C09cAlCho96bq3fD3LYYiDcQPPjmFM/9h663xFzTkNhfmN+ilVKm2uwv+jMJJd A+LcjBO0WQNvKet4yR/GnN7s6yGvADQ+5IWYEqhkBXsUxINUMXQYolt1eWBK8gv2Hk+X XgJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263992; x=1718868792; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RlJIkXaRaXYEOo0FtAy4w5B+d0jhj99dMDrjOpWfyi4=; b=iYGQSn2vDAWKB7QmFPJKb8t/x63rllvEb/XOIbgBpNfzHodFcDJH5CqR2RVVz0scyZ kwy9Wk+2+Qt1v3AJxFlH96Wb1PxsP4b3U/Tw7Yp9RMRCNAfOd3EjnPZCTB8WZilZGNrd dtflOa8aMyqgENlwQBzysSvn6NVT2oxxAV5n5j6UPy6q7Y7KWbfFsmKQCK/b676ckufM FwA31ZnTROyEB4+Qv1Qd0fYtc8Wqdi0kqlCQm8z9h19u2qZvz9ZbbViNrkrqMy23MZY/ +/qh8TDHE9lGGzLMHhJxFEhglUUOKc4XVAV6J9frQN1Cm5uvHLp/yEnWusOxFaBF5AMO TxeA== X-Forwarded-Encrypted: i=1; AJvYcCVwZr6cK+xm9flH6gU5/j+xnYetpoD/zmK8zFTrtoIHGUVZMhUYhmOhwfW+t4giMcRKj4iWfGstu8TOGvAtQ62SwXkAb2mAbTThh4+DtgvE X-Gm-Message-State: AOJu0YzQqrHVhALwz5SdukN8lv5q9dR26kMeIjebRTrzvZOSJ382/dT+ ANSNxtKCy/wdn0nMeoWwsWuL9mNqmAIjK4JkLroXAkEu2Q+8fHudC3gVXYia+dI= X-Google-Smtp-Source: AGHT+IEyM0ONI7ICTx6DjS1SeAdMKnjMDIyvhgY/AOB26Jk18oYzU1QW75t2bRm257+1EVhnYwVCTA== X-Received: by 2002:a05:6a20:3d88:b0:1b2:a889:f7da with SMTP id adf61e73a8af0-1b8a9c887fdmr4702279637.55.1718263991983; Thu, 13 Jun 2024 00:33:11 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:11 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:13 +0800 Subject: [PATCH 8/8] riscv: ftrace: support PREEMPT MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-8-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_003314_100498_785D2DAE X-CRM114-Status: UNSURE ( 9.86 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Now, we can safely enable dynamic ftrace with kernel preemption. Signed-off-by: Andy Chiu --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 80b8d48e1e46..c1493ee1b8cd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -133,7 +133,7 @@ config RISCV select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_GRAPH_RETVAL if HAVE_FUNCTION_GRAPH_TRACER - select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION + select HAVE_FUNCTION_TRACER if !XIP_KERNEL select HAVE_EBPF_JIT if MMU select HAVE_GUP_FAST if MMU select HAVE_FUNCTION_ARG_ACCESS_API