From patchwork Thu Jun 13 07:11:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696241 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9396B13C670 for ; Thu, 13 Jun 2024 07:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263966; cv=none; b=f7E5H72OCVMHVRpHFD6JcO36sir2ySGAYJxFPUdVpJJkR/+D/Dv0hYD7A8w3Z4ILDxYelasY7RaQv0DEMzx7FBEs9AcV3xY55pHaooQJMTyJ8LNLSr51oAji3rBrTgblz9nHputYu3Mh2u5LY5Bvw9dtzPH4QDR/R9mAI9qqcgQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263966; c=relaxed/simple; bh=JiXCsJYbgQAz9+R6zTWp/RApNEHB55TEiq7HnF4rVDs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pmX0ERsVz+E93Y7375G9uy4PzD5shQp4nZ0ILBWpbHovRPXjkcNjRs7b60BjHVj/EPnN2u0N7jxOa4pbRUcwjdVRnKhUJ5wl6p0+tLrUGUvg83TX6dQebhKueUilzQqbLN7s9e5g1MyJ35rp2kUZLdhGs8C9UZ7PjJw5MRRts6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=CDmnzgHr; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="CDmnzgHr" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-705959a2dfbso497628b3a.1 for ; Thu, 13 Jun 2024 00:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263964; x=1718868764; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=90j1lP2iehIU9E2euRd9xJoPIJsKCYmj2Va6/Ket5KI=; b=CDmnzgHrN8I69+1iXs0n+uGJ4tABm3ifoX9XjxQPv/VsPfMX3e9qkxQ9tsrAoSfxvH 1gNjMvnoHWg7g2fpmNDSjf3PnS7G0jxLEVnDOpTe8/RIzFepnyNlyaLaTdHL9mOpyWRk 4x3wAJcrmpR/sI+mXb4HIp8xbD5Wdr9awdaGZJlkf24mSAvOPonAMggz5ceGrBNR0BYS xksIhxbkSZCUlU+4kl9dN+sXxBoRbEJkbR4tkP/NPvrQxXTNAHv1gCjZAB/IYG5c/EfH XHvKh1KtAUBzf4m8gIcC8rN2HMryp1hpNfSJeTOWiN9sL4KexLpwhAd4z/F/tTzuT0xq TrJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263964; x=1718868764; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=90j1lP2iehIU9E2euRd9xJoPIJsKCYmj2Va6/Ket5KI=; b=N2xfuLVqg1qw/fvp/MNsRmbkN4SRAtR1Ds6rqxBDdoSSKUZRsLmykjiTVrvuNbwNuj AFXigVuSZO+uoBQh5hb/CfRAGfd6quBncnoieqwqEbz8honfQHLkkA1ocsMwnvW5sBSx GamzJKVpo0c/xsT2s7tWR/TdRJmhqA7K7kaxF8DebUypyRHuBAXQaiR9USgedG+xWSkn kbOOr+a49dTA+OUTgzFe+h88kE2nXdiGvBu426LoLv2CDJSu622cF3F7Do+pAKGXJl5C T/eWERdO2VYENrM5zoli64VX3qet6doCV6c9aIW/mwiXwweX5oQnJHoGWC9nYXi1RygN NBKw== X-Forwarded-Encrypted: i=1; AJvYcCWmpwTl4wUtPttu1Ls5KCOSGkrSjv4EHm/j5iWfUBYbq0WmPBvaHo6VIPBCccYbkQWJqyYy+7ru7mMTLRxrtgPrtWK6Kt7zpyTFInkKcLCOK7vx X-Gm-Message-State: AOJu0Yy7EFhXZOZGORKq1+qcZyQLYDe8dKcC9KqItHcWHgAhXd+Z1PSt 3KzkyrEcQm3hHO8Cz2J6F7DM5zEbpGEcED4BE1jzqhwwhGqBi0g+GttkQv/8gQ0= X-Google-Smtp-Source: AGHT+IGTwQ5EB55toThK3E4Aeh/60oGzOVaQMTb+lJV8UkAi/aBjTebo3WO7lB+gn84w0BlxoTDk4Q== X-Received: by 2002:a05:6a00:2d84:b0:705:a18a:686a with SMTP id d2e1a72fcca58-705c9389332mr2770164b3a.7.1718263963654; Thu, 13 Jun 2024 00:32:43 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:43 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:06 +0800 Subject: [PATCH 1/8] riscv: stacktrace: convert arch_stack_walk() to noinstr Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-1-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 arch_stack_walk() is called intensively in function_graph when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. As a result, the kernel logs a lot of arch_stack_walk and its sub-functions into the ftrace buffer. However, these functions should not appear on the trace log because they are part of the ftrace itself. This patch references what arm64 does for the smae function. So it further prevent the re-enter kprobe issue, which is also possible on riscv. Related-to: commit 0fbcd8abf337 ("arm64: Prohibit instrumentation on arch_stack_walk()") Fixes: 680341382da5 ("riscv: add CALLER_ADDRx support") Signed-off-by: Andy Chiu Reviewed-by: Alexandre Ghiti --- arch/riscv/kernel/stacktrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c index 528ec7cc9a62..0d3f00eb0bae 100644 --- a/arch/riscv/kernel/stacktrace.c +++ b/arch/riscv/kernel/stacktrace.c @@ -156,7 +156,7 @@ unsigned long __get_wchan(struct task_struct *task) return pc; } -noinline void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, +noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, struct task_struct *task, struct pt_regs *regs) { walk_stackframe(task, regs, consume_entry, cookie); From patchwork Thu Jun 13 07:11:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696242 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5193D13C9D8 for ; Thu, 13 Jun 2024 07:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263969; cv=none; b=IfJ4Yge85+ihLZbW/gnnmQlEYBH1r2nx5zlsWUudcz/sn3gjW5HRQcY/7QvYcjtOyBPku3psSvNVMeio6bWz7Gu2JJF4hypzDO1FMEx2bnQ2o4mMD4V+CbMsXLokzhKVlSgpBqEyxANcGQSfQQSZE/phYb3BNEtXZ+UrPws0fr4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263969; c=relaxed/simple; bh=vgghTaDxluToi5kVZZlG6AV2uIZ41UbFwSgx4K8n064=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DYflUeXAooeKX4ixWzLOYA4DkJUhB9rtyPjSUH+2yUBNfm+YX7cNX4Ss6ZZENrFA1OJaVadzu8BGJqVP4FT6ohCsvpw0aMGeHY+i5SEnOO5Odi1bcWgaJngwVdkKjEqWY7cBpQytTxAA1H+xrIGOeH+I6mVvWdoQhPXRhXuXC3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=etP/LdQK; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="etP/LdQK" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6f4603237e0so519953b3a.0 for ; Thu, 13 Jun 2024 00:32:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263967; x=1718868767; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=FQsuZ21+9cGptdlB0BPgeEv8WU9oI426BLaNkf2S+w0=; b=etP/LdQKyC3qb2zXfttY13aPyp4lbQJSHThwPFHEZQmKBlrzDRfA1y6lR4uMZYATe6 V+OCUa/8NMQONuM38CD388H5XUH/WZa6pTNxoYx2wKr9j3+RQkjchGBgInGwMxnGtzmd cbbUU1FD8S5KoyaGrqyKIMWY/4i5vQr5fUM3mn1JduSSaptEMQqIj4AfFJhxWPzry+MV huaa4WT5nzwFIDPP4/06hjQrXtThG/37yjeUIBNdr7Lu/rJxC8ucf88tLFTeRj/VH+/O m0jR20ArNZH4xunC6yqvVdkLzewbY37UM3hMzfgLirBtsKzsHIrPk01ozwGKRywAwOo1 l4ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263967; x=1718868767; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FQsuZ21+9cGptdlB0BPgeEv8WU9oI426BLaNkf2S+w0=; b=XvqwNVe/ADdA5au4a6KodkSreblXhjChRSnyrze+I4jghVqcyJ2Pruh7FMw3/drPth JLCOthrtjkINN4WI4BJ+YZH9Fz/eb6h8dFtORtna/aDsGwV6qWRm7EZLwAxUi9sFEJbe J7Qx+mIQc2UHyY2doO8ImWo1vFLIbI7N72xGi526gykQWiLT78VOj659U2czBCiHlxaX xr9+ujtlM8Wk+qxeRTSEJUlxNb/4spOvfwgHMk6CuwYoqjhF3DcPdxMz3umj+rt22Q24 n8Hl9v4RudzcSsLPmsc3g4NfLF1Ha1IYanyvmA2hr8H2qtQWx0RF3ucdzqYZjSRqawdk g0NQ== X-Forwarded-Encrypted: i=1; AJvYcCV5NLplGMDfSRUPaDEMyi3PmEHfW7yO6keU4OVHIEmfBt1yfASIlHMToPvwydmqOzCaJq603AhLQ7nvoLmZT5lxt/7uQnmWXjtU3k8hIV2BlDdJ X-Gm-Message-State: AOJu0Yz+D9teHLGYOTASn4JJfXz82HGZaZSxG3FuKCdCqB3ZUojt/ARU XWTLWSS6dXPLJM7EWMURnad+rC0z9mbC4Tj6DWCWk+Do0uoCWkVFLk45sWxvV34= X-Google-Smtp-Source: AGHT+IGkMW8dM0M3dwKshtF1f8Z+mSNjVPk7wRBBSBrIqzVVkpK0qMV/aHa7vJh7XzX910Gr5qBw7Q== X-Received: by 2002:a62:ab02:0:b0:705:d101:90ff with SMTP id d2e1a72fcca58-705d101919bmr283959b3a.16.1718263967547; Thu, 13 Jun 2024 00:32:47 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:47 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:07 +0800 Subject: [PATCH 2/8] tracing: do not trace kernel_text_address() Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-2-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 kernel_text_address() and __kernel_text_address() are called in arch_stack_walk() of riscv. This results in excess amount of un-related traces when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. The situation worsens when function_graph is active, as it calls local_irq_save/restore in each function's entry/exit. This patch adds both functions to notrace, so they won't show up on the trace records. Signed-off-by: Andy Chiu --- kernel/extable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/extable.c b/kernel/extable.c index 71f482581cab..d03fa462fa8b 100644 --- a/kernel/extable.c +++ b/kernel/extable.c @@ -74,7 +74,7 @@ int notrace core_kernel_text(unsigned long addr) return 0; } -int __kernel_text_address(unsigned long addr) +int notrace __kernel_text_address(unsigned long addr) { if (kernel_text_address(addr)) return 1; @@ -91,7 +91,7 @@ int __kernel_text_address(unsigned long addr) return 0; } -int kernel_text_address(unsigned long addr) +int notrace kernel_text_address(unsigned long addr) { bool no_rcu; int ret = 1; From patchwork Thu Jun 13 07:11:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696243 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A8F013CFA4 for ; Thu, 13 Jun 2024 07:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263974; cv=none; b=W8dHYkKRjLouCBLFa26OFxw7es1Yqy4DxihcmrwmbLZEoLtEz7yV0hMNgaw71X9rWSl+MoXCQKiyatvSSJwi4lTHr1BIoRR/uokQkz+XpfqQ0Ci7/G3EGEOI4FQMAiTw4ywK99dWrC8qNlYOzHcO+tBiAeLZdpf8r9DxX0l/JEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263974; c=relaxed/simple; bh=bMHErU1rLfk/wyX5o/DIX3M08fOpuNP508e8HYuo5pE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iL0BSOrN045TQgMO1gj80v6jJ7dp8Kp9wwd5tsr1HN2LBtpkE7wXk8gnIjm38AU4/mo7Pz+37nWetinYMvYZXUgCo3DfuGvE4FbPDVDq8QnlgcLrNTGg1ZanB8vugHVzRSKC+DKEbiRJL8mxzyx9X8Aun5gbkwfz22CrZJrP77k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=IuA+BP23; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="IuA+BP23" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5bacd59e562so363181eaf.2 for ; Thu, 13 Jun 2024 00:32:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263972; x=1718868772; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=KqGOcRG4MSO17L8Yoo0My/BpQLJGME/lNuiKW2jExFw=; b=IuA+BP23jxTOAG9lmxUL54VkgwxfKT7HwSd5s95zMFIDgJ7RuTNGy2P4AR2H3o8t7y Kb3a3HjAHdsi1CIHd+HNjleYdHbBWUsDwsmZ8SrlnqZascKvZhD95KtoZNVAXPY/QnFf wvifrYumjMjlLQJtVJa8x24s8NquzgrXB62yKa0qAQfdiNlMssbHUj+ih0zRU8/+3nZP TPGePrh9SiEcBt5FT6VbPcMtB9VNluBIrS9Au3Vx9NlVZZU9OSCtP/lFGTOUERF7cxZ6 Jws9Vfx1Jx6xbbaVkpqJT9cQfRRWTsSL+873EI33gP95bfCjWt5bf4BPAT1rBc/EM4jp NyBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263972; x=1718868772; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KqGOcRG4MSO17L8Yoo0My/BpQLJGME/lNuiKW2jExFw=; b=nO6bwyRJeGdoCgiBWlut2IQDyKSiFqRRFPyLj3GM1XRtHDANEZPltYhn2ORSDdUIUl cB6Qw7GOkExRjqOu4Djb7RBTt74KN1n0bB0BvK4m9tei18bTNQYbl4mdL3eWp3+JJ7J1 6mgfQ3Ds0jwr5xWhcAtV0MZ+mu9Yz2KZHdBaCM/He0LLuSgQ11a7ICx2mSzsk23wlnAW xFX1JEl2gt4x/TMDcRovfE8TUg3XcVFOEOuxe6njF8ZENh3uxOGIBb/yOpcfiVL3U851 xUmolJ0YIW0RJrmJz7T+ADiLotBRzgKZOP2LbDf0OeZ0aa4pMNqZnTxK4l+neLyaLakw mrDg== X-Forwarded-Encrypted: i=1; AJvYcCVfXrkxdO7Dqyt9gTyuP+b8FjDdLZCsIxnT1NUa9QSRHiiN6QWbXJn0IjlywxKAipkB9ouU/38g4/+oUsujA1JB6CyC9qMmz6oHQ2ipoa7yMyZ2 X-Gm-Message-State: AOJu0Yz+pUwRq6VR7R7KR+/XirM6WFQZPqsZZHHhNtHGkJeZoxbqRtx2 nGPLbaP0ru5yxWvu6d4ZTuU3epwxHCCUVRwJYih/kUFhmOviC51HTEItZxqrgUc= X-Google-Smtp-Source: AGHT+IH5DuWNaDmtQxAXDaBzm13SBjMPGjRqC4tR8s/HjfQ0ZY47iBUeJz4hi/y6Ws3uKj9ioyrEhA== X-Received: by 2002:a05:6871:b29:b0:254:b3cc:a6da with SMTP id 586e51a60fabf-25514c664afmr4230304fac.29.1718263971977; Thu, 13 Jun 2024 00:32:51 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:51 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:08 +0800 Subject: [PATCH 3/8] riscv: ftrace: support fastcc in Clang for WITH_ARGS Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-3-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Evgenii Shatokhin , Andy Chiu X-Mailer: b4 0.12.4 Some caller-saved registers which are not defined as function arguments in the ABI can still be passed as arguments when the kernel is compiled with Clang. As a result, we must save and restore those registers to prevent ftrace from clobbering them. - [1]: https://reviews.llvm.org/D68559 Reported-by: Evgenii Shatokhin Closes: https://lore.kernel.org/linux-riscv/7e7c7914-445d-426d-89a0-59a9199c45b1@yadro.com/ Signed-off-by: Andy Chiu Acked-by: Nathan Chancellor --- arch/riscv/include/asm/ftrace.h | 7 +++++++ arch/riscv/kernel/asm-offsets.c | 7 +++++++ arch/riscv/kernel/mcount-dyn.S | 16 ++++++++++++++-- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 9eb31a7ea0aa..5f81c53dbfd9 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -144,6 +144,13 @@ struct ftrace_regs { unsigned long a5; unsigned long a6; unsigned long a7; +#ifdef CONFIG_CC_IS_CLANG + unsigned long t2; + unsigned long t3; + unsigned long t4; + unsigned long t5; + unsigned long t6; +#endif }; }; }; diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index b09ca5f944f7..db5a26fcc9ae 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -497,6 +497,13 @@ void asm_offsets(void) DEFINE(FREGS_SP, offsetof(struct ftrace_regs, sp)); DEFINE(FREGS_S0, offsetof(struct ftrace_regs, s0)); DEFINE(FREGS_T1, offsetof(struct ftrace_regs, t1)); +#ifdef CONFIG_CC_IS_CLANG + DEFINE(FREGS_T2, offsetof(struct ftrace_regs, t2)); + DEFINE(FREGS_T3, offsetof(struct ftrace_regs, t3)); + DEFINE(FREGS_T4, offsetof(struct ftrace_regs, t4)); + DEFINE(FREGS_T5, offsetof(struct ftrace_regs, t5)); + DEFINE(FREGS_T6, offsetof(struct ftrace_regs, t6)); +#endif DEFINE(FREGS_A0, offsetof(struct ftrace_regs, a0)); DEFINE(FREGS_A1, offsetof(struct ftrace_regs, a1)); DEFINE(FREGS_A2, offsetof(struct ftrace_regs, a2)); diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index 745dd4c4a69c..e988bd26b28b 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -96,7 +96,13 @@ REG_S x8, FREGS_S0(sp) #endif REG_S x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_S x7, FREGS_T2(sp) + REG_S x28, FREGS_T3(sp) + REG_S x29, FREGS_T4(sp) + REG_S x30, FREGS_T5(sp) + REG_S x31, FREGS_T6(sp) +#endif // save the arguments REG_S x10, FREGS_A0(sp) REG_S x11, FREGS_A1(sp) @@ -115,7 +121,13 @@ REG_L x8, FREGS_S0(sp) #endif REG_L x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_L x7, FREGS_T2(sp) + REG_L x28, FREGS_T3(sp) + REG_L x29, FREGS_T4(sp) + REG_L x30, FREGS_T5(sp) + REG_L x31, FREGS_T6(sp) +#endif // restore the arguments REG_L x10, FREGS_A0(sp) REG_L x11, FREGS_A1(sp) From patchwork Thu Jun 13 07:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696244 Received: from mail-oo1-f41.google.com (mail-oo1-f41.google.com [209.85.161.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C72D913D241 for ; Thu, 13 Jun 2024 07:32:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263978; cv=none; b=NHHJwI8IbysuM/+7OqCSbQRksvaS8jHF2x2iYb++k3wMysoX31sO+QlLKF+fzcAEwuq6U5ud8u0IzdDQCUb2EgvX5yt2J8405OdeyMdhGHQn6P+HbZQPqgtQlLo19P6JyWglC2cNH4h9Whj2wLPmDu04QlrLESL7jB1p+sBGw2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263978; c=relaxed/simple; bh=Dx2FUFHDH8Y1zN/f24u2ocIVhF64yQxKbwIv1iUM+u8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tWth5gg6gdPvQkGniRpgTuhCALxjN9PYkbhu6Pr5lcHWahM0C77sVfCcBGMflwqbAylyUGxsy8+C8u93Ht83IUKzTo2076BlHKPruDSzYee6Z9dPKhSDt1DDGLmFDqachfUJDRBRhg6c/KMwN9LRt2OGWIgokuq7QUX6jfzST7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=iDD5YNkb; arc=none smtp.client-ip=209.85.161.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="iDD5YNkb" Received: by mail-oo1-f41.google.com with SMTP id 006d021491bc7-5b9f9e7176eso370252eaf.2 for ; Thu, 13 Jun 2024 00:32:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263976; x=1718868776; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=LTElu2O7G4IRv2D6Es7gQcAecPf3onDHBKt+B4ik+4k=; b=iDD5YNkbvFhJnGsagRANRfyjoIH27Clpz4BtMXzf07CvK7Hm5MlXOx1S6bKu2eKNUI tWPtgX7zKKB5NW4VungXMwvI4Asz+yFRWUprn1TBpNIedqmz1x3mj6BmiwRxOOl5KMG6 UCorkRpIOT/TMUr8ANB68TRGEljCUH6AuJFfYeUnUGZ0dWmWsiHpI9HD6xJ1qlEqeIJH K7NJeBG9vCDi0UVNllGag4NnmxVuvz8APW3UoPS52/w0LhK/p4J7/QEdnE1waTY2y8y/ mm0y9FOJ0OkZ6Q85d4XGNkDoQAn1mRJSd8PfoYeuw3Arrac3A+IXbckQUxMuLo2enK9f XiBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263976; x=1718868776; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LTElu2O7G4IRv2D6Es7gQcAecPf3onDHBKt+B4ik+4k=; b=UpZOPb45P3O79Wbq4d+34gYFc7DBpRq11tLgaiRgRUaqJgV0u/7McLRRVe38e/0cHm DIMZBsmVOQIwHW9wZ9zSgh9VbW/HFtYs9VUn0deiXGd7tL+0Mj/N5/A/7hUy7sxRFuoN JTSB4mgGv2WrnRc4PTg+5Zye2BB2fJ2BrtXENmdIHSYl26oPeZcn0CjPxG9gEOgMKs9i nYZpD+JWi3Mna/wkTyepiNipNVazSyHaAoErWnBVmPV0lltr35rCdDak7mUVFMF0363Y Q3ZvcPpWOJCfqy2NxafoTQYEnd6RUZPAlKW0u7TtRSmH4/zi78D5abNVhI/N+ou5Frr3 ENVA== X-Forwarded-Encrypted: i=1; AJvYcCVAAAobPR5n5DaP7BbJJd8HhCt7noQG/a3lGmQaAqMLgJrMSC6rv8jDHM0jdE7CyVWRoC4MRZwtq1rLbxfR6ROwp/9td6SCoVvr2VBfXpf5BJty X-Gm-Message-State: AOJu0YwJi/xEDmLCp9PuT0JjeosfCQ4Ys59curIXHfW961DxqRhj3oHS 8bL0K8ix3qPQWvTtLstEGpx1Vh0KVyDI5zKdnEnAOLD5LAYWj1Pcz6p3VE0CRpM= X-Google-Smtp-Source: AGHT+IGdJsIqWxldlAuacUU8l0dK4I8dOhgAshBKtsVS5QR/psuALrZ1MwA3KVg0EXK2RXnVq6OGZQ== X-Received: by 2002:a05:6870:c087:b0:254:a9bd:70b2 with SMTP id 586e51a60fabf-2551501e0bfmr4485943fac.59.1718263975852; Thu, 13 Jun 2024 00:32:55 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:55 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:09 +0800 Subject: [PATCH 4/8] riscv: ftrace: align patchable functions to 4 Byte boundary Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-4-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Evgenii Shatokhin , Andy Chiu X-Mailer: b4 0.12.4 We are changing ftrace code patching in order to remove dependency from stop_machine() and enable kernel preemption. This requires us to align functions entry at a 4-B align address. However, -falign-functions on older versions of GCC alone was not strong enoungh to align all functions. In fact, cold functions are not aligned after turning on optimizations. We consider this is a bug in GCC and turn off guess-branch-probility as a workaround to align all functions. GCC bug id: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88345 The option -fmin-function-alignment is able to align all functions properly on newer versions of gcc. So, we add a cc-option to test if the toolchain supports it. Suggested-by: Evgenii Shatokhin Signed-off-by: Andy Chiu --- arch/riscv/Kconfig | 1 + arch/riscv/Makefile | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..80b8d48e1e46 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -203,6 +203,7 @@ config CLANG_SUPPORTS_DYNAMIC_FTRACE config GCC_SUPPORTS_DYNAMIC_FTRACE def_bool CC_IS_GCC depends on $(cc-option,-fpatchable-function-entry=8) + depends on $(cc-option,-fmin-function-alignment=4) || !RISCV_ISA_C config HAVE_SHADOW_CALL_STACK def_bool $(cc-option,-fsanitize=shadow-call-stack) diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 06de9d365088..74628ad8dcf8 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -14,8 +14,13 @@ endif ifeq ($(CONFIG_DYNAMIC_FTRACE),y) LDFLAGS_vmlinux += --no-relax KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY +ifeq ($(CONFIG_CC_IS_CLANG),y) + cflags_ftrace_align := -falign-functions=4 +else + cflags_ftrace_align := -fmin-function-alignment=4 +endif ifeq ($(CONFIG_RISCV_ISA_C),y) - CC_FLAGS_FTRACE := -fpatchable-function-entry=4 + CC_FLAGS_FTRACE := -fpatchable-function-entry=4 $(cflags_ftrace_align) else CC_FLAGS_FTRACE := -fpatchable-function-entry=2 endif From patchwork Thu Jun 13 07:11:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696245 Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF56B13D25B for ; Thu, 13 Jun 2024 07:33:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263982; cv=none; b=dB4mSw0V8009TgzPFjnbi5OW86LxWemhg4OGQNjSry0Wwy8z/rE4RxdHUlvE9zr8V0tjQL9ZHXNJaiFmFq12rS0DX4sqDV59jVyS9qOZfoDWBC89svgjG+N58b406U6o37H0QmdDFghuGpn5YqL7yRBBBpEb+FRL/lZnINgBmuE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263982; c=relaxed/simple; bh=tupMDt20s7S7PlhgqDhuVkvtJDEBbx3ofqpMD3PiZ2k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D/uvXeKYOtEmhPFtUyPhXkziuZuB7L/LRs5MshCUEmyRbNOnSS5H3l3Zw4npoZd9SpxfGm/IlnPqvy+eSfXk29bMJEntuhNLnr9yr7qQeFoMMS7e1FJkKtwgwkHuYJoEz/zxPcdmpOEJMHHaEZN9a+5ZuOh5pP9bmkxHbSpnzf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=FcdI8T+8; arc=none smtp.client-ip=209.85.167.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="FcdI8T+8" Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3d24a231342so39139b6e.1 for ; Thu, 13 Jun 2024 00:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263980; x=1718868780; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xpwccXm6VSga0glYezh9Lcoo9mdtNlokYTZmh5MRn+k=; b=FcdI8T+80T80YU+ZQFMin0zapvV8rJZJ2673hnqQVSA4yfnH1wF0GtzLY8GFUjI5wn /Qop6ysDY0Zyfbm85odiJJy4DZIlxifiGdy6t8H1cXs6dcBTxvpzG7PMOtqlmyZmkxv/ FrB5urjcZktyYh4CpHURWCGNhGcAYVelDTQIPJEczdYfsHt2Kv4ReyGb9Lk6QrIhYR46 1B+1HZlKu0Iae80el5r4j50aFXDJ2zqEP9gv1PqO3T4qH/NNAEm4Ydx6A0tIMfaG881F ad3IleVEHQ8Qlqq1kTnptM1NBfnjp9PPRzGLn9ozRW7EDuIiO3yytScnlxIVIyDGXGcl V3/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263980; x=1718868780; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xpwccXm6VSga0glYezh9Lcoo9mdtNlokYTZmh5MRn+k=; b=By0otqn5YsvAzfzPJK0/t75t5KjQEaoNMELJj/K4p0XOoEauvr6cUV2Vf9+0CtuuYq uBWWFTDKE5AT9hQ81LqSZkpAFk9TISJMezw6ZUlKNRTojuGEyPggMGFudv/o1mBAKkSj xSORf2D2j81TcZmICB3coaprQ/r1SgCx7z/q+05qoueU6yhjbxBpnjTMgBQvfgkHNI+o 3j/5oDYGz/Pj9LfBN7dmafyk2GVDY4+BdYTHdfUr0O27Mt8j2xiMicgTDLU02dOzEy5F i5QpLzWTQNrtKM+miRiEe0VWDeVhwKRoQExpxMP3WhcQpqfEiASiU9UM64Kbo33XrYfl Nbfw== X-Forwarded-Encrypted: i=1; AJvYcCUu3dCBYpy1984Dq2gES7k6tgv92uowqqsItoX3KTcbyOkzCW2sUqS9aW2Yr/S/Z7+viUHZjxqzlYppb8t/WZFIB3FxfpqbGUmRHTIuEBZNkt5H X-Gm-Message-State: AOJu0YzFNoT+43stexaPYfZ+JtGVUNut4ozIzcpeXBtpZ5puGHxdp4/I +KrkLLt/7BWvKKJ9GTlQxOgwn39i8aOozf68VRy+b3oFEeHIn2ETj5rKVvfqk0g= X-Google-Smtp-Source: AGHT+IEzkbVgn0G1G4t0BzOnoC4MTfTDVP2Z/BlOjvozamyY5nqFEfSfIU6N3LGNo64kZ62JzW5oRQ== X-Received: by 2002:a05:6808:1a24:b0:3d2:4a08:2c99 with SMTP id 5614622812f47-3d24a0832b0mr183168b6e.23.1718263979694; Thu, 13 Jun 2024 00:32:59 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.32.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:32:59 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:10 +0800 Subject: [PATCH 5/8] riscv: ftrace: prepare ftrace for atomic code patching Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-5-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 We use an AUIPC+JALR pair to jump into a ftrace trampoline. Since instruction fetch can break down to 4 byte at a time, it is impossible to update two instructions without a race. In order to mitigate it, we initialize the patchable entry to AUIPC + NOP4. Then, the run-time code patching can change NOP4 to JALR to eable/disable ftrcae from a function. This limits the reach of each ftrace entry to +-2KB displacing from ftrace_caller. Starting from the trampoline, we add a level of indirection for it to reach ftrace caller target. Now, it loads the target address from a memory location, then perform the jump. This enable the kernel to update the target atomically. The ordering of reading/updating the targert address should be guarded by generic ftrace code, where it sends smp_rmb ipi. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/ftrace.h | 4 +++ arch/riscv/kernel/ftrace.c | 80 ++++++++++++++++++++++++++--------------- arch/riscv/kernel/mcount-dyn.S | 9 +++-- 3 files changed, 62 insertions(+), 31 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 5f81c53dbfd9..7199383f8c02 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -81,6 +81,7 @@ struct dyn_arch_ftrace { #define JALR_T0 (0x000282e7) #define AUIPC_T0 (0x00000297) #define NOP4 (0x00000013) +#define JALR_RANGE (JALR_SIGN_MASK - 1) #define to_jalr_t0(offset) \ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0) @@ -118,6 +119,9 @@ do { \ * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here. */ #define MCOUNT_INSN_SIZE 8 +#define MCOUNT_AUIPC_SIZE 4 +#define MCOUNT_JALR_SIZE 4 +#define MCOUNT_NOP4_SIZE 4 #ifndef __ASSEMBLY__ struct dyn_ftrace; diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index 87cbd86576b2..f3b09f2d3ecc 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -64,42 +64,64 @@ static int ftrace_check_current_call(unsigned long hook_pos, return 0; } -static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, - bool enable, bool ra) +static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, bool validate) { unsigned int call[2]; - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int replaced[2]; + + make_call_t0(hook_pos, target, call); - if (ra) - make_call_ra(hook_pos, target, call); - else - make_call_t0(hook_pos, target, call); + if (validate) { + /* + * Read the text we want to modify; + * return must be -EFAULT on read error + */ + if (copy_from_kernel_nofault(replaced, (void *)hook_pos, + MCOUNT_INSN_SIZE)) + return -EFAULT; + + if (replaced[0] != call[0]) { + pr_err("%p: expected (%08x) but got (%08x)\n", + (void *)hook_pos, call[0], replaced[0]); + return -EINVAL; + } + } - /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ - if (patch_insn_write((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) + /* Replace the jalr at once. Return -EPERM on write error. */ + if (patch_insn_write((void *)(hook_pos + MCOUNT_AUIPC_SIZE), call + 1, MCOUNT_JALR_SIZE)) return -EPERM; return 0; } -int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +static int __ftrace_modify_call_site(ftrace_func_t *hook_pos, ftrace_func_t target, bool enable) { - unsigned int call[2]; + ftrace_func_t call = target; + ftrace_func_t nops = &ftrace_stub; - make_call_t0(rec->ip, addr, call); - - if (patch_insn_write((void *)rec->ip, call, MCOUNT_INSN_SIZE)) - return -EPERM; + WRITE_ONCE(*hook_pos, enable ? call : nops); return 0; } +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +{ + unsigned long distance, orig_addr; + + orig_addr = (unsigned long)&ftrace_caller; + distance = addr > orig_addr ? addr - orig_addr : orig_addr - addr; + if (distance > JALR_RANGE) + return -EINVAL; + + return __ftrace_modify_call(rec->ip, addr, false); +} + int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int nops[1] = {NOP4}; - if (patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE)) + if (patch_insn_write((void *)(rec->ip + MCOUNT_AUIPC_SIZE), nops, MCOUNT_NOP4_SIZE)) return -EPERM; return 0; @@ -114,10 +136,14 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, */ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) { + unsigned int nops[2]; int out; + make_call_t0(rec->ip, &ftrace_caller, nops); + nops[1] = NOP4; + mutex_lock(&text_mutex); - out = ftrace_make_nop(mod, rec, MCOUNT_ADDR); + out = patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE); mutex_unlock(&text_mutex); if (!mod) @@ -126,12 +152,10 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) return out; } +ftrace_func_t ftrace_call_dest = ftrace_stub; int ftrace_update_ftrace_func(ftrace_func_t func) { - int ret = __ftrace_modify_call((unsigned long)&ftrace_call, - (unsigned long)func, true, true); - - return ret; + return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } struct ftrace_modify_param { @@ -185,7 +209,7 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, if (ret) return ret; - return __ftrace_modify_call(caller, addr, true, false); + return __ftrace_modify_call(caller, addr, true); } #endif @@ -220,17 +244,17 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, prepare_ftrace_return(&fregs->ra, ip, fregs->s0); } #else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ -extern void ftrace_graph_call(void); +ftrace_func_t ftrace_graph_call_dest = ftrace_stub; int ftrace_enable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, true, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, true); } int ftrace_disable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, false, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, false); } #endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* CONFIG_DYNAMIC_FTRACE */ diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index e988bd26b28b..bc06e8ab81cf 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -162,7 +162,8 @@ SYM_FUNC_START(ftrace_caller) mv a3, sp SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) #ifdef CONFIG_FUNCTION_GRAPH_TRACER addi a0, sp, ABI_RA @@ -172,7 +173,8 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) mv a2, s0 #endif SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_graph_call_dest + jalr 0(ra) #endif RESTORE_ABI jr t0 @@ -185,7 +187,8 @@ SYM_FUNC_START(ftrace_caller) PREPARE_ARGS SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) RESTORE_ABI_REGS bnez t1, .Ldirect From patchwork Thu Jun 13 07:11:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696246 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C83F113D283 for ; Thu, 13 Jun 2024 07:33:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263986; cv=none; b=bS/lk4MWuXOyRoVtIEhoN/KStyXUZ9uMVBAeQuDilanFG8Mb5Ze6XGZYrhhgtIpvppJH49ijGUmf0mNOXsrRVFVsW29EIgAETwdJ/Ck66vLtoIc5Qk6ODw7gtCaHIwyY4iOhtryJeexw1qyKOgaJ2VfE/kx3COrTwrKF5vl2VMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263986; c=relaxed/simple; bh=SouKpCcGuPU35PuUtIVf4fx8/4KFcZyhXdY05tRopyw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g2c0LSU34eK7ek6ZlnF/7HPho7O7QAMAft4K8XfYtEraLTseJYbdjdGwOul4piHKKL4EUN5eXTD0xQhiMw3ZxsJ4RuLyX4gRwaITAd9y8MjygPNpN1R9l+s0/tGUOie/+Mebfz0pcHVZ+SJfCn55jSj+FQb7CPakxaRERs6I2Rg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=SzAw/VIh; arc=none smtp.client-ip=209.85.167.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="SzAw/VIh" Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3d21b3da741so332982b6e.2 for ; Thu, 13 Jun 2024 00:33:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263984; x=1718868784; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=wBQdisfSIcfLCxaUXLlDTwMoYdzfD8LhoeZl0/Tj/fA=; b=SzAw/VIhC4REKBJ25UHy2FmqHs22MQouk4M6ydVgr0GxqXQse27QzsYNXGavBehRpz 1NfVjDQQDqSG4QnUBbU36L9kJHu3hyqAXJQoF3+f0dzFXDb+6uDulxdBrb0mhDIPVIa2 pFcmLnD0Ttfc+/+NF12SO/MV4iElDkZkQfn/iYUGVtiYXPB2gH8G27tMmz/iB4P6YHp0 aEH02aL8OjI1KJwB7H1m1rqAuPxHZDYbEPzKVptmGwq8uHbLAcNEcIdsXR1snexZRtiM 8t0whC4n9dKfOoAKBe9QGUuv/1uWefl4Ul8VL4NTwxs2i+w+IRUwPUVceqzRBE1SaIby FVMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263984; x=1718868784; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wBQdisfSIcfLCxaUXLlDTwMoYdzfD8LhoeZl0/Tj/fA=; b=T6KxgCng309ghOPNBWNnfLkVr8FfH1qKubNo60Njm2iq/gJtuiMpQgyjFCbwvBGmJJ g/sqqcDyutyIBtsvsXAMXxdLEoYJRGGadt0g1KGMur2350YfoQ+dZ5/BgHVZ1l/zXxHO FVefyGk8rVyQ0xbUUTqkORHYE5woxmxdhUJ/gOXgETs7oaL1IM+7kaxVaoKXIsZm47Sq yrlxj5nDyQBv9zxd2QdFAFkYheCf0bO14KcFSTN+W2aq6zwrtlaK5o/4ZjLLOjShaGnJ 4s9qPDUk6A1YEwDRNMBFUL7/5X9FXBhfnACNgEMf4QvyE1Ad62ExdRZj+p3XKFnN6+tZ bU2g== X-Forwarded-Encrypted: i=1; AJvYcCVeodxouylZWdtkVCfkGkK9ZwFDalHKNZLPX8P1goulCkHD0cqd6vGmmysyogmNduDIa7uuHgfjq35QoOBNwDTYKS0Td1D6CVY9eM4wYhE01Kfb X-Gm-Message-State: AOJu0YzmZdl5q4nJZM9u8ZwC+oa51hHKvojj6yA1trowMvlSXD7X0bLz SWXcUlZrplgvsA+GKAy22vR/h6vLxqF6Vgqw9qY+QvDgwUb70lOVTc2wMQFbZtg= X-Google-Smtp-Source: AGHT+IHI/57UF4ERYhNsT8IgBFHVZFnNkvkL7rNY4vjoxvdn096FWUD3qDYx3RYqModhDDSYikhCvg== X-Received: by 2002:a05:6870:4412:b0:254:6eb5:ab2f with SMTP id 586e51a60fabf-25514afc016mr4471125fac.6.1718263983791; Thu, 13 Jun 2024 00:33:03 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:03 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:11 +0800 Subject: [PATCH 6/8] riscv: ftrace: do not use stop_machine to update code Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-6-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 Now it is safe to remove dependency from stop_machine() for us to patch code in ftrace. Signed-off-by: Andy Chiu --- arch/riscv/kernel/ftrace.c | 53 ++++------------------------------------------ 1 file changed, 4 insertions(+), 49 deletions(-) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index f3b09f2d3ecc..9a421e151b1d 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -13,23 +13,13 @@ #include #ifdef CONFIG_DYNAMIC_FTRACE -void ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +void arch_ftrace_update_code(int command) { mutex_lock(&text_mutex); - - /* - * The code sequences we use for ftrace can't be patched while the - * kernel is running, so we need to use stop_machine() to modify them - * for now. This doesn't play nice with text_mutex, we use this flag - * to elide the check. - */ - riscv_patch_in_stop_machine = true; -} - -void ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) -{ - riscv_patch_in_stop_machine = false; + command |= FTRACE_MAY_SLEEP; + ftrace_modify_all_code(command); mutex_unlock(&text_mutex); + flush_icache_all(); } static int ftrace_check_current_call(unsigned long hook_pos, @@ -158,41 +148,6 @@ int ftrace_update_ftrace_func(ftrace_func_t func) return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } -struct ftrace_modify_param { - int command; - atomic_t cpu_count; -}; - -static int __ftrace_modify_code(void *data) -{ - struct ftrace_modify_param *param = data; - - if (atomic_inc_return(¶m->cpu_count) == num_online_cpus()) { - ftrace_modify_all_code(param->command); - /* - * Make sure the patching store is effective *before* we - * increment the counter which releases all waiting CPUs - * by using the release variant of atomic increment. The - * release pairs with the call to local_flush_icache_all() - * on the waiting CPU. - */ - atomic_inc_return_release(¶m->cpu_count); - } else { - while (atomic_read(¶m->cpu_count) <= num_online_cpus()) - cpu_relax(); - } - - local_flush_icache_all(); - - return 0; -} - -void arch_ftrace_update_code(int command) -{ - struct ftrace_modify_param param = { command, ATOMIC_INIT(0) }; - - stop_machine(__ftrace_modify_code, ¶m, cpu_online_mask); -} #endif #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS From patchwork Thu Jun 13 07:11:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696247 Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDAA513D508 for ; Thu, 13 Jun 2024 07:33:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263990; cv=none; b=adAnfFxGCaczXywZW2T9Zhcb1Y+LOpiA9oA7xwmVpnVQ/CaZIpoVVyYlVEBE99F7Tom2Rn5/ykURU+rlBOwoQAo0ycVCnuEMCuTSw0eyAuG/9eWZAClDJXQWOjdOWEyInbKBVxACLFyqNUn/CsVUVTi6mVdF1kgfzDEl5h5SzA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263990; c=relaxed/simple; bh=Qip5cL32AGGgHJ2xW9xkkh9vglc7nxfsBldmU/OW3wg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pdNewMoUEkoNGxBJRcqmNzPwdrXmlSIb1dZ+GgI5gGFB5PRTzH0oXs0S7IqUGVCjAHW0Oj21Ic+BVqCE9knWLrE8d7Lx5M0Vq21VNLR78a8dmdIMHjsHfmC/DkCTzomOV/9qiwFf3ybcfgnPTv8VOL8qd78r1NU8DPSmCKqYSkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=iRvRIYIO; arc=none smtp.client-ip=209.85.167.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="iRvRIYIO" Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3c9b94951cfso358450b6e.3 for ; Thu, 13 Jun 2024 00:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263988; x=1718868788; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=TeZhGtu+DoZzH37TzHgHW9NZu5mW4WfgIzkfv6QNhDQ=; b=iRvRIYIOrrNcIUvUYdmLBf3vNeKbUpWLvESA1dCHUUXQt8nVtTaI7UemCGutCRFY8K G9xrTTsoptnQtxLK1WhcmlSMmGE37YVSrGVzz1cyl+5qX5kT7FmtZ8uCbKEgKS/gbxzc 6OE76uclx0Yv2wLQmV4MzDKVBgvq8zCDvuvoyF+vU6VMHM1VmgK5fXjzCU4lnyX5wzMu BhnhthKk1SwoYuk5IrS7XIa0t1wrV7cofVvXD11xhovd+bKyfKss2h5ICWt7dfkYjEfQ ss1GUWcA5KpvZqibYp9FjCxHFMOjtVxr4nUZh4hnqX4E5keSo81mv4VgypAjnuMKhhMi 191A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263988; x=1718868788; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TeZhGtu+DoZzH37TzHgHW9NZu5mW4WfgIzkfv6QNhDQ=; b=wthIBRLV1p66DfKKoILsIUgO3Gt/nLtSYCddCBG79dypxwfwsdEYfDHUB9BwvpoCM1 wTN2FVKDN6UBzcPNkRIT0NwcdLntWMqE79k8pNwhmmx0pcX25CIRc4HjoMuCjP5WvxnT X2749X4QbfuS1fdl8wkGNvzS6zhKC8Q1my+lzOUnoCiEFGntxQujGb9Oe1Pw0pgXtmAG cyPEf6a+mirUBSNcvn2DOB3iDSRqWSZIcLXzbYiGoR4+m9jHOxBP/k2IQgFDSj6p470K 9RMfItpfTqoL9r5fKPCnjMumjzBRIV6LxOff0atmNu+jvydMa9PW84oWDLO8Oq5f704A l4Yg== X-Forwarded-Encrypted: i=1; AJvYcCVzFnt8W4x4XljFl4ABRbLnNfTySSGWcP0gRlloL5vlS24Veth8fljg24CXU2warmbGIxc7ftvx1vgsGutiB4mptmA48Dvz6bHQsnwkLzQYaSDn X-Gm-Message-State: AOJu0Yx/HUiH0+zHxiTbibFSpNZV0/VXTEJvPI88+PsbOHUeJpRQvzzK D4T3XOgWSi40ShFU5nxMiXpobD0f0gQ8xt5pHOfM8lF1j+Jt5DcYF6ankXk5ps3blMFEXcHhhRS B X-Google-Smtp-Source: AGHT+IG/QcH0femSCZzF6g5xhDc4dqSZTxWrgAQgosN/9/estM4RRUpCw1U0PcaMW2D86qVbhMaGwA== X-Received: by 2002:a05:6808:2199:b0:3d2:1c8f:be0c with SMTP id 5614622812f47-3d23e045f11mr5371608b6e.30.1718263988155; Thu, 13 Jun 2024 00:33:08 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:07 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:12 +0800 Subject: [PATCH 7/8] riscv: vector: Support calling schedule() for preemptible Vector Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-7-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 Each function entry implies a call to ftrace infrastructure. And it may call into schedule in some cases. So, it is possible for preemptible kernel-mode Vector to implicitly call into schedule. Since all V-regs are caller-saved, it is possible to drop all V context when a thread voluntarily call schedule(). Besides, we currently don't pass argument through vector register, so we don't have to save/restore V-regs in ftrace trampoline. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/processor.h | 5 +++++ arch/riscv/include/asm/vector.h | 22 +++++++++++++++++++--- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 68c3432dc6ea..02598e168659 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -95,6 +95,10 @@ struct pt_regs; * Thus, the task does not own preempt_v. Any use of Vector will have to * save preempt_v, if dirty, and fallback to non-preemptible kernel-mode * Vector. + * - bit 29: The thread voluntarily calls schedule() while holding an active + * preempt_v. All preempt_v context should be dropped in such case because + * V-regs are caller-saved. Only sstatus.VS=ON is persisted across a + * schedule() call. * - bit 30: The in-kernel preempt_v context is saved, and requries to be * restored when returning to the context that owns the preempt_v. * - bit 31: The in-kernel preempt_v context is dirty, as signaled by the @@ -109,6 +113,7 @@ struct pt_regs; #define RISCV_PREEMPT_V 0x00000100 #define RISCV_PREEMPT_V_DIRTY 0x80000000 #define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000 +#define RISCV_PREEMPT_V_IN_SCHEDULE 0x20000000 /* CPU-specific state of a task */ struct thread_struct { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 731dcd0ed4de..50693cffbe78 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -75,6 +75,11 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline bool riscv_v_is_on(void) +{ + return !!(csr_read(CSR_SSTATUS) & SR_VS); +} + static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) { asm volatile ( @@ -243,6 +248,11 @@ static inline void __switch_to_vector(struct task_struct *prev, struct pt_regs *regs; if (riscv_preempt_v_started(prev)) { + if (riscv_v_is_on()) { + WARN_ON(prev->thread.riscv_v_flags & RISCV_V_CTX_DEPTH_MASK); + riscv_v_disable(); + prev->thread.riscv_v_flags |= RISCV_PREEMPT_V_IN_SCHEDULE; + } if (riscv_preempt_v_dirty(prev)) { __riscv_v_vstate_save(&prev->thread.kernel_vstate, prev->thread.kernel_vstate.datap); @@ -253,10 +263,16 @@ static inline void __switch_to_vector(struct task_struct *prev, riscv_v_vstate_save(&prev->thread.vstate, regs); } - if (riscv_preempt_v_started(next)) - riscv_preempt_v_set_restore(next); - else + if (riscv_preempt_v_started(next)) { + if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) { + next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE; + riscv_v_enable(); + } else { + riscv_preempt_v_set_restore(next); + } + } else { riscv_v_vstate_set_restore(next, task_pt_regs(next)); + } } void riscv_v_vstate_ctrl_init(struct task_struct *tsk); From patchwork Thu Jun 13 07:11:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13696248 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CED8613D52B for ; Thu, 13 Jun 2024 07:33:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263994; cv=none; b=GFqaRwtjGW52ugIqk6K2Duo5osB80+a8GB8ALo3pHyBCFBCh7K86KIjjykkaQd8zAGj2JyD9WiIh63PWzbnTLUaOu1kcouNXsG+mmyywZ8GJoBBdgbEz1Kd4DRZSMC/NihA7amriruiiR8gApv7sWah2p9GzoFAY6x9SC12XTS0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263994; c=relaxed/simple; bh=uoTnrqYSg19kqb0lgMH7SLS/qe9dnaqbOD+HJnS87Ho=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=haQr3eVJqzHK/IG0iGhtYKVhWU73HwGgNhwP66rmWcD4pf6YedtqBrU0z3R73mGEGMawb4YFw4623GCb70Zr8LzFXeND1Ciagq4Mp8FmiwPpyyG3ED78jWTR5Sw4cip1zl/JJEUfsqJ7bZ7uYIQh/TlQbm8ZdvFqLTdOUPngYXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=WlfE5jC2; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="WlfE5jC2" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-6e4dbca52f0so525214a12.0 for ; Thu, 13 Jun 2024 00:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718263992; x=1718868792; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=RlJIkXaRaXYEOo0FtAy4w5B+d0jhj99dMDrjOpWfyi4=; b=WlfE5jC2GSHvYdL9YEYEFvD537xrYazzBJ5mcLQhpz6eXYU1TCCthCwF/9Cx4OGZG5 M5dcRHbYBN8zf7QyWPprIZvpyEqDCAGabmxG0cpu47t9F+v5jRoLq54svsMPwYaSYXxo MbqkKkVE76RkQOkzbk9o1t+nlNWyD/mSjNxXvgmD6kdWjHRL4UiNcm6dbFjq6HcHb8ht jnNHxiZFEfy0Tl1Da6DlSmvzTiKMPMUdoWO3weOiLfM7Mhi/Xg9f3c0rjAToPMUvEsVy byUqA1iW/vapQZGso9cMJL9g//EPx1sODvCJLndZW0DRbnp8AB80IqtZXCNGcBByQluk XS0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263992; x=1718868792; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RlJIkXaRaXYEOo0FtAy4w5B+d0jhj99dMDrjOpWfyi4=; b=Ur7GAjYajI6qy7REdXhflZfiPBxMfG5fiQHRaKR/irl1EnY2Qof1E7ffsV7LvaUP8O RyLLzelZjL47bWhmreNsGTH8y63iCVX5FIvfBK6jt8EB2qfOI9sq4PGCrmwD2pCJmUEY ZNJAOWVni274RatbeYApGMX8la8q5h9g8+E1vZpbcaPsOqPHQo/x475eg/Trllp1ex81 6M8KWSARiexhBrAZ5pUh00r+ivDPEVcfca0qbTGa9fo3iQeOz6BTc/Sw3V7szElPmlIK gUdkAXiUPO+wcIlc0q9Z4MUAND47//cLiVSV3FdGtr4p6K4qA2Shlnmj5xRac4/fmBhU 5UsQ== X-Forwarded-Encrypted: i=1; AJvYcCU/8XoNY1b2GI3G4UqD9+UnyntKYfmuWMutjkLTu2J7nuhuCix4Unz32EvI6nSXCMqhC+pdcTJBOo16/UOIGTNR4fOL5MVZRUOl7wkYEvF0iljq X-Gm-Message-State: AOJu0YwMSAhOyCJp71MPVA+RgJHEYsYyLWcb4am48JL4TNGteZvDrrSW IuJcG64NtpuNwFMom5GQyfECMUS0gjyezQk16yLhMI5EkUIQfwlylLgez1rOSNg= X-Google-Smtp-Source: AGHT+IEyM0ONI7ICTx6DjS1SeAdMKnjMDIyvhgY/AOB26Jk18oYzU1QW75t2bRm257+1EVhnYwVCTA== X-Received: by 2002:a05:6a20:3d88:b0:1b2:a889:f7da with SMTP id adf61e73a8af0-1b8a9c887fdmr4702279637.55.1718263991983; Thu, 13 Jun 2024 00:33:11 -0700 (PDT) Received: from [127.0.1.1] (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc78a1a0sm730028b3a.0.2024.06.13.00.33.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:33:11 -0700 (PDT) From: Andy Chiu Date: Thu, 13 Jun 2024 15:11:13 +0800 Subject: [PATCH 8/8] riscv: ftrace: support PREEMPT Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240613-dev-andyc-dyn-ftrace-v4-v1-8-1a538e12c01e@sifive.com> References: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> In-Reply-To: <20240613-dev-andyc-dyn-ftrace-v4-v1-0-1a538e12c01e@sifive.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Zong Li , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Puranjay Mohan Cc: Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, llvm@lists.linux.dev, Andy Chiu X-Mailer: b4 0.12.4 Now, we can safely enable dynamic ftrace with kernel preemption. Signed-off-by: Andy Chiu --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 80b8d48e1e46..c1493ee1b8cd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -133,7 +133,7 @@ config RISCV select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_GRAPH_RETVAL if HAVE_FUNCTION_GRAPH_TRACER - select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION + select HAVE_FUNCTION_TRACER if !XIP_KERNEL select HAVE_EBPF_JIT if MMU select HAVE_GUP_FAST if MMU select HAVE_FUNCTION_ARG_ACCESS_API