From patchwork Thu Aug 22 12:04:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13773272 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 376901C9450 for ; Thu, 22 Aug 2024 12:04:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328296; cv=none; b=SUv+woCjM+2A1qNwMaiR3kDatH4a8+sqLdac5tzvyq5FCDTm2md23L+iN5unptAr7+AXoSryLL3UwWfHtVuzT8M1qFOf7on27qQvmQ8+ipMtAlYH0lDKYGYEzyTX9nU1B1rlzmXh5xVCvMPqlvEt4A1fDS1WHWu+QXibxNbUDMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328296; c=relaxed/simple; bh=x4Fj8n509ULmZ45N1WACwPWEQuAc2oLxAGwmKjHodr0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZktFRgw05eIl5QlkXoatTz05daoNmsmcXjaf5xxBt4hH3nzFwPEOGI8iukuZiUglege8sQugN335/U8gQJ0rujc+Wyzdd1xkwYw//d3HIgt/MswCYIkXGI7w7qOL9fkO0/B0xR6cBXrtM3tXml2970TvtSqu4wYzF1zpHqe+kcQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BwtKg3Jy; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BwtKg3Jy" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3717ddcae71so301990f8f.3 for ; Thu, 22 Aug 2024 05:04:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724328292; x=1724933092; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=m06R8rBHhIV+fNfH7++C5Q7rVR6HteIR9+x3IVgvVo0=; b=BwtKg3Jy4A7j/nmVN65Il9mrfbmhi/sOSda2DQcHvi05s77inySGxqHn6BhV8TAj+4 Vy2xMx+nxZ5QkH6QWl4ZvEqVmTPcRgLSBBFuqdMOiEEyKgsH8vrHbIdzeK07NAu2etxQ S/Bel1RsNjOQzmAb2DiEJ0KkOkLKzDCQdRZVlYUvBJephgQYr6eMkr9Q9/iAIvFRQNjv /QOQlzryBQRZKKDU4x1qZhc3AaolqV/030CgaD8x+rQL6MByat7mhXmR5t1LflM1RvUd dSLEett6IGDnf0spQFwSmwtjyLcacz8vylcwJlQWD9Gj1YfK+fZCMjxHkt9lUpnhWOcX s31g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724328292; x=1724933092; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m06R8rBHhIV+fNfH7++C5Q7rVR6HteIR9+x3IVgvVo0=; b=RMJW+8XwbaUYgUYHktZ2G4j1AwxEcs/2Rl/WCXZIVO4j8OwzjwR7C8vgPykqqNVY/D wVHkQPTcNJhY7t8F+/JsDvHzuXr9XEznH9ahHh9T+u2A155AQ/OJP2LOK17tx9DROryH HxtblHAcU0LjPXy9sW6wlBTDbrlhEvBPi+eoQUGLrhPDP+VCT0sv6sr/VJxZAA5mRIDr CcyqYokhKpZrkusezQ0HSOl7/KFoV8V4vqburcCPrFqMcNNyjWjc/vvrRA0cpcO8z8bH AGuE5eM8bbpwYYb4RKPgOWjExFpA87BEziq/ldgoURvYTxCh+BxbrEz4OrVpimxuTKJl K93g== X-Gm-Message-State: AOJu0Yz9eIEFP5ey0X+YhOQwIUQJYqJfdJCMkhLOI5xqd6byTpXMVOQD 8b/5Z/mDOZe7h9dtDReyCLSO4GZ/g9Hnj4AASNoaggC4vrxTYa8E8XPk8F63wX8gHbaDudmFWgt 6Xy5wIQO6LU93Rg== X-Google-Smtp-Source: AGHT+IERBhVv08A2uZv+MpnM9lKKuFpvF/ldKWmNQbSwx22l+/nHv3lZyxTumbSPIvwOOIsyXNIWmCzz19pCed0= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:6000:2ad:b0:371:9373:5078 with SMTP id ffacd0b85a97d-372fd59721cmr11543f8f.3.1724328292363; Thu, 22 Aug 2024 05:04:52 -0700 (PDT) Date: Thu, 22 Aug 2024 12:04:13 +0000 In-Reply-To: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3678; i=aliceryhl@google.com; h=from:subject:message-id; bh=x4Fj8n509ULmZ45N1WACwPWEQuAc2oLxAGwmKjHodr0=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBmxylcN8mcEXadzOJVoentM5FyVm0c7KXiYJoT5 nryZ6UFTe2JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZscpXAAKCRAEWL7uWMY5 Rh5pEACltjdooXrOi5jfHOH3q4t2NrSAWmz0Wzo7scorn20GnCAwxRr3y/CF07ko6cGGlugqo7w fw59+gzaz0SDqS70pEVZt5qbsa21Mbm1GrJKocf2mXPLC9onKsU9xOyLlPXlxi+o89Ln3Lo729R IKu+oYEF5yDZC0nRxFTEZsX0YJ/bNKZwEMFsYem9WSrjI4ifDw7HDPifTLoRh1HRJmBRciDe9+f V7Tm/sST0aeMNPsFLh/eCQsQvL4Bf916kOL0jkbZGKSoSS4gJPTDf2569fmJ0PU/Z7/62qsKJaw paEkLJnj/MQLxgslDQ8lazmAMMDoQYH0CqrkBlIzksOwB0ftEuQI3hrV7ELD/5AnUCrup2fWscm jtounuZ8L6EpPLOCSAdPObnCCNgNTpIO3CcN6wt+GW8Flc7OsncgAq2qa8JDNUesF4jHVm9v9u4 hX2UZM5ou9VSVpxvlf2oOYTmZPnNyvrL9DZu9LjXnEwwg+qXpgQKvEvDY6ufRj+kC8SedA+UB3F ulk9WotpkJkTv4PxvDq1h9XI4djWh4uXsdrRizqa+QZAMv/1vjm8bMkaFkElesH5gA5akc2m1NP sNi8Jr2dh7/sqpvIvYrOucLMAcK/bzi7/M17+DSgo1+PyKsqSqlwBJiDqkeh3uvgJJRRh7MaE3U UHQDUWgEeb5N4nQ== X-Mailer: b4 0.13.0 Message-ID: <20240822-tracepoint-v8-1-f0c5899e6fd3@google.com> Subject: [PATCH v8 1/5] rust: add generic static_key_false From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl Add just enough support for static key so that we can use it from tracepoints. Tracepoints rely on `static_key_false` even though it is deprecated, so we add the same functionality to Rust. This patch only provides a generic implementation without code patching (matching the one used when CONFIG_JUMP_LABEL is disabled). Later patches add support for inline asm implementations that use runtime patching. When CONFIG_JUMP_LABEL is unset, `static_key_count` is a static inline function, so a Rust helper is defined for `static_key_count` in this case. If Rust is compiled with LTO, this call should get inlined. The helper can be eliminated once we have the necessary inline asm to make atomic operations from Rust. Signed-off-by: Alice Ryhl --- rust/bindings/bindings_helper.h | 1 + rust/helpers.c | 9 +++++++++ rust/kernel/jump_label.rs | 29 +++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 4 files changed, 40 insertions(+) diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index b940a5777330..8fd092e1b809 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include diff --git a/rust/helpers.c b/rust/helpers.c index 92d3c03ae1bd..5a9bf5209cd8 100644 --- a/rust/helpers.c +++ b/rust/helpers.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -133,6 +134,14 @@ bool rust_helper_refcount_dec_and_test(refcount_t *r) } EXPORT_SYMBOL_GPL(rust_helper_refcount_dec_and_test); +#ifndef CONFIG_JUMP_LABEL +int rust_helper_static_key_count(struct static_key *key) +{ + return static_key_count(key); +} +EXPORT_SYMBOL_GPL(rust_helper_static_key_count); +#endif + __force void *rust_helper_ERR_PTR(long err) { return ERR_PTR(err); diff --git a/rust/kernel/jump_label.rs b/rust/kernel/jump_label.rs new file mode 100644 index 000000000000..011e1fc1d19a --- /dev/null +++ b/rust/kernel/jump_label.rs @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Logic for static keys. +//! +//! C header: [`include/linux/jump_label.h`](srctree/include/linux/jump_label.h). + +/// Branch based on a static key. +/// +/// Takes three arguments: +/// +/// * `key` - the path to the static variable containing the `static_key`. +/// * `keytyp` - the type of `key`. +/// * `field` - the name of the field of `key` that contains the `static_key`. +/// +/// # Safety +/// +/// The macro must be used with a real static key defined by C. +#[macro_export] +macro_rules! static_key_false { + ($key:path, $keytyp:ty, $field:ident) => {{ + let _key: *const $keytyp = ::core::ptr::addr_of!($key); + let _key: *const $crate::bindings::static_key = ::core::ptr::addr_of!((*_key).$field); + + $crate::bindings::static_key_count(_key.cast_mut()) > 0 + }}; +} +pub use static_key_false; diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 274bdc1b0a82..91af9f75d121 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -36,6 +36,7 @@ pub mod firmware; pub mod init; pub mod ioctl; +pub mod jump_label; #[cfg(CONFIG_KUNIT)] pub mod kunit; #[cfg(CONFIG_NET)] From patchwork Thu Aug 22 12:04:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13773273 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01A771C9DD2 for ; Thu, 22 Aug 2024 12:04:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328298; cv=none; b=iVsFfQ9Aw//aTn+900SbQlTaoqKIGBEqeUjHrTXhC+nARzjbnak3yJfWUM73e/+9iQSHxCnx8/dLKePfwbhXuFUYchA0a0LuvYxSu4xqmyT+eWbaffFSyTfX3BkNOF6DkITTGXH5kKJzmgUWchK54sIxWUUH4SPamMsWumxRNww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328298; c=relaxed/simple; bh=mbr+t2kdtAR0gaGGgahPyE7BcbfXEyne0mHE/YfAoMM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VW5o676gkPKYwEqHEaGBkBPckq81IqbxCL9zvY6XM/WQomXO3Um2ICh4UkqScqTFqHtCGHp/DzV+dmnBGcbg4LZLLurvJbyiWJUBqqAL2Mda6AgX6zGp+cLSuDLYJmMp4d+B164DOWUlWVZxIKejQs9IZ4C3//xNFeisWU4NQ0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LiOqnOD/; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LiOqnOD/" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6ad9ffa0d77so16645797b3.2 for ; Thu, 22 Aug 2024 05:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724328295; x=1724933095; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=f8cDHUaReumSMCOwyu04+mqYJDxHlzEMGiEzz4YHS0I=; b=LiOqnOD/8TTEXUz1RJt+FUul9xmT1r8e+a7RJwF1WiFzeTuHLNlpGriokh5pG5BiZ6 c4Ycc4Zge5Dgcg4Oxh1CwvJK2ybUcsk9lCWJ9wsZdJlbxcs3Du0ya3wa25slVY4J0kAq bN7tZW3xoJdSKwgmvXV244Z4akubn2KcMwQLq1gMGkjdZFv+ooC5H4ubHBJECaZ2aS5t yTqdUX7+FPgu5JxZ2AbuHREh88cbk14te1TvIOdaa4XxQzzPezecyGCnQ6dJuXC2/j8m Gsb1SEVwnywQ5zFbhA7bNuFs+XZAIuSMKoWTodr6A2nwHh2wNVeiZSV7hle6ziwRrHtr yT4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724328295; x=1724933095; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=f8cDHUaReumSMCOwyu04+mqYJDxHlzEMGiEzz4YHS0I=; b=C8YAoAem9zFEVE79ch7hU2i8AazdHqS4QV8X3TTeTVKmWya4vLpNYmHItodWrH9nO3 z5dkpkBwHkucKZLYXWUFUMYcSYDxDkmC8vfszw2YDSk2x5q9x5N/+AukPpoJRboXKeds WiD1pT/ZkAbn5DSzEYbBthrmKb4xrYU+PRZmp69OaVkKjOI1W+MxybwpW+PwJgl2mJP6 MiHnoO5y7t+hsSRyEJL40d+K/bTQ2/bNbV25z0/ArCFyCscDsG8uZyX03XaUVxvo87IN aou39/Au9nv8aQ1vK2SRvyCew7odqXmaqiFSPLlHhJvAUqya1vjcBqLIfuHDm8ccFMIK e+NA== X-Gm-Message-State: AOJu0YxsHtNX2UjeGLt+Mro+c7SKhffYrBQcoefvdycQ6f+4W204iYvx IzR/nv+ORjwUOZFkuQgp2exB00k9J3Jad02HCF5Ckbjdt7m5Ut8myyqhTo48Sy4FAR0Y5lWYudp xmzuuB74/i1kvBQ== X-Google-Smtp-Source: AGHT+IFS8l23pd6XKyyBpjZiibiQJuE/dGIDebXAym+Zq17SC6CU8fu8OUtqV4IA2VQql6cypuUkw5hRlPx/fCk= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:690c:3683:b0:62f:1f63:ae4f with SMTP id 00721157ae682-6c09ba0765fmr3529837b3.1.1724328294934; Thu, 22 Aug 2024 05:04:54 -0700 (PDT) Date: Thu, 22 Aug 2024 12:04:14 +0000 In-Reply-To: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=6866; i=aliceryhl@google.com; h=from:subject:message-id; bh=mbr+t2kdtAR0gaGGgahPyE7BcbfXEyne0mHE/YfAoMM=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBmxyldgT9/w/ZnIezX+YKTsIh4Z5lhApHS8iZgy GDuXKIjQROJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZscpXQAKCRAEWL7uWMY5 Ri7YD/0ZsdMONo37tyeCBN+jKz/68J+eujSawLXMswiPYdTohvjITOoKtaLFazXZVIPOFfxDqrO MnNH9Nu53HzI5zwZy5mFohAdI2DyHlD3epSshGT9WBTbAhkBhQtppAStZiGcKhgg4pPuAVs7YB3 b7NfaOR0ZAw+dx6u0ogwlrFB9HeVcrm8R0G5kFpwjGJnCgzWWh/VnTK4j+EINoBxBr82tHq3FSq DZq92HK3Q5sDkph8Z43wEhXzYUznw0mFas3+1Wj3aTgxlBYTvzAdCQdfpQN5+Hom37p8BNfJM7D OLN9kPzMmDEUV4asCHvO748lZ4FUAFQTYTsLFLz4G3kEJWxhmjd1ratMl1+YShBy1p5pI/K5qy9 NO/C1gtukYu+zNURzZA5ssWdj+TLWVFEvfuKS7KK46XkbFn0o0PBn7yJ16z7kfLRRah+y/nNHkP kwg/SvSiYZ4a2dKhWmcpGKzOs4BGYHTEluxQaxUT2EINxWOAO/u1HqWxyvK+C4KDVUGJdUVu0Er 0fBQQXZZnBWp63IQDfkQxt/UXIhGxZpb/+kEVlout+lLTVKx3uXKonl/YvbVyFPe2gLdAyX41Yb V9KNcoRaP8OKgE3RDT61a4JM3IFQA5/x7xNShkujV0gaMl2g3SLuTih4eBQyBEEsU76DrPZF7w1 3EbN95Wt07MWnKQ== X-Mailer: b4 0.13.0 Message-ID: <20240822-tracepoint-v8-2-f0c5899e6fd3@google.com> Subject: [PATCH v8 2/5] rust: add tracepoint support From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl , Carlos Llamas Make it possible to have Rust code call into tracepoints defined by C code. It is still required that the tracepoint is declared in a C header, and that this header is included in the input to bindgen. Instead of calling __DO_TRACE directly, the exported rust_do_trace_ function calls an inline helper function. This is because the `cond` argument does not exist at the callsite of DEFINE_RUST_DO_TRACE. __DECLARE_TRACE always emits an inline static and an extern declaration that is only used when CREATE_RUST_TRACE_POINTS is set. These should not end up in the final binary so it is not a problem that they sometimes are emitted without a user. Reviewed-by: Carlos Llamas Reviewed-by: Gary Guo Signed-off-by: Alice Ryhl Reviewed-by: Boqun Feng --- include/linux/tracepoint.h | 22 +++++++++++++++++- include/trace/define_trace.h | 12 ++++++++++ rust/bindings/bindings_helper.h | 1 + rust/kernel/lib.rs | 1 + rust/kernel/tracepoint.rs | 49 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 84 insertions(+), 1 deletion(-) diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h index 6be396bb4297..5042ca588e41 100644 --- a/include/linux/tracepoint.h +++ b/include/linux/tracepoint.h @@ -237,6 +237,18 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) #define __DECLARE_TRACE_RCU(name, proto, args, cond) #endif +/* + * Declare an exported function that Rust code can call to trigger this + * tracepoint. This function does not include the static branch; that is done + * in Rust to avoid a function call when the tracepoint is disabled. + */ +#define DEFINE_RUST_DO_TRACE(name, proto, args) +#define __DEFINE_RUST_DO_TRACE(name, proto, args) \ + notrace void rust_do_trace_##name(proto) \ + { \ + __rust_do_trace_##name(args); \ + } + /* * Make sure the alignment of the structure in the __tracepoints section will * not add unwanted padding between the beginning of the section and the @@ -252,6 +264,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) extern int __traceiter_##name(data_proto); \ DECLARE_STATIC_CALL(tp_func_##name, __traceiter_##name); \ extern struct tracepoint __tracepoint_##name; \ + extern void rust_do_trace_##name(proto); \ + static inline void __rust_do_trace_##name(proto) \ + { \ + __DO_TRACE(name, \ + TP_ARGS(args), \ + TP_CONDITION(cond), 0); \ + } \ static inline void trace_##name(proto) \ { \ if (static_key_false(&__tracepoint_##name.key)) \ @@ -336,7 +355,8 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) void __probestub_##_name(void *__data, proto) \ { \ } \ - DEFINE_STATIC_CALL(tp_func_##_name, __traceiter_##_name); + DEFINE_STATIC_CALL(tp_func_##_name, __traceiter_##_name); \ + DEFINE_RUST_DO_TRACE(_name, TP_PROTO(proto), TP_ARGS(args)) #define DEFINE_TRACE(name, proto, args) \ DEFINE_TRACE_FN(name, NULL, NULL, PARAMS(proto), PARAMS(args)); diff --git a/include/trace/define_trace.h b/include/trace/define_trace.h index 00723935dcc7..8159294c2041 100644 --- a/include/trace/define_trace.h +++ b/include/trace/define_trace.h @@ -72,6 +72,13 @@ #define DECLARE_TRACE(name, proto, args) \ DEFINE_TRACE(name, PARAMS(proto), PARAMS(args)) +/* If requested, create helpers for calling these tracepoints from Rust. */ +#ifdef CREATE_RUST_TRACE_POINTS +#undef DEFINE_RUST_DO_TRACE +#define DEFINE_RUST_DO_TRACE(name, proto, args) \ + __DEFINE_RUST_DO_TRACE(name, PARAMS(proto), PARAMS(args)) +#endif + #undef TRACE_INCLUDE #undef __TRACE_INCLUDE @@ -129,6 +136,11 @@ # undef UNDEF_TRACE_INCLUDE_PATH #endif +#ifdef CREATE_RUST_TRACE_POINTS +# undef DEFINE_RUST_DO_TRACE +# define DEFINE_RUST_DO_TRACE(name, proto, args) +#endif + /* We may be processing more files */ #define CREATE_TRACE_POINTS diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index 8fd092e1b809..fc6f94729789 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -20,6 +20,7 @@ #include #include #include +#include #include #include diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 91af9f75d121..d00a44b000b6 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -51,6 +51,7 @@ pub mod sync; pub mod task; pub mod time; +pub mod tracepoint; pub mod types; pub mod uaccess; pub mod workqueue; diff --git a/rust/kernel/tracepoint.rs b/rust/kernel/tracepoint.rs new file mode 100644 index 000000000000..cf2d9ad15912 --- /dev/null +++ b/rust/kernel/tracepoint.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Logic for tracepoints. + +/// Declare the Rust entry point for a tracepoint. +/// +/// This macro generates an unsafe function that calls into C, and its safety requirements will be +/// whatever the relevant C code requires. To document these safety requirements, you may add +/// doc-comments when invoking the macro. +#[macro_export] +macro_rules! declare_trace { + ($($(#[$attr:meta])* $pub:vis unsafe fn $name:ident($($argname:ident : $argtyp:ty),* $(,)?);)*) => {$( + $( #[$attr] )* + #[inline(always)] + $pub unsafe fn $name($($argname : $argtyp),*) { + #[cfg(CONFIG_TRACEPOINTS)] + { + // SAFETY: It's always okay to query the static key for a tracepoint. + let should_trace = unsafe { + $crate::macros::paste! { + $crate::jump_label::static_key_false!( + $crate::bindings::[< __tracepoint_ $name >], + $crate::bindings::tracepoint, + key + ) + } + }; + + if should_trace { + $crate::macros::paste! { + // SAFETY: The caller guarantees that it is okay to call this tracepoint. + unsafe { $crate::bindings::[< rust_do_trace_ $name >]($($argname),*) }; + } + } + } + + #[cfg(not(CONFIG_TRACEPOINTS))] + { + // If tracepoints are disabled, insert a trivial use of each argument + // to avoid unused argument warnings. + $( let _unused = $argname; )* + } + } + )*} +} + +pub use declare_trace; From patchwork Thu Aug 22 12:04:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13773274 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DEF11C9EAC for ; Thu, 22 Aug 2024 12:04:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328300; cv=none; b=qq6427rWI9EMQ9yqgGU++pu6W7chr8ZehuB8IaE4J1mQoXMg9xdXuX1N9AyizMZP8wpb9Dec2zXsZE5fymbPzzLliPdp7pFhNO+0TtaB318tnnLA1MQg4oR5Y087+HmAnI2eDDygNDfcqUiVPf3y52A1b2Tz8PjPSFf0DOVjgr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328300; c=relaxed/simple; bh=FjpJXotW1KGtmig+hIm0KQivxPyDztMFhKa3lXhpNM0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IPz1Ht4cW0P6vCUHfsTKrmDjaR1N4Lv/tuvm5ntCewq4tAcAV/TcSX9YIfN7q3MfwSVpiBeuLZnLIKeP6juHyo15nQzDcOyRdWOXNcdaD2m8ku9gFweVyMKpPFFRQ4N1aWsV/uorjwtECX78F9gFa/wID59X56CUFdunlryENa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ad6h3hYG; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ad6h3hYG" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6b71aa9349dso15126597b3.1 for ; Thu, 22 Aug 2024 05:04:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724328298; x=1724933098; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8WXJYVeFW+jGG7DCrz93gCg6JPKvPQ2Wf4cfjKgCwaM=; b=ad6h3hYGznuRmOw22l2BRUiE1CHgMA9crBP+Q+hkqQRZ0ZgtORye1adQZYlXmHktCP hqAnmPi87bzj3bOVN4D3QHJRgadGKsZ6JHa1G5yJPtvehh4aKmfMGeY17+YmRVRaReEH zEU1MFxSEBuPi7DU+Rm9eW6rYjSM8ZXo967ewBPyd9OAsyrcDbW4QlREqXOAWRds8Wjn +IAa6Qs3lX3y0qnv3gJwjzMtsO0/CsOkR7IAvTccjFQ9r9lmz6UHBRlbAMW/VGTHN5yd pbBcs+m9mcwulT0qp2edFHLXGe+X0lrVlDjfSMQVPCR/x2oNNGKUBJFYPFFI4t9jHGhS g9Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724328298; x=1724933098; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8WXJYVeFW+jGG7DCrz93gCg6JPKvPQ2Wf4cfjKgCwaM=; b=SruBu7IXYBPdIIkFzg0H2TVa3dbhnzEdIslvYT0GGMMSMLBjIxksUdHXYMgivJFjgp Wp6AlQJc5BoS8lneP+rhm579yRs6I7/kxtdmr816mJxCSD0UQwyTgv6pMbcczxjJZvvf 3hs15C2iY4RNZDbUYPFHgwo4HHHGlP6TrLg+bPNToPsm1XcyZAMWv2bENtk/+MrAVbyE tI7KqAq6f6t8HbLA9kDMBClC25RMOnO/5B5fBRTj+Iwo8iOtKJBT5mS1psYn3zn5L2xA mIeTCsWXTvYQQv1+DIZeBuOLwV5NGikkcXHix7WSGSn303AnIQlv78xer9rwirnYTiKu iPrQ== X-Gm-Message-State: AOJu0YyItJ+e6AErqiAc2x2KoO7CHIbFoj7ptKEgOqJFY4z9WZOiKP/i 1M+FhKDUTzHxnC/SttwAQwtQH+KQ89ykX4ahbVO6RfZ75qIW6ltT/MwZE1GoovaelCPQMYaJIZH ChYfhRnN121XzUA== X-Google-Smtp-Source: AGHT+IHOwQ7F4WwpIwCYP14vLZEMvblPZGwmqHarUMkdlvedNsRunUgeqjgc8IGJA2wVHg86ug/U3YKoS2Z6kS8= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a25:7483:0:b0:e11:44fb:af26 with SMTP id 3f1490d57ef6-e166540e1eamr8764276.2.1724328297474; Thu, 22 Aug 2024 05:04:57 -0700 (PDT) Date: Thu, 22 Aug 2024 12:04:15 +0000 In-Reply-To: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=4269; i=aliceryhl@google.com; h=from:subject:message-id; bh=FjpJXotW1KGtmig+hIm0KQivxPyDztMFhKa3lXhpNM0=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBmxyldDFylo0W2FYUlH9zPLgV0aNf5JfdXWxC+k EV6BXx+qxOJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZscpXQAKCRAEWL7uWMY5 RmwSD/9lb6aazG/rJFEouodeNF6AgBtP/doqXucmQsYP5lsbpIKM6wD/RrdAfk8KvahqhtiKNZo M80qvgsWx6kR60U8/3k8BpYVdbSLtWq8iy7kDwbqPsW7/lbh6lSSN7po7WTvUGkLxdy55aBL7Sn pxvZeuKi80CsU5j1xBKwqA64hnBiiXrCmIbUYWMsfWCV1QC7IB3G4FGcWnkuPdSj/Sec7odVug3 UGVoS0jRtXoInadAPmAvJimKkQGrM9XN/1U3Y//9PfoOiMapFuQ1CRn8NPVS3wLdpTesPhUn9u7 eexLr2wbBy043Lp3azw2EeBu0AAzJ5FYxBnV7XKV7V+5VISDYLTQky2gSjsC+RTTVAW0ZxL4ExJ 7tcK/uSNtex2S3P2p73VyR46otDTS3YeIjA7K6J8Cw537m+mpFL257yU0gdDeWj4RBhhIEH6JzO Y+Fi1fjq/r5jug7WVznXBOO0L9iN+HL/NGCTv/tRN5iexYDGVO0utnbT5kTx9B3ePtHUT0FmpGN DxAq0+0gRNwsVdxm4jY7ARuozjUBGvHfrKfTKGfwBzWN+t8wjR5lnxyPtKMfDczcG0A9Ii5xWdu wOQZQ3zt/bMwGS3BgCox0QdlxbK6m1+NaZgBJECLHrZEBYyYJo/0uQgI8Kd415KJtvBEaMpr5xh HbPkWR325Lp7a5w== X-Mailer: b4 0.13.0 Message-ID: <20240822-tracepoint-v8-3-f0c5899e6fd3@google.com> Subject: [PATCH v8 3/5] rust: samples: add tracepoint to Rust sample From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl This updates the Rust printing sample to invoke a tracepoint. This ensures that we have a user in-tree from the get-go even though the patch is being merged before its real user. Signed-off-by: Alice Ryhl Reviewed-by: Boqun Feng --- MAINTAINERS | 1 + include/trace/events/rust_sample.h | 31 +++++++++++++++++++++++++++++++ rust/bindings/bindings_helper.h | 1 + samples/rust/Makefile | 3 ++- samples/rust/rust_print.rs | 18 ++++++++++++++++++ samples/rust/rust_print_events.c | 8 ++++++++ 6 files changed, 61 insertions(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index f328373463b0..1acf5bfddfc4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19922,6 +19922,7 @@ C: zulip://rust-for-linux.zulipchat.com P: https://rust-for-linux.com/contributing T: git https://github.com/Rust-for-Linux/linux.git rust-next F: Documentation/rust/ +F: include/trace/events/rust_sample.h F: rust/ F: samples/rust/ F: scripts/*rust* diff --git a/include/trace/events/rust_sample.h b/include/trace/events/rust_sample.h new file mode 100644 index 000000000000..dbc80ca2e465 --- /dev/null +++ b/include/trace/events/rust_sample.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Tracepoints for `samples/rust/rust_print.rs`. + * + * Copyright (C) 2024 Google, Inc. + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM rust_sample + +#if !defined(_RUST_SAMPLE_TRACE_H) || defined(TRACE_HEADER_MULTI_READ) +#define _RUST_SAMPLE_TRACE_H + +#include + +TRACE_EVENT(rust_sample_loaded, + TP_PROTO(int magic_number), + TP_ARGS(magic_number), + TP_STRUCT__entry( + __field(int, magic_number) + ), + TP_fast_assign( + __entry->magic_number = magic_number; + ), + TP_printk("magic=%d", __entry->magic_number) +); + +#endif /* _RUST_SAMPLE_TRACE_H */ + +/* This part must be outside protection */ +#include diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index fc6f94729789..fe97256afe65 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -23,6 +23,7 @@ #include #include #include +#include /* `bindgen` gets confused at certain things. */ const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN; diff --git a/samples/rust/Makefile b/samples/rust/Makefile index 03086dabbea4..f29280ec4820 100644 --- a/samples/rust/Makefile +++ b/samples/rust/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +ccflags-y += -I$(src) # needed for trace events obj-$(CONFIG_SAMPLE_RUST_MINIMAL) += rust_minimal.o -obj-$(CONFIG_SAMPLE_RUST_PRINT) += rust_print.o +obj-$(CONFIG_SAMPLE_RUST_PRINT) += rust_print.o rust_print_events.o subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS) += hostprogs diff --git a/samples/rust/rust_print.rs b/samples/rust/rust_print.rs index 6eabb0d79ea3..6d14b08cac1c 100644 --- a/samples/rust/rust_print.rs +++ b/samples/rust/rust_print.rs @@ -69,6 +69,8 @@ fn init(_module: &'static ThisModule) -> Result { arc_print()?; + trace::trace_rust_sample_loaded(42); + Ok(RustPrint) } } @@ -78,3 +80,19 @@ fn drop(&mut self) { pr_info!("Rust printing macros sample (exit)\n"); } } + +mod trace { + use core::ffi::c_int; + + kernel::declare_trace! { + /// # Safety + /// + /// Always safe to call. + unsafe fn rust_sample_loaded(magic: c_int); + } + + pub(crate) fn trace_rust_sample_loaded(magic: i32) { + // SAFETY: Always safe to call. + unsafe { rust_sample_loaded(magic as c_int) } + } +} diff --git a/samples/rust/rust_print_events.c b/samples/rust/rust_print_events.c new file mode 100644 index 000000000000..a9169ff0edf1 --- /dev/null +++ b/samples/rust/rust_print_events.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2024 Google LLC + */ + +#define CREATE_TRACE_POINTS +#define CREATE_RUST_TRACE_POINTS +#include From patchwork Thu Aug 22 12:04:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13773275 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3D081C9435 for ; Thu, 22 Aug 2024 12:05:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328305; cv=none; b=EOrw1ohuSo7hidinZg1ETPTDWukXOdmBxgpxAK0ngen7LTifApV8zifBan94YuMCzqCruyhpB6Ar99Z1LUY4ZsP9k8YnAO8V26J5HvNrpVUbONMLdPJl5qdFf14cyppBaXMgc3HtokyAT1cgqrWSF5DsxU83Oxz4GGxTBXZ9Fow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328305; c=relaxed/simple; bh=NCijnrFVATFS0Jb3tHGcAM6xeWG4d78Efm1z0mpVoWk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MPtxR3PA6qzGl1KTBnRYBx825W4pp4yjL1Ie0ZGRxthSaCFpDsiDxEEyQa5Qps6dl+vFPvPDCZR1tdTO09/VIdRtyvHUEtv32Ab+YYM82Nh6c8jmT5Nh8LnxQ9H6WjdqTYh8jejFTBTB9Q3MsfATmISKVJVaUAqHHIv1EWZyn0A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NbRQxV0N; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NbRQxV0N" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-371ab0c02e0so531285f8f.3 for ; Thu, 22 Aug 2024 05:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724328300; x=1724933100; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Q375QRvUD5Wva8hc3MN+sMayJuBnYHhQhae9RWdaskQ=; b=NbRQxV0NPteo9j6mMUKMFloL0Hv3V3vQhwltcq8cXQwBqA6sq14hY2oxfhYlAmlcPb emTPEcg15x+XVom5WhZ8TBX8V/1nDSOTJDWhczg2UTleSPifqF+aml1A1inL2ymZ8qDz 3fMVdXhoer9QAQ3sJE9MXn3o2QsDS192u6jc/vQLsu480RFDHLEK5fpLVWx12ULTRN38 Z4kLdq9UAwbUmcUeahnmmTj7YL1tNk5yTus/JAON8/AE+uOnlU/S7WsfLO5P6QzytAPu P0GbJgFP77S6H0HzcR+TEgx2MS+auOB1RC3owVePIZB4kp437z9AuI7bBWqbVcSv7AzT jHEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724328300; x=1724933100; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Q375QRvUD5Wva8hc3MN+sMayJuBnYHhQhae9RWdaskQ=; b=e2dM65lQkKFMijfJGLvlYm0Ivq0hhw1UxaCukt9QjsvqH6Ot56KJsfJ7mDmfGOSm8J XgD+aEgYI5NtetBsCUrHyY4y2EpryFgjymQ1BmqrV5Nxt2BsXrylSV7V03rvgXgE23W8 +BK9sv9QSgzvkro2/KA0VvD+KcSBIkhxM3QYGZGJuokmeH163YDNTIR78ov0tQktHS+w 3yaB61X3uNkHZxJF/53XkLo0gk2JhBrWfw+9HVstR8eIohubBU0uR8HFgEKRGhqTU9rL l9+lyCNq0rQuRnXnv6Cr3MlwdoxrsDqJv/U2RQeJY5l7HBrry/wgoTY78MWS4YfrhSBz oCAw== X-Gm-Message-State: AOJu0YzbPCCRzGJMMhP0qIG4vi+zXGPvbyCKGuRxq7WvbFwZXeh9I37y VP9ixITleniwvpAW5512xic996DGqSnNOLgH6L+Eosr1cbT47QuIMuYOTlgTnvV2XHifbs1hAnN ISk+Htg/yNu1Dtg== X-Google-Smtp-Source: AGHT+IGw3yfAzaYcyD6d0zmHCHRXiA2QSLiNwuvY02gaan1SLSwcPyY0PwxfH/wKsRMOPzJoAId5eBQhi7+XpwQ= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a5d:5392:0:b0:368:371b:3ae with SMTP id ffacd0b85a97d-372fd5902c7mr8464f8f.6.1724328300071; Thu, 22 Aug 2024 05:05:00 -0700 (PDT) Date: Thu, 22 Aug 2024 12:04:16 +0000 In-Reply-To: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=9372; i=aliceryhl@google.com; h=from:subject:message-id; bh=NCijnrFVATFS0Jb3tHGcAM6xeWG4d78Efm1z0mpVoWk=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBmxylehr+vvDlXsGVX+QAyOJmDHQJ8dueLfCPb0 1aADq7Xg3SJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZscpXgAKCRAEWL7uWMY5 RvEiD/9pK5VDC1VlIuqdn/1xgy7FKaKXzMAuZ9DX+dqwtjqh8kmF6asI9lg7b+zWAu71c/J/CJC RSaScqbPYPJ+HvpKfEqipatv8FovDMOM7YowcspUQUAT4CcPlQQ3vTdZfdUVD8/c/hfsWgFz6WI OAB231FNblq1h7IKhnXwNpbb23O9QwyDg7EaYpN3jYLa/oCFvn3O4gedIPo+d6vylLIeVej0+7b OROXwoSwxvIsMFL+e8v01YwJbYe/uwRS/kkf2LWxDwCv6leShBLhXCY/kDSd0Nxjv8hbdiyB0ef D26TdMF1EAvU73JkPQWcsvXpkbN9+SPqAV9X4yfhFur4M7AK8m+gCFr6ecVj4Yb9MMmgt6abyZu 1Rub67vPI8oa3OrU7+lR0xlqX9s0PDM70OkEFAN7q5V4lC+TnQyhmKwu/8HGr39I5KdIyEbPSCC v1+4BItg3uGOFFoGZidk6zVDoQQP+e/wIwXK9u0PYMpdfNQ88o75b8J/tK1sytl9045RqV1WZKx mWy6QuM7Fdj5xdBZ7uO6rQDeySO8zBPHGhvrR/qq6LzdwKEi3zt8D31LO2OJhOWCKfbn/BbdDQp ruagtYvfR/vABAefkqf9zlG1Lr2e2Xohy2qpe8HBXL5YociYOJ5HIffB6gWsGenNKX3SwQZG0Q2 bwo4NPnXyRTaPIg== X-Mailer: b4 0.13.0 Message-ID: <20240822-tracepoint-v8-4-f0c5899e6fd3@google.com> Subject: [PATCH v8 4/5] jump_label: adjust inline asm to be consistent From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl To avoid duplication of inline asm between C and Rust, we need to import the inline asm from the relevant `jump_label.h` header into Rust. To make that easier, this patch updates the header files to expose the inline asm via a new ARCH_STATIC_BRANCH_ASM macro. The header files are all updated to define a ARCH_STATIC_BRANCH_ASM that takes the same arguments in a consistent order so that Rust can use the same logic for every architecture. Suggested-by: Peter Zijlstra (Intel) Acked-by: Peter Zijlstra (Intel) Co-developed-by: Miguel Ojeda Signed-off-by: Miguel Ojeda Signed-off-by: Alice Ryhl --- arch/arm/include/asm/jump_label.h | 14 +++++---- arch/arm64/include/asm/jump_label.h | 20 ++++++++----- arch/loongarch/include/asm/jump_label.h | 16 +++++++---- arch/riscv/include/asm/jump_label.h | 50 ++++++++++++++++++--------------- arch/x86/include/asm/jump_label.h | 38 ++++++++++--------------- 5 files changed, 75 insertions(+), 63 deletions(-) diff --git a/arch/arm/include/asm/jump_label.h b/arch/arm/include/asm/jump_label.h index e4eb54f6cd9f..a35aba7f548c 100644 --- a/arch/arm/include/asm/jump_label.h +++ b/arch/arm/include/asm/jump_label.h @@ -9,13 +9,17 @@ #define JUMP_LABEL_NOP_SIZE 4 +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1:\n\t" \ + WASM(nop) "\n\t" \ + ".pushsection __jump_table, \"aw\"\n\t" \ + ".word 1b, " label ", " key "\n\t" \ + ".popsection\n\t" \ + static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { - asm goto("1:\n\t" - WASM(nop) "\n\t" - ".pushsection __jump_table, \"aw\"\n\t" - ".word 1b, %l[l_yes], %c0\n\t" - ".popsection\n\t" + asm goto(ARCH_STATIC_BRANCH_ASM("%c0", "%l[l_yes]") : : "i" (&((char *)key)[branch]) : : l_yes); return false; diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h index a0a5bbae7229..424ed421cd97 100644 --- a/arch/arm64/include/asm/jump_label.h +++ b/arch/arm64/include/asm/jump_label.h @@ -19,10 +19,14 @@ #define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\"\n\t" \ ".align 3\n\t" \ - ".long 1b - ., %l["#label"] - .\n\t" \ - ".quad %c0 - .\n\t" \ - ".popsection\n\t" \ - : : "i"(key) : : label + ".long 1b - ., " label " - .\n\t" \ + ".quad " key " - .\n\t" \ + ".popsection\n\t" + +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: nop\n\t" \ + JUMP_TABLE_ENTRY(key, label) static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) @@ -30,8 +34,8 @@ static __always_inline bool arch_static_branch(struct static_key * const key, char *k = &((char *)key)[branch]; asm goto( - "1: nop \n\t" - JUMP_TABLE_ENTRY(k, l_yes) + ARCH_STATIC_BRANCH_ASM("%c0", "%l[l_yes]") + : : "i"(k) : : l_yes ); return false; @@ -43,9 +47,11 @@ static __always_inline bool arch_static_branch_jump(struct static_key * const ke const bool branch) { char *k = &((char *)key)[branch]; + asm goto( "1: b %l[l_yes] \n\t" - JUMP_TABLE_ENTRY(k, l_yes) + JUMP_TABLE_ENTRY("%c0", "%l[l_yes]") + : : "i"(k) : : l_yes ); return false; l_yes: diff --git a/arch/loongarch/include/asm/jump_label.h b/arch/loongarch/include/asm/jump_label.h index 29acfe3de3fa..8a924bd69d19 100644 --- a/arch/loongarch/include/asm/jump_label.h +++ b/arch/loongarch/include/asm/jump_label.h @@ -13,18 +13,22 @@ #define JUMP_LABEL_NOP_SIZE 4 -#define JUMP_TABLE_ENTRY \ +/* This macro is also expanded on the Rust side. */ +#define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\" \n\t" \ ".align 3 \n\t" \ - ".long 1b - ., %l[l_yes] - . \n\t" \ - ".quad %0 - . \n\t" \ + ".long 1b - ., " label " - . \n\t" \ + ".quad " key " - . \n\t" \ ".popsection \n\t" +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: nop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { asm goto( - "1: nop \n\t" - JUMP_TABLE_ENTRY + ARCH_STATIC_BRANCH_ASM("%0", "%l[l_yes]") : : "i"(&((char *)key)[branch]) : : l_yes); return false; @@ -37,7 +41,7 @@ static __always_inline bool arch_static_branch_jump(struct static_key * const ke { asm goto( "1: b %l[l_yes] \n\t" - JUMP_TABLE_ENTRY + JUMP_TABLE_ENTRY("%0", "%l[l_yes]") : : "i"(&((char *)key)[branch]) : : l_yes); return false; diff --git a/arch/riscv/include/asm/jump_label.h b/arch/riscv/include/asm/jump_label.h index 1c768d02bd0c..87a71cc6d146 100644 --- a/arch/riscv/include/asm/jump_label.h +++ b/arch/riscv/include/asm/jump_label.h @@ -16,21 +16,28 @@ #define JUMP_LABEL_NOP_SIZE 4 +#define JUMP_TABLE_ENTRY(key, label) \ + ".pushsection __jump_table, \"aw\" \n\t" \ + ".align " RISCV_LGPTR " \n\t" \ + ".long 1b - ., " label " - . \n\t" \ + "" RISCV_PTR " " key " - . \n\t" \ + ".popsection \n\t" + +/* This macro is also expanded on the Rust side. */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + " .align 2 \n\t" \ + " .option push \n\t" \ + " .option norelax \n\t" \ + " .option norvc \n\t" \ + "1: nop \n\t" \ + " .option pop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { asm goto( - " .align 2 \n\t" - " .option push \n\t" - " .option norelax \n\t" - " .option norvc \n\t" - "1: nop \n\t" - " .option pop \n\t" - " .pushsection __jump_table, \"aw\" \n\t" - " .align " RISCV_LGPTR " \n\t" - " .long 1b - ., %l[label] - . \n\t" - " " RISCV_PTR " %0 - . \n\t" - " .popsection \n\t" + ARCH_STATIC_BRANCH_ASM("%0", "%l[label]") : : "i"(&((char *)key)[branch]) : : label); return false; @@ -38,21 +45,20 @@ static __always_inline bool arch_static_branch(struct static_key * const key, return true; } +#define ARCH_STATIC_BRANCH_JUMP_ASM(key, label) \ + " .align 2 \n\t" \ + " .option push \n\t" \ + " .option norelax \n\t" \ + " .option norvc \n\t" \ + "1: j " label " \n\t" \ + " .option pop \n\t" \ + JUMP_TABLE_ENTRY(key, label) + static __always_inline bool arch_static_branch_jump(struct static_key * const key, const bool branch) { asm goto( - " .align 2 \n\t" - " .option push \n\t" - " .option norelax \n\t" - " .option norvc \n\t" - "1: j %l[label] \n\t" - " .option pop \n\t" - " .pushsection __jump_table, \"aw\" \n\t" - " .align " RISCV_LGPTR " \n\t" - " .long 1b - ., %l[label] - . \n\t" - " " RISCV_PTR " %0 - . \n\t" - " .popsection \n\t" + ARCH_STATIC_BRANCH_JUMP_ASM("%0", "%l[label]") : : "i"(&((char *)key)[branch]) : : label); return false; diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h index cbbef32517f0..fb79fa1cf70a 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -12,49 +12,41 @@ #include #include -#define JUMP_TABLE_ENTRY \ +#define JUMP_TABLE_ENTRY(key, label) \ ".pushsection __jump_table, \"aw\" \n\t" \ _ASM_ALIGN "\n\t" \ ".long 1b - . \n\t" \ - ".long %l[l_yes] - . \n\t" \ - _ASM_PTR "%c0 + %c1 - .\n\t" \ + ".long " label " - . \n\t" \ + _ASM_PTR " " key " - . \n\t" \ ".popsection \n\t" +/* This macro is also expanded on the Rust side. */ #ifdef CONFIG_HAVE_JUMP_LABEL_HACK - -static __always_inline bool arch_static_branch(struct static_key *key, bool branch) -{ - asm goto("1:" - "jmp %l[l_yes] # objtool NOPs this \n\t" - JUMP_TABLE_ENTRY - : : "i" (key), "i" (2 | branch) : : l_yes); - - return false; -l_yes: - return true; -} - +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: jmp " label " # objtool NOPs this \n\t" \ + JUMP_TABLE_ENTRY(key, label) #else /* !CONFIG_HAVE_JUMP_LABEL_HACK */ +#define ARCH_STATIC_BRANCH_ASM(key, label) \ + "1: .byte " __stringify(BYTES_NOP5) "\n\t" \ + JUMP_TABLE_ENTRY(key, label) +#endif /* CONFIG_HAVE_JUMP_LABEL_HACK */ static __always_inline bool arch_static_branch(struct static_key * const key, const bool branch) { - asm goto("1:" - ".byte " __stringify(BYTES_NOP5) "\n\t" - JUMP_TABLE_ENTRY - : : "i" (key), "i" (branch) : : l_yes); + int hack_bit = IS_ENABLED(CONFIG_HAVE_JUMP_LABEL_HACK) ? 2 : 0; + asm goto(ARCH_STATIC_BRANCH_ASM("%c0 + %c1", "%l[l_yes]") + : : "i" (key), "i" (hack_bit | branch) : : l_yes); return false; l_yes: return true; } -#endif /* CONFIG_HAVE_JUMP_LABEL_HACK */ - static __always_inline bool arch_static_branch_jump(struct static_key * const key, const bool branch) { asm goto("1:" "jmp %l[l_yes]\n\t" - JUMP_TABLE_ENTRY + JUMP_TABLE_ENTRY("%c0 + %c1", "%l[l_yes]") : : "i" (key), "i" (branch) : : l_yes); return false; From patchwork Thu Aug 22 12:04:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alice Ryhl X-Patchwork-Id: 13773276 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D1E41CB12F for ; Thu, 22 Aug 2024 12:05:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328305; cv=none; b=tKgCZ0V2fZnBCN6kvvSQm5gfljCcNOlpPh2w/5o0ozXP8Q5VWyd8HWb7uEOuBOGKkArbTyfcdZeRbLZ3U4cXce0yh+meUf+FnZnV4GuRylWx4dgcoc8f7kyJe/lzKq7V4x84w2BmogdCWSWUqp3Jcd++KsXtPlmvew8GkFIuBYo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724328305; c=relaxed/simple; bh=od1Acum7ZP3SHNuZV1Frd7Px5i5rAfWdJgysjyMhaoo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oyHXKUkSENyX+erxUnH7X1BO5jJxB5IzquAZeXlhYNrvHOAeJzvX/PVM4KXY/PlmiNOkm2irVZk0B+y/96gDzy+4Xe+EMXlcXB+L1a99lfYjxQCv29cMRZvIfsgAHfEdsrCRQeT3U8mzz/kWGxl5tR3rpXZPc0e5srD3RkCy/Ds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=R99rTTDK; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="R99rTTDK" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6b3fec974e5so14410797b3.1 for ; Thu, 22 Aug 2024 05:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724328303; x=1724933103; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uxEAzZMzJr3y8oSKmglTMtntRHzNRdKVl0l9n/9vlWE=; b=R99rTTDKAFanHgBGddQhFqxBo8W/3/SvpJW6tfqJWHPI3o0XpmIR9lL0e9JH6msPoZ RYz/e1hT5qmnkm+KTm1NPRLhJ7wIShJ7bsujXSSlBEP1nt2t1aK0wduOy/Ror99CE7gE 21CjUBuuT54jdu+F0zRulS4075uIFBHh/eNNAw6wHeP4Npy2IZa3M1NTa/5CHBcOZ6kM SJHA5D3JgM/f+NS0DYZDE1qaMgzSWMYnG/ZpSWu0o14AulARepu0lFz/0QTIgEO08rAs zOJDZfQ/Gi7djXT2lSGqiH/lt8XjQVI+3IU0cqM70IWEfNPCUR4irJmP0wfhygWvZJ/K ngoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724328303; x=1724933103; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uxEAzZMzJr3y8oSKmglTMtntRHzNRdKVl0l9n/9vlWE=; b=p8EC+p4S0DesYm1x86n6/nb2rzWdGODwZ9TmDk51LyWVgiFatSNmqA9CelcX69rLxc 6hUsO5zhrl5GaElv8ZXnwird4du4yfIJXdLWjljJajb3xW0xRsSg4hXaqRuHoGRsM9BN 8K5E4dfspJ3nGcZe0Fqxy5cHNoaYwT10iVqDfZvrMB9ISni05+e/Z/55FBI6ecEfrcux azYqVnVDjUjVXOAaB5iGaVlxjoTQAuWL+nBHpipFHVbi2+gpgOJ69mH01yaO/VZ1SFeY EK76cw4cqJEAFYY+B6SG6GAVCeDyAmEWsO4ANqeHyuN0JSYfPmaYUBoKnEvvUejOsLo+ ReQw== X-Gm-Message-State: AOJu0YwpUXxlOMEuNBDu0Knkojzz6soRNHyVGBj6KMLvWCXXRbgmn/aH f2H8MHJbN54DqvNw0dU+IjG+14/OqOfK6JXyh67jWDAPOmd/g9iqlWVzwuWc/IN6KC0Pmu13VC+ vmEekZ4m14rYgLQ== X-Google-Smtp-Source: AGHT+IGiVlbAew1+icwXyROm+LUwKLbHv3EgJ+ELhtPSIkQV+jPl3FGLr+njU0tLgnmjraKSgCDY+G6v6Qi4DsI= X-Received: from aliceryhl.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:35bd]) (user=aliceryhl job=sendgmr) by 2002:a05:690c:4813:b0:6be:9d4a:f097 with SMTP id 00721157ae682-6c0a0236c74mr298177b3.7.1724328302925; Thu, 22 Aug 2024 05:05:02 -0700 (PDT) Date: Thu, 22 Aug 2024 12:04:17 +0000 In-Reply-To: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240822-tracepoint-v8-0-f0c5899e6fd3@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=8506; i=aliceryhl@google.com; h=from:subject:message-id; bh=od1Acum7ZP3SHNuZV1Frd7Px5i5rAfWdJgysjyMhaoo=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBmxyleWMqafxpffJQSjGJQbTSeoPMOs5rqYzOKA h7koJQoGQSJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCZscpXgAKCRAEWL7uWMY5 RshoD/9FmNAsAln8K8mfbgINydaroarjZhGmYp77z+tBgBU+vnZq3W+3YIcpqwRKHl8Gi3mFOVb OW5B9NMhyyhupwWgDELW5cgPZPUAT00yM0kK+sIVmpV9cxS3lbtT+5thF+ONcWTby43Cywgc6DX Dv2xq3ZdUBFAiUcVz1wy8EQHE8llFacx/qswJVcnRAjlJQKb26/jKadb1BpBYh0SA9yBFxzyHso NZHN/bTSJJ7VR8QjiTT1Y59WzjhkjR3/B3JcgHs8TAnOGP/kdDuldwkkhveUQBkwLNI0zpthfQs mstX0wkLGbi0K7xXR75OHisSb98lx6kM48LbFLpjM3+ZL+tGcqNtOOTubwbHza6j4GBuzDjots7 i+gHzc80Q9z/Sal6mV/oQ8tgOOTBDldQXlaLbxLTG1oFfgRKSXRoYaFowmf0myQZiXSa+nWDfJs E/JYPRErb801FL/Jfu+42W89+vTpAHYEC8UW1uiVKXgONBT1Z3BMYWxInklTnTjGC6AWB38SmMV Surujt2TeSla20M8LkJfsq8mjId+E8sGO3/Fij807tf/blvZ5Kdh1kDpGRyHqELlLutojpxRaja geMR+csmMbMbDT3SggRsWo0zAG2mCRSXS1PXSJJoKpyvfmATpigqjksoCY3eK0alhr4mQL5zNrk Gn0aY4b7TsoYfLw== X-Mailer: b4 0.13.0 Message-ID: <20240822-tracepoint-v8-5-f0c5899e6fd3@google.com> Subject: [PATCH v8 5/5] rust: add arch_static_branch From: Alice Ryhl To: Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Peter Zijlstra , Josh Poimboeuf , Jason Baron , Ard Biesheuvel , Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Boqun Feng , Gary Guo , " =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= " , Benno Lossin , Andreas Hindborg Cc: linux-trace-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Arnd Bergmann , linux-arch@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Sean Christopherson , Uros Bizjak , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mark Rutland , Ryan Roberts , Fuad Tabba , linux-arm-kernel@lists.infradead.org, Paul Walmsley , Palmer Dabbelt , Albert Ou , Anup Patel , Andrew Jones , Alexandre Ghiti , Conor Dooley , Samuel Holland , linux-riscv@lists.infradead.org, Huacai Chen , WANG Xuerui , Bibo Mao , Tiezhu Yang , Andrew Morton , Tianrui Zhao , loongarch@lists.linux.dev, Alice Ryhl To allow the Rust implementation of static_key_false to use runtime code patching instead of the generic implementation, pull in the relevant inline assembly from the jump_label.h header by running the C preprocessor on a .rs.S file. Build rules are added for .rs.S files. Since the relevant inline asm has been adjusted to export the inline asm via the ARCH_STATIC_BRANCH_ASM macro in a consistent way, the Rust side does not need architecture specific code to pull in the asm. It is not possible to use the existing C implementation of arch_static_branch via a Rust helper because it passes the argument `key` to inline assembly as an 'i' parameter. Any attempt to add a C helper for this function will fail to compile because the value of `key` must be known at compile-time. Suggested-by: Peter Zijlstra (Intel) Co-developed-by: Miguel Ojeda Signed-off-by: Miguel Ojeda Signed-off-by: Alice Ryhl --- rust/Makefile | 5 ++- rust/kernel/.gitignore | 3 ++ rust/kernel/arch_static_branch_asm.rs.S | 7 ++++ rust/kernel/jump_label.rs | 64 ++++++++++++++++++++++++++++++++- rust/kernel/lib.rs | 35 ++++++++++++++++++ scripts/Makefile.build | 9 ++++- 6 files changed, 120 insertions(+), 3 deletions(-) diff --git a/rust/Makefile b/rust/Makefile index 043d8737b430..27da24d90b0c 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -36,6 +36,8 @@ always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.c obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.o obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o +always-$(subst y,$(CONFIG_RUST),$(CONFIG_JUMP_LABEL)) += kernel/arch_static_branch_asm.rs + # Avoids running `$(RUSTC)` for the sysroot when it may not be available. ifdef CONFIG_RUST @@ -409,7 +411,8 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \ $(obj)/kernel.o: private rustc_target_flags = --extern alloc \ --extern build_error --extern macros --extern bindings --extern uapi $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \ - $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE + $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o \ + $(obj)/kernel/arch_static_branch_asm.rs FORCE +$(call if_changed_rule,rustc_library) endif # CONFIG_RUST diff --git a/rust/kernel/.gitignore b/rust/kernel/.gitignore new file mode 100644 index 000000000000..d082731007c6 --- /dev/null +++ b/rust/kernel/.gitignore @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +/arch_static_branch_asm.rs diff --git a/rust/kernel/arch_static_branch_asm.rs.S b/rust/kernel/arch_static_branch_asm.rs.S new file mode 100644 index 000000000000..9e373d4f7567 --- /dev/null +++ b/rust/kernel/arch_static_branch_asm.rs.S @@ -0,0 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +// Cut here. + +::kernel::concat_literals!(ARCH_STATIC_BRANCH_ASM("{symb} + {off} + {branch}", "{l_yes}")) diff --git a/rust/kernel/jump_label.rs b/rust/kernel/jump_label.rs index 011e1fc1d19a..ccfd20589c21 100644 --- a/rust/kernel/jump_label.rs +++ b/rust/kernel/jump_label.rs @@ -23,7 +23,69 @@ macro_rules! static_key_false { let _key: *const $keytyp = ::core::ptr::addr_of!($key); let _key: *const $crate::bindings::static_key = ::core::ptr::addr_of!((*_key).$field); - $crate::bindings::static_key_count(_key.cast_mut()) > 0 + #[cfg(not(CONFIG_JUMP_LABEL))] + { + $crate::bindings::static_key_count(_key.cast_mut()) > 0 + } + + #[cfg(CONFIG_JUMP_LABEL)] + $crate::jump_label::arch_static_branch! { $key, $keytyp, $field, false } }}; } pub use static_key_false; + +/// Assert that the assembly block evaluates to a string literal. +#[cfg(CONFIG_JUMP_LABEL)] +const _: &str = include!(concat!(env!("OBJTREE"), "/rust/kernel/arch_static_branch_asm.rs")); + +#[macro_export] +#[doc(hidden)] +#[cfg(CONFIG_JUMP_LABEL)] +#[cfg(not(CONFIG_HAVE_JUMP_LABEL_HACK))] +macro_rules! arch_static_branch { + ($key:path, $keytyp:ty, $field:ident, $branch:expr) => {'my_label: { + $crate::asm!( + include!(concat!(env!("OBJTREE"), "/rust/kernel/arch_static_branch_asm.rs")); + l_yes = label { + break 'my_label true; + }, + symb = sym $key, + off = const ::core::mem::offset_of!($keytyp, $field), + branch = const $crate::jump_label::bool_to_int($branch), + ); + + break 'my_label false; + }}; +} + +#[macro_export] +#[doc(hidden)] +#[cfg(CONFIG_JUMP_LABEL)] +#[cfg(CONFIG_HAVE_JUMP_LABEL_HACK)] +macro_rules! arch_static_branch { + ($key:path, $keytyp:ty, $field:ident, $branch:expr) => {'my_label: { + $crate::asm!( + include!(concat!(env!("OBJTREE"), "/rust/kernel/arch_static_branch_asm.rs")); + l_yes = label { + break 'my_label true; + }, + symb = sym $key, + off = const ::core::mem::offset_of!($keytyp, $field), + branch = const 2 | $crate::jump_label::bool_to_int($branch), + ); + + break 'my_label false; + }}; +} + +#[cfg(CONFIG_JUMP_LABEL)] +pub use arch_static_branch; + +/// A helper used by inline assembly to pass a boolean to as a `const` parameter. +/// +/// Using this function instead of a cast lets you assert that the input is a boolean, and not some +/// other type that can also be cast to an integer. +#[doc(hidden)] +pub const fn bool_to_int(b: bool) -> i32 { + b as i32 +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index d00a44b000b6..c912124b5e6b 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -145,3 +145,38 @@ macro_rules! container_of { ptr.sub(offset) as *const $type }} } + +/// Helper for `.rs.S` files. +#[doc(hidden)] +#[macro_export] +macro_rules! concat_literals { + ($( $asm:literal )* ) => { + ::core::concat!($($asm),*) + }; +} + +/// Wrapper around `asm!` configured for use in the kernel. +/// +/// Uses a semicolon to avoid parsing ambiguities, even though this does not match native `asm!` +/// syntax. +// For x86, `asm!` uses intel syntax by default, but we want to use at&t syntax in the kernel. +#[cfg(target_arch = "x86_64")] +#[macro_export] +macro_rules! asm { + ($($asm:expr),* ; $($rest:tt)*) => { + ::core::arch::asm!( $($asm)*, options(att_syntax), $($rest)* ) + }; +} + +/// Wrapper around `asm!` configured for use in the kernel. +/// +/// Uses a semicolon to avoid parsing ambiguities, even though this does not match native `asm!` +/// syntax. +// For non-x86 arches we just pass through to `asm!`. +#[cfg(not(target_arch = "x86_64"))] +#[macro_export] +macro_rules! asm { + ($($asm:expr),* ; $($rest:tt)*) => { + ::core::arch::asm!( $($asm)*, $($rest)* ) + }; +} diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 72b1232b1f7d..79dde37621e4 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -263,12 +263,13 @@ $(obj)/%.lst: $(obj)/%.c FORCE # Compile Rust sources (.rs) # --------------------------------------------------------------------------- -rust_allowed_features := new_uninit +rust_allowed_features := asm_const,asm_goto,new_uninit # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. rust_common_cmd = \ + OBJTREE=$(abspath $(objtree)) \ RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \ -Zallow-features=$(rust_allowed_features) \ -Zcrate-attr=no_std \ @@ -318,6 +319,12 @@ quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ $(obj)/%.ll: $(obj)/%.rs FORCE +$(call if_changed_dep,rustc_ll_rs) +quiet_cmd_rustc_rs_rs_S = RSCPP $(quiet_modtag) $@ + cmd_rustc_rs_rs_S = $(CPP) $(c_flags) -xc -C -P $< | sed '1,/^\/\/ Cut here.$$/d' >$@ + +$(obj)/%.rs: $(obj)/%.rs.S FORCE + +$(call if_changed_dep,rustc_rs_rs_S) + # Compile assembler sources (.S) # ---------------------------------------------------------------------------