From patchwork Tue Nov 19 15:34:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 13880136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AD6DD44174 for ; Tue, 19 Nov 2024 15:38:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21F466B00AF; Tue, 19 Nov 2024 10:38:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A83E6B00C2; Tue, 19 Nov 2024 10:38:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F17016B00C3; Tue, 19 Nov 2024 10:38:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CF10A6B00AF for ; Tue, 19 Nov 2024 10:38:49 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8B3C8AD2E6 for ; Tue, 19 Nov 2024 15:38:49 +0000 (UTC) X-FDA: 82803251874.28.2D7F5BA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 660A714002A for ; Tue, 19 Nov 2024 15:38:14 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hJEvkfIj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732030636; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=URgg3URVd8Hv8WB8p504Oe6mTtzB0H+jxnfnYfZO9Vc=; b=GOeUUM/eHyJHu4P3wto93Zro9ABQJL85eLOxPUEOn7dXSXAjJawT1e48pngEF2sJjBDQgQ cc5xnOn3n1v6hp/I7fJT6WUT2QZILKy/HNIxsbGCrubcH4emJmx+A13ufS+056TVPt0ClG PuDudRNuf8eeE/6MTUEw1rr1hq/QRg8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hJEvkfIj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of vschneid@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=vschneid@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732030636; a=rsa-sha256; cv=none; b=sD3lKL2FGYpEY7+pqt/TiQX7q6JQl960EJsuOaj51g+Q547XYSlX3tiC7AJqw5h022FBIi SsGsRw7TskVvsRP/TqWBmM/VNKtQRZxTa2lAhPpSfB2K1y863Lv2q/VRElzsb2u1cklomG u7WAihl/Kr6H5r5eaKAOWDvekfGdH+k= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732030726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=URgg3URVd8Hv8WB8p504Oe6mTtzB0H+jxnfnYfZO9Vc=; b=hJEvkfIj1aFxVDKl4OGFI+9GBa2QFeq2uVNCkoX15h6L/saE052u0/Vg+bSkKjRqHa0c7L xpe5/Lkz0xxAlOAA17qexPH8TkwrhxI8GL6aY1gVcazWj4N4SUV8d56/i2sVII8/M6PHoO 1H6z8MeyZizKo/TdYr0aJ4SU58OBcPQ= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-99-3C0haj_PPdCTmZEKdqoqww-1; Tue, 19 Nov 2024 10:38:43 -0500 X-MC-Unique: 3C0haj_PPdCTmZEKdqoqww-1 X-Mimecast-MFC-AGG-ID: 3C0haj_PPdCTmZEKdqoqww Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DC0AB193EF53; Tue, 19 Nov 2024 15:38:32 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.39.194.94]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 613EB30001A2; Tue, 19 Nov 2024 15:38:15 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , Frederic Weisbecker , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Jason Baron , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= , Juri Lelli , Marcelo Tosatti , Yair Podemsky , Daniel Wagner , Petr Tesarik Subject: [RFC PATCH v3 10/15] x86/alternatives: Record text_poke's of JUMP_TYPE_FORCEFUL labels Date: Tue, 19 Nov 2024 16:34:57 +0100 Message-ID: <20241119153502.41361-11-vschneid@redhat.com> In-Reply-To: <20241119153502.41361-1-vschneid@redhat.com> References: <20241119153502.41361-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 660A714002A X-Stat-Signature: hoqfzhonknsuiatxz1o1h1n8yq5uuuyg X-HE-Tag: 1732030694-936654 X-HE-Meta: U2FsdGVkX1/OCbqCXQR9QrFXg7Fs+PpyJ6SJQ3MpRZ5fZODVgsnnxpbmhLdTxLEtwhwRzYe4hXVWeYeiobxHGRQVek1IhqxWcLLVPf4H95y8Qs87B7OBZ62bWt0W6oKe/bSGcijL8Qk6kAl22S0EaMDQ8AUqvcEW6z7GUErpxjR35clNLOEMjRBtHw5A/Ke9LTj+Keup3CYVJsiqXd+FPte7NH85sny4IUzCNeulYgXmxPbsBXoJxTJflHhQ4r5Oa20gY0gqQ1FTPTue8LwzYQK3YQ40ly/MP2JrKNTVbY0EKuHGhMyvLxseTU9OG2wePrxMDaTN2zqLubiD4MTZXonFQAyqMIEctzjRkS1aPRLGPHOOxIikB+bMGAAE+rk5lyxK+fjbgdtrZGr8lS8+gF9pJ1A5x4P8bgzUHYSE2UKnPPAEoeRN5UQX421gQp0sk63mEmOY9Y8mH9z/TGXseNtC4ANnHg4scbc/ZXAxh8h1D5AZAAItEdJPVud7/zjNt7AwKXU9Xj5EX2LaeyqLdt/LuPR8Hd5Oz1iOx6GV2ynpGFd+VpM4oqAnnnrhHhoHlNk1L5AtuHpT1MILqiLOU5zzILzFO2tAjO+GlDGI0mlAcmfmGF0KpUa4uDq2488eE58VXPoDfLdvvVcrjhEXa59poYQxTGm6Nd7bub5zTAhXqvbFiPIIHtvJef8RlsQdesTif9bg5JQnTz31ge6GeaJEq0ETzUGp19djG71hjTKqV1ZC/RmcniCeqk8Vuzi67z8QEoXgt64eh9hb5LS/V9wOUxzxuHNE7XUJFZQ0IFuJxp3CQR0pThLaHESaP86+Q8uXrHy0YiYy6nLcelJCs4eRJEpsh2UdpHRldqy/web7G82orp/PUhxPkTh6imlgRJU8x6/0RQmnLodIiQih7gvnMJvZKtCVeP5svTnV1WBK05xyI76wQMIToWc5tqZf8b/hldDXQlNQm99GmhP sZPJNHY9 vVLuRjoSA8hrnBT9Y217wSNw+dxps2/IlmZwdls2sc9bwFclrTgbuV9juV6YueKlo3+X29cA5nnRq+eFve1O/p5i7to0tmkQxM+uJfugilHAanZVhPfuevwf99FP5JfVya3A/xH9XvSvmy041nYvqCZkxlWBLNyLFEYN0vCUPORlALgMg3oBmDNQ4Jc58NG2gU9OhuaIa0b4Uxvh2YDOnPDb7HCX3V7J7ixLw2Xlm3quMLQlxSNURlV/HZO52BebyCn/QvBWC+F9rhNRb8vNBMZC3PLqu0tk4aIyTtac0Y9fI3bXyLYD98h7d9rO7/OaVudo1HByxpfx2d4/p4YfjKz9p9yL+HbONV08Yrz4aYgHenO5tdN+hclJmUgOPqyxaKOfGMV+6yFrZt+kVf5efxaQCj0xAgsoA37fhGGT9pJzEUIBsfwhpkiFgc4FpmXAesD+LVOGoBMT0Qt6hZYTLNNcXqA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Forceful static keys are used in early entry code where it is unsafe to defer the sync_core() IPIs, and flagged as such via their ->type field. Record that information when creating a text_poke_loc. The text_poke_loc.old field is written to when first iterating a text_poke() entry, and as such can be (ab)used to store this information at the start of text_poke_bp_batch(). Signed-off-by: Valentin Schneider --- arch/x86/include/asm/text-patching.h | 12 ++++++++++-- arch/x86/kernel/alternative.c | 16 ++++++++++------ arch/x86/kernel/jump_label.c | 7 ++++--- 3 files changed, 24 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index 6259f1937fe77..e34de36cab61e 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -38,9 +38,17 @@ extern void *text_poke_copy(void *addr, const void *opcode, size_t len); extern void *text_poke_copy_locked(void *addr, const void *opcode, size_t len, bool core_ok); extern void *text_poke_set(void *addr, int c, size_t len); extern int poke_int3_handler(struct pt_regs *regs); -extern void text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate); +extern void __text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi); +static inline void text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate) +{ + __text_poke_bp(addr, opcode, len, emulate, false); +} -extern void text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate); +extern void __text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi); +static inline void text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate) +{ + __text_poke_queue(addr, opcode, len, emulate, false); +} extern void text_poke_finish(void); #define INT3_INSN_SIZE 1 diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index d17518ca19b8b..954c4c0f7fc58 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2098,7 +2098,10 @@ struct text_poke_loc { u8 opcode; const u8 text[POKE_MAX_OPCODE_SIZE]; /* see text_poke_bp_batch() */ - u8 old; + union { + u8 old; + u8 force_ipi; + }; }; struct bp_patching_desc { @@ -2385,7 +2388,7 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries } static void text_poke_loc_init(struct text_poke_loc *tp, void *addr, - const void *opcode, size_t len, const void *emulate) + const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct insn insn; int ret, i = 0; @@ -2402,6 +2405,7 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr, tp->rel_addr = addr - (void *)_stext; tp->len = len; tp->opcode = insn.opcode.bytes[0]; + tp->force_ipi = force_ipi; if (is_jcc32(&insn)) { /* @@ -2493,14 +2497,14 @@ void text_poke_finish(void) text_poke_flush(NULL); } -void __ref text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate) +void __ref __text_poke_queue(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct text_poke_loc *tp; text_poke_flush(addr); tp = &tp_vec[tp_vec_nr++]; - text_poke_loc_init(tp, addr, opcode, len, emulate); + text_poke_loc_init(tp, addr, opcode, len, emulate, force_ipi); } /** @@ -2514,10 +2518,10 @@ void __ref text_poke_queue(void *addr, const void *opcode, size_t len, const voi * dynamically allocated memory. This function should be used when it is * not possible to allocate memory. */ -void __ref text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate) +void __ref __text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate, bool force_ipi) { struct text_poke_loc tp; - text_poke_loc_init(&tp, addr, opcode, len, emulate); + text_poke_loc_init(&tp, addr, opcode, len, emulate, force_ipi); text_poke_bp_batch(&tp, 1); } diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index f5b8ef02d172c..e03a4f56b30fd 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -101,8 +101,8 @@ __jump_label_transform(struct jump_entry *entry, text_poke_early((void *)jump_entry_code(entry), jlp.code, jlp.size); return; } - - text_poke_bp((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL); + __text_poke_bp((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL, + jump_entry_key(entry)->type & JUMP_TYPE_FORCEFUL); } static void __ref jump_label_transform(struct jump_entry *entry, @@ -135,7 +135,8 @@ bool arch_jump_label_transform_queue(struct jump_entry *entry, mutex_lock(&text_mutex); jlp = __jump_label_patch(entry, type); - text_poke_queue((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL); + __text_poke_queue((void *)jump_entry_code(entry), jlp.code, jlp.size, NULL, + jump_entry_key(entry)->type & JUMP_TYPE_FORCEFUL); mutex_unlock(&text_mutex); return true; }