From patchwork Wed Mar 22 05:02:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 13183543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 244B5C6FD1F for ; Wed, 22 Mar 2023 05:04:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Jqo92sGf4FWZDBI7lMx4LIVQO4uu0oHtvu/itcc9Yjg=; b=iKUtkL2MWpbXor ulRhahQTi+TFRi++2wcmjvKCSn2/rLO5Wf1jbXAXBdYHKEN1MEkruS4EBTZsNCrT5zRObznvbg6hu PvKwtuKGlgZ9FTK2N2OrM8oOgGRaUqJvWa4GtGyh6oRU2d03UF9f7OwrFn2Uw0As2tvDvONuUFnLd 05AdxE/VoH4YknQEfkDtrv5z1QtTJgAqWc6D8Xxzr0bJ16ZAoqkHcESFAo8TmySPCBceJ7PHexaNd VoMEFpGJjrdt3DmPQN/9lCT7F8Jy2yrTXHHeTHmZg+9f8CbU0NXGTUWp1YJ2JE/v0rTBK0DJch+wq hWNqZYwU43sIlNSG0ukQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1peqcG-00Ed6E-1H; Wed, 22 Mar 2023 05:02:28 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1peqcD-00Ed5W-1v for linux-arm-kernel@lists.infradead.org; Wed, 22 Mar 2023 05:02:27 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2815661F29; Wed, 22 Mar 2023 05:02:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F15E6C433D2; Wed, 22 Mar 2023 05:02:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1679461344; bh=ndOGmTUiiV++Xj9S+Ph8LJTM9AhKGZq0wDVwNZYO3GM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=apUXCUknJJO20URvMNoq6w9aOAme+DY66QbeX8xiJzq/k+dOnlE0/4cqHUwgt9YPm WaRDf226gcIKEm1DytvFb043CgM/L07ds+zD66ypJHc9IyNv3A++IMMuWg8exV8K96 pvDnXQvFpTUdcwGW0X3z5A265MmPzAO/oDjF0FwZoZrpiKOZybpkJ282P8T7PTk/OO Nr1VZlAgzRCistKOInNqn1RNdeFU+fJ+/w3kLXRT6naBGRL5VJX8ye96R8OCV30h+B ymW3pdN7J5Psyv6Y3EWyvbSfna2NO4oIusgnQXvAexS4opKMz09BADcfNpLNk9sJVN KY0mkZcnRwIDA== Date: Tue, 21 Mar 2023 22:02:22 -0700 From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Mark Rutland , Jason Baron , Steven Rostedt , Ard Biesheuvel , Christophe Leroy , Paolo Bonzini , Sean Christopherson , Sami Tolvanen , Nick Desaulniers , Will McVicker , Kees Cook , linux-arm-kernel@lists.infradead.org Subject: [PATCH v2.1 01/11] static_call: Improve key type abstraction Message-ID: <20230322050222.3okw7d2cd4y3w4ug@treble> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230321_220225_715334_22A18281 X-CRM114-Status: GOOD ( 18.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make the static_call_key union less fragile by abstracting all knowledge about the type bit into helper functions. Signed-off-by: Josh Poimboeuf --- include/linux/static_call_types.h | 4 +-- kernel/static_call_inline.c | 47 ++++++++++++++++++------- tools/include/linux/static_call_types.h | 4 +-- 3 files changed, 38 insertions(+), 17 deletions(-) diff --git a/include/linux/static_call_types.h b/include/linux/static_call_types.h index 5a00b8b2cf9f..87c3598609e8 100644 --- a/include/linux/static_call_types.h +++ b/include/linux/static_call_types.h @@ -63,8 +63,8 @@ struct static_call_key { union { /* bit 0: 0 = mods, 1 = sites */ unsigned long type; - struct static_call_mod *mods; - struct static_call_site *sites; + struct static_call_mod *_mods; + struct static_call_site *_sites; }; }; diff --git a/kernel/static_call_inline.c b/kernel/static_call_inline.c index 639397b5491c..1328370d3cf6 100644 --- a/kernel/static_call_inline.c +++ b/kernel/static_call_inline.c @@ -115,12 +115,17 @@ static inline bool static_call_key_has_mods(struct static_call_key *key) return !(key->type & 1); } -static inline struct static_call_mod *static_call_key_next(struct static_call_key *key) +static inline struct static_call_mod *static_call_key_mods(struct static_call_key *key) { if (!static_call_key_has_mods(key)) return NULL; - return key->mods; + return key->_mods; +} + +static inline void static_call_key_set_mods(struct static_call_key *key, struct static_call_mod *mods) +{ + key->_mods = mods; } static inline struct static_call_site *static_call_key_sites(struct static_call_key *key) @@ -131,6 +136,12 @@ static inline struct static_call_site *static_call_key_sites(struct static_call_ return (struct static_call_site *)(key->type & ~1); } +static inline void static_call_key_set_sites(struct static_call_key *key, struct static_call_site *sites) +{ + key->_sites = sites; + key->type |= 1; +} + void __static_call_update(struct static_call_key *key, void *tramp, void *func) { struct static_call_site *site, *stop; @@ -154,7 +165,7 @@ void __static_call_update(struct static_call_key *key, void *tramp, void *func) goto done; first = (struct static_call_mod){ - .next = static_call_key_next(key), + .next = static_call_key_mods(key), .mod = NULL, .sites = static_call_key_sites(key), }; @@ -250,8 +261,7 @@ static int __static_call_init(struct module *mod, * static_call_init() before memory allocation works. */ if (!mod) { - key->sites = site; - key->type |= 1; + static_call_key_set_sites(key, site); goto do_transform; } @@ -266,10 +276,10 @@ static int __static_call_init(struct module *mod, */ if (static_call_key_sites(key)) { site_mod->mod = NULL; - site_mod->next = NULL; site_mod->sites = static_call_key_sites(key); + site_mod->next = NULL; - key->mods = site_mod; + static_call_key_set_mods(key, site_mod); site_mod = kzalloc(sizeof(*site_mod), GFP_KERNEL); if (!site_mod) @@ -278,8 +288,9 @@ static int __static_call_init(struct module *mod, site_mod->mod = mod; site_mod->sites = site; - site_mod->next = static_call_key_next(key); - key->mods = site_mod; + site_mod->next = static_call_key_mods(key); + + static_call_key_set_mods(key, site_mod); } do_transform: @@ -406,7 +417,7 @@ static void static_call_del_module(struct module *mod) struct static_call_site *stop = mod->static_call_sites + mod->num_static_call_sites; struct static_call_key *key, *prev_key = NULL; - struct static_call_mod *site_mod, **prev; + struct static_call_mod *site_mod, *prev; struct static_call_site *site; for (site = start; site < stop; site++) { @@ -416,15 +427,25 @@ static void static_call_del_module(struct module *mod) prev_key = key; - for (prev = &key->mods, site_mod = key->mods; + site_mod = static_call_key_mods(key); + if (!site_mod) + continue; + + if (site_mod->mod == mod) { + static_call_key_set_mods(key, site_mod->next); + kfree(site_mod); + continue; + } + + for (prev = site_mod, site_mod = site_mod->next; site_mod && site_mod->mod != mod; - prev = &site_mod->next, site_mod = site_mod->next) + prev = site_mod, site_mod = site_mod->next) ; if (!site_mod) continue; - *prev = site_mod->next; + prev->next = site_mod->next; kfree(site_mod); } } diff --git a/tools/include/linux/static_call_types.h b/tools/include/linux/static_call_types.h index 5a00b8b2cf9f..87c3598609e8 100644 --- a/tools/include/linux/static_call_types.h +++ b/tools/include/linux/static_call_types.h @@ -63,8 +63,8 @@ struct static_call_key { union { /* bit 0: 0 = mods, 1 = sites */ unsigned long type; - struct static_call_mod *mods; - struct static_call_site *sites; + struct static_call_mod *_mods; + struct static_call_site *_sites; }; };