From patchwork Mon Jul 2 18:11:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10502273 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 843AC6035E for ; Mon, 2 Jul 2018 18:29:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74B3D28DB1 for ; Mon, 2 Jul 2018 18:29:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 68A7228DC2; Mon, 2 Jul 2018 18:29:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D821328DBE for ; Mon, 2 Jul 2018 18:29:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=t0bu/PBc2sjZ2iygsfuLL2/lziwreasVK9tlAwsWLTA=; b=ZCC/WqNfdMdLdg4pN6k7tQy33J kzxlLwNSSFLXvpl/UiKxYNAEhZpCvQOqGVk1ct7WE5v5SQxKxH1C5jErMvjqXaLLtv5TbrkQgk9sh ZNWUJrW4MWVcyMmVZL2DIkCpJFxErGMOmypLVF7g34U7LFe6FOuKVtGQ/rHz6nlIy2K2z1uDf8TY3 HOkSBKjgek8IbXKg1jZ/aORt+WGnyEGr/qwHOnyatVxG4ybskxqgrsf/qZUKUflCWl/et5V2UEgWA SF4PsijgnLyrnHFFfxZHqw8cThLSC+6HEyzRDNX7Em60cdra82l6Vtrw7vmzM82NlTzHeScKuoYOb tWySwwGQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3Zp-00081c-4T; Mon, 02 Jul 2018 18:29:29 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3Zk-0007oK-Um for linux-arm-kernel@bombadil.infradead.org; Mon, 02 Jul 2018 18:29:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=bYIEjgYsFCspjdyZn23Ek0/9Xld2X4/AQrtzD7Bk1Ig=; b=kEGrx4BBtLNEECegXpi6DYOsY tBrvwPzRmgHopqtrPMa3vz0kc1um/8IOFWnAoc58ulvB4oIWDXVSi7XdUIrTy2eUfNp5gYhYB6Zxp h02TL21YGtXpSzFy9xwCjXnyreV+DlGbidfej3BVZvD0r1Z+iZvIvulSPuHhqAZwbdHYGpWmrWyGt AZQCh96IflTMzIKqBhv0lqxlXHqzQpzrsCD8cPY5MRLDSm6TMYS7IPmcC9OeRTqt0laR8y9E1rGSN +sLs4khQfnVbwR0P09DcboBw2jfGO4Lz3bCbEUkPc0yIxXwiGLcn8VzR2l3a7rDLkwBGPNjpYWFtt EK24aaE7w==; Received: from mail-wr0-x243.google.com ([2a00:1450:400c:c0c::243]) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3Iz-0004pk-Lz for linux-arm-kernel@lists.infradead.org; Mon, 02 Jul 2018 18:12:08 +0000 Received: by mail-wr0-x243.google.com with SMTP id a12-v6so16440684wro.1 for ; Mon, 02 Jul 2018 11:11:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bYIEjgYsFCspjdyZn23Ek0/9Xld2X4/AQrtzD7Bk1Ig=; b=FkyoYASwtLuwL6Ys+RF95vTBeo00E/h6AYLejkixx4YIeQU0cTYq08FQDO/RhDyvl0 pjVysDdtmPuDQb5zTxUNpxwi9jKuDKYg4aUvAGVXc5v5b5+ozJKscnU+swtD2RuB2VXr MfR7HAW5r8vkudT3oYUv0kfmrzgEd+c5uvu1w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bYIEjgYsFCspjdyZn23Ek0/9Xld2X4/AQrtzD7Bk1Ig=; b=Lm+RXmQQm0RO8YFJ2v5L4wFnbPZCIO24ps0mCGmcoKPY9TL8VwES+EWzc7eda/7qme Yx9W6IFKGkMHaBZgkjASz7c4k/sJOqYWAPKSXnCuz8ka75tx0EXmjKpytlvCelSvMf6t BCRDDaQqrHAUI1MVtiHdD37zAjQgDGDzaSVRcnLBEXTM9hJASEdo+WSSlHE+4WUDDr9z lCr1k/C4CXgHSjAhirbxsc25ZmL/hHEjV8sIKf3iy5NC6e4j7t1Y64+nVMYkvl0Y17UP /X9DQHcTuZBtg3fIVKg7/hBcwFBVOTLLxVdrG+21PqAHTmDpUbxTVtGm6pqgwXJ9uLx4 PQKg== X-Gm-Message-State: APt69E0Y72YmeDhYJWH28SF7R8Tvy5VghGz0lKsAjjlivyouhNffZfWS dEvD2sPBFVXTN33VnYKBEhEGyElCyIw= X-Google-Smtp-Source: AAOMgpcwBTN4GfWSh4LmzewwhWn+MqQ4om1yFsH2NR/SCd1yx2z/yK8fNWZ22NgZ/oAHnGfmeWS5ew== X-Received: by 2002:adf:de84:: with SMTP id w4-v6mr8884955wrl.270.1530555114143; Mon, 02 Jul 2018 11:11:54 -0700 (PDT) Received: from localhost.localdomain (151.21.90.92.rev.sfr.net. [92.90.21.151]) by smtp.gmail.com with ESMTPSA id 189-v6sm10582822wmd.17.2018.07.02.11.11.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Jul 2018 11:11:53 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-arch@vger.kernel.org Subject: [PATCH v2 1/8] kernel/jump_label: abstract jump_entry member accessors Date: Mon, 2 Jul 2018 20:11:38 +0200 Message-Id: <20180702181145.4799-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702181145.4799-1-ard.biesheuvel@linaro.org> References: <20180702181145.4799-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180702_191206_203479_FCD21969 X-CRM114-Status: GOOD ( 22.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kees Cook , Arnd Bergmann , Ard Biesheuvel , Peter Zijlstra , Heiko Carstens , Jessica Yu , Will Deacon , Steven Rostedt , Ingo Molnar , Catalin Marinas , Martin Schwidefsky , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation of allowing architectures to use relative references in jump_label entries [which can dramatically reduce the memory footprint], introduce abstractions for references to the 'code', 'target' and 'key' members of struct jump_entry. Signed-off-by: Ard Biesheuvel --- include/linux/jump_label.h | 34 +++++++++++++++++ kernel/jump_label.c | 40 ++++++++------------ 2 files changed, 49 insertions(+), 25 deletions(-) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index b46b541c67c4..4603a1c88e48 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -119,6 +119,40 @@ struct static_key { #ifdef HAVE_JUMP_LABEL #include + +#ifndef __ASSEMBLY__ + +static inline unsigned long jump_entry_code(const struct jump_entry *entry) +{ + return entry->code; +} + +static inline unsigned long jump_entry_target(const struct jump_entry *entry) +{ + return entry->target; +} + +static inline struct static_key *jump_entry_key(const struct jump_entry *entry) +{ + return (struct static_key *)((unsigned long)entry->key & ~1UL); +} + +static inline bool jump_entry_is_branch(const struct jump_entry *entry) +{ + return (unsigned long)entry->key & 1UL; +} + +static inline bool jump_entry_is_init(const struct jump_entry *entry) +{ + return entry->code == 0; +} + +static inline void jump_entry_set_init(struct jump_entry *entry) +{ + entry->code = 0; +} + +#endif #endif #ifndef __ASSEMBLY__ diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 01ebdf1f9f40..8a3ac4f5f490 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -38,10 +38,10 @@ static int jump_label_cmp(const void *a, const void *b) const struct jump_entry *jea = a; const struct jump_entry *jeb = b; - if (jea->key < jeb->key) + if (jump_entry_key(jea) < jump_entry_key(jeb)) return -1; - if (jea->key > jeb->key) + if (jump_entry_key(jea) > jump_entry_key(jeb)) return 1; return 0; @@ -261,8 +261,8 @@ EXPORT_SYMBOL_GPL(jump_label_rate_limit); static int addr_conflict(struct jump_entry *entry, void *start, void *end) { - if (entry->code <= (unsigned long)end && - entry->code + JUMP_LABEL_NOP_SIZE > (unsigned long)start) + if (jump_entry_code(entry) <= (unsigned long)end && + jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE > (unsigned long)start) return 1; return 0; @@ -321,16 +321,6 @@ static inline void static_key_set_linked(struct static_key *key) key->type |= JUMP_TYPE_LINKED; } -static inline struct static_key *jump_entry_key(struct jump_entry *entry) -{ - return (struct static_key *)((unsigned long)entry->key & ~1UL); -} - -static bool jump_entry_branch(struct jump_entry *entry) -{ - return (unsigned long)entry->key & 1UL; -} - /*** * A 'struct static_key' uses a union such that it either points directly * to a table of 'struct jump_entry' or to a linked list of modules which in @@ -355,7 +345,7 @@ static enum jump_label_type jump_label_type(struct jump_entry *entry) { struct static_key *key = jump_entry_key(entry); bool enabled = static_key_enabled(key); - bool branch = jump_entry_branch(entry); + bool branch = jump_entry_is_branch(entry); /* See the comment in linux/jump_label.h */ return enabled ^ branch; @@ -370,12 +360,12 @@ static void __jump_label_update(struct static_key *key, * An entry->code of 0 indicates an entry which has been * disabled because it was in an init text area. */ - if (entry->code) { - if (kernel_text_address(entry->code)) + if (!jump_entry_is_init(entry)) { + if (kernel_text_address(jump_entry_code(entry))) arch_jump_label_transform(entry, jump_label_type(entry)); else WARN_ONCE(1, "can't patch jump_label at %pS", - (void *)(unsigned long)entry->code); + (void *)jump_entry_code(entry)); } } } @@ -430,8 +420,8 @@ void __init jump_label_invalidate_initmem(void) struct jump_entry *iter; for (iter = iter_start; iter < iter_stop; iter++) { - if (init_section_contains((void *)(unsigned long)iter->code, 1)) - iter->code = 0; + if (init_section_contains((void *)jump_entry_code(iter), 1)) + jump_entry_set_init(iter); } } @@ -441,7 +431,7 @@ static enum jump_label_type jump_label_init_type(struct jump_entry *entry) { struct static_key *key = jump_entry_key(entry); bool type = static_key_type(key); - bool branch = jump_entry_branch(entry); + bool branch = jump_entry_is_branch(entry); /* See the comment in linux/jump_label.h */ return type ^ branch; @@ -565,7 +555,7 @@ static int jump_label_add_module(struct module *mod) continue; key = iterk; - if (within_module(iter->key, mod)) { + if (within_module((unsigned long)key, mod)) { static_key_set_entries(key, iter); continue; } @@ -615,7 +605,7 @@ static void jump_label_del_module(struct module *mod) key = jump_entry_key(iter); - if (within_module(iter->key, mod)) + if (within_module((unsigned long)key, mod)) continue; /* No memory during module load */ @@ -659,8 +649,8 @@ static void jump_label_invalidate_module_init(struct module *mod) struct jump_entry *iter; for (iter = iter_start; iter < iter_stop; iter++) { - if (within_module_init(iter->code, mod)) - iter->code = 0; + if (within_module_init(jump_entry_code(iter), mod)) + jump_entry_set_init(iter); } }