From patchwork Mon Jul 2 18:11:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10502229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 747066035E for ; Mon, 2 Jul 2018 18:12:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C7D528470 for ; Mon, 2 Jul 2018 18:12:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5036528A8E; Mon, 2 Jul 2018 18:12:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C4AEC28470 for ; Mon, 2 Jul 2018 18:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=d0x6j2AKFZCfetsKT4L2t7NeqBDznMsvJcnO3sNEJw0=; b=Dqbre6Z6c2tjgo9rPom0wMvmEg flYzaftrH2YWN07phblUW3+/wxGEEKh3Wz2B6HcsUQQ6rrt1LVL4SwVMouE7F3WcmN8ONDWjBg1/t /q5CblL3byhV1wkdlSI7wRfUAoWj61Mo8WNxTVspmyccNQhor46ilPBhjr7tSr5kKfQfq4vNhnDL5 ozmn6N9rPgxgSyPy6v8dQe25cY4add3edCcQsn6ZhIDQZ1QpfE9/kgv5SugSwl4dqFyhZAJz4yInr ntI3pf9+9hafcawofz0vV3ip/uOEf9eAALezosxub5RE/Es2L61tR1gtfR+IFseQMcwv/Wr/+Jm7S Z/8xVMBg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3JB-00081o-0B; Mon, 02 Jul 2018 18:12:17 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3J8-00080q-An for linux-arm-kernel@bombadil.infradead.org; Mon, 02 Jul 2018 18:12:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=sZUhsZ+WTugw6FSDPAWQoDPS5 i06Xv0V9XS74YCJCBZVpO9Xq2KviCLmxCH6pOZvQy1j5iScA1Fvivf3+/Vt9uPYYDgFP/NceW1UhI y7zwo2zswbZK568fbNCee3dI5Csq1+1pImwXsPtvvtmHm0XCuCno7iC4/OClUADAuUnWuJZB3ekVi /RcXv1LSpiqHfvBpylg9/MaPX8bw3tuuwg3nDtTZULNlM97OFceQXwbLaqgsWFugwViryQQ1a/NlW NQJbJu6ZjjzsxBIqg+bItTrJrku8quMfncq2vwQV7yNJVrisb6OBY+lmevJZTJx70/0H20/L8kuoG 2lBZ9cpiw==; Received: from mail-wr0-x244.google.com ([2a00:1450:400c:c0c::244]) by merlin.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fa3J5-0002RB-8S for linux-arm-kernel@lists.infradead.org; Mon, 02 Jul 2018 18:12:12 +0000 Received: by mail-wr0-x244.google.com with SMTP id h10-v6so16417532wrq.8 for ; Mon, 02 Jul 2018 11:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=L12Ji9yFkOczS8LxdqLIaOyOwP4xEPBvHP1UgNpl+z3lLalzVbfYOgTI0GKhzcnrls UZIePyE850IJ5e69rkXOVAhCKUpEBVKr0gkM64ZjPQ/ZdROpXtXWMMfaSm7qlG5/qQWo draOD5nv3m+K4RcQdV/waomYm1A6BDXnXjz2s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=djv/WvBmZGvxgw9tDiCr9aMHz3eBlC5IQP3GruSAwxkiEFxUBdobXRSZt/lkzahEXs O2OZVBg5Zy7U5UhPbyvylC3NohBpEDdDX097AKSspoIAZHy7mLiBkX6A0Jzoiza+0itE 3A9kQzYgjW4bqYifW7Z18jk+0nVkUMGUxIUk7g902+CF1OOSgIMwlZ/zLUImjKgB6T3C VBjoPL+HbFglIy5mDiRf9lGuXSRHOSmQDY3bcA9n6lsb8sfB97XcTpKTsuT/7j4PL9mq VtMF3o9p1bUkOuQZwdFG61Xaguc4ReAwn45TpdDU67i5jTpIt6ovk9Y7BW1xP7+xUCub eUNQ== X-Gm-Message-State: APt69E3sE761enEObThn95YmAop7B/6HzA7LxM1AYa01DalR8AJc80p9 s4iCfv6alinPj5IbrIbD/Fkq0wYhhnE= X-Google-Smtp-Source: AAOMgpcQVlUUzv1TCSYXqAgcTDl5N5L0ozOlYRn4H8MCiZ+oR1eu3iA3+3NmhFVjcsYoXqFEsf+ihg== X-Received: by 2002:adf:83a6:: with SMTP id 35-v6mr20405339wre.13.1530555116389; Mon, 02 Jul 2018 11:11:56 -0700 (PDT) Received: from localhost.localdomain (151.21.90.92.rev.sfr.net. [92.90.21.151]) by smtp.gmail.com with ESMTPSA id 189-v6sm10582822wmd.17.2018.07.02.11.11.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Jul 2018 11:11:55 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-arch@vger.kernel.org Subject: [PATCH v2 2/8] kernel/jump_label: implement generic support for relative references Date: Mon, 2 Jul 2018 20:11:39 +0200 Message-Id: <20180702181145.4799-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702181145.4799-1-ard.biesheuvel@linaro.org> References: <20180702181145.4799-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180702_141211_314529_619451F8 X-CRM114-Status: GOOD ( 20.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kees Cook , Arnd Bergmann , Ard Biesheuvel , Peter Zijlstra , Heiko Carstens , Jessica Yu , Will Deacon , Steven Rostedt , Ingo Molnar , Catalin Marinas , Martin Schwidefsky , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP To reduce the size taken up by absolute references in jump label entries themselves and the associated relocation records in the .init segment, add support for emitting them as relative references instead. Note that this requires some extra care in the sorting routine, given that the offsets change when entries are moved around in the jump_entry table. Signed-off-by: Ard Biesheuvel --- arch/Kconfig | 3 +++ include/linux/jump_label.h | 28 ++++++++++++++++++++ kernel/jump_label.c | 22 ++++++++++++++- 3 files changed, 52 insertions(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index 2b8b70820002..22fa3792626e 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -348,6 +348,9 @@ config HAVE_PERF_USER_STACK_DUMP config HAVE_ARCH_JUMP_LABEL bool +config HAVE_ARCH_JUMP_LABEL_RELATIVE + bool + config HAVE_RCU_TABLE_FREE bool diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index 4603a1c88e48..871826fd0c3b 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -121,6 +121,32 @@ struct static_key { #include #ifndef __ASSEMBLY__ +#ifdef CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE + +struct jump_entry { + s32 code; + s32 target; + long key; // key may be far away from the core kernel under KASLR +}; + +static inline unsigned long jump_entry_code(const struct jump_entry *entry) +{ + return (unsigned long)&entry->code + entry->code; +} + +static inline unsigned long jump_entry_target(const struct jump_entry *entry) +{ + return (unsigned long)&entry->target + entry->target; +} + +static inline struct static_key *jump_entry_key(const struct jump_entry *entry) +{ + long offset = entry->key & ~1L; + + return (struct static_key *)((unsigned long)&entry->key + offset); +} + +#else static inline unsigned long jump_entry_code(const struct jump_entry *entry) { @@ -137,6 +163,8 @@ static inline struct static_key *jump_entry_key(const struct jump_entry *entry) return (struct static_key *)((unsigned long)entry->key & ~1UL); } +#endif + static inline bool jump_entry_is_branch(const struct jump_entry *entry) { return (unsigned long)entry->key & 1UL; diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 8a3ac4f5f490..d424e1d22d63 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -47,14 +47,34 @@ static int jump_label_cmp(const void *a, const void *b) return 0; } +static void jump_label_swap(void *a, void *b, int size) +{ + long delta = (unsigned long)a - (unsigned long)b; + struct jump_entry *jea = a; + struct jump_entry *jeb = b; + struct jump_entry tmp = *jea; + + jea->code = jeb->code - delta; + jea->target = jeb->target - delta; + jea->key = jeb->key - delta; + + jeb->code = tmp.code + delta; + jeb->target = tmp.target + delta; + jeb->key = tmp.key + delta; +} + static void jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop) { unsigned long size; + void *swapfn = NULL; + + if (IS_ENABLED(CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE)) + swapfn = jump_label_swap; size = (((unsigned long)stop - (unsigned long)start) / sizeof(struct jump_entry)); - sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL); + sort(start, size, sizeof(struct jump_entry), jump_label_cmp, swapfn); } static void jump_label_update(struct static_key *key);