From patchwork Wed Jun 27 16:06:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10492163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F3FB6601A0 for ; Wed, 27 Jun 2018 16:06:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D1AA292C8 for ; Wed, 27 Jun 2018 16:06:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 217B12951C; Wed, 27 Jun 2018 16:06:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A67FF292C8 for ; Wed, 27 Jun 2018 16:06:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VJjRsc7MG7j5Jv7zcVV9f99DemUgkuRnQxmTTDgoVv4=; b=M76D2t3B2kmC9qA0ISlT3i6g2Y g2ORJi2f3QUOjsAjaqDIYfjGTczOPBV6WnYTMkihwN55JZGhKwo0KRHr0WWmm5wOxEndeSmDYR3qq zwpfqr6yOQ7wHZFPuKTTBn1Fv6actGb6mI0NtuecMiWWyQ+JWAHW8ldrX+iT62fi9n+T98k+Fa5D7 /n+bhhNlZ9lPMXAlUCAI8YSMKg/NeYVG1GVFaza6EWmbhDRtSNdfaN51zkV4lZiBNdqOTHN0uu5Dd yPzMcFh1qrCKCUvG0hxZjVL/UfxWy0vt39L6Ih8N2Bcyh9OUwzB2PhQTGbEA73EXubZWdbCTSIZpM 3BSlzaDA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fYCxv-0000Q7-LP; Wed, 27 Jun 2018 16:06:43 +0000 Received: from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fYCxZ-0000Ba-Uu for linux-arm-kernel@lists.infradead.org; Wed, 27 Jun 2018 16:06:23 +0000 Received: by mail-wm0-x242.google.com with SMTP id i139-v6so6416253wmf.4 for ; Wed, 27 Jun 2018 09:06:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4S8kZQAvHa84YpWVvxunUgaXp0tuBTi7qepgqm8/+QE=; b=bNVV4EdSWbYIG3AcgXqoPrgzHv5IuSt1y79hMVSTqlv4GSE4DswGLrww8OA0uUBI4+ hh5hIMZh3Z4SrdgXjLmiRdIWEOb7HOj5fFV/VFoCLlL0JUQYEYCtPuvMJBX+JQBVx9/G tXDhyhXI8X7QnidEH3SaViXefT2ypauu6Asy8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4S8kZQAvHa84YpWVvxunUgaXp0tuBTi7qepgqm8/+QE=; b=pHX/+XXS753l5LuOv1OmDFbT4yZUhwrx+/wIErZjR54CaSwC4VyRvjJXm3PRwFVGGi AOMREF/aANvYD4A47mPYeW3vNP/TcbxzNtVB8AlV8S821fE8yfiq760YvHE7Iuuln6DE pHJ0rhQNd/hXcIUciXA3eFqlS2a12W9umpsNyBWDLVbRfSW0gAHX8adNybitBgHV6r0k LXvlukOie9JA0FctrPQPVUPNDHSAhuntM7CHR3ZgFb0rJIoa5RfChpaAit8JghpQ40w1 nJx6QrlHQa0VNcZzFI+EypxfHqOzfffyBoLtNvU3pyUpUk9PqcusKtUH2OpxtMBklPaB XSew== X-Gm-Message-State: APt69E3j3BvFhmNf+J7Z1kD6DE7ZoM3GPLtohdy3fUdrDQtthGlFQ9R2 mvxgtCOCTMKX8g5WRYTBFdLNaA== X-Google-Smtp-Source: AAOMgpdWFyGN4KR8CEIzYOKzyXFmpA36LSoaMHgh33ZPinoLY9NOznCvWIoUCvwiAqcQLIIPFyvI+A== X-Received: by 2002:a1c:8c55:: with SMTP id o82-v6mr5246007wmd.60.1530115570114; Wed, 27 Jun 2018 09:06:10 -0700 (PDT) Received: from rev02.home ([2a01:cb1d:112:6f00:4d04:66e:dea8:5b23]) by smtp.gmail.com with ESMTPSA id s2-v6sm7055000wrn.75.2018.06.27.09.06.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 09:06:09 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [PATCH 2/5] kernel/jump_label: implement generic support for relative references Date: Wed, 27 Jun 2018 18:06:01 +0200 Message-Id: <20180627160604.8154-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180627160604.8154-1-ard.biesheuvel@linaro.org> References: <20180627160604.8154-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180627_090622_054447_CD2F2124 X-CRM114-Status: GOOD ( 17.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , Ard Biesheuvel , Peter Zijlstra , Catalin Marinas , Will Deacon , Steven Rostedt , Ingo Molnar , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP To reduce the size taken up by absolute references in jump label entries themselves and the associated relocation records in the .init segment, add support for emitting them as 32-bit relative references instead. Note that this requires some extra care in the sorting routine, given that the offsets change when entries are moved around in the jump_entry table. Signed-off-by: Ard Biesheuvel --- arch/Kconfig | 3 +++ include/linux/jump_label.h | 28 ++++++++++++++++++++ kernel/jump_label.c | 20 +++++++++++++- 3 files changed, 50 insertions(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index 2b8b70820002..22fa3792626e 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -348,6 +348,9 @@ config HAVE_PERF_USER_STACK_DUMP config HAVE_ARCH_JUMP_LABEL bool +config HAVE_ARCH_JUMP_LABEL_RELATIVE + bool + config HAVE_RCU_TABLE_FREE bool diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index 86ec0652d3b1..aa203dffe72c 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -121,6 +121,32 @@ struct static_key { #include #ifndef __ASSEMBLY__ +#ifdef CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE + +struct jump_entry { + int code; + int target; + int key; +}; + +static inline unsigned long jump_entry_code(const struct jump_entry *entry) +{ + return (unsigned long)&entry->code + entry->code; +} + +static inline unsigned long jump_entry_target(const struct jump_entry *entry) +{ + return (unsigned long)&entry->target + entry->target; +} + +static inline struct static_key *jump_entry_key(const struct jump_entry *entry) +{ + unsigned long key = (unsigned long)&entry->key + entry->key; + + return (struct static_key *)(key & ~1UL); +} + +#else struct jump_entry; /* defined by the architecture */ @@ -139,6 +165,8 @@ static inline struct static_key *jump_entry_key(const struct jump_entry *entry) return (struct static_key *)((unsigned long)entry->key & ~1UL); } +#endif + static inline bool jump_entry_is_branch(const struct jump_entry *entry) { return (unsigned long)entry->key & 1UL; diff --git a/kernel/jump_label.c b/kernel/jump_label.c index c3524c9b3004..285eff13ecd1 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -49,6 +49,22 @@ static int jump_label_cmp(const void *a, const void *b) return 0; } +static void jump_label_swap(void *a, void *b, int size) +{ + long delta = (unsigned long)a - (unsigned long)b; + struct jump_entry *jea = a; + struct jump_entry *jeb = b; + struct jump_entry tmp = *jea; + + jea->code = jeb->code - delta; + jea->target = jeb->target - delta; + jea->key = jeb->key - delta; + + jeb->code = tmp.code + delta; + jeb->target = tmp.target + delta; + jeb->key = tmp.key + delta; +} + static void jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop) { @@ -56,7 +72,9 @@ jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop) size = (((unsigned long)stop - (unsigned long)start) / sizeof(struct jump_entry)); - sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL); + sort(start, size, sizeof(struct jump_entry), jump_label_cmp, + IS_ENABLED(CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE) ? jump_label_swap + : NULL); } static void jump_label_update(struct static_key *key);