From patchwork Wed Sep 19 06:51:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10605387 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4240F14BD for ; Wed, 19 Sep 2018 06:56:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D7712B41C for ; Wed, 19 Sep 2018 06:56:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 211302B45B; Wed, 19 Sep 2018 06:56:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 937BD2B41C for ; Wed, 19 Sep 2018 06:56:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=EdmQOwti+5/HTdlVsH3weMpYAgka2jS+ENYJWHBwQRo=; b=HLkbKkU2ZgCCKrmpMHVEKp2qXt aM0/O+cDA2u4+P8PMhP7NAxkbTRbeOa+G5o2/mYDGbSZT3/wUQ/LQsbRcogkN3ZUMSKnpnvgoyZjM OAwRE434m1YLp7kWUGKnlKYLTI44JBXcYOi+v83tN3Z9kHme2TGALIxIh6Jjsdig6OPPwMqSX5aNk CGJxMwQ1xKs7Q5rpiLqZpnbnm25JDRX9Yv3Ji+y5GiN0fhpYB6VJbMcHquvjVWKAYl9NpTtVr+/jy ngvMSiEQ8wPr5lRT+qf+D0X8zdPPh9bbFBrxZWj5Y/chL1EqxSxwviRPpDG2xL/1dk/DFiUEqZXsQ OFL3/bfQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g2WP2-0004M6-3l; Wed, 19 Sep 2018 06:56:00 +0000 Received: from mail-it0-x243.google.com ([2607:f8b0:4001:c0b::243]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g2WLj-0001cv-9x for linux-arm-kernel@lists.infradead.org; Wed, 19 Sep 2018 06:52:59 +0000 Received: by mail-it0-x243.google.com with SMTP id j198-v6so16355907ita.0 for ; Tue, 18 Sep 2018 23:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NQAhy2Ey7Mb6Ox0BCHyj7wjUntgBhLLd89LZtlZjBHU=; b=NsN3AGnv7VsZoqzNrVLUZ4ZX6yXb4cVP5GslK5jPZ0Y4tbBxC1B7e1XNykxHAR/3nT Xwjbvtf7nZ33/GqY6GYPE+PxBEbnXSd7i1+38SmgzfPHMcuuuVMJTKvhYZemWDB1m4QH Ja5JsrtYPbYr70dgGUwnDQM80n6TlQiQHp9AQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NQAhy2Ey7Mb6Ox0BCHyj7wjUntgBhLLd89LZtlZjBHU=; b=H54IUPVcZPd302KjzkiIgxKVut7iKm3RWowl1xHq5Dz3bTEX5wkVr1uJXr+/p2icFL 0UPdL3NGIbeBOpXtF5ZUQvmtf7hQUKNEhBKiSvZEW4mTTK9x3pRx0Fu8c/+mUJ46lttC HiXkvNLz9eSAORsBPmkOx3Epv6/DaFpd5kHEI+3y1JelyOfGf2fYo4aqfPNigpIjc5tD q35dorJaPwLj6Z4gyiUA6tjp1F6j8v2Dwb1lYiDonLcQjZfqRTeIvt8GqOLO/iWO9D+o X0NsphIhOdV3unvVA8kOSUgzD99gHTKlPK42uEu+iRlJuETjSMSQgJqKNGqxu+S05KSn VtiQ== X-Gm-Message-State: APzg51DOTYW7Wm3DCq5ONovs4j3vRdU6bHWVfPzBHovWoT9ur49Xz0kB PG8GLZPoeLuliOYiPm6HVt0XkA== X-Google-Smtp-Source: ANB0VdYEogMo8sPSravQfn7FYvX8lZmccL3rlR5nUpxZNJFm1pFyOodUDp8b37oLtQTDBbhwYH8leA== X-Received: by 2002:a24:1ad2:: with SMTP id 201-v6mr20493825iti.18.1537339948813; Tue, 18 Sep 2018 23:52:28 -0700 (PDT) Received: from localhost.localdomain ([209.82.80.116]) by smtp.gmail.com with ESMTPSA id x68-v6sm7939477ita.2.2018.09.18.23.52.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Sep 2018 23:52:28 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Subject: [PATCH v3 5/9] x86: jump_label: switch to jump_entry accessors Date: Tue, 18 Sep 2018 23:51:40 -0700 Message-Id: <20180919065144.25010-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180919065144.25010-1-ard.biesheuvel@linaro.org> References: <20180919065144.25010-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180918_235235_390256_63964919 X-CRM114-Status: GOOD ( 18.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-s390@vger.kernel.org, Kees Cook , Arnd Bergmann , Ard Biesheuvel , Peter Zijlstra , Will Deacon , Jessica Yu , Heiko Carstens , Steven Rostedt , Ingo Molnar , Catalin Marinas , Martin Schwidefsky , Thomas Gleixner , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation of switching x86 to use place-relative references for the code, target and key members of struct jump_entry, replace direct references to the struct members with invocations of the new accessors. This will allow us to make the switch by modifying the accessors only. This incorporates a cleanup of __jump_label_transform() proposed by Peter. Cc: Peter Zijlstra Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/jump_label.c | 62 ++++++++------------ 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index eeea935e9bb5..aac0c1f7e354 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -42,55 +42,40 @@ static void __ref __jump_label_transform(struct jump_entry *entry, void *(*poker)(void *, const void *, size_t), int init) { - union jump_code_union code; + union jump_code_union jmp; const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; + const void *expect, *code; + int line; + + jmp.jump = 0xe9; + jmp.offset = jump_entry_target(entry) - + (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); if (early_boot_irqs_disabled) poker = text_poke_early; if (type == JUMP_LABEL_JMP) { if (init) { - /* - * Jump label is enabled for the first time. - * So we expect a default_nop... - */ - if (unlikely(memcmp((void *)entry->code, default_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - /* - * ...otherwise expect an ideal_nop. Otherwise - * something went horribly wrong. - */ - if (unlikely(memcmp((void *)entry->code, ideal_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = ideal_nop; line = __LINE__; } - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); + code = &jmp.code; } else { - /* - * We are disabling this jump label. If it is not what - * we think it is, then something must have gone wrong. - * If this is the first initialization call, then we - * are converting the default nop to the ideal nop. - */ if (init) { - if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); - if (unlikely(memcmp((void *)entry->code, &code, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = &jmp.code; line = __LINE__; } - memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE); + + code = ideal_nop; } + if (memcmp((void *)jump_entry_code(entry), expect, JUMP_LABEL_NOP_SIZE)) + bug_at((void *)jump_entry_code(entry), line); + /* * Make text_poke_bp() a default fallback poker. * @@ -99,11 +84,14 @@ static void __ref __jump_label_transform(struct jump_entry *entry, * always nop being the 'currently valid' instruction * */ - if (poker) - (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE); - else - text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE, - (void *)entry->code + JUMP_LABEL_NOP_SIZE); + if (poker) { + (*poker)((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); + return; + } + + text_poke_bp((void *)jump_entry_code(entry), code, JUMP_LABEL_NOP_SIZE, + (void *)jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); } void arch_jump_label_transform(struct jump_entry *entry,