From patchwork Tue Sep 27 02:24:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12989707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25A2AC07E9D for ; Tue, 27 Sep 2022 02:29:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vlFLi6QVq/yPjLzDQinzcBNQT27pvsmuezKHlLKNcRs=; b=kjTcdwJXqcXz2P /r9HwcJBz27HAOATopvk+qFcyrLTwaN53C0ACuXWRDW78Pq6OS4ErrNUfRUXVCOGwyiunxrRqPiRw GEPHlJGwjgsTnHTKWuWF0vKK6ik0iOQRIKw4Ho6WvgQCmhoXStgVjhLkoy4tALW6q0lsy2ZQHAJMZ +bNbJDeM671cWDcytrl90joe/n8kyU8mc5G9qGfz41l/U1F01ZwfzOMxHokSidhb3FIx3nsFZjE3v R0guV8JBrP8wjbJNhgBeN8R4tAqd/NMKZsY1CzHnOgqiB52P7qNt2LB1ose1Pmq2GHQrb1CTEApfN 0tw91sF63dpYJh0Ks4Yg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1od0Kx-007q1B-SR; Tue, 27 Sep 2022 02:28:43 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1od0Kk-007pyL-Su; Tue, 27 Sep 2022 02:28:33 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Mc3Pg1x01zpVN2; Tue, 27 Sep 2022 10:25:27 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 27 Sep 2022 10:28:21 +0800 From: Liao Chang To: , , , , , , , , , , CC: , , , Subject: [PATCH 3/3] arm64/kprobe: Optimize the performance of patching single-step slot Date: Tue, 27 Sep 2022 10:24:35 +0800 Message-ID: <20220927022435.129965-4-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220927022435.129965-1-liaochang1@huawei.com> References: <20220927022435.129965-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220926_192831_124637_8B78CB59 X-CRM114-Status: GOOD ( 13.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Since I and D caches are coherent within single-step slot from aarch64_insn_patch_text_nosync(), hence no need to do it again via flush_icache_range(). Acked-by: Will Deacon Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang --- arch/arm64/kernel/probes/kprobes.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index d1d182320245..c9e4d0720285 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -44,13 +44,28 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *); static void __kprobes arch_prepare_ss_slot(struct kprobe *p) { kprobe_opcode_t *addr = p->ainsn.api.insn; - void *addrs[] = {addr, addr + 1}; - u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS}; - /* prepare insn slot */ - aarch64_insn_patch_text(addrs, insns, 2); - - flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE)); + /* + * Prepare insn slot, Mark Rutland points out it depends on a coupe of + * subtleties: + * + * - That the I-cache maintenance for these instructions is complete + * *before* the kprobe BRK is written (and aarch64_insn_patch_text_nosync() + * ensures this, but just omits causing a Context-Synchronization-Event + * on all CPUS). + * + * - That the kprobe BRK results in an exception (and consequently a + * Context-Synchronoization-Event), which ensures that the CPU will + * fetch thesingle-step slot instructions *after* this, ensuring that + * the new instructions are used + * + * It supposes to place ISB after patching to guarantee I-cache maintenance + * is observed on all CPUS, however, single-step slot is installed in + * the BRK exception handler, so it is unnecessary to generate + * Contex-Synchronization-Event via ISB again. + */ + aarch64_insn_patch_text_nosync(addr, p->opcode); + aarch64_insn_patch_text_nosync(addr + 1, BRK64_OPCODE_KPROBES_SS); /* * Needs restoring of return address after stepping xol.