From patchwork Fri Sep 23 08:46:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3F16C6FA86 for ; Fri, 23 Sep 2022 08:52:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5jnKj6mcgjAxium0NpAm7lsZnAggh5CDGL0BFLTmVoU=; b=hXN27c4sa7LS/U P4rWFW70foMG60mnxd72XvFY+Uw4HyRj1PJB7EP2tD336+l4FLvZfs8Rro0G0nhZidcxLjhwVBcop CL9o2vQn8Q/VQj44S2a3jA2L2UoCWe65nGxCivzjsu/fQ4i2KHyr1gPwYEqV+tRRz6xlJV6+aKZ2O GOk7GIRddduqk3gxj1teFpjfBLplVVE7U9J08D7JeyxzBhKuNipA45v/WZlDyL6c4TWZ5/X2TBj6Z aSaJoZeT/qfTxYq1hNwWOpnjqvD4bN6Pjlovlze4OU+m7Apl7RjshHdThQbOSCyyz1ntH/MTI3nob q02ZqO1ZRKaEKbIxGwtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOV-0033TI-Po; Fri, 23 Sep 2022 08:50:47 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033QH-EK; Fri, 23 Sep 2022 08:50:45 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4MYm4j0QwHzpT3m; Fri, 23 Sep 2022 16:47:49 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:39 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 1/3] riscv/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:56 +0800 Message-ID: <20220923084658.99304-2-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694102_8DC1BB13 X-CRM114-Status: UNSURE ( 8.84 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang --- arch/riscv/kernel/probes/kprobes.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index e6e950b7cf32..bc1f39b96e41 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -24,12 +24,14 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *); static void __kprobes arch_prepare_ss_slot(struct kprobe *p) { unsigned long offset = GET_INSN_LENGTH(p->opcode); + kprobe_opcode_t slot[MAX_INSN_SIZE]; p->ainsn.api.restore = (unsigned long)p->addr + offset; - patch_text(p->ainsn.api.insn, p->opcode); - patch_text((void *)((unsigned long)(p->ainsn.api.insn) + offset), - __BUG_INSN_32); + memcpy(slot, &p->opcode, offset); + *(kprobe_opcode_t *)((unsigned long)slot + offset) = __BUG_INSN_32; + patch_text_nosync(p->ainsn.api.insn, slot, + offset + GET_INSN_LENGTH(__BUG_INSN_32)); } static void __kprobes arch_prepare_simulate(struct kprobe *p) From patchwork Fri Sep 23 08:46:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A2D7C6FA82 for ; Fri, 23 Sep 2022 08:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uIXo4YzTa0fGQSMTO5ke0VuPTBrAJg+BM9uDtWOe4uI=; b=3+2s2RC9J6Nyl6 F6gsgzxI5YC9fa63856RyrIplBImsWa1CyY1y1ByqQgwCdVd9GdAw3U5Sri8LBNaaKrAdBCWoOZnU ycUCZ/QEN8bVa539xdNOouCdI/H/GQhZD+xBbZR10eF2uHBio0NRQo8l+ckFp3dcC2g1Z1c94kHfl l3VQbNYp7uL11d9niGqWXkR9mIweUt3OA212zL4l23yddnP5eIUfhxTwuQi+CaklqouBUJHj6X4tZ 0kfKDY9lBDEoHq3kJtkwcpfS2POR4xEqMxKT0qlXY08sW3l/Gu8gSJSr/kv738ucEobzlhvB8mcKk 7tWSSdwCyp9wOGFNyP3w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeP5-0033jX-4o; Fri, 23 Sep 2022 08:51:23 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033R5-EI; Fri, 23 Sep 2022 08:50:47 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4MYm3Q04rLzWgpB; Fri, 23 Sep 2022 16:46:41 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:39 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 2/3] csky/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:57 +0800 Message-ID: <20220923084658.99304-3-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694673_F4E42BCD X-CRM114-Status: UNSURE ( 7.91 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang --- arch/csky/kernel/probes/kprobes.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/csky/kernel/probes/kprobes.c b/arch/csky/kernel/probes/kprobes.c index 3c6e5c725d81..4feb5ce16264 100644 --- a/arch/csky/kernel/probes/kprobes.c +++ b/arch/csky/kernel/probes/kprobes.c @@ -57,7 +57,11 @@ static void __kprobes arch_prepare_ss_slot(struct kprobe *p) p->ainsn.api.restore = (unsigned long)p->addr + offset; - patch_text(p->ainsn.api.insn, p->opcode); + memcpy(p->ainsn.api.insn, &p->opcode, offset); + dcache_wb_range((unsigned long)p->ainsn.api.insn, + (unsigned long)p->ainsn.api.insn + offset); + icache_inv_range((unsigned long)p->ainsn.api.insn, + (unsigned long)p->ainsn.api.insn + offset); } static void __kprobes arch_prepare_simulate(struct kprobe *p) From patchwork Fri Sep 23 08:46:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5C2FECAAD8 for ; Fri, 23 Sep 2022 08:52:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SX5CD+VP96A66smigy3nEEILjGcm1nyT2qjl4LcpDH4=; b=jbEmBULkz3SaO5 AhaMLwvo3i2xWU6XnM8SfcrROuwI+IL6JIi29FxIESwKZLhwalNhdW1EoRYKNPSopaltYblsxXzrK gzYLDypu+xVDVAGVKnELaBPxEzBXRInO9Hqqs2qWRm+NiBlJTSMGIEHv97jJPBmGeeh5Q8BPP2Lai UoDssx1NLggW8qBTOrqFvhtcVm4p2PxfE+BJUt0G+jmqDpuNcC8qjC8gnmR9dNgCDj/VkMVyU8RVS hfgJ9DAUWLMeiZUbfHjTaJYTQ5RaZs3UdvxAb/qN+S3yZekM6SwI6jJbbEvzmsmKyJNhWLpCpvKs+ JqEPxKEusSMWcGaAnGpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOe-0033Vs-Dl; Fri, 23 Sep 2022 08:50:56 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033R6-EI; Fri, 23 Sep 2022 08:50:45 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4MYm3Q4S2lzWgy5; Fri, 23 Sep 2022 16:46:42 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:40 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 3/3] arm64/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:58 +0800 Message-ID: <20220923084658.99304-4-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694985_B17AF750 X-CRM114-Status: UNSURE ( 9.23 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Since I and D caches are coherent within single-step slot from aarch64_insn_patch_text_nosync(), hence no need to do it again via flush_icache_range(). Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang Acked-by: Will Deacon --- arch/arm64/kernel/probes/kprobes.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index d1d182320245..29b98bc12833 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -44,13 +44,10 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *); static void __kprobes arch_prepare_ss_slot(struct kprobe *p) { kprobe_opcode_t *addr = p->ainsn.api.insn; - void *addrs[] = {addr, addr + 1}; - u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS}; /* prepare insn slot */ - aarch64_insn_patch_text(addrs, insns, 2); - - flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE)); + aarch64_insn_patch_text_nosync(addr, p->opcode); + aarch64_insn_patch_text_nosync(addr + 1, BRK64_OPCODE_KPROBES_SS); /* * Needs restoring of return address after stepping xol.