From patchwork Sun Feb 4 03:12:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinghao Jia X-Patchwork-Id: 13544447 X-Patchwork-Delegate: mhiramat@kernel.org Received: from mx0a-00007101.pphosted.com (mx0a-00007101.pphosted.com [148.163.135.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CE5063A1; Sun, 4 Feb 2024 03:57:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.135.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707019077; cv=none; b=ZtLibcZ4NcgQrrLnknlSEqu1BDMwgoz72Sbn/Sb25mL0S4jHJyDuZCOUs/8zfBU8lX9GLmXbQDjt4DcHOg1xzwlzt3fCrjSBgNDeYuvWL6zJOnShpSv5epvUf9KRUdUgCLjhwmOiV2mvcS3CJcWHEG/3Lj0H8qySW6ujVWy0Nek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707019077; c=relaxed/simple; bh=WD//COAAgnwG6ByL9FIxCxWk6nNqTSw772pmIuR8KyY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CTUxzT8Cu2xHAJCDe+iP2gxd1K2kfIrMNcLA6+Z9FL9vWwb+5LdcRmbT8sEi/SCpsUu5RwlcuqDrG8wZ1Ef1BtaBF5aHVZPkpkoc5TavH6FpSH+YL162t1IkIHiXRMHsZI2fe4b9z7kKRz81693Z9JWMdsgxwPIT7/znulAKMLs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu; spf=pass smtp.mailfrom=illinois.edu; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b=KLq57fu1; arc=none smtp.client-ip=148.163.135.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=illinois.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b="KLq57fu1" Received: from pps.filterd (m0166257.ppops.net [127.0.0.1]) by mx0a-00007101.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 4140prQH023253; Sun, 4 Feb 2024 03:13:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=illinois.edu; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=campusrelays; bh=QKbEKF0BOPhRZnolIBDBWcg1t+uTArb4iAsWd5YiF10=; b=KLq57fu1F+VBizIUU7SbpnoOGk2ACVWDAVGSrlHtlBhjKfHgr5LULDfhhGLZifoDbL01 7CJRcq9Ut0tgxrzCD45HA+fgS+WhONZo0/Qo355B4vJTDfn7OlyG8eFDsIRYeApRHg4Y n7lXn4YWKlAQX/4ilWViaF/9gOrEyh0kTVIr5T7iuYvJ7n7EtMu/8HwxHBADz3Up30Dj fpBN8AZ53DZM7kOXSz0w+Xl/Ha1GQUPd8qzMEnNNu3QzbdCHoFoLCi4NQJMgmU08+D5N 39afaFKDEK1TFK9f08xiePsVZx2D2BBQ5t4mLG3vdzv7i+XpjaC0/+IKFiXA5qGkXkP9 xA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-00007101.pphosted.com (PPS) with ESMTPS id 3w1e8n4ktt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 04 Feb 2024 03:13:05 +0000 Received: from m0166257.ppops.net (m0166257.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4143D4Fw010150; Sun, 4 Feb 2024 03:13:05 GMT Received: from localhost.localdomain (oasis.cs.illinois.edu [130.126.137.13]) by mx0a-00007101.pphosted.com (PPS) with ESMTP id 3w1e8n4ktm-2; Sun, 04 Feb 2024 03:13:05 +0000 From: Jinghao Jia To: "Masami Hiramatsu (Google)" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Xin Li Cc: linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org, Jinghao Jia Subject: [PATCH v2 1/3] x86/kprobes: Refactor can_{probe,boost} return type to bool Date: Sat, 3 Feb 2024 21:12:58 -0600 Message-ID: <20240204031300.830475-2-jinghao7@illinois.edu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240204031300.830475-1-jinghao7@illinois.edu> References: <20240204031300.830475-1-jinghao7@illinois.edu> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: BvmB3QU77cDUQOBIJkGSbqBZkiK1PtJh X-Proofpoint-ORIG-GUID: oTAYOSqsYwSJlP7N4HcTMSh_KAGAJsTs X-Spam-Details: rule=cautious_plus_nq_notspam policy=cautious_plus_nq score=0 lowpriorityscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402040022 X-Spam-Score: 0 X-Spam-OrigSender: jinghao7@illinois.edu X-Spam-Bar: Both can_probe and can_boost have int return type but are using int as boolean in their context. Refactor both functions to make them actually return boolean. Signed-off-by: Jinghao Jia Acked-by: Masami Hiramatsu (Google) --- arch/x86/kernel/kprobes/common.h | 2 +- arch/x86/kernel/kprobes/core.c | 33 +++++++++++++++----------------- 2 files changed, 16 insertions(+), 19 deletions(-) diff --git a/arch/x86/kernel/kprobes/common.h b/arch/x86/kernel/kprobes/common.h index c993521d4933..e772276f5aa9 100644 --- a/arch/x86/kernel/kprobes/common.h +++ b/arch/x86/kernel/kprobes/common.h @@ -78,7 +78,7 @@ #endif /* Ensure if the instruction can be boostable */ -extern int can_boost(struct insn *insn, void *orig_addr); +extern bool can_boost(struct insn *insn, void *orig_addr); /* Recover instruction if given address is probed */ extern unsigned long recover_probed_instruction(kprobe_opcode_t *buf, unsigned long addr); diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index e8babebad7b8..644d416441fb 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -137,14 +137,14 @@ NOKPROBE_SYMBOL(synthesize_relcall); * Returns non-zero if INSN is boostable. * RIP relative instructions are adjusted at copying time in 64 bits mode */ -int can_boost(struct insn *insn, void *addr) +bool can_boost(struct insn *insn, void *addr) { kprobe_opcode_t opcode; insn_byte_t prefix; int i; if (search_exception_tables((unsigned long)addr)) - return 0; /* Page fault may occur on this address. */ + return false; /* Page fault may occur on this address. */ /* 2nd-byte opcode */ if (insn->opcode.nbytes == 2) @@ -152,7 +152,7 @@ int can_boost(struct insn *insn, void *addr) (unsigned long *)twobyte_is_boostable); if (insn->opcode.nbytes != 1) - return 0; + return false; for_each_insn_prefix(insn, i, prefix) { insn_attr_t attr; @@ -160,7 +160,7 @@ int can_boost(struct insn *insn, void *addr) attr = inat_get_opcode_attribute(prefix); /* Can't boost Address-size override prefix and CS override prefix */ if (prefix == 0x2e || inat_is_address_size_prefix(attr)) - return 0; + return false; } opcode = insn->opcode.bytes[0]; @@ -181,12 +181,12 @@ int can_boost(struct insn *insn, void *addr) case 0xf6 ... 0xf7: /* Grp3 */ case 0xfe: /* Grp4 */ /* ... are not boostable */ - return 0; + return false; case 0xff: /* Grp5 */ /* Only indirect jmp is boostable */ return X86_MODRM_REG(insn->modrm.bytes[0]) == 4; default: - return 1; + return true; } } @@ -253,20 +253,18 @@ unsigned long recover_probed_instruction(kprobe_opcode_t *buf, unsigned long add } /* Check if paddr is at an instruction boundary */ -static int can_probe(unsigned long paddr) +static bool can_probe(unsigned long paddr) { unsigned long addr, __addr, offset = 0; struct insn insn; kprobe_opcode_t buf[MAX_INSN_SIZE]; if (!kallsyms_lookup_size_offset(paddr, NULL, &offset)) - return 0; + return false; /* Decode instructions */ addr = paddr - offset; while (addr < paddr) { - int ret; - /* * Check if the instruction has been modified by another * kprobe, in which case we replace the breakpoint by the @@ -277,11 +275,10 @@ static int can_probe(unsigned long paddr) */ __addr = recover_probed_instruction(buf, addr); if (!__addr) - return 0; + return false; - ret = insn_decode_kernel(&insn, (void *)__addr); - if (ret < 0) - return 0; + if (insn_decode_kernel(&insn, (void *)__addr) < 0) + return false; #ifdef CONFIG_KGDB /* @@ -290,7 +287,7 @@ static int can_probe(unsigned long paddr) */ if (insn.opcode.bytes[0] == INT3_INSN_OPCODE && kgdb_has_hit_break(addr)) - return 0; + return false; #endif addr += insn.length; } @@ -310,10 +307,10 @@ static int can_probe(unsigned long paddr) */ __addr = recover_probed_instruction(buf, addr); if (!__addr) - return 0; + return false; if (insn_decode_kernel(&insn, (void *)__addr) < 0) - return 0; + return false; if (insn.opcode.value == 0xBA) offset = 12; @@ -324,7 +321,7 @@ static int can_probe(unsigned long paddr) /* This movl/addl is used for decoding CFI. */ if (is_cfi_trap(addr + offset)) - return 0; + return false; } out: From patchwork Sun Feb 4 03:12:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinghao Jia X-Patchwork-Id: 13544451 X-Patchwork-Delegate: mhiramat@kernel.org Received: from mx0a-00007101.pphosted.com (mx0a-00007101.pphosted.com [148.163.135.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF61D6119; Sun, 4 Feb 2024 05:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.135.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707024182; cv=none; b=B/0kZDPhpwCFbw2x2qHl7EnCni75NFyeiXXYZl5egF+9WEYEMyWVsd8UfuUMqcgFKIDH7x7Ryrzyn4ykQdB+rEGIYeg/N5/gc3y8SnATRANdvw+NsyfqK5wjtpdDZRJ7uO7MUbmHGFLqCSMWvFzVY3oMWc1zpZydyko5JFPCZo0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707024182; c=relaxed/simple; bh=UyQtY5NzwuQh5f8YBmtuvP5BCquorDhnoNHeHzzRUOc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W1EE8UgVl31TrHPXuIM+ZQU2tsk876x6n5Uop1U+47Zs4/xZ58YR4jeDoELGlr0Z/EFq8aNfYXPbmEGCbIpz5lUDbMuDP/2rqMInWZkJSiQjaMO8FsQe60Fxj7K7Tad0TZuvqQEQILCM+Kietm9bH5/rk/UPSmWs/7NteGiV3g4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu; spf=pass smtp.mailfrom=illinois.edu; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b=khJzISEW; arc=none smtp.client-ip=148.163.135.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=illinois.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b="khJzISEW" Received: from pps.filterd (m0166257.ppops.net [127.0.0.1]) by mx0a-00007101.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 4142fM2V032217; Sun, 4 Feb 2024 03:13:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=illinois.edu; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=campusrelays; bh=8U/uRPUZ4LmGmqQfXkYTK77RgNRpuaAarihJR0mJhDA=; b=khJzISEWHGYeSlSHd+aovNPF7cTLl2U1ohNvSG4mQCuUQ3P+pgXiDgUkoDeBwkCQgsU8 LRYttbHmF3YtW70lEUaSuKkiT8heve4LAR3fyQZP350l2tFp7iQ1rBGhuCxybt73PiLy YdnvFt502AYf74YviWdk0ybK5dy6/+YMMwZqMUtSPcRztEZcYbJL26+pSibG6DVxSDxB yM5NUVIFckBbOgCAEbvHejYeHtYlS6lCfUlpxeZ2/PM3Mdliz3IftGgY7jJ8LA5SA4Eg EFRkcUFGMBC1KoIhKJzgKz3FpkBO5nMZQaaTCtJfNVO3La0oNSHwj4KgqRKzF5rAJasa Xg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-00007101.pphosted.com (PPS) with ESMTPS id 3w1e8n4ktv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 04 Feb 2024 03:13:06 +0000 Received: from m0166257.ppops.net (m0166257.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4143D4G0010150; Sun, 4 Feb 2024 03:13:06 GMT Received: from localhost.localdomain (oasis.cs.illinois.edu [130.126.137.13]) by mx0a-00007101.pphosted.com (PPS) with ESMTP id 3w1e8n4ktm-3; Sun, 04 Feb 2024 03:13:06 +0000 From: Jinghao Jia To: "Masami Hiramatsu (Google)" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Xin Li Cc: linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org, Jinghao Jia Subject: [PATCH v2 2/3] x86/kprobes: Prohibit kprobing on INT and UD Date: Sat, 3 Feb 2024 21:12:59 -0600 Message-ID: <20240204031300.830475-3-jinghao7@illinois.edu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240204031300.830475-1-jinghao7@illinois.edu> References: <20240204031300.830475-1-jinghao7@illinois.edu> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: pTjztUL61FTrJXA6ja-s8oTry9VD_41U X-Proofpoint-ORIG-GUID: 3fRQU1wk2aq7g_e9smfCVJe3fgw5voOI X-Spam-Details: rule=cautious_plus_nq_notspam policy=cautious_plus_nq score=0 lowpriorityscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402040022 X-Spam-Score: 0 X-Spam-OrigSender: jinghao7@illinois.edu X-Spam-Bar: Both INT (INT n, INT1, INT3, INTO) and UD (UD0, UD1, UD2) serve special purposes in the kernel, e.g., INT3 is used by KGDB and UD2 is involved in LLVM-KCFI instrumentation. At the same time, attaching kprobes on these instructions (particularly UD) will pollute the stack trace dumped in the kernel ring buffer, since the exception is triggered in the copy buffer rather than the original location. Check for INT and UD in can_probe and reject any kprobes trying to attach to these instructions. Suggested-by: Masami Hiramatsu (Google) Signed-off-by: Jinghao Jia --- arch/x86/kernel/kprobes/core.c | 48 +++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 644d416441fb..7a08d6a486c8 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -252,7 +252,28 @@ unsigned long recover_probed_instruction(kprobe_opcode_t *buf, unsigned long add return __recover_probed_insn(buf, addr); } -/* Check if paddr is at an instruction boundary */ +/* Check if insn is INT or UD */ +static inline bool is_exception_insn(struct insn *insn) +{ + /* UD uses 0f escape */ + if (insn->opcode.bytes[0] == 0x0f) { + /* UD0 / UD1 / UD2 */ + return insn->opcode.bytes[1] == 0xff || + insn->opcode.bytes[1] == 0xb9 || + insn->opcode.bytes[1] == 0x0b; + } + + /* INT3 / INT n / INTO / INT1 */ + return insn->opcode.bytes[0] == 0xcc || + insn->opcode.bytes[0] == 0xcd || + insn->opcode.bytes[0] == 0xce || + insn->opcode.bytes[0] == 0xf1; +} + +/* + * Check if paddr is at an instruction boundary and that instruction can + * be probed + */ static bool can_probe(unsigned long paddr) { unsigned long addr, __addr, offset = 0; @@ -291,6 +312,22 @@ static bool can_probe(unsigned long paddr) #endif addr += insn.length; } + + /* Check if paddr is at an instruction boundary */ + if (addr != paddr) + return false; + + __addr = recover_probed_instruction(buf, addr); + if (!__addr) + return false; + + if (insn_decode_kernel(&insn, (void *)__addr) < 0) + return false; + + /* INT and UD are special and should not be kprobed */ + if (is_exception_insn(&insn)) + return false; + if (IS_ENABLED(CONFIG_CFI_CLANG)) { /* * The compiler generates the following instruction sequence @@ -305,13 +342,6 @@ static bool can_probe(unsigned long paddr) * Also, these movl and addl are used for showing expected * type. So those must not be touched. */ - __addr = recover_probed_instruction(buf, addr); - if (!__addr) - return false; - - if (insn_decode_kernel(&insn, (void *)__addr) < 0) - return false; - if (insn.opcode.value == 0xBA) offset = 12; else if (insn.opcode.value == 0x3) @@ -325,7 +355,7 @@ static bool can_probe(unsigned long paddr) } out: - return (addr == paddr); + return true; } /* If x86 supports IBT (ENDBR) it must be skipped. */ From patchwork Sun Feb 4 03:13:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinghao Jia X-Patchwork-Id: 13544448 X-Patchwork-Delegate: mhiramat@kernel.org Received: from mx0a-00007101.pphosted.com (mx0a-00007101.pphosted.com [148.163.135.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E6DD7460; Sun, 4 Feb 2024 04:45:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.135.28 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707021931; cv=none; b=ZtXdMfaoe+rTxU3JegeGxlqoGxsPoeIy3HgpkQg6CNg9ryJdk/xpAOEPe90cS7QsYtE1eb6sWxUxXRjRGaC75fZtHI5ehguSXlOMY809Inu5+U/xH9gGdg4P3y8CaCgqOrMJSCekr99m9qvP1fwwV2fQqNFSu++g3uGnDp532Ts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707021931; c=relaxed/simple; bh=cqAZVRCbOXBgk10uwImR7+eKfgO2Qe7PuU3zufmS+y4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fo+pZO3UaYnCdDpcknXFUTRejfUxJn0rOIetCVGC4NHtESseR7gSVm9vmxa5+JjZlq7mSTR5pM4gnZthjQJozklNlpzq91JJw0YOjyIDjhjPcf6h8NYZm1+3ttBsFI2Vnkr1YUJDhXfrqIlK3S/Attcw9M+zq5rck8EHCBb5/Jc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu; spf=pass smtp.mailfrom=illinois.edu; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b=B9CDCUQN; arc=none smtp.client-ip=148.163.135.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=illinois.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=illinois.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=illinois.edu header.i=@illinois.edu header.b="B9CDCUQN" Received: from pps.filterd (m0166257.ppops.net [127.0.0.1]) by mx0a-00007101.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 4142oQ1A012456; Sun, 4 Feb 2024 03:13:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=illinois.edu; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=campusrelays; bh=UaL3xCx8AUncFg+isg4UuFMjDAHuz/SlcxJjBkpVqCM=; b=B9CDCUQN3zDFBCCGiaUCLDwePzw1IonQjBqfD8408/OH+b+Ua80ddiA2AsDpHy7tbcpK ODw5//Z1iEZEh4gpA2i/izSDqffEMCS76Yb1ObhOeqg84QrW7rkZtyVcklA5WAZxa/Y+ rwIpRrhF5m+Hu9yKEYBQ0LX4XcTTdEMf/qJLs43Ra2T4UbWskLT+jXNw6v8tc63hppT8 S+uOcb9b5Eijqdi/g+iAwCd2AHOo3SRSBBtez5Wc0QJO7/N5CmcxK+TeTBfahrvp/PLT z4WxKpDUM2ecgpSokxHfw9yaXhi4p5A7ZKdalmx635kMDTY0dQ85QUtHZePHbIEUWM8I jg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-00007101.pphosted.com (PPS) with ESMTPS id 3w1e8n4ku0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 04 Feb 2024 03:13:07 +0000 Received: from m0166257.ppops.net (m0166257.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4143D4G2010150; Sun, 4 Feb 2024 03:13:07 GMT Received: from localhost.localdomain (oasis.cs.illinois.edu [130.126.137.13]) by mx0a-00007101.pphosted.com (PPS) with ESMTP id 3w1e8n4ktm-4; Sun, 04 Feb 2024 03:13:07 +0000 From: Jinghao Jia To: "Masami Hiramatsu (Google)" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Xin Li Cc: linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org, Jinghao Jia Subject: [PATCH v2 3/3] x86/kprobes: Boost more instructions from grp2/3/4/5 Date: Sat, 3 Feb 2024 21:13:00 -0600 Message-ID: <20240204031300.830475-4-jinghao7@illinois.edu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240204031300.830475-1-jinghao7@illinois.edu> References: <20240204031300.830475-1-jinghao7@illinois.edu> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: czHbX_K7foO7WJCefQH9zPBPKAETMcJM X-Proofpoint-ORIG-GUID: 9XqJBTkOjiePTMozqFeXhqssPsd8m9Fm X-Spam-Details: rule=cautious_plus_nq_notspam policy=cautious_plus_nq score=0 lowpriorityscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2402040022 X-Spam-Score: 0 X-Spam-OrigSender: jinghao7@illinois.edu X-Spam-Bar: With the instruction decoder, we are now able to decode and recognize instructions with opcode extensions. There are more instructions in these groups that can be boosted: Group 2: ROL, ROR, RCL, RCR, SHL/SAL, SHR, SAR Group 3: TEST, NOT, NEG, MUL, IMUL, DIV, IDIV Group 4: INC, DEC (byte operation) Group 5: INC, DEC (word/doubleword/quadword operation) These instructions are not boosted previously because there are reserved opcodes within the groups, e.g., group 2 with ModR/M.nnn == 110 is unmapped. As a result, kprobes attached to them requires two int3 traps as being non-boostable also prevents jump-optimization. Some simple tests on QEMU show that after boosting and jump-optimization a single kprobe on these instructions with an empty pre-handler runs 10x faster (~1000 cycles vs. ~100 cycles). Since these instructions are mostly ALU operations and do not touch special registers like RIP, let's boost them so that we get the performance benefit. Signed-off-by: Jinghao Jia --- arch/x86/kernel/kprobes/core.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 7a08d6a486c8..530f6d4b34f4 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -169,22 +169,33 @@ bool can_boost(struct insn *insn, void *addr) case 0x62: /* bound */ case 0x70 ... 0x7f: /* Conditional jumps */ case 0x9a: /* Call far */ - case 0xc0 ... 0xc1: /* Grp2 */ case 0xcc ... 0xce: /* software exceptions */ - case 0xd0 ... 0xd3: /* Grp2 */ case 0xd6: /* (UD) */ case 0xd8 ... 0xdf: /* ESC */ case 0xe0 ... 0xe3: /* LOOP*, JCXZ */ case 0xe8 ... 0xe9: /* near Call, JMP */ case 0xeb: /* Short JMP */ case 0xf0 ... 0xf4: /* LOCK/REP, HLT */ - case 0xf6 ... 0xf7: /* Grp3 */ - case 0xfe: /* Grp4 */ /* ... are not boostable */ return false; + case 0xc0 ... 0xc1: /* Grp2 */ + case 0xd0 ... 0xd3: /* Grp2 */ + /* + * AMD uses nnn == 110 as SHL/SAL, but Intel makes it reserved. + */ + return X86_MODRM_REG(insn->modrm.bytes[0]) != 0b110; + case 0xf6 ... 0xf7: /* Grp3 */ + /* AMD uses nnn == 001 as TEST, but Intel makes it reserved. */ + return X86_MODRM_REG(insn->modrm.bytes[0]) != 0b001; + case 0xfe: /* Grp4 */ + /* Only INC and DEC are boostable */ + return X86_MODRM_REG(insn->modrm.bytes[0]) == 0b000 || + X86_MODRM_REG(insn->modrm.bytes[0]) == 0b001; case 0xff: /* Grp5 */ - /* Only indirect jmp is boostable */ - return X86_MODRM_REG(insn->modrm.bytes[0]) == 4; + /* Only INC, DEC, and indirect JMP are boostable */ + return X86_MODRM_REG(insn->modrm.bytes[0]) == 0b000 || + X86_MODRM_REG(insn->modrm.bytes[0]) == 0b001 || + X86_MODRM_REG(insn->modrm.bytes[0]) == 0b100; default: return true; }