From patchwork Wed Oct 2 15:27:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 826DCCF6D39 for ; Wed, 2 Oct 2024 15:27:45 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808898.1220972 (Exim 4.92) (envelope-from ) id 1sw1GL-0007jD-Dr; Wed, 02 Oct 2024 15:27:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808898.1220972; Wed, 02 Oct 2024 15:27:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GL-0007j6-Ax; Wed, 02 Oct 2024 15:27:37 +0000 Received: by outflank-mailman (input) for mailman id 808898; Wed, 02 Oct 2024 15:27:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GK-00070n-KU for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:36 +0000 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [2a00:1450:4864:20::636]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d9c8d053-80d2-11ef-a0ba-8be0dac302b0; Wed, 02 Oct 2024 17:27:35 +0200 (CEST) Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a8a6d1766a7so209487966b.3 for ; Wed, 02 Oct 2024 08:27:35 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:33 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d9c8d053-80d2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882854; x=1728487654; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rlmcJP5mMuj8YOO1ECpxCc4uELbVcUcDRHL7JcNsxfM=; b=ZFPUcU74VORnkol19GRXes2cxs8IQ63GX56evcdMmctNAjjU7ogZFjFcs8pN9b99Hz Ub6BwdXlVhyAAengU6yxljJ5jL/a26+lSt8eUVhkfiWc1hd+i5UswKje0Yu21J+w2KMb e8Xz/gbDBLN/9x/oMtLVkJBjtnxmhOT8vptqY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882854; x=1728487654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rlmcJP5mMuj8YOO1ECpxCc4uELbVcUcDRHL7JcNsxfM=; b=noWsJtqarE4oUMBeF16yd/ij94Pnpn2MfQcTSdSSmX7Ob9b0PKxTDeTPoyKCgCZuew m7IH/drQ71U/4YcCFD+INKXznhh79LFfcPlphtUB8ObTKdEzX1gfiXeTrzY5LiFbZxT+ LHqsqKYzxfW8LppzmGC6XmyF73fIEdeV+SaPTHJ9qxW/XgC+4EmhphJ6E32WNKlYjSCg NyuosxJGbmDuPxbn8M+kIgIhCmZU5y+KesGx7Q2mR9nCzocHQGGe0lNNavLRkOCP89co Tw4Hy50Iy8YEWy2txSnmrmS/KcJpvi5oCNsJ8YqMrnNvbfRN6V7xVGvzyFrqpZMb8/qI qFMw== X-Gm-Message-State: AOJu0YzF0P8e7PNtSAOicm753r3vB6DzDebVLklqKoim7hVbWdd8+87P BP8JEA8JtSj3e3MRxifdSKmfVpP5KJ1I5q9r0jWa5pHUx2ZeweKnApNUWNB5Zhw9EetZBhUhZZr Plxo= X-Google-Smtp-Source: AGHT+IHi8P0+vnqz5tpmJRAAFAvo+luLXAKjODyxsKPUmGnaXXk/W1w7BJkeS7/jdbD1VPXtF1hdeA== X-Received: by 2002:a17:906:db08:b0:a8a:7e24:3230 with SMTP id a640c23a62f3a-a98f8264eddmr337887466b.28.1727882854198; Wed, 02 Oct 2024 08:27:34 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 1/7] x86: Introduce x86_decode_lite() Date: Wed, 2 Oct 2024 16:27:19 +0100 Message-Id: <20241002152725.1841575-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 In order to relocate all IP-relative fields in an alternative replacement block, we need to decode the instructions enough to obtain their length and any relative fields. Full x86_decode() is far too heavyweight, so introduce a minimal form which can make several simplifying assumptions. This a mostly-complete decoder for integer instruction in the onebyte and twobyte maps. Some instructions are intentionally unsupported, owing to being unlikely to find in alternatives, and so as to reduce decode complexity. See the subsequent patch adding a userspace test harness for further details. This logic can decode all alternative blocks that exist in Xen right now. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné v2: * Switch to 0 on failure, rel_sz in bytes * Mostly complete the integer instructions; paird with userspace harness * Put in .init when !CONFIG_LIVEPATCH --- xen/arch/x86/x86_emulate/Makefile | 6 + xen/arch/x86/x86_emulate/decode-lite.c | 311 +++++++++++++++++++++++++ xen/arch/x86/x86_emulate/private.h | 2 + xen/arch/x86/x86_emulate/x86_emulate.h | 14 ++ 4 files changed, 333 insertions(+) create mode 100644 xen/arch/x86/x86_emulate/decode-lite.c diff --git a/xen/arch/x86/x86_emulate/Makefile b/xen/arch/x86/x86_emulate/Makefile index 2e20d65d788a..7a56de877f25 100644 --- a/xen/arch/x86/x86_emulate/Makefile +++ b/xen/arch/x86/x86_emulate/Makefile @@ -6,3 +6,9 @@ obj-y += decode.o obj-$(CONFIG_HVM) += fpu.o obj-y += util.o obj-y += util-xen.o + +ifeq ($(CONFIG_LIVEPATCH),y) +obj-y += decode-lite.o +else +obj-bin-y += decode-lite.init.o +endif diff --git a/xen/arch/x86/x86_emulate/decode-lite.c b/xen/arch/x86/x86_emulate/decode-lite.c new file mode 100644 index 000000000000..6e2581eeab83 --- /dev/null +++ b/xen/arch/x86/x86_emulate/decode-lite.c @@ -0,0 +1,311 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include "private.h" + +#define Imm8 (1 << 0) +#define Imm (1 << 1) +#define Moffs (1 << 2) +#define Branch (1 << 5) /* ... that we care about */ +/* ModRM (1 << 6) */ +#define Known (1 << 7) + +#define ALU_OPS \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|ModRM), \ + (Known|Imm8), \ + (Known|Imm) + +static const uint8_t init_or_livepatch_const onebyte[256] = { + [0x00] = ALU_OPS, /* ADD */ [0x08] = ALU_OPS, /* OR */ + [0x10] = ALU_OPS, /* ADC */ [0x18] = ALU_OPS, /* SBB */ + [0x20] = ALU_OPS, /* AND */ [0x28] = ALU_OPS, /* SUB */ + [0x30] = ALU_OPS, /* XOR */ [0x38] = ALU_OPS, /* CMP */ +/* [0x40 ... 0x4f] = REX prefixes */ + [0x50 ... 0x5f] = (Known), /* PUSH/POP %reg */ + + [0x63] = (Known|ModRM), /* MOVSxd */ + + [0x68] = (Known|Imm), /* PUSH $imm */ + [0x69] = (Known|ModRM|Imm), /* IMUL $imm/r/rm */ + [0x6a] = (Known|Imm8), /* PUSH $imm8 */ + [0x6b] = (Known|ModRM|Imm8), /* PUSH $imm8/r/rm */ + [0x6c ... 0x6f] = (Known), /* INS/OUTS */ + + [0x70 ... 0x7f] = (Known|Branch|Imm8), /* Jcc disp8 */ + [0x80] = (Known|ModRM|Imm8), /* Grp1 */ + [0x81] = (Known|ModRM|Imm), /* Grp1 */ + + [0x83] = (Known|ModRM|Imm8), /* Grp1 */ + [0x84 ... 0x8e] = (Known|ModRM), /* TEST/XCHG/MOV/MOV-SREG/LEA r/rm */ + + [0x90 ... 0x99] = (Known), /* NOP/XCHG rAX/CLTQ/CQTO */ + + [0x9b ... 0x9f] = (Known), /* FWAIT/PUSHF/POPF/SAHF/LAHF */ + + [0xa0 ... 0xa3] = (Known|Moffs), /* MOVABS */ + [0xa4 ... 0xa7] = (Known), /* MOVS/CMPS */ + [0xa8] = (Known|Imm8), /* TEST %al */ + [0xa9] = (Known|Imm), /* TEST %al */ + [0xaa ... 0xaf] = (Known), /* STOS/LODS/SCAS */ + [0xb0 ... 0xb7] = (Known|Imm8), /* MOV $imm8, %reg */ + [0xb8 ... 0xbf] = (Known|Imm), /* MOV $imm{16,32,64}, %reg */ + [0xc0 ... 0xc1] = (Known|ModRM|Imm8), /* Grp2 (ROL..SAR $imm8, %reg) */ + + [0xc3] = (Known), /* RET */ + [0xc6] = (Known|ModRM|Imm8), /* Grp11, Further ModRM decode */ + [0xc7] = (Known|ModRM|Imm), /* Grp11, Further ModRM decode */ + + [0xcb ... 0xcc] = (Known), /* LRET/INT3 */ + [0xcd] = (Known|Imm8), /* INT $imm8 */ + + [0xd0 ... 0xd3] = (Known|ModRM), /* Grp2 (ROL..SAR {$1,%cl}, %reg) */ + + [0xe4 ... 0xe7] = (Known|Imm8), /* IN/OUT $imm8 */ + [0xe8 ... 0xe9] = (Known|Branch|Imm), /* CALL/JMP disp32 */ + [0xeb] = (Known|Branch|Imm8), /* JMP disp8 */ + [0xec ... 0xef] = (Known), /* IN/OUT %dx */ + + [0xf1] = (Known), /* ICEBP */ + [0xf4] = (Known), /* HLT */ + [0xf5] = (Known), /* CMC */ + [0xf6 ... 0xf7] = (Known|ModRM), /* Grp3, Further ModRM decode */ + [0xf8 ... 0xfd] = (Known), /* CLC ... STD */ + [0xfe ... 0xff] = (Known|ModRM), /* Grp4 */ +}; +static const uint8_t init_or_livepatch_const twobyte[256] = { + [0x00 ... 0x03] = (Known|ModRM), /* Grp6/Grp7/LAR/LSL */ + [0x06] = (Known), /* CLTS */ + [0x09] = (Known), /* WBINVD */ + [0x0b] = (Known), /* UD2 */ + + [0x18 ... 0x1f] = (Known|ModRM), /* Grp16 (Hint Nop) */ + [0x20 ... 0x23] = (Known|ModRM), /* MOV %cr/%dr */ + + [0x30 ... 0x33] = (Known), /* WRMSR/RDTSC/RDMSR/RDPMC */ + + [0x40 ... 0x4f] = (Known|ModRM), /* CMOVcc */ + + [0x80 ... 0x8f] = (Known|Branch|Imm), /* Jcc disp32 */ + [0x90 ... 0x9f] = (Known|ModRM), /* SETcc */ + + [0xa0 ... 0xa2] = (Known), /* PUSH/POP %fs/CPUID */ + [0xa3] = (Known|ModRM), /* BT r/rm */ + [0xa4] = (Known|ModRM|Imm8), /* SHLD $imm8 */ + [0xa5] = (Known|ModRM), /* SHLD %cl */ + + [0xa8 ... 0xa9] = (Known), /* PUSH/POP %gs */ + + [0xab] = (Known|ModRM), /* BTS */ + [0xac] = (Known|ModRM|Imm8), /* SHRD $imm8 */ + [0xad ... 0xaf] = (Known|ModRM), /* SHRD %cl/Grp15/IMUL */ + + [0xb0 ... 0xb9] = (Known|ModRM), /* CMPXCHG/LSS/BTR/LFS/LGS/MOVZxx/POPCNT/Grp10 */ + [0xba] = (Known|ModRM|Imm8), /* Grp8 */ + [0xbb ... 0xbf] = (Known|ModRM), /* BSR/BSF/BSR/MOVSX */ + [0xc0 ... 0xc1] = (Known|ModRM), /* XADD */ + [0xc7] = (Known|ModRM), /* Grp9 */ + [0xc8 ... 0xcf] = (Known), /* BSWAP */ +}; + +/* + * Bare minimum x86 instruction decoder to parse the alternative replacement + * instructions and locate the IP-relative references that may need updating. + * + * These are: + * - disp8/32 from near branches + * - RIP-relative memory references + * + * The following simplifications are used: + * - All code is 64bit, and the instruction stream is safe to read. + * - The 67 prefix is not implemented, so the address size is only 64bit. + * + * Inputs: + * @ip The position to start decoding from. + * @end End of the replacement block. Exceeding this is considered an error. + * + * Returns: x86_decode_lite_t + * - On failure, length of 0. + * - On success, length > 0. For rel_sz > 0, rel points at the relative + * field in the instruction stream. + */ +x86_decode_lite_t init_or_livepatch x86_decode_lite(void *ip, void *end) +{ + void *start = ip, *rel = NULL; + unsigned int opc, rel_sz = 0; + uint8_t b, d, rex = 0, osize = 4; + +#define OPC_TWOBYTE (1 << 8) + + /* Mutates IP, uses END. */ +#define FETCH(ty) \ + ({ \ + ty _val; \ + \ + if ( (ip + sizeof(ty)) > end ) \ + goto overrun; \ + _val = *(ty *)ip; \ + ip += sizeof(ty); \ + _val; \ + }) + + for ( ;; ) /* Prefixes */ + { + switch ( b = FETCH(uint8_t) ) + { + case 0x26: /* ES override */ + case 0x2e: /* CS override */ + case 0x36: /* DS override */ + case 0x3e: /* SS override */ + case 0x64: /* FS override */ + case 0x65: /* GS override */ + case 0xf0: /* LOCK */ + case 0xf2: /* REPNE */ + case 0xf3: /* REP */ + break; + + case 0x66: /* Operand size override */ + osize = 2; + break; + + /* case 0x67: Address size override, not implemented */ + + case 0x40 ... 0x4f: /* REX */ + rex = b; + continue; + + default: + goto prefixes_done; + } + rex = 0; /* REX cancelled by subsequent legacy prefix. */ + } + prefixes_done: + + if ( rex & 0x08 ) /* REX.W */ + osize = 8; + + /* Fetch the main opcode byte(s) */ + if ( b == 0x0f ) + { + b = FETCH(uint8_t); + opc = OPC_TWOBYTE | b; + + d = twobyte[b]; + } + else + { + opc = b; + d = onebyte[b]; + } + + if ( unlikely(!(d & Known)) ) + goto unknown; + + if ( d & ModRM ) + { + uint8_t modrm = FETCH(uint8_t); + uint8_t mod = modrm >> 6; + uint8_t reg = (modrm >> 3) & 7; + uint8_t rm = modrm & 7; + + /* ModRM/SIB decode */ + if ( mod == 0 && rm == 5 ) /* RIP relative */ + { + rel = ip; + rel_sz = 4; + FETCH(uint32_t); + } + else if ( mod != 3 && rm == 4 ) /* SIB */ + { + uint8_t sib = FETCH(uint8_t); + uint8_t base = sib & 7; + + if ( mod == 0 && base == 5 ) + goto disp32; + } + + if ( mod == 1 ) /* disp8 */ + FETCH(uint8_t); + else if ( mod == 2 ) /* disp32 */ + { + disp32: + FETCH(uint32_t); + } + + /* ModRM based decode adjustements */ + switch ( opc ) + { + case 0xc7: /* Grp11 XBEGIN is a branch. */ + if ( modrm == 0xf8 ) + d |= Branch; + break; + case 0xf6: /* Grp3 TEST(s) have extra Imm8 */ + if ( reg == 0 || reg == 1 ) + d |= Imm8; + break; + case 0xf7: /* Grp3 TEST(s) have extra Imm */ + if ( reg == 0 || reg == 1 ) + d |= Imm; + break; + } + } + + if ( d & Branch ) + { + /* + * We don't tolerate 66-prefixed call/jmp in alternatives. Some are + * genuinely decoded differently between Intel and AMD CPUs. + * + * We also don't support APX instructions, so don't have to cope with + * JMPABS which is the first branch to have an 8-byte immediate. + */ + if ( osize < 4 ) + goto bad_osize; + + rel = ip; + rel_sz = (d & Imm8) ? 1 : 4; + } + + if ( d & (Imm | Imm8 | Moffs) ) + { + if ( d & Imm8 ) + osize = 1; + else if ( d & Moffs ) + osize = 8; + else if ( osize == 8 && !(opc >= 0xb8 && opc <= 0xbf) ) + osize = 4; + + switch ( osize ) + { + case 1: FETCH(uint8_t); break; + case 2: FETCH(uint16_t); break; + case 4: FETCH(uint32_t); break; + case 8: FETCH(uint64_t); break; + default: goto bad_osize; + } + } + + return (x86_decode_lite_t){ ip - start, rel_sz, rel }; + + bad_osize: + printk(XENLOG_ERR "%s() Bad osize %u in %*ph\n", + __func__, osize, + (int)(unsigned long)(end - start), start); + return (x86_decode_lite_t){ 0, 0, NULL }; + + unknown: + printk(XENLOG_ERR "%s() Unknown opcode in %*ph <%02x> %*ph\n", + __func__, + (int)(unsigned long)(ip - 1 - start), start, b, + (int)(unsigned long)(end - ip), ip); + return (x86_decode_lite_t){ 0, 0, NULL }; + + overrun: + printk(XENLOG_ERR "%s() Decode overrun, got %*ph\n", + __func__, + (int)(unsigned long)(end - start), start); + return (x86_decode_lite_t){ 0, 0, NULL }; + +#undef FETCH +} diff --git a/xen/arch/x86/x86_emulate/private.h b/xen/arch/x86/x86_emulate/private.h index 172270a458bd..c5e0983948ce 100644 --- a/xen/arch/x86/x86_emulate/private.h +++ b/xen/arch/x86/x86_emulate/private.h @@ -9,7 +9,9 @@ #ifdef __XEN__ # include +# include # include +# include # include # include # include diff --git a/xen/arch/x86/x86_emulate/x86_emulate.h b/xen/arch/x86/x86_emulate/x86_emulate.h index 3e819d41746e..01c371057626 100644 --- a/xen/arch/x86/x86_emulate/x86_emulate.h +++ b/xen/arch/x86/x86_emulate/x86_emulate.h @@ -846,4 +846,18 @@ static inline void x86_emul_reset_event(struct x86_emulate_ctxt *ctxt) ctxt->event = (struct x86_event){}; } +/* + * x86_decode_lite(). Very minimal decoder for managing alternatives. + * + * @len is 0 on error, or nonzero on success. If the instruction has a + * relative field, @rel_sz is nonzero, and @rel points at the field. + */ +typedef struct { + uint8_t len; + uint8_t rel_sz; /* bytes: 0, 1 or 4 */ + void *rel; +} x86_decode_lite_t; + +x86_decode_lite_t x86_decode_lite(void *ip, void *end); + #endif /* __X86_EMULATE_H__ */ From patchwork Wed Oct 2 15:27:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 831DDCF6D3A for ; Wed, 2 Oct 2024 15:27:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808901.1220992 (Exim 4.92) (envelope-from ) id 1sw1GP-0008Jd-0j; Wed, 02 Oct 2024 15:27:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808901.1220992; Wed, 02 Oct 2024 15:27:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GO-0008JM-TT; Wed, 02 Oct 2024 15:27:40 +0000 Received: by outflank-mailman (input) for mailman id 808901; Wed, 02 Oct 2024 15:27:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GN-0007h7-IP for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:39 +0000 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [2a00:1450:4864:20::644]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id dade0930-80d2-11ef-99a2-01e77a169b0f; Wed, 02 Oct 2024 17:27:37 +0200 (CEST) Received: by mail-ej1-x644.google.com with SMTP id a640c23a62f3a-a8a7cdfdd80so1116046666b.0 for ; Wed, 02 Oct 2024 08:27:37 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dade0930-80d2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882856; x=1728487656; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BltK+sBNZSyWmrxnW27KxfncArL/nZis+9sjT7JLt9M=; b=uXIT7NRw9hnvWJfJ5XhJ4nTEdBKGqdqnvw19b99x/V/n3HfCpczFQREZxZPtj8bLF+ /YMWVcxM4VCgJPGEa62cop5U7ravFLZCazyW2mwqe0DyBroaoIiAvOKb8B23/Dn+PwbD Yx+N+5VG10IfwnIJsPMVAl11vvsoX+4Aua5Zc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882856; x=1728487656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BltK+sBNZSyWmrxnW27KxfncArL/nZis+9sjT7JLt9M=; b=d9U9Y9Zmm62q2NreFUN95Lhq+0dbZuVEGOVAlH8+I7JPKiTbUi+2rs97hKWiKSm4ix 2OKccRV5xPAxk55EuVKTYGVHLaTWcG5czsdrRkWe4Mrqynuglq/ELDVVYngdLOUKpOcS 7QwXmoy1z8jYYPtFK9n+msit8a+nAZVb9ThkhhpBl7HEytzzsAssb22trr7LJsoG1HcH oUSAFzb/fkoXAPRoW6ApBDqEPrOYkz/Twhy+nXOvvXwdVUHtYGX0HDXP+eDPEboFrirg HKTKNnm+84bLWhfEHFP61WPfaRB3pb2jESmoKeWkQyh6wnJDJCF7gM5Gzv3sy4Gi3GGK 3GcQ== X-Gm-Message-State: AOJu0YwQS+p0XKe/NDyZz4ImOGb3e/jlp+RUXibdZbUf6uFZ98CBfYaq RsdSpX3bX/wIfEnrnEsu+CBRu9T2C7ys6aMWecgzzqn/Y4vqDewqUpvQSNigvLEDzc091SdEnRF NF5EubA== X-Google-Smtp-Source: AGHT+IFfIxycWefXjCr/rq9pvUidEXWNsEmaJjE9ByX/OdNQkr1mKl8cl7dYCbuq0x4vCOupuQsE7g== X-Received: by 2002:a17:907:7d87:b0:a8a:6db5:7e42 with SMTP id a640c23a62f3a-a98f821cc3fmr345093266b.9.1727882856284; Wed, 02 Oct 2024 08:27:36 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 2/7] tests/x86: Introduce a userspace test harness for x86_decode_lite() Date: Wed, 2 Oct 2024 16:27:20 +0100 Message-Id: <20241002152725.1841575-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 All the interesting behaviour is in insns.S. There are 4 interesting cases; "not an instruction we tolerate", or one we do tolerate, split by no relation, disp8 or disp32. The DECL()/END() macros start and terminate the tests_*[] arrays used by C. Between DECL()/END(), a macro named _ adds an entry into the array, including a name and the length of the instruction according to the assembler, while being as visually unintrusive as possible. Plain labels are ad-hoc and there to aid legibility during disassembly. In a couple of cases, the macro named n (for name) allows for choosing a name manually, and is used for cases where the assembler doesn't like the mnemonic. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné v2: * New Despite claiming full APX support in the 2.43 release, binutils trunk doesn't tolerate JMPABS at all. Clang-IAS like it but only when encoded as an immediate, despite the fact the operand should be a moffset and encoded without a $ prefix. https://godbolt.org/z/P4Ph3svha Back to this patch, I can't find any way to get Clang happy with rex.w for explicit prefixing. I suspect we're just going to need to ignore this test case for clang=y. Also, Clang and GAS disagree on needing .allow_index_reg for %riz. --- tools/tests/Makefile | 1 + tools/tests/x86-decode-lite/.gitignore | 1 + tools/tests/x86-decode-lite/Makefile | 45 ++ tools/tests/x86-decode-lite/insns.S | 703 ++++++++++++++++++++++ tools/tests/x86-decode-lite/macro-magic.h | 52 ++ tools/tests/x86-decode-lite/main.c | 111 ++++ tools/tests/x86-decode-lite/x86-emulate.h | 27 + 7 files changed, 940 insertions(+) create mode 100644 tools/tests/x86-decode-lite/.gitignore create mode 100644 tools/tests/x86-decode-lite/Makefile create mode 100644 tools/tests/x86-decode-lite/insns.S create mode 100644 tools/tests/x86-decode-lite/macro-magic.h create mode 100644 tools/tests/x86-decode-lite/main.c create mode 100644 tools/tests/x86-decode-lite/x86-emulate.h diff --git a/tools/tests/Makefile b/tools/tests/Makefile index 1319c3a9d88c..6a8844673abc 100644 --- a/tools/tests/Makefile +++ b/tools/tests/Makefile @@ -5,6 +5,7 @@ SUBDIRS-y := SUBDIRS-y += resource SUBDIRS-$(CONFIG_X86) += cpu-policy SUBDIRS-$(CONFIG_X86) += tsx +SUBDIRS-$(CONFIG_X86) += x86-decode-lite ifneq ($(clang),y) SUBDIRS-$(CONFIG_X86) += x86_emulator endif diff --git a/tools/tests/x86-decode-lite/.gitignore b/tools/tests/x86-decode-lite/.gitignore new file mode 100644 index 000000000000..e726b493c993 --- /dev/null +++ b/tools/tests/x86-decode-lite/.gitignore @@ -0,0 +1 @@ +test-x86-decode-lite diff --git a/tools/tests/x86-decode-lite/Makefile b/tools/tests/x86-decode-lite/Makefile new file mode 100644 index 000000000000..6e8ef21e1c29 --- /dev/null +++ b/tools/tests/x86-decode-lite/Makefile @@ -0,0 +1,45 @@ +XEN_ROOT = $(CURDIR)/../../.. +include $(XEN_ROOT)/tools/Rules.mk + +TARGET := test-x86-decode-lite + +.PHONY: all +all: $(TARGET) + +.PHONY: run +run: $(TARGET) + ./$< + +.PHONY: clean +clean: + $(RM) -- *.o $(TARGET) $(DEPS_RM) + +.PHONY: distclean +distclean: clean + $(RM) -- *~ + +.PHONY: install +install: all + $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN) + $(INSTALL_PROG) $(TARGET) $(DESTDIR)$(LIBEXEC_BIN) + +.PHONY: uninstall +uninstall: + $(RM) -- $(DESTDIR)$(LIBEXEC_BIN)/$(TARGET) + +.PHONY: uninstall +uninstall: + +vpath %.c ../../../xen/arch/x86/x86_emulate + +CFLAGS += $(CFLAGS_xeninclude) -I. -I../../../xen/arch/x86/ +CFLAGS += $(APPEND_CFLAGS) + +LDFLAGS += $(APPEND_LDFLAGS) + +%.o: Makefile + +$(TARGET): main.o insns.o decode-lite.o + $(CC) -o $@ $^ $(LDFLAGS) + +-include $(DEPS_INCLUDE) diff --git a/tools/tests/x86-decode-lite/insns.S b/tools/tests/x86-decode-lite/insns.S new file mode 100644 index 000000000000..a0f938d76905 --- /dev/null +++ b/tools/tests/x86-decode-lite/insns.S @@ -0,0 +1,703 @@ +#include "macro-magic.h" + + .code64 + +#ifndef __clang__ + .allow_index_reg +#endif + .text + +DECL(tests_rel0) +modrm: + /* Mod=0, Reg=0, RM {0..f} */ + _ add %al, (%rax) + _ add %al, (%rcx) + _ add %al, (%rdx) + _ add %al, (%rbx) + _ add %al, (%rsp) /* SIB */ + /*add %al, (%rbp) RIP --> tests_rel4 */ + _ add %al, (%rsi) + _ add %al, (%rdi) + _ add %al, (%r8) + _ add %al, (%r9) + _ add %al, (%r10) + _ add %al, (%r11) + _ add %al, (%r12) /* SIB */ + /*add %al, (%r13) RIP --> tests_rel4 */ + _ add %al, (%r14) + _ add %al, (%r15) + + /* Mod=1, Reg=0, RM {0..f} */ + _ add %al, 0x01(%rax) + _ add %al, 0x01(%rcx) + _ add %al, 0x01(%rdx) + _ add %al, 0x01(%rbx) + _ add %al, 0x01(%rsp) /* SIB */ + _ add %al, 0x01(%rbp) + _ add %al, 0x01(%rsi) + _ add %al, 0x01(%rdi) + _ add %al, 0x01(%r8) + _ add %al, 0x01(%r9) + _ add %al, 0x01(%r10) + _ add %al, 0x01(%r11) + _ add %al, 0x01(%r12) /* SIB */ + _ add %al, 0x01(%r13) + _ add %al, 0x01(%r14) + _ add %al, 0x01(%r15) + + /* Mod=2, Reg=0, RM {0..f} */ + _ add %al, 0x7f000001(%rax) + _ add %al, 0x7f000001(%rcx) + _ add %al, 0x7f000001(%rdx) + _ add %al, 0x7f000001(%rbx) + _ add %al, 0x7f000001(%rsp) /* SIB */ + _ add %al, 0x7f000001(%rbp) + _ add %al, 0x7f000001(%rsi) + _ add %al, 0x7f000001(%rdi) + _ add %al, 0x7f000001(%r8) + _ add %al, 0x7f000001(%r9) + _ add %al, 0x7f000001(%r10) + _ add %al, 0x7f000001(%r11) + _ add %al, 0x7f000001(%r12) /* SIB */ + _ add %al, 0x7f000001(%r13) + _ add %al, 0x7f000001(%r14) + _ add %al, 0x7f000001(%r15) + + /* Mod=3, Reg=0, RM {0..f} */ + _ add %al, %al + _ add %al, %cl + _ add %al, %dl + _ add %al, %bl + _ add %al, %ah + _ add %al, %ch + _ add %al, %dh + _ add %al, %dl + _ add %al, %r8b + _ add %al, %r9b + _ add %al, %r10b + _ add %al, %r11b + _ add %al, %r12b + _ add %al, %r13b + _ add %al, %r14b + _ add %al, %r15b + +sib: + /* Mod=0, Reg=0, RM=4, SIB S=3, I=0, B {0..f} */ + _ add %al, (%rax, %rax, 8) + _ add %al, (%rcx, %rax, 8) + _ add %al, (%rdx, %rax, 8) + _ add %al, (%rbx, %rax, 8) + _ add %al, (%rsp, %rax, 8) + _ add %al, ( , %rax, 8) /* "none", %rbp encoded with mod=1/2 */ + _ add %al, (%rsi, %rax, 8) + _ add %al, (%rdi, %rax, 8) + _ add %al, (%r8, %rax, 8) + _ add %al, (%r9, %rax, 8) + _ add %al, (%r10, %rax, 8) + _ add %al, (%r11, %rax, 8) + _ add %al, (%r12, %rax, 8) + _ rex.b add %al,(,%rax, 8) /* "none", %r13 encoded with mod=1/2 */ + _ add %al, (%r14, %rax, 8) + _ add %al, (%r15, %rax, 8) + + /* Mod=1, Reg=0, RM=4, SIB S=3, I=0, B {0..f} */ + _ add %al, 0x01(%rax, %rax, 8) + _ add %al, 0x01(%rcx, %rax, 8) + _ add %al, 0x01(%rdx, %rax, 8) + _ add %al, 0x01(%rbx, %rax, 8) + _ add %al, 0x01(%rsp, %rax, 8) + _ add %al, 0x01(%rbp, %rax, 8) + _ add %al, 0x01(%rsi, %rax, 8) + _ add %al, 0x01(%rdi, %rax, 8) + _ add %al, 0x01(%r8, %rax, 8) + _ add %al, 0x01(%r9, %rax, 8) + _ add %al, 0x01(%r10, %rax, 8) + _ add %al, 0x01(%r11, %rax, 8) + _ add %al, 0x01(%r12, %rax, 8) + _ add %al, 0x01(%r13, %rax, 8) + _ add %al, 0x01(%r14, %rax, 8) + _ add %al, 0x01(%r15, %rax, 8) + + /* Mod=2, Reg=0, RM=4, SIB S=3, I=0, B {0..f} */ + _ add %al, 0x7f000001(%rax, %rax, 8) + _ add %al, 0x7f000001(%rcx, %rax, 8) + _ add %al, 0x7f000001(%rdx, %rax, 8) + _ add %al, 0x7f000001(%rbx, %rax, 8) + _ add %al, 0x7f000001(%rsp, %rax, 8) + _ add %al, 0x7f000001(%rbp, %rax, 8) + _ add %al, 0x7f000001(%rsi, %rax, 8) + _ add %al, 0x7f000001(%rdi, %rax, 8) + _ add %al, 0x7f000001(%r8, %rax, 8) + _ add %al, 0x7f000001(%r9, %rax, 8) + _ add %al, 0x7f000001(%r10, %rax, 8) + _ add %al, 0x7f000001(%r11, %rax, 8) + _ add %al, 0x7f000001(%r12, %rax, 8) + _ add %al, 0x7f000001(%r13, %rax, 8) + _ add %al, 0x7f000001(%r14, %rax, 8) + _ add %al, 0x7f000001(%r15, %rax, 8) + + /* Mod=0, Reg=0, RM=4, SIB S=3, I=4, B {0..f} */ + _ add %al, (%rax, %riz, 8) + _ add %al, (%rcx, %riz, 8) + _ add %al, (%rdx, %riz, 8) + _ add %al, (%rbx, %riz, 8) + _ add %al, (%rsp, %riz, 8) + _ add %al, ( , %riz, 8) /* %rbp encoded with mod=1/2 */ + _ add %al, (%rsi, %riz, 8) + _ add %al, (%rdi, %riz, 8) + _ add %al, (%r8, %riz, 8) + _ add %al, (%r9, %riz, 8) + _ add %al, (%r10, %riz, 8) + _ add %al, (%r11, %riz, 8) + _ add %al, (%r12, %riz, 8) + _ rex.b add %al,(,%riz, 8) /* %r13 encoded with mod=1/2 */ + _ add %al, (%r14, %riz, 8) + _ add %al, (%r15, %riz, 8) + + /* Mod=1, Reg=0, RM=4, SIB S=3, I=4, B {0..f} */ + _ add %al, 0x01(%rax, %riz, 8) + _ add %al, 0x01(%rcx, %riz, 8) + _ add %al, 0x01(%rdx, %riz, 8) + _ add %al, 0x01(%rbx, %riz, 8) + _ add %al, 0x01(%rsp, %riz, 8) + _ add %al, 0x01(%rbp, %riz, 8) + _ add %al, 0x01(%rsi, %riz, 8) + _ add %al, 0x01(%rdi, %riz, 8) + _ add %al, 0x01(%r8, %riz, 8) + _ add %al, 0x01(%r9, %riz, 8) + _ add %al, 0x01(%r10, %riz, 8) + _ add %al, 0x01(%r11, %riz, 8) + _ add %al, 0x01(%r12, %riz, 8) + _ add %al, 0x01(%r13, %riz, 8) + _ add %al, 0x01(%r14, %riz, 8) + _ add %al, 0x01(%r15, %riz, 8) + + /* Mod=2, Reg=0, RM=4, SIB S=3, I=4, B {0..f} */ + _ add %al, 0x7f000001(%rax, %riz, 8) + _ add %al, 0x7f000001(%rcx, %riz, 8) + _ add %al, 0x7f000001(%rdx, %riz, 8) + _ add %al, 0x7f000001(%rbx, %riz, 8) + _ add %al, 0x7f000001(%rsp, %riz, 8) + _ add %al, 0x7f000001(%rbp, %riz, 8) + _ add %al, 0x7f000001(%rsi, %riz, 8) + _ add %al, 0x7f000001(%rdi, %riz, 8) + _ add %al, 0x7f000001(%r8, %riz, 8) + _ add %al, 0x7f000001(%r9, %riz, 8) + _ add %al, 0x7f000001(%r10, %riz, 8) + _ add %al, 0x7f000001(%r11, %riz, 8) + _ add %al, 0x7f000001(%r12, %riz, 8) + _ add %al, 0x7f000001(%r13, %riz, 8) + _ add %al, 0x7f000001(%r14, %riz, 8) + _ add %al, 0x7f000001(%r15, %riz, 8) + + .macro alu_ops op + _ \op %al, (%rax) + _ \op %eax, (%rax) + _ \op (%rax), %al + _ \op (%rax), %eax + _ \op $1, %al + _ \op $0x7f000001, %eax + + /* Vary osize on imm fields */ + _ data16 \op $1, %al + _ rex.w \op $1, %al + _ data16 rex.w \op $1, %al + + _ \op $0x7f01, %ax + _ \op $0x7f000001, %rax + _ data16 \op $0x7f000001, %rax + .endm + +onebyte_row_0x: + alu_ops add + alu_ops or + +onebyte_row_1x: + alu_ops adc + alu_ops sbb + +onebyte_row_2x: + alu_ops and + .code32 + _ es nop + .code64 + alu_ops sub + _ cs nop + +onebyte_row_3x: + alu_ops xor + .code32 + _ ss nop + .code64 + alu_ops cmp + _ ds nop + +/* onebyte_row_4x --> rex prefixes */ + +onebyte_row_5x: + _ push %rax + _ push %rcx + _ push %rdx + _ push %rbx + _ push %rsp + _ push %rbp + _ push %rsi + _ push %rdi + _ pop %rax + _ pop %rcx + _ pop %rdx + _ pop %rbx + _ pop %rsp + _ pop %rbp + _ pop %rsi + _ pop %rdi + +onebyte_row_6x: + /*pusha,popa,bound --> not supported */ + _ movsxd (%rax), %eax + _ movslq (%rax), %rax + _ fs nop + _ gs nop + _ data16 nop + /* addr32 --> not supported */ + _ pushq $0x7f000001 + _ pushw $0x7f01 + _ rex.w pushq $0x7f000001 + _ imul $0x7f01, %ax, %ax + _ imul $0x7f000001, %eax, %eax + _ imul $0x7f000001, %rax, %rax + _ pushq $0 + _ pushw $0 + _ rex.w pushq $0 + _ imul $0, %ax, %ax + _ imul $0, %eax, %eax + _ imul $0, %rax, %rax + _ insb + _ insw + _ insl + _ outsb + _ outsw + _ outsl + +/* onebyte_row_7x: --> Jcc disp8 */ + +onebyte_row_8x: + _ add $0, %cl /* Grp1 */ + _ data16 add $0, %cl + _ rex.w add $0, %cl + _ add $0x7f01, %cx + _ add $0x7f000001, %ecx + _ add $0x7f000001, %rcx + _ add $0, %cx + _ add $0, %ecx + _ add $0, %rcx + _ test %cl, %cl + _ test %ecx, %ecx + _ xchg %cl, %cl + _ xchg %ecx, %ecx + _ mov %cl, (%rax) + _ mov %ecx, (%rax) + _ mov (%rax), %cl + _ mov (%rax), %ecx + _ mov %cs, (%rax) + _ lea (%rax), %eax + _ mov (%rax), %cs + /*pop mem --> Grp1a, Not supported (XOP prefix adjacent) */ + +onebyte_row_9x: + _ nop + _ pause + _ xchg %ax, %ax + _ xchg %eax, %eax + _ xchg %rax, %rax + _ rex.w xchg %rax, %rax + _ cltq + _ cqto + _ wait + _ pushf + _ popf + _ sahf + _ lahf + +onebyte_row_ax: + _ mov 0x8000000000000001, %al + _ mov 0x8000000000000001, %ax + _ mov 0x8000000000000001, %eax + _ mov 0x8000000000000001, %rax + _ mov %al, 0x8000000000000001 + _ mov %ax, 0x8000000000000001 + _ mov %eax, 0x8000000000000001 + _ mov %rax, 0x8000000000000001 + _ movsb + _ movsl + _ cmpsb + _ cmpsl + _ test $0, %al + _ test $0x80000001, %eax + _ test $0x7f000001, %rax + _ stosb + _ stosl + _ lodsb + _ lodsl + _ scasb + _ scasl + +onebyte_row_bx: + _ mov $0, %al + _ mov $0, %cl + _ mov $0x7f01, %ax + _ mov $0x7f01, %cx + _ mov $0x7f000001, %eax + _ mov $0x7f000001, %ecx + _ mov $0x7f00000000000001, %rax + _ mov $0x7f00000000000001, %rcx + +onebyte_row_cx: + _ rol $0, %al /* Grp2 */ + _ rol $0, %ax + _ rol $0, %eax + _ rol $0, %rax + /*ret $0 --> not supported */ + _ ret + /*les,lds --> not supported */ + _ movb $0, (%rax) /* Grp11 */ + _ movw $0, (%rax) + _ movl $0, (%rax) + _ movq $0, (%rax) + /*xbegin (Grp11) --> disp32 */ + /*enter,leave,lretq $0 --> not supported */ + _ lretq + _ int3 + _ int $0 + /*into,iret --> not supported */ + +onebyte_row_dx: + _ rol $1, %al /* Grp2 */ + _ rol $1, %ax + _ rol $1, %eax + _ rol $1, %rax + _ rol %cl, %al + _ rol %cl, %ax + _ rol %cl, %eax + _ rol %cl, %rax + /*aam,aad,setalc,xlat,d8...df --> not supported */ + +onebyte_row_ex: + /*loop{ne,e,},jrcxz --> not supported */ + _ in $0, %al + _ in $0, %eax + _ out %al, $0 + _ out %eax, $0 + /*call,jmp --> disp32 */ + /*ljmp --> not supported */ + /*jmp --> disp8 */ + _ in %dx, %al + _ in %dx, %eax + _ out %al, %dx + _ out %eax, %dx + +onebyte_row_fx: + _ lock addb $0, (%rax) + n "icebp" .byte 0xf1 /* icebp */ + _ repne nop + _ repe nop + _ hlt + _ cmc + _ test $0, %cl /* Grp3, /0 has extra Imm{8,} */ + _ not %cl + _ test $0x7f01, %cx + _ not %cx + _ test $0x7f000001, %ecx + _ not %ecx + _ test $0x7f000001, %rcx + _ not %rcx + _ clc + _ stc + _ cli + _ sti + _ cld + _ std + _ inc %cl /* Grp4 */ + _ dec %cl + _ inc %ecx /* Grp5 */ + _ dec %ecx + _ call *(%rax) + _ lcall *(%rax) + _ jmp *(%rax) + _ ljmp *(%rax) + _ push (%rax) + +twobyte_row_0x: + _ sldt (%rax) /* Grp6 */ + _ sgdt (%rax) /* Grp7 */ + _ lar (%rax), %eax + _ lsl (%rax), %eax + _ clts + _ wbinvd + _ ud2a + +twobyte_row_1x: + _ prefetchnta (%rax) /* Grp16 (Hint Nop) */ + _ nopl (%rax) + +twobyte_row_2x: + _ mov %cr0, %rax + _ mov %dr0, %rax + _ mov %rax, %cr0 + _ mov %rax, %dr0 + +twobyte_row_3x: + _ wrmsr + _ rdtsc + _ rdmsr + _ rdpmc + +twobyte_row_4x: + _ cmovo (%rax), %eax + _ cmovg (%rax), %eax + +/* twobyte_row_8x: --> Jcc disp32 */ + +twobyte_row_9x: + _ seto (%rax) + _ setg (%rax) + +twobyte_row_ax: + _ push %fs + _ pop %fs + _ cpuid + _ bt %eax, (%rax) + _ shld $0, %ax, (%rax) + _ shld $0, %eax, (%rax) + _ shld $0, %rax, (%rax) + _ shld %cl, %ax, (%rax) + _ shld %cl, %eax, (%rax) + _ shld %cl, %rax, (%rax) + _ push %gs + _ pop %gs + /*rsm --> not supported */ + _ bts %eax, (%rax) + _ shrd $0, %ax, (%rax) + _ shrd $0, %eax, (%rax) + _ shrd $0, %rax, (%rax) + _ shrd %cl, %ax, (%rax) + _ shrd %cl, %eax, (%rax) + _ shrd %cl, %rax, (%rax) + _ fxsave (%rax) /* Grp15 */ + _ imul (%rax), %eax + +twobyte_row_bx: + _ cmpxchg %al, (%rax) + _ cmpxchg %eax, (%rax) + _ lss (%rax), %eax + _ btr %eax, (%rax) + _ lfs (%rax), %eax + _ lgs (%rax), %eax + _ movzbl (%rax), %eax + _ movzwl (%rax), %eax + _ popcnt (%rax), %eax + _ ud1 (%rax), %eax /* Grp10 */ + _ bt $0, %ax /* Grp8 */ + _ bt $0, %eax + _ bt $0, %rax + _ btc %eax, (%rax) + _ bsf (%rax), %eax + _ bsr (%rax), %eax + _ movsbl (%rax), %eax + _ movswl (%rax), %eax + +twobyte_row_cx: + _ xadd %al, (%rax) + _ xadd %eax, (%rax) + _ cmpxchg8b (%rax) /* Grp9 */ + _ bswap %eax + _ bswap %edi + +END(tests_rel0) + +DECL(tests_rel1) +disp8: +1: + _ jo 1b + _ jno 1b + _ jb 1b + _ jae 1b + _ je 1b + _ jne 1b + _ jbe 1b + _ ja 1b + _ js 1b + _ jns 1b + _ jp 1b + _ jnp 1b + _ jl 1b + _ jge 1b + _ jle 1b + _ jg 1b + _ jmp 1b + +disp8_rex: + _ rex.w jo 1b + _ rex.w jno 1b + _ rex.w jb 1b + _ rex.w jae 1b + _ rex.w je 1b + _ rex.w jne 1b + _ rex.w jbe 1b + _ rex.w ja 1b + _ rex.w js 1b + _ rex.w jns 1b + _ rex.w jp 1b + _ rex.w jnp 1b + _ rex.w jl 1b + _ rex.w jge 1b + _ rex.w jle 1b + _ rex.w jg 1b + _ rex.w jmp 1b +END(tests_rel1) + +DECL(tests_rel4) +disp32: + _ call other_section + _ jmp other_section + _ jo other_section + _ jno other_section + _ jb other_section + _ jae other_section + _ je other_section + _ jne other_section + _ jbe other_section + _ ja other_section + _ js other_section + _ jns other_section + _ jp other_section + _ jnp other_section + _ jl other_section + _ jge other_section + _ jle other_section + _ jg other_section + _ xbegin other_section + +disp32_rex: + _ rex.w call other_section + _ rex.w jmp other_section + _ rex.w jo other_section + _ rex.w jno other_section + _ rex.w jb other_section + _ rex.w jae other_section + _ rex.w je other_section + _ rex.w jne other_section + _ rex.w jbe other_section + _ rex.w ja other_section + _ rex.w js other_section + _ rex.w jns other_section + _ rex.w jp other_section + _ rex.w jnp other_section + _ rex.w jl other_section + _ rex.w jge other_section + _ rex.w jle other_section + _ rex.w jg other_section + _ rex.w xbegin other_section + +riprel: + _ add %al, 0(%rip) + _ rex.b add %al, 0(%rip) + + _ addb $1, 0(%rip) + _ rex.b addb $1, 0(%rip) + + _ addl $0x7f000001, 0(%rip) + _ rex.b addl $0x7f000001, 0(%rip) +END(tests_rel4) + +DECL(tests_unsup) + +unsup_prefix: /* Prefixes unimplemented for simplicity. */ + _ vaddpd %zmm0, %zmm0, %zmm0 /* 0x62 EVEX */ + _ addr32 nop /* 0x67 Address size override */ + _ bextr $0, %eax, %eax /* 0x8f XOP */ + _ bextr %eax, %eax, %eax /* 0xc4 VEX3 */ + _ vaddpd %ymm0, %ymm0, %ymm0 /* 0xc5 VEX2 */ + n "jmpabs 0" .byte 0xd5, 0x00, 0xa1, 0x01, 0, 0, 0, 0, 0, 0, 0x80 /* 0xd5 REX2 */ + _ fadds (%rax) /* 0xd8 ... 0xdf ESCAPE (x87) */ + _ femms /* 0x0f,0x0e ... 0x0f 3DNOW */ + +unsup_branch: +1: + _ loopne 1b + _ loope 1b + _ loop 1b + _ jrcxz 1b + +opsize_branch: /* 66-prefixed branches are decoded differently by vendors */ + _ data16 call other_section + _ data16 jmp other_section + _ data16 jo other_section + _ data16 jno other_section + _ data16 jb other_section + _ data16 jae other_section + _ data16 je other_section + _ data16 jne other_section + _ data16 jbe other_section + _ data16 ja other_section + _ data16 js other_section + _ data16 jns other_section + _ data16 jp other_section + _ data16 jnp other_section + _ data16 jl other_section + _ data16 jge other_section + _ data16 jle other_section + _ data16 jg other_section + _ data16 xbegin other_section + +not_64bit: /* Not valid/encodable in 64bit mode */ + .code32 + _ push %es + _ pop %es + _ push %cs + _ push %ss + _ pop %ss + _ push %ds + _ pop %ds + _ daa + _ das + _ aaa + _ aas + _ pusha + _ popa + _ bound %eax, (%eax) + /*arpl %ax, %ax --> movsxd in 64bit mode */ + /* Grp1 */ + _ lcall $-1, $-1 + _ les (%eax), %eax + _ lds (%eax), %eax + _ into + _ aam $0 + _ aad $0 /* Also REX2, also not supported */ + n "salc" .byte 0xd6 + _ ljmp $-1, $-1 + .code64 + +unsup_insn: /* Instructions that would complicated decode, or shouldn't be used */ + _ ret $0 + _ enter $0, $0 + _ leave + _ lretq $0 + _ iretq + _ xlat + _ syscall + _ sysretl + _ invd + _ sysenter + _ sysexitl + _ rsm + +END(tests_unsup) + + /* This is here to cause jmps to use their disp32 form. */ + .section .text.other_section, "ax", @progbits +other_section: + int3 + + /* Mark this file as not needing executable stacks. */ + .section .note.GNU-stack, "", @progbits diff --git a/tools/tests/x86-decode-lite/macro-magic.h b/tools/tests/x86-decode-lite/macro-magic.h new file mode 100644 index 000000000000..b6b59906c941 --- /dev/null +++ b/tools/tests/x86-decode-lite/macro-magic.h @@ -0,0 +1,52 @@ +#ifndef X86_DECODE_LITE_LINKAGE_H +#define X86_DECODE_LITE_LINKAGE_H + +/* Start a 'struct test' array */ +.macro start_arr aname + .pushsection .data.rel.ro.\aname, "aw", @progbits + .globl \aname + .type \aname, STT_OBJECT +\aname: + .popsection + + /* Declare a macro wrapping \aname */ + .macro pushsection_arr + .pushsection .data.rel.ro.\aname, "aw", @progbits + .endm +.endm + +/* Macro 'n' to wrap the metadata of an instruction. Name can be different. */ +.macro n name:req insn:vararg + /* Emit the instruction, with start & end markers. */ +.Ls\@: \insn +.Le\@: + + /* Emit \name as a string. */ + .pushsection .rosdata.str1, "aMS", @progbits, 1 +.Ln\@: .asciz "\name" + .popsection + + /* Emit an entry into the array. */ + pushsection_arr + .quad .Ln\@, .Ls\@, .Le\@ - .Ls\@ + .popsection +.endm + +/* Macro '_' where the name is the instruction itself. */ +.macro _ insn:vararg + n "\insn" \insn +.endm + +/* Finish a 'struct test' array */ +.macro finish_arr aname + pushsection_arr + .quad 0, 0, 0 + .size \aname, . - \aname + .popsection + .purgem pushsection_arr +.endm + +#define DECL(aname) start_arr aname +#define END(aname) finish_arr aname + +#endif /* X86_DECODE_LITE_LINKAGE_H */ diff --git a/tools/tests/x86-decode-lite/main.c b/tools/tests/x86-decode-lite/main.c new file mode 100644 index 000000000000..cdae7de8e90e --- /dev/null +++ b/tools/tests/x86-decode-lite/main.c @@ -0,0 +1,111 @@ +/* + * Userspace test harness for x86_decode_lite(). + */ +#include + +#include "x86-emulate.h" + +static unsigned int nr_failures; +#define fail(t, fmt, ...) \ +({ \ + const unsigned char *insn = (t)->ip; \ + \ + nr_failures++; \ + \ + (void)printf(" Fail '%s' [%02x", (t)->name, *insn); \ + for ( unsigned int i = 1; i < (t)->len; i++ ) \ + printf(" %02x", insn[i]); \ + printf("]\n"); \ + \ + (void)printf(fmt, ##__VA_ARGS__); \ +}) + +struct test { + const char *name; + void *ip; + unsigned long len; +}; + +extern const struct test +/* Defined in insns.S, ends with sentinel */ + tests_rel0[], /* No relocatable entry */ + tests_rel1[], /* disp8 */ + tests_rel4[], /* disp32 or RIP-relative */ + tests_unsup[]; /* Unsupported instructions */ + +static inline void run_tests(const struct test *tests, unsigned int rel_sz) +{ + printf("Test rel%u\n", rel_sz); + + for ( unsigned int i = 0; tests[i].name; ++i ) + { + const struct test *t = &tests[i]; + x86_decode_lite_t r; + + /* + * Don't end strictly at t->len. This provides better diagnostics if + * too many bytes end up getting consumed. + */ + r = x86_decode_lite(t->ip, t->ip + /* t->len */ 20); + + if ( r.len == 0 ) + { + fail(t, " Failed to decode instruction\n"); + + if ( r.rel_sz != 0 || r.rel ) + fail(t, " Rel/sz despite no decode\n"); + + continue; + } + + if ( r.len != t->len ) + { + fail(t, " Expected length %lu, got %u\n", + t->len, r.len); + continue; + } + + if ( r.rel_sz != rel_sz ) + { + fail(t, " Expected relocation size %u, got %u\n", + rel_sz, r.rel_sz); + continue; + } + + if ( r.rel_sz && + (r.rel < t->ip || + r.rel > t->ip + t->len || + r.rel + r.rel_sz > t->ip + t->len) ) + { + fail(t, " Rel [%p,+%u) outside insn [%p,+%lu)\n", + r.rel, r.rel_sz, t->ip, t->len); + continue; + } + } +} + +static void run_tests_unsup(const struct test *tests) +{ + printf("Test unsup\n"); + + for ( unsigned int i = 0; tests[i].name; ++i ) + { + const struct test *t = &tests[i]; + x86_decode_lite_t r = x86_decode_lite(t->ip, t->ip + t->len); + + if ( r.len ) + fail(t, " Got len %u\n", r.len); + } +} + +int main(int argc, char **argv) +{ + printf("Tests for x86_decode_lite()\n"); + + run_tests(tests_rel0, 0); + run_tests(tests_rel1, 1); + run_tests(tests_rel4, 4); + run_tests_unsup(tests_unsup); + + return !!nr_failures; +} diff --git a/tools/tests/x86-decode-lite/x86-emulate.h b/tools/tests/x86-decode-lite/x86-emulate.h new file mode 100644 index 000000000000..558dab1b768e --- /dev/null +++ b/tools/tests/x86-decode-lite/x86-emulate.h @@ -0,0 +1,27 @@ +#ifndef X86_EMULATE_H +#define X86_EMULATE_H + +#include +#include +#include +#include +#include + +#include +#include + +#include + +#define ASSERT assert + +#define printk(...) + +#define likely +#define unlikely +#define cf_check +#define init_or_livepatch +#define init_or_livepatch_const + +#include "x86_emulate/x86_emulate.h" + +#endif /* X86_EMULATE_H */ From patchwork Wed Oct 2 15:27:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DC36CF6D38 for ; Wed, 2 Oct 2024 15:27:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808902.1220998 (Exim 4.92) (envelope-from ) id 1sw1GP-0008RA-K4; Wed, 02 Oct 2024 15:27:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808902.1220998; Wed, 02 Oct 2024 15:27:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GP-0008Pb-Dr; Wed, 02 Oct 2024 15:27:41 +0000 Received: by outflank-mailman (input) for mailman id 808902; Wed, 02 Oct 2024 15:27:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GN-0007h7-Vx for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:39 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id db2f3685-80d2-11ef-99a2-01e77a169b0f; Wed, 02 Oct 2024 17:27:38 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a7a843bef98so986424266b.2 for ; Wed, 02 Oct 2024 08:27:38 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:36 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: db2f3685-80d2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882857; x=1728487657; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/OYNqhpKRgZXD8+6+AhbIpT8QNeVjNGqQsR7u8ru59I=; b=bgZ6vCv6CL3Yk4zQrlPy9O+4COEFiqNa2NUzpA3sziLKHuovyQmdlHA/8ge3XwFeQw SOs2o4DqyIXezOuCPbnD17FpHtV3J1iZzuxunOsCQNJLPQBbQ1/yDWy+P4f8L0eKCnNx 4wLCaaiSVI8uebUwmjj0nOCtrxAUJx+jNosuk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882857; x=1728487657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/OYNqhpKRgZXD8+6+AhbIpT8QNeVjNGqQsR7u8ru59I=; b=lyqkjF9C4IY0B5hxgilOpGRA3nCV5MkZWzvbOldh7PTWzLVe0BFvjtW/LijU78NKnH ugg0uJP/49KHp77yY75+h68x8n0aRSsAV23rodPSK6QMAd+NRWE+3o/IH6fP8YqmxFxP R0Xz6GKqALvfq8XHTcaX7YsmGtu/ZsiKVzjZ5pNMU+uoSpZxhGvaR8UAcix8YPbrrwN9 kjyhVyaGABxBjcuL5lvR62w3RAtBqxJGtqe33k0/vvf0/qRNTJWjv5Fw8f7Aih3W7C+H IfJ4Ry34huY15OYVP3jhKfDNN6DwvuQ9MhERN/d+LxSWl+RUd4XoJaCrkyWGSJNrV6tF peMw== X-Gm-Message-State: AOJu0YzR3Ujw5VrCSr8lX1RAS3R8Sy7OublKndq3DFW7k+IdsmH0xhkH zoIQkOD4uytE5E092WTHxA3FJhk+K93KEG/NzXBduznnZRkoNjytybv3Ni9fdbLgxgSunV6cvmp 0dDc= X-Google-Smtp-Source: AGHT+IGyps52t9eHsJjMQR/F6NQEncBxc2igwnQUXEn+2dI/dBTnAiVOxq0yisZwo57aulhs1Sa6DQ== X-Received: by 2002:a17:907:703:b0:a8b:6ee7:ba17 with SMTP id a640c23a62f3a-a98f83ff431mr349107466b.55.1727882856976; Wed, 02 Oct 2024 08:27:36 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 3/7] x86/alternative: Walk all replacements during self tests Date: Wed, 2 Oct 2024 16:27:21 +0100 Message-Id: <20241002152725.1841575-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 When self tests are active, walk all alternative replacements with x86_decode_lite(). This checks that we can decode all instructions, and also lets us check that disp8's don't leave the replacement block as such a case will definitely malfunction. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné v2: * Rebase over API changes in patch 1 * Use +%lu and drop casts * Swap to CONFIG_SELF_TESTS --- xen/arch/x86/alternative.c | 52 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 1ba35cb9ede9..2dc759abd858 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #define MAX_PATCH_LEN (255-1) @@ -490,6 +491,57 @@ static void __init _alternative_instructions(bool force) void __init alternative_instructions(void) { arch_init_ideal_nops(); + + /* + * Walk all replacement instructions with x86_decode_lite(). This checks + * both that we can decode all instructions within the replacement, and + * that any near branch with a disp8 stays within the alternative itself. + */ + if ( IS_ENABLED(CONFIG_SELF_TESTS) ) + { + struct alt_instr *a; + + for ( a = __alt_instructions; + a < __alt_instructions_end; ++a ) + { + void *repl = ALT_REPL_PTR(a); + void *ip = repl, *end = ip + a->repl_len; + + if ( !a->repl_len ) + continue; + + for ( x86_decode_lite_t res; ip < end; ip += res.len ) + { + const int8_t *d8; + const void *target; + + res = x86_decode_lite(ip, end); + + if ( res.len == 0 ) + { + printk("Alt for %ps [%*ph]\n", + ALT_ORIG_PTR(a), a->repl_len, repl); + panic(" Unable to decode instruction at +%lu in alternative\n", + ip - repl); + } + + if ( res.rel_sz != 1 ) + continue; + + d8 = res.rel; + target = ip + res.len + *d8; + + if ( target < repl || target > end ) + { + printk("Alt for %ps [%*ph]\n", + ALT_ORIG_PTR(a), a->repl_len, repl); + panic(" 'JMP/Jcc disp8' at +%lu leaves alternative block\n", + ip - repl); + } + } + } + } + _alternative_instructions(false); } From patchwork Wed Oct 2 15:27:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03487CF6D3C for ; Wed, 2 Oct 2024 15:27:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808903.1221011 (Exim 4.92) (envelope-from ) id 1sw1GQ-0000NZ-Om; Wed, 02 Oct 2024 15:27:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808903.1221011; Wed, 02 Oct 2024 15:27:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GQ-0000Mo-L1; Wed, 02 Oct 2024 15:27:42 +0000 Received: by outflank-mailman (input) for mailman id 808903; Wed, 02 Oct 2024 15:27:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GP-0007h7-1W for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:41 +0000 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [2a00:1450:4864:20::632]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id dc034904-80d2-11ef-99a2-01e77a169b0f; Wed, 02 Oct 2024 17:27:39 +0200 (CEST) Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-a7aa086b077so972951566b.0 for ; Wed, 02 Oct 2024 08:27:39 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dc034904-80d2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882858; x=1728487658; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Kg1YWqxiODnPwg0lFMS5tFuPYMahV9xXqAF9qlsMRt8=; b=Q+JcH8yGgr2yd5wTyIbo+x6Uqnt8CWHYOQBl5/BmO9/oQxBS8Py9rMn0jRA5OqI96L eUhF25obs0iEM8tC63pDo/u2riF1Xs1QOk2zhhVdkvrNRTiLg2YtfVWgnfNgNVk9RIJr oYRhOcHXgDrdCHLUvJ1HrPf5/4AUpYn8PQMn8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882858; x=1728487658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kg1YWqxiODnPwg0lFMS5tFuPYMahV9xXqAF9qlsMRt8=; b=cBg9aVY8pYeI/YhNOy1zn9+xhSgiMnEAyNGsyklMuxH+4eliCXhKvo3xOhade1aU8/ aTmWyQ3coqF+JNNjcqM5QOSBOtl405JwwBxR2qFj2mjWDeN6vGGa8D9G92xwi0JWkg9I 1MnoK2eUWs3aDnc6c/bsjNeqq5UCV9uzB+IpsutaPpMGZnMtYRsstNiY2TFj13zpAPDE nKEe0rF+j/4PhvnsSJyOyY0LekVxA/EpqnDrNxEGGk6XNZDFVzrGK/avIQEDR930HNVG 8DlOsS7GDaVWB8vaoVmr5ZhOUHpDm1p1fNLNjXWd47KXD7/wt0XVFveYhWWh0xK7I3Ig ZNaQ== X-Gm-Message-State: AOJu0YyKxG8EHwoT+x+4R1RY2wg/MVlnvGK99/VAhKMXibeppcJRD6Rm dRtoaAfJoc0Fk1MuO2pYXeyfDHSJOMLTHAAR7BO+Gy3P+rSlg2UlDm5YRZOD8bF6A9BnLKsf5tG lWgI= X-Google-Smtp-Source: AGHT+IHTV3eeo5X12pIQK8EIKOA8qaSxG31trYp1WwulYTh2n8MBG0DiWK277bEIyrZ7R7rCYZLcFg== X-Received: by 2002:a17:907:8690:b0:a8b:6ee7:ba10 with SMTP id a640c23a62f3a-a98f8270815mr332802566b.33.1727882858549; Wed, 02 Oct 2024 08:27:38 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 4/7] x86/alternative: Indent the relocation logic Date: Wed, 2 Oct 2024 16:27:22 +0100 Message-Id: <20241002152725.1841575-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... to make subsequent patches legible. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné --- xen/arch/x86/alternative.c | 124 +++++++++++++++++++------------------ 1 file changed, 63 insertions(+), 61 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 2dc759abd858..5ea7f51b007c 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -264,78 +264,80 @@ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, memcpy(buf, repl, a->repl_len); - /* 0xe8/0xe9 are relative branches; fix the offset. */ - if ( a->repl_len >= 5 && (*buf & 0xfe) == 0xe8 ) { - /* - * Detect the special case of indirect-to-direct branch patching: - * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; already - * checked above), - * - replacement's displacement is -5 (pointing back at the very - * insn, which makes no sense in a real replacement insn), - * - original is an indirect CALL/JMP (opcodes 0xFF/2 or 0xFF/4) - * using RIP-relative addressing. - * Some branch destinations may still be NULL when we come here - * the first time. Defer patching of those until the post-presmp- - * initcalls re-invocation (with force set to true). If at that - * point the branch destination is still NULL, insert "UD2; UD0" - * (for ease of recognition) instead of CALL/JMP. - */ - if ( a->cpuid == X86_FEATURE_ALWAYS && - *(int32_t *)(buf + 1) == -5 && - a->orig_len >= 6 && - orig[0] == 0xff && - orig[1] == (*buf & 1 ? 0x25 : 0x15) ) + /* 0xe8/0xe9 are relative branches; fix the offset. */ + if ( a->repl_len >= 5 && (*buf & 0xfe) == 0xe8 ) { - long disp = *(int32_t *)(orig + 2); - const uint8_t *dest = *(void **)(orig + 6 + disp); - - if ( dest ) + /* + * Detect the special case of indirect-to-direct branch patching: + * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; already + * checked above), + * - replacement's displacement is -5 (pointing back at the very + * insn, which makes no sense in a real replacement insn), + * - original is an indirect CALL/JMP (opcodes 0xFF/2 or 0xFF/4) + * using RIP-relative addressing. + * Some branch destinations may still be NULL when we come here + * the first time. Defer patching of those until the post-presmp- + * initcalls re-invocation (with force set to true). If at that + * point the branch destination is still NULL, insert "UD2; UD0" + * (for ease of recognition) instead of CALL/JMP. + */ + if ( a->cpuid == X86_FEATURE_ALWAYS && + *(int32_t *)(buf + 1) == -5 && + a->orig_len >= 6 && + orig[0] == 0xff && + orig[1] == (*buf & 1 ? 0x25 : 0x15) ) { - /* - * When building for CET-IBT, all function pointer targets - * should have an endbr64 instruction. - * - * If this is not the case, leave a warning because - * something is probably wrong with the build. A CET-IBT - * enabled system might have exploded already. - * - * Otherwise, skip the endbr64 instruction. This is a - * marginal perf improvement which saves on instruction - * decode bandwidth. - */ - if ( IS_ENABLED(CONFIG_XEN_IBT) ) + long disp = *(int32_t *)(orig + 2); + const uint8_t *dest = *(void **)(orig + 6 + disp); + + if ( dest ) { - if ( is_endbr64(dest) ) - dest += ENDBR64_LEN; - else - printk(XENLOG_WARNING - "altcall %ps dest %ps has no endbr64\n", - orig, dest); + /* + * When building for CET-IBT, all function pointer targets + * should have an endbr64 instruction. + * + * If this is not the case, leave a warning because + * something is probably wrong with the build. A CET-IBT + * enabled system might have exploded already. + * + * Otherwise, skip the endbr64 instruction. This is a + * marginal perf improvement which saves on instruction + * decode bandwidth. + */ + if ( IS_ENABLED(CONFIG_XEN_IBT) ) + { + if ( is_endbr64(dest) ) + dest += ENDBR64_LEN; + else + printk(XENLOG_WARNING + "altcall %ps dest %ps has no endbr64\n", + orig, dest); + } + + disp = dest - (orig + 5); + ASSERT(disp == (int32_t)disp); + *(int32_t *)(buf + 1) = disp; } - - disp = dest - (orig + 5); - ASSERT(disp == (int32_t)disp); - *(int32_t *)(buf + 1) = disp; - } - else if ( force ) - { - buf[0] = 0x0f; - buf[1] = 0x0b; - buf[2] = 0x0f; - buf[3] = 0xff; - buf[4] = 0xff; + else if ( force ) + { + buf[0] = 0x0f; + buf[1] = 0x0b; + buf[2] = 0x0f; + buf[3] = 0xff; + buf[4] = 0xff; + } + else + continue; } + else if ( force && system_state < SYS_STATE_active ) + ASSERT_UNREACHABLE(); else - continue; + *(int32_t *)(buf + 1) += repl - orig; } else if ( force && system_state < SYS_STATE_active ) ASSERT_UNREACHABLE(); - else - *(int32_t *)(buf + 1) += repl - orig; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); a->priv = 1; From patchwork Wed Oct 2 15:27:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4848CCF6D39 for ; Wed, 2 Oct 2024 15:27:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808904.1221023 (Exim 4.92) (envelope-from ) id 1sw1GS-0000h4-4n; Wed, 02 Oct 2024 15:27:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808904.1221023; Wed, 02 Oct 2024 15:27:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GR-0000g7-V4; Wed, 02 Oct 2024 15:27:43 +0000 Received: by outflank-mailman (input) for mailman id 808904; Wed, 02 Oct 2024 15:27:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GQ-00070n-E7 for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:42 +0000 Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [2a00:1450:4864:20::12d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id dd84ee6f-80d2-11ef-a0ba-8be0dac302b0; Wed, 02 Oct 2024 17:27:42 +0200 (CEST) Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-5389917ef34so8060512e87.2 for ; Wed, 02 Oct 2024 08:27:42 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:38 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dd84ee6f-80d2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882861; x=1728487661; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5lEtOt7sJb/gqHGmXN1HeKQkSgPlz46zJUquNaFSqHg=; b=IbFFFTol0BkenkkcaEl9GD18KalA1qN950jkkIiCGZBZLpQ1CgdChHaWMFoZ98poQa Msg/0RAktQZsO+mqeNsa6rbEAqKYE1dRSC36c+ySkBCSVc31UiHPp3aV5fiWH4VbVKt/ CmmZGLHLJqkzarLYyH+TXX22F2LRE6T6idtSM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882861; x=1728487661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5lEtOt7sJb/gqHGmXN1HeKQkSgPlz46zJUquNaFSqHg=; b=H32bKwzqbi61V65LLz099JrvCCu+s2SIoJNNmFYfihgmDn3mKcn4CRp/QroRqFlueH /DoaTLixsBBLYYGR157uZ0OO2Aj2VDTLDK1ChEISli7RrPReraVX51OZaqWjZ+QYL6oC bO7dFbz2mXGJ3L37Pf01gw/+s5V0vwKIhQaLecRO/aXuAT2KJpUu/X0/6+gV+31KZMTq Msalk85ynxWKmDSnUsbSIzATv3KE6GOAdkFxxIez8X3H9Y7vOGpRzD9yTHEZx0K5Qmof pMhsmMaoi1GsVvJUk0qxUNUSqnVsbXhkjMKD/OtyaUUMM0VgGAgQ8f4gNbP+VW9IyZhl 8a0w== X-Gm-Message-State: AOJu0Yxpn9LAlznVmOAjrmQdYINIwPYX+ABVfgQohZw8zyyHHlqaXrxY 8agYCsc+OeTehIHBcY6kzBqBUDv2QF17IJ6lu6Ow/PPFnWRZWRLe3gn+TU8XwDnpPSnRxyabwJ9 C4qg= X-Google-Smtp-Source: AGHT+IFMz20lzYRyDTFBCZuZGX8TbKWpGePrGdiPdDMdq/Mk7wnanI7JzKks4Wc91CB+zV8GtQaQpg== X-Received: by 2002:a05:6512:2245:b0:535:6cde:5c4d with SMTP id 2adb3069b0e04-539a065c3e0mr2158383e87.3.1727882860654; Wed, 02 Oct 2024 08:27:40 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 5/7] x86/alternative: Replace a continue with a goto Date: Wed, 2 Oct 2024 16:27:23 +0100 Message-Id: <20241002152725.1841575-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 A subsequent patch is going to insert a loop, which interferes with the continue in the devirtualisation logic. Replace it with a goto, and a paragraph explaining why we intentionally avoid setting a->priv = 1. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné --- xen/arch/x86/alternative.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index 5ea7f51b007c..b745f112154a 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -328,7 +328,15 @@ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, buf[4] = 0xff; } else - continue; + { + /* + * The function pointer we're wanting to devirtualise + * is still NULL, and we're not sealing yet. Leave + * the alternative fully un-processed, in order to try + * again next time around. + */ + goto skip_this_alternative; + } } else if ( force && system_state < SYS_STATE_active ) ASSERT_UNREACHABLE(); @@ -343,6 +351,7 @@ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, add_nops(buf + a->repl_len, total_len - a->repl_len); text_poke(orig, buf, total_len); + skip_this_alternative:; } /* From patchwork Wed Oct 2 15:27:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1FB6CF6D3E for ; Wed, 2 Oct 2024 15:27:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808905.1221032 (Exim 4.92) (envelope-from ) id 1sw1GU-00014w-D8; Wed, 02 Oct 2024 15:27:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808905.1221032; Wed, 02 Oct 2024 15:27:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GU-00014a-79; Wed, 02 Oct 2024 15:27:46 +0000 Received: by outflank-mailman (input) for mailman id 808905; Wed, 02 Oct 2024 15:27:44 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GS-0007h7-2V for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:44 +0000 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [2a00:1450:4864:20::635]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ddca10f1-80d2-11ef-99a2-01e77a169b0f; Wed, 02 Oct 2024 17:27:42 +0200 (CEST) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a8d24f98215so232419666b.1 for ; Wed, 02 Oct 2024 08:27:42 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:40 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ddca10f1-80d2-11ef-99a2-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882861; x=1728487661; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kKsfnfRQgiEGAkaEc12PpOtnWnQ5PN0UH+J+RGQf2wM=; b=p39tyRywlNP6KMhvZaoy8017ChgOGyzamCzTw7owvE7vfz1AU872Ibo4WW3NY3W5K8 lWDK3yBwt5EU21pgZHdWcjiftI7Z1I2GhfVr7DeOyZRnc6yhkLwr9R/a66J2Un1yHq0J o3r5elTtpLZP7E0EoZITdQQhH2p2MEMrAQ1r4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882861; x=1728487661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kKsfnfRQgiEGAkaEc12PpOtnWnQ5PN0UH+J+RGQf2wM=; b=m/ljHGeKeWTJaJ2ZX3l03glw6plOCn+2i1+pPdELW/fS5VJ4jAYYnVxxNOg7UDRcze PB7kvLtE5xcqK2eHoafnZVN6MDnv66rrmlGl99CiUrx/C+RRuOevdnhu/GIFattr1Nfh 9MntphfLc4s8KUxRCclTai5XoHiYHMipOV9RtjaIUQz7mxAjo2s6MLlLoSEaWCueXfUh FlDebufl6NDrAxScoSIyoozBKZ0bWCwTOx6qfRR0KmzBL7NONHJ6BkDwaDi18d1fjz23 gi2zSNf5FSAFrefLNTPpjHj797EnwPmNvsmJOQMiH+6jr73ihy0k7MLUN/W7fMqNkhQv df6Q== X-Gm-Message-State: AOJu0Yw0fRVAnL9U7UEHLYWnThbsLG1SKpKZEo3dYi0txXJLzBfSruRK 0CH60hazesz7YJz84O7mkQvjv35S+zb4RJOZ2tt8IJRAdQcDxftqt+KUXaWE8ZZzqxYAMn2xseh KS6E= X-Google-Smtp-Source: AGHT+IH8hwKmXlccN1whSvfTN4yPqkVDyedjNvhrBux41Exg2tJba+6aRWj33Y7UXrUf1gJuuf5KpA== X-Received: by 2002:a17:906:794c:b0:a90:b6e8:6919 with SMTP id a640c23a62f3a-a98f83615e1mr357768666b.48.1727882861402; Wed, 02 Oct 2024 08:27:41 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 6/7] x86/alternative: Relocate all insn-relative fields Date: Wed, 2 Oct 2024 16:27:24 +0100 Message-Id: <20241002152725.1841575-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Right now, relocation of displacements is restricted to finding 0xe8/e9 as the first byte of the replacement, but this is overly restrictive. Use x86_decode_lite() to find and adjust all insn-relative fields. As with disp8's not leaving the replacemnet block, some disp32's don't either. e.g. the RSB stuffing loop. These stay unmodified. For now, leave the altcall devirtualisation alone. These require more care to transform into the new scheme. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné --- xen/arch/x86/alternative.c | 46 +++++++++++++++++++++++++++++++------- 1 file changed, 38 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index b745f112154a..da1a4af56377 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -264,10 +264,31 @@ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, memcpy(buf, repl, a->repl_len); + /* Walk buf[] and adjust any insn-relative operands. */ + if ( a->repl_len ) { - /* 0xe8/0xe9 are relative branches; fix the offset. */ - if ( a->repl_len >= 5 && (*buf & 0xfe) == 0xe8 ) + uint8_t *ip = buf, *end = ip + a->repl_len; + + for ( x86_decode_lite_t res; ip < end; ip += res.len ) { + int32_t *d32; + const uint8_t *target; + + res = x86_decode_lite(ip, end); + + if ( res.len == 0 ) + { + printk("Alt for %ps [%*ph]\n" + " Unable to decode instruction at +%lu in alternative\n", + ALT_ORIG_PTR(a), a->repl_len, repl, ip - repl); + return -EINVAL; + } + + if ( res.rel_sz != 4 ) + continue; + + d32 = res.rel; + /* * Detect the special case of indirect-to-direct branch patching: * - replacement is a direct CALL/JMP (opcodes 0xE8/0xE9; already @@ -337,14 +358,23 @@ static int init_or_livepatch _apply_alternatives(struct alt_instr *start, */ goto skip_this_alternative; } + + continue; + } + + target = ip + res.len + *d32; + + if ( target >= buf && target <= end ) + { + /* + * Target doesn't leave the replacement block. e.g. RSB + * stuffing. Leave it unmodified. + */ + continue; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); - else - *(int32_t *)(buf + 1) += repl - orig; + + *d32 += repl - orig; } - else if ( force && system_state < SYS_STATE_active ) - ASSERT_UNREACHABLE(); } a->priv = 1; From patchwork Wed Oct 2 15:27:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13819976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5B99CF6D3D for ; Wed, 2 Oct 2024 15:27:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.808906.1221036 (Exim 4.92) (envelope-from ) id 1sw1GU-0001A8-Qt; Wed, 02 Oct 2024 15:27:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 808906.1221036; Wed, 02 Oct 2024 15:27:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GU-00018O-J8; Wed, 02 Oct 2024 15:27:46 +0000 Received: by outflank-mailman (input) for mailman id 808906; Wed, 02 Oct 2024 15:27:44 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sw1GS-00070n-G8 for xen-devel@lists.xenproject.org; Wed, 02 Oct 2024 15:27:44 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id deb2ef59-80d2-11ef-a0ba-8be0dac302b0; Wed, 02 Oct 2024 17:27:44 +0200 (CEST) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a8d6ac24a3bso187271366b.1 for ; Wed, 02 Oct 2024 08:27:44 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c299ac8esm880024566b.222.2024.10.02.08.27.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Oct 2024 08:27:41 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: deb2ef59-80d2-11ef-a0ba-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1727882863; x=1728487663; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bUmCjdSTNNz/bYZNosiPwxy4sAkcSDlDvTaqnGpBKLQ=; b=SwZhQ+6WsrrkThYLfUhF3ZhBf8FNy1hE+2faWX3m0EMLqLW9B7Hqr+oar23AjEF9iw r9XJisJBpLM56SvmCiI4blgEfi+7vubVrAemYaQfMA7LUdr3mdv1A4o2FKb+LjSL7o9s 1bwTmds51XIq55tobkeJwhbTUD0HINeLit00o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727882863; x=1728487663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bUmCjdSTNNz/bYZNosiPwxy4sAkcSDlDvTaqnGpBKLQ=; b=iyW51Zk88pbF4VzDjsD0cBk6XxPjSxBbJVgoWif/vGGZWWXI0nq/cdbiey7rd1QRZO HM4Ug/bRNe40NzfHtbIrLZRI/YDfTH7rBPkoiTnSo2eQTa/IiBtxqsqVXP6MYuIWqkvL PFSoUN0ov+reeGcCNvqhe2MuP2rkMNk7EZrlzPqw5Z0gB2M9+PvzyyJ1flFpdj/Qs1ig wyG/OzmdwdIbVPK2Xft/Cd1im9/zTZr9uhMPP9cQbx2XMfPr+hCNHqzCgcH5/+W4eu1I 2nietFMwv5u0P78hnIwUj2itKQ+MOQUjiSuyr6zRPvzd8wuUP4J48CmgbpUl511PBNAz lQrg== X-Gm-Message-State: AOJu0YwFk3y8smbOb1yUTGvVA0PDxw75o6rdtib9WAqubdTmag5h5lqJ +KaOs42EfGPHqmQsK9pZE1KyK3i3HL/23STlqgLj94oSP4WY+XhZYGbSfYKUr3tmJ5nifVydWgL aP/I= X-Google-Smtp-Source: AGHT+IGpegr0xxYsCZ+z9qYO4dCzrS7LRSj3afqU4XsleDv+WYP+JIFC1n1LDhSpV8GKeuZQvfoQzg== X-Received: by 2002:a17:906:c107:b0:a90:3494:2aa9 with SMTP id a640c23a62f3a-a990a03470emr1993866b.2.1727882863066; Wed, 02 Oct 2024 08:27:43 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 7/7] x86/spec-ctrl: Introduce and use DO_COND_BHB_SEQ Date: Wed, 2 Oct 2024 16:27:25 +0100 Message-Id: <20241002152725.1841575-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241002152725.1841575-1-andrew.cooper3@citrix.com> References: <20241002152725.1841575-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Now that alternatives can fix up call displacements even when they're not the first instruction of the replacement, move the SCF_entry_bhb conditional inside the replacement block. This removes a conditional branch from the fastpaths of BHI-unaffected hardware. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné --- xen/arch/x86/hvm/vmx/entry.S | 12 +++---- xen/arch/x86/include/asm/spec_ctrl_asm.h | 43 +++++++++++++----------- 2 files changed, 30 insertions(+), 25 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S index 6aaeb28a5b96..e901a41f252a 100644 --- a/xen/arch/x86/hvm/vmx/entry.S +++ b/xen/arch/x86/hvm/vmx/entry.S @@ -62,12 +62,12 @@ FUNC(vmx_asm_vmexit_handler) * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses RETs * itself so must be after we've perfomed all the RET-safety we can. */ - testb $SCF_entry_bhb, CPUINFO_scf(%rsp) - jz .L_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L_skip_bhb: + .macro VMX_BHB_SEQ fn:req + DO_COND_BHB_SEQ \fn scf=CPUINFO_scf(%rsp) + .endm + ALTERNATIVE_2 "", \ + "VMX_BHB_SEQ fn=clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "VMX_BHB_SEQ fn=clear_bhb_tsx", X86_SPEC_BHB_TSX ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_VMX /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */ diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 729a830411eb..559dad88f967 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -92,6 +92,21 @@ .L\@_skip: .endm +.macro DO_COND_BHB_SEQ fn:req, scf=%bl +/* + * Requires SCF (defaults to %rbx), fn=clear_bhb_{loops,tsx} + * Clobbers %rax, %rcx + * + * Conditionally use a BHB clearing software sequence. + */ + testb $SCF_entry_bhb, \scf + jz .L\@_skip_bhb + + call \fn + +.L\@_skip_bhb: +.endm + .macro DO_OVERWRITE_RSB tmp=rax, xu /* * Requires nothing @@ -277,12 +292,9 @@ * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses RETs * itself so must be after we've perfomed all the RET-safety we can. */ - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_PV .endm @@ -322,12 +334,9 @@ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), \ X86_FEATURE_SC_MSR_PV - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX ALTERNATIVE "lfence", "", X86_SPEC_NO_LFENCE_ENTRY_INTR .endm @@ -433,13 +442,9 @@ * Clear the BHB to mitigate BHI. Used on eIBRS parts, and uses RETs * itself so must be after we've perfomed all the RET-safety we can. */ - testb $SCF_entry_bhb, %bl - jz .L\@_skip_bhb - - ALTERNATIVE_2 "", \ - "call clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ - "call clear_bhb_tsx", X86_SPEC_BHB_TSX -.L\@_skip_bhb: + ALTERNATIVE_2 "", \ + "DO_COND_BHB_SEQ clear_bhb_loops", X86_SPEC_BHB_LOOPS, \ + "DO_COND_BHB_SEQ clear_bhb_tsx", X86_SPEC_BHB_TSX lfence .endm