From patchwork Tue Jan 19 10:24:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gary Lin X-Patchwork-Id: 12029855 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F1FC43381 for ; Tue, 19 Jan 2021 13:49:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D77A5206E4 for ; Tue, 19 Jan 2021 13:49:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405961AbhASNtE (ORCPT ); Tue, 19 Jan 2021 08:49:04 -0500 Received: from de-smtp-delivery-102.mimecast.com ([62.140.7.102]:48667 "EHLO de-smtp-delivery-102.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390580AbhASKa0 (ORCPT ); Tue, 19 Jan 2021 05:30:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1611052140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Et/09oALV20Ju9+eB7wAnivLNxilXefAZQNBnrd3cRE=; b=Vmq2DkKWW9QUd3KO+gfghjU8ob6+Q7/gOhRAfG3Eviun3+WesRCNGKUaN2pB9mQ4F0U6kD BgXHWEy5T4QJNoc6fV5fLfniZq2oAaz4E/r0xGhFDWwnFDk/9YEua8MsgJ068jqmXG0vX/ KqyYLBnU2oAOFEDu0URp5eMmooxzvjw= Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-14-1dnMtNziN8a0XMrZGANP5A-1; Tue, 19 Jan 2021 11:25:28 +0100 X-MC-Unique: 1dnMtNziN8a0XMrZGANP5A-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DB3ejDiKbRyb5Tl1bF16AL09LRVTzhCwZ579C37CnCY888Yfo7T3CcJM821IG9Yp/tOEjL4bkTgLGU+Y44B3V2GNDJcwPFiu3+cI3AFPPZget9Up0Aq8/y7vCjN7nxOIuexhW/8p6t38dioAEDO00hY9gzWVFcxjY59/SBMv60uGFTIYgHHK+i9wsYddF+O0vWfiLPrQkptt1gbJF+y6nlYfSu1Y0W4+3YfCe+kdky/nEia/znw6CLsRZVCDwr1IZF+tOEsWgmYbEXh5rBfvnkUfiqZpzDXcv3lFXM/1bB3DvDN94iFMRQ1b8mGDSsSmwMWZg0svHswLqHQMl3VVrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hRAWZJ+gwEt/3tPZe71MZ696YnggxgqO6cFB3mjroUg=; b=ITcOxNzNTtV1aHaNzhWpoOjHYJ7hM6jqU/cOoh4cbiKW9Tu7JhXjtLNCKbv0zTsIgpewANDo97zlm9lcm8I+/IxuMFSIlZ8Qoj6gU3G72FeNnbCSFm6WHssQknafVny7ET7Wi6wmkanfx5CjYykNn/UVKKyLOrHx25db8c5hVaUQl7534CiQr4UjlolGVkobuaOJVLj5hWhQcGatHnODB+b1nqH7spEzOv4tgVekGgIvbX8mkVbUCfx3gLpkk/XKweHIZeMvk5zqf2V5v0LbVdI0JL/EPuUJZYwkTMnjupdAaUfaPng94iKttB9Qa4V0y8IhHGuA1UucLeJyplSdVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com; Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) by DB6PR04MB3109.eurprd04.prod.outlook.com (2603:10a6:6:f::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.14; Tue, 19 Jan 2021 10:25:27 +0000 Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313]) by DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313%6]) with mapi id 15.20.3763.014; Tue, 19 Jan 2021 10:25:27 +0000 From: Gary Lin To: netdev@vger.kernel.org, bpf@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann CC: Eric Dumazet , Andrii Nakryiko , andreas.taschner@suse.com Subject: [PATCH v4 1/3] bpf,x64: pad NOPs to make images converge more easily Date: Tue, 19 Jan 2021 18:24:59 +0800 Message-ID: <20210119102501.511-2-glin@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210119102501.511-1-glin@suse.com> References: <20210119102501.511-1-glin@suse.com> X-Originating-IP: [111.240.113.126] X-ClientProxiedBy: FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) To DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from GaryLaptop.prv.suse.net (111.240.113.126) by FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.6 via Frontend Transport; Tue, 19 Jan 2021 10:25:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 97faf096-f4bc-44ca-6ba1-08d8bc648a01 X-MS-TrafficTypeDiagnostic: DB6PR04MB3109: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cThOngYad6TmqawTMXI8jENnVIIKy3WKF/OTAoeD0cLA3z8OgikBZ0RV2VdN6QJtUZrkjNHqdBDgTXyO+6QJaPIZ8NdC5d0VcCp9LkANKsKyNd04z56KWHJ/dN0A9V8m1DEO9IqJgUdCbQwcKJZolC1qx+7KZpF/X76Y9H5+s4CEsZmx20rccrqRFPMxRfUkbHY33mKBpSrLumZVFrQomO4U42G0SIHqH3/J7gkvBZX6srWXOI80gbvBhS5DjyjyyTOqsHnrHH4OFoOth3BjB34GuqacfmPCqzNF5Y7s3+H6y9aZTtNDn8SFmJropjuNfl0y71WHcwep8+a3WseAMfTBqf1uWo3uP5pRNcyhJGTY+qlakBDqVykvsoA9BleN1UtIJKT77Vv/lQsfK6XccQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB3PR0402MB3641.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(376002)(366004)(346002)(83380400001)(1076003)(186003)(26005)(6486002)(52116002)(316002)(956004)(16526019)(6512007)(6506007)(4326008)(107886003)(8676002)(5660300002)(36756003)(66556008)(2906002)(8936002)(66946007)(66476007)(86362001)(2616005)(478600001)(110136005)(6666004)(54906003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: sAi/WGraxw9yFAiNDj1TywOzQ46KucwjOM4dkXOoKi1bdH2fT/AgPwE7iP5zuIIwQ1IWcvMJQ+0EG4IKAcX2dHaUpqrKynb7NmF8YuMJqWJ1K8uiyzp7Pc9JymkoGa5BDKGolGxzh+DkLFKnQwvyVE9Dk81SU1OfS/pE1xZkk5pSKQnISL8cX1KaL87C5X0X2p+dVCkwXGXAK9FDu0GaYk7aIINp1yErcebd8X6uTjuJZYt/MbDa2IT9YR6MNEdlF8tQQzUP7+d13XI7AiZBc2RM89RCAXFapar4oBVKLTIOOoSfxEVz2/zIS74TXbXHas812ITjQYZisu+mF64Q15l3UM5SB9+UguJyobLDJW1ywCRxjKCbEvw2sxNKZa8bYqvptD9mBBSaEGOdbkS2G5KHnkoL2NVYDJnfHAEmxIfVOV9UDYiDCKjBfdf3VRE7mWG15PWW5Sn3J68ywvWetsxxl7+2DpNDeAUNeHRQOO2CgZicnwt3J8bUN39r3EWqA8qDViVIMlgO0dPHZ0HEDn29P8z60JauKDRrO4nHqsJ20lssiOMQidxB7vU2mUkaCMTVi30APX7jSLDlv5NwVdX4ANp0E06SFIAu0SZTQUqc1Mqki+Zchx3mof7q5VbRFBoKgLcFKmXt7UQ/lnbJrlOM5xVhjI35YrCDtQfbWyWiozbQoT61V/8CmYwB7Qkxe+G9eq2qKhtBVG0sel2T4DoGyER5shsMovRDk8ExZFKhojjpqPBJqEtwrgTUaNKKpwdz/WIs/u19Oy6Zzq/ZScn30emYJy0imcT/ZErc3bCoF7LP5AJ1mFOv0KosrU+fCS01DKzczHn69O7g2btI032YpWV4iKSA16J/+6lrWS+iBHuea8d8Jj9XDVXNjTvkuklZlNH/108UJBSMsPlISAxMGx1D4b7r0b6+D+wzgmvCaVl0BgxCXw0/GdNY7q7uyTXcBBkCVRg7I9ZNl97q54X41W2oSCsabTxe/7n/oM+gK/mzBzT/OhEHTJA2fAzL X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97faf096-f4bc-44ca-6ba1-08d8bc648a01 X-MS-Exchange-CrossTenant-AuthSource: DB3PR0402MB3641.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2021 10:25:27.5922 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H0FIcvVpp/pI3qTQ8SmSrT4skidtsCjn8sr9mncm/x3mLEBowbE95503CQBffpUQ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR04MB3109 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The x64 bpf jit expects bpf images converge within the given passes, but it could fail to do so with some corner cases. For example: l0: ja 40 l1: ja 40 [... repeated ja 40 ] l39: ja 40 l40: ret #0 This bpf program contains 40 "ja 40" instructions which are effectively NOPs and designed to be replaced with valid code dynamically. Ideally, bpf jit should optimize those "ja 40" instructions out when translating the bpf instructions into x64 machine code. However, do_jit() can only remove one "ja 40" for offset==0 on each pass, so it requires at least 40 runs to eliminate those JMPs and exceeds the current limit of passes(20). In the end, the program got rejected when BPF_JIT_ALWAYS_ON is set even though it's legit as a classic socket filter. To make bpf images more likely converge within 20 passes, this commit pads some instructions with NOPs in the last 5 passes: 1. conditional jumps A possible size variance comes from the adoption of imm8 JMP. If the offset is imm8, we calculate the size difference of this BPF instruction between the previous and the current pass and fill the gap with NOPs. To avoid the recalculation of jump offset, those NOPs are inserted before the JMP code, so we have to subtract the 2 bytes of imm8 JMP when calculating the NOP number. 2. BPF_JA There are two conditions for BPF_JA. a.) nop jumps If this instruction is not optimized out in the previous pass, instead of removing it, we insert the equivalent size of NOPs. b.) label jumps Similar to condition jumps, we prepend NOPs right before the JMP code. To make the code concise, emit_nops() is modified to use the signed len and return the number of inserted NOPs. For bpf-to-bpf, we always enable padding for the extra pass since there is only one extra run and the jump padding doesn't affected the images that converge without padding. After applying this patch, the corner case was loaded with the following jit code: flen=45 proglen=77 pass=17 image=ffffffffc03367d4 from=jump pid=10097 JIT code: 00000000: 0f 1f 44 00 00 55 48 89 e5 53 41 55 31 c0 45 31 JIT code: 00000010: ed 48 89 fb eb 30 eb 2e eb 2c eb 2a eb 28 eb 26 JIT code: 00000020: eb 24 eb 22 eb 20 eb 1e eb 1c eb 1a eb 18 eb 16 JIT code: 00000030: eb 14 eb 12 eb 10 eb 0e eb 0c eb 0a eb 08 eb 06 JIT code: 00000040: eb 04 eb 02 66 90 31 c0 41 5d 5b c9 c3 0: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0] 5: 55 push rbp 6: 48 89 e5 mov rbp,rsp 9: 53 push rbx a: 41 55 push r13 c: 31 c0 xor eax,eax e: 45 31 ed xor r13d,r13d 11: 48 89 fb mov rbx,rdi 14: eb 30 jmp 0x46 16: eb 2e jmp 0x46 ... 3e: eb 06 jmp 0x46 40: eb 04 jmp 0x46 42: eb 02 jmp 0x46 44: 66 90 xchg ax,ax 46: 31 c0 xor eax,eax 48: 41 5d pop r13 4a: 5b pop rbx 4b: c9 leave 4c: c3 ret At the 16th pass, 15 jumps were already optimized out, and one jump was replaced with NOPs at 44 and the image converged at the 17th pass. v4: - Add the detailed comments about the possible padding bytes v3: - Copy the instructions of prologue separately or the size calculation of the first BPF instruction would include the prologue. - Replace WARN_ONCE() with pr_err() and EFAULT - Use MAX_PASSES in the for loop condition check - Remove the "padded" flag from x64_jit_data. For the extra pass of subprogs, padding is always enabled since it won't hurt the images that converge without padding. v2: - Simplify the sample code in the description and provide the jit code - Check the expected padding bytes with WARN_ONCE - Move the 'padded' flag to 'struct x64_jit_data' Signed-off-by: Gary Lin --- arch/x86/net/bpf_jit_comp.c | 140 ++++++++++++++++++++++++++++-------- 1 file changed, 112 insertions(+), 28 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 1d4d50199293..b7a2911bda77 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -869,8 +869,31 @@ static void detect_reg_usage(struct bpf_insn *insn, int insn_cnt, } } +static int emit_nops(u8 **pprog, int len) +{ + u8 *prog = *pprog; + int i, noplen, cnt = 0; + + while (len > 0) { + noplen = len; + + if (noplen > ASM_NOP_MAX) + noplen = ASM_NOP_MAX; + + for (i = 0; i < noplen; i++) + EMIT1(ideal_nops[noplen][i]); + len -= noplen; + } + + *pprog = prog; + + return cnt; +} + +#define INSN_SZ_DIFF (((addrs[i] - addrs[i - 1]) - (prog - temp))) + static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, - int oldproglen, struct jit_context *ctx) + int oldproglen, struct jit_context *ctx, bool jmp_padding) { bool tail_call_reachable = bpf_prog->aux->tail_call_reachable; struct bpf_insn *insn = bpf_prog->insnsi; @@ -880,7 +903,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, bool seen_exit = false; u8 temp[BPF_MAX_INSN_SIZE + BPF_INSN_SAFETY]; int i, cnt = 0, excnt = 0; - int proglen = 0; + int ilen, proglen = 0; u8 *prog = temp; int err; @@ -894,7 +917,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, bpf_prog_was_classic(bpf_prog), tail_call_reachable, bpf_prog->aux->func_idx != 0); push_callee_regs(&prog, callee_regs_used); - addrs[0] = prog - temp; + + ilen = prog - temp; + if (image) + memcpy(image + proglen, temp, ilen); + proglen += ilen; + addrs[0] = proglen; + prog = temp; for (i = 1; i <= insn_cnt; i++, insn++) { const s32 imm32 = insn->imm; @@ -903,8 +932,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 b2 = 0, b3 = 0; s64 jmp_offset; u8 jmp_cond; - int ilen; u8 *func; + int nops; switch (insn->code) { /* ALU */ @@ -1502,6 +1531,30 @@ st: if (is_imm8(insn->off)) } jmp_offset = addrs[i + insn->off] - addrs[i]; if (is_imm8(jmp_offset)) { + if (jmp_padding) { + /* To keep the jmp_offset valid, the extra bytes are + * padded before the jump insn, so we substract the + * 2 bytes of jmp_cond insn from INSN_SZ_DIFF. + * + * If the previous pass already emits an imm8 + * jmp_cond, then this BPF insn won't shrink, so + * "nops" is 0. + * + * On the other hand, if the previous pass emits an + * imm32 jmp_cond, the extra 4 bytes(*) is padded to + * keep the image from shrinking further. + * + * (*) imm32 jmp_cond is 6 bytes, and imm8 jmp_cond + * is 2 bytes, so the size difference is 4 bytes. + */ + nops = INSN_SZ_DIFF - 2; + if (nops != 0 && nops != 4) { + pr_err("unexpected jmp_cond padding: %d bytes\n", + nops); + return -EFAULT; + } + cnt += emit_nops(&prog, nops); + } EMIT2(jmp_cond, jmp_offset); } else if (is_simm32(jmp_offset)) { EMIT2_off32(0x0F, jmp_cond + 0x10, jmp_offset); @@ -1524,11 +1577,55 @@ st: if (is_imm8(insn->off)) else jmp_offset = addrs[i + insn->off] - addrs[i]; - if (!jmp_offset) - /* Optimize out nop jumps */ + if (!jmp_offset) { + /* + * If jmp_padding is enabled, the extra nops will + * be inserted. Otherwise, optimize out nop jumps. + */ + if (jmp_padding) { + /* There are 3 possible conditions. + * (1) This BPF_JA is already optimized out in + * the previous run, so there is no need + * to pad any extra byte (0 byte). + * (2) The previous pass emits an imm8 jmp, + * so we pad 2 bytes to match the previous + * insn size. + * (3) Similarly, the previous pass emits an + * imm32 jmp, and 5 bytes is padded. + */ + nops = INSN_SZ_DIFF; + if (nops != 0 && nops != 2 && nops != 5) { + pr_err("unexpected nop jump padding: %d bytes\n", + nops); + return -EFAULT; + } + cnt += emit_nops(&prog, nops); + } break; + } emit_jmp: if (is_imm8(jmp_offset)) { + if (jmp_padding) { + /* To avoid breaking jmp_offset, the extra bytes + * are padded before the actual jmp insn, so + * 2 bytes is substracted from INSN_SZ_DIFF. + * + * If the previous pass already emits an imm8 + * jmp, there is nothing to pad (0 byte). + * + * If it emits an imm32 jmp (5 bytes) previously + * and now an imm8 jmp (2 bytes), then we pad + * (5 - 2 = 3) bytes to stop the image from + * shrinking further. + */ + nops = INSN_SZ_DIFF - 2; + if (nops != 0 && nops != 3) { + pr_err("unexpected jump padding: %d bytes\n", + nops); + return -EFAULT; + } + cnt += emit_nops(&prog, INSN_SZ_DIFF - 2); + } EMIT2(0xEB, jmp_offset); } else if (is_simm32(jmp_offset)) { EMIT1_off32(0xE9, jmp_offset); @@ -1671,26 +1768,6 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, return 0; } -static void emit_nops(u8 **pprog, unsigned int len) -{ - unsigned int i, noplen; - u8 *prog = *pprog; - int cnt = 0; - - while (len > 0) { - noplen = len; - - if (noplen > ASM_NOP_MAX) - noplen = ASM_NOP_MAX; - - for (i = 0; i < noplen; i++) - EMIT1(ideal_nops[noplen][i]); - len -= noplen; - } - - *pprog = prog; -} - static void emit_align(u8 **pprog, u32 align) { u8 *target, *prog = *pprog; @@ -2065,6 +2142,9 @@ struct x64_jit_data { struct jit_context ctx; }; +#define MAX_PASSES 20 +#define PADDING_PASSES (MAX_PASSES - 5) + struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) { struct bpf_binary_header *header = NULL; @@ -2074,6 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) struct jit_context ctx = {}; bool tmp_blinded = false; bool extra_pass = false; + bool padding = false; u8 *image = NULL; int *addrs; int pass; @@ -2110,6 +2191,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) image = jit_data->image; header = jit_data->header; extra_pass = true; + padding = true; goto skip_init_addrs; } addrs = kmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL); @@ -2135,8 +2217,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) * may converge on the last pass. In such case do one more * pass to emit the final image. */ - for (pass = 0; pass < 20 || image; pass++) { - proglen = do_jit(prog, addrs, image, oldproglen, &ctx); + for (pass = 0; pass < MAX_PASSES || image; pass++) { + if (!padding && pass >= PADDING_PASSES) + padding = true; + proglen = do_jit(prog, addrs, image, oldproglen, &ctx, padding); if (proglen <= 0) { out_image: image = NULL; From patchwork Tue Jan 19 10:25:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gary Lin X-Patchwork-Id: 12029851 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF717C433DB for ; Tue, 19 Jan 2021 13:49:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8ACC207B1 for ; Tue, 19 Jan 2021 13:49:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405930AbhASNtC (ORCPT ); Tue, 19 Jan 2021 08:49:02 -0500 Received: from de-smtp-delivery-102.mimecast.com ([62.140.7.102]:52172 "EHLO de-smtp-delivery-102.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390569AbhASKa0 (ORCPT ); Tue, 19 Jan 2021 05:30:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1611052140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yB0csRxSD99waBXb/MVj7U+cqmbggdEYvS9+OCXQWwk=; b=aMxZh1TtHvxxZMu5/Hc2oSe3zTLVpF8fen5PvKfIzo7x5ebYp1DF7UlCYrnhrvlcJmplAb CUElKfxZIGe4m+5ypuxsHOFl9tvp4+g92VIEPd5EnNNPr+4fiXI6T/7ahhNysbAHGJz/Rp jU+gUzFY9GyKd9Ocxt3hiKw6WmZBNhA= Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02lp2050.outbound.protection.outlook.com [104.47.6.50]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-14-CisVifiaMNmnz6RFdfTYyg-2; Tue, 19 Jan 2021 11:25:32 +0100 X-MC-Unique: CisVifiaMNmnz6RFdfTYyg-2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=STE8qbTl0sMHEmaaqYRG5OqKJGFyxuiNUvE6VdXdo1n9cU1Z/WITepEnGuJ3tuR8hFQ92/KmqTvPnnEsPPLTRrIfNbzQqiCpFwVj77724VLrgZEtrrtl/sbaapNhtm5S0qSZg4T3kHI0DlLk+nfZ9OsIo5WyQktBacgoz3V6Uc+fzobsf47DD1JQIXoIKBIOKMa2FyctkLGnjsBLyFRjDy3wzD8vT0jF63D79TNKbKncW+0KBY6+I8ewVQdpusYoryiibF/Qccx5obEZnsXmjWuqUH0ZlVoPJn0jQxd1bmXcVB8MhOmJmQikPCA3z5dDH3rKur3QPty+uT/KdMl4rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EiPEfNmnPGyGHDv9RJ4rOy1Wt8uwYiAlb4GTKbKYxP8=; b=a/7XPFh4Clns2D4sc2KV71RLxSWLd6Jzco8vTJ2dxjhRwNwgpsphl9tk+m0f5OtoxcF7zLNK6ujzBd1/xZkI9YKW4XPVcvJ4Z/8WAAlSHPWBUOjqs7HAJbfa1sfi/DyQgKBIXOpGb9znxL7i3cY66au07eM0qYAgWMDl6D3zval8r/WkCOSpdxpXtVwNKymUP4uFNU+aoTTNBjq5S313aZvbyKdxbuxYUm+Zl68UcOsqoyyPqmnE1qTDIpqv4AmTZ7+1WQwRNPcq9S0PAV6XyEP6GVAo9f/mP6BP6/L7pf7rz+CBEfm5M1jW76EVWmGuAnFpv7iSJKfFf4SrGoBcxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com; Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) by DB6PR04MB3109.eurprd04.prod.outlook.com (2603:10a6:6:f::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.14; Tue, 19 Jan 2021 10:25:30 +0000 Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313]) by DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313%6]) with mapi id 15.20.3763.014; Tue, 19 Jan 2021 10:25:30 +0000 From: Gary Lin To: netdev@vger.kernel.org, bpf@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann CC: Eric Dumazet , Andrii Nakryiko , andreas.taschner@suse.com Subject: [PATCH v4 2/3] test_bpf: remove EXPECTED_FAIL flag from bpf_fill_maxinsns11 Date: Tue, 19 Jan 2021 18:25:00 +0800 Message-ID: <20210119102501.511-3-glin@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210119102501.511-1-glin@suse.com> References: <20210119102501.511-1-glin@suse.com> X-Originating-IP: [111.240.113.126] X-ClientProxiedBy: FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) To DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from GaryLaptop.prv.suse.net (111.240.113.126) by FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.6 via Frontend Transport; Tue, 19 Jan 2021 10:25:28 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d1c8b0b7-a330-4335-af37-08d8bc648bcb X-MS-TrafficTypeDiagnostic: DB6PR04MB3109: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1284; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /Yf2etPIq2ZazqosfLaocxPHhTPr82JrAgkN92GmU+dw5h2yOiMgIG/CdcYQtJOUQX0ZDteXXTu0+adyuyBJmPOMthaA5Zb5ERwL3H62zvjbYNuFlQqGp+4bnR8hSPkbMt98mrr0LqRzgQ/h0PQ2t0GSQFnlRUSTqBgDZtjk36FoRKbnnBMIAXKRkQCkGemyNYzca7v2k2BddJkcGORj1tsNnIcmSn2E0HQqYOONss6nUkgrZAmGzO9cZMCrceLT17GSB6CngCtDDpLubqPMbtz4yR6xW+Jivy5zlqXtHef6xH2z8sfDX4xu19TR3YyQJb8ki4KdzbQCp7kHcXCJjUJU8XErQwy66v/1vYJQ0muT9zvzh96GzESY1dqIZ2ItuxQ4iqEray9afem3H5cV/w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB3PR0402MB3641.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(39860400002)(136003)(376002)(366004)(346002)(83380400001)(1076003)(186003)(26005)(6486002)(52116002)(316002)(956004)(16526019)(6512007)(6506007)(4326008)(107886003)(8676002)(5660300002)(36756003)(66556008)(2906002)(8936002)(66946007)(66476007)(86362001)(2616005)(478600001)(110136005)(6666004)(54906003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: rQptqlvNQyid0pXjaGoN26lH/plhx/Xo3gVEjuRZMAG3EiSPJ1a/bg/2JLa3zd4+bcP4yBFs7QD/APVTdM9SrBdmQxEwk1aRTMpL2NgiQ4lk+nKBBeoMtpYpSzigtk3sMmZPOJx4SpbCcchu32UpIQvlhANW4Rpx3lRBeAG8Fw6fDNR3FkHrJiGXZTrIxsYiyVJS47zPDxhM52kzvx8JpOR6pswrVFTXd6Vg7ilTWB3xHFZpn18ll8EHDmqcPHKceXxAJNB3pwv5q/eUcc1bED+h7IhYYdUx2vHKuqzq/XijmVKWSkQVZCaRI61ExE/lLyvKPX59Luamnky4k6kfjpZDBJg7EUx0iAKoy+knSCgmNHrQK6F43h80cdHo93Y79q9ygEo/Emkw1PLIrHf7imKluw6Pe7fkm8mSUx0VqS1ATTij2wo07N3nerraV8rqqD2kLSZDD4TCgs6Ma76ANucqzLGfGUU6PfEOohHnH42QsLFDdAuJugqoUb/eX39e44u1yUp9/UQQzWeiGlHcaLdGvdFiMOkfluGHDH5CNKibUgtBQPfbH3avgyo8zoc2gtom7xaRBraMKkvfoJScph7QKzK6vTDefVVq+sjZ3ntA8GHZfNA07w9VkX+Law+CkWpHsGlCGvm1LkxZFGLZxofGuaX/wFJM7aXiMQg0y6hCb1e8/4dwse/9SrlDzuX61tWrufrJJeijfNFrDNL1REUtU+qcmkLBk7ZQqFKQvTdjgWvtPcqxAiN7VFQEZ1IaXxzLjIpkYm/Ml1yVgFZ9JosqB+16zM9/ociTdHgDZQmLLtFxqXihFWCcxyHgY88rE+Y572RdPT1kKlTzcX/BX5q1LXKeNJ4pUofOzDm7WhGbh9OeI5TGcbIVrfB3xg5xcMnAqUPcRuJQwxU+ZHjjn3F5UZI1ebTob/HAk8mR5wK7DoppwP2eWKHJTwHxXfjSJUdBL/WaFxT7+hgNnSHZqwk8E0eH6AIbQllhc7faQNkMlc1SXdCJmlg8MDECA2Dj X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: d1c8b0b7-a330-4335-af37-08d8bc648bcb X-MS-Exchange-CrossTenant-AuthSource: DB3PR0402MB3641.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2021 10:25:30.5745 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fkv1Y4+zi5qkD26nKGISpsYFDXGxXkjacdj5otRFIgcKPvuGWTGL8rM8mT0w0g5V X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR04MB3109 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net With NOPs padding, x64 jit now can handle the jump cases like bpf_fill_maxinsns11(). Signed-off-by: Gary Lin --- lib/test_bpf.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/lib/test_bpf.c b/lib/test_bpf.c index 49ec9e8d8aed..4dc4dcbecd12 100644 --- a/lib/test_bpf.c +++ b/lib/test_bpf.c @@ -345,7 +345,7 @@ static int __bpf_fill_ja(struct bpf_test *self, unsigned int len, static int bpf_fill_maxinsns11(struct bpf_test *self) { - /* Hits 70 passes on x86_64, so cannot get JITed there. */ + /* Hits 70 passes on x86_64 and triggers NOPs padding. */ return __bpf_fill_ja(self, BPF_MAXINSNS, 68); } @@ -5318,15 +5318,10 @@ static struct bpf_test tests[] = { { "BPF_MAXINSNS: Jump, gap, jump, ...", { }, -#if defined(CONFIG_BPF_JIT_ALWAYS_ON) && defined(CONFIG_X86) - CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL, -#else CLASSIC | FLAG_NO_DATA, -#endif { }, { { 0, 0xababcbac } }, .fill_helper = bpf_fill_maxinsns11, - .expected_errcode = -ENOTSUPP, }, { "BPF_MAXINSNS: jump over MSH", From patchwork Tue Jan 19 10:25:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gary Lin X-Patchwork-Id: 12029853 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 462ABC433E9 for ; Tue, 19 Jan 2021 13:49:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C443207B1 for ; Tue, 19 Jan 2021 13:49:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405982AbhASNtG (ORCPT ); Tue, 19 Jan 2021 08:49:06 -0500 Received: from de-smtp-delivery-102.mimecast.com ([62.140.7.102]:28714 "EHLO de-smtp-delivery-102.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390572AbhASKa0 (ORCPT ); Tue, 19 Jan 2021 05:30:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1611052140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nhWA5JxYXlHyY8G9GP9kKMoeAua8zPUto7O7q9ccu4c=; b=W9VvWW3N27N6136os0ea6KeR0ZXs0fPXubbi9vmI6A+OwZaXtz+3OIOaA0kJ+A/Rd/em3W 9tlKoMJ07SmD6sPHdImeDJ9+PoJUtHtz+HEfc+4NY3IKdnC/0q6O4Thcz7aXb0LVK9EUL9 YH2/5M8yYjI2ecSmDu1/ndL6WYBxsOQ= Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02lp2051.outbound.protection.outlook.com [104.47.6.51]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-16-dvSWzTAQP3eGMi-8I9tbQg-1; Tue, 19 Jan 2021 11:25:35 +0100 X-MC-Unique: dvSWzTAQP3eGMi-8I9tbQg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KOdEGCkJKTJjsm9lJCdZXa0S86/Te0U9PGm+TvaMDmUxoTKWsbXj0gZABLJpnU+PXaheeX3xW5bx6O7GT/+0HfkYuYToWcpPWVzIxIUUmjBc8ViLpn655V8uLHjIT7t3MwGKYDe+FKDNFiWEVwnwlABsMydK9a4aLYYuF7utzxbVWsJKpeTn1fQ0HxQq1KNUTwubhn8PZrhPRYWJNtQnajq9Gv4rY+xV0d65GzDGhTaB+6lXmlWPHVdV3jAT3Kf2GiSo6/NEzdcx4Rx/WvkdwmXhI5OBTuFZ2eaYYvz9vd7WCUMawrCvHFNIkiqNUH9NXJ7J35y6nnp9jrAAnW2lUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ustf11rjWTfbLqoyEYFfYL0TMSoEMf15WGLRvDebExs=; b=oN4K26ZqDTKmnLuftCsbynFJHgRtzDIqG1fLYPxo4LlkZqZDUtJwInmbA30G04EaF0nAR72XaObNnxvAaVrA44DsKPyr7SmOTvOGMU4LT9RPbaBHGeD0dQTackkdY++kaCiAI+Nc8IYDMIoLYG13WJk9FrpnAfilysmXPQh8SV9f5wK4BjOcd0PEK1+kTCdWZubzGxEoezxTkGIMdWKvV+DRTSf+jv/Yon0Vf5FvoRgcCNwrPM8RJffQmHPErMGzxRoNdJmW3N02EpHmftsX1A0QiwsR2XTglDrwoV8kUHymtJnSsXjeDqgsUELVRJjSx4vlhcwSuiiXJPeb+VhA6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=suse.com; Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) by DBBPR04MB7964.eurprd04.prod.outlook.com (2603:10a6:10:1e9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.11; Tue, 19 Jan 2021 10:25:33 +0000 Received: from DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313]) by DB3PR0402MB3641.eurprd04.prod.outlook.com ([fe80::80c9:1fa3:ae84:7313%6]) with mapi id 15.20.3763.014; Tue, 19 Jan 2021 10:25:33 +0000 From: Gary Lin To: netdev@vger.kernel.org, bpf@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann CC: Eric Dumazet , Andrii Nakryiko , andreas.taschner@suse.com Subject: [PATCH v4 3/3] selftests/bpf: Add verifier tests for x64 jit jump padding Date: Tue, 19 Jan 2021 18:25:01 +0800 Message-ID: <20210119102501.511-4-glin@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210119102501.511-1-glin@suse.com> References: <20210119102501.511-1-glin@suse.com> X-Originating-IP: [111.240.113.126] X-ClientProxiedBy: FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) To DB3PR0402MB3641.eurprd04.prod.outlook.com (2603:10a6:8:b::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from GaryLaptop.prv.suse.net (111.240.113.126) by FR2P281CA0023.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:14::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.6 via Frontend Transport; Tue, 19 Jan 2021 10:25:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1ccc1452-ac5a-4829-c2c0-08d8bc648d99 X-MS-TrafficTypeDiagnostic: DBBPR04MB7964: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: R7ENsDMOtGTuh1nKdAAG3fpM6jT5dy0yHc71pLv8sUXWRSyghYjeg8T/5mhHLgDSHJ6hqB4BUeADq/4poUGywl+R1OruQJoVaNha3IDLWH7iusKQLyU/uk5SwcLgT3epUjdtxcrSjVT66Dyb2pcfYFVhOCdX7HmPGle5p9iZvIrIu4k8zbF3VKG53ZJGZJxQWhn1YcnNUkwih0lYwS7FWqnNKhks4MDxYCglkO3/RLf/NF6/n/w2kSSLdQ/uylunKV8KplvMs1/VCGTkT0REXM1AMgYRwMiAQY/N5Nj1s92odYbxjGEYzF6dSRzM+9IMmIJU9qarkVTvL8XEP96Ugq8aEyZc4Rf/opUakdAb0+WOk8pCqIx0bCVEjaOlleo+b34a+ydWxVaS8A+QDrYT6Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB3PR0402MB3641.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(396003)(366004)(376002)(136003)(39860400002)(346002)(6666004)(2906002)(36756003)(6486002)(316002)(110136005)(52116002)(8676002)(186003)(1076003)(2616005)(16526019)(86362001)(956004)(5660300002)(66476007)(6512007)(4326008)(26005)(6506007)(66556008)(107886003)(8936002)(66946007)(54906003)(478600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: gS8xm+IzptgIrz9A9XtfShwt+1U8aJL0tviLS24OI5Rz40HZSDBljmlJr3G270GnBK01h0mqk+bL9rQIfVpow26LJnLHWbEln8jxoyEFIVAQVbjoaHTPejpDGY2GFjMKrNT8kM5wRv4HL0AoBpERRqBryikQwY0lUQCr8OoaKtv5pOxqpKtufpIgW2B8dlmFXlmw19vVBJslsDhbiPOTDPkQZqxmED2CAt2axUf2qKdPC4mEG0D7RZPt9RV9+dpZgo8q3HpeMp9btO8nXzgLVqRceTIxnAOYm8rs+X6q+iNz96Pzu16+3MAngwpNnc2u7J7p3Sh3BhhrzZbPtOHr5RvO2PgjnWN/8eSPux0Xx1DwCXlimb5MR1lOKM1qCBWAsUnhsplZ/zou0J9xl/Uz/52vNV7YspY9Ld4dpX+3W0AXBhZ5F5t9TDp9jyYvVKPyPjXbFbhi+4pFNjb8CxV4Ia4/Q7De5dbayprYLxrqEMglAGSFC+MPXQxrSnkNvW7JsnVf0D7hzxq2QIwJvcV7hPttvyYwUzoBJ2Dmi4GejdlwbFvpCMKe5w8BZxgFOJMIWBHFwwCLGXzh9iv8hDNwNJ8bvX5iFBmjKF6ucMP+5NOIo5Y5exuc0JCMpgWRt3MkkprZX0gATu0QQdlJzSMVPaIQy7aog6xxBzG4jJIuemfW1qNp8hPYb/0uubwCRwRqZloVybJffva9S5DCPO6umOkpFYjwj31iDgJF37BzalG9Zi7UWeZKzwdWTW+BzNHasexQSqc3KBIwr/m7AO42ssRk2Ge7h2eXmEoWoPTwcaHR/FXTcJUk+/5yZpNsODCG1rkzG52deiRINvj7knfvGcKIaZjHtPxTCNeYNQocWk90m0TeOrEoLAexA2kEDUanX8QmvFfgXvJmMHcV1ZEqyIkX6GaT5p4yf00kyp1fnoOZ6UBKCYrK2gFmwDwhUofcrF3I0qca5P7UKNw/2+Wib/WyXAIsZCcQ6VzICUo5ikY9eWvgfjUqUQRIZV6R1lMI X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1ccc1452-ac5a-4829-c2c0-08d8bc648d99 X-MS-Exchange-CrossTenant-AuthSource: DB3PR0402MB3641.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2021 10:25:33.5848 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: r9w3ha7H/JVvfy1xM+fBcWJPoBzVvLJDYrt89yhOORdtKU4IIcfdnASRkT/e941x X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7964 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net There are 3 tests added into verifier's jit tests to trigger x64 jit jump padding. The first test can be represented as the following assembly code: 1: bpf_call bpf_get_prandom_u32 2: if r0 == 1 goto pc+128 3: if r0 == 2 goto pc+128 ... 129: if r0 == 128 goto pc+128 130: goto pc+128 131: goto pc+127 ... 256: goto pc+2 257: goto pc+1 258: r0 = 1 259: ret We first store a random number to r0 and add the corresponding conditional jumps (2~129) to make verifier believe that those jump instructions from 130 to 257 are reachable. When the program is sent to x64 jit, it starts to optimize out the NOP jumps backwards from 257. Since there are 128 such jumps, the program easily reaches 15 passes and triggers jump padding. Here is the x64 jit code of the first test: 0: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0] 5: 66 90 xchg ax,ax 7: 55 push rbp 8: 48 89 e5 mov rbp,rsp b: e8 4c 90 75 e3 call 0xffffffffe375905c 10: 48 83 f8 01 cmp rax,0x1 14: 0f 84 fe 04 00 00 je 0x518 1a: 48 83 f8 02 cmp rax,0x2 1e: 0f 84 f9 04 00 00 je 0x51d ... f6: 48 83 f8 18 cmp rax,0x18 fa: 0f 84 8b 04 00 00 je 0x58b 100: 48 83 f8 19 cmp rax,0x19 104: 0f 84 86 04 00 00 je 0x590 10a: 48 83 f8 1a cmp rax,0x1a 10e: 0f 84 81 04 00 00 je 0x595 ... 500: 0f 84 83 01 00 00 je 0x689 506: 48 81 f8 80 00 00 00 cmp rax,0x80 50d: 0f 84 76 01 00 00 je 0x689 513: e9 71 01 00 00 jmp 0x689 518: e9 6c 01 00 00 jmp 0x689 ... 5fe: e9 86 00 00 00 jmp 0x689 603: e9 81 00 00 00 jmp 0x689 608: 0f 1f 00 nop DWORD PTR [rax] 60b: eb 7c jmp 0x689 60d: eb 7a jmp 0x689 ... 683: eb 04 jmp 0x689 685: eb 02 jmp 0x689 687: 66 90 xchg ax,ax 689: b8 01 00 00 00 mov eax,0x1 68e: c9 leave 68f: c3 ret As expected, a 3 bytes NOPs is inserted at 608 due to the transition from imm32 jmp to imm8 jmp. A 2 bytes NOPs is also inserted at 687 to replace a NOP jump. The second test case is tricky. Here is the assembly code: 1: bpf_call bpf_get_prandom_u32 2: if r0 == 1 goto pc+2048 3: if r0 == 2 goto pc+2048 ... 2049: if r0 == 2048 goto pc+2048 2050: goto pc+2048 2051: goto pc+16 2052: goto pc+15 ... 2064: goto pc+3 2065: goto pc+2 2066: goto pc+1 ... [repeat "goto pc+16".."goto pc+1" 127 times] ... 4099: r0 = 2 4100: ret There are 4 major parts of the program. 1) 1~2049: Those are instructions to make 2050~4098 reachable. Some of them also could generate the padding for jmp_cond. 2) 2050: This is the target instruction for the imm32 nop jmp padding. 3) 2051~4098: The repeated "goto 1~16" instructions are designed to be consumed by the nop jmp optimization. In the end, those instrucitons become 128 continuous 0 offset jmp and are optimized out in 1 pass, and this make insn 2050 an imm32 nop jmp in the next pass, so that we can trigger the 5 bytes padding. 4) 4099~4100: Those are the instructions to end the program. The x64 jit code is like this: 0: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0] 5: 66 90 xchg ax,ax 7: 55 push rbp 8: 48 89 e5 mov rbp,rsp b: e8 bc 7b d5 d3 call 0xffffffffd3d57bcc 10: 48 83 f8 01 cmp rax,0x1 14: 0f 84 7e 66 00 00 je 0x6698 1a: 48 83 f8 02 cmp rax,0x2 1e: 0f 84 74 66 00 00 je 0x6698 24: 48 83 f8 03 cmp rax,0x3 28: 0f 84 6a 66 00 00 je 0x6698 2e: 48 83 f8 04 cmp rax,0x4 32: 0f 84 60 66 00 00 je 0x6698 38: 48 83 f8 05 cmp rax,0x5 3c: 0f 84 56 66 00 00 je 0x6698 42: 48 83 f8 06 cmp rax,0x6 46: 0f 84 4c 66 00 00 je 0x6698 ... 666c: 48 81 f8 fe 07 00 00 cmp rax,0x7fe 6673: 0f 1f 40 00 nop DWORD PTR [rax+0x0] 6677: 74 1f je 0x6698 6679: 48 81 f8 ff 07 00 00 cmp rax,0x7ff 6680: 0f 1f 40 00 nop DWORD PTR [rax+0x0] 6684: 74 12 je 0x6698 6686: 48 81 f8 00 08 00 00 cmp rax,0x800 668d: 0f 1f 40 00 nop DWORD PTR [rax+0x0] 6691: 74 05 je 0x6698 6693: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0] 6698: b8 02 00 00 00 mov eax,0x2 669d: c9 leave 669e: c3 ret Since insn 2051~4098 are optimized out right before the padding pass, there are several conditional jumps from the first part are replaced with imm8 jmp_cond, and this triggers the 4 bytes padding, for example at 6673, 6680, and 668d. On the other hand, Insn 2050 is replaced with the 5 bytes nops at 6693. The third test is to invoke the first and second tests as subprogs to test bpf2bpf. Per the system log, there was one more jit happened with only one pass and the same jit code was produced. v4: - Add the second test case which triggers jmp_cond padding and imm32 nop jmp padding. - Add the new test case as another subprog Signed-off-by: Gary Lin --- tools/testing/selftests/bpf/test_verifier.c | 72 +++++++++++++++++++++ tools/testing/selftests/bpf/verifier/jit.c | 24 +++++++ 2 files changed, 96 insertions(+) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 777a81404fdb..e0e65ff30a2d 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -296,6 +296,78 @@ static void bpf_fill_scale(struct bpf_test *self) } } +static int bpf_fill_torturous_jumps_insn_1(struct bpf_insn *insn) +{ + unsigned int len = 259, hlen = 128; + int i; + + insn[0] = BPF_EMIT_CALL(BPF_FUNC_get_prandom_u32); + for (i = 1; i <= hlen; i++) { + insn[i] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, i, hlen); + insn[i + hlen] = BPF_JMP_A(hlen - i); + } + insn[len - 2] = BPF_MOV64_IMM(BPF_REG_0, 1); + insn[len - 1] = BPF_EXIT_INSN(); + + return len; +} + +static int bpf_fill_torturous_jumps_insn_2(struct bpf_insn *insn) +{ + unsigned int len = 4100, jmp_off = 2048; + int i, j; + + insn[0] = BPF_EMIT_CALL(BPF_FUNC_get_prandom_u32); + for (i = 1; i <= jmp_off; i++) { + insn[i] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, i, jmp_off); + } + insn[i++] = BPF_JMP_A(jmp_off); + for (; i <= jmp_off * 2 + 1; i+=16) { + for (j = 0; j < 16; j++) { + insn[i + j] = BPF_JMP_A(16 - j - 1); + } + } + + insn[len - 2] = BPF_MOV64_IMM(BPF_REG_0, 2); + insn[len - 1] = BPF_EXIT_INSN(); + + return len; +} + +static void bpf_fill_torturous_jumps(struct bpf_test *self) +{ + struct bpf_insn *insn = self->fill_insns; + int i = 0; + + switch (self->retval) { + case 1: + self->prog_len = bpf_fill_torturous_jumps_insn_1(insn); + return; + case 2: + self->prog_len = bpf_fill_torturous_jumps_insn_2(insn); + return; + case 3: + /* main */ + insn[i++] = BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, 4); + insn[i++] = BPF_RAW_INSN(BPF_JMP|BPF_CALL, 0, 1, 0, 262); + insn[i++] = BPF_ST_MEM(BPF_B, BPF_REG_10, -32, 0); + insn[i++] = BPF_MOV64_IMM(BPF_REG_0, 3); + insn[i++] = BPF_EXIT_INSN(); + + /* subprog 1 */ + i += bpf_fill_torturous_jumps_insn_1(insn + i); + + /* subprog 2 */ + i += bpf_fill_torturous_jumps_insn_2(insn + i); + + self->prog_len = i; + return; + default: + self->prog_len = 0; + break; + } +} + /* BPF_SK_LOOKUP contains 13 instructions, if you need to fix up maps */ #define BPF_SK_LOOKUP(func) \ /* struct bpf_sock_tuple tuple = {} */ \ diff --git a/tools/testing/selftests/bpf/verifier/jit.c b/tools/testing/selftests/bpf/verifier/jit.c index c33adf344fae..df215e004566 100644 --- a/tools/testing/selftests/bpf/verifier/jit.c +++ b/tools/testing/selftests/bpf/verifier/jit.c @@ -105,3 +105,27 @@ .result = ACCEPT, .retval = 2, }, +{ + "jit: torturous jumps, imm8 nop jmp and pure jump padding", + .insns = { }, + .fill_helper = bpf_fill_torturous_jumps, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, + .result = ACCEPT, + .retval = 1, +}, +{ + "jit: torturous jumps, imm32 nop jmp and jmp_cond padding", + .insns = { }, + .fill_helper = bpf_fill_torturous_jumps, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, + .result = ACCEPT, + .retval = 2, +}, +{ + "jit: torturous jumps in subprog", + .insns = { }, + .fill_helper = bpf_fill_torturous_jumps, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, + .result = ACCEPT, + .retval = 3, +},