From patchwork Thu Jun 13 01:38:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13695769 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DBAF168D0 for ; Thu, 13 Jun 2024 01:38:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242706; cv=none; b=BaM4iOw2/fZEoMKsio0uLy5CpWSOeqb9rZi+qiHXljfm9D4Guqv5JLk2IFPIrp6mo2jCW59BCbWv7f9GB7bADmvdJmpnIpFsuW0iF0IZ2HZmStnw6QgN4axHZR7bgIzbp03D+PM5Cuu34uaqIDVoVGIKVGWPshZ9AvESbZctKGA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242706; c=relaxed/simple; bh=vTNhC6ujDjkHa+aMh1zTZRDke/yxsmbKQ9NSFXn4DTE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kryP0cW8uCRQoYyAC/S0yobVhKhDRqLZJu8urrlXQim+veOzsd6tp4UlrYkH4OpFeeM/5EMeFSUxNO9dw21aF77NEVrk28VcVw3tovlku45JZOGPw4q5eaDIo0y+j/bzw2nquh+W+4LyVPJ/gct1rkKhJznMdRYu4njgZZ2tmkA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MyYtSMUU; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MyYtSMUU" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1f6f1677b26so3648815ad.0 for ; Wed, 12 Jun 2024 18:38:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718242704; x=1718847504; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RCw1jrs3zUrHyZp0xVHEyZoD6RGIp5bFSRxbPOgyZ/E=; b=MyYtSMUUrsKA5lBTfrprwVu6/uY0k6DXennbG2PRRH0A8J8gRHbe7Xb4l7KrluadLx 4xRO74cavYU958VS2SyCQuK3piEUEvsy1ps66k7e6/5ZU4+tlrL6P+py3BGKbWlAPS5Q phRz0TFu801gtt02kLHpez1IH+gLFbnhXYE8kZaq8kWczHkAqLXxh68k3wBqgFfZs7zO AT9faN0ewTBAlh2e6tHquI6zW37PtcM2WEgr4LgIOmOFuy57ZQHbkKwUPAG1uq6aEGDP grTkUzF2VaRcJuQhL7YZyPmpKLh+n10p4NfNPmqpA1SmqckM6gGj9h2a3/4GppUNAOpS 6dgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718242704; x=1718847504; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RCw1jrs3zUrHyZp0xVHEyZoD6RGIp5bFSRxbPOgyZ/E=; b=iYpYG0cTnm1JGxCkdrcTocZHsm/GpvsdhrpagaOZ6Gnf7G7mKfCWoNC58xpHrVvsI+ DP4p+A36vR4irA9Hk3VTH5HpF/maJ6Hhp9P1wITn4jGmVeQ829L8PVnBLZbP7UjlP//r u4lKnlgL43LGDEEJAAOnXq4JUojVo/QkIR94hT2TLw63UCOW7SVDIJmjlB2FgQwA2CBj VHERo0Qbu8+vJprFGqeYPzkvh0h29fqp8MG/7juRHlgdhgNaFDZITxvwpRGKUiuXVS7W 2mG3vXqmRn26Nj3bkRadI3GuFjvP68Xrq1YE4s3od+/PibI4pKF2wrt6e99YrA449xKd aseA== X-Gm-Message-State: AOJu0YzaBIGLBjG/PJ8ekRisPrUdCsG3vZVtjSD5lWbZg4jjm5ZJ4Lme RK+EkRYFVZocAvAI9YGuFf8ivZuvrQELOslLip/Ij82/IaCWBc1jHqIB2w== X-Google-Smtp-Source: AGHT+IHW/JmhnOa/zleQr5R3qNvjTkTmnWfEOJms97W+7DKP13j4EV5484Pog3iluf+O47AlVI3R0w== X-Received: by 2002:a17:902:ccc4:b0:1f7:3a70:9e71 with SMTP id d9443c01a7336-1f84e2cb510mr20409345ad.13.1718242703608; Wed, 12 Jun 2024 18:38:23 -0700 (PDT) Received: from macbook-pro-49.dhcp.thefacebook.com ([2620:10d:c090:400::5:b914]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855f339fbsm1279905ad.262.2024.06.12.18.38.22 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 12 Jun 2024 18:38:23 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, memxor@gmail.com, eddyz87@gmail.com, kernel-team@fb.com Subject: [PATCH v3 bpf-next 1/4] bpf: Relax tuple len requirement for sk helpers. Date: Wed, 12 Jun 2024 18:38:12 -0700 Message-Id: <20240613013815.953-2-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240613013815.953-1-alexei.starovoitov@gmail.com> References: <20240613013815.953-1-alexei.starovoitov@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov __bpf_skc_lookup() safely handles incorrect values of tuple len, hence we can allow zero to be passed as tuple len. This patch alone doesn't make an observable verifier difference. It's a trivial improvement that might simplify bpf programs. Acked-by: Eduard Zingerman Signed-off-by: Alexei Starovoitov --- net/core/filter.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/net/core/filter.c b/net/core/filter.c index 7c46ecba3b01..cb133232a887 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -6815,7 +6815,7 @@ static const struct bpf_func_proto bpf_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6834,7 +6834,7 @@ static const struct bpf_func_proto bpf_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6853,7 +6853,7 @@ static const struct bpf_func_proto bpf_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6877,7 +6877,7 @@ static const struct bpf_func_proto bpf_tc_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6901,7 +6901,7 @@ static const struct bpf_func_proto bpf_tc_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6925,7 +6925,7 @@ static const struct bpf_func_proto bpf_tc_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6963,7 +6963,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6987,7 +6987,7 @@ static const struct bpf_func_proto bpf_xdp_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7011,7 +7011,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7031,7 +7031,7 @@ static const struct bpf_func_proto bpf_sock_addr_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7050,7 +7050,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7069,7 +7069,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; From patchwork Thu Jun 13 01:38:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13695770 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C4ED168D0 for ; Thu, 13 Jun 2024 01:38:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242711; cv=none; b=eDv/H6ymQX4YVf+PbBJtOY07EAjaskS0kpGzgcxhLI2KhXRBgdMh1muipzG2m0YEUBBawTpNNCGXpDXUPUjGNTO8pxNMUGYy83ZK++dx55TLShPnSQCK0N/dG+Y3pjXSwzLC9/fiozXwHj3a1gDhIPLaB1h24rmh6uu3gK0SKps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242711; c=relaxed/simple; bh=zgngmNcyOk//N6htq6R9dUV+AgfmK5lbqflXO/+aJgU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JCwqscmFVOfQISKrlO+T1YeSFn2tumRuIVsDNBr+Lu+OeLW9FQ4Mo5t+g8GFgXC2DGxtZbrQU3DFArrepmlW1thrDwnHwexA4lEU6kOSrHaNlhNnvmu29IwZJTxpMJHdv6SL0r+HNxqOp3DjDB+xDjOxl2g4arO1d9QqrbCFnZc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=F0mLJlEz; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F0mLJlEz" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-681ad081695so354075a12.3 for ; Wed, 12 Jun 2024 18:38:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718242708; x=1718847508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=njrV3v7woP1cH7iimvXxDwEswIq9Y84fcQp3rkMvhAU=; b=F0mLJlEzWN0U2O6++Jibrge5F07YQnnOZAzGEagaa9AEhiS0WweF42nLCErXROiDhU Ll+ovQglygS8HnSTgOcDLEPkc3090YUC7iqQTZ6NZZ/WGIwfM30pyLhegdb8Vewk7uia dYH/Fos6Th/n/XBT6xFsdoPliXyqhbxzxDr79/zEdTiD30Jzd0KwfPTsCCa5kHnExmsP YHPDAupZ0vyiXsiYLAIrVKtu+Oq/6OL0uSMbXKJkdo6ZU9Ee8vZIhhdjo9Sl/bXKL6Sk 8zYAj+WkThm074xEtCU6Siu+Q5UZjLJcZNCnY/G2NmHZTXduWeChl0BsJBg+HkiTH9d2 8mkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718242708; x=1718847508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=njrV3v7woP1cH7iimvXxDwEswIq9Y84fcQp3rkMvhAU=; b=ZNP5yxdMqZUgIaT11Ie+rgE906qeAiV2uqKHiAHO2NWPC2Q+LUaO/mC+3am+dyuOad c1g/UFeaeOj6IZ8eH8CcQWA/1ATQwmMFaOxbWnmm9q98KsBoG8hYwkqeIEtdHM6FtNn5 8YL5aEEzpAs+OFZ0yUnL+qo15i6TWGRPtNB0ugxE0Abi36z7295kMV3amlJ/wMYyJGUL +liSXzfDpNlXvwdsaXw37SEXJTMICUA2RNFqrEeMVaVdSaCFkvRIk8k6LfwQ8/W99zv2 WyZf/m/3DD0C2A0VVydpnGg9eM1jRO+7avx8Xlbho1f5c0Qt8D1MO6gfWGwQamL7TApK wwFw== X-Gm-Message-State: AOJu0YyvF0NMJb94KXzFMVp07CYhnVLh+1y3MYi5keO7kV/xMNeMkXhl SEPVKOOp3kpf2UFhlVt03Y5n8AgP9Dbe/7rXn4eizqwGviX/Lz+71fUaaQ== X-Google-Smtp-Source: AGHT+IEkYyC3JUAYYRiuGqQifRu3/QcB8zMOBU3SpQjrtAAB1l5aww3T1ebNvcr5ZEk530cXSFgVag== X-Received: by 2002:a05:6a20:6a0f:b0:1b5:d173:338c with SMTP id adf61e73a8af0-1b8a9b4ebb0mr4381133637.23.1718242707499; Wed, 12 Jun 2024 18:38:27 -0700 (PDT) Received: from macbook-pro-49.dhcp.thefacebook.com ([2620:10d:c090:400::5:b914]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c4a7602690sm2600317a91.28.2024.06.12.18.38.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 12 Jun 2024 18:38:27 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, memxor@gmail.com, eddyz87@gmail.com, kernel-team@fb.com Subject: [PATCH v3 bpf-next 2/4] bpf: Track delta between "linked" registers. Date: Wed, 12 Jun 2024 18:38:13 -0700 Message-Id: <20240613013815.953-3-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240613013815.953-1-alexei.starovoitov@gmail.com> References: <20240613013815.953-1-alexei.starovoitov@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Compilers can generate the code r1 = r2 r1 += 0x1 if r2 < 1000 goto ... use knowledge of r2 range in subsequent r1 operations So remember constant delta between r2 and r1 and update r1 after 'if' condition. Unfortunately LLVM still uses this pattern for loops with 'can_loop' construct: for (i = 0; i < 1000 && can_loop; i++) The "undo" pass was introduced in LLVM https://reviews.llvm.org/D121937 to prevent this optimization, but it cannot cover all cases. Instead of fighting middle end optimizer in BPF backend teach the verifier about this pattern. Signed-off-by: Alexei Starovoitov Acked-by: Eduard Zingerman --- include/linux/bpf_verifier.h | 12 ++- kernel/bpf/log.c | 4 +- kernel/bpf/verifier.c | 95 ++++++++++++++++--- .../testing/selftests/bpf/verifier/precise.c | 22 ++--- 4 files changed, 109 insertions(+), 24 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 50aa87f8d77f..2b54e25d2364 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -73,7 +73,10 @@ enum bpf_iter_state { struct bpf_reg_state { /* Ordering of fields matters. See states_equal() */ enum bpf_reg_type type; - /* Fixed part of pointer offset, pointer types only */ + /* + * Fixed part of pointer offset, pointer types only. + * Or constant delta between "linked" scalars with the same ID. + */ s32 off; union { /* valid when type == PTR_TO_PACKET */ @@ -167,6 +170,13 @@ struct bpf_reg_state { * Similarly to dynptrs, we use ID to track "belonging" of a reference * to a specific instance of bpf_iter. */ + /* + * Upper bit of ID is used to remember relationship between "linked" + * registers. Example: + * r1 = r2; both will have r1->id == r2->id == N + * r1 += 10; r1->id == N | BPF_ADD_CONST and r1->off == 10 + */ +#define BPF_ADD_CONST (1U << 31) u32 id; /* PTR_TO_SOCKET and PTR_TO_TCP_SOCK could be a ptr returned * from a pointer-cast helper, bpf_sk_fullsock() and diff --git a/kernel/bpf/log.c b/kernel/bpf/log.c index 4bd8f17a9f24..3f4ae92e549f 100644 --- a/kernel/bpf/log.c +++ b/kernel/bpf/log.c @@ -708,7 +708,9 @@ static void print_reg_state(struct bpf_verifier_env *env, verbose(env, "%s", btf_type_name(reg->btf, reg->btf_id)); verbose(env, "("); if (reg->id) - verbose_a("id=%d", reg->id); + verbose_a("id=%d", reg->id & ~BPF_ADD_CONST); + if (reg->id & BPF_ADD_CONST) + verbose(env, "%+d", reg->off); if (reg->ref_obj_id) verbose_a("ref_obj_id=%d", reg->ref_obj_id); if (type_is_non_owning_ref(reg->type)) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 20ac9cfd54dd..983bc4608adc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3991,7 +3991,7 @@ static bool idset_contains(struct bpf_idset *s, u32 id) u32 i; for (i = 0; i < s->count; ++i) - if (s->ids[i] == id) + if (s->ids[i] == (id & ~BPF_ADD_CONST)) return true; return false; @@ -4001,7 +4001,7 @@ static int idset_push(struct bpf_idset *s, u32 id) { if (WARN_ON_ONCE(s->count >= ARRAY_SIZE(s->ids))) return -EFAULT; - s->ids[s->count++] = id; + s->ids[s->count++] = id & ~BPF_ADD_CONST; return 0; } @@ -4438,8 +4438,20 @@ static bool __is_pointer_value(bool allow_ptr_leaks, static void assign_scalar_id_before_mov(struct bpf_verifier_env *env, struct bpf_reg_state *src_reg) { - if (src_reg->type == SCALAR_VALUE && !src_reg->id && - !tnum_is_const(src_reg->var_off)) + if (src_reg->type != SCALAR_VALUE) + return; + + if (src_reg->id & BPF_ADD_CONST) { + /* + * The verifier is processing rX = rY insn and + * rY->id has special linked register already. + * Cleared it, since multiple rX += const are not supported. + */ + src_reg->id = 0; + src_reg->off = 0; + } + + if (!src_reg->id && !tnum_is_const(src_reg->var_off)) /* Ensure that src_reg has a valid ID that will be copied to * dst_reg and then will be used by find_equal_scalars() to * propagate min/max range. @@ -14034,6 +14046,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, struct bpf_func_state *state = vstate->frame[vstate->curframe]; struct bpf_reg_state *regs = state->regs, *dst_reg, *src_reg; struct bpf_reg_state *ptr_reg = NULL, off_reg = {0}; + bool alu32 = (BPF_CLASS(insn->code) != BPF_ALU64); u8 opcode = BPF_OP(insn->code); int err; @@ -14056,11 +14069,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, if (dst_reg->type != SCALAR_VALUE) ptr_reg = dst_reg; - else - /* Make sure ID is cleared otherwise dst_reg min/max could be - * incorrectly propagated into other registers by find_equal_scalars() - */ - dst_reg->id = 0; + if (BPF_SRC(insn->code) == BPF_X) { src_reg = ®s[insn->src_reg]; if (src_reg->type != SCALAR_VALUE) { @@ -14124,7 +14133,43 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, verbose(env, "verifier internal error: no src_reg\n"); return -EINVAL; } - return adjust_scalar_min_max_vals(env, insn, dst_reg, *src_reg); + err = adjust_scalar_min_max_vals(env, insn, dst_reg, *src_reg); + if (err) + return err; + /* + * Compilers can generate the code + * r1 = r2 + * r1 += 0x1 + * if r2 < 1000 goto ... + * use r1 in memory access + * So remember constant delta between r2 and r1 and update r1 after + * 'if' condition. + */ + if (env->bpf_capable && BPF_OP(insn->code) == BPF_ADD && + dst_reg->id && is_reg_const(src_reg, alu32)) { + u64 val = reg_const_value(src_reg, alu32); + + if ((dst_reg->id & BPF_ADD_CONST) || + /* prevent overflow in find_equal_scalars() later */ + val > (u32)S32_MAX) { + /* + * If the register already went through rX += val + * we cannot accumulate another val into rx->off. + */ + dst_reg->off = 0; + dst_reg->id = 0; + } else { + dst_reg->id |= BPF_ADD_CONST; + dst_reg->off = val; + } + } else { + /* + * Make sure ID is cleared otherwise dst_reg min/max could be + * incorrectly propagated into other registers by find_equal_scalars() + */ + dst_reg->id = 0; + } + return 0; } /* check validity of 32-bit and 64-bit arithmetic operations */ @@ -15096,12 +15141,36 @@ static bool try_match_pkt_pointers(const struct bpf_insn *insn, static void find_equal_scalars(struct bpf_verifier_state *vstate, struct bpf_reg_state *known_reg) { + struct bpf_reg_state fake_reg; struct bpf_func_state *state; struct bpf_reg_state *reg; bpf_for_each_reg_in_vstate(vstate, state, reg, ({ - if (reg->type == SCALAR_VALUE && reg->id == known_reg->id) + if (reg->type != SCALAR_VALUE || reg == known_reg) + continue; + if ((reg->id & ~BPF_ADD_CONST) != (known_reg->id & ~BPF_ADD_CONST)) + continue; + if ((!(reg->id & BPF_ADD_CONST) && !(known_reg->id & BPF_ADD_CONST)) || + reg->off == known_reg->off) { copy_register_state(reg, known_reg); + } else { + s32 saved_off = reg->off; + + fake_reg.type = SCALAR_VALUE; + __mark_reg_known(&fake_reg, (s32)reg->off - (s32)known_reg->off); + + /* reg = known_reg; reg += delta */ + copy_register_state(reg, known_reg); + /* + * Must preserve off, id and add_const flag, + * otherwise another find_equal_scalars() will be incorrect. + */ + reg->off = saved_off; + + scalar32_min_max_add(reg, &fake_reg); + scalar_min_max_add(reg, &fake_reg); + reg->var_off = tnum_add(reg->var_off, fake_reg.var_off); + } })); } @@ -16730,6 +16799,10 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold, } if (!rold->precise && exact == NOT_EXACT) return true; + if ((rold->id & BPF_ADD_CONST) != (rcur->id & BPF_ADD_CONST)) + return false; + if ((rold->id & BPF_ADD_CONST) && (rold->off != rcur->off)) + return false; /* Why check_ids() for scalar registers? * * Consider the following BPF code: diff --git a/tools/testing/selftests/bpf/verifier/precise.c b/tools/testing/selftests/bpf/verifier/precise.c index 0a9293a57211..90643ccc221d 100644 --- a/tools/testing/selftests/bpf/verifier/precise.c +++ b/tools/testing/selftests/bpf/verifier/precise.c @@ -39,12 +39,12 @@ .result = VERBOSE_ACCEPT, .errstr = "mark_precise: frame0: last_idx 26 first_idx 20\ - mark_precise: frame0: regs=r2 stack= before 25\ - mark_precise: frame0: regs=r2 stack= before 24\ - mark_precise: frame0: regs=r2 stack= before 23\ - mark_precise: frame0: regs=r2 stack= before 22\ - mark_precise: frame0: regs=r2 stack= before 20\ - mark_precise: frame0: parent state regs=r2 stack=:\ + mark_precise: frame0: regs=r2,r9 stack= before 25\ + mark_precise: frame0: regs=r2,r9 stack= before 24\ + mark_precise: frame0: regs=r2,r9 stack= before 23\ + mark_precise: frame0: regs=r2,r9 stack= before 22\ + mark_precise: frame0: regs=r2,r9 stack= before 20\ + mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 19 first_idx 10\ mark_precise: frame0: regs=r2,r9 stack= before 19\ mark_precise: frame0: regs=r9 stack= before 18\ @@ -100,11 +100,11 @@ .errstr = "26: (85) call bpf_probe_read_kernel#113\ mark_precise: frame0: last_idx 26 first_idx 22\ - mark_precise: frame0: regs=r2 stack= before 25\ - mark_precise: frame0: regs=r2 stack= before 24\ - mark_precise: frame0: regs=r2 stack= before 23\ - mark_precise: frame0: regs=r2 stack= before 22\ - mark_precise: frame0: parent state regs=r2 stack=:\ + mark_precise: frame0: regs=r2,r9 stack= before 25\ + mark_precise: frame0: regs=r2,r9 stack= before 24\ + mark_precise: frame0: regs=r2,r9 stack= before 23\ + mark_precise: frame0: regs=r2,r9 stack= before 22\ + mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 20 first_idx 20\ mark_precise: frame0: regs=r2,r9 stack= before 20\ mark_precise: frame0: parent state regs=r2,r9 stack=:\ From patchwork Thu Jun 13 01:38:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13695771 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D1E2A1CF for ; Thu, 13 Jun 2024 01:38:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242714; cv=none; b=W+rWz5fRu3iBHNPoR4UNoQ80AJsW0OqZ9rUa4hhSt9S8OM1dXQANzUYd5saEoBZBqgeubejOckEiOxKhW6OPHH4o1r4Oat4Upbi19z5DpApWtjnlYN6/hjM9BHNr7iz27dcjjySoRFB1QWnmKZFRWTchafxJsrSV5u84s3XjECI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242714; c=relaxed/simple; bh=Gm5gtUs7ESel2//jbT0TijXe29JcPEixmsWoeLbvHjg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Gp1UVNebmaV9yIQI8YbGpvM8MgaXSmPUlYKT0Zt5FwMe9tfGXIi6G2tRfkHUfotIDMeA3aJQLvN6jcLJl1Lmzf3TPUqwz1QZGDkD4PHtf2VhvT4OoUpQORcTzIh7Jc5Y8gkG/zCmuULf8qJAV9Xj/hPk+5vQSXUb+bsl9HqOcJo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=V+xsrkfX; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="V+xsrkfX" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1f44b5b9de6so4351385ad.3 for ; Wed, 12 Jun 2024 18:38:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718242711; x=1718847511; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Us1aqdKDZ5u/C9Le8gUMjYEff67H98STOGSj9ePhdrk=; b=V+xsrkfXYXoTXFGfMegh7xbtQtiznoslh6HC21A9bb0umT3BV79JkTwgp1c3pb9HGS JkN3hhYJZRwetFNThOInA80mRkH4Ty5D/1FinwBki4/UuwUMtm9YWrix/2T6m+wxYkXY XxVZnVP4nTgoJF2FVgXg4YG/6CFKIqXur/z6pGaI0hJG2gNiZNT4oQ4HSPNH4uQvDmT9 NVqjrHIZVFIb+Zaws23/yugEkcdBVo9vN8mZLWGw4KN0sejlUiljJNunLhdAAry+ssQy MhVTQaZQTXNofMks8F97lmNe6a9W9jVXZbijlefFYwaTalnCS8QmRr3oYHwvra5skBFH lEvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718242711; x=1718847511; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Us1aqdKDZ5u/C9Le8gUMjYEff67H98STOGSj9ePhdrk=; b=nqJxq/veGhfzijM5x47QnXh0zKWHbSS1OumYi6z5gIe01i/sQnJ9E++3Ad0wK5+SbK yRpABc21WORwL5acpce+cKo2qQPX7LLHhKME/aokKkHH3gvt3g0PHY3df7ykHDYHnTho rr4GVm62N2zCDNfDbKPU5nb+4k8b+R4aaSGrOLqQWvjUgVnWL1jl6TDSoz/P/2oOTzpt 6B40CjgEIz46zP4zygRQqhFqzLHwTBLk7aOZvl7BFbuq5p0HdpajHNLPonBsON1mNpap hKw344OAfV1pkVulaO/lRja7GRk5xuPr4b2YGjLZ9TGZHlc8NeeORT1DbLgrmxA1acmk RMsg== X-Gm-Message-State: AOJu0Yzz6Zl5GpfEgszmMoHlw1eeeXlLy+u4PvxFiRY+RsoZZ/JWDmEu zDQVBPtGxOBe/gZ+stSYre4rBkZ2nIVBm/BabW9vxkKvcKZlO7ZenmOZLA== X-Google-Smtp-Source: AGHT+IFGYl3IWcCKw/3V5YEGdYNfP0q8xYgH0dQ5VFnUgwUMLiE2J5bX7I8hEcKapsBK1G4Upru6LQ== X-Received: by 2002:a17:902:c944:b0:1f7:2091:978 with SMTP id d9443c01a7336-1f83b60919fmr40866005ad.37.1718242711255; Wed, 12 Jun 2024 18:38:31 -0700 (PDT) Received: from macbook-pro-49.dhcp.thefacebook.com ([2620:10d:c090:400::5:b914]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855f2fe6dsm1287355ad.257.2024.06.12.18.38.30 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 12 Jun 2024 18:38:30 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, memxor@gmail.com, eddyz87@gmail.com, kernel-team@fb.com Subject: [PATCH v3 bpf-next 3/4] bpf: Support can_loop/cond_break on big endian Date: Wed, 12 Jun 2024 18:38:14 -0700 Message-Id: <20240613013815.953-4-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240613013815.953-1-alexei.starovoitov@gmail.com> References: <20240613013815.953-1-alexei.starovoitov@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Add big endian support for can_loop/cond_break macros. Acked-by: Yonghong Song Signed-off-by: Alexei Starovoitov --- .../testing/selftests/bpf/bpf_experimental.h | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 3d9e4b8c6b81..82b73c37b50b 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -351,6 +351,7 @@ l_true: \ l_continue:; \ }) #else +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ #define can_loop \ ({ __label__ l_break, l_continue; \ bool ret = true; \ @@ -376,6 +377,33 @@ l_true: \ l_break: break; \ l_continue:; \ }) +#else +#define can_loop \ + ({ __label__ l_break, l_continue; \ + bool ret = true; \ + asm volatile goto("1:.byte 0xe5; \ + .byte 0; \ + .long (((%l[l_break] - 1b - 8) / 8) & 0xffff) << 16; \ + .short 0" \ + :::: l_break); \ + goto l_continue; \ + l_break: ret = false; \ + l_continue:; \ + ret; \ + }) + +#define cond_break \ + ({ __label__ l_break, l_continue; \ + asm volatile goto("1:.byte 0xe5; \ + .byte 0; \ + .long (((%l[l_break] - 1b - 8) / 8) & 0xffff) << 16; \ + .short 0" \ + :::: l_break); \ + goto l_continue; \ + l_break: break; \ + l_continue:; \ + }) +#endif #endif #ifndef bpf_nop_mov From patchwork Thu Jun 13 01:38:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13695772 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52ADE168D0 for ; Thu, 13 Jun 2024 01:38:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242718; cv=none; b=DFsFa97+dd8wVywzRIzeKQ8941VHRkBObdFxf8/pQ0e+9tqlAM5p3Ezjao1rz5X1NXkchhv08Jk9PWOdSCtyrdV2iU5ryHi9WuIPgaHcfg37FvvWXkHE0HrFBPm3QTXXflyJEqXT417jT1HmTpgid4dEQ3HseRriP3hq+vl7Sog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718242718; c=relaxed/simple; bh=0C7YFIXX4V6L4MugzHRCmje1MAySHz560R7vtuAISy8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CEYEkXZzcAjwHT7rPdPIOWS9Q/T0LltbH4R7kKNPf6m9eHIEgW14K9MYzd+6dIylN+UQMpQfh9cWJhiVGb16l0rGqVYHHxmxz+3szoHxjZDxnXLgTCFs8FqfHw7DXJ4HY1SaTSxqMkzRU7gZIE7c7CtCVd/ewCzpjLvbOiTx514= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bHFiH7JP; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bHFiH7JP" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-705c7e2d31cso424294b3a.3 for ; Wed, 12 Jun 2024 18:38:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718242715; x=1718847515; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9hkdNXD0ldSnRyH//rXl3FD1wqGeU2gjckNjPaQlUqg=; b=bHFiH7JPJaXZNevGE+2f3OQ2OyeXXn3dk+Rh3JwjTrIMTEISAN0sbDqZIpK66MFDmh fXZC8uEmxLzcva0AoDOpjRNjf8LDYEP0S3wxe3i5KLdJSa4M3tfmaozRMMVPU3K20ZiP InW+BQ5qyd1NYJxtK7D7WQbVwfIe6iveuhAIpIJPM+g8HPjbMbfEjtvdoWNPhobfw5wg fmxjxHCi6fKCerW0r4ShFu0tE5OHRmRh3dnT5yt+13Bj4dfblhNfgFaU6ng5Cg7q08M9 oVzTGlkM2sWBFHpoEo9ip8ccC54LR2d5QSmx8MLujHkOh1sf5zWVZyidu5MARzlRlL9m BRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718242715; x=1718847515; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9hkdNXD0ldSnRyH//rXl3FD1wqGeU2gjckNjPaQlUqg=; b=j1PRV2TiahHeXqSrajjmxBCBzqaYEblmS8hH2ByFqQSbGylURY6uR16s/44EdqjcyO onjvGD0PFLoDKApIdZfXxUB7W+mHu2rz0+Ag14ft81YF6bIo7Rs98xilw8lDKyjTybGt w6W/CYxLsjX3v4RiYzikF6nGVrSfv+oKnGHPWlL1DnE3k07giycAB9Q8SOTN3t48P7Yu inMvklbXQtE8VAHI1/fu765+5tbpER70MKHc81l2zvVj0HSVOXnkM8xsjRIs963c4FQl 4ERA56Pkd6fHSZ7Dql9zx9i0RgTA8cU5urC+E7gOmPTzIWadOJGKmEo5K9vIwLKhR09i IF9A== X-Gm-Message-State: AOJu0YxiqlPx+RyMlVXcroPNRxZcZvs28wEblm0Y5vmDvjcYHxAWFYy7 /yUhC/RyhPcXBXc4rsZEqLfKdL1MMWTQlsSfvO8R3yjvzghxbci2FzbAoQ== X-Google-Smtp-Source: AGHT+IGVJ/g4W+OaG1pLnTH0oRjnOys18OTCN3HlFCUMBRwPKUcOoUDNtRVPkPtkmDhl5OYCJSpAPA== X-Received: by 2002:a05:6a00:2347:b0:703:ed39:fbb7 with SMTP id d2e1a72fcca58-705bcdf4708mr4158633b3a.5.1718242714946; Wed, 12 Jun 2024 18:38:34 -0700 (PDT) Received: from macbook-pro-49.dhcp.thefacebook.com ([2620:10d:c090:400::5:b914]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc91e154sm217999b3a.44.2024.06.12.18.38.33 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 12 Jun 2024 18:38:34 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, memxor@gmail.com, eddyz87@gmail.com, kernel-team@fb.com Subject: [PATCH v3 bpf-next 4/4] selftests/bpf: Add tests for add_const Date: Wed, 12 Jun 2024 18:38:15 -0700 Message-Id: <20240613013815.953-5-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240613013815.953-1-alexei.starovoitov@gmail.com> References: <20240613013815.953-1-alexei.starovoitov@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Improve arena based tests and add several C and asm tests with specific pattern. These tests would have failed without add_const verifier support. Also add several loop_inside_iter*() tests that are not related to add_const, but nice to have. Signed-off-by: Alexei Starovoitov --- .../testing/selftests/bpf/progs/arena_htab.c | 16 +- .../bpf/progs/verifier_iterating_callbacks.c | 236 ++++++++++++++++++ 2 files changed, 249 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/arena_htab.c b/tools/testing/selftests/bpf/progs/arena_htab.c index 1e6ac187a6a0..cd598348725e 100644 --- a/tools/testing/selftests/bpf/progs/arena_htab.c +++ b/tools/testing/selftests/bpf/progs/arena_htab.c @@ -18,25 +18,35 @@ void __arena *htab_for_user; bool skip = false; int zero = 0; +char __arena arr1[100000]; +char arr2[1000]; SEC("syscall") int arena_htab_llvm(void *ctx) { #if defined(__BPF_FEATURE_ADDR_SPACE_CAST) || defined(BPF_ARENA_FORCE_ASM) struct htab __arena *htab; + char __arena *arr = arr1; __u64 i; htab = bpf_alloc(sizeof(*htab)); cast_kern(htab); htab_init(htab); + cast_kern(arr); + /* first run. No old elems in the table */ - for (i = zero; i < 1000; i++) + for (i = zero; i < 100000 && can_loop; i++) { htab_update_elem(htab, i, i); + arr[i] = i; + } - /* should replace all elems with new ones */ - for (i = zero; i < 1000; i++) + /* should replace some elems with new ones */ + for (i = zero; i < 1000 && can_loop; i++) { htab_update_elem(htab, i, i); + /* Access mem to make the verifier use bounded loop logic */ + arr2[i] = i; + } cast_user(htab); htab_for_user = htab; #else diff --git a/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c b/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c index bd676d7e615f..53679252e8a1 100644 --- a/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c +++ b/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c @@ -405,4 +405,240 @@ int cond_break5(const void *ctx) return cnt1 > 1 && cnt2 > 1 ? 1 : 0; } +#define ARR2_SZ 1000 +SEC(".data.arr2") +char arr2[ARR2_SZ]; + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + __u64 i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < ARR2_SZ) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter_signed(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + long i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < ARR2_SZ && i >= 0) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +volatile const int limit = ARR2_SZ; + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter_volatile_limit(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + __u64 i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < limit) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +#define ARR_LONG_SZ 1000 + +SEC(".data.arr_long") +long arr_long[ARR_LONG_SZ]; + +SEC("socket") +__success +int test1(const void *ctx) +{ + long i; + + for (i = 0; i < ARR_LONG_SZ && can_loop; i++) + arr_long[i] = i; + return 0; +} + +SEC("socket") +__success +int test2(const void *ctx) +{ + __u64 i; + + for (i = zero; i < ARR_LONG_SZ && can_loop; i++) { + barrier_var(i); + arr_long[i] = i; + } + return 0; +} + +SEC(".data.arr_foo") +struct { + int a; + int b; +} arr_foo[ARR_LONG_SZ]; + +SEC("socket") +__success +int test3(const void *ctx) +{ + __u64 i; + + for (i = zero; i < ARR_LONG_SZ && can_loop; i++) { + barrier_var(i); + arr_foo[i].a = i; + arr_foo[i].b = i; + } + return 0; +} + +SEC("socket") +__success +int test4(const void *ctx) +{ + long i; + + for (i = zero + ARR_LONG_SZ - 1; i < ARR_LONG_SZ && i >= 0 && can_loop; i--) { + barrier_var(i); + arr_foo[i].a = i; + arr_foo[i].b = i; + } + return 0; +} + +char buf[10] SEC(".data.buf"); + +SEC("socket") +__description("check add const") +__success +__naked void check_add_const(void) +{ + /* typical LLVM generated loop with may_goto */ + asm volatile (" \ + call %[bpf_ktime_get_ns]; \ + if r0 > 9 goto l1_%=; \ +l0_%=: r1 = %[buf]; \ + r2 = r0; \ + r1 += r2; \ + r3 = *(u8 *)(r1 +0); \ + .byte 0xe5; /* may_goto */ \ + .byte 0; /* regs */ \ + .short 4; /* off of l1_%=: */ \ + .long 0; /* imm */ \ + r0 = r2; \ + r0 += 1; \ + if r2 < 9 goto l0_%=; \ + exit; \ +l1_%=: r0 = 0; \ + exit; \ +" : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__msg("*(u8 *)(r7 +0) = r0") +__msg("invalid access to map value, value_size=10 off=10 size=1") +__naked void check_add_const_3regs(void) +{ + asm volatile ( + "r6 = %[buf];" + "r7 = %[buf];" + "call %[bpf_ktime_get_ns];" + "r1 = r0;" /* link r0.id == r1.id == r2.id */ + "r2 = r0;" + "r1 += 1;" /* r1 == r0+1 */ + "r2 += 2;" /* r2 == r0+2 */ + "if r0 > 8 goto 1f;" /* r0 range [0, 8] */ + "r6 += r1;" /* r1 range [1, 9] */ + "r7 += r2;" /* r2 range [2, 10] */ + "*(u8 *)(r6 +0) = r0;" /* safe, within bounds */ + "*(u8 *)(r7 +0) = r0;" /* unsafe, out of bounds */ + "1: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__msg("*(u8 *)(r8 -1) = r0") +__msg("invalid access to map value, value_size=10 off=10 size=1") +__naked void check_add_const_3regs_2if(void) +{ + asm volatile ( + "r6 = %[buf];" + "r7 = %[buf];" + "r8 = %[buf];" + "call %[bpf_ktime_get_ns];" + "if r0 < 2 goto 1f;" + "r1 = r0;" /* link r0.id == r1.id == r2.id */ + "r2 = r0;" + "r1 += 1;" /* r1 == r0+1 */ + "r2 += 2;" /* r2 == r0+2 */ + "if r2 > 11 goto 1f;" /* r2 range [0, 11] -> r0 range [-2, 9]; r1 range [-1, 10] */ + "if r0 s< 0 goto 1f;" /* r0 range [0, 9] -> r1 range [1, 10]; r2 range [2, 11]; */ + "r6 += r0;" /* r0 range [0, 9] */ + "r7 += r1;" /* r1 range [1, 10] */ + "r8 += r2;" /* r2 range [2, 11] */ + "*(u8 *)(r6 +0) = r0;" /* safe, within bounds */ + "*(u8 *)(r7 -1) = r0;" /* safe */ + "*(u8 *)(r8 -1) = r0;" /* unsafe */ + "1: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__flag(BPF_F_TEST_STATE_FREQ) +__naked void check_add_const_regsafe_off(void) +{ + asm volatile ( + "r8 = %[buf];" + "call %[bpf_ktime_get_ns];" + "r6 = r0;" + "call %[bpf_ktime_get_ns];" + "r7 = r0;" + "call %[bpf_ktime_get_ns];" + "r1 = r0;" /* same ids for r1 and r0 */ + "if r6 > r7 goto 1f;" /* this jump can't be predicted */ + "r1 += 1;" /* r1.off == +1 */ + "goto 2f;" + "1: r1 += 100;" /* r1.off == +100 */ + "goto +0;" /* verify r1.off in regsafe() after this insn */ + "2: if r0 > 8 goto 3f;" /* r0 range [0,8], r1 range either [1,9] or [100,108]*/ + "r8 += r1;" + "*(u8 *)(r8 +0) = r0;" /* potentially unsafe, buf size is 10 */ + "3: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + char _license[] SEC("license") = "GPL";