From patchwork Sat Jan 27 17:52:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534218 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 312312D600; Sat, 27 Jan 2024 17:52:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377978; cv=none; b=uOs5zZ+dJxCGPKj6jmS6LibNmeUZguL2czKT7/S9TE+UPIZ5yvt9dXlb+m+tm6qXhW0jAu19T9Zd1UzrvWr7g6CuU+Y3IzVYd2ueWpLu6oNafzVcyrXvZgODfCisS2XLYh7FlmSZhbw7Pvmt33C5r+cgYmhih1uh6/4WeChyToc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377978; c=relaxed/simple; bh=VAiI/hRTYsl5Wq3jPFieFFhFWOTUmvZ8ipOqR5Ko1Gw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lvUWxVCkTbQlQZlj5/HTIdMJBGMP0ARzUUAtOQ7jmsjsZ8JYX91aqLtMyOO8dvun61A7VU78HmPNYRFr7DwHiaMK6cJqizP38p3bEO3+mSdZISoNG7oUGMre1Xw8eveoHbw8T1wNN/vYE9Eid+hEMwb/S6HDBZr3hW/SpA7Nu+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MUgCB3to; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MUgCB3to" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-40ef0643722so3485635e9.3; Sat, 27 Jan 2024 09:52:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377975; x=1706982775; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=140WdK2PKIttOzK0HYik4RpplzTQ3zPqG6HYXa/6NHA=; b=MUgCB3to8TTtMTqdvNPsW7gOlgBRCmBYcNn8YdvI2Cv9oEHFiBT7HKF63pC5SvUuzA 7fvYz9SXwNbmhzTldJtp014Vs8+Oz4hF0bPiPq7ZDSX8Qn0+cXbVdZGALONgEklCdyOS qttLpMRIMT1waWjy7KZ/f13toL1x10NQwVcRBQPfPDFoSpa7t/x9KRvBnNSSRV2COZhr yDBJr/mBVZXQ8PtSFe8X0HxQuL18NlzMs5LLxvhXCegP1TUFOtgygbY2yL3IaIDYUxFh SjsaRfDAjWwmmofDu5N2204VoiWM5gj2ddiXyj9XEZNebbGz4gVPjcrZLEo0riW4UKBq 2qOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377975; x=1706982775; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=140WdK2PKIttOzK0HYik4RpplzTQ3zPqG6HYXa/6NHA=; b=BBZifjBS65gstCElx2df2YFsG8YNgolAm4IjPcEJDu3TsEle297S1p2S0sdYhUzSiH lkiIzFFyhSdEJ7Br1eBYZ0PdCqEM+XZZEWneRgmZqmVUGVB6jexzmkgwl9Pcpcx17ABm mZxD4itBcCfo5Fkqbpgl2AMXVqmi6H8tuAl5edAjODQrdFOSnk0KWmzJtN02fycJg/7f 9cvKukeoxO2/rxhLPuAz7SHu+FbTdnqOikV7BEm/Ng0h6f4kqGyGaob/m+sYx1k1ZDTd wWfk1Ztg6QcRt7HyI3J59nWWfRQdbtXo78vxZ/w05B1Kd3zSMQSzNzq1ipQLLsYYb6ZG /0OA== X-Gm-Message-State: AOJu0YzvAvs/DwALFx4CmCc44h0DR828J2h2zkVznBR8BpUdwyz+2hIO 3Dnpr40jYiAiiCiVJ7qHg2B/L5CKLJY8xfTN77PCtA3HOI0DVMWy X-Google-Smtp-Source: AGHT+IGIROIv3bVw5y84GVGgIcSolBbb5CdoA23ygrQqxfrk56y86APtbiL1QNuKZjDtgk4FbvbgFw== X-Received: by 2002:adf:e511:0:b0:33a:e75a:6aea with SMTP id j17-20020adfe511000000b0033ae75a6aeamr274965wrm.73.1706377975132; Sat, 27 Jan 2024 09:52:55 -0800 (PST) Received: from localhost ([185.220.101.170]) by smtp.gmail.com with ESMTPSA id l1-20020a50cbc1000000b00557aa8d72c9sm1801444edi.25.2024.01.27.09.52.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:52:54 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org, Maxim Mikityanskiy Subject: [PATCH bpf-next v3 1/6] bpf: Track spilled unbounded scalars Date: Sat, 27 Jan 2024 19:52:32 +0200 Message-ID: <20240127175237.526726-2-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Maxim Mikityanskiy Support the pattern where an unbounded scalar is spilled to the stack, then boundary checks are performed on the src register, after which the stack frame slot is refilled into a register. Before this commit, the verifier didn't treat the src register and the stack slot as related if the src register was an unbounded scalar. The register state wasn't copied, the id wasn't preserved, and the stack slot was marked as STACK_MISC. Subsequent boundary checks on the src register wouldn't result in updating the boundaries of the spilled variable on the stack. After this commit, the verifier will preserve the bond between src and dst even if src is unbounded, which permits to do boundary checks on src and refill dst later, still remembering its boundaries. Such a pattern is sometimes generated by clang when compiling complex long functions. One test is adjusted to reflect that now unbounded scalars are tracked. Signed-off-by: Maxim Mikityanskiy Acked-by: Eduard Zingerman --- kernel/bpf/verifier.c | 16 +--------------- .../selftests/bpf/progs/verifier_spill_fill.c | 6 +++--- 2 files changed, 4 insertions(+), 18 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c5d68a9d8acc..f4b9fd0006d1 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4380,20 +4380,6 @@ static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32) return subreg32 ? tnum_subreg(reg->var_off).value : reg->var_off.value; } -static bool __is_scalar_unbounded(struct bpf_reg_state *reg) -{ - return tnum_is_unknown(reg->var_off) && - reg->smin_value == S64_MIN && reg->smax_value == S64_MAX && - reg->umin_value == 0 && reg->umax_value == U64_MAX && - reg->s32_min_value == S32_MIN && reg->s32_max_value == S32_MAX && - reg->u32_min_value == 0 && reg->u32_max_value == U32_MAX; -} - -static bool register_is_bounded(struct bpf_reg_state *reg) -{ - return reg->type == SCALAR_VALUE && !__is_scalar_unbounded(reg); -} - static bool __is_pointer_value(bool allow_ptr_leaks, const struct bpf_reg_state *reg) { @@ -4504,7 +4490,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env, return err; mark_stack_slot_scratched(env, spi); - if (reg && !(off % BPF_REG_SIZE) && register_is_bounded(reg) && env->bpf_capable) { + if (reg && !(off % BPF_REG_SIZE) && reg->type == SCALAR_VALUE && env->bpf_capable) { bool reg_value_fits; reg_value_fits = get_reg_width(reg) <= BITS_PER_BYTE * size; diff --git a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c index 7013a9694163..317806451762 100644 --- a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c +++ b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c @@ -452,9 +452,9 @@ l0_%=: r1 >>= 16; \ SEC("raw_tp") __log_level(2) __success -__msg("fp-8=0m??mmmm") -__msg("fp-16=00mm??mm") -__msg("fp-24=00mm???m") +__msg("fp-8=0m??scalar()") +__msg("fp-16=00mm??scalar()") +__msg("fp-24=00mm???scalar()") __naked void spill_subregs_preserve_stack_zero(void) { asm volatile ( From patchwork Sat Jan 27 17:52:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534219 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99BA32D600; Sat, 27 Jan 2024 17:53:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377982; cv=none; b=Bi+rd2yghQkjEuYJY31TxeT8HffgjQadArYKbnX3csU5qZglW+P7TCCmLy+6xDpDaT/u+w1spxqBGoH/awm3WoM8r+5k1lkyROOPtk/rL2gove4wAbliyoWaOlAe0tZi4IGJpT7AKfyZWBjT9M9ex5fKdrafznCUzIeFWmj1nPg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377982; c=relaxed/simple; bh=JGo/mjFx+TLftCJ2ezCKct2I+QubavYYCAKzoptDZiI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pw0Fkpox1w9kLpI9vBGMpZzVTrEcC3IRQeSxRO75wiBG0G692+tKfs6VKYcTOwmLd96YipxjvehsNGrnAZypvEy61FKWSCMvu4hZ2Foadf1aKWsjShQi3bcrRw2d3cQsg8aZ1Omn9g+QK8Ow8E3dQb0dKjaDglycEKIH+3nQXp8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mANLIUib; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mANLIUib" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a35385da5bbso81903866b.3; Sat, 27 Jan 2024 09:53:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377979; x=1706982779; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7MI9xHIKdb/2hEcLbW3yAAuFkUxSisIY5fJfgmqAUs4=; b=mANLIUibW7i1jBMS73iEtbXa5zicER8a/+RQYk+vQ3Sq0fuDZ4K9WF+fm1sSfJlXPL y+MM4Tp8BgAsSU8u6p82Tiq1gLFF6IySaaOWsk5hQiXHC3eFT7sRV3vLktbBA0+P0gpr HGCoGTEmDUA8qT/l5q6pPwtWIubBdXD9FkVyeod+Jcj/1g19feOOm3/EouNrS6yompOj XyceVD4p3asQ3oWSsMdXoNN+AKzaQ5uaBzmU8qeYGrGg3cTKwVz8rTaAH01n0ZQZ7PVO I968NumTR/JD+OCH4CdY9cv+BdsnBgHoQ3jz5YcDcUwCf7zKrqvrivIlSeWBufHoCozz tsrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377979; x=1706982779; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7MI9xHIKdb/2hEcLbW3yAAuFkUxSisIY5fJfgmqAUs4=; b=B9EvDm+hmekf7GileRaUdwdwOt2zx1X4rnxwkdMJO6pzpOslbjWw+h5gm6iMZw1kvQ oQUEBRMQJWFIicyHbNByFVF7OnKHJvSPNVn285KVq9ookVYsbFgeBl8cZFke6uRvV9xg Gy4aj+ySDUHc18hHUDrIqXEGMyyCwpnUOwPHoGZGkF1tOljuBenZbW2DcbgZEQl6oiuF TMlLJ8kpwvSKbCcTKZ6Ogo1aS4XwXBvbXs7jSWDGcXaNYE8Q5Y/rHBGoppRe33Q4K5E/ 9Zy5nnkFlLknbljNnmBbC46wkKjpGvbdRdf+NpklG45wa+HsMuT6KUiOT/VXJE2DC7DF 8MTA== X-Gm-Message-State: AOJu0Yx5Y/AEvRNFwXs/lHqzRrcZ11uSQP52XWdxXTbKRDVL/629qvTi 4ZRKvazKHIvXPJDqWTThcyXPRfAVkLrZIBI5iaGt4Y8C4Qgu6iAB X-Google-Smtp-Source: AGHT+IGyVthHiGfPiNW/2LIzp6rHrBafev1u7RUi5gVhSs3gwx+oZRs2/tflkGDfEU/1Xnd/LTseqQ== X-Received: by 2002:a17:906:451:b0:a27:c4d2:c649 with SMTP id e17-20020a170906045100b00a27c4d2c649mr1355554eja.59.1706377978529; Sat, 27 Jan 2024 09:52:58 -0800 (PST) Received: from localhost ([185.220.101.170]) by smtp.gmail.com with ESMTPSA id s12-20020a17090699cc00b00a2cd74b743csm1960325ejn.3.2024.01.27.09.52.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:52:58 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org, Maxim Mikityanskiy Subject: [PATCH bpf-next v3 2/6] selftests/bpf: Test tracking spilled unbounded scalars Date: Sat, 27 Jan 2024 19:52:33 +0200 Message-ID: <20240127175237.526726-3-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Maxim Mikityanskiy The previous commit added tracking for unbounded scalars on spill. Add the test case to check the new functionality. Signed-off-by: Maxim Mikityanskiy Acked-by: Eduard Zingerman --- .../selftests/bpf/progs/verifier_spill_fill.c | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c index 317806451762..f9803005e1c0 100644 --- a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c +++ b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c @@ -940,4 +940,31 @@ l0_%=: r0 = 0; \ : __clobber_all); } +SEC("xdp") +__description("spill unbounded reg, then range check src") +__success __retval(0) +__naked void spill_unbounded(void) +{ + asm volatile (" \ + /* Produce an unbounded scalar. */ \ + call %[bpf_get_prandom_u32]; \ + /* Spill r0 to stack. */ \ + *(u64*)(r10 - 8) = r0; \ + /* Boundary check on r0. */ \ + if r0 > 16 goto l0_%=; \ + /* Fill r0 from stack. */ \ + r0 = *(u64*)(r10 - 8); \ + /* Boundary check on r0 with predetermined result. */\ + if r0 <= 16 goto l0_%=; \ + /* Dead branch: the verifier should prune it. Do an invalid memory\ + * access if the verifier follows it. \ + */ \ + r0 = *(u64*)(r9 + 0); \ +l0_%=: r0 = 0; \ + exit; \ +" : + : __imm(bpf_get_prandom_u32) + : __clobber_all); +} + char _license[] SEC("license") = "GPL"; From patchwork Sat Jan 27 17:52:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534220 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 080532D600; Sat, 27 Jan 2024 17:53:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377985; cv=none; b=JdmgiqD+nDSUCPURHcrTkzQYV3I7zvKrV1+HLtUVTDOL1uvdpV0T7zuCzYMJ1UuJqhyUwUUAGlZmffFRfsySg4yTJ55MLMyI3BcjLt2zwh/ITDVrcmnrMdVhuuPx677QhPoXbXr5cU3jyX7/1qP59QhcU9BTCajnuSyuRNY0hYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377985; c=relaxed/simple; bh=tCrlPxhS+T295vFqO3J48QdUkPVcQ+VfKAzqeN3uN0M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=E2eG6emKyt0VMNIzC/HsUyZWx6kOPgB2gLBny5HRwwGwlxOqWalRx4f5yFQ1ywFRHnZF08XQbbKaLZVMFP56kCj3MHbmtoyWcjzoq+vNEt4UGf6FMYPKlMDWkMZFzKJoB/HRSoBdaOlUiTU/NxK+VfQJYFqfMWcBNDX2yG3okQs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EyScnCXP; arc=none smtp.client-ip=209.85.208.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EyScnCXP" Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-556c3f0d6c5so971224a12.2; Sat, 27 Jan 2024 09:53:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377982; x=1706982782; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xjKlkVBdyalMJl+fAj4IHpaZGfCfC+zDeIDRIDfHkAM=; b=EyScnCXPOYvgwoHuiCPKp+11uUJUIAMtC910rs+BeaZmi0PVXSk+EUe/Z0n0Fl403p ethgNMh5Lg4aPwsEDV6AzMVji2lYD4Z17LMjc4QIW41tmucdh5dCghQuxoQMoTwrKjy9 bBdLa/Y4Y/4NynP6sHmKp77brzbp457YHz3Wvia/Y3ZpJMSonYbXRogbMQ+V2J17Qhav T2z7dwY6yIYt7VFlqeJGz5Jz5EVBMlcCV1blyaZBSt67vlcMgn0Nke/KThF7GAotgrW0 SUR0b9TPkbaaSg7IfYbiU8BcW/xyaGMd8Yn4UBd7AWG/9E+hfGZZKB0GWKsp94VoJRPV cc3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377982; x=1706982782; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xjKlkVBdyalMJl+fAj4IHpaZGfCfC+zDeIDRIDfHkAM=; b=ZFeVDXM+bn3K/FIOjbLy4T3q760lb5MQwwL12dVY+h2gUjaOob7cRTvqlm/EHhKnpW zJPawMvCypIw3cIzl9EnO7CzU0ia7dz3uTjXjIKzmNZyfpI4tzcY5zgn/9h9EXecpzd2 Snw7GxDJaGT3Ag2i2lFPzzf7V55ukBnQivRICJCIKduwc1rzaBNyv4j/I9cWA+1vFgpu itCm4LJaVHopl31ituCCBIe4JsvTkhW0YO/dj/E9oEZ7o1f2DvukYa465wiVY72ShNnh j2Vhgf2P12Nzbj7OPtGYOCCtIxfIQOvjfB46Y6Fwbi3MC+SNoU3td8pBKchh5byDyBOf V3uw== X-Gm-Message-State: AOJu0Yy5/s7Dd81Ix6ShdO56+RQB/TePpv8BFBNWfkjtgTC2L8W+TN4H h5L1+TB9yfdmf9+1r5VESFOUKOC3ILtcfQwdB1FgQ20hMi7ATxpi X-Google-Smtp-Source: AGHT+IGaxqP7sp/rWBwKiujJl8zWECq4wDfkan4iJ1aJVv0+iMoH6HIMjU2DFggA86BuvNzLipV7KQ== X-Received: by 2002:a05:6402:b2a:b0:55d:151:ebf2 with SMTP id bo10-20020a0564020b2a00b0055d0151ebf2mr979224edb.20.1706377982068; Sat, 27 Jan 2024 09:53:02 -0800 (PST) Received: from localhost ([185.220.101.170]) by smtp.gmail.com with ESMTPSA id e10-20020a50ec8a000000b0055c643f4f8asm1852147edr.32.2024.01.27.09.53.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:53:01 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org, Maxim Mikityanskiy Subject: [PATCH bpf-next v3 3/6] bpf: Preserve boundaries and track scalars on narrowing fill Date: Sat, 27 Jan 2024 19:52:34 +0200 Message-ID: <20240127175237.526726-4-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Maxim Mikityanskiy When the width of a fill is smaller than the width of the preceding spill, the information about scalar boundaries can still be preserved, as long as it's coerced to the right width (done by coerce_reg_to_size). Even further, if the actual value fits into the fill width, the ID can be preserved as well for further tracking of equal scalars. Implement the above improvements, which makes narrowing fills behave the same as narrowing spills and MOVs between registers. Two tests are adjusted to accommodate for endianness differences and to take into account that it's now allowed to do a narrowing fill from the least significant bits. reg_bounds_sync is added to coerce_reg_to_size to correctly adjust umin/umax boundaries after the var_off truncation, for example, a 64-bit value 0xXXXXXXXX00000000, when read as a 32-bit, gets umin = 0, umax = 0xFFFFFFFF, var_off = (0x0; 0xffffffff00000000), which needs to be synced down to umax = 0, otherwise reg_bounds_sanity_check doesn't pass. Signed-off-by: Maxim Mikityanskiy --- include/linux/bpf_verifier.h | 9 +++++++ kernel/bpf/verifier.c | 15 ++++++++--- .../selftests/bpf/progs/verifier_spill_fill.c | 26 ++++++++++++++----- 3 files changed, 39 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 7f5816482a10..897be9dd6f12 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -918,6 +918,15 @@ static inline void mark_verifier_state_scratched(struct bpf_verifier_env *env) env->scratched_stack_slots = ~0ULL; } +static inline bool bpf_stack_narrow_access_ok(int off, int fill_size, int spill_size) +{ +#ifdef __BIG_ENDIAN + off -= spill_size - fill_size; +#endif + + return !(off % BPF_REG_SIZE); +} + const char *reg_type_str(struct bpf_verifier_env *env, enum bpf_reg_type type); const char *dynptr_type_str(enum bpf_dynptr_type type); const char *iter_type_str(const struct btf *btf, u32 btf_id); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f4b9fd0006d1..81b08a0e9e2c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4778,7 +4778,8 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env, if (dst_regno < 0) return 0; - if (!(off % BPF_REG_SIZE) && size == spill_size) { + if (size <= spill_size && + bpf_stack_narrow_access_ok(off, size, spill_size)) { /* The earlier check_reg_arg() has decided the * subreg_def for this insn. Save it first. */ @@ -4786,6 +4787,12 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env, copy_register_state(&state->regs[dst_regno], reg); state->regs[dst_regno].subreg_def = subreg_def; + + /* Break the relation on a narrowing fill. + * coerce_reg_to_size will adjust the boundaries. + */ + if (get_reg_width(reg) > size * BITS_PER_BYTE) + state->regs[dst_regno].id = 0; } else { int spill_cnt = 0, zero_cnt = 0; @@ -6061,10 +6068,10 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size) * values are also truncated so we push 64-bit bounds into * 32-bit bounds. Above were truncated < 32-bits already. */ - if (size < 4) { + if (size < 4) __mark_reg32_unbounded(reg); - reg_bounds_sync(reg); - } + + reg_bounds_sync(reg); } static void set_sext64_default_val(struct bpf_reg_state *reg, int size) diff --git a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c index f9803005e1c0..3e5d063ea7e8 100644 --- a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c +++ b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c @@ -217,7 +217,7 @@ __naked void uninit_u32_from_the_stack(void) SEC("tc") __description("Spill a u32 const scalar. Refill as u16. Offset to skb->data") -__failure __msg("invalid access to packet") +__success __retval(0) __naked void u16_offset_to_skb_data(void) { asm volatile (" \ @@ -225,13 +225,19 @@ __naked void u16_offset_to_skb_data(void) r3 = *(u32*)(r1 + %[__sk_buff_data_end]); \ w4 = 20; \ *(u32*)(r10 - 8) = r4; \ - r4 = *(u16*)(r10 - 8); \ + " +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + "r4 = *(u16*)(r10 - 8);" +#else + "r4 = *(u16*)(r10 - 6);" +#endif + " \ r0 = r2; \ - /* r0 += r4 R0=pkt R2=pkt R3=pkt_end R4=umax=65535 */\ + /* r0 += r4 R0=pkt R2=pkt R3=pkt_end R4=20 */\ r0 += r4; \ - /* if (r0 > r3) R0=pkt,umax=65535 R2=pkt R3=pkt_end R4=umax=65535 */\ + /* if (r0 > r3) R0=pkt,off=20 R2=pkt R3=pkt_end R4=20 */\ if r0 > r3 goto l0_%=; \ - /* r0 = *(u32 *)r2 R0=pkt,umax=65535 R2=pkt R3=pkt_end R4=20 */\ + /* r0 = *(u32 *)r2 R0=pkt,off=20 R2=pkt R3=pkt_end R4=20 */\ r0 = *(u32*)(r2 + 0); \ l0_%=: r0 = 0; \ exit; \ @@ -268,7 +274,7 @@ l0_%=: r0 = 0; \ } SEC("tc") -__description("Spill a u32 const scalar. Refill as u16 from fp-6. Offset to skb->data") +__description("Spill a u32 const scalar. Refill as u16 from MSB. Offset to skb->data") __failure __msg("invalid access to packet") __naked void _6_offset_to_skb_data(void) { @@ -277,7 +283,13 @@ __naked void _6_offset_to_skb_data(void) r3 = *(u32*)(r1 + %[__sk_buff_data_end]); \ w4 = 20; \ *(u32*)(r10 - 8) = r4; \ - r4 = *(u16*)(r10 - 6); \ + " +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + "r4 = *(u16*)(r10 - 6);" +#else + "r4 = *(u16*)(r10 - 8);" +#endif + " \ r0 = r2; \ /* r0 += r4 R0=pkt R2=pkt R3=pkt_end R4=umax=65535 */\ r0 += r4; \ From patchwork Sat Jan 27 17:52:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534221 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDFF02D604; Sat, 27 Jan 2024 17:53:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377992; cv=none; b=m/C6a8spubuDj2LNRt+9hJRfBm0JjMemNv1FMbsAjtlblAQbm3W8OUPLiq0IXaSp6krDwa5/riLzwp1AvZ7C4GwnVN7qpCb1qCpHg9SEZ5NZD75/B2xbnrEK8mvDISzshItpjW3aZd3YC6n0WQ30ALeInONPIHKqEeJ09jxJcjM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377992; c=relaxed/simple; bh=VfWPiZ1vRW6H3hWfF649onKg9BA90vIGQzyb70j3pPM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=phVKuevtv26Y6XaiF921MzfGtxz+AYcw0ZFqMNaLCLDEXvfYquIxJP8mxtjbq4948IzyDt9c42Jh4VuM81mmEM0FqJ1tAHwbVy5B6bFf12UPLRXNjd5JBvmZrwN628EEcK/2i8AQDTIUhQRjsqC8jnNaFESlg98vdXK+ECB5hFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dZVRHHgh; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dZVRHHgh" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-40e5afc18f5so21055775e9.3; Sat, 27 Jan 2024 09:53:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377989; x=1706982789; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=szKrLwGLdzZiKf2ZM55P0AxDZ1c6ft6AGkc0dEme8Fw=; b=dZVRHHghITzSYQfhGL7+df6GFZJJjM5PPs2QJyMH6kHCM2+i5YO6g5HvPS+8X2S7MI LCzhPkM334gzJlxNDwSG4kX3/ivyG+2T01NM6tef6tO/6Um5Tn0O8FMn10G0wqpMne0p MJ2U8GptRLwZFHbQvf7XYAmUQBVnOa1M7ut32csBjI/IIsRPWSqnvF8V7Lw/yV3jovVb 6eGlyyxtbzEzvhEV0tdWWfgTZv7HUoloqSWuSazeH7qWmFjSNRc7sID6euzKVXwPVsQs a6pV3ccgBqLw35l9bxnwnfjfDhZ+1giO5eN03vO1I1sW74UUkZ9miihvAwl058DtKpK4 nJnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377989; x=1706982789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=szKrLwGLdzZiKf2ZM55P0AxDZ1c6ft6AGkc0dEme8Fw=; b=tQLAFVucO1K3LCwJEUcJbnu29ye72VAeKILBZN2SiV20GLiKdoxztUVtBK09RbQoTD tTAuCcLOiidKhFKkCeUyCQRu4wb+lwsLc7gcCcunk2MRYYCbbTQzDhnkvBqNQd6ScGlw RhIQYmrXuLflyUPZSKuP7JrH8wPSC2Xm8p2j8RtI8YsqnY+TSUeOpAKGBw4RVX2FwqeN GRFyKrUW+iFGUzAS6muQkf/nz+Pb2MBYNe+VJYMQndwpQQxF3D6ZnoK4Gv8Hgx9ykHOf EYjKQIfSFXGdCBeA21PRc8n4r5o1fF6UVkvQ8KMxg4nsJW49iTP2gMbzjAKuuZVougwO jLcg== X-Gm-Message-State: AOJu0Yzp61yiVTNPkB9YQrrGU2MgksdqRDuN07jruR3HPGEqmVqQAxZL ndFxfZy8meRRXtBWegX7OCI+0zLnQqhmcpzKH7hmGNiyQiekeD7B X-Google-Smtp-Source: AGHT+IFQkR6le9DuoGJyY0i1r5xR5M4gVp5kXBnw6HCIJujSrmh2WLnMm3dgc+giJwlriNU8Cv4WEg== X-Received: by 2002:adf:d1c4:0:b0:33a:e6b3:a924 with SMTP id b4-20020adfd1c4000000b0033ae6b3a924mr733982wrd.130.1706377988851; Sat, 27 Jan 2024 09:53:08 -0800 (PST) Received: from localhost ([185.220.101.170]) by smtp.gmail.com with ESMTPSA id se27-20020a170906ce5b00b00a349318ea10sm1954018ejb.199.2024.01.27.09.53.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:53:08 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org, Maxim Mikityanskiy Subject: [PATCH bpf-next v3 4/6] selftests/bpf: Add test cases for narrowing fill Date: Sat, 27 Jan 2024 19:52:35 +0200 Message-ID: <20240127175237.526726-5-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Maxim Mikityanskiy The previous commit allowed to preserve boundaries and track IDs of scalars on narrowing fills. Add test cases for that pattern. Signed-off-by: Maxim Mikityanskiy Acked-by: Eduard Zingerman --- .../selftests/bpf/progs/verifier_spill_fill.c | 111 ++++++++++++++++++ 1 file changed, 111 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c index 3e5d063ea7e8..7f3b1319bd99 100644 --- a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c +++ b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c @@ -979,4 +979,115 @@ l0_%=: r0 = 0; \ : __clobber_all); } +SEC("xdp") +__description("32-bit fill after 64-bit spill") +__success __retval(0) +__naked void fill_32bit_after_spill_64bit(void) +{ + asm volatile(" \ + /* Randomize the upper 32 bits. */ \ + call %[bpf_get_prandom_u32]; \ + r0 <<= 32; \ + /* 64-bit spill r0 to stack. */ \ + *(u64*)(r10 - 8) = r0; \ + /* 32-bit fill r0 from stack. */ \ + " +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + "r0 = *(u32*)(r10 - 8);" +#else + "r0 = *(u32*)(r10 - 4);" +#endif + " \ + /* Boundary check on r0 with predetermined result. */\ + if r0 == 0 goto l0_%=; \ + /* Dead branch: the verifier should prune it. Do an invalid memory\ + * access if the verifier follows it. \ + */ \ + r0 = *(u64*)(r9 + 0); \ +l0_%=: exit; \ +" : + : __imm(bpf_get_prandom_u32) + : __clobber_all); +} + +SEC("xdp") +__description("32-bit fill after 64-bit spill of 32-bit value should preserve ID") +__success __retval(0) +__naked void fill_32bit_after_spill_64bit_preserve_id(void) +{ + asm volatile (" \ + /* Randomize the lower 32 bits. */ \ + call %[bpf_get_prandom_u32]; \ + w0 &= 0xffffffff; \ + /* 64-bit spill r0 to stack - should assign an ID. */\ + *(u64*)(r10 - 8) = r0; \ + /* 32-bit fill r1 from stack - should preserve the ID. */\ + " +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + "r1 = *(u32*)(r10 - 8);" +#else + "r1 = *(u32*)(r10 - 4);" +#endif + " \ + /* Compare r1 with another register to trigger find_equal_scalars. */\ + r2 = 0; \ + if r1 != r2 goto l0_%=; \ + /* The result of this comparison is predefined. */\ + if r0 == r2 goto l0_%=; \ + /* Dead branch: the verifier should prune it. Do an invalid memory\ + * access if the verifier follows it. \ + */ \ + r0 = *(u64*)(r9 + 0); \ + exit; \ +l0_%=: r0 = 0; \ + exit; \ +" : + : __imm(bpf_get_prandom_u32) + : __clobber_all); +} + +SEC("xdp") +__description("32-bit fill after 64-bit spill should clear ID") +__failure __msg("math between ctx pointer and 4294967295 is not allowed") +__naked void fill_32bit_after_spill_64bit_clear_id(void) +{ + asm volatile (" \ + r6 = r1; \ + /* Roll one bit to force the verifier to track both branches. */\ + call %[bpf_get_prandom_u32]; \ + r0 &= 0x8; \ + /* Put a large number into r1. */ \ + r1 = 0xffffffff; \ + r1 <<= 32; \ + r1 += r0; \ + /* 64-bit spill r1 to stack - should assign an ID. */\ + *(u64*)(r10 - 8) = r1; \ + /* 32-bit fill r2 from stack - should clear the ID. */\ + " +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ + "r2 = *(u32*)(r10 - 8);" +#else + "r2 = *(u32*)(r10 - 4);" +#endif + " \ + /* Compare r2 with another register to trigger find_equal_scalars.\ + * Having one random bit is important here, otherwise the verifier cuts\ + * the corners. If the ID was mistakenly preserved on fill, this would\ + * cause the verifier to think that r1 is also equal to zero in one of\ + * the branches, and equal to eight on the other branch.\ + */ \ + r3 = 0; \ + if r2 != r3 goto l0_%=; \ +l0_%=: r1 >>= 32; \ + /* The verifier shouldn't propagate r2's range to r1, so it should\ + * still remember r1 = 0xffffffff and reject the below.\ + */ \ + r6 += r1; \ + r0 = *(u32*)(r6 + 0); \ + exit; \ +" : + : __imm(bpf_get_prandom_u32) + : __clobber_all); +} + char _license[] SEC("license") = "GPL"; From patchwork Sat Jan 27 17:52:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534222 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 195B92D04C; Sat, 27 Jan 2024 17:53:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377996; cv=none; b=VpGCbiYUTPX9NpAd/RNijYpUAYX6yIjssDCnTwbvyJ44qV2QvEKsX3FV1Af5u9pUzqUFDylz0SOOhBEUaerWwxz6ijwuqeqWlk/9KjWL2by9rWfgn3E42WFqXG2WxrxtvvhIFK8acRLvqnTiEGqTslVikSlhu3iRh3locUuVn2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706377996; c=relaxed/simple; bh=ppuOJbWTKXG47/qhgqZIDczj38nPaUZRQ1q+3ti7jxU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Qfi8aI3vuHyrZcz731kRSD+lyxDi2ooYiqqv5qJSo7TRkdwU0V4rjZ2nJ3qHVXiwENDOLdHawSZkCOVa0RWTm9N4GKlrpxGz/O08GDgh2F+ZZzt3TWSWq2s5yAOs03XmvHiakWL/gVLrU1fuyL2HOlXHO1Uc7HjBhXNKVEYSMDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eobIvT+0; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eobIvT+0" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a30e445602cso533592066b.0; Sat, 27 Jan 2024 09:53:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377993; x=1706982793; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gih8YjOYCnRXo89Ub0lhyCjdzkl+eWifpUUwXHuNba8=; b=eobIvT+0zLotwb9EHhtOwEMb2AGAoLIk7blvKh2i9x2F1u6WLYnTg2merdKjligSnL 9EghsI1AgI7rZOQORcAK+TPIpd9MPN1CAikDa27DmPP5N+Ih1wYx50hq0bywxxv4+IEF i+Pblhz/xjrn5jvXBOKb2yPxpEi950GEw9DbK/88HL7p8tGQBC5648WWjDEFYpK1ip3O OvFAtv9mLO/aFk+6/sxFuCE2tfM0YfFX7L5wnq1Fixiw3TBcrZCm9fsYDI1UM9o1m/fz XkYrWepso/Fkpk2Mx1rAu4GjYJ2IIJon6/x8uOgvZ/GNaa///EoaRPg6JvF5tLt0JCeN WFgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377993; x=1706982793; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gih8YjOYCnRXo89Ub0lhyCjdzkl+eWifpUUwXHuNba8=; b=FB/FrjszdJGvYOM5V9AealGPsGBcm/s1hWpPQNg0aO7n5Y7q/6WYC4rGnRfe/ihOL/ Z6NKb+r1MfONCfJTIHRI8j3APGguL5t2Q0nS54QUaiBoW8w+pQS7hs7WPP2vSGK5rSC3 bcnA/jG7Em6QN5MFrbfzk1mIFYQZAErMgmVq+40102/VLdDSS3GiZAAasqnbhs0J9QL1 0hActsQoVwOCwJ7ppvVajRJY+nkgUuFvDubR3qcLCUrrNygG904L3SPa3c9Gj+YHjofz mmkf3ozlpDZUu11xHBODH1bMTHXpKMhrymrjZNSb5WnAj4noX3PuCksI/G5AJASFlc59 hK6g== X-Gm-Message-State: AOJu0Yz68s0wewl0Ku0xiKuQGL67MWd11qm/DTcZzrRe+WvBt+wpMsnQ skD/OVw4kNNC5VgrOQYrOIXUUPIXyUDf3svM/9xW54hpJsk61zWA X-Google-Smtp-Source: AGHT+IH/9Nj5WDlsPbIDD1PwT1FxXbii1wmvr/nrJGYxhoi6DMNAq+DmzYtYnoPRfoyLnVY+Q57aig== X-Received: by 2002:a17:907:76e9:b0:a2b:130c:a897 with SMTP id kg9-20020a17090776e900b00a2b130ca897mr3789358ejc.17.1706377992884; Sat, 27 Jan 2024 09:53:12 -0800 (PST) Received: from localhost ([185.220.101.170]) by smtp.gmail.com with ESMTPSA id vu2-20020a170907a64200b00a2cc5199dd0sm1967844ejc.135.2024.01.27.09.53.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:53:12 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v3 5/6] bpf: handle scalar spill vs all MISC in stacksafe() Date: Sat, 27 Jan 2024 19:52:36 +0200 Message-ID: <20240127175237.526726-6-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eduard Zingerman When check_stack_read_fixed_off() reads value from an spi all stack slots of which are set to STACK_{MISC,INVALID}, the destination register is set to unbound SCALAR_VALUE. Exploit this fact by allowing stacksafe() to use a fake unbound scalar register to compare 'mmmm mmmm' stack value in old state vs spilled 64-bit scalar in current state and vice versa. Veristat results after this patch show some gains: ./veristat -C -e file,prog,states -f 'states_pct>10' not-opt after File Program States (DIFF) ----------------------- --------------------- --------------- bpf_overlay.o tail_rev_nodeport_lb4 -45 (-15.85%) bpf_xdp.o tail_lb_ipv4 -541 (-19.57%) pyperf100.bpf.o on_event -680 (-10.42%) pyperf180.bpf.o on_event -2164 (-19.62%) pyperf600.bpf.o on_event -9799 (-24.84%) strobemeta.bpf.o on_event -9157 (-65.28%) xdp_synproxy_kern.bpf.o syncookie_tc -54 (-19.29%) xdp_synproxy_kern.bpf.o syncookie_xdp -74 (-24.50%) Signed-off-by: Eduard Zingerman --- kernel/bpf/verifier.c | 72 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 69 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 81b08a0e9e2c..6a174c4849ba 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1155,6 +1155,12 @@ static bool is_spilled_scalar_reg(const struct bpf_stack_state *stack) stack->spilled_ptr.type == SCALAR_VALUE; } +static bool is_spilled_scalar_reg64(const struct bpf_stack_state *stack) +{ + return stack->slot_type[0] == STACK_SPILL && + stack->spilled_ptr.type == SCALAR_VALUE; +} + /* Mark stack slot as STACK_MISC, unless it is already STACK_INVALID, in which * case they are equivalent, or it's STACK_ZERO, in which case we preserve * more precise STACK_ZERO. @@ -2264,8 +2270,7 @@ static void __reg_assign_32_into_64(struct bpf_reg_state *reg) } /* Mark a register as having a completely unknown (scalar) value. */ -static void __mark_reg_unknown(const struct bpf_verifier_env *env, - struct bpf_reg_state *reg) +static void __mark_reg_unknown_imprecise(struct bpf_reg_state *reg) { /* * Clear type, off, and union(map_ptr, range) and @@ -2277,10 +2282,20 @@ static void __mark_reg_unknown(const struct bpf_verifier_env *env, reg->ref_obj_id = 0; reg->var_off = tnum_unknown; reg->frameno = 0; - reg->precise = !env->bpf_capable; + reg->precise = false; __mark_reg_unbounded(reg); } +/* Mark a register as having a completely unknown (scalar) value, + * initialize .precise as true when not bpf capable. + */ +static void __mark_reg_unknown(const struct bpf_verifier_env *env, + struct bpf_reg_state *reg) +{ + __mark_reg_unknown_imprecise(reg); + reg->precise = !env->bpf_capable; +} + static void mark_reg_unknown(struct bpf_verifier_env *env, struct bpf_reg_state *regs, u32 regno) { @@ -16474,6 +16489,43 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold, } } +static struct bpf_reg_state unbound_reg; + +static __init int unbound_reg_init(void) +{ + __mark_reg_unknown_imprecise(&unbound_reg); + unbound_reg.live |= REG_LIVE_READ; + return 0; +} +late_initcall(unbound_reg_init); + +static bool is_stack_all_misc(struct bpf_verifier_env *env, + struct bpf_stack_state *stack) +{ + u32 i; + + for (i = 0; i < ARRAY_SIZE(stack->slot_type); ++i) { + if ((stack->slot_type[i] == STACK_MISC) || + (stack->slot_type[i] == STACK_INVALID && env->allow_uninit_stack)) + continue; + return false; + } + + return true; +} + +static struct bpf_reg_state *scalar_reg_for_stack(struct bpf_verifier_env *env, + struct bpf_stack_state *stack) +{ + if (is_spilled_scalar_reg64(stack)) + return &stack->spilled_ptr; + + if (is_stack_all_misc(env, stack)) + return &unbound_reg; + + return NULL; +} + static bool stacksafe(struct bpf_verifier_env *env, struct bpf_func_state *old, struct bpf_func_state *cur, struct bpf_idmap *idmap, bool exact) { @@ -16512,6 +16564,20 @@ static bool stacksafe(struct bpf_verifier_env *env, struct bpf_func_state *old, if (i >= cur->allocated_stack) return false; + /* 64-bit scalar spill vs all slots MISC and vice versa. + * Load from all slots MISC produces unbound scalar. + * Construct a fake register for such stack and call + * regsafe() to ensure scalar ids are compared. + */ + old_reg = scalar_reg_for_stack(env, &old->stack[spi]); + cur_reg = scalar_reg_for_stack(env, &cur->stack[spi]); + if (old_reg && cur_reg) { + if (!regsafe(env, old_reg, cur_reg, idmap, exact)) + return false; + i += BPF_REG_SIZE - 1; + continue; + } + /* if old state was safe with misc data in the stack * it will be safe with zero-initialized stack. * The opposite is not true From patchwork Sat Jan 27 17:52:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Mikityanskiy X-Patchwork-Id: 13534231 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B4062D04C; Sat, 27 Jan 2024 17:53:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706378001; cv=none; b=AvlVNk0Fz7MoeWdP32QBrQJq/bgqQDVVAHsj77Ie9+jQ7Q7wo0/frwOw4wAkMROCBXlXU0VbWf3Ucn232hc+oOVRKCgbjolXRdhGtmukZOuF1Fo4Rm8cKbyXGeegn1mEmlhMPN+vNfLJVblf6RinD3h//OJOEmyZwZEUmXswS3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706378001; c=relaxed/simple; bh=FPhNJPCfmYlJH1B1BBcfKiI2kTB7BgMYUwbeyONYl9U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G0vv9m5a1/nWuu6rpF8oeqCE3uVcxb+b0VO3TxOK0jXPZGwlzA/5kDjbK9LNavshZ4NB52cpWFX2iwgB5j4UT7X7BO2BXAlqvf7WOhA58Rrvqjy1GzxMh1TzMpq/rv7wpCIn/605uUDiZlf1LO+mfbrqiMiXcxjtVwukL7prc+A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=A2xWN2N/; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="A2xWN2N/" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-40ea5653f6bso23599085e9.3; Sat, 27 Jan 2024 09:53:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706377998; x=1706982798; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ozV2KGMrFgsHS1gPSO2eoaYS6GdUgNAcChmUCNMwCAk=; b=A2xWN2N/3TH5acS9vdnC2a6EbjT8DXrgKvBHzRvcNp/FlKc5rm59QHPq8D/n+grnOW 74khNB/XL4bvRKHjFjSNcQaBBdFMEdWQxlLiBbH4EgcKLTJGzvf1Xqp44FZKEsTo7E/m EBp3vqpQuq3fFazEV8hfvG8LZCledtcvGZc0gVn/xnI4XtxHku1KM9eq9upaeEYjc7G9 txmBRzdR5WtXXJWQL4MrAb8IByZfXws9DnnoO41mIWxU9q9Gyez0C+nPGja/qrnCauLv 4bXpAGXbA9pJQ1XwpeG/L9qrNPHumdYGXgWplGwfZ38Hd4GIR8ai6OOPMElcZo3u7I04 BSyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706377998; x=1706982798; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ozV2KGMrFgsHS1gPSO2eoaYS6GdUgNAcChmUCNMwCAk=; b=pzqcL/ynOjTls6RNuZvHV2gVcPytnUAp48uKLKSxY/RktchXH+m4QJyGxF0Jj+UqRH QLj8dAsibT5soFc26H6aaIU2CuWIftBP+0j9Y8S3/xGncuMmOYSIB9BoWVZCK2pHJC8V OhpGSSYlqaGXqmCSOA5Id+KV494n1NKFDJryVa2riYqe3KrQTSJCro8lzur7HaZIq4CK 4GIyEzUT8PRpCTIgK4wNJblf7roLecJDPNwJxL6qR5sm46wQ88JYDqt8xmon0Yb1FtCo Dca6bCJvPBsoM+o/DyrAByjYeSrDFlkmsrjeD/AHvKRRIjwwWWRre8JYN5NGJiTWaSd5 a/cg== X-Gm-Message-State: AOJu0YxU9kyzFEWeqj822maN8dqNv3MAL46FhOYM60I3fuN7JCZETSnJ cPiptemjNx0UrF3tOEO79GJBvplDRrTIwLIVnOb+ua8jcCHkRf3C X-Google-Smtp-Source: AGHT+IGyZ4sp0ye/xpHnStxlAoFfsZB17xRWshIdpLqRd365oDnOYY4p7kscEoVnzsy9MR58aETrBQ== X-Received: by 2002:adf:f351:0:b0:333:44e2:16b7 with SMTP id e17-20020adff351000000b0033344e216b7mr1414501wrp.49.1706377998257; Sat, 27 Jan 2024 09:53:18 -0800 (PST) Received: from localhost ([185.100.87.41]) by smtp.gmail.com with ESMTPSA id g4-20020aa7d1c4000000b0055d1a75bd0csm1803307edp.57.2024.01.27.09.53.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jan 2024 09:53:17 -0800 (PST) From: Maxim Mikityanskiy To: Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Shung-Hsi Yu Cc: John Fastabend , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v3 6/6] selftests/bpf: states pruning checks for scalar vs STACK_MISC Date: Sat, 27 Jan 2024 19:52:37 +0200 Message-ID: <20240127175237.526726-7-maxtram95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240127175237.526726-1-maxtram95@gmail.com> References: <20240127175237.526726-1-maxtram95@gmail.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eduard Zingerman Check that stacksafe() compares spilled scalars with STACK_MISC. The following combinations are explored: - old spill of imprecise scalar is equivalent to cur STACK_{MISC,INVALID} (plus error in unpriv mode); - old spill of precise scalar is not equivalent to cur STACK_MISC; - old STACK_MISC is equivalent to cur scalar; - old STACK_MISC is not equivalent to cur non-scalar. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/progs/verifier_spill_fill.c | 154 ++++++++++++++++++ 1 file changed, 154 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c index 7f3b1319bd99..85e48069c9e6 100644 --- a/tools/testing/selftests/bpf/progs/verifier_spill_fill.c +++ b/tools/testing/selftests/bpf/progs/verifier_spill_fill.c @@ -1090,4 +1090,158 @@ l0_%=: r1 >>= 32; \ : __clobber_all); } +/* stacksafe(): check if stack spill of an imprecise scalar in old state + * is considered equivalent to STACK_{MISC,INVALID} in cur state. + */ +SEC("socket") +__success __log_level(2) +__msg("8: (79) r1 = *(u64 *)(r10 -8)") +__msg("8: safe") +__msg("processed 11 insns") +/* STACK_INVALID should prevent verifier in unpriv mode from + * considering states equivalent and force an error on second + * verification path (entry - label 1 - label 2). + */ +__failure_unpriv +__msg_unpriv("8: (79) r1 = *(u64 *)(r10 -8)") +__msg_unpriv("9: (95) exit") +__msg_unpriv("8: (79) r1 = *(u64 *)(r10 -8)") +__msg_unpriv("invalid read from stack off -8+2 size 8") +__flag(BPF_F_TEST_STATE_FREQ) +__naked void old_imprecise_scalar_vs_cur_stack_misc(void) +{ + asm volatile( + /* get a random value for branching */ + "call %[bpf_ktime_get_ns];" + "if r0 == 0 goto 1f;" + /* conjure scalar at fp-8 */ + "r0 = 42;" + "*(u64*)(r10 - 8) = r0;" + "goto 2f;" +"1:" + /* conjure STACK_{MISC,INVALID} at fp-8 */ + "call %[bpf_ktime_get_ns];" + "*(u16*)(r10 - 8) = r0;" + "*(u16*)(r10 - 4) = r0;" +"2:" + /* read fp-8, should be considered safe on second visit */ + "r1 = *(u64*)(r10 - 8);" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + +/* stacksafe(): check that stack spill of a precise scalar in old state + * is not considered equivalent to STACK_MISC in cur state. + */ +SEC("socket") +__success __log_level(2) +/* verifier should visit 'if r1 == 0x2a ...' two times: + * - once for path entry - label 2; + * - once for path entry - label 1 - label 2. + */ +__msg("if r1 == 0x2a goto pc+0") +__msg("if r1 == 0x2a goto pc+0") +__msg("processed 15 insns") +__flag(BPF_F_TEST_STATE_FREQ) +__naked void old_precise_scalar_vs_cur_stack_misc(void) +{ + asm volatile( + /* get a random value for branching */ + "call %[bpf_ktime_get_ns];" + "if r0 == 0 goto 1f;" + /* conjure scalar at fp-8 */ + "r0 = 42;" + "*(u64*)(r10 - 8) = r0;" + "goto 2f;" +"1:" + /* conjure STACK_MISC at fp-8 */ + "call %[bpf_ktime_get_ns];" + "*(u64*)(r10 - 8) = r0;" + "*(u32*)(r10 - 4) = r0;" +"2:" + /* read fp-8, should not be considered safe on second visit */ + "r1 = *(u64*)(r10 - 8);" + /* use r1 in precise context */ + "if r1 == 42 goto +0;" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + +/* stacksafe(): check if STACK_MISC in old state is considered + * equivalent to stack spill of a scalar in cur state. + */ +SEC("socket") +__success __log_level(2) +__msg("8: (79) r0 = *(u64 *)(r10 -8)") +__msg("8: safe") +__msg("processed 11 insns") +__flag(BPF_F_TEST_STATE_FREQ) +__naked void old_stack_misc_vs_cur_scalar(void) +{ + asm volatile( + /* get a random value for branching */ + "call %[bpf_ktime_get_ns];" + "if r0 == 0 goto 1f;" + /* conjure STACK_{MISC,INVALID} at fp-8 */ + "call %[bpf_ktime_get_ns];" + "*(u16*)(r10 - 8) = r0;" + "*(u16*)(r10 - 4) = r0;" + "goto 2f;" +"1:" + /* conjure scalar at fp-8 */ + "r0 = 42;" + "*(u64*)(r10 - 8) = r0;" +"2:" + /* read fp-8, should be considered safe on second visit */ + "r0 = *(u64*)(r10 - 8);" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + +/* stacksafe(): check that STACK_MISC in old state is not considered + * equivalent to stack spill of a non-scalar in cur state. + */ +SEC("socket") +__success __log_level(2) +/* verifier should process exit instructions twice: + * - once for path entry - label 2; + * - once for path entry - label 1 - label 2. + */ +__msg("r1 = *(u64 *)(r10 -8)") +__msg("exit") +__msg("r1 = *(u64 *)(r10 -8)") +__msg("exit") +__msg("processed 11 insns") +__flag(BPF_F_TEST_STATE_FREQ) +__naked void old_stack_misc_vs_cur_ctx_ptr(void) +{ + asm volatile( + /* remember context pointer in r9 */ + "r9 = r1;" + /* get a random value for branching */ + "call %[bpf_ktime_get_ns];" + "if r0 == 0 goto 1f;" + /* conjure STACK_MISC at fp-8 */ + "call %[bpf_ktime_get_ns];" + "*(u64*)(r10 - 8) = r0;" + "*(u32*)(r10 - 4) = r0;" + "goto 2f;" +"1:" + /* conjure context pointer in fp-8 */ + "*(u64*)(r10 - 8) = r9;" +"2:" + /* read fp-8, should not be considered safe on second visit */ + "r1 = *(u64*)(r10 - 8);" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + char _license[] SEC("license") = "GPL";