From patchwork Wed Aug 14 03:30:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762804 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f175.google.com (mail-yw1-f175.google.com [209.85.128.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1BE91EEE9 for ; Wed, 14 Aug 2024 03:30:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606217; cv=none; b=Vp200LqHALAkkT/3OM0ro6HwG6i4fea9ykWwaapf3VDxVADR60TZixsNpv7ngUUV3Ic4u/cByaXiIkItbCwi6FAE4fvvwpj1IgCraasqxT3I49F8sVnkDQwVgZVzpWevxVZlYs+ZB9woW8VFrQtX2g0P1ejSsnzqJ1vVOE/lbAg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606217; c=relaxed/simple; bh=6uguzTwBipT1pGt4iZ9oV71qd0fpV8tMOJjn79NSbc8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r+wGD9kKAx4lfk2bIeRYRzrtvzPoxAZ/bEgC/mf5zZrQhFu0WlVFClfEM1qiJkkbnM1eqirbT0e6ZhE+oL+rQz49MdSpzKma8kII9ZmQ8jIXP7JOLWDB0x2ZJX04d3Bi8UagNP+j0L/eUujRoVQq4l7o/HpnPg5CgIdmcWQU3ng= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kJ2lpoQr; arc=none smtp.client-ip=209.85.128.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kJ2lpoQr" Received: by mail-yw1-f175.google.com with SMTP id 00721157ae682-690af536546so62017067b3.3 for ; Tue, 13 Aug 2024 20:30:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606214; x=1724211014; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B4W3sp2O8bZa06ldyNi6Xkm8a0QLS7BrI4ar8WNRouE=; b=kJ2lpoQrP0aYOht2aV/e9rrVTDIDXEwWWW05LkrZ/PRD1LqUFXwwAgVT/T8LSJN9yK 6r6XuA2nvSopMIuMkA3SrzRL2P2RlI/bVTH2XrL3rOoFeTpyXJkhr/kEDM796L2Wu2+6 8XK6UZjJV3QkfhLBCAVbLEHKmgWi6+Ks0thuC2McwQ0Vj+/Vq8ER2IM/p3qoFIiy6Zb4 x27H4hTE0kgAZzfx6KsIqwJABp21+W/ORUoXgkcJ+ZlrRhwCAX7SNqjhSSR2823/vGvS EPnwsagXuB38GkwaEWUq0bMnVn6aMM5t5xzsTbI01l6JjSL+j1g5COScwL9zFYDSQ+5p NNpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606214; x=1724211014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B4W3sp2O8bZa06ldyNi6Xkm8a0QLS7BrI4ar8WNRouE=; b=sR9k9Cee4g7kgtqMisn1ECPn+Dcrk19IpJy2K9tzvfLJ7mg2Ab+nPZzr8m2TcQjug+ Y5NDWUmZjX1Yn69lurYjaPzb1tljLvAm59JFBW6+7mQ80jTO2gcPtJLI0Bi3TSrnZ5Lj bp0/DFo8fZpDcEOM+QWYjPjDXpuKqi/JHOGhQO32czSDPBR0wp3cHydMplMudBgtGqu+ VgiBsxxV92GInMs00hAbsuGSihABgdbX8KSf+TYraIwSi3Dz0u5+avGfpRPUlSF6UNM/ swq/NxyHrYqYUTLhzevytxvlHKqFBYmI51sfJcX72+uVgaLqn92j6EtS0HIQtweMhcVY alWA== X-Gm-Message-State: AOJu0Yw1AL26neHs0P89epw4kKEJCswgzR8JxXrMWL+uwsg/v5nwACEr PosJVeyXwqWsU7FPXkmCLDiw7rSbak7/kD2A5U1lGGxYOijQovJGUn+PwqKx X-Google-Smtp-Source: AGHT+IEJFczEH9ELL9G9xWR5BRbkZDpPLOKkKg6K9OPH62H2oYV6Q4FhRAARNJwGqOduPFrzdjKmXw== X-Received: by 2002:a05:690c:3302:b0:651:6888:a018 with SMTP id 00721157ae682-6ac9764531fmr16074917b3.26.1723606214374; Tue, 13 Aug 2024 20:30:14 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:14 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 1/7] bpf: define BPF_UPTR a new enumerator of btf_field_type. Date: Tue, 13 Aug 2024 20:30:04 -0700 Message-Id: <20240814033010.2980635-2-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Define BPF_UPTR, and modify functions that describe attributes of a field type. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 9f35df07e86d..954e476b5605 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -203,6 +203,7 @@ enum btf_field_type { BPF_GRAPH_ROOT = BPF_RB_ROOT | BPF_LIST_HEAD, BPF_REFCOUNT = (1 << 9), BPF_WORKQUEUE = (1 << 10), + BPF_UPTR = (1 << 11), }; typedef void (*btf_dtor_kfunc_t)(void *); @@ -322,6 +323,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type) return "kptr"; case BPF_KPTR_PERCPU: return "percpu_kptr"; + case BPF_UPTR: + return "uptr"; case BPF_LIST_HEAD: return "bpf_list_head"; case BPF_LIST_NODE: @@ -350,6 +353,7 @@ static inline u32 btf_field_type_size(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: return sizeof(u64); case BPF_LIST_HEAD: return sizeof(struct bpf_list_head); @@ -379,6 +383,7 @@ static inline u32 btf_field_type_align(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: return __alignof__(u64); case BPF_LIST_HEAD: return __alignof__(struct bpf_list_head); @@ -419,6 +424,7 @@ static inline void bpf_obj_init_field(const struct btf_field *field, void *addr) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: break; default: WARN_ON_ONCE(1); From patchwork Wed Aug 14 03:30:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762805 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0A712BB1C for ; Wed, 14 Aug 2024 03:30:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606218; cv=none; b=r/+jPP0fGI5Erk+lreu9r5mfjfCrXIB3bWG2Nd7CnmsFJMZTBI+e9J688JWKiL47cDXcLOSYCO/A0bGmwlPg9/NMzosTeQEf8hXlCsTcwqNN8It4oxFqRrZsAFvEelwar9cdf7pq4NdSlVFGZvRw837NrPGuYl/Wpx9mQX5P1xE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606218; c=relaxed/simple; bh=Wg4qmiKbu07OPOW7gvELVSHVyPKbKWmHubbg6/DTWLQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T0ykYV+SVyP/bfsnIZbfDw07H3h1V0x5sQVEFZkC/7ZcdedlKy9MzvUSpGRtEd1LtvFrzJOjXPiBu1H4k7IRTuCVq8Flyv1TcvxPbzkaiNDhEa1ghyRsvaeAgae6JF+6/J/9kKeHQJnL4g8RotMlCJiRU3o9uRfvyy8td/mIOvk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=b4mr6TJG; arc=none smtp.client-ip=209.85.128.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b4mr6TJG" Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-66108213e88so55030317b3.1 for ; Tue, 13 Aug 2024 20:30:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606215; x=1724211015; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UbLF3v+0Qr91UjWGKHwc1e2DoPoHow16O9w04Ex1jM4=; b=b4mr6TJG9Ma6XD0wGYXQwdTE1nDF4qk2gI2/t25CXeC0YONPWqdeXbImbwe6XRV3wT p1N4zoC9Dp1Nym9e2uwd0cJku/M4hvfLFWMuFIG9wL3lNGOFyUh/ve3P5QAjSNW/xt4I 6MAN7CFTVEL8iizsVRG7U00McA+t1kY5Q+Ke6TTQ8CyBU2aScw7SxtZnkVA0q/8HMEZC YMjML6vMan12qIB65uyTlTFaXznYVeM6uZjm1I4IsyShRjy24uz7una99FqF9JoxNRq6 vRLGV/l6W294/rX7+AwQhmfB2X4CDgVWCFNqHmYs4NlJIeXChx9b9ig138YLZyPqFQdW qJ3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606215; x=1724211015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UbLF3v+0Qr91UjWGKHwc1e2DoPoHow16O9w04Ex1jM4=; b=jvqrmHAnc8GWNynCi1G/bw9NPRCFv3URgNVh4WhqBxD5ls6AE2FbPYCmRgJzVejDFq KbnuaG4Zt6J1CVIvhnMyW7Vxk9jCm53Dul6Qnpap02390Hxy9bHBfadelmT5TjznmzLJ yjlK5X6ShDDyI5dj9N8HDIf/iaQUprmsUXSzZHu+OS1UA4XkF6tpt2tre1r1mq/+XBh8 71XA86fTzQaLulEwXW8MkU/l/TKghFWV6AT28IEf5BIhIGUn8Ijwkdh+cuOs7lDxYkic QZ81HVc3t8Ee95mcUGOjy0+YT6tPONGKBEbJtGiPZ8CsH0OJCLraOtyryoVVeWJIaXem y+LQ== X-Gm-Message-State: AOJu0Yzg8vwQ4MLa8aaRFQf893ryOUfXZRtBrNrFadBX4XJE5mI4cucr CS6w1npkYXSOqpiRMgTlFiKO79Qg0jYGQBItuDYvi07MKPnJKOwWfyUyvktW X-Google-Smtp-Source: AGHT+IGTIpShZBciimQolUrUozrkWhFYBhHqtyVo5mwskzG2Ekiy5rjAxwnSTlVq5uOLMyXP/JVXZA== X-Received: by 2002:a05:690c:4787:b0:65b:cdab:2678 with SMTP id 00721157ae682-6ac96bb55e1mr14475297b3.15.1723606215504; Tue, 13 Aug 2024 20:30:15 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:15 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 2/7] bpf: Parse and support "uptr" tag. Date: Tue, 13 Aug 2024 20:30:05 -0700 Message-Id: <20240814033010.2980635-3-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Parse "uptr" tag from BTF, map it to BPF_UPTR, and support it in related functions. "uptr" tag is used to annotate a field in a struct type is a uptr, which is used to share a block memory between user programs and BPF programs. Signed-off-by: Kui-Feng Lee --- kernel/bpf/btf.c | 5 +++++ kernel/bpf/syscall.c | 2 ++ 2 files changed, 7 insertions(+) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index c4506d788c85..9db3e7d2fa66 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3361,6 +3361,8 @@ static int btf_find_kptr(const struct btf *btf, const struct btf_type *t, type = BPF_KPTR_REF; else if (!strcmp("percpu_kptr", __btf_name_by_offset(btf, t->name_off))) type = BPF_KPTR_PERCPU; + else if (!strcmp("uptr", __btf_name_by_offset(btf, t->name_off))) + type = BPF_UPTR; else return -EINVAL; @@ -3538,6 +3540,7 @@ static int btf_repeat_fields(struct btf_field_info *info, case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: case BPF_LIST_HEAD: case BPF_RB_ROOT: break; @@ -3664,6 +3667,7 @@ static int btf_find_field_one(const struct btf *btf, case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: ret = btf_find_kptr(btf, var_type, off, sz, info_cnt ? &info[0] : &tmp); if (ret < 0) @@ -3988,6 +3992,7 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: ret = btf_parse_kptr(btf, &rec->fields[i], &info_arr[i]); if (ret < 0) goto end; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 65dcd92d0b2c..fed4a2145f81 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -548,6 +548,7 @@ void btf_record_free(struct btf_record *rec) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: if (rec->fields[i].kptr.module) module_put(rec->fields[i].kptr.module); btf_put(rec->fields[i].kptr.btf); @@ -596,6 +597,7 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: btf_get(fields[i].kptr.btf); if (fields[i].kptr.module && !try_module_get(fields[i].kptr.module)) { ret = -ENXIO; From patchwork Wed Aug 14 03:30:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762806 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9AC03BBE9 for ; Wed, 14 Aug 2024 03:30:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606219; cv=none; b=LgeNIRgxgje8xRHkPdtVX/6cWQx8VMxx5odtdsq7SBY4yacqbalGcya96/KrQ7DDG8yLMzqZtl5uapt+k3KlQ5udJdajSXtxmDCwdZ6VZemQIwpA4UJkH+cjU92jyRI13V42QJ/29YLDEi9x9YghkUrhfisZL7wrqXxKkOdtOas= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606219; c=relaxed/simple; bh=2kOtRv7qB6Foh6TYAjMuRKyqQS09rtUXbv1HfzFhBpI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JVKZ42P9YZYI+F11Tn/1HsOI6HhlWcLMuGeDPwM1P9rOHa5pqL/U5JR3tNUh8k7WmuRSZ7aUMoGPa45cfEx4dU2lrq6itTr08bELCiZyMp/Jw/8b319Jr3NC9YygkeJU62Sne5oV5pIef8Ns0oNoKSdZW8f2UqvSFTe1lb+6pAw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mR4USPn3; arc=none smtp.client-ip=209.85.128.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mR4USPn3" Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-66599ca3470so59057477b3.2 for ; Tue, 13 Aug 2024 20:30:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606217; x=1724211017; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yQuSW7ZjyolAixejdxlo5ouKzCwC+WVNTWhjPrUWT8U=; b=mR4USPn3vk/DbQ8znFr9tmj+lKqBzAyDIqMeSg1fcRNNNBeS/l4YufJvSmFbEr8hP9 RzN/3Mxvk1g/XhR394UTRg600VZHkk6NhzWdOtmbSyV7IOH8sDc1l5eugytRhmZEEfMJ 1973QCI5TDRRIlDhNJ1ju1aH7KG4DIsAuqw56EWsvXIN1YWPMRoZy9DCx7tFCR9RLTK8 cCLO5C1UbUN4Qg9la9bPzHvTcU4mG9r2wCGSc65IDan9KlB2OHKtxh0CZH+peKU2HY1I Y5r9vcbWcyPQHu2qQfN2SCf5mduW4DSWj5DZEUu49UN5rWCIl61/pBwRQsRuXIfgMAWu ZiVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606217; x=1724211017; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yQuSW7ZjyolAixejdxlo5ouKzCwC+WVNTWhjPrUWT8U=; b=ItSP+PQ3ASE668GECNGY5c8uKJZYyow0SlqsBUpOLz+AUFvuu3Ru3UfOeSNBnkUY7M ZIa+SEhsLInXKqbzQ8APOAcKok09VY41BU+RMRt+XsG9OjM9xRvaxDjBPRr/SsOedRR6 nuxoTD2h8sqSBCUrei97fTtED9dE314Yc48gMRQNcFIPDFE9ikhvIB2pwuusxbNMTjwE I51yDJ0GC80R+7chgnUcKTLRG+iEbIgW7XKvnQsmyaFBY6/sicdJ9Yd5H1Ys7rnJPlcG XCm4d+GnzZ6ufHtkjrrn9HVbDDouVPmqTt2oYyYj0kcJavXfWxUC/tc3gdlGoHiMNtPF sPQA== X-Gm-Message-State: AOJu0YwLdYGQStDOccNVKs03bq2uD8Wxh3P1UwHs8aEUqyzMzdntVNcL A13nbnnmAXmOcyn5K/U9l4G9JedaQHYmV4Jir8kytDxx1G2IOV9H/eLtiDXC X-Google-Smtp-Source: AGHT+IFTjt8hdnrdZ3eu+mx20nMdPS4U4+x98qaNTM7mTckQB1RPjlwuEvlmCHNCtyJHrlc8h4oOSQ== X-Received: by 2002:a05:690c:ecc:b0:64b:69f0:f8f2 with SMTP id 00721157ae682-6ac9621a6fcmr16845777b3.3.1723606216636; Tue, 13 Aug 2024 20:30:16 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:16 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 3/7] bpf: Handle BPF_UPTR in verifier. Date: Tue, 13 Aug 2024 20:30:06 -0700 Message-Id: <20240814033010.2980635-4-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Give PTR_TO_MEM | PTR_MAYBE_NULL to the memory pointed by an uptr with the size of the pointed type to make them readable and writable. Signed-off-by: Kui-Feng Lee --- kernel/bpf/verifier.c | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e3932f8ce10a..5bc5b37b63cc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5340,6 +5340,10 @@ static int map_kptr_match_type(struct bpf_verifier_env *env, int perm_flags; const char *reg_name = ""; + if (kptr_field->type == BPF_UPTR) + /* BPF programs should not change any user kptr */ + return -EACCES; + if (btf_is_kernel(reg->btf)) { perm_flags = PTR_MAYBE_NULL | PTR_TRUSTED | MEM_RCU; @@ -5488,6 +5492,29 @@ static u32 btf_ld_kptr_type(struct bpf_verifier_env *env, struct btf_field *kptr return ret; } +static int mark_uptr_ld_reg(struct bpf_verifier_env *env, u32 regno, + struct btf_field *field) +{ + struct bpf_reg_state *val_reg; + const struct btf_type *t; + u32 type_id, tsz; + + val_reg = reg_state(env, regno); + type_id = field->kptr.btf_id; + t = btf_type_id_size(field->kptr.btf, &type_id, &tsz); + if (!t) { + verbose(env, "The type of uptr is invalid"); + return -EACCES; + } + + mark_reg_known_zero(env, cur_regs(env), regno); + val_reg->type = PTR_TO_MEM | PTR_MAYBE_NULL; + val_reg->mem_size = tsz; + val_reg->id = ++env->id_gen; + + return 0; +} + static int check_map_kptr_access(struct bpf_verifier_env *env, u32 regno, int value_regno, int insn_idx, struct btf_field *kptr_field) @@ -5516,9 +5543,16 @@ static int check_map_kptr_access(struct bpf_verifier_env *env, u32 regno, verbose(env, "store to referenced kptr disallowed\n"); return -EACCES; } + if (class != BPF_LDX && kptr_field->type == BPF_UPTR) { + verbose(env, "store to uptr disallowed\n"); + return -EACCES; + } if (class == BPF_LDX) { val_reg = reg_state(env, value_regno); + if (kptr_field->type == BPF_UPTR) + return mark_uptr_ld_reg(env, value_regno, kptr_field); + /* We can simply mark the value_regno receiving the pointer * value from map as PTR_TO_BTF_ID, with the correct type. */ @@ -5576,6 +5610,7 @@ static int check_map_access(struct bpf_verifier_env *env, u32 regno, case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: + case BPF_UPTR: if (src != ACCESS_DIRECT) { verbose(env, "kptr cannot be accessed indirectly by helper\n"); return -EACCES; @@ -6956,7 +6991,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn return err; if (tnum_is_const(reg->var_off)) kptr_field = btf_record_find(reg->map_ptr->record, - off + reg->var_off.value, BPF_KPTR); + off + reg->var_off.value, BPF_KPTR | BPF_UPTR); if (kptr_field) { err = check_map_kptr_access(env, regno, value_regno, insn_idx, kptr_field); } else if (t == BPF_READ && value_regno >= 0) { From patchwork Wed Aug 14 03:30:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762807 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C992E3FBA5 for ; Wed, 14 Aug 2024 03:30:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606221; cv=none; b=MtuTWOCm2gtlWI59LQWdP01Osefmb3ed42n3girSJMYgBkgfzl3zg1vNEMq95Laul9r9c5wVJnDVzjUOWcwhxoPbitZrygCcfApV5CltnnP0D5/p63bC4u3rz4pee9ZCF0p9tmdikX4qIhTdB03cS+cfuW/U3L6vZeewPS+WZY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606221; c=relaxed/simple; bh=TxXltHGbAB9FV5PI/f1qTCaWCXoT6t9TOXMrIAoxN4s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oRtYnJgHJz8JoP7w+Y4/TyVjkiqZYXWtUJhMc25fZBsidFQg2lJV72vDZrIwRNpHpRPXuHQB+Yz4wGFPhXMDZSkuc5LWtVo0z73WJ+AtQu+olU1ZNZPXxn7UkopO9QrGaLfZry+LxBuWJ0hGQ08Gew6um4rz9zV1C8WXndrywIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JIbC0hMf; arc=none smtp.client-ip=209.85.128.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JIbC0hMf" Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-699d8dc6744so4169757b3.0 for ; Tue, 13 Aug 2024 20:30:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606219; x=1724211019; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/WlaZ8Qk4gImq5cOnYJqxcFCkNXR6u9I92jjLqb+PSo=; b=JIbC0hMfRpBqhfnmjOBlZLlRo7K9EOloGdD+p5pKc1zz/DnxIvzJTQTf38zfkRSUAF c2KuNfDhYrNvqr7nyzaoEM4eqw2q0WnzCGcGg04VggJ/A6ObXogu+/ycYJybU02r3V73 mket2GHrzKwMVOh1L2jTPZHNlu6bFNELLn7tv12+4U2BeaOhNM+alNyHIvEEv+0vwBX7 RHU7ZySYnTWnDQxN1JGFTq3GRvJd1AlJcdUK2LcNHOay0tHXFTnApbasI2AZDdVjGri8 z8m2jkhON2sGmqEMOH0VnMDwPw7EyP89Id3ur9VS8ddWXpgZHYBcXbflB47hdZVwBR9x Da2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606219; x=1724211019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/WlaZ8Qk4gImq5cOnYJqxcFCkNXR6u9I92jjLqb+PSo=; b=RZqkpQb4NsS0spnf0DAtTFF5Rk6Eu16x5kZkkCdVeH3V6A/6bnLO0fqzJT5du7HFLn OUKcn3I9v6N2uN0vmNNu0y5s60Ec6PYQiaiUTtwGXqrQKqgkILb97H9g9WFl2pAct96d uwqT/Rlw+ckvsoEb+MIym+Ka0GrDXl//UTwLO6H8uw7h15ZgIjA0Nqfi0kgCZVxG0/3d 7m8J6Wvun4974rLwuJnCJKmUkP0HfkTQz5JEWSx0buFUjdDUxhUP5nhjCp8lMlUiV07I 2x/7fVE65kTHtv7aQ6kFF3ApUoPwtoA9W81TKIMBnOic2R+jvblQcjqlJHerGvcRtv3j m/lg== X-Gm-Message-State: AOJu0YweIo2GG4jZ/nufNxIfALyxbhZSzC86fVutRkso/6XS7tUMSGCr CfHJjOD6IyT5JNe8D6Ki7r91t5mJyKR6jeo9ccBF/mJ9wxFErsbXmvCddu1S X-Google-Smtp-Source: AGHT+IG5ACL4Q/2MmoPXNV2oWkutSyNCViq3kQcmU5wjEwc7tftN0jDn9wF8InfAfMMzhsDH58rX8w== X-Received: by 2002:a05:690c:dd5:b0:65f:8d14:d37d with SMTP id 00721157ae682-6ad19995235mr5406827b3.1.1723606217949; Tue, 13 Aug 2024 20:30:17 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:17 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee , linux-mm@kvack.org Subject: [RFC bpf-next v3 4/7] bpf: add helper functions of pinning and converting BPF_UPTR. Date: Tue, 13 Aug 2024 20:30:07 -0700 Message-Id: <20240814033010.2980635-5-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC The helper functions manage uptrs in BPF map values from userspace. These uptrs point to user memory, so we must pin and convert them into kernel address space for BPF programs to access them in the kernel. These helper functions will be utilized by the upcoming patches. To access uptrs in BPF programs, they are pinned using pin_user_pages_fast(), but the conversion to kernel addresses is actually done by page_address(). The uptrs can be unpinned using unpin_user_pages(). Currently, the memory block pointed to by a uptr must reside in a single memory page, as crossing multiple pages is not supported. Cc: linux-mm@kvack.org Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 30 ++++++++++ kernel/bpf/helpers.c | 20 +++++++ kernel/bpf/syscall.c | 132 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 182 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 954e476b5605..886c818ff555 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -477,6 +477,8 @@ static inline void bpf_long_memcpy(void *dst, const void *src, u32 size) data_race(*ldst++ = *lsrc++); } +void bpf_obj_unpin_uptr(const struct btf_field *field, void *addr); + /* copy everything but bpf_spin_lock, bpf_timer, and kptrs. There could be one of each. */ static inline void bpf_obj_memcpy(struct btf_record *rec, void *dst, void *src, u32 size, @@ -503,6 +505,34 @@ static inline void bpf_obj_memcpy(struct btf_record *rec, memcpy(dst + curr_off, src + curr_off, size - curr_off); } +static inline void bpf_obj_uptrcpy(struct btf_record *rec, + void *dst, void *src) +{ + int i; + + if (IS_ERR_OR_NULL(rec)) + return; + + for (i = 0; i < rec->cnt; i++) { + u32 next_off = rec->fields[i].offset; + void *addr; + + if (rec->fields[i].type == BPF_UPTR) { + /* Unpin old address. + * + * Alignments are guaranteed by btf_find_field_one(). + */ + addr = *(void **)(dst + next_off); + if (addr) + bpf_obj_unpin_uptr(&rec->fields[i], addr); + + *(void **)(dst + next_off) = *(void **)(src + next_off); + } + } +} + +void copy_map_uptr_locked(struct bpf_map *map, void *dst, void *src, bool lock_src); + static inline void copy_map_value(struct bpf_map *map, void *dst, void *src) { bpf_obj_memcpy(map->record, dst, src, map->value_size, false); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d02ae323996b..d588b52605b9 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -388,6 +388,26 @@ void copy_map_value_locked(struct bpf_map *map, void *dst, void *src, preempt_enable(); } +/* Copy map value and uptr from src to dst, with lock_src indicating + * whether src or dst is locked. + */ +void copy_map_uptr_locked(struct bpf_map *map, void *src, void *dst, + bool lock_src) +{ + struct bpf_spin_lock *lock; + + if (lock_src) + lock = src + map->record->spin_lock_off; + else + lock = dst + map->record->spin_lock_off; + preempt_disable(); + __bpf_spin_lock_irqsave(lock); + copy_map_value(map, dst, src); + bpf_obj_uptrcpy(map->record, dst, src); + __bpf_spin_unlock_irqrestore(lock); + preempt_enable(); +} + BPF_CALL_0(bpf_jiffies64) { return get_jiffies_64(); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index fed4a2145f81..d504f5eb955a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -155,6 +155,138 @@ static void maybe_wait_bpf_programs(struct bpf_map *map) synchronize_rcu(); } +void bpf_obj_unpin_uptr(const struct btf_field *field, void *addr) +{ + struct page *pages[1]; + u32 size, type_id; + int npages; + void *ptr; + + type_id = field->kptr.btf_id; + btf_type_id_size(field->kptr.btf, &type_id, &size); + if (size == 0) + return; + + ptr = (void *)((intptr_t)addr & PAGE_MASK); + + npages = (((intptr_t)addr + size + ~PAGE_MASK) - (intptr_t)ptr) >> PAGE_SHIFT; + if (WARN_ON_ONCE(npages > 1)) + return; + + pages[0] = virt_to_page(ptr); + unpin_user_pages(pages, 1); +} + +/* Unpin uptr fields in the record up to cnt */ +static void bpf_obj_unpin_uptrs_cnt(struct btf_record *rec, int cnt, void *src) +{ + u32 next_off; + void **kaddr_ptr; + int i; + + for (i = 0; i < cnt; i++) { + if (rec->fields[i].type != BPF_UPTR) + continue; + + next_off = rec->fields[i].offset; + kaddr_ptr = src + next_off; + if (*kaddr_ptr) { + bpf_obj_unpin_uptr(&rec->fields[i], *kaddr_ptr); + *kaddr_ptr = NULL; + } + } +} + +/* Find all BPF_UPTR fields in the record, pin the user memory, map it + * to kernel space, and update the addresses in the source memory. + * + * The map value passing from userspace may contain user kptrs pointing to + * user memory. This function pins the user memory and maps it to kernel + * memory so that BPF programs can access it. + */ +static int bpf_obj_trans_pin_uptrs(struct btf_record *rec, void *src, u32 size) +{ + u32 type_id, tsz, npages, next_off; + void *uaddr, *kaddr, **uaddr_ptr; + const struct btf_type *t; + struct page *pages[1]; + int i, err; + + if (IS_ERR_OR_NULL(rec)) + return 0; + + if (!btf_record_has_field(rec, BPF_UPTR)) + return 0; + + for (i = 0; i < rec->cnt; i++) { + if (rec->fields[i].type != BPF_UPTR) + continue; + + next_off = rec->fields[i].offset; + if (next_off + sizeof(void *) > size) { + err = -EFAULT; + goto rollback; + } + uaddr_ptr = src + next_off; + uaddr = *uaddr_ptr; + if (!uaddr) + continue; + + /* Make sure the user memory takes up at most one page */ + type_id = rec->fields[i].kptr.btf_id; + t = btf_type_id_size(rec->fields[i].kptr.btf, &type_id, &tsz); + if (!t) { + err = -EFAULT; + goto rollback; + } + if (tsz == 0) { + *uaddr_ptr = NULL; + continue; + } + npages = (((intptr_t)uaddr + tsz + ~PAGE_MASK) - + ((intptr_t)uaddr & PAGE_MASK)) >> PAGE_SHIFT; + if (npages > 1) { + /* Allow only one page */ + err = -EFAULT; + goto rollback; + } + + /* Pin the user memory */ + err = pin_user_pages_fast((intptr_t)uaddr, 1, FOLL_LONGTERM | FOLL_WRITE, pages); + if (err < 0) + goto rollback; + + /* Map to kernel space */ + kaddr = page_address(pages[0]); + if (unlikely(!kaddr)) { + WARN_ON_ONCE(1); + unpin_user_pages(pages, 1); + err = -EFAULT; + goto rollback; + } + *uaddr_ptr = kaddr + ((intptr_t)uaddr & ~PAGE_MASK); + } + + return 0; + +rollback: + /* Unpin the user memory of earlier fields */ + bpf_obj_unpin_uptrs_cnt(rec, i, src); + + return err; +} + +static void bpf_obj_unpin_uptrs(struct btf_record *rec, void *src) +{ + if (IS_ERR_OR_NULL(rec)) + return; + + if (!btf_record_has_field(rec, BPF_UPTR)) + return; + + bpf_obj_unpin_uptrs_cnt(rec, rec->cnt, src); +} + static int bpf_map_update_value(struct bpf_map *map, struct file *map_file, void *key, void *value, __u64 flags) { From patchwork Wed Aug 14 03:30:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762808 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 396353BBD7 for ; Wed, 14 Aug 2024 03:30:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606221; cv=none; b=XzjqvdTQcG9qR+EETi5NrBPNKzL7wR97Oe+QP5UpK1JpYlNeSh7lycpAyjYad3n+ZOIbJZEEyc2DsSMQMQwgAWTkJN73bzzOT3ffJny6do+0UCnXHpHjdTJN0BgsA/iAlsl3kTUKBZseHHI5smlBNKnwKzyeqKcQGNjcwWhWWBM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606221; c=relaxed/simple; bh=3RXSElFPxPpuRa/jOxAfV9bbnHRH02MiIHG6wFsVRhg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=guWorKOOX7d2VBz6/+DWnapyBSXc8ELzmrq2zo/RR/RK7ClG46SdHwyBGeHVNYo7TSc2I3smAzYSSjJHZ9z+mPOkOOZK6n2vWDXa136b7AgqBoE1KXt+Id7XPEx08t5cITBeUnlat7Vco6pU61fhvIONYvHWmFMoWJxCwob6FdU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lOmzIFxz; arc=none smtp.client-ip=209.85.128.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lOmzIFxz" Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-685cc5415e8so63178887b3.3 for ; Tue, 13 Aug 2024 20:30:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606219; x=1724211019; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5DLFRUkWcH/DHpAQ3G3GNX5Iu0FHC4ZM55orknhqP+w=; b=lOmzIFxzZMCAgmkpUtL58LlnkfYE79ZzgNYcXFqM2p16zSXa5T2qdTqP6S7+lprNWw 4YyBTaBf8wdNRyc2z8hML5AaOoHh3M+oIhjIjh4LgA8r2Reyk+cbrr6V7AxcBxAiXqyG AwFhHgRhhJ7/jF+wwBPRQw7MH4g2i36qRA7L6RAFagQ8W1QT1+g0RSccH++2ZaWEQg71 l0OoVxl4iUM0d9h2r4VN5WbcnhHWdeTtSJZI99fuRf4D9bS3Vu+NdjKazdu3WeGAqQgh p8jR588kt+IXa9zX8CPqpoeMPA/oJkJvBVlsK9z4c3EGDXbzb6oJX+wyS5f6Ug4zqorX 7HmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606219; x=1724211019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5DLFRUkWcH/DHpAQ3G3GNX5Iu0FHC4ZM55orknhqP+w=; b=qkBzR/x+xCZQ92OiCc8qbcXPUXSVLEHer1PZXhRxBI8y1PXiocIN0hD5Dm/cTkRmzi 2D/0nxpaJV5Y1tIWR7IaT4yNY85J3q+q6ufKUK+Y+XAA1RLlXHhltBx/55r+YsGCOR8C w+NgiJlE8wn8ef26PodxEi+oIID2Ati+KIyEmjHGPwapvoXnyGDP1pA/LRHUwi7G93dY JEb5naDWugfOU0l+3HFwkaqSNCv5bNJ3L0MwTXrYErlkgpoZyI92ZV8K3wT47OSvK2DM P4UkqO/u7KTsu5AaECxiytQc1SO4vhx+gAafxma6nlhK1lUw5y5JtmvreEJj8Ohs8k16 4MPg== X-Gm-Message-State: AOJu0YwDtzpYDLNXomHVFQ2zP8HPsH5yb+8Q+LR+RDyYWgUXETSYvlv/ xmbmFAEfwhm9tMg4BUUs7LJtxV3dZt2YVtp9yi1UUKYDP7RPMYAphU1cNO8i X-Google-Smtp-Source: AGHT+IG5naNP1r+B5pmwh8q0atzk3SeDT4CJa+pjtriXsKo7BcLLR4RqGiZBj5PIyLF+Kxzk33OPFA== X-Received: by 2002:a05:690c:4501:b0:64a:7040:2d8a with SMTP id 00721157ae682-6ac96f89bfcmr16131747b3.23.1723606219067; Tue, 13 Aug 2024 20:30:19 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:18 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 5/7] bpf: pin, translate, and unpin __uptr from syscalls. Date: Tue, 13 Aug 2024 20:30:08 -0700 Message-Id: <20240814033010.2980635-6-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC When a user program updates a map value, every uptr will be pinned and translated to an address in the kernel. This process is initiated by calling bpf_map_update_elem() from user programs. Currently, uptr is only supported by task storage maps and can only be set by user programs through syscalls. When the value of an uptr is overwritten or destroyed, the memory pointed to by the old value must be unpinned. This is ensured by calling bpf_obj_uptrcpy() and copy_map_uptr_locked() when updating map value and by bpf_obj_free_fields() when destroying map value. Signed-off-by: Kui-Feng Lee --- kernel/bpf/bpf_local_storage.c | 23 ++++++++++++++----- kernel/bpf/syscall.c | 40 +++++++++++++++++++++++++++++++--- 2 files changed, 55 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index c938dea5ddbf..2fafad53b9d9 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -99,8 +99,11 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner, } if (selem) { - if (value) + if (value) { copy_map_value(&smap->map, SDATA(selem)->data, value); + if (smap->map.map_type == BPF_MAP_TYPE_TASK_STORAGE) + bpf_obj_uptrcpy(smap->map.record, SDATA(selem)->data, value); + } /* No need to call check_and_init_map_value as memory is zero init */ return selem; } @@ -575,8 +578,13 @@ bpf_local_storage_update(void *owner, struct bpf_local_storage_map *smap, if (err) return ERR_PTR(err); if (old_sdata && selem_linked_to_storage_lockless(SELEM(old_sdata))) { - copy_map_value_locked(&smap->map, old_sdata->data, - value, false); + if (smap->map.map_type == BPF_MAP_TYPE_TASK_STORAGE && + btf_record_has_field(smap->map.record, BPF_UPTR)) + copy_map_uptr_locked(&smap->map, old_sdata->data, + value, false); + else + copy_map_value_locked(&smap->map, old_sdata->data, + value, false); return old_sdata; } } @@ -607,8 +615,13 @@ bpf_local_storage_update(void *owner, struct bpf_local_storage_map *smap, goto unlock; if (old_sdata && (map_flags & BPF_F_LOCK)) { - copy_map_value_locked(&smap->map, old_sdata->data, value, - false); + if (smap->map.map_type == BPF_MAP_TYPE_TASK_STORAGE && + btf_record_has_field(smap->map.record, BPF_UPTR)) + copy_map_uptr_locked(&smap->map, old_sdata->data, + value, false); + else + copy_map_value_locked(&smap->map, old_sdata->data, + value, false); selem = SELEM(old_sdata); goto unlock; } diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index d504f5eb955a..1854aeb13ff7 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -287,8 +287,8 @@ static void bpf_obj_unpin_uptrs(struct btf_record *rec, void *src) bpf_obj_unpin_uptrs_cnt(rec, rec->cnt, src); } -static int bpf_map_update_value(struct bpf_map *map, struct file *map_file, - void *key, void *value, __u64 flags) +static int bpf_map_update_value_inner(struct bpf_map *map, struct file *map_file, + void *key, void *value, __u64 flags) { int err; @@ -340,6 +340,29 @@ static int bpf_map_update_value(struct bpf_map *map, struct file *map_file, return err; } +static int bpf_map_update_value(struct bpf_map *map, struct file *map_file, + void *key, void *value, __u64 flags) +{ + int err; + + if (map->map_type == BPF_MAP_TYPE_TASK_STORAGE) { + /* Pin user memory can lead to context switch, so we need + * to do it before potential RCU lock. + */ + err = bpf_obj_trans_pin_uptrs(map->record, value, + bpf_map_value_size(map)); + if (err) + return err; + } + + err = bpf_map_update_value_inner(map, map_file, key, value, flags); + + if (err && map->map_type == BPF_MAP_TYPE_TASK_STORAGE) + bpf_obj_unpin_uptrs(map->record, value); + + return err; +} + static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, __u64 flags) { @@ -846,6 +869,11 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj) field->kptr.dtor(xchgd_field); } break; + case BPF_UPTR: + if (*(void **)field_ptr) + bpf_obj_unpin_uptr(field, *(void **)field_ptr); + *(void **)field_ptr = NULL; + break; case BPF_LIST_HEAD: if (WARN_ON_ONCE(rec->spin_lock_off < 0)) continue; @@ -1231,7 +1259,7 @@ static int map_check_btf(struct bpf_map *map, struct bpf_token *token, map->record = btf_parse_fields(btf, value_type, BPF_SPIN_LOCK | BPF_TIMER | BPF_KPTR | BPF_LIST_HEAD | - BPF_RB_ROOT | BPF_REFCOUNT | BPF_WORKQUEUE, + BPF_RB_ROOT | BPF_REFCOUNT | BPF_WORKQUEUE | BPF_UPTR, map->value_size); if (!IS_ERR_OR_NULL(map->record)) { int i; @@ -1287,6 +1315,12 @@ static int map_check_btf(struct bpf_map *map, struct bpf_token *token, goto free_map_tab; } break; + case BPF_UPTR: + if (map->map_type != BPF_MAP_TYPE_TASK_STORAGE) { + ret = -EOPNOTSUPP; + goto free_map_tab; + } + break; case BPF_LIST_HEAD: case BPF_RB_ROOT: if (map->map_type != BPF_MAP_TYPE_HASH && From patchwork Wed Aug 14 03:30:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762809 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80380383A1 for ; Wed, 14 Aug 2024 03:30:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606222; cv=none; b=del/ts0YPvJvgYxH/Q8kEF9UBPuRLi0UDdxZu77YoiMEYSVvhBiDFrNZZVW0Gy3Ui5MdJDR/MqAFToZT1SVgiUx1NLu9UsrSv6DfjGqQldN+S8A2G9S8bl8BLmlGMtg4F0tiqtHyhUK7TyED23ak48X045aZ6lixev5n61yfbjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606222; c=relaxed/simple; bh=5nFJ7ehyCvQFnjG/IjO9tfwq3gab+NR0irsJ84qK06k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mblk2XK3ceAd7O2IIqGxR7DI4ksmWn9JHs+AiepkQaES7DyqhtaZEKCdk4mD3lUTKBWIBvrV4uiyYh2kulRNf/LY28e1Z7NjfTL9+oSUM7rFsJY+1jZbEs3jYLZJMOgWZ/PAmFMSv9JAmDRBKcLg6tF9hhMgvm1gCkMitrlUhpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Gc7T9Gc/; arc=none smtp.client-ip=209.85.128.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Gc7T9Gc/" Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-6512866fa87so55531537b3.2 for ; Tue, 13 Aug 2024 20:30:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606220; x=1724211020; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wA7s9K430JAGPCORlT7sLO0AYsuu7LQN2Z75vPP4SaA=; b=Gc7T9Gc/Q74Q8Exl30itSWWfLef3LZsK/sujZD1Yfx29w+bdU9vDBYOLVktV80axsP hkZkulGE6t9wFK1ZTopnykzWPndAxhYpyJG4cX2b5a+0BRei/rXBqd55qFpBLfwLd7wy v0zoJDK7YC0tw+p3tA70ckxkCtmS5Fjkfx6bNYE0dB9zt0ioYf5jcp7GJHHLBvrICyTR 99AS32rwwsZxs/rpa1+Dmi25z/jQLaTDntmu3PrApJlKvcX2YwRGr/KMcoE3mTzimytR 97oJKW9/speMofNeasYAA3sRLern1FufRrXRhHaELonP6Lc9n0nng6MRhoJMVZwzU/2d WztQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606220; x=1724211020; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wA7s9K430JAGPCORlT7sLO0AYsuu7LQN2Z75vPP4SaA=; b=XxlwvkHeCA/h8C02qqLHgw0M6OEYd1QN9icCZI0wsMrEj4aYeY6K1r+L5/LG0JRwZ6 id8WyD0Iiu9o1fuZu848c5LjvMbqGQ96ChqbODlIItFeYkomR5b3mIy8KVFA9jXI2NK3 j1Ji25T1KoRZeKxtdQ85RZKvDGfKfIK9PBkNqWt5S28jiPR5HQG7/rcfyvMzPxbe1whC ZDjffLq2KdIE2rfvbAnDROWHmF7pZ+dAUtpx5syL0TJGHRtviK573m9GYKcO/2MJkakk slAoh9btZmhSmUpn8vTR43fHrE4ysJO5BJwIONLyMCwEzPBBZDF6TV5PXcPF2nisby6u 1nhw== X-Gm-Message-State: AOJu0YzD0XOUzNJQP4ioJruHPZTPBuRmUc/1AzyN6b8wHeQHJkuqF2Mq dTfc73lRc5ZxH3/ThdGq49aSy9XTK0zjasg1JNJE205CvxRuz/Ae9FDsqzoG X-Google-Smtp-Source: AGHT+IHhkEU8y3rou4wvxD5yjHRSHQSxLSBZBNJ0KWmWaucD7T4SqW6lpNvzqBYiuN6r8j7MJy3lgw== X-Received: by 2002:a05:690c:2a93:b0:647:e079:da73 with SMTP id 00721157ae682-6ac97b0f293mr13299927b3.10.1723606220189; Tue, 13 Aug 2024 20:30:20 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:19 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 6/7] libbpf: define __uptr. Date: Tue, 13 Aug 2024 20:30:09 -0700 Message-Id: <20240814033010.2980635-7-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Make __uptr available to BPF programs to enable them to define uptrs. Signed-off-by: Kui-Feng Lee --- tools/lib/bpf/bpf_helpers.h | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 305c62817dd3..7ff9d947b976 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -185,6 +185,7 @@ enum libbpf_tristate { #define __kptr_untrusted __attribute__((btf_type_tag("kptr_untrusted"))) #define __kptr __attribute__((btf_type_tag("kptr"))) #define __percpu_kptr __attribute__((btf_type_tag("percpu_kptr"))) +#define __uptr __attribute__((btf_type_tag("uptr"))) #if defined (__clang__) #define bpf_ksym_exists(sym) ({ \ From patchwork Wed Aug 14 03:30:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13762810 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com [209.85.128.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 886F248CCD for ; Wed, 14 Aug 2024 03:30:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606224; cv=none; b=g76cOIS7ROB6Yfn4V3U7q+FfYQ/mcwQwStleY3XZfIAwIdnboCprmd0C42agL0JXe4iiyJsMuWORWHgVdc6Uvfic2kpthDj908jj3w4CSRebZtk6Slnt6WZFALhPLEg3kogjsTbyNceD1wZLi7/1qJg1kilb82qiDlfBa3mE96o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723606224; c=relaxed/simple; bh=D34mojUig8vZuHPyC8TaZRR0VKlsSdjtAlsD/lb7vLI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VKEBOOVmCgJ0WmKsrpdu56gL26ZNy910rMm2B1QsGHY1xuvYvKddO1MlYOn/TdXoQtZNu8A/2UN6jgj312MGbrEGDlamLzP+1teQCXUIFItO9oPeChceFe8DbcAiZTUvigUyix4htT9j6wLfwlOhCuNso4eA9syixAp9GZnH7Hc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kwH/NqxO; arc=none smtp.client-ip=209.85.128.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kwH/NqxO" Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-699d8dc6744so4169967b3.0 for ; Tue, 13 Aug 2024 20:30:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723606221; x=1724211021; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T3GlU5Fpsfnwd7Yq8PAloGm0P5P3SvGYUSsiYTJ2NW0=; b=kwH/NqxOLqRYPpgEIoQLolGIT3MImB3i96S7sa+ZlaI8AVLtguJcEBqgreI24yFosf vt4w/OkC9NDZfi55Z4kn/dLPv5Ms53+AaqizglVIeXYdIN8kZpCk74rfEfEvQoHKrzgj RRePNR0pKC8Mdkil4S3QN+9Dn73717ug5vbxocL9I/0evuzlylY3yxr9mxvMN+V1CG87 VmYYDhZTrFHd2yPZ5RGCHDtCMI3KLUt6N9xD1x535joL9t8B4vBYb07v4BfIQdwhdyJi 6RSxKJ/ANTv6Nbo7/e85J9ThUc9HQRGdwrTiYNcV3rgsIrXDAEOXfeXQQfhscGs09pYw WrZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723606221; x=1724211021; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T3GlU5Fpsfnwd7Yq8PAloGm0P5P3SvGYUSsiYTJ2NW0=; b=XrZbIADwiyrBYmOGRjCgmbsNzPw2pRczIy7wM7iJy1qdQfSIghFTAqJYrnyfbZRdi1 0MLhgOCHI8MDvItFMeWf187QYNYhH+hVEYRUqYN+4ujDZN1CmuoIp025TYYyLzyXoNDS gB/Fj5UVqBT0CCPIZdTc+LyEGBB8jOcDBrP0OpuTVCxP009+DxMDjz/zYNg2NzZ6e/kV g1Rgl3wzIZq7v7bKyjCiafjUKmpfBnGysjNtTvmtsh8gqSOc2rcx8yi4SXlViMbRdYgI nfwaR97L2WoVOdYqv+VxHEqGWsgUCCOWKHABAxZhPWP9w++6O7yZ4WnWJ98IkoiQmaFZ SWMg== X-Gm-Message-State: AOJu0YwUniUAjpDcJLdOMg+PbtrxgUyCpG66bOfB7ZtcmI8dcjh8h1D8 ykdEXlAMfs1quVqhyv06yUZxaVnoSylnT3QvYKUze+wfi2/xtDKXHIaPdamN X-Google-Smtp-Source: AGHT+IG1fN46HQC9uLwSXiCb7uAWRd38nA2uQ8LTbRKgs0aXPusyQ4q8ASS3xuzsnANY2vfkEKMthQ== X-Received: by 2002:a05:690c:b16:b0:62f:60db:326 with SMTP id 00721157ae682-6a9e9c39937mr46499737b3.20.1723606221328; Tue, 13 Aug 2024 20:30:21 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:3c23:99cc:16a9:8b68]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6a0a451b597sm15109587b3.117.2024.08.13.20.30.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 20:30:20 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [RFC bpf-next v3 7/7] selftests/bpf: test __uptr on the value of a task storage map. Date: Tue, 13 Aug 2024 20:30:10 -0700 Message-Id: <20240814033010.2980635-8-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240814033010.2980635-1-thinker.li@gmail.com> References: <20240814033010.2980635-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Make sure the memory of uptrs have been mapped to the kernel properly. Also ensure the values of uptrs in the kernel haven't been copied to userspace. Signed-off-by: Kui-Feng Lee --- .../bpf/prog_tests/task_local_storage.c | 106 ++++++++++++++++++ .../selftests/bpf/progs/task_ls_uptr.c | 65 +++++++++++ 2 files changed, 171 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/task_ls_uptr.c diff --git a/tools/testing/selftests/bpf/prog_tests/task_local_storage.c b/tools/testing/selftests/bpf/prog_tests/task_local_storage.c index c33c05161a9e..5709b083021c 100644 --- a/tools/testing/selftests/bpf/prog_tests/task_local_storage.c +++ b/tools/testing/selftests/bpf/prog_tests/task_local_storage.c @@ -5,6 +5,7 @@ #include #include #include +#include #include /* For SYS_xxx definitions */ #include #include @@ -14,6 +15,20 @@ #include "task_ls_recursion.skel.h" #include "task_storage_nodeadlock.skel.h" +struct user_data { + int a; + int b; + int result; +}; + +struct value_type { + struct user_data *udata; +}; + +#define MAGIC_VALUE 0xabcd1234 + +#include "task_ls_uptr.skel.h" + static void test_sys_enter_exit(void) { struct task_local_storage *skel; @@ -40,6 +55,95 @@ static void test_sys_enter_exit(void) task_local_storage__destroy(skel); } +static struct user_data user_data __attribute__((aligned(16))) = { + .a = 1, + .b = 2, +}; + +static void test_uptr(void) +{ + struct task_ls_uptr *skel = NULL; + int task_fd = -1, ev_fd = -1; + struct value_type value; + int err, wstatus; + __u64 dummy = 1; + pid_t pid; + + value.udata = &user_data; + + task_fd = sys_pidfd_open(getpid(), 0); + if (!ASSERT_NEQ(task_fd, -1, "sys_pidfd_open")) + goto out; + + ev_fd = eventfd(0, 0); + if (!ASSERT_NEQ(ev_fd, -1, "eventfd")) + goto out; + + skel = task_ls_uptr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open_and_load")) + goto out; + + err = bpf_map_update_elem(bpf_map__fd(skel->maps.datamap), &task_fd, &value, 0); + if (!ASSERT_OK(err, "update_datamap")) + exit(1); + + err = task_ls_uptr__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto out; + + fflush(stdout); + fflush(stderr); + + pid = fork(); + if (pid < 0) + goto out; + + /* Call syscall in the child process, but access the map value of + * the parent process in the BPF program to check if the user kptr + * is translated/mapped correctly. + */ + if (pid == 0) { + /* child */ + + /* Overwrite the user_data in the child process to check if + * the BPF program accesses the user_data of the parent. + */ + user_data.a = 0; + user_data.b = 0; + + /* Wait for the parent to set child_pid */ + read(ev_fd, &dummy, sizeof(dummy)); + + exit(0); + } + + skel->bss->parent_pid = syscall(SYS_gettid); + skel->bss->child_pid = pid; + + write(ev_fd, &dummy, sizeof(dummy)); + + err = waitpid(pid, &wstatus, 0); + ASSERT_EQ(err, pid, "waitpid"); + skel->bss->child_pid = 0; + + ASSERT_EQ(MAGIC_VALUE + user_data.a + user_data.b, + user_data.result, "result"); + + /* Check if user programs can access the value of user kptrs + * through bpf_map_lookup_elem(). Make sure the kernel value is not + * leaked. + */ + err = bpf_map_lookup_elem(bpf_map__fd(skel->maps.datamap), &task_fd, &value); + if (!ASSERT_OK(err, "bpf_map_lookup_elem")) + goto out; + ASSERT_EQ(value.udata, NULL, "lookup_udata"); + +out: + task_ls_uptr__destroy(skel); + close(ev_fd); + close(task_fd); +} + static void test_exit_creds(void) { struct task_local_storage_exit_creds *skel; @@ -237,4 +341,6 @@ void test_task_local_storage(void) test_recursion(); if (test__start_subtest("nodeadlock")) test_nodeadlock(); + if (test__start_subtest("uptr")) + test_uptr(); } diff --git a/tools/testing/selftests/bpf/progs/task_ls_uptr.c b/tools/testing/selftests/bpf/progs/task_ls_uptr.c new file mode 100644 index 000000000000..473e6890d522 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/task_ls_uptr.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include +#include +#include "task_kfunc_common.h" + +char _license[] SEC("license") = "GPL"; + +struct user_data { + int a; + int b; + int result; +}; + +struct value_type { + struct user_data __uptr *udata; +}; + +struct { + __uint(type, BPF_MAP_TYPE_TASK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct value_type); +} datamap SEC(".maps"); + +#define MAGIC_VALUE 0xabcd1234 + +/* This is a workaround to avoid clang generating a forward reference for + * struct user_data. This is a known issue and will be fixed in the future. + */ +struct user_data __dummy; + +pid_t child_pid = 0; +pid_t parent_pid = 0; + +SEC("tp_btf/sys_enter") +int BPF_PROG(on_enter, struct pt_regs *regs, long id) +{ + struct task_struct *task, *data_task; + struct value_type *ptr; + struct user_data *udata; + + task = bpf_get_current_task_btf(); + if (task->pid != child_pid) + return 0; + + data_task = bpf_task_from_pid(parent_pid); + if (!data_task) + return 0; + + ptr = bpf_task_storage_get(&datamap, data_task, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_task_release(data_task); + if (!ptr) + return 0; + + udata = ptr->udata; + if (!udata) + return 0; + udata->result = MAGIC_VALUE + udata->a + udata->b; + + return 0; +}