From patchwork Mon Apr 29 21:36:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647775 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 633561836D3 for ; Mon, 29 Apr 2024 21:36:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426575; cv=none; b=bF7buQ+b2FdzvQA9SyMxJ1NhKNJBmiQMUCBtwVJ7SAtyUXQ7vj+4X9b+Zf5IJ4wJfnf2KyQUWmldU6kX4O3MI5SoBxkL485dmy6QjJ9xLFl5o2RgIzbPpy2DN2oMntD9eAOTZNovVStotx1gy0NbC03Ab4Mp+NUegGlxtyUnQkM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426575; c=relaxed/simple; bh=Uqyx1b4ju9DUldDzncpwOrl3eanbI/lLteuJ3sVxMPc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Tbq2vk4UF6iDMp4etMSphwGNNUnElNrNzfhJryh6Br16y8OP3C770Fqnh9raQPxqmtd8idX/9ouqQ9D4uDy/mYflnuJgJuzOzSUxlhL5dm3mypWyluTHb+sYdXQwYKZzz5Zd0h7tUERIkVYNE0iySD6hOPMYcLci9yISzInZzUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Kr1OM3bY; arc=none smtp.client-ip=209.85.167.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Kr1OM3bY" Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-3c709e5e4f9so3159137b6e.3 for ; Mon, 29 Apr 2024 14:36:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426573; x=1715031373; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pOV52fyH/vltZ6XgT2t4cJ1myqxEU4aZMpQVbrAe+Yg=; b=Kr1OM3bY9AcKVSbxDetNpC70I/rV1D+r/J56xpKaPcq4ijGM74oelYIR/SNTeSnSQd XSac9pFQA9eEpMWUokh5OKCqLCLbNNci6SWMb4eb7omVHIi4QPWKkISr3T/6t7j+p2x/ O5Z73rlXN88hOIT32Ef0JZTHhdIE40+dwTQHZgJGLLE07N2bPn9DlhQ02O5XtE9AtSg8 6siiO4a2L0KrroUWvLIbGvYS7EQi5NWdIuADWDWTHHXWWDm4JuGcEUgG6F0YCnGVDPct XMgG2DuS9L7G/G9/GxY+7IoXR0drky3KUFwpH9dE2+KicJbaD4j7A0/0BLvrMrGL6tfy Lj4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426573; x=1715031373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pOV52fyH/vltZ6XgT2t4cJ1myqxEU4aZMpQVbrAe+Yg=; b=nxunA91Am97NgKg6SwHAvuSoQaDPSb9bGTFW73CuNcW1pddlM+fhkbGbK0b3yLYS6y bz9DiJwaDHkhsFwzCNcC9ZRphDrazJpE85a/H7TC9ZQafu26fMgqKe0xOZyaxMm9fU8G 2A2Wf3PPoMw4cRmJVJib0uleIGLt7JEqIFw5VJJL3e3F2NxY+phDzPuJxhNzPfVMNVEO naT4onNupRmEtodXlLT7xBdHjKtVYccejNyPPaWGdVtaMfjQlgESSJgVXOzObiD+rmWK JHX9wvTp+k3cNWmJRzfn6KFcra/XYZDBQZG3BfA0gLLVYgytSC0QqF+N0mXypAJLtSWn iqJw== X-Gm-Message-State: AOJu0YxFQqDKCGbTbS+Y8Jr0OQU6vpYG2eKayPXeaSI0uOfzXrCnVVwa pGB9VM7XbGiWUlvXXnlK9Zl9FDIMdN89cr4eucO7E42aBRrPBOZizZbf6g== X-Google-Smtp-Source: AGHT+IE9zzXuVRTvCW8pq3+0O6Q+a6gcNCnLwlUgSF0rJnc+D9l4cl6791uXh6Hj7ZOsUfKl1I0vjw== X-Received: by 2002:a05:6808:e85:b0:3c8:531f:ca40 with SMTP id k5-20020a0568080e8500b003c8531fca40mr15106373oil.35.1714426573302; Mon, 29 Apr 2024 14:36:13 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:12 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 1/6] bpf: add a pointer of the attached link to bpf_struct_ops_map. Date: Mon, 29 Apr 2024 14:36:04 -0700 Message-Id: <20240429213609.487820-2-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net To facilitate the upcoming unregistring struct_ops objects from the systems consuming objects, a pointer of the attached link is added to allow for accessing the attached link of a bpf_struct_ops_map directly from the map itself. Previously, a st_map could be attached to multiple links. This patch now enforces only one link attached at most. Signed-off-by: Kui-Feng Lee --- kernel/bpf/bpf_struct_ops.c | 47 ++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 86c7884abaf8..072e3416c987 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -20,6 +20,8 @@ struct bpf_struct_ops_value { #define MAX_TRAMP_IMAGE_PAGES 8 +struct bpf_struct_ops_link; + struct bpf_struct_ops_map { struct bpf_map map; struct rcu_head rcu; @@ -39,6 +41,8 @@ struct bpf_struct_ops_map { void *image_pages[MAX_TRAMP_IMAGE_PAGES]; /* The owner moduler's btf. */ struct btf *btf; + /* The link is attached by this map. */ + struct bpf_struct_ops_link __rcu *attached; /* uvalue->data stores the kernel struct * (e.g. tcp_congestion_ops) that is more useful * to userspace than the kvalue. For example, @@ -1048,6 +1052,22 @@ static bool bpf_struct_ops_valid_to_reg(struct bpf_map *map) smp_load_acquire(&st_map->kvalue.common.state) == BPF_STRUCT_OPS_STATE_READY; } +/* Set the attached link of a map. + * + * Return the current value of the st_map->attached. + */ +static inline struct bpf_struct_ops_link *map_attached(struct bpf_struct_ops_map *st_map, + struct bpf_struct_ops_link *st_link) +{ + return unrcu_pointer(cmpxchg(&st_map->attached, NULL, st_link)); +} + +/* Reset the attached link of a map */ +static inline void map_attached_null(struct bpf_struct_ops_map *st_map) +{ + rcu_assign_pointer(st_map->attached, NULL); +} + static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) { struct bpf_struct_ops_link *st_link; @@ -1061,6 +1081,7 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) * bpf_struct_ops_link_create() fails to register. */ st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); + map_attached_null(st_map); bpf_map_put(&st_map->map); } kfree(st_link); @@ -1125,9 +1146,21 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map goto err_out; } + if (likely(st_map != old_st_map) && map_attached(st_map, st_link)) { + /* The map is already in use */ + err = -EBUSY; + goto err_out; + } + err = st_map->st_ops_desc->st_ops->update(st_map->kvalue.data, old_st_map->kvalue.data); - if (err) + if (err) { + if (st_map != old_st_map) + map_attached_null(st_map); goto err_out; + } + + if (likely(st_map != old_st_map)) + map_attached_null(old_st_map); bpf_map_inc(new_map); rcu_assign_pointer(st_link->map, new_map); @@ -1172,20 +1205,28 @@ int bpf_struct_ops_link_create(union bpf_attr *attr) } bpf_link_init(&link->link, BPF_LINK_TYPE_STRUCT_OPS, &bpf_struct_ops_map_lops, NULL); + if (map_attached(st_map, link)) { + err = -EBUSY; + goto err_out; + } + err = bpf_link_prime(&link->link, &link_primer); if (err) - goto err_out; + goto err_out_attached; err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data); if (err) { bpf_link_cleanup(&link_primer); + /* The link has been free by bpf_link_cleanup() */ link = NULL; - goto err_out; + goto err_out_attached; } RCU_INIT_POINTER(link->map, map); return bpf_link_settle(&link_primer); +err_out_attached: + map_attached_null(st_map); err_out: bpf_map_put(map); kfree(link); From patchwork Mon Apr 29 21:36:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647776 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC2CD184106 for ; Mon, 29 Apr 2024 21:36:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426577; cv=none; b=Gx26Nwn42GJsmIOkjxbHRnX9czk5qphLWJz/GEzHW+Bn2uIslJ5IUU5ZFFONSmhJO6iv9dT7fol3RIKi2x2qzFJVbTGDVCtSz0NUruIFG+TfK19Q6qIV4JkVxpGdm64Wtw8PHoMPSf7uupIHmy9QhVSRiYJS1wAU1YB8oHfR0Gc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426577; c=relaxed/simple; bh=KR3nntE0MSV0xmElGXWcN2MeqwUZwsO4XLNYJPvTbIs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B78gVbQT1fwz6IEDteE5iAbU+sxB/2bhtZdVNuNALBjzg94DeqT8w/ncKTLoaF/Ty2ZmbwHKPhOlwBYQCHW9xMe+bcVSs83MyakkphwwZXruta9tYvPQovpw/fI417kuY2o+NWnWfOzjd0zWRa4hJMp+ixnW3wSCqSrlEBbr0mY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fQAoWCbn; arc=none smtp.client-ip=209.85.167.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fQAoWCbn" Received: by mail-oi1-f178.google.com with SMTP id 5614622812f47-3c844b6edbbso3018389b6e.1 for ; Mon, 29 Apr 2024 14:36:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426574; x=1715031374; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tiPf+3XkCJLmBitbMVDk2ZnNXM0bO3q3Bz9cKB/E+HA=; b=fQAoWCbnmZv9MUjXZbTAVKZg6Ak1GgbgKme7kOna59rta6UYoRA0zT1JUC7D/zRfc7 eorVuO4ViFS6ezwgXAonY/DHRRS6+yBTPCT0P0iroIiT2Kdn/Nvs1m2Prv6AOvbgBEBr cmYjCMZrzZ/RdM/M8sfpo61RqIbVF/x4S56vLttF9UomxFNZ7N9bx+b/QXmKlQuvfskU GfnyYbjgapL9myKPCckCazhv2/QJCpA81W0uhNbe4xW5rBvNEFqQc45EU6K0sIfN3gtC uoIfE0AyAzPBAspNMFngP8yANknry77vlMOYG22Rt0WnKnyIwtQJc2JBGZXwTpKVtS+a 8NQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426574; x=1715031374; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tiPf+3XkCJLmBitbMVDk2ZnNXM0bO3q3Bz9cKB/E+HA=; b=vjqgPpHv25GXU5BWLmbQlk2CiUxLaa2MpX/BpdyQeZ5QOi+2ylbQS9JOovMBOrXb2D owQrCNnY0326NP3qeBJ96j+VskaIehlhLKcJ8cB91BHuXBE6dy6n/JdxYMB2+B9eZIQP KNPchMlNvn08jPDfQMQE1OrcnRJfsPnB88wUFSOzHf/P/WAHsVl05L4+WxXH/TkfpS57 e81YEV4ysf3/2SXlUJS3of+ES0CgfgwmjNoRAvaVSlYB90YXDAH5H3FvYWIo1RAX49+6 dzCLwKl/25N+SrRcOid3yYed6AdfpXg1got2bIedy1pPfuojMOnS7O+keEZJGw7IJOz8 c33w== X-Gm-Message-State: AOJu0YyX9JKWTzGkK2ftZihKmoMBpjWWJ2rbb4Qo6GrgguGF8IlM4FuV jCthZ990TSsksTybWoW2JgES+/nPqUdBFoBwKVMbOnvH9WKNek4VmaR48Q== X-Google-Smtp-Source: AGHT+IGHBsGhoUnJbogxRfTNrzCvIDjSEDw6VH8OUcfgXfdQarpvCCUJoKKtoWCsy1dB8SEMOU1iGg== X-Received: by 2002:a05:6808:1887:b0:3c7:5091:54e5 with SMTP id bi7-20020a056808188700b003c7509154e5mr15881779oib.21.1714426574543; Mon, 29 Apr 2024 14:36:14 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:14 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 2/6] bpf: export bpf_link_inc_not_zero(). Date: Mon, 29 Apr 2024 14:36:05 -0700 Message-Id: <20240429213609.487820-3-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Make bpf_link_inc_not_zero() available to be used by bpf_struct_ops.c later. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 5034c1b4ded7..8a1500764332 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2324,6 +2324,7 @@ int bpf_link_prime(struct bpf_link *link, struct bpf_link_primer *primer); int bpf_link_settle(struct bpf_link_primer *primer); void bpf_link_cleanup(struct bpf_link_primer *primer); void bpf_link_inc(struct bpf_link *link); +struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link); void bpf_link_put(struct bpf_link *link); int bpf_link_new_fd(struct bpf_link *link); struct bpf_link *bpf_link_get_from_fd(u32 ufd); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 7d392ec83655..4a2f95c4b2ac 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -5364,7 +5364,7 @@ static int link_detach(union bpf_attr *attr) return ret; } -static struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link) +struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link) { return atomic64_fetch_add_unless(&link->refcnt, 1, 0) ? link : ERR_PTR(-ENOENT); } From patchwork Mon Apr 29 21:36:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647777 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08DDF1836D3 for ; Mon, 29 Apr 2024 21:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426578; cv=none; b=FQ1ZPew7awp7evCoVQpnC75+c1IsN0Z+pJ9AnKNQX6kccbAjtMpHofUz9Q/SC0Kpmi9W/ev11zVj3Q5YTm6GNWeKa3s4nZfLzqWYlfzTuT12FbVEsRF7l6iywQWo/0qbNGthGh59wKtMM7PQKSzJG1Iw4ulvStoJZPEx89O5dTI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426578; c=relaxed/simple; bh=4uakBWvaHA1KxlgcBwgGIfbSSivWPoJn0awGMiKF3PU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IPHj24+BIY2fm/ci3FHacX3kwlR6/Kd4hrAZjA8KNYkdQ00GbnNaRLwW1rOD+yQu06awH45q01vSfQOenO1zt58bcCb2I79EqSyiG6q6D2GOI6ZIpPt1E9jbQHZfNfQX7g52F500C2mpnYYcOjP+CUkLL68D/+ShEsbEduaIMJo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S1FZ9ZkZ; arc=none smtp.client-ip=209.85.167.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S1FZ9ZkZ" Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-3c8641b41e7so1031127b6e.3 for ; Mon, 29 Apr 2024 14:36:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426575; x=1715031375; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JTnbJwGNcO/RFWMVGphrMvNsaylBPZQXpHG6wXSmgfw=; b=S1FZ9ZkZqR6yOVO06MlJGk7nsFScfUW9Ll9jjwhGRP2kNRafUwV56xt7lOm5UiIIhh azlwXPYVK4Y+RVnU8yOImZNlp2n7MpWkf7ox0COUpHGIKGFrMYqjLFm2MTwau4y+nkCc +xOZozEZbagaXPvojFStdtxeBU5BMB9M5D7nXMdQDFwW+8dKSa6ggoafTkhJmFV1GTjM js7/d1SOFxZnScy7gdn0XuI2gE4TUqRXWjuHrsQf+4MM+/Rd488IDDN2iMq8KBe+b/l4 byQY+1S+7eF3n652n/J+ObBFJUJHOuRXZkHX4EStcuOqsZ22VsFsLXA6zObEVuSulKGj vKsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426576; x=1715031376; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JTnbJwGNcO/RFWMVGphrMvNsaylBPZQXpHG6wXSmgfw=; b=TQv6ASRB2XAsZdsP9uQZt6cIWpcYYQZ/BPQS76PHNaiu0jLlyaQqwLXVxxmXoB8T/d LvN2DDyBRA/ZnRdU6MnRzqZEiGbSPCopoLHA+0zEvkRhnBUj8pQh7YY7lWrykHyK4abt zwP5lr7syo14VMAYWp20mPgMoa3+TCuSeC8QRObBDdZVxK5ahOV19OJGhDDIjKOzb8tC 167MyFcB8EPcIvEQhP6MBTWGUEb30d4BbPt9BjqhfY+hgd1cFnZghrb+N6bkYwA80QeE /k/LipJvXUduNl+r2yGDJ/BrRl+Yr2joRzmtaPFU4EkkkNW2NBihtVPNMYI0WZddT14A Ecew== X-Gm-Message-State: AOJu0YzOIezNdRWGtumZZi5wXLqsopNQKp3GhTsJdZsJTExKH5/FwN7Z tDOUZf0oH730nsnJl++BWwOFu82EdtdWB1WybZ9ZLUDsLsTkNNAiwN057w== X-Google-Smtp-Source: AGHT+IFdXfEjMzMa14okT2LtWsnbYfAG2UcSHCK1peFu+3Nl3EPd6b+BlpPOUFfiEJ+bISA1W7LpOA== X-Received: by 2002:aca:2814:0:b0:3c7:306f:c7b with SMTP id 20-20020aca2814000000b003c7306f0c7bmr1096273oix.19.1714426575561; Mon, 29 Apr 2024 14:36:15 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:15 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 3/6] bpf: provide a function to unregister struct_ops objects from consumers. Date: Mon, 29 Apr 2024 14:36:06 -0700 Message-Id: <20240429213609.487820-4-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net bpf_struct_ops_kvalue_unreg() unregisters the struct_ops map specified by the pointer passed in. A subsystem could use this function to unregister a struct_ops object that was previously registered to it. In effect, bpf_struct_ops_kvalue_unreg() detaches the corresponding st_map of an object from the link if there is one. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 6 +++ kernel/bpf/bpf_struct_ops.c | 97 ++++++++++++++++++++++++++++++++++--- 2 files changed, 97 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 8a1500764332..eeeed4b1bd32 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1793,6 +1793,7 @@ static inline void bpf_module_put(const void *data, struct module *owner) module_put(owner); } int bpf_struct_ops_link_create(union bpf_attr *attr); +bool bpf_struct_ops_kvalue_unreg(void *data); #ifdef CONFIG_NET /* Define it here to avoid the use of forward declaration */ @@ -1843,6 +1844,11 @@ static inline void bpf_struct_ops_desc_release(struct bpf_struct_ops_desc *st_op { } +static inline bool bpf_struct_ops_kvalue_unreg(void *data) +{ + return false; +} + #endif #if defined(CONFIG_CGROUP_BPF) && defined(CONFIG_BPF_LSM) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 072e3416c987..8e79b02a1ccb 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -1077,9 +1077,6 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) st_map = (struct bpf_struct_ops_map *) rcu_dereference_protected(st_link->map, true); if (st_map) { - /* st_link->map can be NULL if - * bpf_struct_ops_link_create() fails to register. - */ st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); map_attached_null(st_map); bpf_map_put(&st_map->map); @@ -1087,6 +1084,83 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) kfree(st_link); } +/* Called from the subsystem that consume the struct_ops. + * + * The caller should protected this function by holding rcu_read_lock() to + * ensure "data" is valid. However, this function may unlock rcu + * temporarily. The caller should not rely on the preceding rcu_read_lock() + * after returning from this function. + * + * Return true if unreg() success. If a call fails, it means some other + * task has unrgistered or is unregistering the same object. + */ +bool bpf_struct_ops_kvalue_unreg(void *data) +{ + struct bpf_struct_ops_map *st_map = + container_of(data, struct bpf_struct_ops_map, kvalue.data); + enum bpf_struct_ops_state prev_state; + struct bpf_struct_ops_link *st_link; + bool ret = false; + + /* The st_map and st_link should be protected by rcu_read_lock(), + * or they may have been free when we try to increase their + * refcount. + */ + if (IS_ERR(bpf_map_inc_not_zero(&st_map->map))) + /* The map is already gone */ + return false; + + prev_state = cmpxchg(&st_map->kvalue.common.state, + BPF_STRUCT_OPS_STATE_INUSE, + BPF_STRUCT_OPS_STATE_TOBEFREE); + if (prev_state == BPF_STRUCT_OPS_STATE_INUSE) { + st_map->st_ops_desc->st_ops->unreg(data); + /* Pair with bpf_map_inc() for reg() */ + bpf_map_put(&st_map->map); + /* Pair with bpf_map_inc_not_zero() above */ + bpf_map_put(&st_map->map); + return true; + } + if (prev_state != BPF_STRUCT_OPS_STATE_READY) + goto fail; + + /* With BPF_F_LINK */ + + st_link = rcu_dereference(st_map->attached); + if (!st_link || !bpf_link_inc_not_zero(&st_link->link)) + /* The map is on the way to unregister */ + goto fail; + + rcu_read_unlock(); + mutex_lock(&update_mutex); + + if (rcu_dereference_protected(st_link->map, true) != &st_map->map) + /* The map should be unregistered already or on the way to + * be unregistered. + */ + goto fail_unlock; + + st_map->st_ops_desc->st_ops->unreg(data); + + map_attached_null(st_map); + rcu_assign_pointer(st_link->map, NULL); + /* Pair with bpf_map_get() in bpf_struct_ops_link_create() or + * bpf_map_inc() in bpf_struct_ops_map_link_update(). + */ + bpf_map_put(&st_map->map); + + ret = true; + +fail_unlock: + mutex_unlock(&update_mutex); + rcu_read_lock(); + bpf_link_put(&st_link->link); +fail: + bpf_map_put(&st_map->map); + return ret; +} +EXPORT_SYMBOL_GPL(bpf_struct_ops_kvalue_unreg); + static void bpf_struct_ops_map_link_show_fdinfo(const struct bpf_link *link, struct seq_file *seq) { @@ -1096,7 +1170,8 @@ static void bpf_struct_ops_map_link_show_fdinfo(const struct bpf_link *link, st_link = container_of(link, struct bpf_struct_ops_link, link); rcu_read_lock(); map = rcu_dereference(st_link->map); - seq_printf(seq, "map_id:\t%d\n", map->id); + if (map) + seq_printf(seq, "map_id:\t%d\n", map->id); rcu_read_unlock(); } @@ -1109,7 +1184,8 @@ static int bpf_struct_ops_map_link_fill_link_info(const struct bpf_link *link, st_link = container_of(link, struct bpf_struct_ops_link, link); rcu_read_lock(); map = rcu_dereference(st_link->map); - info->struct_ops.map_id = map->id; + if (map) + info->struct_ops.map_id = map->id; rcu_read_unlock(); return 0; } @@ -1134,6 +1210,10 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map mutex_lock(&update_mutex); old_map = rcu_dereference_protected(st_link->map, lockdep_is_held(&update_mutex)); + if (!old_map) { + err = -EINVAL; + goto err_out; + } if (expected_old_map && old_map != expected_old_map) { err = -EPERM; goto err_out; @@ -1214,14 +1294,19 @@ int bpf_struct_ops_link_create(union bpf_attr *attr) if (err) goto err_out_attached; + /* Init link->map before calling reg() in case being unregistered + * immediately. + */ + RCU_INIT_POINTER(link->map, map); + err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data); if (err) { + rcu_assign_pointer(link->map, NULL); bpf_link_cleanup(&link_primer); /* The link has been free by bpf_link_cleanup() */ link = NULL; goto err_out_attached; } - RCU_INIT_POINTER(link->map, map); return bpf_link_settle(&link_primer); From patchwork Mon Apr 29 21:36:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647778 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 573A6184106 for ; Mon, 29 Apr 2024 21:36:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426579; cv=none; b=Sl0ka7YugMqIMnv7diREAgKh6ynRb7xpBxKOCYBFoBPIhDLZshopZ6dUra6w2LKM/7v3BfCQZhz/tRYeq8gHhHyaW0MJyNRPoRZTfTeNgvosVrbxW9j04LYaCVTNcHLKlXOZueseyhst0Gx3KE2Zv15QHwAvKQzGa0i0NgKlIFA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426579; c=relaxed/simple; bh=qPWMaoapf8etuzRHFlLLredcVqscaPwHU3Og4UJsWKQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bG7UzZu8vlE5CCU7usHRK6az28Yam2yXCvAjevH2jd/pgwLKSRDIhOTHBcUszFDHkj/BsjYbXZLNjAtqnEgZPI2ismzeFcsIosPuZHyBdllOo0XaZ9jOZTYQCFCblywsdE0ZWdSjf5VTEnc0Fq+x8dYlGrgei1wJ8s6aOSwrgow= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FooPZrTj; arc=none smtp.client-ip=209.85.210.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FooPZrTj" Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-6eb55942409so2418691a34.1 for ; Mon, 29 Apr 2024 14:36:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426577; x=1715031377; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+SyQcBUr1RoB63injLEssF5Dc8uqHZAPs77gRcJceqo=; b=FooPZrTjp8yBL0Vt3njGul57z1Zvb33SZCcOHsw6Pp3M561KMZ8c3zBksMDs74iAKc R9nxH5Wr5ZIpN8uNJeeV60Qd+qAPaA5qleWlot8RiOrKzyoZQaPuk7Ci0hQAjd171qTt E/QcjECVn9FtHyZZe/HdBiX1IjE8H8WrKiG7lJliICigxiZPSM68j3kfIOgXUFy/+Pmf GGrGzrFAIUoSNFM4qu/hr4zfsIdOyYHyT4yiu0jcScREpKkrnCA/sbWEhcXeV2/4fobt HWBo0a7aPf8Pj1PQRde8JeB0pRDM4c1iTWilBpyKuzmKLKQP7IXTtpW2boSkjHQiYXov Of0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426577; x=1715031377; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+SyQcBUr1RoB63injLEssF5Dc8uqHZAPs77gRcJceqo=; b=Z5UR8fatT4+nowdbjY9CtDXtzCAwOsOlbBjM/27Orcx6RsGb1j7coLMC4qJf9dUWkW iK+2ecl8eZ59OSzfOUrT5+brLU9ksZFOWAGAsdyfZfYsRbrrEvPbz6H2WmcH86xQoJzf SwaZ7BtMvvRlNRmiSEYQdw6Ax1hEczWEj+9z0RWsJLwlqzcSEK1ZdwFEizBUxbyx4DFE bevmCYLYuCF3p3m/eA1spcZBYEhfx+Cmb9BtcetLHzzuesUr5RrtwUrKWgRA5bBQkpA9 EBLujxV/D109fcyT7TVyaWGQn8et6ZP51GpFB/ZwSUczUi5tAKs9TBWDr9WduEWvd7tp dTTQ== X-Gm-Message-State: AOJu0YxZ5CqCT0MLxadyZ+HmishxC+KqoC6MR6OlmIYoMLXIf+g1gav2 I92q62TV+kdweBUeIx6iCJNzwcn5inFiS36hmJtDnNUBribrnBB6VkrHoA== X-Google-Smtp-Source: AGHT+IGSvpRcOUmGZbBouxAesp79MVsHG1Co9AzGYKXloTIQLa46GaZ+9gk7EiEJwhexu4o2a06wjw== X-Received: by 2002:a05:6808:608f:b0:3c7:2364:e506 with SMTP id de15-20020a056808608f00b003c72364e506mr1114811oib.29.1714426576882; Mon, 29 Apr 2024 14:36:16 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:16 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 4/6] bpf: detach a bpf_struct_ops_map from a link. Date: Mon, 29 Apr 2024 14:36:07 -0700 Message-Id: <20240429213609.487820-5-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Allow user space programs to detach a struct_ops map from the attached link. Signed-off-by: Kui-Feng Lee --- kernel/bpf/bpf_struct_ops.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 8e79b02a1ccb..4a8a7e5ffc56 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -1252,8 +1252,36 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map return err; } +static int bpf_struct_ops_map_link_detach(struct bpf_link *link) +{ + struct bpf_struct_ops_link *st_link; + struct bpf_struct_ops_map *st_map; + struct bpf_map *map; + + mutex_lock(&update_mutex); + + st_link = container_of(link, struct bpf_struct_ops_link, link); + map = rcu_dereference_protected(st_link->map, lockdep_is_held(&update_mutex)); + if (!map) { + mutex_unlock(&update_mutex); + return -ENOENT; + } + st_map = container_of(map, struct bpf_struct_ops_map, map); + + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); + + map_attached_null(st_map); + rcu_assign_pointer(st_link->map, NULL); + bpf_map_put(map); + + mutex_unlock(&update_mutex); + + return 0; +} + static const struct bpf_link_ops bpf_struct_ops_map_lops = { .dealloc = bpf_struct_ops_map_link_dealloc, + .detach = bpf_struct_ops_map_link_detach, .show_fdinfo = bpf_struct_ops_map_link_show_fdinfo, .fill_link_info = bpf_struct_ops_map_link_fill_link_info, .update_map = bpf_struct_ops_map_link_update, From patchwork Mon Apr 29 21:36:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647779 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AA771836D3 for ; Mon, 29 Apr 2024 21:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426580; cv=none; b=D0g/mxPRrB5TfK/9sVRzLB/Ct4lTWVuUCX+CqtfihWp6ibdxjmdeOtulfjn5jaml51sNGNlTodxH2s2zHqdX5/ZkHkdozLx28XbnqpP/zoiNKVK8qZ/7BfdnDMaom3JSjg6U7VFcwgHTFf+Yg+/JLsR8ByBHF3M8qAZUFCMZb80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426580; c=relaxed/simple; bh=Z2LkQHTsgWCWz/Q/PwJpz3s4i9iXX/Tm62a7jtcZ9YY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=p3w0SJrNBQSbzbjAfnuP6WVPZ1ds1vTNbX8PsVeR5y50Bu4Li5AzizD7PjB1wK2WBbxd0xG7azmSAb3QKSwG8tOjgG6RR2eZjwSZPhpJs4g8f2675ZQ/0luFnQ3tQXk+p75ghI57BXe9HHn5O3+ESwML56V+MhcH1vBNHY1v2/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=M2XjgASh; arc=none smtp.client-ip=209.85.167.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M2XjgASh" Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3c70e46f3caso3021271b6e.2 for ; Mon, 29 Apr 2024 14:36:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426578; x=1715031378; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YdRBves7cYltUiulBFm9u6fqywNCkscd2bo/pr1PFdU=; b=M2XjgAShGf+bX7PlxCjPF70xqyPa9ukLJMpSInJ+2vv0p5zrSxdqBGVIzwplA8KivU W+h9VTCGILsXaW83hhJ0ZZZnBNQTX0E5YKzqiup8PV/coDqQztZElvcoMppcI8V/7VE1 mpLYqgUvwGF3WCMbC1MSt9dbyEagrKCcs0+5m+/pNgXBUV335Ey9I+tosgbTmgNAhAN5 obBNRey8q/DzJ3Mzk2BcaFZj9T4gg6xofOrcVH4L6Un0hg0v33jSXrXb57mMkJbjU9Zq ekR3pzayVtdy1im98jDXjBkalh3VWu1meDW3SLTQ0Ropq1Rph+jEDrr/Ub2s+20i7+6e WMrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426578; x=1715031378; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YdRBves7cYltUiulBFm9u6fqywNCkscd2bo/pr1PFdU=; b=qx9dHcnY7obJ101RsPhrgdId+xJTWQCjWrQCIYiX8UEiuILk5zJvgjb3revuGtd5hh XioET1/qqmVNnmPGHbLlBeCTZ/KAj+lgsNQhtZYoTuvpaRZFAJBL6O5Xe6UfH3gqV9y0 5BezV/nBC6po7PmCuW2ZCBkcVGItSUIBJYNCdjKBhkyP+j66FBiyjn01BsM2ML0qQ7XU 4x8bUnM8hCzXT5jQHKAqFF8lFXvyT2xmo5VLmi+n58JUGEZEwzRdWTJUBLeatYk4ck9p lo5afze/SkFZCF6e8IT98AgDgJNjG5h2A7yHEXrWGRRcKHfcmtFhywKC6bImbfN+jULK sjnA== X-Gm-Message-State: AOJu0Yz4r8EdV/odqlL8yWLGc+69YdBC4QYdhWBV91TT1dhfKjKCoH3W /2fSTVEVq0Xlbod8+Cna1AbNZbf3TqKcsstLn/xMhbGrYTJLN1fvMby7cQ== X-Google-Smtp-Source: AGHT+IFUAViiagLy3qu1uxkemLfnUfebGP5gF9AJbQe7nY1uM9rn9B+J0YCqteFXQU+V6e8V1i+3+Q== X-Received: by 2002:a05:6808:18a3:b0:3c7:5084:ad17 with SMTP id bi35-20020a05680818a300b003c75084ad17mr15254657oib.43.1714426577956; Mon, 29 Apr 2024 14:36:17 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:17 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 5/6] bpf: support epoll from bpf struct_ops links. Date: Mon, 29 Apr 2024 14:36:08 -0700 Message-Id: <20240429213609.487820-6-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add epoll support to bpf struct_ops links to trigger EPOLLHUP event upon detachment. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 2 ++ kernel/bpf/bpf_struct_ops.c | 14 ++++++++++++++ kernel/bpf/syscall.c | 15 +++++++++++++++ 3 files changed, 31 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index eeeed4b1bd32..a4550b927352 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1574,6 +1574,7 @@ struct bpf_link { const struct bpf_link_ops *ops; struct bpf_prog *prog; struct work_struct work; + wait_queue_head_t wait_hup; }; struct bpf_link_ops { @@ -1587,6 +1588,7 @@ struct bpf_link_ops { struct bpf_link_info *info); int (*update_map)(struct bpf_link *link, struct bpf_map *new_map, struct bpf_map *old_map); + __poll_t (*poll)(struct file *file, struct poll_table_struct *pts); }; struct bpf_tramp_link { diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 4a8a7e5ffc56..f19b6a76591a 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -12,6 +12,7 @@ #include #include #include +#include struct bpf_struct_ops_value { struct bpf_struct_ops_common_value common; @@ -1149,6 +1150,8 @@ bool bpf_struct_ops_kvalue_unreg(void *data) */ bpf_map_put(&st_map->map); + wake_up_interruptible_poll(&st_link->link.wait_hup, EPOLLHUP); + ret = true; fail_unlock: @@ -1276,15 +1279,26 @@ static int bpf_struct_ops_map_link_detach(struct bpf_link *link) mutex_unlock(&update_mutex); + wake_up_interruptible_poll(&st_link->link.wait_hup, EPOLLHUP); + return 0; } +static __poll_t bpf_struct_ops_map_link_poll(struct file *file, + struct poll_table_struct *ptrs) +{ + struct bpf_struct_ops_link *st_link = file->private_data; + + return (st_link->map) ? 0 : EPOLLHUP | EPOLLERR; +} + static const struct bpf_link_ops bpf_struct_ops_map_lops = { .dealloc = bpf_struct_ops_map_link_dealloc, .detach = bpf_struct_ops_map_link_detach, .show_fdinfo = bpf_struct_ops_map_link_show_fdinfo, .fill_link_info = bpf_struct_ops_map_link_fill_link_info, .update_map = bpf_struct_ops_map_link_update, + .poll = bpf_struct_ops_map_link_poll, }; int bpf_struct_ops_link_create(union bpf_attr *attr) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 4a2f95c4b2ac..b4dbca04d4f5 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2990,6 +2990,7 @@ void bpf_link_init(struct bpf_link *link, enum bpf_link_type type, link->id = 0; link->ops = ops; link->prog = prog; + init_waitqueue_head(&link->wait_hup); } static void bpf_link_free_id(int id) @@ -3108,6 +3109,19 @@ static void bpf_link_show_fdinfo(struct seq_file *m, struct file *filp) } #endif +static __poll_t bpf_link_poll(struct file *file, struct poll_table_struct *pts) +{ + struct bpf_link *link = file->private_data; + + if (link->ops->poll) { + poll_wait(file, &link->wait_hup, pts); + + return link->ops->poll(file, pts); + } + + return 0; +} + static const struct file_operations bpf_link_fops = { #ifdef CONFIG_PROC_FS .show_fdinfo = bpf_link_show_fdinfo, @@ -3115,6 +3129,7 @@ static const struct file_operations bpf_link_fops = { .release = bpf_link_release, .read = bpf_dummy_read, .write = bpf_dummy_write, + .poll = bpf_link_poll, }; static int bpf_link_alloc_id(struct bpf_link *link) From patchwork Mon Apr 29 21:36:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13647780 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B885184124 for ; Mon, 29 Apr 2024 21:36:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426581; cv=none; b=Z+cZuEY+V7iERSqmLMNK5796PuYUEi+3bUO+GsrCw9/rijQmoiFyljDGvemMDKLPlG6VFyuy77e1gbb1Q1JQLQJKlBnLSSxwgnBz/83THQS2eBY7TaL2X6SR9BmpciTBAzAQm/C8WacnF/yFN5H5vyn70vvPTEY30ll77/guqnI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714426581; c=relaxed/simple; bh=I/HQdIzxClxpiYMOP+mYxG3ROF0r7hwbDSt5dUNidLo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ByeO4Rcv0M1HEGwOViK2wiy0RCY0kBbG+oJwC6g8k0IgasYTCULU34JGibL3wGttqOm7rFxTnObydjX75n3zVn6TsDwQNLIMJbqEDKbbrCBksZTwby4gx9wuzIiUCCPVBGWzKvsAHXXvmPTReInvN4G8o6rUxG4YnukJGEhs/Zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LdkMAbnP; arc=none smtp.client-ip=209.85.167.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LdkMAbnP" Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3c844b6edbbso3018420b6e.1 for ; Mon, 29 Apr 2024 14:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714426579; x=1715031379; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TQI0s/zOku12P4kv+jI56KO8d44pJV8Wjk4rHVkZmnw=; b=LdkMAbnPV8Eo7XQh9+TvAabeCeeMsCEroVVvGGIwjBD9ZbrRGPnWKcPdH5UMWtWSqD 5n8zHl3XIkh16dhhp+LwxzQHYwrR8Ov8QU3RUe5hbkFarSiRJJ88tTttF9A68KKOtcga 3CwUC0lcECNZHNrk87Wtes3KB5DYSMtzQrtZqN23SXZmaqzpLlq/cy2GBr6hSaOYwLEh sVwrDXrf33rj/C+yj9UzBUNlN2q2h5zHDD2bhV9zF4MLMPXNtBCeBAkb/WawBeoRoMwr eM+//yjRSWHLJwQ7SwAAu/0sBtktIkQA5FYrYbWgNBao+/bcQwfynJMjjgS//LIwGe6U crKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714426579; x=1715031379; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TQI0s/zOku12P4kv+jI56KO8d44pJV8Wjk4rHVkZmnw=; b=DyacqcIii/GUFiZ41JZOF9iT7hAvLhCWgXBhA4V7KCZA4xihO6vSlSwjE7DADYCZRJ XdHf1HuKtC5M1cHcCHyXCnhnaai12Zxc9YHV3Zxqp5zO5YOtAz2z+Vqgk5cPVzd1a/AT 7Pxz+cpknwj0W9ijns7Ijjggy5fnVIZ1Zh4TlD6/ixlH7LllKUhEcauIuxrzxmkLzWvP ZenxJOu9UPaYdLichD6tYoUHsg2bSFMjiZxvD5ImbxdBocmhQ/4OoAaDPo7cr0vgOa3t /SQ9mGieJFJ6mA67q6N+V8tDC/TR1H0/aTgpUTb5bbJN2co4Pugda3tJnaz0sjOE9UkX Ljiw== X-Gm-Message-State: AOJu0YymVrhVm7erDz+4/45SJ4aIfK8BbG+59DirhwPe66dZZIqVdmdO 7QazfuNDLkxcVLMqQo9g7DJ2pcgOrhC3D/AXqB2W/kZ1JiW2bTxpfpEXZA== X-Google-Smtp-Source: AGHT+IFYonbBO6nCTjwLruMcb9WeXKTbPlfMQwIOOppMX1c+zlfzWKL9z1JA1P8m6D2LqPqEtUjV3Q== X-Received: by 2002:a05:6808:211d:b0:3c8:5493:e25f with SMTP id r29-20020a056808211d00b003c85493e25fmr14579023oiw.12.1714426578955; Mon, 29 Apr 2024 14:36:18 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:b805:4ca7:fd75:4bf]) by smtp.gmail.com with ESMTPSA id x5-20020a05680801c500b003c8642321c9sm714034oic.50.2024.04.29.14.36.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 14:36:18 -0700 (PDT) From: Kui-Feng Lee To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next 6/6] selftests/bpf: test detaching struct_ops links. Date: Mon, 29 Apr 2024 14:36:09 -0700 Message-Id: <20240429213609.487820-7-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240429213609.487820-1-thinker.li@gmail.com> References: <20240429213609.487820-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Verify whether a user space program is informed through epoll with EPOLLHUP when a struct_ops object is detached or unregistered using the function bpf_struct_ops_kvalue_unreg() or BPF_LINK_DETACH. Signed-off-by: Kui-Feng Lee --- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 18 ++- .../selftests/bpf/bpf_testmod/bpf_testmod.h | 1 + .../bpf/prog_tests/test_struct_ops_module.c | 104 ++++++++++++++++++ .../selftests/bpf/progs/struct_ops_module.c | 7 ++ 4 files changed, 126 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 39ad96a18123..e526ccfad8bf 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -539,16 +539,20 @@ static int bpf_testmod_ops_init_member(const struct btf_type *t, const struct btf_member *member, void *kdata, const void *udata) { + struct bpf_testmod_ops *ops = kdata; + if (member->offset == offsetof(struct bpf_testmod_ops, data) * 8) { /* For data fields, this function has to copy it and return * 1 to indicate that the data has been handled by the * struct_ops type, or the verifier will reject the map if * the value of the data field is not zero. */ - ((struct bpf_testmod_ops *)kdata)->data = ((struct bpf_testmod_ops *)udata)->data; - return 1; - } - return 0; + ops->data = ((struct bpf_testmod_ops *)udata)->data; + } else if (member->offset == offsetof(struct bpf_testmod_ops, do_unreg) * 8) { + ops->do_unreg = ((struct bpf_testmod_ops *)udata)->do_unreg; + } else + return 0; + return 1; } static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = { @@ -572,6 +576,12 @@ static int bpf_dummy_reg(void *kdata) if (ops->test_2) ops->test_2(4, ops->data); + if (ops->do_unreg) { + rcu_read_lock(); + bpf_struct_ops_kvalue_unreg(kdata); + rcu_read_unlock(); + } + return 0; } diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h index 23fa1872ee67..ee8d4a2cd187 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h @@ -43,6 +43,7 @@ struct bpf_testmod_ops { int b; } unsupported; int data; + bool do_unreg; /* The following pointers are used to test the maps having multiple * pages of trampolines. diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c index 7cf2b9ddd3e1..b4d3b29114ca 100644 --- a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c @@ -3,6 +3,8 @@ #include #include +#include + #include "struct_ops_module.skel.h" static void check_map_info(struct bpf_map_info *info) @@ -160,6 +162,104 @@ static void test_struct_ops_incompatible(void) struct_ops_module__destroy(skel); } +static void test_detach_link(void) +{ + struct epoll_event ev, events[2]; + struct struct_ops_module *skel; + struct bpf_link *link = NULL; + int fd, epollfd = -1, nfds; + int err; + + skel = struct_ops_module__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open_and_load")) + return; + + link = bpf_map__attach_struct_ops(skel->maps.testmod_1); + if (!ASSERT_OK_PTR(link, "attach_struct_ops")) + goto cleanup; + + fd = bpf_link__fd(link); + if (!ASSERT_GE(fd, 0, "link_fd")) + goto cleanup; + + epollfd = epoll_create1(0); + if (!ASSERT_GE(epollfd, 0, "epoll_create1")) + goto cleanup; + + ev.events = EPOLLHUP; + ev.data.fd = fd; + err = epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, &ev); + if (!ASSERT_OK(err, "epoll_ctl")) + goto cleanup; + + err = bpf_link__detach(link); + if (!ASSERT_OK(err, "detach_link")) + goto cleanup; + + nfds = epoll_wait(epollfd, events, 2, 500); + if (!ASSERT_EQ(nfds, 1, "epoll_wait")) + goto cleanup; + + if (!ASSERT_EQ(events[0].data.fd, fd, "epoll_wait_fd")) + goto cleanup; + if (!ASSERT_TRUE(events[0].events & EPOLLHUP, "events[0].events")) + goto cleanup; + +cleanup: + close(epollfd); + bpf_link__destroy(link); + struct_ops_module__destroy(skel); +} + +/* Test bpf_struct_ops_kvalue_unreg() */ +static void test_do_unreg(void) +{ + struct epoll_event ev, events[2]; + struct struct_ops_module *skel; + struct bpf_link *link = NULL; + int fd, epollfd = -1, nfds; + int err; + + skel = struct_ops_module__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open_and_load")) + return; + + /* bpf_testmod will unregister this map immediately through the + * function bpf_struct_ops_kvalue_unreg() since "do_unreg" is true. + */ + link = bpf_map__attach_struct_ops(skel->maps.testmod_do_unreg); + if (!ASSERT_OK_PTR(link, "attach_struct_ops")) + goto cleanup; + + fd = bpf_link__fd(link); + if (!ASSERT_GE(fd, 0, "link_fd")) + goto cleanup; + + epollfd = epoll_create1(0); + if (!ASSERT_GE(epollfd, 0, "epoll_create1")) + goto cleanup; + + ev.events = EPOLLHUP; + ev.data.fd = fd; + err = epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, &ev); + if (!ASSERT_OK(err, "epoll_ctl")) + goto cleanup; + + nfds = epoll_wait(epollfd, events, 2, 500); + if (!ASSERT_EQ(nfds, 1, "epoll_wait")) + goto cleanup; + + if (!ASSERT_EQ(events[0].data.fd, fd, "epoll_wait_fd")) + goto cleanup; + if (!ASSERT_TRUE(events[0].events & EPOLLHUP, "events[0].events")) + goto cleanup; + +cleanup: + close(epollfd); + bpf_link__destroy(link); + struct_ops_module__destroy(skel); +} + void serial_test_struct_ops_module(void) { if (test__start_subtest("test_struct_ops_load")) @@ -168,5 +268,9 @@ void serial_test_struct_ops_module(void) test_struct_ops_not_zeroed(); if (test__start_subtest("test_struct_ops_incompatible")) test_struct_ops_incompatible(); + if (test__start_subtest("test_detach_link")) + test_detach_link(); + if (test__start_subtest("test_do_unreg")) + test_do_unreg(); } diff --git a/tools/testing/selftests/bpf/progs/struct_ops_module.c b/tools/testing/selftests/bpf/progs/struct_ops_module.c index 63b065dae002..7a697a7dd0ac 100644 --- a/tools/testing/selftests/bpf/progs/struct_ops_module.c +++ b/tools/testing/selftests/bpf/progs/struct_ops_module.c @@ -81,3 +81,10 @@ struct bpf_testmod_ops___incompatible testmod_incompatible = { .test_2 = (void *)test_2, .data = 3, }; + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_do_unreg = { + .test_1 = (void *)test_1, + .test_2 = (void *)test_2, + .do_unreg = true, +};