From patchwork Tue Jan 28 08:46:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952306 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 999F21E521; Tue, 28 Jan 2025 08:46:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054004; cv=none; b=m5QkMRWNoM8zuVnhrtNcVatdXiXVX+ePmmJZR/W0RNmUAYAueHRkCYGjVeFnYfDoHdqm3BIX2CCFMmbw8XTfGNKHLx8XOpWH9JYLxZ3w9WoUK4GD6XeXmrsWigNQekTS7M/xWxwu9GFKU8hQFv/fgz3fz7gfcTGilqI8y218WGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054004; c=relaxed/simple; bh=wrbYn3EK4NGao4rFHvVEf04A0D3Y5+FtZCFu0/lzvec=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZnsJSFXgbC52e1/qlUKBb3RwB0Ub851Oyt9h+jBtrjxc325/aZXvfa5XS8inovfqvt3xy6/trJWgCFV6zjhfOV51XsSCKXI7fNUsa/Nm7eyrvsKXHTllhiZAF/jdgrQw6xusCM9Z97nzKAuJASwq02a7bmOKjkQnlDaCNS+pQNg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AORW5FDB; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AORW5FDB" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-216401de828so97129535ad.3; Tue, 28 Jan 2025 00:46:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054002; x=1738658802; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bEQ9weNA2CyTmGDk3/BYG9T6+PKgmDlsp/plMw8ftEE=; b=AORW5FDBOFFNi24d+PUbxMYhu9ud9siEhSoLHiqxOLfM/oIxXfUwYKzs6dF07ASRhT UXdlOLWLoZPU2S/lDtN4Ss2E+ur/F6AKEUdbsnbcKpC3EG9QlMw7Ud0y7UGbXmtK4iis qVKVA9P9GknqYh9ft7xKxi59YxiAverIL4LZw8tu2YRF5JFle34yh+5n2wTO9zly6M97 oj48ybQgej3uRJ+dFnm7JDcj9CQgj4VCYxWBs57CrNAu9pwZfZBAyyVmD7nu9/8ufThy lRKbMLTmihy4grZ8Dwa06uqiE2o0uckax049wuBRti9SZtpXPtkFfZjP8KlgDL5d/bRA zl/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054002; x=1738658802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bEQ9weNA2CyTmGDk3/BYG9T6+PKgmDlsp/plMw8ftEE=; b=m6rJrIs82joetgujfgXPsJ2Q+3ZoKLMdf4P2nvzni4of9URwrXL4dt9qyzZlHcR6q1 Zg1LrIx5rXHA4/a5jw6HCNhUIU8UZcmRdQhmN+lemeVtrZw4IluN7yPYXgMW34ndrRvx HwNU3m+akkuwvIeAePi5dOEn8I3S3oEOiBUDJqvSrMCCTp6ToU01rQdR/VE3BcfGz2Km e994dSLeyqXMBq/48FOsXSHcoICqdCUDAcYgzvPHDRSeyFNnlqc5fRoyRkBp+3UcX1J2 RDZbDQdsQdJWV36xMff9IuL2/+ZIm8odLACv5Djy5sfxWl5y6O0iT6bHw0DQ1pWsdjlH Zh7g== X-Forwarded-Encrypted: i=1; AJvYcCUoNrl0zXf5b0nWfKDqe6EWeJHgrwWLvE3GVEhrp53zGGCKFQxnIdZL39BQ22ghP8DQBK/gQvQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1QG9gUPYXx+tPbKXt6Z9uj/zXRfZ23/OTKvdK6ggGoMPn2dy9 wWIgfOwrh1TY2X4sgaY/TUFjgaPIKcoCrqLEzbD+HnXh0FxATUBBs3bBSTTVs50= X-Gm-Gg: ASbGncu7cybuaoznl1la7H/hq0mBxxpDZNkvFOZPyH3uYD9rbkc3MUsMpfAxlxpeKJG GhiagP4vIOXFMtoiPf1vG8vkZ93w6Y/U/wKPJO8iiraxYG+F7TlVk9hpDMqBBt1NczlmK0O5HnI ojPyn2h61d5ZolrfcR/CcvcdjJYI/R0y3XkX4JPBuNuE6PCIFfl5ULckSybHTjRWNjF+wZ162GF Ir1uqTdIcS2U6HCoAuD+3lRxIZW6CstkZ3fC091ERWJ9bFvMWYFrLU4NrDHCXIcspYxRgb+HVU0 4AeTUP1LA8PkTYyYcDXHijrhtV6xs7lRcSkfX4EZj70tbax72C4mag== X-Google-Smtp-Source: AGHT+IF4hUdT+Q2G6c+vUpbwu9O87JppL5I67ZUk3t65MgkG6I3J6peyc1vWRpImgLU05TvzYAY5fA== X-Received: by 2002:a17:902:d48f:b0:216:282d:c697 with SMTP id d9443c01a7336-21c35544097mr616315645ad.27.1738054001706; Tue, 28 Jan 2025 00:46:41 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.46.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:46:41 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 01/13] net-timestamp: add support for bpf_setsockopt() Date: Tue, 28 Jan 2025 16:46:08 +0800 Message-Id: <20250128084620.57547-2-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Users can write the following code to enable the bpf extension: bpf_setsockopt(skops, SOL_SOCKET, SK_BPF_CB_FLAGS, &flags, sizeof(flags)); Signed-off-by: Jason Xing --- include/net/sock.h | 3 +++ include/uapi/linux/bpf.h | 8 ++++++++ net/core/filter.c | 23 +++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 35 insertions(+) diff --git a/include/net/sock.h b/include/net/sock.h index 8036b3b79cd8..7916982343c6 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -303,6 +303,7 @@ struct sk_filter; * @sk_stamp: time stamp of last packet received * @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only * @sk_tsflags: SO_TIMESTAMPING flags + * @sk_bpf_cb_flags: used in bpf_setsockopt() * @sk_use_task_frag: allow sk_page_frag() to use current->task_frag. * Sockets that can be used under memory reclaim should * set this to false. @@ -445,6 +446,8 @@ struct sock { u32 sk_reserved_mem; int sk_forward_alloc; u32 sk_tsflags; +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG)) + u32 sk_bpf_cb_flags; __cacheline_group_end(sock_write_rxtx); __cacheline_group_begin(sock_write_tx); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 2acf9b336371..6116eb3d1515 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6913,6 +6913,13 @@ enum { BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F, }; +/* Definitions for bpf_sk_cb_flags */ +enum { + SK_BPF_CB_TX_TIMESTAMPING = 1<<0, + SK_BPF_CB_MASK = (SK_BPF_CB_TX_TIMESTAMPING - 1) | + SK_BPF_CB_TX_TIMESTAMPING +}; + /* List of known BPF sock_ops operators. * New entries can only be added at the end */ @@ -7091,6 +7098,7 @@ enum { TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */ + SK_BPF_CB_FLAGS = 1009, /* Used to set socket bpf flags */ }; enum { diff --git a/net/core/filter.c b/net/core/filter.c index 2ec162dd83c4..1c6c07507a78 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5222,6 +5222,25 @@ static const struct bpf_func_proto bpf_get_socket_uid_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +static int sk_bpf_set_get_cb_flags(struct sock *sk, char *optval, bool getopt) +{ + u32 sk_bpf_cb_flags; + + if (getopt) { + *(u32 *)optval = sk->sk_bpf_cb_flags; + return 0; + } + + sk_bpf_cb_flags = *(u32 *)optval; + + if (sk_bpf_cb_flags & ~SK_BPF_CB_MASK) + return -EINVAL; + + sk->sk_bpf_cb_flags = sk_bpf_cb_flags; + + return 0; +} + static int sol_socket_sockopt(struct sock *sk, int optname, char *optval, int *optlen, bool getopt) @@ -5238,6 +5257,7 @@ static int sol_socket_sockopt(struct sock *sk, int optname, case SO_MAX_PACING_RATE: case SO_BINDTOIFINDEX: case SO_TXREHASH: + case SK_BPF_CB_FLAGS: if (*optlen != sizeof(int)) return -EINVAL; break; @@ -5247,6 +5267,9 @@ static int sol_socket_sockopt(struct sock *sk, int optname, return -EINVAL; } + if (optname == SK_BPF_CB_FLAGS) + return sk_bpf_set_get_cb_flags(sk, optval, getopt); + if (getopt) { if (optname == SO_BINDTODEVICE) return -EINVAL; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 2acf9b336371..70366f74ef4e 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7091,6 +7091,7 @@ enum { TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */ + SK_BPF_CB_FLAGS = 1009, /* Used to set socket bpf flags */ }; enum { From patchwork Tue Jan 28 08:46:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952307 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAC1B1DED73; Tue, 28 Jan 2025 08:46:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054009; cv=none; b=bGWRSXu/HGrJfi0aSr+00KHZFyS1K5Y8i2z6TbpIddp4bwX3Y8yD18bhND1NKYTyYKo+q3y82ehSsiIqsr+bMisu0lGeAbU6lNF/2u/YhplG9kJfaTAZ51Yx4Tt2F0xdsBLap6UMecaV70bIevUTGJz+vn03X2s453eDl1+NdPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054009; c=relaxed/simple; bh=9wh26/MXjHI/IdCoIuwtAX4FzIaDdiPAMzB0iAcmueM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Se2Jyj6GrId5fjOa0l9fnVqUQkz/JN1RuUReVnE7sBcyIamh2Rss+T4Z34UpxH9s7BzteJ//pAqJ3Ohyzf5Ts6EMFoRZMDMOy2p4LsdX1iBr1lYqKavBUh7jRr1zWFkUM4L68hXJsaXqX0Ond7c3xLZt8sp1wYqBnOOkpJW3j4Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=l0rYk6tr; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="l0rYk6tr" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-21675fd60feso118992725ad.2; Tue, 28 Jan 2025 00:46:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054007; x=1738658807; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KGmDd20z27/LS1vdWXNTJekDTBka5SLdUJCvGwv1Mqo=; b=l0rYk6tr3EFgS6j+E7iSE+MJj01fNTBxxTmQIyIcr6hn2Q8px9m7zloimbtAHPDqQo bBY2/oxoB5RWCMcQK0l92hmskaCxWbAS6fLxCv5XqDUzzcFqxJjzYMILMKyw4LMMUjW5 4611cGF2ih8V81qbLCgvAfrYlIiAxYyqUYZOCTzTPOS8s1Xyr7D/H+tqQAP10sNRg5wO jt93Rvtr7B9QoNXsjInBnQHqWDKbfCz+onKssHZb93vATVzlHlNRsjQR7UB9e2+7jYPK gITscBgOoMAlMXb6FaVd/Fbi2+dya9gRG+e62JS+5BsIN7Iz1PkpwIwgqYcn9Fqqe0kS YjSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054007; x=1738658807; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KGmDd20z27/LS1vdWXNTJekDTBka5SLdUJCvGwv1Mqo=; b=IkzgjTjxqApHFYYxzCYGOD1wrZKf964H+5HiJG2pDg6eyVUQRIOT/9GMMgLlQPInNX ho3NXcQHUSJXDM7ojM+fGemY6axqD0wjHCwtZmV2kbxCeXxsrEJqCJrp7bD6vhF9MeSx ye8SD9CCEB92Vp3S4SfYYOOFl2jouhPh/j+An606hPfVZMTnlGbGFykWdp/m6QVEmfal g4qiTgPMLSGZnTp+UGvd/kvQurFWTX/Qo64+X34kp2YCLwYMyNLQh7UqOLVgL6l7qs22 PdZywtNSkDjP0yRipIB87evqZFdhfqt41i1+b4VYiCcwyBNZReLxbqmg3MjgjRpXgiNu TAcw== X-Forwarded-Encrypted: i=1; AJvYcCVytbKEsKKneAVXGUC3HmJzqVQh42hJuLJhYjcHyZg80raA7DpGHF7BbwFdb9BCuwIFk8PlvJE=@vger.kernel.org X-Gm-Message-State: AOJu0YylWVC3ynXATN63QDQzILQKxPx9egK2YjK5Ug58YYo8kj8sfp59 WBBy17JPmr0cYTqAIct66B/r501qWDMBIT3StIDaCqxsq8WNW2egpw8MDR3Yi2k= X-Gm-Gg: ASbGncvkuidEi1i2AsDfTW1Dsuxt9CEr9+hd7jUuJ9QXudNHuVGUv5fxhLzAzrOt0PX mWxAJb5BxDL25JpESSY2LBSdzmn23TAQVggDZxADoTAfKE721H34Lnrny13N9oPnqh7tB01Xdce tiAtd35bJapX3TRTL/lXBzrJTK8x6I9uE9T0e/U0FF4cOrYbnNsjchVg38MT1LwBimiuGerql9i 6D1DklxmOaSNmUd7z0QFtRyOWmfuTBURlfsbjHmbaC7yyJjQc/+4y0zM9D+W2ZuECeMsz9cQRGW 2uF2EkfUsZObA8mzKmeUTAoikTaX5VNqYXQRAqq8dUIrxHT4gzROaA== X-Google-Smtp-Source: AGHT+IHNuKAw2dWKiRqkrBGbYZtOUvt6LdU9GJGk4vFwLSscNliHmlKwL8YlwCg7eMs3GppJESOOfA== X-Received: by 2002:a17:903:186:b0:216:69ca:772a with SMTP id d9443c01a7336-21c3563ee3fmr748349215ad.53.1738054006889; Tue, 28 Jan 2025 00:46:46 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:46:46 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 02/13] net-timestamp: prepare for timestamping callbacks use Date: Tue, 28 Jan 2025 16:46:09 +0800 Message-Id: <20250128084620.57547-3-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Later, I would introduce four callback points to report information to user space based on this patch. As to skb initialization here, people can follow these three steps as below to fetch the shared info from the exported skb in the bpf prog: 1. skops_kern = bpf_cast_to_kern_ctx(skops); 2. skb = skops_kern->skb; 3. shinfo = bpf_core_cast(skb->head + skb->end, struct skb_shared_info); More details can be seen in the last selftest patch of the series. Signed-off-by: Jason Xing --- include/net/sock.h | 7 +++++++ net/core/sock.c | 15 +++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/net/sock.h b/include/net/sock.h index 7916982343c6..6f4d54faba92 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2923,6 +2923,13 @@ int sock_set_timestamping(struct sock *sk, int optname, struct so_timestamping timestamping); void sock_enable_timestamps(struct sock *sk); +#if defined(CONFIG_CGROUP_BPF) +void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op); +#else +static inline void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op) +{ +} +#endif void sock_no_linger(struct sock *sk); void sock_set_keepalive(struct sock *sk); void sock_set_priority(struct sock *sk, u32 priority); diff --git a/net/core/sock.c b/net/core/sock.c index eae2ae70a2e0..41db6407e360 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -948,6 +948,21 @@ int sock_set_timestamping(struct sock *sk, int optname, return 0; } +#if defined(CONFIG_CGROUP_BPF) +void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op) +{ + struct bpf_sock_ops_kern sock_ops; + + memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); + sock_ops.op = op; + sock_ops.is_fullsock = 1; + sock_ops.sk = sk; + bpf_skops_init_skb(&sock_ops, skb, 0); + /* Timestamping bpf extension supports only TCP and UDP full socket */ + __cgroup_bpf_run_filter_sock_ops(sk, &sock_ops, CGROUP_SOCK_OPS); +} +#endif + void sock_set_keepalive(struct sock *sk) { lock_sock(sk); From patchwork Tue Jan 28 08:46:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952308 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E76451DC994; Tue, 28 Jan 2025 08:46:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054014; cv=none; b=bfwtrDnchaxmta/l6pLbB8QTW7vQlOwnjJLGA/vjifUO36QmhxgHKBVbb/gG+dd/i9dj3gXk1cQYw4Y+YMAxI9i5WbTJXIU+eCI43Eiyfv0Cson3swzQqmaX3DoU3E+GsJIXFXy/r3pZrRY7sOmbbfZi37nCs3XXgBvjhC612zU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054014; c=relaxed/simple; bh=M2E4pG3pbPE82kxEJw8nQ4OUbiqFhi/0Xn+Hz7M6Qgo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tXdueZCum2cVTG80wREWW5R1lGuPxxJjCKIXGtLez0BbV1dFVgSGZ9Wl8Rzz+jFoma89t8K9UxiC59kU+yeKLjHfApgQGfO/670l/JfLmeV51Jfgy7x6otPYvkAbQw7bF9RlcRKGEzp5AMRcZn1fFmf3+94kV6/ZeoUHBhuiELs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IhLa1oME; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IhLa1oME" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-216281bc30fso19133115ad.0; Tue, 28 Jan 2025 00:46:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054012; x=1738658812; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hfGMmgvbumdhb3rVXrvLEWEq/NKmW/MgCi6J1BYZIt8=; b=IhLa1oMEjVHQjwnVAnMnPgiBzHhNNsESmHuFAYY3g8BnLX556SujNXvlXPqEFy3Q0Q p3K3RHk8nLtttFJIiI5vEc5juQ6KjSiBtvx7q56X6TfaIkZeoDUtHjYbnghMqWmTUqfD sJOm0eEWSJ0DfeioLd1ef6W36PF0ZGIzc98UKroIfOIxBd66ijSqKixy/jp+h0umkfEx oAS5KxXeHJw5oObvcAtH+G+gT/9fO4a24fdJuWbqka341+vSzl1mbbkM9p6V6+WgkpGB br3HgqiYqbccM3jfaDQ8Uym/eb/0WtkickyYeFj2a0IcsdcHmM6WhMsYg4umDAwWIZRl coAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054012; x=1738658812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hfGMmgvbumdhb3rVXrvLEWEq/NKmW/MgCi6J1BYZIt8=; b=qjmEhishpViewrRkjr76esArjaOiQHGUCbvsZEzwThwqoDH9HGkZulMLLj+7SkOp0Z NvHGs1SDtWkN2vADmWkfKoZMO/VLvLX3CRcLKMF5LoypFQpi9F0NUUFdV/B6PVen+OFS CmLPifsySoRVFVdrlz20L3YpjKdF3iivrDwIp67utn4lQIwxcgETqS43i4PgNHP6+lwa 6PxeGk1JZEu1EdI6Npm6GYzDC3yiiiu4DTasdMoH67Ipx3r+5HvfzL+J/qMid01J5UkV jBiW/LODJ3rhnkz3TI6tBj4VpSIQjLKuOZABbKWbSqux5xmP/FrIobULxp7CWV9y13Z/ q5LA== X-Forwarded-Encrypted: i=1; AJvYcCXAijDHZd5nCPCdk9Te1JsyF9sqcR6eLwC1aj1ZdQOdbkEzrUoJSSOAhatLjQrRZ7DbxphDCkw=@vger.kernel.org X-Gm-Message-State: AOJu0YyvvhOwjfhyzpnZ7R02AGO0ubrDa4wrXTKtHGCuI7VPA/qYtX0E cKFy+hqY0Aduuvt6STPpOglwJpymqpwwNaXotjYEIjKp4eiuUy05 X-Gm-Gg: ASbGncubfNi/bNTrGr3Q7s3cJZkWsT2IbUPQTZ5BVcm9GTdjuksdTczvh6LGmx4gVjh yP3Ljt5VMuJlOzX8UxyCuwzAKC3pQq9/b2FCJR9DIqv+kW4pztudmX7JgPMmScd37pbFjEpUKmQ eY883PtbmvIDgWnNNF7W3xfyEC6RllJEbjlHvPGVtHGXAm33LRlZRktND4E+9YLr9JH6d3AWs8k L3I5t8s5n29Nvrke26T21UyQmrTWGiQzt2wDhA9WewlnqJrvJGv5sUngNTSFEjIQQPCVlVTbSda A4X5A13Nh/kwg0gWK0PmkoA3jUx4PvOJulqveIDy+B+tOzBs5d5+PA== X-Google-Smtp-Source: AGHT+IFgJjjUcQLw4Ei6BKrKXmw3ZwpIzC1uCEX4VfueXD/mftnBhrBzvhQwyx4+6hZKgLvbyNdwFQ== X-Received: by 2002:a17:902:f681:b0:216:1cfa:2bda with SMTP id d9443c01a7336-21c3564858emr700146585ad.43.1738054012064; Tue, 28 Jan 2025 00:46:52 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:46:51 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 03/13] bpf: stop unsafely accessing TCP fields in bpf callbacks Date: Tue, 28 Jan 2025 16:46:10 +0800 Message-Id: <20250128084620.57547-4-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The "allow_tcp_access" flag is added to indicate that the callback site has a tcp_sock locked. Applying the new member allow_tcp_access in the existing callbacks where is_fullsock is set to 1 can help us stop UDP socket accessing struct tcp_sock and stop TCP socket without sk lock protecting does the similar thing, or else it could be catastrophe leading to panic. To keep it simple, instead of distinguishing between read and write access, we disallow all read/write access to the tcp_sock through the older bpf_sock_ops ctx. The new timestamping callbacks can use newer helpers to read everything from a sk (e.g. bpf_core_cast), so nothing is lost. Signed-off-by: Jason Xing --- include/linux/filter.h | 5 +++++ include/net/tcp.h | 1 + net/core/filter.c | 8 ++++---- net/ipv4/tcp_input.c | 2 ++ net/ipv4/tcp_output.c | 2 ++ 5 files changed, 14 insertions(+), 4 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index a3ea46281595..1569e9f31a8c 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -1508,6 +1508,11 @@ struct bpf_sock_ops_kern { void *skb_data_end; u8 op; u8 is_fullsock; + u8 allow_tcp_access; /* Indicate that the callback site + * has a tcp_sock locked. Then it + * would be safe to access struct + * tcp_sock. + */ u8 remaining_opt_len; u64 temp; /* temp and everything after is not * initialized to 0 before calling diff --git a/include/net/tcp.h b/include/net/tcp.h index 5b2b04835688..293047694710 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -2649,6 +2649,7 @@ static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args) memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); if (sk_fullsock(sk)) { sock_ops.is_fullsock = 1; + sock_ops.allow_tcp_access = 1; sock_owned_by_me(sk); } diff --git a/net/core/filter.c b/net/core/filter.c index 1c6c07507a78..dc0e67c5776a 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -10381,10 +10381,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, } \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ - is_fullsock), \ + allow_tcp_access), \ fullsock_reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, \ - is_fullsock)); \ + allow_tcp_access)); \ *insn++ = BPF_JMP_IMM(BPF_JEQ, fullsock_reg, 0, jmp); \ if (si->dst_reg == si->src_reg) \ *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ @@ -10469,10 +10469,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, temp)); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ - is_fullsock), \ + allow_tcp_access), \ reg, si->dst_reg, \ offsetof(struct bpf_sock_ops_kern, \ - is_fullsock)); \ + allow_tcp_access)); \ *insn++ = BPF_JMP_IMM(BPF_JEQ, reg, 0, 2); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, sk),\ diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index eb82e01da911..77185479ed5e 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -169,6 +169,7 @@ static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb) memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); sock_ops.op = BPF_SOCK_OPS_PARSE_HDR_OPT_CB; sock_ops.is_fullsock = 1; + sock_ops.allow_tcp_access = 1; sock_ops.sk = sk; bpf_skops_init_skb(&sock_ops, skb, tcp_hdrlen(skb)); @@ -185,6 +186,7 @@ static void bpf_skops_established(struct sock *sk, int bpf_op, memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); sock_ops.op = bpf_op; sock_ops.is_fullsock = 1; + sock_ops.allow_tcp_access = 1; sock_ops.sk = sk; /* sk with TCP_REPAIR_ON does not have skb in tcp_finish_connect */ if (skb) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 0e5b9a654254..695749807c09 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -522,6 +522,7 @@ static void bpf_skops_hdr_opt_len(struct sock *sk, struct sk_buff *skb, sock_owned_by_me(sk); sock_ops.is_fullsock = 1; + sock_ops.allow_tcp_access = 1; sock_ops.sk = sk; } @@ -567,6 +568,7 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb, sock_owned_by_me(sk); sock_ops.is_fullsock = 1; + sock_ops.allow_tcp_access = 1; sock_ops.sk = sk; } From patchwork Tue Jan 28 08:46:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952309 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 227631DF73D; Tue, 28 Jan 2025 08:46:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054019; cv=none; b=P/PtyvlGGicH7huW68aB/IFQrOM18T69MqQZMInKIsv28aHdRhh1oz04Nzh7W/kRSBFe1o6DxFQ7mJ+dN36QtDHcq2eSgxFnGsjPwMDnaPmvPktekjMMsmubozSgIy44vrl1LzMEmNZ/yBWaCqzCu7nuiDCrFSmDDRop6xIK5LM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054019; c=relaxed/simple; bh=6gjefGTlSgHBj050nZsylf4QqygtaEzfR2BlPPxYz58=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Nr1HTlCKOW9T2TDAZXEGAa6EWtxC69oXVAqePrMwHq25YBr2QAoTDYyaEzHyrM/dm/IRHjMCE4nzqnhEIqSusSutRlE/AYn/UcinHPEIJvt7i2PO/yMVguowfPvN7WXE4VTs7wJ0Eut9LRTtpQQGOibFD108kuRYMuBctZBzfP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ho6FYe5u; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ho6FYe5u" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-21670dce0a7so111644805ad.1; Tue, 28 Jan 2025 00:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054017; x=1738658817; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=edPp/qi7ns9gzyrhgsfr5YC2VgBldDClUoe6GDZO/bk=; b=ho6FYe5uoGbPozDHa+dZvteLE53oAwqCcW2dRO14Bo7V+gCMlz+i6TuqFcySJ/vsyP eWqUae2zTHJ5NuvL78wP1hszdvWLbdUA4sZdRdJXEFjX6Sm5B5S98dWkOKiFuwoM3P9T DVjAgjQI/VqlInojBMdcWOr+nlI9SrnVBjH+wPVVcFJRsPJ+YEQA8mZQ5V6bslgM21Fw Zn69rA7CPMBvzUsjLXBwzMOBXuJ8z28zavAhZzXln3H7ECq4mcujL2RHn8RnyqxKYOpZ /edHnr/rNFDloBaw60h4a7jg3Qq+ACEIiZZ0VHqJvtart1+W83EVE0JjfQm0U3WlTl1e rLPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054017; x=1738658817; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=edPp/qi7ns9gzyrhgsfr5YC2VgBldDClUoe6GDZO/bk=; b=P/Gj6as32cYyEOwOgjHq/gt9JnqEk0IEA/s4dbWinRhCAbAzTBVLAIHHW8mFlDOd8i fkdzXR0WRxH/xGRdBMck7aqz9XdoIw9lgsiUaqlYtAIeAr+vKR71VDrL4beLK6h2ZiQd Reu+Yai8m+uDNYbqA1Qflv6qqpdwwTVLD74wO8d3xS/oC6QkiNfzndvQubpWEm5b+LW/ tlQEQ5rvF8K2PTQ0KSvC7dGCP+XwuavIehNJb/idLwoobjJvTVuyFV/ZOwO3T7HJz+WG /3pK+DFdSKLp7yKAzovFFUrOn2urUTt1jghvC9qYthICddTblWH+qkozC/5MHaX/qYvB Of2A== X-Forwarded-Encrypted: i=1; AJvYcCXc2venDYkjR0zUZ0TPb6mLQnL5xJbnDM8EccjAsfL8YmQNNgHbhYY0qETot/TwDe3dR18MA08=@vger.kernel.org X-Gm-Message-State: AOJu0YwC0+dP911pzHjMOTqcfL7sURooLqBO07pLkBZTWltaNUhk5ZHo 6UD/JBtPJpNwVhKbEwB2rnbXpUTJKLk8PNX65bI4VXVOLLoNVBzn X-Gm-Gg: ASbGnctHjGeIWsWGKlQ+NKD0NHwjv6nlFfmpDTAIf0GUdNusiOwJ4P4Bow2JHo1pnsr xjs1hPMnRoEebfTRwNPqYhN/ycfJiJpwJWpivDrvn5nSA3f0b/n9ELxckWzfLBrOsyT4YjOxcUU QKZ9Xk4M5GRXv29YKBGRhzdFpoldnkkrdD6fCM+iDS9OlXyd4Iq+6CX8XnpD5JVpvgU4mhdqhcR 8KPVCpvlwCpLhczpn5ICwiPieAi1ZqS2RNlR65iM1ApMzaYzmWhJrn+laLhp5zbL1CDXhCF9Jnp 6OOvNmoCYgHHV4k/0bKPRm+ae8btHksn50jaWVLp/BwUiOW1OTYIsQ== X-Google-Smtp-Source: AGHT+IGqOP6e5eDSV1c9md94CzB6589bQXmg3JQW6RzNab63CO5tgoesB3YFSvcefl/J193jWpWlTA== X-Received: by 2002:a17:902:d4c4:b0:216:55a1:35a with SMTP id d9443c01a7336-21c3558bef3mr586256735ad.30.1738054017268; Tue, 28 Jan 2025 00:46:57 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.46.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:46:56 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 04/13] bpf: stop calling some sock_op BPF CALLs in new timestamping callbacks Date: Tue, 28 Jan 2025 16:46:11 +0800 Message-Id: <20250128084620.57547-5-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Simply disallow calling bpf_sock_ops_setsockopt/getsockopt, bpf_sock_ops_cb_flags_set, and the bpf_sock_ops_load_hdr_opt for the new timestamping callbacks for the safety consideration. Besides, In the next round, we will support the UDP proto for SO_TIMESTAMPING bpf extension, so we need to ensure there is no safety problem, which is usually caused by UDP socket trying to access TCP fields. Signed-off-by: Jason Xing --- net/core/filter.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/core/filter.c b/net/core/filter.c index dc0e67c5776a..d3395ffe058e 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5523,6 +5523,11 @@ static int __bpf_setsockopt(struct sock *sk, int level, int optname, return -EINVAL; } +static bool is_locked_tcp_sock_ops(struct bpf_sock_ops_kern *bpf_sock) +{ + return bpf_sock->op <= BPF_SOCK_OPS_WRITE_HDR_OPT_CB; +} + static int _bpf_setsockopt(struct sock *sk, int level, int optname, char *optval, int optlen) { @@ -5673,6 +5678,9 @@ static const struct bpf_func_proto bpf_sock_addr_getsockopt_proto = { BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, int, level, int, optname, char *, optval, int, optlen) { + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen); } @@ -5758,6 +5766,9 @@ static int bpf_sock_ops_get_syn(struct bpf_sock_ops_kern *bpf_sock, BPF_CALL_5(bpf_sock_ops_getsockopt, struct bpf_sock_ops_kern *, bpf_sock, int, level, int, optname, char *, optval, int, optlen) { + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + if (IS_ENABLED(CONFIG_INET) && level == SOL_TCP && optname >= TCP_BPF_SYN && optname <= TCP_BPF_SYN_MAC) { int ret, copy_len = 0; @@ -5800,6 +5811,9 @@ BPF_CALL_2(bpf_sock_ops_cb_flags_set, struct bpf_sock_ops_kern *, bpf_sock, struct sock *sk = bpf_sock->sk; int val = argval & BPF_SOCK_OPS_ALL_CB_FLAGS; + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + if (!IS_ENABLED(CONFIG_INET) || !sk_fullsock(sk)) return -EINVAL; @@ -7609,6 +7623,9 @@ BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock, u8 search_kind, search_len, copy_len, magic_len; int ret; + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + /* 2 byte is the minimal option len except TCPOPT_NOP and * TCPOPT_EOL which are useless for the bpf prog to learn * and this helper disallow loading them also. From patchwork Tue Jan 28 08:46:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952310 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E3861DED67; Tue, 28 Jan 2025 08:47:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054025; cv=none; b=ZKrYV5IpttAbYV1f6gOD8p+HR5U4UkQYjeleEqbtxGgy5U5mKQPE6ql0c/ta7OnU8HRXzoMENxsn9IywT6ZZpoIKru/Em4eaVKTfSJKhcox2tMvXbfU8+LYZCo0CpqMtNDNDuDU7OHPtbDHgfe4pIuwyPJ1pTQ9peoGAKa+veP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054025; c=relaxed/simple; bh=QApQtC5XUqZW26wLg8a9HCJozP8Mzr+LBUJglp2VALk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QhNHULuSwGGahM3XV4DvnDaGdaGfp/NzJ6Ro8727LHRO8qdAYyEtREbcaD9NtuoyjMkUPS1sKZOMTNAkm4QQa0rJaQVFektPi3+2Rj/ZAFGnRJlNQ9REcarMMU2Dy63lsAH4cmgDn7R8QHGNopQoEedcD8M+3M+eU4OmjTGWbfg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=D5Nvi4VC; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D5Nvi4VC" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2166651f752so110262745ad.3; Tue, 28 Jan 2025 00:47:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054023; x=1738658823; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BzNnDEADoQ3Tn/NnieMSGDXghu1f1Kdjynrdkn9Gsf4=; b=D5Nvi4VCR3zXFz556fXmRj2eFmL/3ZjXB9EEpRSaCHJ7De7tWj6dl2oTswAa2IggMT 2wtg+A8Ptc8ujK8gJ4CU2m4Nsjm0rHAeE8vjj1xqgD6dxrl/NPG3Ag4IFe95eorjxfdA 0meZqFbVfJRFgtJbsHoPOn6isHJ/aSATBvHG8paPf36LZV7MpxZwUIdFJGYLwzRx4SHv KlNnYLEc1l71uAYF1yyEiUyRZjqFYNukjHa1yvCE6FH1o3LfA4ikZaloOUKsizZ8sxf0 6Xx0opLOhM2IgZ8zVT5OYePwqAvmoNzQq6ZqpPVCAyRfCqa1KI5WB9PMidoywYoa/Z02 n1CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054023; x=1738658823; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BzNnDEADoQ3Tn/NnieMSGDXghu1f1Kdjynrdkn9Gsf4=; b=QwRlRbCyEjQ4C4Uxa2RnodHNun+ONqDN/GFc66ElvH/B3gAFY/kfytWVjPuks5r1Yh tPUFh16fQpLPlS1VT3IeOwmL7o4Ocy1xLLZliJO/foUsuwUTBsF5hI4+uzGtpmD0fR/a HpMcnUVypiWYQoiYhVDDNCytdorqfAx7hcIkE9pJgB5MMOP+X/VaR6z2lfdUskkRLp3N wmYM01TfJvMTcuZ6y9EtKb2cvay1O04o41XgF1Qrwg5BpV2WdIAMEW/hpGzmwXUM7hCD h/K3haTgYGDxN/xkqSnBMo0qKBCAx2xDKLhYDWXhBOgpkdsM2Sdankt8RN1q9ExcO5p3 /U1g== X-Forwarded-Encrypted: i=1; AJvYcCVxqQExREK2DDbzzv2h1O+hQ1F0ERC8YIjkdBqzVU2LZrWkhPk4TMtdozFsHRmbB8tWojaBCKI=@vger.kernel.org X-Gm-Message-State: AOJu0Yww+duDFVFC+504ejvkSbARDWDGGqnCXNOW31y80nEQ3tWTqaPW +vKKrFCH3WmtnAIickzIPi1qN9MyZE5lBz25fDNwVUoHkK26Ekfk X-Gm-Gg: ASbGncsqkPP3vGuTC1Pa+hyU6+LQrvlAq16eNhMuMR2p1NLIQW4EzR3FyF7oxX3XyIr J7gUd2OPpdfxUKyPu8F2peamDfJafiWUSnE+BLEmvxtRqlhWZGVniBg7GM6MIi7O9BrUWZCkoSf Zc6aMG13rumj50c7sfDbdX4flZjC/27316Ii1inJsrLliXczPD7EmptAWkrE88WzIctRawkaUla fstUfv+IiO+t3f7TFOTaCdClDIkv755ghVUrOQeJ2kZCTrtSU/eu07U4u3n68SlMXKDGavNF4Un eWr8MQ1LmkZijIjVcsK+XP7JPiUjS+/ryYN2jKDFcqwqs65/OQuFmg== X-Google-Smtp-Source: AGHT+IHT4vI6hH5UHmjiil8kr+jszmy7QoRwyrP7c3v14WFztu/DTBGJ8BY38gKu44cp3581f9I0VA== X-Received: by 2002:a17:903:2312:b0:216:69ca:770b with SMTP id d9443c01a7336-21c3550eaacmr736564255ad.12.1738054022485; Tue, 28 Jan 2025 00:47:02 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.46.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:02 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 05/13] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING Date: Tue, 28 Jan 2025 16:46:12 +0800 Message-Id: <20250128084620.57547-6-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net No functional changes here. I add skb_enable_app_tstamp() to test if the orig_skb matches the usage of application SO_TIMESTAMPING and skb_sw_tstamp_tx() to distinguish the software and hardware timestamp when tsflag is SCM_TSTAMP_SND. Also, I deliberately distinguish the the software and hardware SCM_TSTAMP_SND timestamp by passing 'sw' parameter in order to avoid such a case where hardware may go wrong and pass a NULL hwstamps, which is even though unlikely to happen. If it really happens, bpf prog will finally consider it as a software timestamp. It will be hardly recognized. Let's make the timestamping part more robust. After this patch, I will soon add checks about bpf SO_TIMESTAMPING. In this way, we can support two modes parallelly. Signed-off-by: Jason Xing --- include/linux/skbuff.h | 13 +++++++------ net/core/dev.c | 2 +- net/core/skbuff.c | 32 ++++++++++++++++++++++++++++++-- net/ipv4/tcp_input.c | 3 ++- 4 files changed, 40 insertions(+), 10 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index bb2b751d274a..dfc419281cc9 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -39,6 +39,7 @@ #include #include #include +#include /** * DOC: skb checksums @@ -4533,18 +4534,18 @@ void skb_complete_tx_timestamp(struct sk_buff *skb, void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb, struct skb_shared_hwtstamps *hwtstamps, - struct sock *sk, int tstype); + struct sock *sk, bool sw, int tstype); /** - * skb_tstamp_tx - queue clone of skb with send time stamps + * skb_tstamp_tx - queue clone of skb with send HARDWARE timestamps * @orig_skb: the original outgoing packet * @hwtstamps: hardware time stamps, may be NULL if not available * * If the skb has a socket associated, then this function clones the * skb (thus sharing the actual data and optional structures), stores - * the optional hardware time stamping information (if non NULL) or - * generates a software time stamp (otherwise), then queues the clone - * to the error queue of the socket. Errors are silently ignored. + * the optional hardware time stamping information (if non NULL) then + * queues the clone to the error queue of the socket. Errors are + * silently ignored. */ void skb_tstamp_tx(struct sk_buff *orig_skb, struct skb_shared_hwtstamps *hwtstamps); @@ -4565,7 +4566,7 @@ static inline void skb_tx_timestamp(struct sk_buff *skb) { skb_clone_tx_timestamp(skb); if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP) - skb_tstamp_tx(skb, NULL); + __skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND); } /** diff --git a/net/core/dev.c b/net/core/dev.c index afa2282f2604..d77b8389753e 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4501,7 +4501,7 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) skb_assert_len(skb); if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP)) - __skb_tstamp_tx(skb, NULL, NULL, skb->sk, SCM_TSTAMP_SCHED); + __skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SCHED); /* Disable soft irqs for various locks below. Also * stops preemption for RCU. diff --git a/net/core/skbuff.c b/net/core/skbuff.c index a441613a1e6c..6042961dfc02 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5539,10 +5539,35 @@ void skb_complete_tx_timestamp(struct sk_buff *skb, } EXPORT_SYMBOL_GPL(skb_complete_tx_timestamp); +static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) +{ + int flag; + + switch (tstype) { + case SCM_TSTAMP_SCHED: + flag = SKBTX_SCHED_TSTAMP; + break; + case SCM_TSTAMP_SND: + flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP; + break; + case SCM_TSTAMP_ACK: + if (TCP_SKB_CB(skb)->txstamp_ack) + return true; + fallthrough; + default: + return false; + } + + if (skb_shinfo(skb)->tx_flags & flag) + return true; + + return false; +} + void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb, struct skb_shared_hwtstamps *hwtstamps, - struct sock *sk, int tstype) + struct sock *sk, bool sw, int tstype) { struct sk_buff *skb; bool tsonly, opt_stats = false; @@ -5551,6 +5576,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, if (!sk) return; + if (!skb_enable_app_tstamp(orig_skb, tstype, sw)) + return; + tsflags = READ_ONCE(sk->sk_tsflags); if (!hwtstamps && !(tsflags & SOF_TIMESTAMPING_OPT_TX_SWHW) && skb_shinfo(orig_skb)->tx_flags & SKBTX_IN_PROGRESS) @@ -5599,7 +5627,7 @@ EXPORT_SYMBOL_GPL(__skb_tstamp_tx); void skb_tstamp_tx(struct sk_buff *orig_skb, struct skb_shared_hwtstamps *hwtstamps) { - return __skb_tstamp_tx(orig_skb, NULL, hwtstamps, orig_skb->sk, + return __skb_tstamp_tx(orig_skb, NULL, hwtstamps, orig_skb->sk, false, SCM_TSTAMP_SND); } EXPORT_SYMBOL_GPL(skb_tstamp_tx); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 77185479ed5e..62252702929d 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -3330,7 +3330,8 @@ static void tcp_ack_tstamp(struct sock *sk, struct sk_buff *skb, if (!before(shinfo->tskey, prior_snd_una) && before(shinfo->tskey, tcp_sk(sk)->snd_una)) { tcp_skb_tsorted_save(skb) { - __skb_tstamp_tx(skb, ack_skb, NULL, sk, SCM_TSTAMP_ACK); + __skb_tstamp_tx(skb, ack_skb, NULL, sk, true, + SCM_TSTAMP_ACK); } tcp_skb_tsorted_restore(skb); } } From patchwork Tue Jan 28 08:46:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952311 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AEE51DF727; Tue, 28 Jan 2025 08:47:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054029; cv=none; b=T3FAvkJdtYUn7kAxe32R85LqeK4nyDLjluP06dSwupOQ0INp623Zjqi7mjvGsTEUWwjznL4bCxc2/Q+CuoI9cqK+IhvDEkJvVH7p0HhP1eioDQzjfJPQ4OITU4FPwY79E+FqH6OUtjz9GYTatiG4verT9kOBQpRT/YpnwGWES00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054029; c=relaxed/simple; bh=EtQXoTBugP9oThfrVJpumYlGe78MM7EKLtfwPsC3lz0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ukJXgVUaGHea0YDgADVb2UnXw0r+Z+wa3oJJ4tq6jHDZ4nBMYr3JXTgnkVflHb49Z+gqLXkosweHEOSPJC1McQ5ijWwrIh/d2WN5s6P5HtnoPe2Vv1JHfRPbRQNvkxLeAEBF4y6mO88gS5etFPp2gBZHVo6hEraPV/dDMk+xClI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=j3u4DfsV; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="j3u4DfsV" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21680814d42so87552425ad.2; Tue, 28 Jan 2025 00:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054027; x=1738658827; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9YI7fXIJNvl0GMc/bb7iVhvTzXGdQVV12ax4M++Lb08=; b=j3u4DfsVrc/F+zfD309hB1iryZ5mOsBMdaXt7xyKyqXyoq63E3sCOa3M5pH1w8+pl4 th7jGU0ckI1mBEXKqZO/ptvBAQb0cBx+MheRbmk2gfhZQQbUaj3ARC3f3cXk2rxd5iDe KxT6g54U33yzQd9UJSZgPiOlcki9S5Y10L7dYz/MqtgHQUi/oIoDpe5BCRNkqdmDDwe6 A/iQObs6fUW0UUk1fzqNhN3bBaGeeGgOrlvGJhX9UEpPP4YY1A7FYsn4Q4LzaNdGJPqd 69GEbWBb//pFBCkiNQf6Ca1hvzjjsQFkZKeg0C3dDHn5yV4ZGupeKBdgxGVaAXGzoIyg H9ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054027; x=1738658827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9YI7fXIJNvl0GMc/bb7iVhvTzXGdQVV12ax4M++Lb08=; b=bXqT+kz/AfjvUuPI2CVKIwENBsZa8hyYlTMmYC+LpwGPbA40uv+RO7QiKUl8/UOJzI SucX2rnJAMb+5io0+Srx2dy5XHF88HQhg6KqR5Rhep8VRNUaPZgDerkyHZNTNlEa84Wb qydaqKPFRafwEGshOHu12vIPLHvM7TyBbJSCMS8oPEziBkZ71Ya0NcASCpX+FgLTyQFr MzwkcIo8M5X6lmeQnup+hj0Kw4rfq35KZw6iNyaeVYnJBS5kXeyEuJZPpBhmslwno4QP Ce6jc8CzWV9GNMNsgOb+W1uZShFQfX5xvDxzS0IWIx94N76u01lQfuK/QAJ9WPhJDww0 9wSQ== X-Forwarded-Encrypted: i=1; AJvYcCXo/i8sh70MWRdUQEuU+I8zOoE6B8Q8k2hniLNRwykInkFugBRwtwdXD0yk154Qan1Fl9Us8bk=@vger.kernel.org X-Gm-Message-State: AOJu0YzZuithX9/oh8+ELpoaKCdEMedhjA3p10B/RR7Eh+TU7YiRQLJp v5fK4wAzzMXGnKHjaKkvqWNpez8XuZJzuZ83EN3w/OtOGMlaws8b X-Gm-Gg: ASbGnctvkyDYRuF9LPg66RXULPluE0CAnaqjz+mxhNZJVQkdwHjM8GL8eqcxpwvp8+J 04sUrfuqDzlJ6eQgjbKqdLeFyCJsDCHeb04KB30NXOEpeWFIxq4j7C+iFhXdBbpGbo/r0bbokLt wbS5CzcpLyPJEpxwy9oHFaZAx/6n9bR0H/P9GxeCPCxQae6kriypMKxfekf8CJzUa9xQenjQkH3 cU6iWVJWSoDeNqgX+2uunJON96iuXHaDCcAJCmSuuI7RwZEgcz+1hSwYFNStkO5x5RIGY/dCxH0 fbTYJ7GWGS4mU8a9bKBTMu6HROlgmk9634w3+r5ceXj+Ysnl1jhxFg== X-Google-Smtp-Source: AGHT+IFrmdHW539M3buebpADp0DPSeyoOL/nbvGJLY38UFxJG53G/Yy0LsEltUEYFozcxbJylkQEYw== X-Received: by 2002:a17:902:cf0e:b0:212:4c82:e3d4 with SMTP id d9443c01a7336-21c356093bcmr687653195ad.46.1738054027618; Tue, 28 Jan 2025 00:47:07 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:07 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 06/13] net-timestamp: support SCM_TSTAMP_SCHED for bpf extension Date: Tue, 28 Jan 2025 16:46:13 +0800 Message-Id: <20250128084620.57547-7-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Introducing SKBTX_BPF is used as an indicator telling us whether the skb should be traced by the bpf prog. Signed-off-by: Jason Xing --- include/linux/skbuff.h | 6 +++++- include/uapi/linux/bpf.h | 4 ++++ net/core/dev.c | 3 ++- net/core/skbuff.c | 23 +++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 ++++ 5 files changed, 38 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index dfc419281cc9..35c2e864dd4b 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -490,10 +490,14 @@ enum { /* generate software time stamp when entering packet scheduling */ SKBTX_SCHED_TSTAMP = 1 << 6, + + /* used for bpf extension when a bpf program is loaded */ + SKBTX_BPF = 1 << 7, }; #define SKBTX_ANY_SW_TSTAMP (SKBTX_SW_TSTAMP | \ - SKBTX_SCHED_TSTAMP) + SKBTX_SCHED_TSTAMP | \ + SKBTX_BPF) #define SKBTX_ANY_TSTAMP (SKBTX_HW_TSTAMP | \ SKBTX_HW_TSTAMP_USE_CYCLES | \ SKBTX_ANY_SW_TSTAMP) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 6116eb3d1515..30d2c078966b 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7032,6 +7032,10 @@ enum { * by the kernel or the * earlier bpf-progs. */ + BPF_SOCK_OPS_TS_SCHED_OPT_CB, /* Called when skb is passing through + * dev layer when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/dev.c b/net/core/dev.c index d77b8389753e..4f291459d6b1 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4500,7 +4500,8 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) skb_reset_mac_header(skb); skb_assert_len(skb); - if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP)) + if (unlikely(skb_shinfo(skb)->tx_flags & + (SKBTX_SCHED_TSTAMP | SKBTX_BPF))) __skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SCHED); /* Disable soft irqs for various locks below. Also diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6042961dfc02..d19d577b996f 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5564,6 +5564,24 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) return false; } +static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype) +{ + int op; + + if (!sk) + return; + + switch (tstype) { + case SCM_TSTAMP_SCHED: + op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; + break; + default: + return; + } + + bpf_skops_tx_timestamping(sk, skb, op); +} + void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb, struct skb_shared_hwtstamps *hwtstamps, @@ -5576,6 +5594,11 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, if (!sk) return; + /* bpf extension feature entry */ + if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF) + skb_tstamp_tx_bpf(orig_skb, sk, tstype); + + /* application feature entry */ if (!skb_enable_app_tstamp(orig_skb, tstype, sw)) return; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 70366f74ef4e..eed91b7296b7 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7025,6 +7025,10 @@ enum { * by the kernel or the * earlier bpf-progs. */ + BPF_SOCK_OPS_TS_SCHED_OPT_CB, /* Called when skb is passing through + * dev layer when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Jan 28 08:46:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952312 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99C4D1DED67; Tue, 28 Jan 2025 08:47:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054035; cv=none; b=FkqyFDpTKMU8YCmGbmHbuiS6/krulnr+XVWGwp2tJE4lSfXcyhnHVZHkjzJAWoRBw1Paw9o1jVzgJOt0nA/JtxawS0zIEWs9r0TYCGo3yD8m8+p5VE7q7EhhDgWfwQ9cdcCHcK0pBH4HonNdCZRzPZJ4zmKH8/VTYJDUb0P1iR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054035; c=relaxed/simple; bh=yq4u9sCpQJpiQpcF2aG5NYY4OkdQ+GiMFtfVSCms1oc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Scy4N7lOjbexsFh10lBlRvi46alD/q2Majn0CUKv+3x839gekCjSqi/tet+9YlnJaJzgW7uBTwu/gsECxRQplzRILNMqCILXiRf0rgv6Py6hTrx2njkHrE4bEdrd+bf1kZWSm7lSSdn3SB7QfuNRzMb/Y53vNE/EKhYbntEXmjY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Dc2hJWS9; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Dc2hJWS9" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21636268e43so117386305ad.2; Tue, 28 Jan 2025 00:47:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054033; x=1738658833; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YIucsfFknM8P1g1XyuwxrXdMWrf/7m2l5cECZ0rTqwI=; b=Dc2hJWS9ElsJ7CXhPq1lz2m58REjXfoWwHSFakpEyl6n1dcSLo9PgzZUpHFxq0hqrD mNw/zSKca6KR6ohF5QmlIGHEuzuQV6+2yiKQYe+ZC8FsssCVCqpLGdf0IKdrB0OW1p6U Q7PW6qhc/aq2zXZbdj3amdCMNYYvepdV+hKeunm5SEOgZJUMGLrBntw9baX3y22Rlxfi laf7Z8hs2dFXt954WCTvdDIe23G0ih5ws3T/kDXeakRcYkhORJGIHq+a1BZFDQMbW2hC RYlusrME61hkiazZgT/6rn7cZjIQY7doiENFafJc8bZh2KRXeJqHzCthiC5yYMiIrL1v SNGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054033; x=1738658833; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YIucsfFknM8P1g1XyuwxrXdMWrf/7m2l5cECZ0rTqwI=; b=uIw7q2RZELik4iFEWBZ5PzPDkNfNj7G9BQU6YFQS84fy1axt+Q9zGr6Q7iR+2GRat4 hcD8CkdQVX2MUS0bdrvD9/P8yCp7uliOO5gU2nuab3DWRxKsjBcOp4z3F7wcJeU6lQRS cRecJtsIseN6AJHhrLbrTlc3mE64iJbdbsTHvsXh/eJUHMBo6EKWzs8MO+Ai2I2dmJd9 KZFncRIbQkTRA3CBMI8+U5io/B510sNU7QMuAWKbbzTgcT31D6ZZX/ncSYmCztm6eRL5 8YvHh6+r7Lpuj+JwnEumu5KooFVs0Oz+edhR0cApOFvoB90j4K9cR94aSsrjZg5sA7ZH /lYQ== X-Forwarded-Encrypted: i=1; AJvYcCXz5svMufcZVkaI0aImT+KG7ZhIHwjMiWpOMG3nnv43qgVFD1bSkouwCeR5uC4YI2HAzdcWXXs=@vger.kernel.org X-Gm-Message-State: AOJu0Yx3oai1b8gjOKLIaeVDzFXJliaVVdlZodI1zq77iFuOgIPUFR2z 5JFe+0+m6nBmag3xSTM442ASsqu1hp1MC57pRkw4nNshFFV7+QxR X-Gm-Gg: ASbGncvn+4FjhlQWCmwwDjRwH5V0Hz9JXW62C+cyq/oAaQbuC+bxcP/H1ul7prKrEx0 nIsyX8Tg4Wzc/VK59kwbjXvF/kO8jHcwmisXLvoNaOWGq738xXbQvXdMBT+n6T0jPbrroGR6cXn mD0dlwVC00uyAnC0t7EB93gvPV2FO/sySD/Gd/9qpNTTyecesv/X5XOmYp6GGQc6PUJL1HF8hOr F65iTvpq6Vs8C19K9PLI0aHoVbuRcH0Hdnavvu/c5o/zK3hiUmzKi3b0OxK0LeKrFA9SfQMGiS3 zgyHKz/n4w8E0DWwKwndDcJydDo5NsG5vE9cdG9xav0YDSI/Nbd7Dg== X-Google-Smtp-Source: AGHT+IGqkybKjdMNj4H4r8LackhtuAO/uVSjScfRAJQkPM2hqjJAVMMhpdC/eeEbV4v/sPMRAC8OcA== X-Received: by 2002:a17:902:e74a:b0:216:56c7:98a7 with SMTP id d9443c01a7336-21c35618995mr732278255ad.53.1738054032787; Tue, 28 Jan 2025 00:47:12 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:12 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 07/13] net-timestamp: support sw SCM_TSTAMP_SND for bpf extension Date: Tue, 28 Jan 2025 16:46:14 +0800 Message-Id: <20250128084620.57547-8-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Support SCM_TSTAMP_SND case. Then we will get the software timestamp when the driver is about to send the skb. Later, I will support the hardware timestamp. Signed-off-by: Jason Xing --- include/linux/skbuff.h | 2 +- include/uapi/linux/bpf.h | 4 ++++ net/core/skbuff.c | 10 ++++++++-- tools/include/uapi/linux/bpf.h | 4 ++++ 4 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 35c2e864dd4b..de8d3bd311f5 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4569,7 +4569,7 @@ void skb_tstamp_tx(struct sk_buff *orig_skb, static inline void skb_tx_timestamp(struct sk_buff *skb) { skb_clone_tx_timestamp(skb); - if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP) + if (skb_shinfo(skb)->tx_flags & (SKBTX_SW_TSTAMP | SKBTX_BPF)) __skb_tstamp_tx(skb, NULL, NULL, skb->sk, true, SCM_TSTAMP_SND); } diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 30d2c078966b..6a1083bcf779 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7036,6 +7036,10 @@ enum { * dev layer when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SW_OPT_CB, /* Called when skb is about to send + * to the nic when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index d19d577b996f..288eb9869827 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5564,7 +5564,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) return false; } -static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype) +static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, + int tstype, bool sw) { int op; @@ -5575,6 +5576,11 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, int tstype) case SCM_TSTAMP_SCHED: op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; break; + case SCM_TSTAMP_SND: + if (!sw) + return; + op = BPF_SOCK_OPS_TS_SW_OPT_CB; + break; default: return; } @@ -5596,7 +5602,7 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, /* bpf extension feature entry */ if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF) - skb_tstamp_tx_bpf(orig_skb, sk, tstype); + skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw); /* application feature entry */ if (!skb_enable_app_tstamp(orig_skb, tstype, sw)) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index eed91b7296b7..9bd1c7c77b17 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7029,6 +7029,10 @@ enum { * dev layer when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SW_OPT_CB, /* Called when skb is about to send + * to the nic when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Jan 28 08:46:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952313 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC5951DE4E3; Tue, 28 Jan 2025 08:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054041; cv=none; b=cQACwkWKituYCKSMIcfnTtV0xBVRWhZLeId8PUu6ZCVZookl6Thj9plAncqCazoBes9WPU4OuRmooA+osSuZeVYHH82gflOebvUqzwTbd8yMf1GkVJzqNtaAxvw7iY+p48LLhSk4VgK/XuMsqRIou0bM61HPuf+myUUzQtUglcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054041; c=relaxed/simple; bh=BdNB/GGRAZb8/epyZCnM9bOLQPXObRKF9ZTMJJ2FsWk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gjYyyWnYu7mgGQEsCD12CASfd5ppk0EAcZU4WcOb4t28vIS8CRYUdY0SOtrf4lKc8QmyZaZRIiddhOMusSkng3qpL1ayBI+lEaB96/TX/Jb1Aim33uRlO992pQMbyK3W/YU9Gc8u0s0N8YFSPLItFfMucZhRRUQbKUANUJZfSjU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aesQOHB0; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aesQOHB0" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-21a7ed0155cso87710115ad.3; Tue, 28 Jan 2025 00:47:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054038; x=1738658838; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cLm7BAECDHVjsR7lqQUOHfEEMf58hoJW0MlcXzl0Asc=; b=aesQOHB0WkK2NXCYiUf3F+dBmA0KBa6luYN7xC6pMne1Qa0ai4Ymn5s5P1WHqXW1Uq thXTzOfTTKiy3z5Ue7ebtx1YtIiubJmU43G7oRKSoXxiy7mDHtdMCEZbMQdjSE8vAsD8 fNntlFytDC6x1KBUHSy1l6KInE7pa6P1RAxEllwGn+fOfUwLwAtOVbSVMKhJAzXSmKl2 OfRlAlSSKpUW2Eg5ltJrb+LwDU3iIzI1uBlraCZ0CWF28PsO/0eB9U8olaytpMRYr0pF Tsouc6tMFPrhxhfWsClLbmJQplyCnA7+n3HaWxFvAk1cuuPcsRVASE6YLZolyqhJaIfo C4kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054038; x=1738658838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cLm7BAECDHVjsR7lqQUOHfEEMf58hoJW0MlcXzl0Asc=; b=cw1EEeWK4hbDlLJzXyj1FhHZ+2N3NGakXE+G3pRfNxaxmR8AKjOCeqqPUumB8XR/y1 WFZBCI7Pd4MQyt/2Oa8gkgnXiMWzBzFQFQ2IEOAZlbWaAM8ChZ1EuuBDpx4C092xzl7N 0aZbgfbmhIXkCJ8WHrit4QRh8C7ZhFZ6XXXYMkwn+F98Wd9MjOr51fuoPVHqBAA+LT+b yAPRAq8rfw/1FLcpyx54nD312DqZ1Ub2KEOVn8/sGvRZyHG+CHfQRy1bGqvxhvS0NS3G jn30QOu6hcWmTS9eNAPidUPS2WJgSIrZRYjF0XAthCyBTNMFM0c1xcFbcuoMc9VSUUn+ LHuA== X-Forwarded-Encrypted: i=1; AJvYcCWcixDkF/5fV338MR07384rL5FryEgtX1Jza+m3D88OWNDPAw/JmWM+2YXhHq9IgT23WsMsj7s=@vger.kernel.org X-Gm-Message-State: AOJu0YznzIiVNuaE/lRA3S5UPC1uKeVQ2T9UgORejMv05wSGFVJHGvm9 fm5b8sbU12vqsI1iVdPxxXMojgJ/q03JLZ0HQ+Ye3qS4hW6Huwnf X-Gm-Gg: ASbGncsLgg9o2NqdzifsdCRoAgvWTxRGiBJNDlT9Nujnn+1vl9O4KngoK5PtKHfygx+ P17N3Ei89W/0LqqebUd7IZKqyaogg9RnLGii2rc9/M/mjYRJg+Ito5dEoHl8HATzfH++oQrScNm Gk4inyFoYkROj7Fzl6J+L+4EXDrUfue75iML6asXKhMpY9kTwHhkjnzWUwk2kNIrxCcZbft5jhg Iwbi0+EKqsNgfKWmnU+Mc5Ctsl/3mpgzhOHXIoeTR62i+pa274TTCBRy7lPyc+3PWapJgUilpsp QSYHD04smOAES6wfPMwOxFjP+fyQgZALu5UyoHvmS9aYaIyWfih4gg== X-Google-Smtp-Source: AGHT+IEQRAbljmz3IUYoIAWT6T/2agG5jGLBP0XowJYvZjdG+EQrmuF60AkaX21Ml7ciCoMu5yiwyg== X-Received: by 2002:a17:903:2b0f:b0:216:69ca:7707 with SMTP id d9443c01a7336-21c35586204mr663584025ad.30.1738054037965; Tue, 28 Jan 2025 00:47:17 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:17 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 08/13] net-timestamp: support hw SCM_TSTAMP_SND for bpf extension Date: Tue, 28 Jan 2025 16:46:15 +0800 Message-Id: <20250128084620.57547-9-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net In this patch, we finish the hardware part. Then bpf program can fetch the hwstamp from skb directly. To avoid changing so many callers using SKBTX_HW_TSTAMP from drivers, use this simple modification like this patch does to support printing hardware timestamp. Signed-off-by: Jason Xing --- include/linux/skbuff.h | 4 +++- include/uapi/linux/bpf.h | 7 +++++++ net/core/skbuff.c | 11 ++++++----- net/dsa/user.c | 2 +- net/socket.c | 2 +- tools/include/uapi/linux/bpf.h | 7 +++++++ 6 files changed, 25 insertions(+), 8 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index de8d3bd311f5..df2d790ae36b 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -471,7 +471,7 @@ struct skb_shared_hwtstamps { /* Definitions for tx_flags in struct skb_shared_info */ enum { /* generate hardware time stamp */ - SKBTX_HW_TSTAMP = 1 << 0, + __SKBTX_HW_TSTAMP = 1 << 0, /* generate software time stamp when queueing packet to NIC */ SKBTX_SW_TSTAMP = 1 << 1, @@ -495,6 +495,8 @@ enum { SKBTX_BPF = 1 << 7, }; +#define SKBTX_HW_TSTAMP (__SKBTX_HW_TSTAMP | SKBTX_BPF) + #define SKBTX_ANY_SW_TSTAMP (SKBTX_SW_TSTAMP | \ SKBTX_SCHED_TSTAMP | \ SKBTX_BPF) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 6a1083bcf779..4c3566f623c2 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7040,6 +7040,13 @@ enum { * to the nic when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_HW_OPT_CB, /* Called in hardware phase when + * SK_BPF_CB_TX_TIMESTAMPING feature + * is on. At the same time, hwtstamps + * of skb is initialized as the + * timestamp that hardware just + * generates. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 288eb9869827..c769feae5162 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5548,7 +5548,7 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) flag = SKBTX_SCHED_TSTAMP; break; case SCM_TSTAMP_SND: - flag = sw ? SKBTX_SW_TSTAMP : SKBTX_HW_TSTAMP; + flag = sw ? SKBTX_SW_TSTAMP : __SKBTX_HW_TSTAMP; break; case SCM_TSTAMP_ACK: if (TCP_SKB_CB(skb)->txstamp_ack) @@ -5565,7 +5565,8 @@ static bool skb_enable_app_tstamp(struct sk_buff *skb, int tstype, bool sw) } static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, - int tstype, bool sw) + int tstype, bool sw, + struct skb_shared_hwtstamps *hwtstamps) { int op; @@ -5577,9 +5578,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; break; case SCM_TSTAMP_SND: + op = sw ? BPF_SOCK_OPS_TS_SW_OPT_CB : BPF_SOCK_OPS_TS_HW_OPT_CB; if (!sw) - return; - op = BPF_SOCK_OPS_TS_SW_OPT_CB; + *skb_hwtstamps(skb) = *hwtstamps; break; default: return; @@ -5602,7 +5603,7 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, /* bpf extension feature entry */ if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF) - skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw); + skb_tstamp_tx_bpf(orig_skb, sk, tstype, sw, hwtstamps); /* application feature entry */ if (!skb_enable_app_tstamp(orig_skb, tstype, sw)) diff --git a/net/dsa/user.c b/net/dsa/user.c index 291ab1b4acc4..ae715bf0ae75 100644 --- a/net/dsa/user.c +++ b/net/dsa/user.c @@ -897,7 +897,7 @@ static void dsa_skb_tx_timestamp(struct dsa_user_priv *p, { struct dsa_switch *ds = p->dp->ds; - if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) + if (!(skb_shinfo(skb)->tx_flags & __SKBTX_HW_TSTAMP)) return; if (!ds->ops->port_txtstamp) diff --git a/net/socket.c b/net/socket.c index 262a28b59c7f..70eabb510ce6 100644 --- a/net/socket.c +++ b/net/socket.c @@ -676,7 +676,7 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags) u8 flags = *tx_flags; if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE) { - flags |= SKBTX_HW_TSTAMP; + flags |= __SKBTX_HW_TSTAMP; /* PTP hardware clocks can provide a free running cycle counter * as a time base for virtual clocks. Tell driver to use the diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 9bd1c7c77b17..974b7f61d11f 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7033,6 +7033,13 @@ enum { * to the nic when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_HW_OPT_CB, /* Called in hardware phase when + * SK_BPF_CB_TX_TIMESTAMPING feature + * is on. At the same time, hwtstamps + * of skb is initialized as the + * timestamp that hardware just + * generates. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Jan 28 08:46:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952314 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 076F61DE4E3; Tue, 28 Jan 2025 08:47:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054045; cv=none; b=r7rdN49wWkv0ry3lKTZpPDqdU//4dVj1OVt2oilTEEIhda6vjXUn7vbUq/Iu0vCP0a96klhtHhiEF4hv43PG6BamKQX7LQsauack+VTQ09r1O5iBYEP2mGykpTkZfWjhPeabxGhZXUNLouzkdopLJjlviCn2gJ2uDLhu+l9cfVI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054045; c=relaxed/simple; bh=vWZydBKP59s3FpOObSwDDoUwuCZmN9vaGxIbWHzZSIg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qAbnn+vsrh+Aa6qiiYkI+wigONn09q4ApKBOtDy/MgTa8sTFPDgUgiicWnvhQEyepn4FYJqCm6VRevrzS6vBAb95PiHDZS+UPJZeu8jZoueD3zf4ssst7jGyYxCqTGVCKFagLGvwzL1pLiYh2sKIuJDzVmMkOR7Uts4izkqJ/kg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KHzOQhfn; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KHzOQhfn" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-216401de828so97138025ad.3; Tue, 28 Jan 2025 00:47:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054043; x=1738658843; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1I2k2ii3QrIAPoqerPtjO/Qv/9nXmEL2qWUmYgFeL2w=; b=KHzOQhfnrVSJO8ZVKWUwdzroOUqrfmv529X48X6LTiqq5Vw1xWWDvQ0Zc1D9j4oQ9X VPXjlyeN2znSo9l4BUvxtff+PE1aE1dGAmqBd3WHvesnqdc19jchA1lNNVi92N/elzxL MC93XZVUCnRVKyThE7mBpWo6KwHwXPUzzq5pLO9hKM4YDc7Rja08HQHBJ9D0wxLf3pS2 zZza3LqTUC5kJar8DoYWcTKuPni6DzgzXJoc34xhGAabCGEcr6isGC8ObPOp/vVSs0iv 6UfhF6ay6cJgpjHqCI2sEPbJ30PdQJn7ZAt4jrmb5tiEWzlpUfnjHdyAXfvRKjIiDg3b rISw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054043; x=1738658843; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1I2k2ii3QrIAPoqerPtjO/Qv/9nXmEL2qWUmYgFeL2w=; b=BIK/FIu3oUwZ9CX61avjUTiBmhV5dIR+Zd8B0GK0lGDp2EgRncRKI5wXFQhiYwjNl+ M1Tqri/K2YnXUDUTb083cyIGUD0xbYXlSptZObZPXDxdw/v0O8kP7lyotlbNmnAziRGE 8yCORwWxv6qqHCKYJI5Ut94DIg3pt6WkeU4tFcSZPmQbbKlns0WMnapTpCxdMuaanHAP KYyMOwiGy9CuD5NMTlpzgQxGYGC+wkHecOO5cen21oAgz3D/oLtQKN8PEFLN//oUwJh2 ooSltZG4qP7hxsbedYX0lBZMCRKiX61og+LK2m5PlfT5QwuMUeESi9P2tMDGwNjRqKXn 6Yvg== X-Forwarded-Encrypted: i=1; AJvYcCVP8G53uJMyle0AErrRYuGPSOBDqQFbWtJn1YdYZK5vYbMhS3+bC4w9fTyGgb5VeysyAcLcTOQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyM1zkG4n+IkPmfDsyi3ly1YXWsR2ZZcmOInMMgnbKjFYVNilMc vI1/Qvozbgl2xiW5RARfVyWyQNIf6K0mok0IdLAyGNj/CYoy5I3O X-Gm-Gg: ASbGncsqPX4UqWqpHq92iI/RtWqHhD0xR0LJicjYLcvymnPDJ0glQW573Wxlh7zfJ2c RotczdnYwynRWA0rm7OkkrVWcwxDo35IMqlwBldkqhADIAXbq7tVQ6UFuhIAx2yf/1e0Z/AkHoh w+ZUfk9cVTYtNYb3n9aBzfpRMyz6tWtWmdwPjSfOmWFGSj6QQcI4JNJkwvoxPCSY9Ni3qtxbkwF kNivmwI6F+LI7CNt6UJ/FfhMCeWEQl8hz6o6xc1kISF5dbtfCxo051YtRhemv0tvEvED/o/gJbB 1CNI+pddUqBis4JdlcnJJsKY6sNtgx12QucUrijUB7szjvRRAMhwaQ== X-Google-Smtp-Source: AGHT+IEOgpIoxWkHGtV5xl3H/Q4LFOGvxPyFx7ldYG0RAQS5TXEGE+OdRQFP9qOd9i7OF3SRh7gh+w== X-Received: by 2002:a17:902:ea08:b0:21d:3bd7:afe8 with SMTP id d9443c01a7336-21d3bd7b2acmr469423555ad.49.1738054043149; Tue, 28 Jan 2025 00:47:23 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:22 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 09/13] net-timestamp: support SCM_TSTAMP_ACK for bpf extension Date: Tue, 28 Jan 2025 16:46:16 +0800 Message-Id: <20250128084620.57547-10-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Handle the ACK timestamp case. Actually testing SKBTX_BPF flag can work, but we need to Introduce a new txstamp_ack_bpf to avoid cache line misses in tcp_ack_tstamp(). To be more specific, in most cases, normal flows would not access skb_shinfo as txstamp_ack is zero, so that this function won't appear in the hot spot lists. Introducing a new member txstamp_ack_bpf works similarly. Signed-off-by: Jason Xing --- include/net/tcp.h | 3 ++- include/uapi/linux/bpf.h | 5 +++++ net/core/skbuff.c | 3 +++ net/ipv4/tcp_input.c | 3 ++- net/ipv4/tcp_output.c | 5 +++++ tools/include/uapi/linux/bpf.h | 5 +++++ 6 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 293047694710..88429e422301 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -959,9 +959,10 @@ struct tcp_skb_cb { __u8 sacked; /* State flags for SACK. */ __u8 ip_dsfield; /* IPv4 tos or IPv6 dsfield */ __u8 txstamp_ack:1, /* Record TX timestamp for ack? */ + txstamp_ack_bpf:1, /* ack timestamp for bpf use */ eor:1, /* Is skb MSG_EOR marked? */ has_rxtstamp:1, /* SKB has a RX timestamp */ - unused:5; + unused:4; __u32 ack_seq; /* Sequence number ACK'd */ union { struct { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 4c3566f623c2..800122a8abe5 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7047,6 +7047,11 @@ enum { * timestamp that hardware just * generates. */ + BPF_SOCK_OPS_TS_ACK_OPT_CB, /* Called when all the skbs in the + * same sendmsg call are acked + * when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index c769feae5162..33340e0b094f 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5582,6 +5582,9 @@ static void skb_tstamp_tx_bpf(struct sk_buff *skb, struct sock *sk, if (!sw) *skb_hwtstamps(skb) = *hwtstamps; break; + case SCM_TSTAMP_ACK: + op = BPF_SOCK_OPS_TS_ACK_OPT_CB; + break; default: return; } diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 62252702929d..c8945f5be31b 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -3323,7 +3323,8 @@ static void tcp_ack_tstamp(struct sock *sk, struct sk_buff *skb, const struct skb_shared_info *shinfo; /* Avoid cache line misses to get skb_shinfo() and shinfo->tx_flags */ - if (likely(!TCP_SKB_CB(skb)->txstamp_ack)) + if (likely(!TCP_SKB_CB(skb)->txstamp_ack && + !TCP_SKB_CB(skb)->txstamp_ack_bpf)) return; shinfo = skb_shinfo(skb); diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 695749807c09..fc84ca669b76 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -1556,6 +1556,7 @@ static void tcp_adjust_pcount(struct sock *sk, const struct sk_buff *skb, int de static bool tcp_has_tx_tstamp(const struct sk_buff *skb) { return TCP_SKB_CB(skb)->txstamp_ack || + TCP_SKB_CB(skb)->txstamp_ack_bpf || (skb_shinfo(skb)->tx_flags & SKBTX_ANY_TSTAMP); } @@ -1572,7 +1573,9 @@ static void tcp_fragment_tstamp(struct sk_buff *skb, struct sk_buff *skb2) shinfo2->tx_flags |= tsflags; swap(shinfo->tskey, shinfo2->tskey); TCP_SKB_CB(skb2)->txstamp_ack = TCP_SKB_CB(skb)->txstamp_ack; + TCP_SKB_CB(skb2)->txstamp_ack_bpf = TCP_SKB_CB(skb)->txstamp_ack_bpf; TCP_SKB_CB(skb)->txstamp_ack = 0; + TCP_SKB_CB(skb)->txstamp_ack_bpf = 0; } } @@ -3213,6 +3216,8 @@ void tcp_skb_collapse_tstamp(struct sk_buff *skb, shinfo->tskey = next_shinfo->tskey; TCP_SKB_CB(skb)->txstamp_ack |= TCP_SKB_CB(next_skb)->txstamp_ack; + TCP_SKB_CB(skb)->txstamp_ack_bpf |= + TCP_SKB_CB(next_skb)->txstamp_ack_bpf; } } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 974b7f61d11f..06e68d772989 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7040,6 +7040,11 @@ enum { * timestamp that hardware just * generates. */ + BPF_SOCK_OPS_TS_ACK_OPT_CB, /* Called when all the skbs in the + * same sendmsg call are acked + * when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Jan 28 08:46:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952315 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DC671DE4E3; Tue, 28 Jan 2025 08:47:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054050; cv=none; b=NJFrlmAFJFlPTPfGpOlACzIG+7Aq/7E4Xp6Cag0p148wJGcZk+uL7hqR3ginArOIL0scQgws/hvXcdNnwwLHVfvqX4W8Z4yWiqm/mnsXa74aWIdcdY/CjUZb463wTTvf7LfnHuqT8QYsf7wxgneAxCAVjt/MkqZAfynlA3KpqPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054050; c=relaxed/simple; bh=qgLgc+vtWhxd3wDw4kFYbDMlFJVL1MhDz8zW3gWKabs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SqZ0nVp9AGg68rDuGg8N1ARP7LMAPMAtiZGQoxn6tSngKSkm+xMiiFla304BAk1xBsiugZHohdUFRAB5QpZ2eYpZ913WEA67Sg9UE5EOyA0dRNg3xaiLWGWf2nTL9oYIGuCITmXIGzpmyCISYqiy5tHPwDxWyM9QBG7NCFgd9eU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Os+ezm76; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Os+ezm76" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-216395e151bso68161735ad.0; Tue, 28 Jan 2025 00:47:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054048; x=1738658848; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KivH3GOTURvpBe4gz30uCxSbbTX4Z4qxExfXxl7mwmc=; b=Os+ezm76gyzffeguTHujn0IxilGZFzdq9/VN6GWG+r7RAIr1O9E0wyhRDmTjGBGj44 V7Q8w/GkbZS1C28osEABoUVNBSzbkkXdjegUlNVvq2+wTp82DUUOA0anBSqUP736IrV6 69un6dg4X1xrO87tjPAloI05/g+gMm4fPFu2k6LCOLzBmhBUZo+vVPS5K7HHX942kp3a i6Ga/UF7H1KDJ2vNgLGLwTgAiJT6q8NXC8BjMBQr405WDraUzpVu5SRhmgDzzeZdLY2b RWz4tecZGG9ugd21YeuYzDpJugAIyuu7GkXnHc727VvI4TFViHwzoUAwczOW6j4Z0dVL uOkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054048; x=1738658848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KivH3GOTURvpBe4gz30uCxSbbTX4Z4qxExfXxl7mwmc=; b=r6hmCS5Y0Mc6zxxLFKaIcPQFF7WjDyDyoQFz8/2LfSvSOFn7pTMQTbHG+XsOTRLXy9 RW7It0Nan/PfQpWrXLohpDdfcMXQ403e0irISEn2XDag0ZaEfzHMJyVyCXVxs+e3P82T X79/ebueI22kaARcMzYdfeI9esDLnxTpof/i/7bLPsFbqxHoNmyV2dTx12Lf5uxtJLDJ mqfze3LymXHrEdR742y8EM3gbMsFHdxiR0mePeaFTs/tnNsticTqFJRMHlDdBd4dru5x p0WA79H4LcHT3D/NJCeI0TnQF/ESVvh9yjw0KMYvg5cA14FGb1tsWm5NuS+2j1a0k6+s lBUA== X-Forwarded-Encrypted: i=1; AJvYcCXmziBdLnZsi1KLkonROQyWTnzzyFu5yTI62t34E4r4EMFsW2+Cdkok3Sxowq4GCzAcrOzKOJ4=@vger.kernel.org X-Gm-Message-State: AOJu0YyAQVuQ7u8bc0C0OB2Zo/kSeWRgw9zLLnfao77kiVl9OcAJxf5n kh3Rnn+ej+bYyzT0eQiI/YK2wST3TzOGqVk10+vIbp72SSaf3e++ X-Gm-Gg: ASbGncueO72qipOKpJr90A04NwhZt6VMBzqfPwfdxi2QBwkEKQ6lHX7gFCUa6CF945L +Tadc9YkS/QuE+0Iwf9bgQFU3iNPhVFQErNPdpizNbH4pRJPZqXxghwvEvu97zxZ1thaIWFpBX0 f5eLkYLRmIoEs8p2DiiCeWqZcVY0i2P8TBhevXFciBDjCQVWPIn1LneF5Nggkw20guI2mNhHj62 W1r/3Jpng65DDdHRa5t/nmWyH33LR8P3byWuenlBR9+5eF+rwQMeZJU87pxIwha3FVIgGqQhmdP Nmw81QD/RjOAhz72Y1QuumYmDCvrj5PcroB/hPHKh3ZY1XL/HxGzVQ== X-Google-Smtp-Source: AGHT+IEepHaPQF3Lq7vjMgHI8gu1HtaC3+TluzYakou8W3Z/w4XOR+IN0pFo8+36poE6BHZURKGWaw== X-Received: by 2002:a17:902:f687:b0:216:4b6f:dde5 with SMTP id d9443c01a7336-21dcc4a500bmr37897205ad.13.1738054048399; Tue, 28 Jan 2025 00:47:28 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:28 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 10/13] net-timestamp: make TCP tx timestamp bpf extension work Date: Tue, 28 Jan 2025 16:46:17 +0800 Message-Id: <20250128084620.57547-11-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Make partial of the feature work finally. After this, user can fully use the bpf prog to trace the tx path for TCP type. Signed-off-by: Jason Xing --- net/ipv4/tcp.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0d704bda6c41..0a41006b10d1 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -492,6 +492,15 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } + + if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) { + struct skb_shared_info *shinfo = skb_shinfo(skb); + struct tcp_skb_cb *tcb = TCP_SKB_CB(skb); + + tcb->txstamp_ack_bpf = 1; + shinfo->tx_flags |= SKBTX_BPF; + shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; + } } static bool tcp_stream_is_readable(struct sock *sk, int target) From patchwork Tue Jan 28 08:46:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952316 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B39CB1DF738; Tue, 28 Jan 2025 08:47:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054056; cv=none; b=pAUlzkB39E3Sp6ykRyBPGWy9JvapDh+z0zoL1tRZiHoX8qI3kTQfMPZ9lxV8mQ8B3v45rd6++ScqKrU5bczIAQnTIZiL/QmOtMeCf1dmFUhWMz0bx1K3Mlio0Zoc8JxLizcN2Z1SBH7foqCL7vu07bJ3+tUDkC6UewYDI7TXYGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054056; c=relaxed/simple; bh=aNb8u7WdCXGl59VHWWfhDdMK5wqD7qDrJJnP35x5Hvc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CSnGRHWW/p8tj9RNrO9gyHAF9u0+it918lGND7oQp0PR72OeMPqUdpd8Xl+FffvRXDphvtZ0VoqASmtHqoh6CIwgpK3xzXpWMplUaEJ12N260VEF5sKi6ry9/DB48CPuoDvCSqotMyTbvzshrNNK8AemYwzzesmSdS3jiMMGwEI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lM+YGB2B; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lM+YGB2B" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2164b662090so103812355ad.1; Tue, 28 Jan 2025 00:47:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054054; x=1738658854; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=60jA2uSFXnbKbsU07z2IqYkTW4O6QVW5Lrq9IN0HxbY=; b=lM+YGB2BU1WEbN8ycBawS4AByovzmcwv5tVK7vUnqb7blGEHC8OXejsD+yRnp9tl2v FMTrwSUhUY7jANFuMl/BupXst/maQiSkSf1yIsMizWtq2VVQQ3YuOg7525BpJT7PjDxW IkTZe83oGkduELsS4VW+eoc14fzBNgxsxLKJYEEhMEuR78CHz0Fsro8krqkh7Bza7UuD tmqcFkMH0NW0ZVG4OBk4tcwfD1zWO5krmRoIuSkWprvTO/Sg7rPRccArpNnPbAGBTZIA tCDsQDh9MnmAk5uC8AjFtMoXY3bl/iZRNKjBxV8H1sw9NKI2RVfCPlFNjtJ18CKRJp9T mPgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054054; x=1738658854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=60jA2uSFXnbKbsU07z2IqYkTW4O6QVW5Lrq9IN0HxbY=; b=SCdZLryGi2cwcC4EXQn2enUT07meoiDWZzvTu3mOd28rkltfemCQzIz0h3aSjXf9Co zeqqzMyH/y5Ll9sstV7yd3jK1eUA/Z7kDsEo2dIOXRmSoOy6Yrmb2KpoWKnjh+DdwJdH VHZdqCwy1MvgOFKQVl34fHFSkPZdVYX6MXe0o+zofEI99+NrkWQHTcJgeD24BwRvSRDC LMKN39mf9MFC7MAg2I58F0llWuSDte7bTvhT7pqJyUTdxI/P/FdKaJn5mIfBG+ncFLMK 9hpHbiSOV8nfAhmchNFQaxIm7no3qttzzoBLaaBPt6SSbMWyInNm3Hkry7XtIb3KVu0B TD6w== X-Forwarded-Encrypted: i=1; AJvYcCUBxpuoAj7VFc5ShHUHdchsK00WQd91XZlg27qGpZXR9hvkFs0xOI4uP0pbeNHvX7nKzhRPJ74=@vger.kernel.org X-Gm-Message-State: AOJu0Yx5f9N1ls1/qN7uLAgMyRKbl5IZSNsoRlIBCur5pNZbFQq/rt/O YCDCCURrZ529N3kDJ+g9cgpHmpE3xGv0xFgCwyP5NiPsKWXr3YPz X-Gm-Gg: ASbGncu05cQ9d7VC6q6Yvh7gYhyUzClbR0rmRiD5mRiNyGdqNwdDfHlAiovvlQ+WMIf oMLVuD3ZszTjpwh3QaF4h0VX1NIRd/rmOxIyFao5j9zRcrpGNSxD8jICtcYq1kvKb6SY9NOBeE/ XbZSvF5CWtq/5AaEotdshbSTFCGaZijSKbHoaAuDT+4RF8JNun87HG05yhPRON+jGp3mCnHdVrt BPNiEfpiF7aXWh+x9PTF0ZPPojIgpy04lGST294CYpyE6pYXct0fXW7dka/Fy+La+B7lIJFUa8N zTOmG8PUCUwi5QR0caSx3PMc3ivfivxClNDY3WKK4dqYqSNkqg00Ow== X-Google-Smtp-Source: AGHT+IEWl/dzQtDB8tEnoABnbUkYynUMTwx7kv3c9y2hpEh+t3iCo8rtNWpyeIRlAwhePHFK38UYjA== X-Received: by 2002:a17:903:703:b0:216:31aa:1308 with SMTP id d9443c01a7336-21c355e35c9mr538630815ad.34.1738054053867; Tue, 28 Jan 2025 00:47:33 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:33 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 11/13] net-timestamp: add a new callback in tcp_tx_timestamp() Date: Tue, 28 Jan 2025 16:46:18 +0800 Message-Id: <20250128084620.57547-12-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Introduce the callback to correlate tcp_sendmsg timestamp with other points, like SND/SW/ACK. We can let bpf trace the beginning of tcp_sendmsg_locked() and fetch the socket addr, so that in tcp_tx_timestamp() we can correlate the tskey with the socket addr. It is accurate since they are under the protect of socket lock. More details can be found in the selftest. Signed-off-by: Jason Xing --- include/uapi/linux/bpf.h | 7 +++++++ net/ipv4/tcp.c | 1 + tools/include/uapi/linux/bpf.h | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 800122a8abe5..accb3b314fff 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7052,6 +7052,13 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SND_CB, /* Called when every sendmsg syscall + * is triggered. For TCP, it stays + * in the last send process to + * correlate with tcp_sendmsg timestamp + * with other timestamping callbacks, + * like SND/SW/ACK. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0a41006b10d1..b2f1fd216df1 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -500,6 +500,7 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) tcb->txstamp_ack_bpf = 1; shinfo->tx_flags |= SKBTX_BPF; shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; + bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB); } } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 06e68d772989..384502996cdd 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7045,6 +7045,13 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SND_CB, /* Called when every sendmsg syscall + * is triggered. For TCP, it stays + * in the last send process to + * correlate with tcp_sendmsg timestamp + * with other timestamping callbacks, + * like SND/SW/ACK. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Jan 28 08:46:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952317 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16A051DED67; Tue, 28 Jan 2025 08:47:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054062; cv=none; b=mkQl1+NqWtEVDlST34m82P6xMJ1aWvXve6ODsPjbhgx8nkHbuIA31DKCx+ZvQfrp+9dBrVEnb/+trGYGgyH7AFxftWIp+uf5T+C2BfB8PaPJLGXsRMW8ufG648BcatzgPL8ZIakZAd5O2wGjFfsk/jNywIfDHE627VxPIF+w9MQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054062; c=relaxed/simple; bh=799BK+r9QuZWg9Gtk48q6q1sa2nNf/sTykN5qorac7k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Z1F2dUsdt6Y3poK+Nv9OltQ38pHXdugIIi+aUa/Mj35WNU2/BlOlFLjFUq43Fm5l3tz8/u8ANvb6OyIjJal0ys2kbaotMmF7Z3qWAVl2mOmeOtf6cB52bfSKGokd5xKsW9QpAVUd9PxkI5tw6XAtMTscLc6ALYpfSqpW10/iaOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fQtln9N4; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fQtln9N4" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2164b1f05caso91349195ad.3; Tue, 28 Jan 2025 00:47:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054060; x=1738658860; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XNUgFFnrif8jgeaUElfsrSn4qMqyCaNOvtGck2c3Qpc=; b=fQtln9N4UQuZO35sIvgYF5ZsaQwDafdNKloPgDlTsfrYCbEuqyUV6+VOwNsKlcZo65 ZHYS4H8g9qZ5UDHtmOFOv4/g3nRI1wm+JP9e/DPHeOxKW/bmSfZ5QCSWQ40iUi1F6f8x 52TFbqRtT9F7Nhwt+IBcJR+TbW+ljpXfWv/10CbG6lIBUMQ0+VRnOBABdD076UI+cW3r Zgjwwax/alcFlfPeeIlIQ6VEdXZb975dzDVnLSoLGDbIjUShHENTN5jySPATuj6s4QU4 0QRIYY0ANHpZkoKnANwKgchbu9PvvQcaFlW81YgiRPYLHBuzcB/fN0XV+/eG5N1bffJa 1fRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054060; x=1738658860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XNUgFFnrif8jgeaUElfsrSn4qMqyCaNOvtGck2c3Qpc=; b=sdNiJDhOuuh4fjpJ/jKKED5pm7Vj+J8GlPBL9eOh1ioBiEXEzWpvYGUuETS9AfVJcG Fht3Wsz11QLSMiFcF3EZ4/EO2BarppyEd89SB8avmKFr4l3E348efVyouKkQiPDi2l76 0YL4IsKAp8C+iIH8IfE+Br7m2cXtmrwgYG06DadggNb02vuJnXUwKMjx9hc57tz2HHhZ WS5ee84zWNiTdJzj+Lk/wKqXNavn2jVIWvm9f11QNfTL9qm1XsGP7tIlbZ1MhzbOhARA 7lwz7zuFO17+KOl3yefl2TF2Udp/JCo3D5N64Sn7coGH94AavrlD7ENMx5I2GtkbSMGO QqOg== X-Forwarded-Encrypted: i=1; AJvYcCUIRfxW7aKyhBYiOlLn5q41/sZVpikpdAldGfv51mofJfdYRov1WPvNaxPN5SziEJnK8FRBEys=@vger.kernel.org X-Gm-Message-State: AOJu0YzeSwGQWJckJAngTivbLr/xkgDk43SNkvUsljVEtypx53exLYpQ on8jvQfMNlKIpvnN+uOoVPbUBV0bE71luqD5GcmvfXdfHcUgt4RC X-Gm-Gg: ASbGncsYKsW+2XT0oeQyara9CbLchTxbFNzw84ZWUbviKanWvk6nQsKw8qEJHUxyt3n zOOcHid9LzVouwfqv7cL1XQsIbtbYd5nwCADW419W9LAfQ2Nu9/ft6d86GxE9j7GANhgYSeuq+o OP4gokoAKyRELqb+0zgtc/fZTK0irPzySapGvoBdjDjLNbDuETsQ0wyHrSyPEpbJ5CYOEDe8q1Q 9+GhaAa8SWszBdTm64Q0T746qebA+Zj1g0UVZw1T0aPOwAAeKsScgll5VIpLOci2gR07VXFn8dj hCL0fLtyQeaRJGeXHZL6naXA7RmmZWWj/Qd+OtyBbrDSVzrU8v46GQ== X-Google-Smtp-Source: AGHT+IGeOLZdZJxvcoHQ3oD57LURLSjLYP9tMx5Q1Wtn3wIXuY8cVkpR6xi+1D6q/lqpupBQI7gljg== X-Received: by 2002:a17:902:cf0b:b0:216:2af7:a2a3 with SMTP id d9443c01a7336-21c3560726cmr767853725ad.53.1738054059083; Tue, 28 Jan 2025 00:47:39 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:38 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 12/13] net-timestamp: introduce cgroup lock to avoid affecting non-bpf cases Date: Tue, 28 Jan 2025 16:46:19 +0800 Message-Id: <20250128084620.57547-13-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Introducing the lock to avoid affecting the applications which are not using timestamping bpf feature. Signed-off-by: Jason Xing --- net/ipv4/tcp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index b2f1fd216df1..a2ac57543b6d 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -493,7 +493,8 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } - if (SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) { + if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) && + SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) { struct skb_shared_info *shinfo = skb_shinfo(skb); struct tcp_skb_cb *tcb = TCP_SKB_CB(skb); From patchwork Tue Jan 28 08:46:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13952318 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A01951DED67; Tue, 28 Jan 2025 08:47:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054068; cv=none; b=BL+eQNyrmT7nM4OkgM1cglG786u16lx7STIVxWvaWRFvP13JFNh7ThP5HVWrAWtk4GS+0jCVCFLlkd30wfslxd9pwhHvdmQ8wwHXKeVDA8cY31lPGTnODMnrgikXhQQZJmn2Qn8la3W8y12jBcA2a6Tbn4gJ07+fqBuStMIZSAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738054068; c=relaxed/simple; bh=HSsKDqfsjp5fopc3ikPD6v9u+CpaF3o6GtowEZi30Uc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Q/VVCYPwK8d/wEi5xej5K4r6O+M51rGGnXW7ChWdaaFB6zOS8ZdH9O/SPzq5k/W6xHYNMW0PbJHDB5a1/4OOWbvE7oEVbqp5tR6WWZwlzgGYv3xsBjZu5/nrbxU4puiej6OqVbwXD45BwEMxA1sBNPPXtDCSlQtlD5IhInMLTds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WlKLIwIh; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WlKLIwIh" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-216401de828so97142495ad.3; Tue, 28 Jan 2025 00:47:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738054066; x=1738658866; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gbXhFqWfxeW6KXHCx6cQeU+eZI9I0iF185XwKjE3BUM=; b=WlKLIwIhCliqybl//M2OjL5EmWK/kWy6YnUhHTY91KKtWWCq01Slwv2MMZQk2nXySC triNCcGQd/e0/jk02UF0gkVZBFVSBXVn5XlWodAgthqyA/WlmqLKLtidnghEOxE/2PCv fz2s73JGu0xN5aXX7h6aKgCTVYxBhZj32OUXGV9v+Mo4Dk1zxXobsS8iOx+Cgvf/vCeC wIE4NXDTygclLXT+TdQVR4TX761cTGCc+USswpTinNBlDf79W0sYtUJrzcHNLg/qkeDV DQXUucn7ZMoRPx5a7beWKqOYgsX6WM5tBqiz+Qj1FhM5vZ32ItsfWSNjib7uS8q41Ge4 Aa4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738054066; x=1738658866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gbXhFqWfxeW6KXHCx6cQeU+eZI9I0iF185XwKjE3BUM=; b=uQpbzd20GMi3csihxNJmT4UOzFx5HZcDxIdlrk9cH1KDWXzfsjn4RSnRh6AVOY5wLc WOTETSdxV3+k95VavPmlRB5VX782eW27NbleN3oXeLCgmEirVbat6vadLQEStyo4vfUS 8NPZ+1jrsh6Z6FPdEo0fvsqWlbxXlbpO9VEYJFOPCCSlluMZ5PPsqpV8IhYGD9ZPkDtr 8uqX5unOgyz5A3ZbLeDFvG+ln8tJve4pPKqSvbxXo14F080tAMFiv/FC0q4OSr1NK6o/ f9fv6i8mNvW9HJobZAXpAmCfN6i+cCN0JlKeOH9dwb3P0R7JJv0g5HOUoW2EpRgIW9ci NbOA== X-Forwarded-Encrypted: i=1; AJvYcCXhJeRcVhnZgHo9KXkDUlFKnwN7zGhJFJGAG1/H2Ci7JqpXECC4IotWXH/nKekTHW/k1fL0bJY=@vger.kernel.org X-Gm-Message-State: AOJu0YwIvrVQitSA3n+xhzMVBrD1WPWH94HkxrHxEqunXmriwiqLzIfC c5SNrBWyOllq65Q3QPBpbm3gahYKmx0JcxEQTsbInRR+0gmapmZ3 X-Gm-Gg: ASbGnctf4sAkEiSY+g1Go53zXsmQ9pJP9nxa3rUgkg9E9VBqWVpZ7wx0DsvHJuID0sA 5exQeE2dzu/WRSaKjy3sDC/STKRrjQXbO3Lx+10MR/bF9C+4rUfMcE3C9DFmRTP3W5cA5za2dip qwekEnnn7jHYM4WoICLK10gmaQEpCuAN0HRxErgf6ignG3rJg8oXgaO4imATJGWBSUknJrIJ15J MdaYncsswnxP+woksup/HovA3L+EGbCWrhtp6JfvB4/ofuagmIN8oKOnXac4cDDlqY3NzZm1x2u nLkEoMaNnlOEtM4MACuM7lIC9nhl9YM4RNd3vvchV1ROm4Ge0m3tCd2Vc4i7n/Pl X-Google-Smtp-Source: AGHT+IFYAf/5X0dP5hKm17V+bZbPN1OiCMGd8qqMzz+iQbVbiNy7gVThbh6KIQjfd0zaoHY5YTfbUQ== X-Received: by 2002:a17:902:c94d:b0:216:55a1:369 with SMTP id d9443c01a7336-21c3540179amr628897295ad.18.1738054065737; Tue, 28 Jan 2025 00:47:45 -0800 (PST) Received: from KERNELXING-MC1.tencent.com ([111.201.25.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da414e2b8sm75873275ad.205.2025.01.28.00.47.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 00:47:45 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, horms@kernel.org Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v7 13/13] bpf: add simple bpf tests in the tx path for so_timestamping feature Date: Tue, 28 Jan 2025 16:46:20 +0800 Message-Id: <20250128084620.57547-14-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250128084620.57547-1-kerneljasonxing@gmail.com> References: <20250128084620.57547-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Only check if we pass those three key points after we enable the bpf extension for so_timestamping. During each point, we can choose whether to print the current timestamp. Signed-off-by: Jason Xing --- .../bpf/prog_tests/so_timestamping.c | 86 +++++ .../selftests/bpf/progs/so_timestamping.c | 299 ++++++++++++++++++ 2 files changed, 385 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/so_timestamping.c create mode 100644 tools/testing/selftests/bpf/progs/so_timestamping.c diff --git a/tools/testing/selftests/bpf/prog_tests/so_timestamping.c b/tools/testing/selftests/bpf/prog_tests/so_timestamping.c new file mode 100644 index 000000000000..ee7fdc381609 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/so_timestamping.c @@ -0,0 +1,86 @@ +#define _GNU_SOURCE +#include +#include +#include +#include + +#include "test_progs.h" +#include "cgroup_helpers.h" +#include "network_helpers.h" + +#include "so_timestamping.skel.h" + +#define CG_NAME "/so-timestamping-test" + +static const char addr4_str[] = "127.0.0.1"; +static const char addr6_str[] = "::1"; +static struct so_timestamping *skel; +static int cg_fd; + +static void test_tcp(int family) +{ + struct so_timestamping__bss *bss = skel->bss; + char buf[] = "testing testing"; + int sfd = -1, cfd = -1; + int n; + + memset(bss, 0, sizeof(*bss)); + + sfd = start_server(family, SOCK_STREAM, + family == AF_INET6 ? addr6_str : addr4_str, 0, 0); + if (!ASSERT_OK_FD(sfd, "start_server")) + goto out; + + cfd = connect_to_fd(sfd, 0); + if (!ASSERT_OK_FD(cfd, "connect_to_fd_server")) + goto out; + + n = write(cfd, buf, sizeof(buf)); + if (!ASSERT_EQ(n, sizeof(buf), "send to server")) + goto out; + + ASSERT_EQ(bss->nr_active, 1, "nr_active"); + ASSERT_EQ(bss->nr_snd, 2, "nr_snd"); + ASSERT_EQ(bss->nr_sched, 1, "nr_sched"); + ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw"); + ASSERT_EQ(bss->nr_ack, 1, "nr_ack"); + +out: + if (sfd >= 0) + close(sfd); + if (cfd >= 0) + close(cfd); +} + +void test_so_timestamping(void) +{ + struct netns_obj *ns; + + cg_fd = test__join_cgroup(CG_NAME); + if (cg_fd < 0) + return; + + ns = netns_new("so_timestamping_ns", true); + if (!ASSERT_OK_PTR(ns, "create ns")) + return; + + skel = so_timestamping__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open and load skel")) + goto done; + + if (!ASSERT_OK(so_timestamping__attach(skel), "attach skel")) + goto done; + + skel->links.skops_sockopt = + bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd); + if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup")) + goto done; + + test_tcp(AF_INET6); + test_tcp(AF_INET); + +done: + so_timestamping__destroy(skel); + netns_free(ns); + close(cg_fd); +} diff --git a/tools/testing/selftests/bpf/progs/so_timestamping.c b/tools/testing/selftests/bpf/progs/so_timestamping.c new file mode 100644 index 000000000000..a893859ffe32 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/so_timestamping.c @@ -0,0 +1,299 @@ +#include "vmlinux.h" +#include "bpf_tracing_net.h" +#include +#include +#include +#include "bpf_misc.h" +#include "bpf_kfuncs.h" +#define BPF_PROG_TEST_TCP_HDR_OPTIONS +#include "test_tcp_hdr_options.h" +#include + +#define SK_BPF_CB_FLAGS 1009 +#define SK_BPF_CB_TX_TIMESTAMPING 1 + +int nr_active; +int nr_snd; +int nr_passive; +int nr_sched; +int nr_txsw; +int nr_ack; + +struct sockopt_test { + int opt; + int new; +}; + +static const struct sockopt_test sol_socket_tests[] = { + { .opt = SK_BPF_CB_FLAGS, .new = SK_BPF_CB_TX_TIMESTAMPING, }, + { .opt = 0, }, +}; + +struct loop_ctx { + void *ctx; + const struct sock *sk; +}; + +struct sk_stg { + __u64 sendmsg_ns; /* record ts when sendmsg is called */ +}; + +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct sk_stg); +} sk_stg_map SEC(".maps"); + + +struct delay_info { + u64 sendmsg_ns; /* record ts when sendmsg is called */ + u32 sched_delay; /* SCHED_OPT_CB - sendmsg_ns */ + u32 sw_snd_delay; /* SW_OPT_CB - SCHED_OPT_CB */ + u32 ack_delay; /* ACK_OPT_CB - SW_OPT_CB */ +}; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, u32); + __type(value, struct delay_info); + __uint(max_entries, 1024); +} time_map SEC(".maps"); + +static u64 delay_tolerance_nsec = 10000000000; /* 10 second as an example */ + +static int bpf_test_sockopt_int(void *ctx, const struct sock *sk, + const struct sockopt_test *t, + int level) +{ + int new, opt, tmp; + + opt = t->opt; + new = t->new; + + if (bpf_setsockopt(ctx, level, opt, &new, sizeof(new))) + return 1; + + if (bpf_getsockopt(ctx, level, opt, &tmp, sizeof(tmp)) || + tmp != new) + return 1; + + return 0; +} + +static int bpf_test_socket_sockopt(__u32 i, struct loop_ctx *lc) +{ + const struct sockopt_test *t; + + if (i >= ARRAY_SIZE(sol_socket_tests)) + return 1; + + t = &sol_socket_tests[i]; + if (!t->opt) + return 1; + + return bpf_test_sockopt_int(lc->ctx, lc->sk, t, SOL_SOCKET); +} + +static int bpf_test_sockopt(void *ctx, const struct sock *sk) +{ + struct loop_ctx lc = { .ctx = ctx, .sk = sk, }; + int n; + + n = bpf_loop(ARRAY_SIZE(sol_socket_tests), bpf_test_socket_sockopt, &lc, 0); + if (n != ARRAY_SIZE(sol_socket_tests)) + return -1; + + return 0; +} + +static bool bpf_test_access_sockopt(void *ctx) +{ + const struct sockopt_test *t; + int tmp, ret, i = 0; + int level = SOL_SOCKET; + + t = &sol_socket_tests[i]; + + for (; t->opt;) { + ret = bpf_setsockopt(ctx, level, t->opt, (void *)&t->new, sizeof(t->new)); + if (ret != -EOPNOTSUPP) + return true; + + ret = bpf_getsockopt(ctx, level, t->opt, &tmp, sizeof(tmp)); + if (ret != -EOPNOTSUPP) + return true; + + if (++i >= ARRAY_SIZE(sol_socket_tests)) + break; + } + + return false; +} + +/* Adding a simple test to see if we can get an expected value */ +static bool bpf_test_access_load_hdr_opt(struct bpf_sock_ops *skops) +{ + struct tcp_opt reg_opt; + int load_flags = 0; + int ret; + + reg_opt.kind = TCPOPT_EXP; + reg_opt.len = 0; + reg_opt.data32 = 0; + ret = bpf_load_hdr_opt(skops, ®_opt, sizeof(reg_opt), load_flags); + if (ret != -EOPNOTSUPP) + return true; + + return false; +} + +/* Adding a simple test to see if we can get an expected value */ +static bool bpf_test_access_cb_flags_set(struct bpf_sock_ops *skops) +{ + int ret; + + ret = bpf_sock_ops_cb_flags_set(skops, 0); + if (ret != -EOPNOTSUPP) + return true; + + return false; +} + +/* In the timestamping callbacks, we're not allowed to call the following + * BPF CALLs for the safety concern. Return false if expected. + */ +static int bpf_test_access_bpf_calls(struct bpf_sock_ops *skops, + const struct sock *sk) +{ + if (bpf_test_access_sockopt(skops)) + return true; + + if (bpf_test_access_load_hdr_opt(skops)) + return true; + + if (bpf_test_access_cb_flags_set(skops)) + return true; + + return false; +} + +static bool bpf_test_delay(struct bpf_sock_ops *skops, const struct sock *sk) +{ + struct bpf_sock_ops_kern *skops_kern; + u64 timestamp = bpf_ktime_get_ns(); + struct skb_shared_info *shinfo; + struct delay_info dinfo = {0}; + struct delay_info *val; + struct sk_buff *skb; + struct sk_stg *stg; + u64 prior_ts, delay; + u32 tskey; + + if (bpf_test_access_bpf_calls(skops, sk)) + return false; + + skops_kern = bpf_cast_to_kern_ctx(skops); + skb = skops_kern->skb; + shinfo = bpf_core_cast(skb->head + skb->end, struct skb_shared_info); + tskey = shinfo->tskey; + if (!tskey) + return false; + + if (skops->op == BPF_SOCK_OPS_TS_SND_CB) { + stg = bpf_sk_storage_get(&sk_stg_map, (void *)sk, 0, 0); + if (!stg) + return false; + dinfo.sendmsg_ns = stg->sendmsg_ns; + bpf_map_update_elem(&time_map, &tskey, &dinfo, BPF_ANY); + goto out; + } + + val = bpf_map_lookup_elem(&time_map, &tskey); + if (!val) + return false; + + switch (skops->op) { + case BPF_SOCK_OPS_TS_SCHED_OPT_CB: + delay = val->sched_delay = timestamp - val->sendmsg_ns; + break; + case BPF_SOCK_OPS_TS_SW_OPT_CB: + prior_ts = val->sched_delay + val->sendmsg_ns; + delay = val->sw_snd_delay = timestamp - prior_ts; + break; + case BPF_SOCK_OPS_TS_ACK_OPT_CB: + prior_ts = val->sw_snd_delay + val->sched_delay + val->sendmsg_ns; + delay = val->ack_delay = timestamp - prior_ts; + break; + } + + if (delay >= delay_tolerance_nsec) + return false; + + /* Since it's the last one, remove from the map after latency check */ + if (skops->op == BPF_SOCK_OPS_TS_ACK_OPT_CB) + bpf_map_delete_elem(&time_map, &tskey); + +out: + return true; +} + +SEC("fentry/tcp_sendmsg_locked") +int BPF_PROG(trace_tcp_sendmsg_locked, struct sock *sk, struct msghdr *msg, size_t size) +{ + u64 timestamp = bpf_ktime_get_ns(); + u32 flag = sk->sk_bpf_cb_flags; + struct sk_stg *stg; + + if (!flag) + return 0; + + stg = bpf_sk_storage_get(&sk_stg_map, sk, 0, + BPF_SK_STORAGE_GET_F_CREATE); + if (!stg) + return 0; + + stg->sendmsg_ns = timestamp; + nr_snd += 1; + return 0; +} + +SEC("sockops") +int skops_sockopt(struct bpf_sock_ops *skops) +{ + struct bpf_sock *bpf_sk = skops->sk; + const struct sock *sk; + + if (!bpf_sk) + return 1; + + sk = (struct sock *)bpf_skc_to_tcp_sock(bpf_sk); + if (!sk) + return 1; + + switch (skops->op) { + case BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: + nr_active += !bpf_test_sockopt(skops, sk); + break; + case BPF_SOCK_OPS_TS_SND_CB: + if (bpf_test_delay(skops, sk)) + nr_snd += 1; + break; + case BPF_SOCK_OPS_TS_SCHED_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_sched += 1; + break; + case BPF_SOCK_OPS_TS_SW_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_txsw += 1; + break; + case BPF_SOCK_OPS_TS_ACK_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_ack += 1; + break; + } + + return 1; +} + +char _license[] SEC("license") = "GPL";