From patchwork Wed Dec 9 14:37:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Geva, Erez" X-Patchwork-Id: 11961773 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55BB7C4361B for ; Wed, 9 Dec 2020 15:43:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19CA522AAC for ; Wed, 9 Dec 2020 15:43:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730572AbgLIPnP (ORCPT ); Wed, 9 Dec 2020 10:43:15 -0500 Received: from thoth.sbs.de ([192.35.17.2]:55305 "EHLO thoth.sbs.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726110AbgLIPnO (ORCPT ); Wed, 9 Dec 2020 10:43:14 -0500 X-Greylist: delayed 3797 seconds by postgrey-1.27 at vger.kernel.org; Wed, 09 Dec 2020 10:43:13 EST Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 0B9EbPmO022112 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 9 Dec 2020 15:37:25 +0100 Received: from tsnlaptop.atstm41.nbgm.siemens.de ([144.145.220.34]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id 0B9EbGp1002581; Wed, 9 Dec 2020 15:37:22 +0100 From: Erez Geva To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, Alexey Kuznetsov , Arnd Bergmann , Cong Wang , "David S . Miller" , Hideaki YOSHIFUJI , Jakub Kicinski , Jamal Hadi Salim , Jiri Pirko , Alexei Starovoitov , Colin Ian King , Daniel Borkmann , Eric Dumazet , Eyal Birger , "Gustavo A . R . Silva" , Jakub Sitnicki , John Ogness , Jon Rosen , Kees Cook , Mao Wenan , Marc Kleine-Budde , Martin KaFai Lau , Matthieu Baerts , Andrei Vagin , Dmitry Safonov <0x7f454c46@gmail.com>, "Eric W . Biederman" , Ingo Molnar , John Stultz , Miaohe Lin , Michal Kubecek , Or Cohen , Oleg Nesterov , Peter Zijlstra , Richard Cochran , Stefan Schmidt , Willem de Bruijn , Xie He , Stephen Boyd , Thomas Gleixner , Vladis Dronov , Sebastian Andrzej Siewior , Frederic Weisbecker Cc: Vinicius Costa Gomes , Vedang Patel , Ines Molzahn , Simon Sudler , Andreas Meisinger , Andreas Bucher , Henning Schild , Jan Kiszka , Andreas Zirkler , Ermin Sakic , An Ninh Nguyen , Michael Saenger , Bernd Maehringer , Gisela Greinert , Erez Geva , Erez Geva Subject: [PATCH 1/3] Add TX sending hardware timestamp. Date: Wed, 9 Dec 2020 15:37:04 +0100 Message-Id: <20201209143707.13503-2-erez.geva.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201209143707.13503-1-erez.geva.ext@siemens.com> References: <20201209143707.13503-1-erez.geva.ext@siemens.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Configure and send TX sending hardware timestamp from user space application to the socket layer, to provide to the TC ETC Qdisc, and pass it to the interface network driver. - New flag for the SO_TXTIME socket option. - New access auxiliary data header to pass the TX sending hardware timestamp. - Add the hardware timestamp to the socket cookie. - Copy the TX sending hardware timestamp to the socket cookie. Signed-off-by: Erez Geva Reported-by: kernel test robot --- include/net/sock.h | 2 ++ include/uapi/asm-generic/socket.h | 3 +++ include/uapi/linux/net_tstamp.h | 3 ++- net/core/sock.c | 9 +++++++++ 4 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/net/sock.h b/include/net/sock.h index a5c6ae78df77..dd5bfd42b4e2 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -859,6 +859,7 @@ enum sock_flags { SOCK_SELECT_ERR_QUEUE, /* Wake select on error queue */ SOCK_RCU_FREE, /* wait rcu grace period in sk_destruct() */ SOCK_TXTIME, + SOCK_HW_TXTIME, SOCK_XDP, /* XDP is attached */ SOCK_TSTAMP_NEW, /* Indicates 64 bit timestamps always */ }; @@ -1690,6 +1691,7 @@ void sk_send_sigurg(struct sock *sk); struct sockcm_cookie { u64 transmit_time; + u64 transmit_hw_time; u32 mark; u16 tsflags; }; diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h index 77f7c1638eb1..16265b00c25a 100644 --- a/include/uapi/asm-generic/socket.h +++ b/include/uapi/asm-generic/socket.h @@ -119,6 +119,9 @@ #define SO_DETACH_REUSEPORT_BPF 68 +#define SO_HW_TXTIME 69 +#define SCM_HW_TXTIME SO_HW_TXTIME + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 || (defined(__x86_64__) && defined(__ILP32__)) diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h index 7ed0b3d1c00a..dd51c9a99b1f 100644 --- a/include/uapi/linux/net_tstamp.h +++ b/include/uapi/linux/net_tstamp.h @@ -162,8 +162,9 @@ struct scm_ts_pktinfo { enum txtime_flags { SOF_TXTIME_DEADLINE_MODE = (1 << 0), SOF_TXTIME_REPORT_ERRORS = (1 << 1), + SOF_TXTIME_USE_HW_TIMESTAMP = (1 << 2), - SOF_TXTIME_FLAGS_LAST = SOF_TXTIME_REPORT_ERRORS, + SOF_TXTIME_FLAGS_LAST = SOF_TXTIME_USE_HW_TIMESTAMP, SOF_TXTIME_FLAGS_MASK = (SOF_TXTIME_FLAGS_LAST - 1) | SOF_TXTIME_FLAGS_LAST }; diff --git a/net/core/sock.c b/net/core/sock.c index 727ea1cc633c..317dce54321b 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1227,6 +1227,8 @@ int sock_setsockopt(struct socket *sock, int level, int optname, break; } sock_valbool_flag(sk, SOCK_TXTIME, true); + sock_valbool_flag(sk, SOCK_HW_TXTIME, + sk_txtime.flags & SOF_TXTIME_USE_HW_TIMESTAMP); sk->sk_clockid = sk_txtime.clockid; sk->sk_txtime_deadline_mode = !!(sk_txtime.flags & SOF_TXTIME_DEADLINE_MODE); @@ -2378,6 +2380,13 @@ int __sock_cmsg_send(struct sock *sk, struct msghdr *msg, struct cmsghdr *cmsg, return -EINVAL; sockc->transmit_time = get_unaligned((u64 *)CMSG_DATA(cmsg)); break; + case SCM_HW_TXTIME: + if (!sock_flag(sk, SOCK_HW_TXTIME)) + return -EINVAL; + if (cmsg->cmsg_len != CMSG_LEN(sizeof(u64))) + return -EINVAL; + sockc->transmit_hw_time = get_unaligned((u64 *)CMSG_DATA(cmsg)); + break; /* SCM_RIGHTS and SCM_CREDENTIALS are semantically in SOL_UNIX. */ case SCM_RIGHTS: case SCM_CREDENTIALS: From patchwork Wed Dec 9 14:37:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Geva, Erez" X-Patchwork-Id: 11961801 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FC6DC433FE for ; Wed, 9 Dec 2020 15:59:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0940323B52 for ; Wed, 9 Dec 2020 15:59:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730863AbgLIP67 (ORCPT ); Wed, 9 Dec 2020 10:58:59 -0500 Received: from david.siemens.de ([192.35.17.14]:59031 "EHLO david.siemens.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729303AbgLIP67 (ORCPT ); Wed, 9 Dec 2020 10:58:59 -0500 Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 0B9EbS7n007624 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 9 Dec 2020 15:37:28 +0100 Received: from tsnlaptop.atstm41.nbgm.siemens.de ([144.145.220.34]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id 0B9EbGp2002581; Wed, 9 Dec 2020 15:37:25 +0100 From: Erez Geva To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, Alexey Kuznetsov , Arnd Bergmann , Cong Wang , "David S . Miller" , Hideaki YOSHIFUJI , Jakub Kicinski , Jamal Hadi Salim , Jiri Pirko , Alexei Starovoitov , Colin Ian King , Daniel Borkmann , Eric Dumazet , Eyal Birger , "Gustavo A . R . Silva" , Jakub Sitnicki , John Ogness , Jon Rosen , Kees Cook , Mao Wenan , Marc Kleine-Budde , Martin KaFai Lau , Matthieu Baerts , Andrei Vagin , Dmitry Safonov <0x7f454c46@gmail.com>, "Eric W . Biederman" , Ingo Molnar , John Stultz , Miaohe Lin , Michal Kubecek , Or Cohen , Oleg Nesterov , Peter Zijlstra , Richard Cochran , Stefan Schmidt , Willem de Bruijn , Xie He , Stephen Boyd , Thomas Gleixner , Vladis Dronov , Sebastian Andrzej Siewior , Frederic Weisbecker Cc: Vinicius Costa Gomes , Vedang Patel , Ines Molzahn , Simon Sudler , Andreas Meisinger , Andreas Bucher , Henning Schild , Jan Kiszka , Andreas Zirkler , Ermin Sakic , An Ninh Nguyen , Michael Saenger , Bernd Maehringer , Gisela Greinert , Erez Geva , Erez Geva Subject: [PATCH 2/3] Pass TX sending hardware timestamp to a socket's buffer. Date: Wed, 9 Dec 2020 15:37:05 +0100 Message-Id: <20201209143707.13503-3-erez.geva.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201209143707.13503-1-erez.geva.ext@siemens.com> References: <20201209143707.13503-1-erez.geva.ext@siemens.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Pass TX sending hardware timestamp from the socket layer to a socket's buffer. The TC ETC Qdisc will pass it to the interface network driver. - Add the hardware timestamp to the IP cork, to support the use of IP/UDP with the TX sending hardware timestamp when sending through the TC ETF Qdisc - Pass the TX sending hardware timestamp to a socket's buffer using the hardware timestamp on the socket's buffer shared information. Note: The socket buffer's hardware timestamp is used by the network interface driver to send the actual sending timestamp back to the application. The timestamp is used by the TC ETF before the buffer arrives in the network interface driver. Signed-off-by: Erez Geva --- include/net/inet_sock.h | 1 + net/ipv4/ip_output.c | 2 ++ net/ipv4/raw.c | 1 + net/ipv6/ip6_output.c | 2 ++ net/ipv6/raw.c | 1 + net/packet/af_packet.c | 3 +++ 6 files changed, 10 insertions(+) diff --git a/include/net/inet_sock.h b/include/net/inet_sock.h index 89163ef8cf4b..e74e8dabf63d 100644 --- a/include/net/inet_sock.h +++ b/include/net/inet_sock.h @@ -160,6 +160,7 @@ struct inet_cork { char priority; __u16 gso_size; u64 transmit_time; + u64 transmit_hw_time; u32 mark; }; diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 879b76ae4435..416598c864e3 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1282,6 +1282,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, cork->mark = ipc->sockc.mark; cork->priority = ipc->priority; cork->transmit_time = ipc->sockc.transmit_time; + cork->transmit_hw_time = ipc->sockc.transmit_hw_time; cork->tx_flags = 0; sock_tx_timestamp(sk, ipc->sockc.tsflags, &cork->tx_flags); @@ -1545,6 +1546,7 @@ struct sk_buff *__ip_make_skb(struct sock *sk, skb->priority = (cork->tos != -1) ? cork->priority: sk->sk_priority; skb->mark = cork->mark; skb->tstamp = cork->transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(cork->transmit_hw_time); /* * Steal rt from cork.dst to avoid a pair of atomic_inc/atomic_dec * on dst refcount diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c index 7d26e0f8bdae..213a47fb2df8 100644 --- a/net/ipv4/raw.c +++ b/net/ipv4/raw.c @@ -378,6 +378,7 @@ static int raw_send_hdrinc(struct sock *sk, struct flowi4 *fl4, skb->priority = sk->sk_priority; skb->mark = sockc->mark; skb->tstamp = sockc->transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(sockc->transmit_hw_time); skb_dst_set(skb, &rt->dst); *rtp = NULL; diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 749ad72386b2..8cff05f5aa8a 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1375,6 +1375,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, cork->base.length = 0; cork->base.transmit_time = ipc6->sockc.transmit_time; + cork->base.transmit_hw_time = ipc6->sockc.transmit_hw_time; return 0; } @@ -1841,6 +1842,7 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, skb->mark = cork->base.mark; skb->tstamp = cork->base.transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(cork->base.transmit_hw_time); skb_dst_set(skb, dst_clone(&rt->dst)); IP6_UPD_PO_STATS(net, rt->rt6i_idev, IPSTATS_MIB_OUT, skb->len); diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index 6e4ab80a3b94..417f21867782 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -648,6 +648,7 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length, skb->priority = sk->sk_priority; skb->mark = sockc->mark; skb->tstamp = sockc->transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(sockc->transmit_hw_time); skb_put(skb, length); skb_reset_network_header(skb); diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 7a18ffff8551..c762d5bcecc2 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -1986,6 +1986,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, skb->priority = sk->sk_priority; skb->mark = sk->sk_mark; skb->tstamp = sockc.transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(sockc.transmit_hw_time); skb_setup_tx_timestamp(skb, sockc.tsflags); @@ -2500,6 +2501,7 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, skb->priority = po->sk.sk_priority; skb->mark = po->sk.sk_mark; skb->tstamp = sockc->transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(sockc->transmit_hw_time); skb_setup_tx_timestamp(skb, sockc->tsflags); skb_zcopy_set_nouarg(skb, ph.raw); @@ -2975,6 +2977,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) skb->priority = sk->sk_priority; skb->mark = sockc.mark; skb->tstamp = sockc.transmit_time; + skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(sockc.transmit_hw_time); if (has_vnet_hdr) { err = virtio_net_hdr_to_skb(skb, &vnet_hdr, vio_le()); From patchwork Wed Dec 9 14:37:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Geva, Erez" X-Patchwork-Id: 11961797 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35414C4361B for ; Wed, 9 Dec 2020 15:55:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0053F22B47 for ; Wed, 9 Dec 2020 15:55:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732041AbgLIPze (ORCPT ); Wed, 9 Dec 2020 10:55:34 -0500 Received: from goliath.siemens.de ([192.35.17.28]:45088 "EHLO goliath.siemens.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729877AbgLIPz2 (ORCPT ); Wed, 9 Dec 2020 10:55:28 -0500 X-Greylist: delayed 4530 seconds by postgrey-1.27 at vger.kernel.org; Wed, 09 Dec 2020 10:55:21 EST Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 0B9EbVs3007322 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 9 Dec 2020 15:37:31 +0100 Received: from tsnlaptop.atstm41.nbgm.siemens.de ([144.145.220.34]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id 0B9EbGp3002581; Wed, 9 Dec 2020 15:37:28 +0100 From: Erez Geva To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, Alexey Kuznetsov , Arnd Bergmann , Cong Wang , "David S . Miller" , Hideaki YOSHIFUJI , Jakub Kicinski , Jamal Hadi Salim , Jiri Pirko , Alexei Starovoitov , Colin Ian King , Daniel Borkmann , Eric Dumazet , Eyal Birger , "Gustavo A . R . Silva" , Jakub Sitnicki , John Ogness , Jon Rosen , Kees Cook , Mao Wenan , Marc Kleine-Budde , Martin KaFai Lau , Matthieu Baerts , Andrei Vagin , Dmitry Safonov <0x7f454c46@gmail.com>, "Eric W . Biederman" , Ingo Molnar , John Stultz , Miaohe Lin , Michal Kubecek , Or Cohen , Oleg Nesterov , Peter Zijlstra , Richard Cochran , Stefan Schmidt , Willem de Bruijn , Xie He , Stephen Boyd , Thomas Gleixner , Vladis Dronov , Sebastian Andrzej Siewior , Frederic Weisbecker Cc: Vinicius Costa Gomes , Vedang Patel , Ines Molzahn , Simon Sudler , Andreas Meisinger , Andreas Bucher , Henning Schild , Jan Kiszka , Andreas Zirkler , Ermin Sakic , An Ninh Nguyen , Michael Saenger , Bernd Maehringer , Gisela Greinert , Erez Geva , Erez Geva Subject: [PATCH 3/3] The TC ETF Qdisc pass the hardware timestamp to the interface driver. Date: Wed, 9 Dec 2020 15:37:06 +0100 Message-Id: <20201209143707.13503-4-erez.geva.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201209143707.13503-1-erez.geva.ext@siemens.com> References: <20201209143707.13503-1-erez.geva.ext@siemens.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The ETF pass the TX sending hardware timestamp to the network interface driver. - Add new flag to the ETF Qdisc setting that mandate the use of the hardware timestamp in a socket's buffer. - The ETF Qdisc pass the TX sending hardware timestamp to the network interface driver. Signed-off-by: Erez Geva --- include/uapi/linux/pkt_sched.h | 1 + net/sched/sch_etf.c | 59 +++++++++++++++++++++++++++------- 2 files changed, 49 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h index 9e7c2c607845..51e2b57bfa81 100644 --- a/include/uapi/linux/pkt_sched.h +++ b/include/uapi/linux/pkt_sched.h @@ -1056,6 +1056,7 @@ struct tc_etf_qopt { #define TC_ETF_DEADLINE_MODE_ON _BITUL(0) #define TC_ETF_OFFLOAD_ON _BITUL(1) #define TC_ETF_SKIP_SOCK_CHECK _BITUL(2) +#define TC_ETF_USE_HW_TIMESTAMP _BITUL(3) }; enum { diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c index c48f91075b5c..67eace3e180f 100644 --- a/net/sched/sch_etf.c +++ b/net/sched/sch_etf.c @@ -23,11 +23,13 @@ #define DEADLINE_MODE_IS_ON(x) ((x)->flags & TC_ETF_DEADLINE_MODE_ON) #define OFFLOAD_IS_ON(x) ((x)->flags & TC_ETF_OFFLOAD_ON) #define SKIP_SOCK_CHECK_IS_SET(x) ((x)->flags & TC_ETF_SKIP_SOCK_CHECK) +#define USE_HW_TIMESTAMP(x) ((x)->flags & TC_ETF_USE_HW_TIMESTAMP) struct etf_sched_data { bool offload; bool deadline_mode; bool skip_sock_check; + bool use_hw_timestamp; int clockid; int queue; s32 delta; /* in ns */ @@ -75,7 +77,7 @@ static inline int validate_input_params(struct tc_etf_qopt *qopt, static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb) { struct etf_sched_data *q = qdisc_priv(sch); - ktime_t txtime = nskb->tstamp; + ktime_t hwtxtime, txtime = nskb->tstamp; struct sock *sk = nskb->sk; ktime_t now; @@ -88,6 +90,9 @@ static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb) if (!sock_flag(sk, SOCK_TXTIME)) return false; + if (!q->use_hw_timestamp != !sock_flag(sk, SOCK_HW_TXTIME)) + return false; + /* We don't perform crosstimestamping. * Drop if packet's clockid differs from qdisc's. */ @@ -99,7 +104,11 @@ static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb) skip: now = q->get_time(); - if (ktime_before(txtime, now) || ktime_before(txtime, q->last)) + if (q->use_hw_timestamp) + hwtxtime = skb_hwtstamps(nskb)->hwtstamp; + else + hwtxtime = txtime; + if (ktime_before(txtime, now) || ktime_before(hwtxtime, q->last)) return false; return true; @@ -173,16 +182,33 @@ static int etf_enqueue_timesortedlist(struct sk_buff *nskb, struct Qdisc *sch, return qdisc_drop(nskb, sch, to_free); } - while (*p) { - struct sk_buff *skb; + if (q->use_hw_timestamp) { + ktime_t hwtxtime = skb_hwtstamps(nskb)->hwtstamp; + + while (*p) { + struct sk_buff *skb; - parent = *p; - skb = rb_to_skb(parent); - if (ktime_compare(txtime, skb->tstamp) >= 0) { - p = &parent->rb_right; - leftmost = false; - } else { - p = &parent->rb_left; + parent = *p; + skb = rb_to_skb(parent); + if (ktime_compare(hwtxtime, skb_hwtstamps(skb)->hwtstamp) >= 0) { + p = &parent->rb_right; + leftmost = false; + } else { + p = &parent->rb_left; + } + } + } else { + while (*p) { + struct sk_buff *skb; + + parent = *p; + skb = rb_to_skb(parent); + if (ktime_compare(txtime, skb->tstamp) >= 0) { + p = &parent->rb_right; + leftmost = false; + } else { + p = &parent->rb_left; + } } } rb_link_node(&nskb->rbnode, parent, p); @@ -245,6 +271,10 @@ static void timesortedlist_remove(struct Qdisc *sch, struct sk_buff *skb) qdisc_bstats_update(sch, skb); + /* Pass hardware time to driver and to last */ + if (q->use_hw_timestamp) + skb->tstamp = skb_hwtstamps(skb)->hwtstamp; + q->last = skb->tstamp; sch->q.qlen--; @@ -393,6 +423,10 @@ static int etf_init(struct Qdisc *sch, struct nlattr *opt, q->offload = OFFLOAD_IS_ON(qopt); q->deadline_mode = DEADLINE_MODE_IS_ON(qopt); q->skip_sock_check = SKIP_SOCK_CHECK_IS_SET(qopt); + q->use_hw_timestamp = USE_HW_TIMESTAMP(qopt); + /* deadline mode can not coexist with using hardware time */ + if (q->use_hw_timestamp && q->deadline_mode) + return -EOPNOTSUPP; switch (q->clockid) { case CLOCK_REALTIME: @@ -484,6 +518,9 @@ static int etf_dump(struct Qdisc *sch, struct sk_buff *skb) if (q->skip_sock_check) opt.flags |= TC_ETF_SKIP_SOCK_CHECK; + if (q->use_hw_timestamp) + opt.flags |= TC_ETF_USE_HW_TIMESTAMP; + if (nla_put(skb, TCA_ETF_PARMS, sizeof(opt), &opt)) goto nla_put_failure;