From patchwork Thu Jan 27 00:36:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725955 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66C72C433EF for ; Thu, 27 Jan 2022 00:36:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233937AbiA0Agk (ORCPT ); Wed, 26 Jan 2022 19:36:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233908AbiA0Agk (ORCPT ); Wed, 26 Jan 2022 19:36:40 -0500 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5DA4C06173B; Wed, 26 Jan 2022 16:36:39 -0800 (PST) Received: by mail-ed1-x532.google.com with SMTP id n10so1444550edv.2; Wed, 26 Jan 2022 16:36:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BZekbOX/mBkxJ3VLK+5yn7uRMRqRi36JXBTZy8Qw+hU=; b=aRARrOuufabe73tJxz7WsJYCFwSCAb0ziKrgi23PM7QAo7XrZCEecBr3jEc1RB4fkN BL/twvydFqXcFIhS23ShhtTuMDmKGL2oei08Ld5C/2r/uib/H3/OSVzZhBm1/u8te4L+ VTPLM6tHwfdDvNVlEWNOKK9i9J8fo+LRDXSZqqySUPoHq8PLipA4F1miP4AP6SlLUmIU GgNnXKdXXGiRAN734tNuO5tXiqcglbDZbICEsfZCFjF8yhfsxrtjvOuwJslsO+6vk3TB iUh4jwO9iLpgLVD7JtfV2vy2UgFQkx3JY9wHyeNCflJozVCFR9eHATje8JBpsCKP1Bzt DpuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BZekbOX/mBkxJ3VLK+5yn7uRMRqRi36JXBTZy8Qw+hU=; b=znz6VmydKrZShzBIvD1rZ+w/e71Uk9NJkl4obWTQPlWh0JOOfjCgwyAE9EzAaafgsW JjU2EQjY/wczYmBnIFORy/SyTh/8NOmD0fkHnAw0GcbbwAtCqM2JsplLiSXNTLgZ0Hp6 NvN7gFmcF84yoS0gF9owzQiPQBRXDC+JQwy/uyxsKB7hxX6Ba9Ae9UAj1sNUy6nSQ9TQ 1j9kcnNAkCzfs6TR9KKaaj39uHAHlUc8aMx9/psvlS9nYAUTQHSisXzfKV6kYUudgkNp F5QaJEk7WFPpw6/33Yyh0xKZ2ZkwhpBfnATImoK9RNqUJc5IPJzcPHIyf+JgPJH76vQp GH1A== X-Gm-Message-State: AOAM531QZVK+hqNT8jPXWkxdRb36AjPU2PEtDdvPb4R12PRZfj3bcQJC EXzmPXglCbWONFWOcxi84DiasnX+kQA= X-Google-Smtp-Source: ABdhPJz2DXu1wNwL31EPFWgvtRsZOMbsQKZRPkeiVPVq9FFjHYRhTDKCaan0+jb9U6gUdfOsthDKgg== X-Received: by 2002:a50:e616:: with SMTP id y22mr1353944edm.277.1643243798103; Wed, 26 Jan 2022 16:36:38 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:37 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 01/10] ipv6: optimise dst refcounting on skb init Date: Thu, 27 Jan 2022 00:36:22 +0000 Message-Id: <647e136457fb586846933244448652547dd306f9.1643243772.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org __ip6_make_skb() gets a cork->dst ref, hands it over to skb and shortly after puts cork->dst. Save two atomics by stealing it without extra referencing, ip6_cork_release() handles NULL cork->dst. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 2995f8d89e7e..14d607ccfeea 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1807,6 +1807,15 @@ int ip6_append_data(struct sock *sk, } EXPORT_SYMBOL_GPL(ip6_append_data); +static void ip6_cork_steal_dst(struct sk_buff *skb, struct inet_cork_full *cork) +{ + struct dst_entry *dst = cork->base.dst; + + cork->base.dst = NULL; + cork->base.flags &= ~IPCORK_ALLFRAG; + skb_dst_set(skb, dst); +} + static void ip6_cork_release(struct inet_cork_full *cork, struct inet6_cork *v6_cork) { @@ -1889,7 +1898,7 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, skb->tstamp = cork->base.transmit_time; - skb_dst_set(skb, dst_clone(&rt->dst)); + ip6_cork_steal_dst(skb, cork); IP6_UPD_PO_STATS(net, rt->rt6i_idev, IPSTATS_MIB_OUT, skb->len); if (proto == IPPROTO_ICMPV6) { struct inet6_dev *idev = ip6_dst_idev(skb_dst(skb)); From patchwork Thu Jan 27 00:36:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725956 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56CC8C433EF for ; Thu, 27 Jan 2022 00:36:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233989AbiA0Agm (ORCPT ); Wed, 26 Jan 2022 19:36:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233952AbiA0Agl (ORCPT ); Wed, 26 Jan 2022 19:36:41 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE137C06161C; Wed, 26 Jan 2022 16:36:40 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id p12so1391900edq.9; Wed, 26 Jan 2022 16:36:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UxDNgTAE0cSB8ZQYqqnLx7ARkRkGdeZUWmVAeW7wE8w=; b=UxNN2e4QMjiLRB3KjFk+d0964K5zS0bXfNJq5W2+9QhwH7rL8wHwommwe6im9aR0AW d+tb1OD2FL48Ue8s8kgOEq0lGC1BKcGReMkE2NUl0EP2iHQmm6n+7cgzkchvY60mB7TX CcGrk9IDvDPw/nKbBiEb3GmJF3FDr+PdQU0p2ogqN8nq3drU/dhrN3dsUG9+7+ysLRHc 3JNN8rFwme1Gw4O85Jv/U5yEB5JsVa0NYWM/2VBvI4cLSnYmdTaWBdQAt42v0OBEw8Xr qhnlmOZUxIgwmp92EIscfExtu1NPReSGR1k0nAUlRtueAL6fA3tfQj/8TBysIjZO/7kO ostg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UxDNgTAE0cSB8ZQYqqnLx7ARkRkGdeZUWmVAeW7wE8w=; b=nfNy/UyAvrGe91dPOAcU9R7ZLpwyH96kL4W/B8UfoxVGo0BjEEcIcOUG03K4GVxjKK crQ3dYh5VwBxg6QzgE7m9j0Yo/VM1XD83b7hFEKV8s6KXvQ/te2jQNrf7yW3QtkIqJaZ Wek214noTo3xBmoocz8vxO3Gj6g5x/J13J+a16GMGBYyhXsdREV+a1/ROYmUljcVh6bl nbQlFtQUGc4/gxmmacJIP3oW+f2Ze/hGt0mZlZrd6su/CT+Lwp69ALOnKXAXXBDWVeK4 4hhoTXlqd39GWoZUtxv/yhuBjmGceFE6TQXQaJa3DOsYYbc5ElUrFijHJVaoJ5y3DXDo ct7g== X-Gm-Message-State: AOAM531wtpUjBivDqd4IbxPRQL86w12wXZPRRywfz74LoLDBOUTUA94z thTGyY+wqvXe+DbQjz3CofX2szqcqU8= X-Google-Smtp-Source: ABdhPJwshQN22vMvwSvlXUoR6HtvGLQALyZ8baZxG/wowQuClI85f3xbh7x0WKqRxzGhByAZEhZxjA== X-Received: by 2002:a05:6402:2898:: with SMTP id eg24mr1456329edb.142.1643243799013; Wed, 26 Jan 2022 16:36:39 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:38 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 02/10] udp6: shuffle up->pending AF_INET bits Date: Thu, 27 Jan 2022 00:36:23 +0000 Message-Id: <54bfbd199f8e371333082a123de432211248eaae.1643243772.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Corked AF_INET for ipv6 socket doesn't appear to be the hottest case, so move it out of the common path under up->pending check to remove overhead. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 528b81ef19c9..e221a6957b1f 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1363,9 +1363,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } } - if (up->pending == AF_INET) - return udp_sendmsg(sk, msg, len); - /* Rough check on arithmetic overflow, better check is made in ip6_append_data(). */ @@ -1374,6 +1371,8 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) getfrag = is_udplite ? udplite_getfrag : ip_generic_getfrag; if (up->pending) { + if (up->pending == AF_INET) + return udp_sendmsg(sk, msg, len); /* * There are pending frames. * The socket lock must be held while it's corked. From patchwork Thu Jan 27 00:36:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725957 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6456AC433F5 for ; Thu, 27 Jan 2022 00:36:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234030AbiA0Agp (ORCPT ); Wed, 26 Jan 2022 19:36:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233973AbiA0Agl (ORCPT ); Wed, 26 Jan 2022 19:36:41 -0500 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B9CBC06161C; Wed, 26 Jan 2022 16:36:41 -0800 (PST) Received: by mail-ej1-x62a.google.com with SMTP id s5so2216336ejx.2; Wed, 26 Jan 2022 16:36:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PIyy1S882MJEdPQ345mTMhYul/4iG3UJWg4AQv8ZfsM=; b=Vs+eFN3GvnoXPrSn/v5w2h17QddAImMR3zY6a+2yAQ6VLg7WZnc71MOtbzAKD0xwvK QxVgP+VRshzGKldX7TwBTXQZ1AqIE3T50lfxHrR10kenB5njR2do3XRfXgwUN4xWEMIb eNJhbRPy9iGXPPpt2YX7bdecUsgomSGwbLDhcM64VkHcjFI18aQy7dOIjC8ymwJB17os BvX+wzwgl6WyVNEA+wD+QSCUsrPT93grzeXhAj3IY78YyOB022/d326ELOuPPvNc64Jh aVNf+Weu6zv3sCP/hJ+otFQ7Fwyi1d+1HixSrfRu5quGR/k8OVUEx6Govl1Wx3kw04Zf gMmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PIyy1S882MJEdPQ345mTMhYul/4iG3UJWg4AQv8ZfsM=; b=XhnAkxhQcc9BWEq0hBS4s1gh+s3J3XFB1EU6ub5TVFffPBdOBaeUtZh1EWqKLITqj4 jLZCZzdrWzODNfNTQ3aYO0aHpXQJ0ulUsrrA0rp1wn9SpnYjQCQ0aMdzaD1M8roqv1Oa z1P/1SlFgEnNPp7M4KNYPho3RvhoydLE6ZC8c+9yL2sJT1tyDfLTUNTyvkdEXduEBMot ztxwqVHiDsMvPwS3fH3xdoPybqyMQXb08nNSn0Es7vh7xfr/LK1AdexSDdikP599vOYT PmylskMSh1S0tSK6zVMdgGyi/cYCrvlydLsbVre87w6i0+4EWmxnIauyJoMRAVEBJuPo e7Sw== X-Gm-Message-State: AOAM530sXVw63ViA8xgPIIWtmj59R7AxgpJc0WDSXRV1+nfj89XCVHLD r6W56JRQlj8sLmPFQw4dMnlbPmAN3NM= X-Google-Smtp-Source: ABdhPJwT7dr42MSzLWWRj+EZ2vzWIe6zAxtvQU0DfaOJvqiWn+Fy6Z8OtOl2cHPSRlwi0dMUnC8aHg== X-Received: by 2002:a17:907:2d94:: with SMTP id gt20mr1085743ejc.118.1643243800011; Wed, 26 Jan 2022 16:36:40 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:39 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 03/10] ipv6: remove daddr temp buffer in __ip6_make_skb Date: Thu, 27 Jan 2022 00:36:24 +0000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org ipv6_push_nfrag_opts() doesn't change passed daddr, and so __ip6_make_skb() doesn't actually need to keep an on-stack copy of fl6->daddr. Set initially final_dst to fl6->daddr, ipv6_push_nfrag_opts() will override it if needed, and get rid of extra copies. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 14d607ccfeea..4acd577d5ec5 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1843,7 +1843,7 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, { struct sk_buff *skb, *tmp_skb; struct sk_buff **tail_skb; - struct in6_addr final_dst_buf, *final_dst = &final_dst_buf; + struct in6_addr *final_dst; struct ipv6_pinfo *np = inet6_sk(sk); struct net *net = sock_net(sk); struct ipv6hdr *hdr; @@ -1873,9 +1873,9 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, /* Allow local fragmentation. */ skb->ignore_df = ip6_sk_ignore_df(sk); - - *final_dst = fl6->daddr; __skb_pull(skb, skb_network_header_len(skb)); + + final_dst = &fl6->daddr; if (opt && opt->opt_flen) ipv6_push_frag_opts(skb, opt, &proto); if (opt && opt->opt_nflen) @@ -1895,7 +1895,6 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, skb->priority = sk->sk_priority; skb->mark = cork->base.mark; - skb->tstamp = cork->base.transmit_time; ip6_cork_steal_dst(skb, cork); From patchwork Thu Jan 27 00:36:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725958 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FB3AC433FE for ; Thu, 27 Jan 2022 00:36:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233952AbiA0Agt (ORCPT ); Wed, 26 Jan 2022 19:36:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234004AbiA0Agn (ORCPT ); Wed, 26 Jan 2022 19:36:43 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 848DDC061749; Wed, 26 Jan 2022 16:36:42 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id p12so1391968edq.9; Wed, 26 Jan 2022 16:36:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Owvd8F/4d0c7x5po9puWXUYvkgw1reQHYkIouzUNwg8=; b=MkuJbGgjL1dT1EY2gDe0pRvIOCi2WqXM1ttb2U8n8dkpQbkjk37m23aqjOmyCy0DuH MwmVIClLw6NU38AOLggaW/wiGjwr3QVKrTXNx00NjO5F0596PsLq3fDms6mxXp219ltd WIcWB2G7Bxuf5AKoOj7JSe6TBi6SWFVH3idkjWnyR64a5FpHVc6AVMqdVsYpGWi7zxIc 8pRWD8ea8fxb7eVZ0cqcTU2IWC84qzhJ1Kd5LJ0VIH1NcuVNfV5epq9lLbOmiREaLq+5 KGCuRiGRTaN6oUpYFWYS3RbCcjLQVUJ0Zazh9NvQ7LwOE95PzniPddLPWw4Szl4U0QY9 Mwpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Owvd8F/4d0c7x5po9puWXUYvkgw1reQHYkIouzUNwg8=; b=i67QmQh8vJapyLBbxky3K2vtVs3c7g1KZf59Si5VDjQvnVqKBycyXhxSf41bQT5zxG pFJLSMkrKIUccahMtQ30bfu1nyWgU3HN4qTbnlOdXj+USHUcqO6E1uxAcSxGPbElUJq9 OtCzFPQ+LAPemTyd0t0w3+x9jjY8mD1QaLNhGLpLTYv3zxjE6t2ng2E1TVMqw9Ey2t/r uo2QQYybDhS6xbCMOcUgplIKb8rQA6BZTLYAj9L3XNndW4RrNV2YVrAEdTniF8r1OYhh +MUt7VXLQvgtOoZhyH2KlHqB5Pq5WjHPGM6UC5lO9knkb7FGYtcjt923RgqVpHj+AtHe le2Q== X-Gm-Message-State: AOAM533s+H3PAOBzG4DSmdw65K+JbhUIC2l/H92Rrj2tR12++lEp/2IY U9YFmcEWTE3XHZsQ0Y3dm8HoP0b43f4= X-Google-Smtp-Source: ABdhPJzxbKWXnssnXJN5sgEHoB8hnWwsH5a0J/70J+3ajgY7rVorb7h2O8g2cOzoPqQm5DS2r419PQ== X-Received: by 2002:a05:6402:27c8:: with SMTP id c8mr1362036ede.87.1643243800938; Wed, 26 Jan 2022 16:36:40 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:40 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 04/10] ipv6: clean up cork setup/release Date: Thu, 27 Jan 2022 00:36:25 +0000 Message-Id: <7c774539bb6c4d55f62026328c22fc32ab5562da.1643243773.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Clean up ip6_setup_cork() and ip6_cork_release() adding a local variable for v6_cork->opt. It's a preparation patch for further changes. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 44 +++++++++++++++++++++---------------------- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 4acd577d5ec5..88349e49717a 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1354,7 +1354,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, { struct ipv6_pinfo *np = inet6_sk(sk); unsigned int mtu; - struct ipv6_txoptions *opt = ipc6->opt; + struct ipv6_txoptions *nopt, *opt = ipc6->opt; /* * setup for corking @@ -1363,32 +1363,28 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, if (WARN_ON(v6_cork->opt)) return -EINVAL; - v6_cork->opt = kzalloc(sizeof(*opt), sk->sk_allocation); - if (unlikely(!v6_cork->opt)) + nopt = v6_cork->opt = kzalloc(sizeof(*opt), sk->sk_allocation); + if (unlikely(!nopt)) return -ENOBUFS; - v6_cork->opt->tot_len = sizeof(*opt); - v6_cork->opt->opt_flen = opt->opt_flen; - v6_cork->opt->opt_nflen = opt->opt_nflen; + nopt->tot_len = sizeof(*opt); + nopt->opt_flen = opt->opt_flen; + nopt->opt_nflen = opt->opt_nflen; - v6_cork->opt->dst0opt = ip6_opt_dup(opt->dst0opt, - sk->sk_allocation); - if (opt->dst0opt && !v6_cork->opt->dst0opt) + nopt->dst0opt = ip6_opt_dup(opt->dst0opt, sk->sk_allocation); + if (opt->dst0opt && !nopt->dst0opt) return -ENOBUFS; - v6_cork->opt->dst1opt = ip6_opt_dup(opt->dst1opt, - sk->sk_allocation); - if (opt->dst1opt && !v6_cork->opt->dst1opt) + nopt->dst1opt = ip6_opt_dup(opt->dst1opt, sk->sk_allocation); + if (opt->dst1opt && !nopt->dst1opt) return -ENOBUFS; - v6_cork->opt->hopopt = ip6_opt_dup(opt->hopopt, - sk->sk_allocation); - if (opt->hopopt && !v6_cork->opt->hopopt) + nopt->hopopt = ip6_opt_dup(opt->hopopt, sk->sk_allocation); + if (opt->hopopt && !nopt->hopopt) return -ENOBUFS; - v6_cork->opt->srcrt = ip6_rthdr_dup(opt->srcrt, - sk->sk_allocation); - if (opt->srcrt && !v6_cork->opt->srcrt) + nopt->srcrt = ip6_rthdr_dup(opt->srcrt, sk->sk_allocation); + if (opt->srcrt && !nopt->srcrt) return -ENOBUFS; /* need source address above miyazawa*/ @@ -1820,11 +1816,13 @@ static void ip6_cork_release(struct inet_cork_full *cork, struct inet6_cork *v6_cork) { if (v6_cork->opt) { - kfree(v6_cork->opt->dst0opt); - kfree(v6_cork->opt->dst1opt); - kfree(v6_cork->opt->hopopt); - kfree(v6_cork->opt->srcrt); - kfree(v6_cork->opt); + struct ipv6_txoptions *opt = v6_cork->opt; + + kfree(opt->dst0opt); + kfree(opt->dst1opt); + kfree(opt->hopopt); + kfree(opt->srcrt); + kfree(opt); v6_cork->opt = NULL; } From patchwork Thu Jan 27 00:36:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725961 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50C92C433FE for ; Thu, 27 Jan 2022 00:36:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234141AbiA0Agu (ORCPT ); Wed, 26 Jan 2022 19:36:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234017AbiA0Ago (ORCPT ); Wed, 26 Jan 2022 19:36:44 -0500 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4188C06161C; Wed, 26 Jan 2022 16:36:43 -0800 (PST) Received: by mail-ej1-x62f.google.com with SMTP id me13so2041868ejb.12; Wed, 26 Jan 2022 16:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YcffHEbuuMh1GAJfl6EBV2ha6pRMQcnaQyLatKJI1BU=; b=NJewKiI1hoOJlwu68lbQdEmd/sffYw/XmfEjJ4VdhGOPyGDaFqBAWBu9wG4o7K47KP G/Wse3mP0M2NMYYBvQdwofMOkKKjGvPXWzekT6KVZ+JaRIKTcBbVBXH5/VvUWsjYr1DL qgt8YulxKlwjbsryMYckZVLV1U0fz66hr+NneHbt+iizcrIsdPzuPrXgPKxaTk5V7CLI +b4ge5ClmMpqGKLD6sX1DbgJfZoGZTPK83H1yKjrwu1RtEb1tY0IlhqGWwF0RfpagP8d TygiZ7jlGxFww0jy8tK2n0oA45KfczLOMHIX+JD7U54BpZj/EgWhcDTVZKKxY94Kjn8T 6u8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YcffHEbuuMh1GAJfl6EBV2ha6pRMQcnaQyLatKJI1BU=; b=U1WMYrt9270U4lq+tMnv/EiiG+RVSlU7BNtAlGjCDtFHcz8QoACbwrE0CxIY6Wjijb HbCKs9bHVLLf9IbdnoiCf/NW52H7Edv7hRPrsWeJ5bISADr+F4EhYk/GUjvRLFEhyrPu jSI3xJMvieP9p2XGlBMCp/M8jEcSyuoi4nnwQUy+D/94qZjqNKzGnYnUlPJ3wAKii6pB PHX1H2ZMbM19/6Cc8w3RIsIEN6bHn+JozEHeK6TLEYWLlE1B8sZpLqmVftPc5a48NmR6 eJdCGcKXgCfFFkgubEUGmETERKBs3OhR9bcyns6Uby0xG1xTbp+PnvGSgxR5yv6Uvb1F E8fw== X-Gm-Message-State: AOAM530B2XArin2CeizoxkpLKjN37BhRzqmx8DO5bCmmmzXxRr6vM9Gm q1iNc5iaAgCGBD4hWzPQ+GQRvLvSSI4= X-Google-Smtp-Source: ABdhPJx9TKtR0SROmZRu0f30+P9TVzjqvb48KlZozpjt5DoHtYKk+aw+FII5MXHsrcbnsHbkCDEGTw== X-Received: by 2002:a17:906:1b0b:: with SMTP id o11mr1054429ejg.109.1643243801828; Wed, 26 Jan 2022 16:36:41 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:41 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 05/10] ipv6: don't zero inet_cork_full::fl after use Date: Thu, 27 Jan 2022 00:36:26 +0000 Message-Id: <120aefd3d8bc4392f98edec3148779db39ef1454.1643243773.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org It doesn't appear there is any reason for ip6_cork_release() to zero cork->fl, it'll be fully filled on next initialisation. This 88 bytes memset accounts to 0.3-0.5% of total CPU cycles. It's also needed in following patches and allows to remove an extar flow copy in udp_v6_push_pending_frames(). Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 1 - net/ipv6/udp.c | 10 ++-------- 2 files changed, 2 insertions(+), 9 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 88349e49717a..b8fdda9ac797 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1831,7 +1831,6 @@ static void ip6_cork_release(struct inet_cork_full *cork, cork->base.dst = NULL; cork->base.flags &= ~IPCORK_ALLFRAG; } - memset(&cork->fl, 0, sizeof(cork->fl)); } struct sk_buff *__ip6_make_skb(struct sock *sk, diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index e221a6957b1f..3af1eea739a8 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1266,23 +1266,17 @@ static int udp_v6_push_pending_frames(struct sock *sk) { struct sk_buff *skb; struct udp_sock *up = udp_sk(sk); - struct flowi6 fl6; int err = 0; if (up->pending == AF_INET) return udp_push_pending_frames(sk); - /* ip6_finish_skb will release the cork, so make a copy of - * fl6 here. - */ - fl6 = inet_sk(sk)->cork.fl.u.ip6; - skb = ip6_finish_skb(sk); if (!skb) goto out; - err = udp_v6_send_skb(skb, &fl6, &inet_sk(sk)->cork.base); - + err = udp_v6_send_skb(skb, &inet_sk(sk)->cork.fl.u.ip6, + &inet_sk(sk)->cork.base); out: up->len = 0; up->pending = 0; From patchwork Thu Jan 27 00:36:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725959 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1CC8C4332F for ; Thu, 27 Jan 2022 00:36:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234105AbiA0Agt (ORCPT ); Wed, 26 Jan 2022 19:36:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234027AbiA0Ago (ORCPT ); Wed, 26 Jan 2022 19:36:44 -0500 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 643E9C06173B; Wed, 26 Jan 2022 16:36:44 -0800 (PST) Received: by mail-ed1-x531.google.com with SMTP id w25so585189edt.7; Wed, 26 Jan 2022 16:36:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WDqDCJ5NL4LuLdFrIrWhPcUdc5NRFY/X6Wj3QPHSD+0=; b=dbwE2UZJaBfWlzco9zRjfOlhlIpzG/xZyaJKOUSu1yt8bXlyguuEM3KNKyantOJUF4 dgllJb9j1ScQTFkVM1YVkge+GlR+ux+OJ9/x3VRAPVCuxLokWthjDuf50zpd+dXo7SQL 7K+P1M4+OrTgtEMQpm5xZgpCv4zMzfYzG+dm8FInh+AHEmMG2CFFPpImilkUoIRl88DI 286GejQIlZEZQG1A7FofDIRjAUQy75vBgy2SCouCY664B9uegMoamGqAnDvmDEkmGHJD cQ47/qzcadSYWhik+yABV3Axd8xA9zklQrlU2qK4fkH3IL6q5rbT/zzuqkeLTgLJ0cqc zn0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WDqDCJ5NL4LuLdFrIrWhPcUdc5NRFY/X6Wj3QPHSD+0=; b=6NnnH/p0U65a08gyneraubt9WrM91YDjcfffRXgy4aMtc46xXCC4ojU9yKVLhZ/KDn buON435Ux1sUJTcMgZRD9I4jCpf0gpLTiWe32SmXqctp4kIK5h7lvK3alIMhGEFsMR7G 9bYwzEn5tn58PjHGgPFq5fH068UmghdTBHPbIWqeSo3p6mwWF+64YXGG+QAI6BLl8LIv ov01p91pJp5KEDwymvQ+HP7xFhaT74Km+TfC7qWRNrqnMSl6TWP3tfNgS9gAPqor5pFg IScmMWqT+tZXIHbZ52B2JbMNzWTTILUWl2ZyVpA/IHx5pHYQUKYMvvV/fqf8G0bL2DFG WFYA== X-Gm-Message-State: AOAM531WBE/d3ervkU7Bav1lPtotp8217jCHX9cibUksM2mwOJvOtRuf 7/V7jOjt6kbLvoTa4K+sSY1aUTAB61U= X-Google-Smtp-Source: ABdhPJzgnH6LO5Q9T0RHAKza0klM/MRdhz7JaOnKxE0xu+F/36Z3S3hBnpOCXzUUK+WDQhEZ0hPS/Q== X-Received: by 2002:a05:6402:40cd:: with SMTP id z13mr1384319edb.119.1643243802763; Wed, 26 Jan 2022 16:36:42 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:42 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 06/10] ipv6: pass full cork into __ip6_append_data() Date: Thu, 27 Jan 2022 00:36:27 +0000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Convert a struct inet_cork argument in __ip6_append_data() to struct inet_cork_full. As one struct contains another inet_cork is still can be accessed via ->base field. It's a preparation patch making further changes a bit cleaner. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index b8fdda9ac797..62da09819750 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1424,7 +1424,7 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, static int __ip6_append_data(struct sock *sk, struct flowi6 *fl6, struct sk_buff_head *queue, - struct inet_cork *cork, + struct inet_cork_full *cork_full, struct inet6_cork *v6_cork, struct page_frag *pfrag, int getfrag(void *from, char *to, int offset, @@ -1433,6 +1433,7 @@ static int __ip6_append_data(struct sock *sk, unsigned int flags, struct ipcm6_cookie *ipc6) { struct sk_buff *skb, *skb_prev = NULL; + struct inet_cork *cork = &cork_full->base; unsigned int maxfraglen, fragheaderlen, mtu, orig_mtu, pmtu; struct ubuf_info *uarg = NULL; int exthdrlen = 0; @@ -1797,7 +1798,7 @@ int ip6_append_data(struct sock *sk, transhdrlen = 0; } - return __ip6_append_data(sk, fl6, &sk->sk_write_queue, &inet->cork.base, + return __ip6_append_data(sk, fl6, &sk->sk_write_queue, &inet->cork, &np->cork, sk_page_frag(sk), getfrag, from, length, transhdrlen, flags, ipc6); } @@ -1993,7 +1994,7 @@ struct sk_buff *ip6_make_skb(struct sock *sk, if (ipc6->dontfrag < 0) ipc6->dontfrag = inet6_sk(sk)->dontfrag; - err = __ip6_append_data(sk, fl6, &queue, &cork->base, &v6_cork, + err = __ip6_append_data(sk, fl6, &queue, cork, &v6_cork, ¤t->task_frag, getfrag, from, length + exthdrlen, transhdrlen + exthdrlen, flags, ipc6); From patchwork Thu Jan 27 00:36:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725960 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 575F7C43219 for ; Thu, 27 Jan 2022 00:36:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234129AbiA0Agu (ORCPT ); Wed, 26 Jan 2022 19:36:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234040AbiA0Agp (ORCPT ); Wed, 26 Jan 2022 19:36:45 -0500 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38D8CC061751; Wed, 26 Jan 2022 16:36:45 -0800 (PST) Received: by mail-ej1-x629.google.com with SMTP id a8so2084978ejc.8; Wed, 26 Jan 2022 16:36:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f76VCmw6L1xq+F3n85MyvmtgQ0vjtDYX1QQFC+2rnWw=; b=BHocNtloAkh991Vev1wdeaPq962WdUAWrH0gTeYZa5Qr4GkgzAlj91092gbZ3hfows KW81zysv2dFH8rJoLeR9krplJUAw0Bj/q01FRPIetqDjbHD/NAxAsm7jwJoH6/1eaD72 hAFJReMR99f5ntiNhg20spSk/BsnOxRntaz1bbYcuWi5yWLLZxxYoR3O2FHFZaSFNVcY zxZLJEyb84nNW50KML1Qj5glsrBFsPmtpi0qVKCU+W8OkB5YKY8+t/ZsFBjp2Gar5/0D QcLuXoGPAI9bucHHvZMBcUHHwEPRdyfCYpBZ/RiHnAcRjWSKADI4w48lNVLod9nClZnP T/+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f76VCmw6L1xq+F3n85MyvmtgQ0vjtDYX1QQFC+2rnWw=; b=xe+MpBDmodfcos5vbq3IbBUcQZiU1aRwI37s5sNi9k5DBXftGbI18Bo4+9OEfOcSxU 2pbC7IDpvXcCsaLyCSjGyE6kQVgY0oyXc4Nwmg/jYpIzZys9zLdynRlXMiMVqFqEZGYM uyC3Oowl6k+EBzLKAyNztIaceUiA94SEpOFOsZPhcQEc2hYBJ3cIKrXkZUMxA8HEgrtf uSrgsnst8TaqDpt1ejC5fYicGkwhiW8y1P1eKU3FZb5vEaU+Q1G8zGf+cnKQv4KI5tPW LjcDoWUJzbej92/En4b+rzEtzqombNl/pImqj+brDyweflyCZI1tcvN1HxPrEIltuEg9 ozTA== X-Gm-Message-State: AOAM533KTpcbfuPkaqxISbS6mJQwQzJJ5XauKrTzitwv95dmQvD9YoB3 avkpCMUyEH1AFi6yyrD/uy+JbGjYB8A= X-Google-Smtp-Source: ABdhPJz+kvaEfq525bjPdM95RrJW6W70jSOXDDrflqgVF2/p/oiYVG0gwdaQh75KbJ2Qr69+ttFUqw== X-Received: by 2002:a17:907:7d93:: with SMTP id oz19mr1064241ejc.350.1643243803620; Wed, 26 Jan 2022 16:36:43 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:43 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 07/10] udp6: pass flow in ip6_make_skb together with cork Date: Thu, 27 Jan 2022 00:36:28 +0000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Another preparation patch. inet_cork_full already contains a field for iflow, so we can avoid passing a separate struct iflow6 into __ip6_append_data() and ip6_make_skb(), and use the flow stored in inet_cork_full. Make sure callers set cork->fl, i.e. we init it in ip6_append_data() and before calling ip6_make_skb(). Signed-off-by: Pavel Begunkov --- include/net/ipv6.h | 2 +- net/ipv6/ip6_output.c | 20 +++++++++----------- net/ipv6/udp.c | 4 +++- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 3afcb128e064..5e0b56d66724 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -1020,7 +1020,7 @@ struct sk_buff *ip6_make_skb(struct sock *sk, int getfrag(void *from, char *to, int offset, int len, int odd, struct sk_buff *skb), void *from, int length, int transhdrlen, - struct ipcm6_cookie *ipc6, struct flowi6 *fl6, + struct ipcm6_cookie *ipc6, struct rt6_info *rt, unsigned int flags, struct inet_cork_full *cork); diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 62da09819750..0cc490f2cfbf 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1350,7 +1350,7 @@ static void ip6_append_data_mtu(unsigned int *mtu, static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, struct inet6_cork *v6_cork, struct ipcm6_cookie *ipc6, - struct rt6_info *rt, struct flowi6 *fl6) + struct rt6_info *rt) { struct ipv6_pinfo *np = inet6_sk(sk); unsigned int mtu; @@ -1391,7 +1391,6 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, } dst_hold(&rt->dst); cork->base.dst = &rt->dst; - cork->fl.u.ip6 = *fl6; v6_cork->hop_limit = ipc6->hlimit; v6_cork->tclass = ipc6->tclass; if (rt->dst.flags & DST_XFRM_TUNNEL) @@ -1422,7 +1421,6 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, } static int __ip6_append_data(struct sock *sk, - struct flowi6 *fl6, struct sk_buff_head *queue, struct inet_cork_full *cork_full, struct inet6_cork *v6_cork, @@ -1434,6 +1432,7 @@ static int __ip6_append_data(struct sock *sk, { struct sk_buff *skb, *skb_prev = NULL; struct inet_cork *cork = &cork_full->base; + struct flowi6 *fl6 = &cork_full->fl.u.ip6; unsigned int maxfraglen, fragheaderlen, mtu, orig_mtu, pmtu; struct ubuf_info *uarg = NULL; int exthdrlen = 0; @@ -1786,19 +1785,19 @@ int ip6_append_data(struct sock *sk, * setup for corking */ err = ip6_setup_cork(sk, &inet->cork, &np->cork, - ipc6, rt, fl6); + ipc6, rt); if (err) return err; + inet->cork.fl.u.ip6 = *fl6; exthdrlen = (ipc6->opt ? ipc6->opt->opt_flen : 0); length += exthdrlen; transhdrlen += exthdrlen; } else { - fl6 = &inet->cork.fl.u.ip6; transhdrlen = 0; } - return __ip6_append_data(sk, fl6, &sk->sk_write_queue, &inet->cork, + return __ip6_append_data(sk, &sk->sk_write_queue, &inet->cork, &np->cork, sk_page_frag(sk), getfrag, from, length, transhdrlen, flags, ipc6); } @@ -1967,9 +1966,8 @@ struct sk_buff *ip6_make_skb(struct sock *sk, int getfrag(void *from, char *to, int offset, int len, int odd, struct sk_buff *skb), void *from, int length, int transhdrlen, - struct ipcm6_cookie *ipc6, struct flowi6 *fl6, - struct rt6_info *rt, unsigned int flags, - struct inet_cork_full *cork) + struct ipcm6_cookie *ipc6, struct rt6_info *rt, + unsigned int flags, struct inet_cork_full *cork) { struct inet6_cork v6_cork; struct sk_buff_head queue; @@ -1986,7 +1984,7 @@ struct sk_buff *ip6_make_skb(struct sock *sk, cork->base.opt = NULL; cork->base.dst = NULL; v6_cork.opt = NULL; - err = ip6_setup_cork(sk, cork, &v6_cork, ipc6, rt, fl6); + err = ip6_setup_cork(sk, cork, &v6_cork, ipc6, rt); if (err) { ip6_cork_release(cork, &v6_cork); return ERR_PTR(err); @@ -1994,7 +1992,7 @@ struct sk_buff *ip6_make_skb(struct sock *sk, if (ipc6->dontfrag < 0) ipc6->dontfrag = inet6_sk(sk)->dontfrag; - err = __ip6_append_data(sk, fl6, &queue, cork, &v6_cork, + err = __ip6_append_data(sk, &queue, cork, &v6_cork, ¤t->task_frag, getfrag, from, length + exthdrlen, transhdrlen + exthdrlen, flags, ipc6); diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 3af1eea739a8..44b7ca9bd78e 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1533,9 +1533,11 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) struct inet_cork_full cork; struct sk_buff *skb; + cork.fl.u.ip6 = fl6; + skb = ip6_make_skb(sk, getfrag, msg, ulen, sizeof(struct udphdr), &ipc6, - &fl6, (struct rt6_info *)dst, + (struct rt6_info *)dst, msg->msg_flags, &cork); err = PTR_ERR(skb); if (!IS_ERR_OR_NULL(skb)) From patchwork Thu Jan 27 00:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725962 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0C71C433F5 for ; Thu, 27 Jan 2022 00:36:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234243AbiA0Ag4 (ORCPT ); Wed, 26 Jan 2022 19:36:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234061AbiA0Agr (ORCPT ); Wed, 26 Jan 2022 19:36:47 -0500 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60FDCC061755; Wed, 26 Jan 2022 16:36:46 -0800 (PST) Received: by mail-ej1-x62f.google.com with SMTP id a8so2085043ejc.8; Wed, 26 Jan 2022 16:36:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FrxWZOLdX8kzfbgGNePsf48Jt9sSElrC1nnUoAy2AoA=; b=qRVi1tV3qc4J+oggyap7jLI53x0g5mclcHCmXWQJ6YRhtQ1QuRxZduobIJ2Xs4/Dyg gTvxE0fRxkqGCAVr8nV7Nh6xU6zXiEBoGB8TAvz8X02rSLPRxHOrg2mo838CFf1ztWUj d8WjxBonx9K548UCm3bQvozJ62oKomGKv8CYd5XuoI40FvSn1Jm76lPnXgAT8JQBEkH2 debcEhiq0TflGDh4tnofJb3TQaxj4xHiRHeW73vNbCu2cwu16RNWMAK5hzrQ0VQWr2Dw Vpdw7rFZxS7rEQYDV7bBNn7Ch6qKWGLEGO8qm+w9bNjz8ftW8LKbCno2bmlTsoTxMaUY YJmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FrxWZOLdX8kzfbgGNePsf48Jt9sSElrC1nnUoAy2AoA=; b=q8pq9kZqN2FKYmSCr5e4FFz+t+Izb0oApF5KH/gGuWRuquz+xRFkhZv9kqeKHvJ6dL 19ZQqQl/LOik+w25sRJGBbnB/Dn7BUOnCl4huuxJKr0dZmnYBeKKKy3a8di6XFMaeip0 X9CzzN228s5XrlxBibRyxhc/uCi7+2GBT8Jd0sD8dsZeubZXY3T4/d9xwjkIHLzdHfj6 YLvR0hrkFTMd3MuPxf5nXTmTajLw2DjnspdYajB/Y5yid+tc39ZAIkgx6NvaQI6v28aa 5acha0biabJMYaZVupQfNqjIcH8LQeSQ8YNmCmzmvC2HJEgYNckqJadJBbe8xDI0kLnr JwWQ== X-Gm-Message-State: AOAM5310Hmtxpe4f5x77tjc5Iap8iERbxTVIckDv6DQiwaAhH9FmuCv1 xATixnQWttjC5iG1JdMnfwdWJuxzbDc= X-Google-Smtp-Source: ABdhPJzqHYg2qgHRxJB2Pco/QXkWa/FVGiLU+gGBZihqlCnglWCVfgmZoSjXNAqMnq6Yf/5NYPuL2w== X-Received: by 2002:a17:906:7948:: with SMTP id l8mr1016303ejo.636.1643243804758; Wed, 26 Jan 2022 16:36:44 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:44 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 08/10] udp6: don't make extra copies of iflow Date: Thu, 27 Jan 2022 00:36:29 +0000 Message-Id: <46cf12d281bede877a156d4ac224c21809ee3764.1643243773.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org udpv6_sendmsg() first initialises an on-stack 88B struct flowi6 and then copies it into cork, which is expensive. Avoid the copy in corkless case by initialising on-stack cork->fl directly. The main part is a couple of lines under !corkreq check. The rest converts fl6 variable to be a pointer. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 85 +++++++++++++++++++++++++------------------------- 1 file changed, 42 insertions(+), 43 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 44b7ca9bd78e..cfcf08c3df4d 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1294,7 +1294,8 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) struct ipv6_txoptions *opt = NULL; struct ipv6_txoptions *opt_to_free = NULL; struct ip6_flowlabel *flowlabel = NULL; - struct flowi6 fl6; + struct inet_cork_full cork; + struct flowi6 *fl6 = &cork.fl.u.ip6; struct dst_entry *dst; struct ipcm6_cookie ipc6; int addr_len = msg->msg_namelen; @@ -1384,19 +1385,19 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } ulen += sizeof(struct udphdr); - memset(&fl6, 0, sizeof(fl6)); + memset(fl6, 0, sizeof(*fl6)); if (sin6) { if (sin6->sin6_port == 0) return -EINVAL; - fl6.fl6_dport = sin6->sin6_port; + fl6->fl6_dport = sin6->sin6_port; daddr = &sin6->sin6_addr; if (np->sndflow) { - fl6.flowlabel = sin6->sin6_flowinfo&IPV6_FLOWINFO_MASK; - if (fl6.flowlabel&IPV6_FLOWLABEL_MASK) { - flowlabel = fl6_sock_lookup(sk, fl6.flowlabel); + fl6->flowlabel = sin6->sin6_flowinfo&IPV6_FLOWINFO_MASK; + if (fl6->flowlabel & IPV6_FLOWLABEL_MASK) { + flowlabel = fl6_sock_lookup(sk, fl6->flowlabel); if (IS_ERR(flowlabel)) return -EINVAL; } @@ -1413,24 +1414,24 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (addr_len >= sizeof(struct sockaddr_in6) && sin6->sin6_scope_id && __ipv6_addr_needs_scope_id(__ipv6_addr_type(daddr))) - fl6.flowi6_oif = sin6->sin6_scope_id; + fl6->flowi6_oif = sin6->sin6_scope_id; } else { if (sk->sk_state != TCP_ESTABLISHED) return -EDESTADDRREQ; - fl6.fl6_dport = inet->inet_dport; + fl6->fl6_dport = inet->inet_dport; daddr = &sk->sk_v6_daddr; - fl6.flowlabel = np->flow_label; + fl6->flowlabel = np->flow_label; connected = true; } - if (!fl6.flowi6_oif) - fl6.flowi6_oif = sk->sk_bound_dev_if; + if (!fl6->flowi6_oif) + fl6->flowi6_oif = sk->sk_bound_dev_if; - if (!fl6.flowi6_oif) - fl6.flowi6_oif = np->sticky_pktinfo.ipi6_ifindex; + if (!fl6->flowi6_oif) + fl6->flowi6_oif = np->sticky_pktinfo.ipi6_ifindex; - fl6.flowi6_uid = sk->sk_uid; + fl6->flowi6_uid = sk->sk_uid; if (msg->msg_controllen) { opt = &opt_space; @@ -1440,14 +1441,14 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) err = udp_cmsg_send(sk, msg, &ipc6.gso_size); if (err > 0) - err = ip6_datagram_send_ctl(sock_net(sk), sk, msg, &fl6, + err = ip6_datagram_send_ctl(sock_net(sk), sk, msg, fl6, &ipc6); if (err < 0) { fl6_sock_release(flowlabel); return err; } - if ((fl6.flowlabel&IPV6_FLOWLABEL_MASK) && !flowlabel) { - flowlabel = fl6_sock_lookup(sk, fl6.flowlabel); + if ((fl6->flowlabel&IPV6_FLOWLABEL_MASK) && !flowlabel) { + flowlabel = fl6_sock_lookup(sk, fl6->flowlabel); if (IS_ERR(flowlabel)) return -EINVAL; } @@ -1464,16 +1465,17 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) opt = ipv6_fixup_options(&opt_space, opt); ipc6.opt = opt; - fl6.flowi6_proto = sk->sk_protocol; - fl6.flowi6_mark = ipc6.sockc.mark; - fl6.daddr = *daddr; - if (ipv6_addr_any(&fl6.saddr) && !ipv6_addr_any(&np->saddr)) - fl6.saddr = np->saddr; - fl6.fl6_sport = inet->inet_sport; + fl6->flowi6_proto = sk->sk_protocol; + fl6->flowi6_mark = ipc6.sockc.mark; + fl6->daddr = *daddr; + if (ipv6_addr_any(&fl6->saddr) && !ipv6_addr_any(&np->saddr)) + fl6->saddr = np->saddr; + fl6->fl6_sport = inet->inet_sport; if (cgroup_bpf_enabled(CGROUP_UDP6_SENDMSG) && !connected) { err = BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK(sk, - (struct sockaddr *)sin6, &fl6.saddr); + (struct sockaddr *)sin6, + &fl6->saddr); if (err) goto out_no_dst; if (sin6) { @@ -1489,32 +1491,32 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) err = -EINVAL; goto out_no_dst; } - fl6.fl6_dport = sin6->sin6_port; - fl6.daddr = sin6->sin6_addr; + fl6->fl6_dport = sin6->sin6_port; + fl6->daddr = sin6->sin6_addr; } } - if (ipv6_addr_any(&fl6.daddr)) - fl6.daddr.s6_addr[15] = 0x1; /* :: means loopback (BSD'ism) */ + if (ipv6_addr_any(&fl6->daddr)) + fl6->daddr.s6_addr[15] = 0x1; /* :: means loopback (BSD'ism) */ - final_p = fl6_update_dst(&fl6, opt, &final); + final_p = fl6_update_dst(fl6, opt, &final); if (final_p) connected = false; - if (!fl6.flowi6_oif && ipv6_addr_is_multicast(&fl6.daddr)) { - fl6.flowi6_oif = np->mcast_oif; + if (!fl6->flowi6_oif && ipv6_addr_is_multicast(&fl6->daddr)) { + fl6->flowi6_oif = np->mcast_oif; connected = false; - } else if (!fl6.flowi6_oif) - fl6.flowi6_oif = np->ucast_oif; + } else if (!fl6->flowi6_oif) + fl6->flowi6_oif = np->ucast_oif; - security_sk_classify_flow(sk, flowi6_to_flowi_common(&fl6)); + security_sk_classify_flow(sk, flowi6_to_flowi_common(fl6)); if (ipc6.tclass < 0) ipc6.tclass = np->tclass; - fl6.flowlabel = ip6_make_flowinfo(ipc6.tclass, fl6.flowlabel); + fl6->flowlabel = ip6_make_flowinfo(ipc6.tclass, fl6->flowlabel); - dst = ip6_sk_dst_lookup_flow(sk, &fl6, final_p, connected); + dst = ip6_sk_dst_lookup_flow(sk, fl6, final_p, connected); if (IS_ERR(dst)) { err = PTR_ERR(dst); dst = NULL; @@ -1522,7 +1524,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } if (ipc6.hlimit < 0) - ipc6.hlimit = ip6_sk_dst_hoplimit(np, &fl6, dst); + ipc6.hlimit = ip6_sk_dst_hoplimit(np, fl6, dst); if (msg->msg_flags&MSG_CONFIRM) goto do_confirm; @@ -1530,18 +1532,15 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) /* Lockless fast path for the non-corking case */ if (!corkreq) { - struct inet_cork_full cork; struct sk_buff *skb; - cork.fl.u.ip6 = fl6; - skb = ip6_make_skb(sk, getfrag, msg, ulen, sizeof(struct udphdr), &ipc6, (struct rt6_info *)dst, msg->msg_flags, &cork); err = PTR_ERR(skb); if (!IS_ERR_OR_NULL(skb)) - err = udp_v6_send_skb(skb, &fl6, &cork.base); + err = udp_v6_send_skb(skb, fl6, &cork.base); goto out; } @@ -1563,7 +1562,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) ipc6.dontfrag = np->dontfrag; up->len += ulen; err = ip6_append_data(sk, getfrag, msg, ulen, sizeof(struct udphdr), - &ipc6, &fl6, (struct rt6_info *)dst, + &ipc6, fl6, (struct rt6_info *)dst, corkreq ? msg->msg_flags|MSG_MORE : msg->msg_flags); if (err) udp_v6_flush_pending_frames(sk); @@ -1598,7 +1597,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) do_confirm: if (msg->msg_flags & MSG_PROBE) - dst_confirm_neigh(dst, &fl6.daddr); + dst_confirm_neigh(dst, &fl6->daddr); if (!(msg->msg_flags&MSG_PROBE) || len) goto back_from_confirm; err = 0; From patchwork Thu Jan 27 00:36:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725963 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1A36C4332F for ; Thu, 27 Jan 2022 00:36:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234040AbiA0Ag5 (ORCPT ); Wed, 26 Jan 2022 19:36:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234071AbiA0Ags (ORCPT ); Wed, 26 Jan 2022 19:36:48 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6520C061748; Wed, 26 Jan 2022 16:36:47 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id u24so1377589eds.11; Wed, 26 Jan 2022 16:36:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mi+XPD0Dq+TrYWW1iZbOOEuiNokMVOo4lyf3EzZclVM=; b=gzkcPtjcGKG3ROk4gg/haSeVuFp1ik6ZH/VGo9HdXJO8TvGjTZMUG6a1wdbdBJBTVD WbMe43y/k8nWmPgVxEnvbYPtFprw7vuplrABcP/XJlwS4wHDGYiH3BRg8oNRQOjduhDg GYvHBK2oMbDu5rxuuFZKxnOUdGvwFFXOLT6Hr6752BqiRk8G+e55tWanJM0aOIjrDrmO igxEn0iXpLNvEl98MYgcyoovpmouQ9rp+L8UZbIVRKPbvLD5Kgd4qItPDrtMwQzG4511 bOB5juTa/jkOuaHYQFx0vWDLOOIxk0yY1G2kxvGE0dok8Hlh0DN9bn3AZ9IX30dAQH5T E8Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mi+XPD0Dq+TrYWW1iZbOOEuiNokMVOo4lyf3EzZclVM=; b=uMBTT3eIK/ZPbnVS8/YDTKzXB+OxWi3NPSlv8bE9peTUZIbydeSQmTiY8V3KM76lTG 1zzylw054izE9wjZPLtdY0110e1Rdqct87xyTHFcOhALzvyuuyDvWIv4USFbLFjDbeB4 8QqHMuCbFwXsc2/775Fg4h5v7K/39ZnboZSczz3R8XFh10fA3JM9mpOxudmTdPwYyKTH DRPUoBZs5GLwoNdiZ5nnCNGqnJPeNxfBRA/DERwYbHjekeGz/Q21LudKclWtHutAUgRT Uk2iMGbdhlLvIKy6spob8vLFMVIadUe4kHDXah9gMf64uX5kM2WJrpJoxjgPtYta9H1C G25A== X-Gm-Message-State: AOAM531LaLPyQ3iR1/aJp2AmAfeiMFcEQjMbyHQoHpelTSHCYm2CNPm+ 6wZFiqkNdwb/K9rdDSxxgiYrEAUJUcg= X-Google-Smtp-Source: ABdhPJwiP8z9E9KeZuiw7aip3tEDkd9tV8HYv9hbSCG6N9NvFb9+kDb7Er9ehV2bra0q4zvv0v8rpQ== X-Received: by 2002:a05:6402:348c:: with SMTP id v12mr1415681edc.384.1643243805648; Wed, 26 Jan 2022 16:36:45 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:45 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 09/10] ipv6: optimise dst refcounting on cork init Date: Thu, 27 Jan 2022 00:36:30 +0000 Message-Id: <6c2a202cb1b22936624133d189bf2d95a1ab8eac.1643243773.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org udpv6_sendmsg() doesn't need dst after calling ip6_make_skb(), so instead of taking an additional reference inside ip6_setup_cork() and releasing the initial one afterwards, we can hand over a reference into ip6_make_skb() saving two atomics. The only other user of ip6_setup_cork() is ip6_append_data() and it requires an extra dst_hold(). Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 13 +++++++++---- net/ipv6/udp.c | 3 ++- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 0cc490f2cfbf..0c6c971ce0a5 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1356,6 +1356,11 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, unsigned int mtu; struct ipv6_txoptions *nopt, *opt = ipc6->opt; + /* callers pass dst together with a reference, set it first so + * ip6_cork_release() can put it down even in case of an error. + */ + cork->base.dst = &rt->dst; + /* * setup for corking */ @@ -1389,8 +1394,6 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, /* need source address above miyazawa*/ } - dst_hold(&rt->dst); - cork->base.dst = &rt->dst; v6_cork->hop_limit = ipc6->hlimit; v6_cork->tclass = ipc6->tclass; if (rt->dst.flags & DST_XFRM_TUNNEL) @@ -1784,6 +1787,7 @@ int ip6_append_data(struct sock *sk, /* * setup for corking */ + dst_hold(&rt->dst); err = ip6_setup_cork(sk, &inet->cork, &np->cork, ipc6, rt); if (err) @@ -1974,15 +1978,16 @@ struct sk_buff *ip6_make_skb(struct sock *sk, int exthdrlen = (ipc6->opt ? ipc6->opt->opt_flen : 0); int err; - if (flags & MSG_PROBE) + if (flags & MSG_PROBE) { + dst_release(&rt->dst); return NULL; + } __skb_queue_head_init(&queue); cork->base.flags = 0; cork->base.addr = 0; cork->base.opt = NULL; - cork->base.dst = NULL; v6_cork.opt = NULL; err = ip6_setup_cork(sk, cork, &v6_cork, ipc6, rt); if (err) { diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index cfcf08c3df4d..c6872596b408 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1541,7 +1541,8 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) err = PTR_ERR(skb); if (!IS_ERR_OR_NULL(skb)) err = udp_v6_send_skb(skb, fl6, &cork.base); - goto out; + /* ip6_make_skb steals dst reference */ + goto out_no_dst; } lock_sock(sk); From patchwork Thu Jan 27 00:36:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12725964 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 544CAC433EF for ; Thu, 27 Jan 2022 00:37:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234029AbiA0Ag7 (ORCPT ); Wed, 26 Jan 2022 19:36:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234079AbiA0Ags (ORCPT ); Wed, 26 Jan 2022 19:36:48 -0500 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55620C06175B; Wed, 26 Jan 2022 16:36:48 -0800 (PST) Received: by mail-ej1-x633.google.com with SMTP id k25so2186986ejp.5; Wed, 26 Jan 2022 16:36:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qenSIoAXZ6o9VOjrqRc2EOQyMOxrV68LqnhSggaVs6I=; b=Czo+ENsmCpOPbGSPPdD9nN2KoWZnqE/IAEJ7gbZVSsPS6NR/v3zqkGBqNe/V9WVlVE RfhQF5Tz/kzZpADsL4/V8tOAnHeFscEsasnOJLnP6C5Xxkg8KkcFjYrqIsXzx/7O2kee uV+Jx5SNcqw77Utc0qio9q8HsG8hJVmKxXsYXacZ1lMfhhbYTlbDgG3O9dw2p4EWkERw Zkj79vBK1iV/VCpWXojsHFBaWXnWZybQ/XBRxwBQUGWbL0Au/n05Ars+BdAU0dFQmwGv Nk2PYphgsZlxUUpxDfNBSyYQfOd3CX73hw37DLIIOcW+W72J+zy3MYYo85sNOZN6EUC0 d6MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qenSIoAXZ6o9VOjrqRc2EOQyMOxrV68LqnhSggaVs6I=; b=qVBeqhJs758JjEoQjputXIBSla9U2UnKHeN4Snx/bsyacSTjCnJGjVy79S2aH2BBp6 E6laoUta2GHl/FxvAyCKCvD8eQF3u8+scDKdiAVHcs0VYdXB8UbWP/aSGxtbyeHfIGdO quQzeMt0CKNSTh4GqfqGvWwWQBu40uprBHbm7+qWrsMHWPEJBnjk7+SZQLu8GrD6JAlS QEMu3hO0cQcP+5QPpVX8ButBlj0PqRogbc9PXczzQNHPQdt8o5dfWcfxVKyjFQ4W1YOc o8l5QHb4j90E9MdgijKE+M7P7JCBidU3AxLWOyklTkAY/ac4IYMTda+bbSoZMTKPs02V wAxw== X-Gm-Message-State: AOAM5304KLHI/jmR+6BF4o1jVirMQUuuR+noFg/41KwiA2QZk09UdP1i jj0Pm1pELqXoew/4q5a7M5KdfICpDwk= X-Google-Smtp-Source: ABdhPJyB36Kzr0C+6YwvLEkXBnQfdKBZ0Ds1xbfLzopBJQQw5iRj0AVgVxxERXvaRauDU2VTDIx8fQ== X-Received: by 2002:a17:907:6089:: with SMTP id ht9mr998920ejc.612.1643243806756; Wed, 26 Jan 2022 16:36:46 -0800 (PST) Received: from 127.0.0.1localhost ([85.255.234.222]) by smtp.gmail.com with ESMTPSA id op27sm8039235ejb.103.2022.01.26.16.36.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jan 2022 16:36:46 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Willem de Bruijn , linux-kernel@vger.kernel.org, Paolo Abeni , Pavel Begunkov Subject: [PATCH net-next v2 10/10] ipv6: partially inline ipv6_fixup_options Date: Thu, 27 Jan 2022 00:36:31 +0000 Message-Id: <5c6bda8c6f78228fd58586a4160edcc374011a26.1643243773.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Inline a part of ipv6_fixup_options() to avoid extra overhead on function call if opt is NULL. Signed-off-by: Pavel Begunkov --- include/net/ipv6.h | 12 ++++++++++-- net/ipv6/exthdrs.c | 8 ++++---- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 5e0b56d66724..082f30256f59 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -437,8 +437,16 @@ struct ipv6_txoptions *ipv6_renew_options(struct sock *sk, struct ipv6_txoptions *opt, int newtype, struct ipv6_opt_hdr *newopt); -struct ipv6_txoptions *ipv6_fixup_options(struct ipv6_txoptions *opt_space, - struct ipv6_txoptions *opt); +struct ipv6_txoptions *__ipv6_fixup_options(struct ipv6_txoptions *opt_space, + struct ipv6_txoptions *opt); + +static inline struct ipv6_txoptions * +ipv6_fixup_options(struct ipv6_txoptions *opt_space, struct ipv6_txoptions *opt) +{ + if (!opt) + return NULL; + return __ipv6_fixup_options(opt_space, opt); +} bool ipv6_opt_accepted(const struct sock *sk, const struct sk_buff *skb, const struct inet6_skb_parm *opt); diff --git a/net/ipv6/exthdrs.c b/net/ipv6/exthdrs.c index 77e34aec7e82..658d5eabaf7e 100644 --- a/net/ipv6/exthdrs.c +++ b/net/ipv6/exthdrs.c @@ -1344,14 +1344,14 @@ ipv6_renew_options(struct sock *sk, struct ipv6_txoptions *opt, return opt2; } -struct ipv6_txoptions *ipv6_fixup_options(struct ipv6_txoptions *opt_space, - struct ipv6_txoptions *opt) +struct ipv6_txoptions *__ipv6_fixup_options(struct ipv6_txoptions *opt_space, + struct ipv6_txoptions *opt) { /* * ignore the dest before srcrt unless srcrt is being included. * --yoshfuji */ - if (opt && opt->dst0opt && !opt->srcrt) { + if (opt->dst0opt && !opt->srcrt) { if (opt_space != opt) { memcpy(opt_space, opt, sizeof(*opt_space)); opt = opt_space; @@ -1362,7 +1362,7 @@ struct ipv6_txoptions *ipv6_fixup_options(struct ipv6_txoptions *opt_space, return opt; } -EXPORT_SYMBOL_GPL(ipv6_fixup_options); +EXPORT_SYMBOL_GPL(__ipv6_fixup_options); /** * fl6_update_dst - update flowi destination address with info given