From patchwork Sun Apr 3 13:06:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799536 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7567CC433F5 for ; Sun, 3 Apr 2022 13:08:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245645AbiDCNKH (ORCPT ); Sun, 3 Apr 2022 09:10:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236847AbiDCNKH (ORCPT ); Sun, 3 Apr 2022 09:10:07 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9AAE2714E for ; Sun, 3 Apr 2022 06:08:12 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id u16so10665489wru.4 for ; Sun, 03 Apr 2022 06:08:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r6mHSY3rMp/h+hhho43L4evOYkmu9O/j90Xx3k1ptBQ=; b=B0eFqH01T/qQ/h0E9kkmFYGzyvqYzGGrzEXr7jAFfgWvJZFosPjM5+xMgsygUValhr 9poJ34IhNAt/NZrSJMzlDChqR8AQUlc9gFWrVC09QjskR0YMvhXs/ONEpHMceQJcPCLT BS+flwvKjqpZrcg1d/rsn1ZRQ6+hz/cG7SX7g0CqPIU4/OxbH3BwcYToKUfG3UAoI1Ow OV1U4mTX67khDao3xARGoSrMgpuW9TQsmPoFSa7YXayoPphdmCAfd2so2LOY8x29QCgv a0RaJ6AhcXsBM4zEqX93WVu6QOIfo6bf/xyXJBs+ZRlUCAONZtzixohaa/XIESOSoQtp ggGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r6mHSY3rMp/h+hhho43L4evOYkmu9O/j90Xx3k1ptBQ=; b=SaUd6QlAGFCl4hwk8zjbnLa2qwN6SZmAHsBU1XFK8onV7AjLgvhvA6nSBf33vUB8I9 VdMsuq9TKSZb6cRYITRd8jXtXSrs8Ja6UxJ5W0+K6tjJ/mU3bbp+If69LqWApb84Aii/ OP44BZaqCg25Fm5GTPPAZPTY/ZK00W96TX7kah8HQn1/5A/lK5XcaYFx9JIzJ4vdjD/M MdYQsc9uslbbEw+YrHKaXkDvI8M2HTPSURv31n9aqUebs/YY4TTM8tjCtFXOUh3Aeneu XvrKH+BAk5uIsG52Qwb8QR0UJKx9BjLAHUMOA19ar7SNBCo524U+0BswWHrLzO/VJR8d Vslw== X-Gm-Message-State: AOAM531bLIyLkikLbAZru1sg3pkpNhZjS4Duwx6WPgNPk6H4CrCqYQr3 OkA2nqIAGZk5TxDhM/bY9E67qcgqBko= X-Google-Smtp-Source: ABdhPJxiHQYGdnx7eepEY9TOiLoaiDVWu9PJjMboNxMBZK56LDymHRhpQvkNJ9B+WprdpTrkgDJbAg== X-Received: by 2002:adf:dd49:0:b0:206:ce4:a004 with SMTP id u9-20020adfdd49000000b002060ce4a004mr2604472wrm.161.1648991291057; Sun, 03 Apr 2022 06:08:11 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:10 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 01/27] sock: deduplicate ->sk_wmem_alloc check Date: Sun, 3 Apr 2022 14:06:13 +0100 Message-Id: <2ba680102bcbdb9f0165aef89c41a43c9ada03c6.1648981570.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The main ->sk_wmem_alloc check in sock_def_write_space() almost completely repeats sock_writeable() apart from small differences like rounding, so we should be able to replace the first check and remove extra sock_writeable(). Signed-off-by: Pavel Begunkov --- net/core/sock.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index 1180a0cb0110..f5766d6e27cb 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3174,15 +3174,14 @@ static void sock_def_write_space(struct sock *sk) /* Do not wake up a writer until he can make "significant" * progress. --DaveM */ - if ((refcount_read(&sk->sk_wmem_alloc) << 1) <= READ_ONCE(sk->sk_sndbuf)) { + if (sock_writeable(sk)) { wq = rcu_dereference(sk->sk_wq); if (skwq_has_sleeper(wq)) wake_up_interruptible_sync_poll(&wq->wait, EPOLLOUT | EPOLLWRNORM | EPOLLWRBAND); /* Should agree with poll, otherwise some programs break */ - if (sock_writeable(sk)) - sk_wake_async(sk, SOCK_WAKE_SPACE, POLL_OUT); + sk_wake_async(sk, SOCK_WAKE_SPACE, POLL_OUT); } rcu_read_unlock(); From patchwork Sun Apr 3 13:06:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799540 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0719C433FE for ; Sun, 3 Apr 2022 13:08:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235661AbiDCNKQ (ORCPT ); Sun, 3 Apr 2022 09:10:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235242AbiDCNKK (ORCPT ); Sun, 3 Apr 2022 09:10:10 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16CD527154 for ; Sun, 3 Apr 2022 06:08:14 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id p189so4336364wmp.3 for ; Sun, 03 Apr 2022 06:08:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4tX9kkoFz5yjBrZACb+aZ/fkC3N1J9HdMrLaUNmfdJg=; b=f5aCWOX9MkjJiRsUA8aaJF86pIMSGy4B0KfIjEDZHD9mrZyIejY7r2md7eca/VIlbs 0k9eOwj0fq8bZ5rM6jypCSgrft78WddDd+qQQdfb2btzE+iKUz0wORGhlzAuscE8Qsym h4HGtgm7JmPwGYsxb5iKmiV2XXnJ4z5v/lLPkrHkbV2BF/5cJAMZWhEUghrnkm3A2c1P RRzMkHG0XUviSY5tJFRISKkc7vIPL8zJ7AR++XAf8Pq/qMHUwbYCQz3vOFXAS/fU9iMR rnEq2pjyLCXPHWRdzZHO5aFgsKtW5O9Ypkc6991RB/qhO/bv62JqPf04qjBD6ikKaUW/ Vohw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4tX9kkoFz5yjBrZACb+aZ/fkC3N1J9HdMrLaUNmfdJg=; b=nI0GPaYjXTEBSDRENXpQQUmbagRHbdbm/RHHD3Kx0V36bo47L0q6S1c2jb1TvRw5Nf udVE4gna87sjGiIe89Xf93ebw+yM2Pds6t0MuOLIqbMb211PB0cX8CJO3CgUdiDKIKfB IUy8UeWbr4GFN6bmWFgM7PTxOZRaBjEHQlP8HT83sdR6a/fpiy+bRusf+BOKKUpWYwpQ GsN+lakNX4Myw94i1LEGFheCkndbJ/iHDFlGkUFTnVrweVzNOEGJX81CFmTqGmPnaWv7 d6xKWKdU5vmtYk4DeweKyWOK2Pm8n9Ve9aXQ0wCmfyXgcGBJVTUhbTEASmO7mnQPuT5J MUIA== X-Gm-Message-State: AOAM5308y8nS7AuA7R3Z/HfSEb9R20UgK/cc8xG0Idu16XkfeuIgWc9F tIfI2i7sBQ3tkCFJLo12OUzrIsRlH5Y= X-Google-Smtp-Source: ABdhPJwjDkK/x/u2DToX7rJmhd65hMehjkMAhhYq2Wi3UYTyrohJ3jpablwHmOMmVERtWJWTOVtMhA== X-Received: by 2002:a05:600c:4f86:b0:38c:bd13:e074 with SMTP id n6-20020a05600c4f8600b0038cbd13e074mr15857864wmq.97.1648991292461; Sun, 03 Apr 2022 06:08:12 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:12 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 02/27] sock: optimise sock_def_write_space send refcounting Date: Sun, 3 Apr 2022 14:06:14 +0100 Message-Id: <769468f1e09dc13caefaa5cebc3ed1e04f747bcc.1648981570.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org sock_def_write_space() is extensively used by UDP and there is some room for optimisation. When sock_wfree() needs to do ->sk_write_space(), it modifies ->sk_wmem_alloc in two steps. First, it puts all but one refs and calls ->sk_write_space(), and then puts down remaining 1. That's needed because the callback relies on ->sk_wmem_alloc being subbed but something should hold the socket alive. The idea behind this patch is to take advantage of SOCK_RCU_FREE and ensure the socket is not freed by wrapping ->sk_write_space() in an RCU section. Then we can remove one extra refcount atomic. Note: not all callbacks might be RCU prepared, so we carve out a sock_def_write_space() specific path. Signed-off-by: Pavel Begunkov --- net/core/sock.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/net/core/sock.c b/net/core/sock.c index f5766d6e27cb..9389bb602c64 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -144,6 +144,8 @@ static DEFINE_MUTEX(proto_list_mutex); static LIST_HEAD(proto_list); +static void sock_def_write_space(struct sock *sk); + /** * sk_ns_capable - General socket capability test * @sk: Socket to use a capability on or through @@ -2300,8 +2302,20 @@ void sock_wfree(struct sk_buff *skb) { struct sock *sk = skb->sk; unsigned int len = skb->truesize; + bool free; if (!sock_flag(sk, SOCK_USE_WRITE_QUEUE)) { + if (sock_flag(sk, SOCK_RCU_FREE) && + sk->sk_write_space == sock_def_write_space) { + rcu_read_lock(); + free = refcount_sub_and_test(len, &sk->sk_wmem_alloc); + sock_def_write_space(sk); + rcu_read_unlock(); + if (unlikely(free)) + __sk_free(sk); + return; + } + /* * Keep a reference on sk_wmem_alloc, this will be released * after sk_write_space() call From patchwork Sun Apr 3 13:06:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799538 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5849DC433F5 for ; Sun, 3 Apr 2022 13:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358416AbiDCNKO (ORCPT ); Sun, 3 Apr 2022 09:10:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358401AbiDCNKL (ORCPT ); Sun, 3 Apr 2022 09:10:11 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86E4C26AF9 for ; Sun, 3 Apr 2022 06:08:15 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id k124-20020a1ca182000000b0038c9cf6e2a6so4117054wme.0 for ; Sun, 03 Apr 2022 06:08:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m586UMPc+B3np9B42WYPK+e0b7Q4+XzQ7XMprCZx6c4=; b=gDyP2tsYut+gj8ph5dlR5+2PAPLSInI0FO0UXCj19rfbn0VwwxyxPAgWx7t71Ty1qg v14wEgNM0DP7WyUpUpvc48nZAegn2HddwCwy3SzSwrHib0rB2FNecwz3U8d1w1IXNYY/ KH3NDkB9V9sLx5JyZZq7R/hsKm94I+XmMYf2gsXjU/CgFrA8oA2cT0174t1TTItmwmvo Q9+ox0TtyCPhrchjuNCVYnRcOyc5TDxDHP/Au3bmmIKr+F88WlXyLP0pJTcnget2jRh6 qurlSOMCAjn+89nsuRW9igN88F/fc3o6eSC1euamGwKwukqdQl0nBJmsHd6oDL/3Ad7s HJSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m586UMPc+B3np9B42WYPK+e0b7Q4+XzQ7XMprCZx6c4=; b=PPtWzWBb64XvIrBFZ3VeW6yBgrUVrMak4SBvGFjwIkh+DRh78gwqIrPj+CY5Stedn5 1MrPkVBICAfvXyzsjjQm3yj0JIGpOqGvDVNXE1I+qa0FFHw/9bbHoZcm/9VMLRnroKv6 N2ojRVtCtmHCYKjyZM+bBxNqMCCK3o9GYfCSieYv8VSp6w88+0TqeByNiuRLBY0Jy8a8 FOLbjZEvBkDgQwNmDZprSfBppeqOAeozPIgRA6zKT+BKrjM+GflUctOidznQdnZMkBvV Zv+G3Fn3sg1fukYNYx+x5EvfW+HHI2JauRtfMs2mwQpwpN6njpUMjTS0A/Um7+7ILOiu gE1w== X-Gm-Message-State: AOAM532l0Jt5/7kBDABKM951+QPlKCauz1U1Vi3ycqhoDjjwQ5VSGk0v b95OZH/lozwByOAmefUBoGGDG9sBu8g= X-Google-Smtp-Source: ABdhPJzQb2V2Hn4VNMmmWA7C44O1+TfB2Wtnw38HEpwIczxMJF+8SGQDeEaYwt57fMq6Nq/S3oZepA== X-Received: by 2002:a05:600c:4ed2:b0:38c:93ad:4825 with SMTP id g18-20020a05600c4ed200b0038c93ad4825mr15554453wmq.181.1648991293951; Sun, 03 Apr 2022 06:08:13 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:13 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 03/27] sock: optimise sock_def_write_space barriers Date: Sun, 3 Apr 2022 14:06:15 +0100 Message-Id: <488b0ee4247ae055503be8c01c6a96427c226f56.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Now we have a separate path for sock_def_write_space() and can go one step further. When it's called from sock_wfree() we know that there is a preceding atomic for putting down ->sk_wmem_alloc. We can use it to replace to replace smb_mb() with a less expensive smp_mb__after_atomic(). It also removes an extra RCU read lock/unlock as a small bonus. Signed-off-by: Pavel Begunkov --- net/core/sock.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/net/core/sock.c b/net/core/sock.c index 9389bb602c64..b1a8f47fda55 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -144,6 +144,7 @@ static DEFINE_MUTEX(proto_list_mutex); static LIST_HEAD(proto_list); +static void sock_def_write_space_wfree(struct sock *sk); static void sock_def_write_space(struct sock *sk); /** @@ -2309,7 +2310,7 @@ void sock_wfree(struct sk_buff *skb) sk->sk_write_space == sock_def_write_space) { rcu_read_lock(); free = refcount_sub_and_test(len, &sk->sk_wmem_alloc); - sock_def_write_space(sk); + sock_def_write_space_wfree(sk); rcu_read_unlock(); if (unlikely(free)) __sk_free(sk); @@ -3201,6 +3202,29 @@ static void sock_def_write_space(struct sock *sk) rcu_read_unlock(); } +/* An optimised version of sock_def_write_space(), should only be called + * for SOCK_RCU_FREE sockets under RCU read section and after putting + * ->sk_wmem_alloc. + */ +static void sock_def_write_space_wfree(struct sock *sk) +{ + /* Do not wake up a writer until he can make "significant" + * progress. --DaveM + */ + if (sock_writeable(sk)) { + struct socket_wq *wq = rcu_dereference(sk->sk_wq); + + /* rely on refcount_sub from sock_wfree() */ + smp_mb__after_atomic(); + if (wq && waitqueue_active(&wq->wait)) + wake_up_interruptible_sync_poll(&wq->wait, EPOLLOUT | + EPOLLWRNORM | EPOLLWRBAND); + + /* Should agree with poll, otherwise some programs break */ + sk_wake_async(sk, SOCK_WAKE_SPACE, POLL_OUT); + } +} + static void sock_def_destruct(struct sock *sk) { } From patchwork Sun Apr 3 13:06:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799539 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6794C433EF for ; Sun, 3 Apr 2022 13:08:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358424AbiDCNKP (ORCPT ); Sun, 3 Apr 2022 09:10:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235661AbiDCNKL (ORCPT ); Sun, 3 Apr 2022 09:10:11 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B623227159 for ; Sun, 3 Apr 2022 06:08:16 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id d29so4584454wra.10 for ; Sun, 03 Apr 2022 06:08:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cAQX/zAboHQHlBe7X9iztRMX0Jxv73yIQeqQNfVA2p0=; b=AQ2jgQ0nb1Ib0s7lPkTtMtx+3Rp9IttydMIpoldCY58nI37Nbin204jHnQp7xvtrNM ILH+oBzSnzZYJDLBS+fUhJBE/ZZ7vUFgXkb2i0tmizSxNoa/D51WNd1LedCqfDicP0/I IjW+X8ppLE3umvG12ypmzxfXE9LMv7NPdthKjn/B0h0DUFalqiruCvfUXVR6acGEGC3n xnJp/piyRWbvSdh3qd60QCCuDrwWncczF669oYovMHnGXDF6gdiTfFIpEPjn5KWyViX6 4kf34R9MN1TQetGr5ANtiLHHQaNuWTbDvavsNlEXKIe98+Lc+3wDSxYco9Ua7JXo+SAY Cfug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cAQX/zAboHQHlBe7X9iztRMX0Jxv73yIQeqQNfVA2p0=; b=P1kRM4fajSM2+CdrCE6I2DpC2K0WllgW0JeJCX0/QmnC2Opv/dsKghNp/oa91fPD+v UQ68Uelw7wQOhY6UR8eOCYoRrZ1sjwLl8ovK7EGFiKdyEMCdoqm2K97VgLkHft6UZinK aW0pThdmh1byM7U4tnvYj+oE+vCJ1OMHVvwZ+k+c/3NLR90SRys4coBId1gA/rxzwxpR 3VmdgS1+VruLBTOIqJwbDah70BodLaQKM0LS1T84w64lnvE+cD7oGjgtL645IPXESUI0 fg9kboF+XzuVkMUn81Avjrujsna+wGtW2QS3E7WCPhebVbcMAeqm8v+FL+mfzPOLROfB ejzA== X-Gm-Message-State: AOAM530IlddgTYBvQfd3G4Y2bimvCPYHAf4GMBGc5gD5qMfP54ncav1a J/6Z5e1tyxbsFtPCLH3JqYqIrquzKYY= X-Google-Smtp-Source: ABdhPJzGkedqER16IvxV5imy4bKWlJJ1o1BcyQ9HTZOwLBXDl7fzYcMqhVDcmSr2LCPHM/Wc/o97Ag== X-Received: by 2002:a5d:54ce:0:b0:205:133d:c152 with SMTP id x14-20020a5d54ce000000b00205133dc152mr13819458wrv.334.1648991295184; Sun, 03 Apr 2022 06:08:15 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:14 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 04/27] skbuff: drop zero check from skb_zcopy_set Date: Sun, 3 Apr 2022 14:06:16 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Only two skb_zcopy_set() callers may pass a null skb, so kill a null check from the function, which can't be easily compiled out and hand code where needed. This will also help with further patches. Signed-off-by: Pavel Begunkov --- include/linux/skbuff.h | 2 +- net/ipv4/ip_output.c | 3 ++- net/ipv6/ip6_output.c | 3 ++- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 3a30cae8b0a5..f5de5c9cc3da 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1679,7 +1679,7 @@ static inline void skb_zcopy_init(struct sk_buff *skb, struct ubuf_info *uarg) static inline void skb_zcopy_set(struct sk_buff *skb, struct ubuf_info *uarg, bool *have_ref) { - if (skb && uarg && !skb_zcopy(skb)) { + if (uarg && !skb_zcopy(skb)) { if (unlikely(have_ref && *have_ref)) *have_ref = false; else diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 00b4bf26fd93..f864b8c48e42 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1027,7 +1027,8 @@ static int __ip_append_data(struct sock *sk, paged = true; } else { uarg->zerocopy = 0; - skb_zcopy_set(skb, uarg, &extra_uref); + if (skb) + skb_zcopy_set(skb, uarg, &extra_uref); } } diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index e23f058166af..e9b039f56637 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1529,7 +1529,8 @@ static int __ip6_append_data(struct sock *sk, paged = true; } else { uarg->zerocopy = 0; - skb_zcopy_set(skb, uarg, &extra_uref); + if (skb) + skb_zcopy_set(skb, uarg, &extra_uref); } } From patchwork Sun Apr 3 13:06:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799541 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4ED9C433F5 for ; Sun, 3 Apr 2022 13:08:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358542AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358421AbiDCNKP (ORCPT ); Sun, 3 Apr 2022 09:10:15 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A9E72717D for ; Sun, 3 Apr 2022 06:08:18 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id d7so10623976wrb.7 for ; Sun, 03 Apr 2022 06:08:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L1VtlVNeW0B4aRj8TAPTIudNNvRffgx/xr8HTpKXaaw=; b=Z4s1dCHD1hvK9Sd6pD3i0LprmTCayrEFXW3HZU/mXPqtQ6SVOSyfVeyKm1WWgTnUaK 51a1RM+JVZQtg6Mj/wvGzLazNC/KqkzMDw5PSdxIZRy1387JRIY1KqL+HbYdqo245+1H ou/SwpIaMuz5dz6JhHzMGua+hMKeKuMh7GABpeXAGqqJC6jIw4EhiSgrMOauOeSeFJdE EcVX+NOlCJIQeycddwwfgS3T5nWE9cXARsoGgKBROHz3PyHNhN0QyUwUSvk+uij84AXD 2fdk66GmlfroSg219pIGVjkNN7Anx4T/r48bRkN/3rUZ/laPI3NEP9L3nivxRc5HgAyJ IfgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L1VtlVNeW0B4aRj8TAPTIudNNvRffgx/xr8HTpKXaaw=; b=wuuPHgjSVbrHlsXbCbKrQ26RHW1I48fAm2EkmslGrQth1WV66WEI91rBRGreW8I7QY UUoCqqVOpbLCz7/9EofwxBE70+lGEZk2Nv43Iy0KHjIHgpHITGuhceWk2X0MxnRsfklE RA2l3IlKQlL1z4nddF9YrLaXd8Hq5BbRBZSVQcY7SYjyZSHrpIZckBWh7R5BM/dYJlEP oOYnUSR+qg/fWTHbnxz5VovQ+VpncDXvCjSacnxPQTU+7GwcYh086TbJWUBR3wLOEKKk 19MIjxpXE0ad6cVCurvg4dLimLTZDv35vz0IuKNRM1tZFOEFXylDaQOs3vP3Kzfs8Zsb vIHw== X-Gm-Message-State: AOAM5332X9WTT2aiVAUQ6cx4k4Vib/Zsc44VKKt0oWQ5YVz8v+Kccc2T bSdmodbSFMWFWhPORRwzs9jOZKqMF6A= X-Google-Smtp-Source: ABdhPJy+5Z3aXlocAGQzHFLIjaxgUfMehYWi5J6671yK/q85NUAUAGGebtln9schsjaBw+LIa6dYmw== X-Received: by 2002:a05:6000:508:b0:1e4:a027:d147 with SMTP id a8-20020a056000050800b001e4a027d147mr13876685wrf.315.1648991296598; Sun, 03 Apr 2022 06:08:16 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:16 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 05/27] skbuff: drop null check from skb_zcopy Date: Sun, 3 Apr 2022 14:06:17 +0100 Message-Id: <62db530f5b9875c820c97d4b6b2f30b511d94468.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org skb_zcopy() is used all around the networkong code including generic paths. Many callers pass only a non-null skb, so we can remove it from there and fix up several callers that would be affected. It removes extra checks from zerocopy paths but also sheds some bytes from the binary. text data bss dec hex filename 8521472 0 0 8521472 820700 arch/x86/boot/bzImage 8521056 0 0 8521056 820560 arch/x86/boot/bzImage delta=416B Signed-off-by: Pavel Begunkov --- include/linux/skbuff.h | 2 +- net/core/dev.c | 2 +- net/core/skbuff.c | 3 ++- net/ipv4/ip_output.c | 7 +++++-- net/ipv4/tcp.c | 5 ++++- net/ipv6/ip6_output.c | 7 +++++-- 6 files changed, 18 insertions(+), 8 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index f5de5c9cc3da..10f94b1909da 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1649,7 +1649,7 @@ static inline struct skb_shared_hwtstamps *skb_hwtstamps(struct sk_buff *skb) static inline struct ubuf_info *skb_zcopy(struct sk_buff *skb) { - bool is_zcopy = skb && skb_shinfo(skb)->flags & SKBFL_ZEROCOPY_ENABLE; + bool is_zcopy = skb_shinfo(skb)->flags & SKBFL_ZEROCOPY_ENABLE; return is_zcopy ? skb_uarg(skb) : NULL; } diff --git a/net/core/dev.c b/net/core/dev.c index 8a5109479dbe..4842a398f08d 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2286,7 +2286,7 @@ void dev_queue_xmit_nit(struct sk_buff *skb, struct net_device *dev) } out_unlock: if (pt_prev) { - if (!skb_orphan_frags_rx(skb2, GFP_ATOMIC)) + if (!skb2 || !skb_orphan_frags_rx(skb2, GFP_ATOMIC)) pt_prev->func(skb2, skb->dev, pt_prev, skb->dev); else kfree_skb(skb2); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 10bde7c6db44..7680314038b4 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -893,7 +893,8 @@ EXPORT_SYMBOL(skb_dump); */ void skb_tx_error(struct sk_buff *skb) { - skb_zcopy_clear(skb, true); + if (skb) + skb_zcopy_clear(skb, true); } EXPORT_SYMBOL(skb_tx_error); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index f864b8c48e42..ab10b1f94669 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1018,10 +1018,13 @@ static int __ip_append_data(struct sock *sk, csummode = CHECKSUM_PARTIAL; if (flags & MSG_ZEROCOPY && length && sock_flag(sk, SOCK_ZEROCOPY)) { - uarg = msg_zerocopy_realloc(sk, length, skb_zcopy(skb)); + if (skb) + uarg = skb_zcopy(skb); + extra_uref = !uarg; /* only ref on new uarg */ + + uarg = msg_zerocopy_realloc(sk, length, uarg); if (!uarg) return -ENOBUFS; - extra_uref = !skb_zcopy(skb); /* only ref on new uarg */ if (rt->dst.dev->features & NETIF_F_SG && csummode == CHECKSUM_PARTIAL) { paged = true; diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index cf18fbcbf123..add71b703520 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1205,7 +1205,10 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (flags & MSG_ZEROCOPY && size && sock_flag(sk, SOCK_ZEROCOPY)) { skb = tcp_write_queue_tail(sk); - uarg = msg_zerocopy_realloc(sk, size, skb_zcopy(skb)); + if (skb) + uarg = skb_zcopy(skb); + + uarg = msg_zerocopy_realloc(sk, size, uarg); if (!uarg) { err = -ENOBUFS; goto out_err; diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index e9b039f56637..f1ada6f2af7d 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1520,10 +1520,13 @@ static int __ip6_append_data(struct sock *sk, csummode = CHECKSUM_PARTIAL; if (flags & MSG_ZEROCOPY && length && sock_flag(sk, SOCK_ZEROCOPY)) { - uarg = msg_zerocopy_realloc(sk, length, skb_zcopy(skb)); + if (skb) + uarg = skb_zcopy(skb); + extra_uref = !uarg; /* only ref on new uarg */ + + uarg = msg_zerocopy_realloc(sk, length, uarg); if (!uarg) return -ENOBUFS; - extra_uref = !skb_zcopy(skb); /* only ref on new uarg */ if (rt->dst.dev->features & NETIF_F_SG && csummode == CHECKSUM_PARTIAL) { paged = true; From patchwork Sun Apr 3 13:06:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799543 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AD57C433F5 for ; Sun, 3 Apr 2022 13:08:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358633AbiDCNKc (ORCPT ); Sun, 3 Apr 2022 09:10:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358468AbiDCNKR (ORCPT ); Sun, 3 Apr 2022 09:10:17 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49BB8393F5 for ; Sun, 3 Apr 2022 06:08:19 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id p189so4336427wmp.3 for ; Sun, 03 Apr 2022 06:08:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vbNrUqcL+5B6jcAJE+OFILYZJBfw/FalxK3WQ8Mdyzs=; b=mKOweO1WNQC1Zb05N6p78EA1oUpHKUxcRWq4Wzpdl8elw9NzOahMaEm3JGxINiK0+z GT6v/26M3yK/6nPJtwk/7Cvn5H2ZJ3SJZ4pD1RQaQ3dP5/UKTvTF0gYIR+bxi3XYWuGk EZOtU63Nik2aWbeAkuvN7HLBbdGJRBCLYLf+bGPvsuv2du1TXaHgAKxUvKC1ypHAiH5S z5bLlWqgFGFJHE3upWEjKvX2+5A/+6YIM0gub1BX478K+9sfS0f7yJnvf9CrfPhofWLZ yHQqd9ZgbA66v/8mwfAnC5LXI8HGBITRxyJ5hPrkQ1aFiTDlcanmtdmQ5NOlnbKlTvsv K1QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vbNrUqcL+5B6jcAJE+OFILYZJBfw/FalxK3WQ8Mdyzs=; b=8RgnJJIRCCeCwg7HmSYMnnsHB6LneEZCb4T5YNnjnpJk9Zm9trOXfPSllbdGB5TnmA 9Xt67t6iGXLuCpbzOOHwRyNKCQAJ5odrhcWT/kXaNS6dLU1uEDdawzHDBvtdBt4O8y2L tngdOvQ9Et+SEqFT8oqKbwCmzHiZIrX82hyz/xQFZwxZLfeaBRcI4TUFm228YZSrpm2l dw7WIHFIq4oS+ArK1tC5V3/UrUHR9djMIezEmFhsWXAlYnqXTJi2ymYAhJ8kiWXD6vHW pIPnZGOE4crko77cXtldehLsdI2TtAYYTuEhtaHvmcp4duqmoh5j1OMTFw4VYUSO+I4/ 5d1g== X-Gm-Message-State: AOAM531woPAvKGMpjAO/+MVbnWDjj3j3Sh9YANI2aOSknAgrdBeaMWj+ OG5xtfEZJ/YWGbLgc61qefUID7IuAD0= X-Google-Smtp-Source: ABdhPJz7HbfRMzb7o1fNHPI2tavbXmldCXPyJOU4wO94Oey4Z5NWWjCFtONF5BpQTjMuTRlh2bjv+Q== X-Received: by 2002:a05:600c:5113:b0:38d:d8f:67e9 with SMTP id o19-20020a05600c511300b0038d0d8f67e9mr15422593wms.107.1648991297600; Sun, 03 Apr 2022 06:08:17 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:17 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 06/27] net: xen: set zc flags only when there is ubuf Date: Sun, 3 Apr 2022 14:06:18 +0100 Message-Id: <3b75e002109483819858dd303ca91981b33ee0d9.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In preparation to changing zc ubuf invariants, set SKBFL_ZEROCOPY_ENABLE IFF there is a ubuf set. Signed-off-by: Pavel Begunkov --- drivers/net/xen-netback/interface.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c index fe8e21ad8ed9..0a0c36a38fd4 100644 --- a/drivers/net/xen-netback/interface.c +++ b/drivers/net/xen-netback/interface.c @@ -55,7 +55,8 @@ void xenvif_skb_zerocopy_prepare(struct xenvif_queue *queue, struct sk_buff *skb) { - skb_shinfo(skb)->flags |= SKBFL_ZEROCOPY_ENABLE; + if (skb_uarg(skb)) + skb_shinfo(skb)->flags |= SKBFL_ZEROCOPY_ENABLE; atomic_inc(&queue->inflight_packets); } From patchwork Sun Apr 3 13:06:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799542 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF13BC433EF for ; Sun, 3 Apr 2022 13:08:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358768AbiDCNKa (ORCPT ); Sun, 3 Apr 2022 09:10:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358477AbiDCNK1 (ORCPT ); Sun, 3 Apr 2022 09:10:27 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE01027154 for ; Sun, 3 Apr 2022 06:08:20 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id q19so3713807wrc.6 for ; Sun, 03 Apr 2022 06:08:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KmKRZVdSh5FqmA1REwzHCTiq5E4GC2nAqX9kULvJvZU=; b=Cu8L3mnPMzui1x64BdWt4PsYpsr2zA9c6DkJosOJxgnmzUepCnTCZwPP0Q2mtQAkcr Cbin67FnEj60ccgXmDtetiPrVMMN+i4hDGOpTYVxJd/DD/XemhHM+3P2atwDob1EX5FI cOmgmJEhrNQxWLi5k/VfhNgP2vgOYYCdtym+oX5RsRUzHL7uXpPDhlEqjrENjEqg571I v0hNp16wypSOp84P5PVK73N6Ho20+VqFaXBx2dfIZJmj+ICGHqfavjlUAEv7MgviieSi 3kchbfawhxqH+qRPhnc2CbtHQvZKADg+AfAhz6emBCBu6VbxD7Utp0CRMnFuZZC2JoIj QRAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KmKRZVdSh5FqmA1REwzHCTiq5E4GC2nAqX9kULvJvZU=; b=4cIJNNWkW/E8N6usBH3KV4vZFvzZv4CWzledy49y0WfArD6QonCfBR4U3QWJzy01Mg wZJD+kcvVGF0UYEWDqRIKSc5gQ5ieeSP99DgDTTm9NMWX0V//Lz6QPNlGgg5Nkxr1NZh gJeMU7ssDfb5Rrh4ItM1RFhqXltsB4EF7DLzzO7a90PNJ5fgMuKtIAjgdjSC1HGbb5OD QEGpgou2EMBMr/IA5lSZx61GEsYE/qt+UbcCnXAVnp8IQIi218WnTRsYdDGSavgBqJ73 zNvjlXScnFCe890F3WpXxpdUhSEDWqqphDhypsKSVKeqBgRvfk4/gCvyJM/iQP3alT/y VLYQ== X-Gm-Message-State: AOAM531Xx3XatKcgwfM6BvDLHDt52QiG5/OkNlHAnTirfKby8+edoDQR kvnucN2xUZjXKH3LBVdIe1IOTrRkhDY= X-Google-Smtp-Source: ABdhPJxfBtOGdTRtS5Q8J8UNaO4B68ZAkrwtnKarkic0oIXyxuj/Wsn49OfKuOppPgSrGeJNyoLwhQ== X-Received: by 2002:a05:6000:186d:b0:204:110a:d832 with SMTP id d13-20020a056000186d00b00204110ad832mr13935618wri.47.1648991299105; Sun, 03 Apr 2022 06:08:19 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:18 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 07/27] skbuff: introduce skb_is_zcopy() Date: Sun, 3 Apr 2022 14:06:19 +0100 Message-Id: <3cf9407f8fa1b73845d0a23e2f7445b5633d626d.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add a new helper function called skb_is_zcopy() for checking for an skb zerocopy status. Before we were using skb_zcopy() for that, but it's slightly heavier and generates extra code. Note: since the previous patch we should have a ubuf set IFF an skb is SKBFL_ZEROCOPY_ENABLE marked apart from nouarg cases. Signed-off-by: Pavel Begunkov --- include/linux/skbuff.h | 25 +++++++++++++++---------- net/core/skbuff.c | 15 +++++++-------- 2 files changed, 22 insertions(+), 18 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 10f94b1909da..410850832b6a 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1647,11 +1647,14 @@ static inline struct skb_shared_hwtstamps *skb_hwtstamps(struct sk_buff *skb) return &skb_shinfo(skb)->hwtstamps; } -static inline struct ubuf_info *skb_zcopy(struct sk_buff *skb) +static inline bool skb_is_zcopy(struct sk_buff *skb) { - bool is_zcopy = skb_shinfo(skb)->flags & SKBFL_ZEROCOPY_ENABLE; + return skb_shinfo(skb)->flags & SKBFL_ZEROCOPY_ENABLE; +} - return is_zcopy ? skb_uarg(skb) : NULL; +static inline struct ubuf_info *skb_zcopy(struct sk_buff *skb) +{ + return skb_is_zcopy(skb) ? skb_uarg(skb) : NULL; } static inline bool skb_zcopy_pure(const struct sk_buff *skb) @@ -1679,7 +1682,7 @@ static inline void skb_zcopy_init(struct sk_buff *skb, struct ubuf_info *uarg) static inline void skb_zcopy_set(struct sk_buff *skb, struct ubuf_info *uarg, bool *have_ref) { - if (uarg && !skb_zcopy(skb)) { + if (uarg && !skb_is_zcopy(skb)) { if (unlikely(have_ref && *have_ref)) *have_ref = false; else @@ -1723,11 +1726,13 @@ static inline void net_zcopy_put_abort(struct ubuf_info *uarg, bool have_uref) /* Release a reference on a zerocopy structure */ static inline void skb_zcopy_clear(struct sk_buff *skb, bool zerocopy_success) { - struct ubuf_info *uarg = skb_zcopy(skb); - if (uarg) { - if (!skb_zcopy_is_nouarg(skb)) + if (skb_is_zcopy(skb)) { + if (!skb_zcopy_is_nouarg(skb)) { + struct ubuf_info *uarg = skb_zcopy(skb); + uarg->callback(skb, uarg, zerocopy_success); + } skb_shinfo(skb)->flags &= ~SKBFL_ALL_ZEROCOPY; } @@ -3023,7 +3028,7 @@ static inline void skb_orphan(struct sk_buff *skb) */ static inline int skb_orphan_frags(struct sk_buff *skb, gfp_t gfp_mask) { - if (likely(!skb_zcopy(skb))) + if (likely(!skb_is_zcopy(skb))) return 0; if (!skb_zcopy_is_nouarg(skb) && skb_uarg(skb)->callback == msg_zerocopy_callback) @@ -3034,7 +3039,7 @@ static inline int skb_orphan_frags(struct sk_buff *skb, gfp_t gfp_mask) /* Frags must be orphaned, even if refcounted, if skb might loop to rx path */ static inline int skb_orphan_frags_rx(struct sk_buff *skb, gfp_t gfp_mask) { - if (likely(!skb_zcopy(skb))) + if (likely(!skb_is_zcopy(skb))) return 0; return skb_copy_ubufs(skb, gfp_mask); } @@ -3591,7 +3596,7 @@ static inline int skb_add_data(struct sk_buff *skb, static inline bool skb_can_coalesce(struct sk_buff *skb, int i, const struct page *page, int off) { - if (skb_zcopy(skb)) + if (skb_is_zcopy(skb)) return false; if (i) { const skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1]; diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7680314038b4..f7842bfdd7ae 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -1350,14 +1350,13 @@ int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, struct msghdr *msg, int len, struct ubuf_info *uarg) { - struct ubuf_info *orig_uarg = skb_zcopy(skb); struct iov_iter orig_iter = msg->msg_iter; int err, orig_len = skb->len; /* An skb can only point to one uarg. This edge case happens when * TCP appends to an skb, but zerocopy_realloc triggered a new alloc. */ - if (orig_uarg && uarg != orig_uarg) + if (skb_is_zcopy(skb) && uarg != skb_zcopy(skb)) return -EEXIST; err = __zerocopy_sg_from_iter(sk, skb, &msg->msg_iter, len); @@ -1380,9 +1379,9 @@ EXPORT_SYMBOL_GPL(skb_zerocopy_iter_stream); static int skb_zerocopy_clone(struct sk_buff *nskb, struct sk_buff *orig, gfp_t gfp_mask) { - if (skb_zcopy(orig)) { - if (skb_zcopy(nskb)) { - /* !gfp_mask callers are verified to !skb_zcopy(nskb) */ + if (skb_is_zcopy(orig)) { + if (skb_is_zcopy(nskb)) { + /* !gfp_mask callers are verified to !skb_is_zcopy(nskb) */ if (!gfp_mask) { WARN_ON_ONCE(1); return -ENOMEM; @@ -1721,8 +1720,8 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, if (skb_cloned(skb)) { if (skb_orphan_frags(skb, gfp_mask)) goto nofrags; - if (skb_zcopy(skb)) - refcount_inc(&skb_uarg(skb)->refcnt); + if (skb_is_zcopy(skb)) + net_zcopy_get(skb_uarg(skb)); for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) skb_frag_ref(skb, i); @@ -3535,7 +3534,7 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen) if (skb_headlen(skb)) return 0; - if (skb_zcopy(tgt) || skb_zcopy(skb)) + if (skb_is_zcopy(tgt) || skb_is_zcopy(skb)) return 0; todo = shiftlen; From patchwork Sun Apr 3 13:06:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799544 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB9C5C433FE for ; Sun, 3 Apr 2022 13:08:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358635AbiDCNKg (ORCPT ); Sun, 3 Apr 2022 09:10:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358503AbiDCNK1 (ORCPT ); Sun, 3 Apr 2022 09:10:27 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9813393FD for ; Sun, 3 Apr 2022 06:08:21 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id n63-20020a1c2742000000b0038d0c31db6eso4097087wmn.1 for ; Sun, 03 Apr 2022 06:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wUEmFx91uw/Q5RHRXDGBiQki94Rhwniq9a+OnZPUQPQ=; b=Gggwz4tvUuiIfZvwMcDNkdjT4yQAAg6EJ8S6lKLlqdgeI2csZI919zItNmie876Cfg wJ6MzltEhscJgWzsB7ZHxfKTqWssbelza3vzjsiPRZ+NzOk3FMObxjsO8QU2ukQry1d/ 6olXxOy8l3QDSmeDJKOpZh1IYdHIYXSfQCHNkNitmLOnyVuzRpFIElttoqbRZgzoAOie Dk26p93sYvp8tpL/ldm5VTDS0cq+/876Z7YI5QBnQvH7bF8QtoYjDxpyflfhk0pZfM3H 6Wg6yfwC1tgn3FBrpHwtiwgq3iZBGfsZWAFfVDRj+tQJG5lxQzopqVvUuB2RqL45jyeN EnXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wUEmFx91uw/Q5RHRXDGBiQki94Rhwniq9a+OnZPUQPQ=; b=s9nwlr9L1RRw7PIkJ5iiw3xJOjcnmvtgsXuUpTlQFeX5bdJyPRLmywlMONBhth5mua y+dXthIrz2Th3b+UEg/HUtzYaH+oOLfLVN3/0Bx4+W8gOGCjLHp8n5pD7lfAa3MCUKmq i8q6Hhjyo9g2dyUJMqc0lnvnINwKe2PxP/fkywJ0HVKCggdiYI+m7v4t3/tS1dut9JNE Qxp+xoSWd3Zrt/pFFS89gPCtcC/siqztAaCeCqche17Ul7d9sVvXrST8CLwRuJzqQemV Z5FXeSXhEf4u/F9LFGYbQFc+9FYZDNx5z0o4rqya0I/HCyej7UJhTgmmF8d81GQySF25 Mdug== X-Gm-Message-State: AOAM5323PffP/oMvC25sUuFEkP2j3p+AeDcjHkz+e2jP3JAKxMo249ih iyuOsUu2cRAsyxMg2vNLz3xgt2nxeos= X-Google-Smtp-Source: ABdhPJwFX/6Bsvd9ZOQWMHQ3nm68Nv84FzwFFEZN0hDFPf9i4EB7qJYR9eeUKLA+ipacLRxr90CEYQ== X-Received: by 2002:a05:600c:3511:b0:38c:d035:cddb with SMTP id h17-20020a05600c351100b0038cd035cddbmr15714893wmq.74.1648991300170; Sun, 03 Apr 2022 06:08:20 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:19 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 08/27] skbuff: optimise alloc_skb_with_frags() Date: Sun, 3 Apr 2022 14:06:20 +0100 Message-Id: <6011c226ac8c8d3c178675dfc53ca1c5d84c1095.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Some users of alloc_skb_with_frags() including UDP pass zero datalen, Extract and inline the pure skb alloc part of it. We also save on needlessly pre-setting errcode ptr and with other small refactorings. Signed-off-by: Pavel Begunkov --- include/linux/skbuff.h | 41 ++++++++++++++++++++++++++++++++++++----- net/core/skbuff.c | 31 ++++++++++++------------------- 2 files changed, 48 insertions(+), 24 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 410850832b6a..ebc4ad36c3a2 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1300,11 +1300,42 @@ static inline struct sk_buff *alloc_skb(unsigned int size, return __alloc_skb(size, priority, 0, NUMA_NO_NODE); } -struct sk_buff *alloc_skb_with_frags(unsigned long header_len, - unsigned long data_len, - int max_page_order, - int *errcode, - gfp_t gfp_mask); +struct sk_buff *alloc_skb_frags(struct sk_buff *skb, + unsigned long data_len, + int max_page_order, + int *errcode, + gfp_t gfp_mask); + +/** + * alloc_skb_with_frags - allocate skb with page frags + * + * @header_len: size of linear part + * @data_len: needed length in frags + * @max_page_order: max page order desired. + * @errcode: pointer to error code if any + * @gfp_mask: allocation mask + * + * This can be used to allocate a paged skb, given a maximal order for frags. + */ +static inline struct sk_buff *alloc_skb_with_frags(unsigned long header_len, + unsigned long data_len, + int max_page_order, + int *errcode, + gfp_t gfp_mask) +{ + struct sk_buff *skb; + + skb = alloc_skb(header_len, gfp_mask); + if (unlikely(!skb)) { + *errcode = -ENOBUFS; + return NULL; + } + + if (!data_len) + return skb; + return alloc_skb_frags(skb, data_len, max_page_order, errcode, gfp_mask); +} + struct sk_buff *alloc_skb_for_msg(struct sk_buff *first); /* Layout of fast clones : [skb1][skb2][fclone_ref] */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index f7842bfdd7ae..2c787d964a60 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5955,40 +5955,32 @@ int skb_mpls_dec_ttl(struct sk_buff *skb) EXPORT_SYMBOL_GPL(skb_mpls_dec_ttl); /** - * alloc_skb_with_frags - allocate skb with page frags + * alloc_skb_frags - allocate page frags for skb * - * @header_len: size of linear part + * @skb: buffer * @data_len: needed length in frags * @max_page_order: max page order desired. * @errcode: pointer to error code if any * @gfp_mask: allocation mask * - * This can be used to allocate a paged skb, given a maximal order for frags. + * This can be used to allocate pages for skb, given a maximal order for frags. */ -struct sk_buff *alloc_skb_with_frags(unsigned long header_len, - unsigned long data_len, - int max_page_order, - int *errcode, - gfp_t gfp_mask) +struct sk_buff *alloc_skb_frags(struct sk_buff *skb, + unsigned long data_len, + int max_page_order, + int *errcode, + gfp_t gfp_mask) { int npages = (data_len + (PAGE_SIZE - 1)) >> PAGE_SHIFT; unsigned long chunk; - struct sk_buff *skb; struct page *page; int i; - *errcode = -EMSGSIZE; /* Note this test could be relaxed, if we succeed to allocate * high order pages... */ - if (npages > MAX_SKB_FRAGS) - return NULL; - - *errcode = -ENOBUFS; - skb = alloc_skb(header_len, gfp_mask); - if (!skb) - return NULL; - + if (unlikely(npages > MAX_SKB_FRAGS)) + goto failure; skb->truesize += npages << PAGE_SHIFT; for (i = 0; npages > 0; i++) { @@ -6022,9 +6014,10 @@ struct sk_buff *alloc_skb_with_frags(unsigned long header_len, failure: kfree_skb(skb); + *errcode = -EMSGSIZE; return NULL; } -EXPORT_SYMBOL(alloc_skb_with_frags); +EXPORT_SYMBOL(alloc_skb_frags); /* carve out the first off bytes from skb when off < headlen */ static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off, From patchwork Sun Apr 3 13:06:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799545 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69774C433F5 for ; Sun, 3 Apr 2022 13:08:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235242AbiDCNKh (ORCPT ); Sun, 3 Apr 2022 09:10:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358532AbiDCNK2 (ORCPT ); Sun, 3 Apr 2022 09:10:28 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F29CB13B for ; Sun, 3 Apr 2022 06:08:22 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id d7so10624152wrb.7 for ; Sun, 03 Apr 2022 06:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mqCJk3Tcq92QKRFxYlavWwmlfxI03jKy0XINWiL2FrI=; b=ODZ2mADnavhrYfzE4peIovQEBGpbGp0Zxz8mZTXMXjYBfcq8Lt9Ms/WrRbsqJQECli RT/Ish8qaFIP40Bqj2MqG7C+zNQkDZDX7We6ZejACxeY4RnnAfrXU1gO3Psm/f0WMBBN /AY9X96CPkt4sgyBF4u5uB9XCVKGDEaDTGujnrjrLFq1pOfnLzNncj5+C68mcm6Tp79B UNYvYi8Z8pqVV5reHIeqNv0tBf07L/jeLvLYo/zo6OVrzpuMxCzDIChpxbX1RkRjLb5j 7BB88ZMlOQ/2cBsZMkZowOeyT8cMCRlgseX/5Fdvmh1xPir0Dw6YVBz2++YIiIpxNRKu KKEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mqCJk3Tcq92QKRFxYlavWwmlfxI03jKy0XINWiL2FrI=; b=vRDGjf4E1MvHhwIJFC7m/v8d1/5ediBuoD1G4aVHPsFVn0FXt3alZ75x1xSIuHXmhh R7GYnOiiHAwtpiy1I6cUBGxE8u8EuV+pQKJW9CrCHgmpW1QF7TpnuXm7sJpv1ITK9ZC7 GF7es4JFQXmLdoGFoC4H1GxVf5sCoUDG/3RlatUTj9rf/8YrF2YyQLw6ti0GaV9la4Kp yH59HgcTL/jkAqnRXDchhQ2YnKHsCjBAphqm/18kLE+nQICrXspzLCNFqEQr6A3qsZ3V +VB4XlG1owscbQaUnAhkwMZcG2KzF/rCBxTLMIMS/zr8LVv2uazhFEYbRVWZEG7PbBUH eAZQ== X-Gm-Message-State: AOAM531bkXPHVmxSnNvB9IL6Qy+qJbqyqN3AqCWCRsSi0nf9grWFcoHY vbATKgWA56oiTAdO52dKCbN36LLbhyc= X-Google-Smtp-Source: ABdhPJzPtZMFtKNp1/YY5bwirlHKXf5w7ygwPDf8Qn4vpsNiRJ/FeTbu5IvDw95vQCJfrVn2cxFrtg== X-Received: by 2002:adf:fa09:0:b0:206:10e7:c7d9 with SMTP id m9-20020adffa09000000b0020610e7c7d9mr766779wrr.549.1648991301342; Sun, 03 Apr 2022 06:08:21 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:21 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 09/27] net: inline sock_alloc_send_skb Date: Sun, 3 Apr 2022 14:06:21 +0100 Message-Id: <02d5e2ea08dc28f3e22245c6c1110a108e576abc.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org sock_alloc_send_skb() is simple and just proxying to another function, so we can inline it and cut associated overhead. Signed-off-by: Pavel Begunkov --- include/net/sock.h | 10 ++++++++-- net/core/sock.c | 7 ------- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index c4b91fc19b9c..9dab633c3caf 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1825,11 +1825,17 @@ int sock_getsockopt(struct socket *sock, int level, int op, char __user *optval, int __user *optlen); int sock_gettstamp(struct socket *sock, void __user *userstamp, bool timeval, bool time32); -struct sk_buff *sock_alloc_send_skb(struct sock *sk, unsigned long size, - int noblock, int *errcode); struct sk_buff *sock_alloc_send_pskb(struct sock *sk, unsigned long header_len, unsigned long data_len, int noblock, int *errcode, int max_page_order); + +static inline struct sk_buff *sock_alloc_send_skb(struct sock *sk, + unsigned long size, + int noblock, int *errcode) +{ + return sock_alloc_send_pskb(sk, size, 0, noblock, errcode, 0); +} + void *sock_kmalloc(struct sock *sk, int size, gfp_t priority); void sock_kfree_s(struct sock *sk, void *mem, int size); void sock_kzfree_s(struct sock *sk, void *mem, int size); diff --git a/net/core/sock.c b/net/core/sock.c index b1a8f47fda55..77e37556e0c3 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2626,13 +2626,6 @@ struct sk_buff *sock_alloc_send_pskb(struct sock *sk, unsigned long header_len, } EXPORT_SYMBOL(sock_alloc_send_pskb); -struct sk_buff *sock_alloc_send_skb(struct sock *sk, unsigned long size, - int noblock, int *errcode) -{ - return sock_alloc_send_pskb(sk, size, 0, noblock, errcode, 0); -} -EXPORT_SYMBOL(sock_alloc_send_skb); - int __sock_cmsg_send(struct sock *sk, struct msghdr *msg, struct cmsghdr *cmsg, struct sockcm_cookie *sockc) { From patchwork Sun Apr 3 13:06:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799557 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 048B2C433FE for ; Sun, 3 Apr 2022 13:09:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358589AbiDCNLF (ORCPT ); Sun, 3 Apr 2022 09:11:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358536AbiDCNK2 (ORCPT ); Sun, 3 Apr 2022 09:10:28 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 525271029 for ; Sun, 3 Apr 2022 06:08:24 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id 123-20020a1c1981000000b0038b3616a71aso4074099wmz.4 for ; Sun, 03 Apr 2022 06:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b4iQGbstpUOSsgNXmdUB67X47VRarOvzLhVtfaDU4Wo=; b=l+m80QqAOYTCuIdmyreEEQ6q7ged2RbmlEsNYc4X0pumPjYe3KlzonzAEjkmzxh9l/ YW3eYunumE0mxPhHHhJ1n5zWmLKPeU0X/OnvnMNhcoalFlxiLSzwmGJUVx1jnydAIqx7 Z8cX4eZbKy0APRYjCPIU5L/SvvNOy9wbHS5Z+pEEI+L3Gm/wxVdF4ouvHonyI8y422CX Yb/AGdHbskXMJ6EIi0I1l9MpvKId56p6HZWPJtuYwCqDc1NOic/pY/OiaGpF3Kpc08Wj KqxSol7jcLfAOJtM7D5Jx9rl8LfgYmDOyI67nQoak27wNJpo2td+tLi6o5g+w6mPAsrs 4BJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b4iQGbstpUOSsgNXmdUB67X47VRarOvzLhVtfaDU4Wo=; b=uN1nXaAmxzKiAFhMfBWl/q1FuXkFuDr1R/zJA3gOOfg5xgGmDEt9t7sxL+dWXbdkGC FZtfPPDxscrceZFhdWjoZh1RMlocnTsh/pR+SWKmXgs5d4eu1kf8DR8eJX6uok5FzIXG rhqD2jB/DFxdQ+tgiPrajR3zmfXgiVtNVFlf4m8PHEjGb7pWDxTCVhvGOFODuDtD7KeB XKPQjww0nfUB2c27Voyacn2aC7m+2nZys77qM8Yvl6SFRh6gziWP8F7ihuS89941Mr4h ugreGMv45lAYD+4nfMXmppbP5ncyoiQdalkaWLgGS7NIUcn20vLaIQBXd/WpO4U3y37w XEqA== X-Gm-Message-State: AOAM532EM/44rEDfti9FKdjSkpVyOO7hUeHnVgnYUZqRMFhlMb6icClh qI9lZ/ikP5tFQEzOb9q9UxGGHyL1XB8= X-Google-Smtp-Source: ABdhPJxZczJk9E6frV2ldr1xnn1H/SaZHONMZE/fI4XArZ9Pyv4loDOyWRDR3/jqKxRGPZfyIkLVCQ== X-Received: by 2002:a05:600c:1ca7:b0:38e:27a6:5546 with SMTP id k39-20020a05600c1ca700b0038e27a65546mr15501660wms.188.1648991302573; Sun, 03 Apr 2022 06:08:22 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:22 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 10/27] net: inline part of skb_csum_hwoffload_help Date: Sun, 3 Apr 2022 14:06:22 +0100 Message-Id: <8ca1d5139e1820ac7e55166f164a56b2993d10e1.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Inline a part of skb_csum_hwoffload_help() responsible for skipping for HW-accelerated cases. Signed-off-by: Pavel Begunkov --- include/linux/netdevice.h | 13 ++++++++++--- net/core/dev.c | 11 ++++------- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index cd7a597c55b1..a4e41f7edc47 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -4699,9 +4699,16 @@ extern u8 netdev_rss_key[NETDEV_RSS_KEY_LEN] __read_mostly; void netdev_rss_key_fill(void *buffer, size_t len); int skb_checksum_help(struct sk_buff *skb); -int skb_crc32c_csum_help(struct sk_buff *skb); -int skb_csum_hwoffload_help(struct sk_buff *skb, - const netdev_features_t features); +int __skb_csum_hwoffload_help(struct sk_buff *skb, + const netdev_features_t features); + +static inline int skb_csum_hwoffload_help(struct sk_buff *skb, + const netdev_features_t features) +{ + if ((features & NETIF_F_HW_CSUM) && !skb_csum_is_sctp(skb)) + return 0; + return __skb_csum_hwoffload_help(skb, features); +} struct sk_buff *__skb_gso_segment(struct sk_buff *skb, netdev_features_t features, bool tx_path); diff --git a/net/core/dev.c b/net/core/dev.c index 4842a398f08d..6044b6124edc 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3233,7 +3233,7 @@ int skb_checksum_help(struct sk_buff *skb) } EXPORT_SYMBOL(skb_checksum_help); -int skb_crc32c_csum_help(struct sk_buff *skb) +static inline int skb_crc32c_csum_help(struct sk_buff *skb) { __le32 crc32c_csum; int ret = 0, offset, start; @@ -3572,16 +3572,13 @@ static struct sk_buff *validate_xmit_vlan(struct sk_buff *skb, return skb; } -int skb_csum_hwoffload_help(struct sk_buff *skb, - const netdev_features_t features) +int __skb_csum_hwoffload_help(struct sk_buff *skb, + const netdev_features_t features) { if (unlikely(skb_csum_is_sctp(skb))) return !!(features & NETIF_F_SCTP_CRC) ? 0 : skb_crc32c_csum_help(skb); - if (features & NETIF_F_HW_CSUM) - return 0; - if (features & (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM)) { switch (skb->csum_offset) { case offsetof(struct tcphdr, check): @@ -3592,7 +3589,7 @@ int skb_csum_hwoffload_help(struct sk_buff *skb, return skb_checksum_help(skb); } -EXPORT_SYMBOL(skb_csum_hwoffload_help); +EXPORT_SYMBOL(__skb_csum_hwoffload_help); static struct sk_buff *validate_xmit_skb(struct sk_buff *skb, struct net_device *dev, bool *again) { From patchwork Sun Apr 3 13:06:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799546 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB2B2C433EF for ; Sun, 3 Apr 2022 13:08:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358481AbiDCNKj (ORCPT ); Sun, 3 Apr 2022 09:10:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358587AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B43263B6 for ; Sun, 3 Apr 2022 06:08:25 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id h4so10584264wrc.13 for ; Sun, 03 Apr 2022 06:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X8Z6isJG5NFeUxeEjvkxdgE+kQKIsemrCR/6308ZWXw=; b=XOeMqe5q8E5eT6wxLBvYkQ1jFIn+UZEesnikCxzlIVF+XX/SUKWC7+XI8JOjTVG0wA E5IR2kqtBLNlLb1sr2sWpL1ipp5PwS0d4jV/K/Ke9/oJ3x22t21i26D9N5DSo9CbiMNX XsDljfeDaixss3zJX7/AAMhyHnSrStMFM5bXcUwCmVlEBY6l1aUdb0NfN4T74B2AJ7Qo KQh09O2/hdPt0F7bWXiE0RP/SLZlgTPCrxT09SUkk/NWRb6s6ROOciThAq4nwCsbwl+l /WqgClk8RyfyDDLKR6D4AX5vaBOHXU2mTuo2TyDLzG8lEmPlBEdhg3vCoxTISSmUtF0O 9HWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X8Z6isJG5NFeUxeEjvkxdgE+kQKIsemrCR/6308ZWXw=; b=Zqc6NdFj51F+VQYx/tT99jSovoYy0EauPCxh5vjr41yiaRE6TuHQ6KQzOJZj90o4Tk zcss4x6701dRJG2E/S3DjjrYTBI4opN943aeCL652+glZY0T/nSecu5sOtSeEaSh74Tv F3AylW/az+uhjLpnWWL5ittFNqDALGIEaE0SY4bbBZWKWk9wTuKi0ym4Tx3N9kgQlKAf vjsFozLfcE7iZUWH+fqW5pcUpeVtHRT7cztJS2e68y2wZY/YVK21k0uUt6g4xUjdAxGf eI+Q5VIIkjYWYTpoerwioTSN8kWNTpzG5lCpo5rm0gz7xFq1NejTEfA25tktwa2BWndn w5kg== X-Gm-Message-State: AOAM532KHpRRkXWkyBvMo1yFk7UZMXgMoD8SHiFKy6egadpLxFn7E0/f e9EiXEJvjF66+eYTPfEz2A2hKBCq6Jo= X-Google-Smtp-Source: ABdhPJxi5nzzCUeZzNns6kQOmLTC0sjPJvdC2nY/SX4hR8NornadvfThMqBuejxJ69zpRVJHkti6CQ== X-Received: by 2002:adf:916d:0:b0:206:db9:7ce9 with SMTP id j100-20020adf916d000000b002060db97ce9mr2122089wrj.556.1648991303655; Sun, 03 Apr 2022 06:08:23 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:23 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 11/27] net: inline skb_zerocopy_iter_dgram Date: Sun, 3 Apr 2022 14:06:23 +0100 Message-Id: <37734b05228dd2c03821a650b5568eb94a801528.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org skb_zerocopy_iter_dgram() is a small proxy function, inline it. For that, move __zerocopy_sg_from_iter into linux/skbuff.h Signed-off-by: Pavel Begunkov --- include/linux/skbuff.h | 36 ++++++++++++++++++++++-------------- net/core/datagram.c | 2 -- net/core/datagram.h | 15 --------------- net/core/skbuff.c | 7 ------- 4 files changed, 22 insertions(+), 38 deletions(-) delete mode 100644 net/core/datagram.h diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index ebc4ad36c3a2..93a50ac6b9c4 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -647,20 +647,6 @@ struct ubuf_info { int mm_account_pinned_pages(struct mmpin *mmp, size_t size); void mm_unaccount_pinned_pages(struct mmpin *mmp); -struct ubuf_info *msg_zerocopy_alloc(struct sock *sk, size_t size); -struct ubuf_info *msg_zerocopy_realloc(struct sock *sk, size_t size, - struct ubuf_info *uarg); - -void msg_zerocopy_put_abort(struct ubuf_info *uarg, bool have_uref); - -void msg_zerocopy_callback(struct sk_buff *skb, struct ubuf_info *uarg, - bool success); - -int skb_zerocopy_iter_dgram(struct sk_buff *skb, struct msghdr *msg, int len); -int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, - struct msghdr *msg, int len, - struct ubuf_info *uarg); - /* This data is invariant across clones and lives at * the end of the header data, ie. at skb->end. */ @@ -1670,6 +1656,28 @@ static inline void skb_set_end_offset(struct sk_buff *skb, unsigned int offset) } #endif +struct ubuf_info *msg_zerocopy_alloc(struct sock *sk, size_t size); +struct ubuf_info *msg_zerocopy_realloc(struct sock *sk, size_t size, + struct ubuf_info *uarg); + +void msg_zerocopy_put_abort(struct ubuf_info *uarg, bool have_uref); + +void msg_zerocopy_callback(struct sk_buff *skb, struct ubuf_info *uarg, + bool success); + +int __zerocopy_sg_from_iter(struct sock *sk, struct sk_buff *skb, + struct iov_iter *from, size_t length); + +static inline int skb_zerocopy_iter_dgram(struct sk_buff *skb, + struct msghdr *msg, int len) +{ + return __zerocopy_sg_from_iter(skb->sk, skb, &msg->msg_iter, len); +} + +int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, + struct msghdr *msg, int len, + struct ubuf_info *uarg); + /* Internal */ #define skb_shinfo(SKB) ((struct skb_shared_info *)(skb_end_pointer(SKB))) diff --git a/net/core/datagram.c b/net/core/datagram.c index ee290776c661..bd78b974baa5 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -62,8 +62,6 @@ #include #include -#include "datagram.h" - /* * Is a socket 'connection oriented' ? */ diff --git a/net/core/datagram.h b/net/core/datagram.h deleted file mode 100644 index bcfb75bfa3b2..000000000000 --- a/net/core/datagram.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ - -#ifndef _NET_CORE_DATAGRAM_H_ -#define _NET_CORE_DATAGRAM_H_ - -#include - -struct sock; -struct sk_buff; -struct iov_iter; - -int __zerocopy_sg_from_iter(struct sock *sk, struct sk_buff *skb, - struct iov_iter *from, size_t length); - -#endif /* _NET_CORE_DATAGRAM_H_ */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 2c787d964a60..65ac779eb5cd 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -80,7 +80,6 @@ #include #include -#include "datagram.h" #include "sock_destructor.h" struct kmem_cache *skbuff_head_cache __ro_after_init; @@ -1340,12 +1339,6 @@ void msg_zerocopy_put_abort(struct ubuf_info *uarg, bool have_uref) } EXPORT_SYMBOL_GPL(msg_zerocopy_put_abort); -int skb_zerocopy_iter_dgram(struct sk_buff *skb, struct msghdr *msg, int len) -{ - return __zerocopy_sg_from_iter(skb->sk, skb, &msg->msg_iter, len); -} -EXPORT_SYMBOL_GPL(skb_zerocopy_iter_dgram); - int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, struct msghdr *msg, int len, struct ubuf_info *uarg) From patchwork Sun Apr 3 13:06:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799556 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 390AEC433EF for ; Sun, 3 Apr 2022 13:09:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358637AbiDCNLD (ORCPT ); Sun, 3 Apr 2022 09:11:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358589AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D19EAE52 for ; Sun, 3 Apr 2022 06:08:26 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id c7so10754530wrd.0 for ; Sun, 03 Apr 2022 06:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZHbqEh9VJDFpVedQY3QtrqYMVk07AEtmjeXokOr+nLg=; b=hAa118a6tlSth1mxXI6QGB/r6rT9KjdqK06rnWMflTjmflpbXsLWNnI0vhOznT2bGw iB86XKJElc1bO3YiS+iCkayTIR1Rpl/ws7BUiZ+m6E9oSyx4MEfQVA0uk99RtaLOYT/R RHNiiPtdP2GvnftekVuT/NMdPygeG0Pv+DJZUXDaD5kIxYU8470YBi3AUIJhzmwp6oKF s5OYDEzEevKsTuZ/vn2ORVIRgjt1oixy3klsZBnHvGRRsb0kcePSAB7WMKV/fwXOGvdO iLXcOzkwQvb2hH+eb3CH9HnYSPJPKo0iuCSCkVvI+anzZGCE4W+7fLM7T7r4Kb+yiF0Q 4u7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZHbqEh9VJDFpVedQY3QtrqYMVk07AEtmjeXokOr+nLg=; b=AL2armbAVyzv38IynE+Kyp4au9U0bDwblJSs8EKimzVFUuWfAhHUrIL6q09c1lZPEB k+NIVtceoNwzGTpGae08SppDiTc80M2coQvDmU5s5ch5g2IXuIoDN2JH+MHDmmmK0PXz PqCGUrBDr250elfUcLljGbxAgONXZKtlvCk68lAxDyXJ55WmLfZD8j/50nd2JdclKIUn 0RJD2OG4GZWzZVcV96VTZrrCgXcm3SpHVmJyRJ0vumB/LyUbqzv6ZzN7Zd+LECqm3dKt HQgViHcPZwsdRJRJ+z41umH1t3IBpa3OGIRq49DfL2+Fr/8V7tp/DeojIAHaHEuXvrtR tRNg== X-Gm-Message-State: AOAM531e0cWoNQFrP1UfErhoZj2F8fCrVFEkJQmkImmDzcKlr4VDYFqg D81BSbge++hmzXJ8bqLzRqx7q5LubgE= X-Google-Smtp-Source: ABdhPJx12JZDaMqZD6265XqQvX6hlwY3lrAcAuOtaG8nksSfH2UM31AdSmdE5SNu5p+/PIADwwq6Vw== X-Received: by 2002:a05:6000:1687:b0:205:80b7:afca with SMTP id y7-20020a056000168700b0020580b7afcamr13199136wrd.665.1648991304712; Sun, 03 Apr 2022 06:08:24 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:24 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 12/27] ipv6: inline ip6_local_out() Date: Sun, 3 Apr 2022 14:06:24 +0100 Message-Id: <8f78232221e0f6e3ae136a183f0f11cb201d1063.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org ip6_local_out() is simple, inline it. Signed-off-by: Pavel Begunkov --- include/net/ipv6.h | 13 ++++++++++++- net/ipv6/output_core.c | 12 ------------ 2 files changed, 12 insertions(+), 13 deletions(-) diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 213612f1680c..0320bea599c9 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -1074,7 +1074,18 @@ void ip6_protocol_deliver_rcu(struct net *net, struct sk_buff *skb, int nexthdr, bool have_final); int __ip6_local_out(struct net *net, struct sock *sk, struct sk_buff *skb); -int ip6_local_out(struct net *net, struct sock *sk, struct sk_buff *skb); + +static inline int ip6_local_out(struct net *net, struct sock *sk, + struct sk_buff *skb) +{ + int err; + + err = __ip6_local_out(net, sk, skb); + if (likely(err == 1)) + err = dst_output(net, sk, skb); + + return err; +} /* * Extension header (options) processing diff --git a/net/ipv6/output_core.c b/net/ipv6/output_core.c index 2880dc7d9a49..f657e713561b 100644 --- a/net/ipv6/output_core.c +++ b/net/ipv6/output_core.c @@ -151,15 +151,3 @@ int __ip6_local_out(struct net *net, struct sock *sk, struct sk_buff *skb) dst_output); } EXPORT_SYMBOL_GPL(__ip6_local_out); - -int ip6_local_out(struct net *net, struct sock *sk, struct sk_buff *skb) -{ - int err; - - err = __ip6_local_out(net, sk, skb); - if (likely(err == 1)) - err = dst_output(net, sk, skb); - - return err; -} -EXPORT_SYMBOL_GPL(ip6_local_out); From patchwork Sun Apr 3 13:06:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799547 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E288C433FE for ; Sun, 3 Apr 2022 13:08:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358512AbiDCNKk (ORCPT ); Sun, 3 Apr 2022 09:10:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358421AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3D0D10FD7 for ; Sun, 3 Apr 2022 06:08:27 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id q19so3714063wrc.6 for ; Sun, 03 Apr 2022 06:08:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AtsblrA0Q/EAB3Ej64TC+StZTR9fs4Fmy33efi83Z1w=; b=P897ZHRZ42Kc/2lUY9RVPcWKaJeVIAp7YYv3U0oHfQo3U7EaHzoU0HRw/vWxVMDC8w rAvY4dFbWcBe67Gkfyd4/oRW8DBa+IKIRHzlEM0VThkebCaRpuu84yyDGjFFKgisyhz0 XpgKB2nOtNsKYEVtKM6i54LAY0eCyAMVdLGOms5p957NAE3IgcPbjr1gGUokRNsc4/Tk 6EWnAV1OU2tzX4iUeuhjTjh9Hg5jWgxUO4ikw2VVMfP1HJ9QkFLxMc3ov8VbaEZcC1Is xMF3taVLU3csWDLMXLpklCdnIDKC+6dmpfQX6nvx5hJX8kU5B8nrWvOV53f9mO9MjVWT mU9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AtsblrA0Q/EAB3Ej64TC+StZTR9fs4Fmy33efi83Z1w=; b=e7tz7YGqgYOtrRkrW6EdZhU4URzSjCCd7b1o1dEDbtiYaAT2Xc8H/zabOgixkLGgBP Zi8Ecg9FNS1pXjqUMDAn8yC2Nzv0Cd6FSZ6W7Bw8DoVxA+1QljTX7+IINyMSyPJDcywr boQoTPrbcJahLiLmDG/VyQ4QleJqLVTl2jMD/9t32CtOzGEbWtCxeZQPZ6gVJlwFQuh3 rMwOfc3P0V8hJ26NFs/n3ueH5Cg+q3/fCoucDr6rv3gTtkwZ3dUtIrfQ3I2xFWv+VTYh Jq2gLH5/nhoxko2mpLeIAIfdfzJXccg7hX09v4WKXwPZlcYEVUXzd2hOkDRmCHLF0m2F KDnA== X-Gm-Message-State: AOAM532Q8oCtcLaR64epj0t/ol9+oPVvLGsp3M/9V/ypxxu6teWsZ+sz vRM5merH9eSC94Zf14pSPzcf1SNfiiw= X-Google-Smtp-Source: ABdhPJykKr0h6gUD5OGv+7JbuG5Wf20yulnbZcXLVl4KTljwEcGay5yJ8e68xfgd1aEGMIucdW30qA== X-Received: by 2002:a5d:6945:0:b0:206:bd5:bf90 with SMTP id r5-20020a5d6945000000b002060bd5bf90mr3092425wrw.252.1648991305990; Sun, 03 Apr 2022 06:08:25 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:25 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 13/27] ipv6: help __ip6_finish_output() inlining Date: Sun, 3 Apr 2022 14:06:25 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org There are two callers of __ip6_finish_output(), both are in ip6_finish_output(). We can combine the call sites into one and handle return code after, that will inline __ip6_finish_output(). Note, error handling under NET_XMIT_CN will only return 0 if __ip6_finish_output() succeded, and in this case it return 0. Considering that NET_XMIT_SUCCESS is 0, it'll be returning exactly the same result for it as before. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 1 - 1 file changed, 1 deletion(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index f1ada6f2af7d..39f3e4bee9e6 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -198,7 +198,6 @@ static int ip6_finish_output(struct net *net, struct sock *sk, struct sk_buff *s ret = BPF_CGROUP_RUN_PROG_INET_EGRESS(sk, skb); switch (ret) { case NET_XMIT_SUCCESS: - return __ip6_finish_output(net, sk, skb); case NET_XMIT_CN: return __ip6_finish_output(net, sk, skb) ? : ret; default: From patchwork Sun Apr 3 13:06:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799555 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28EB4C433F5 for ; Sun, 3 Apr 2022 13:09:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358541AbiDCNK7 (ORCPT ); Sun, 3 Apr 2022 09:10:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358637AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4752140E7 for ; Sun, 3 Apr 2022 06:08:28 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id l62-20020a1c2541000000b0038e4570af2fso4081661wml.5 for ; Sun, 03 Apr 2022 06:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B1wqCJjycXccmpr0OYQdIB9liIfmIlJGjy/dtfC04bA=; b=F8D9fPrvJHsv70/isrhgUpVhKDrU2CZ1ttwfMfHxf1QPlvisbBcLgSAI/GrBp/+ZmD SjJqT3ZTAoALzh0/4lfmBwHohnDoeFOKZZhYbQ1/mBIBWN3brlYGZEVj54PpwsjA1Phu ym/FRdSHFSayM3txPHWCNZVLiwrx0lCbiQMWiYtfWt/w3GuQezWhW0htfG6K+WboyNLt TbEZGsX7vD4uidYDh69kkN9OGIkXp8hYVf8oLEpkMpIe0g/gNIojlqpmZPc3WLbehWv9 qKdW5tGldJwvCLpAzPccqgIDxvKt/wXf6SHTkFF8d15CseeU6PejaA0CPuev3Md7ebiN X3VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B1wqCJjycXccmpr0OYQdIB9liIfmIlJGjy/dtfC04bA=; b=Q67862myUWbsgIGdR8SKZUtr5/nYpEnu0AsLxnPykNFqQlprm6tL3Sd6oVwA1uFuAN 7FRbMj+6g4dSKFKay8P0i5a67wfUl6FuNx+byo2PqRM8au9D2AtDwQK+vjM2h6cS1Dig tWDigNOWxswG3uoQsHuXcJ1EFgMxzoPO06OYy3yr+fztghPnmy6o0caHUb65rB7qr/bK n1jhZ/GSytkShNe/inCTe7OESpEGeZ8EZjPrBkYuaZ77GVJRL2ln/SX/eYru9ABiRrZC mrrHdCQ084au7nUfmV8jxPSxVTcqENgtwdG40rWAja8/Id1GY5P2AmrMbzOVsTEqallc MZHg== X-Gm-Message-State: AOAM533VImurV+K8CvIh7o/PFiCmkqXnpgOu9sfCl7vn91gbYB4jzCXj 4wQLvZ/QA/QIoLp8WEwhivpvSZEd3J4= X-Google-Smtp-Source: ABdhPJxQtlvBKOj2wHtVmrJURAHX/271L5lridHtzhlh+bleyHmqf5qQ1jWgFI6RCh6qMWwYZghI2Q== X-Received: by 2002:a05:600c:35cc:b0:38c:73e8:7dd5 with SMTP id r12-20020a05600c35cc00b0038c73e87dd5mr15702465wmq.196.1648991307256; Sun, 03 Apr 2022 06:08:27 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:26 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 14/27] ipv6: refactor ip6_finish_output2() Date: Sun, 3 Apr 2022 14:06:26 +0100 Message-Id: <53e9e0d4c60a54b0c1070619e7104dbe32b5f937.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Throw neigh checks in ip6_finish_output2() under a single slow path if, so we don't have the overhead in the hot path. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 39f3e4bee9e6..4319364a4a8c 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -119,19 +119,21 @@ static int ip6_finish_output2(struct net *net, struct sock *sk, struct sk_buff * rcu_read_lock_bh(); nexthop = rt6_nexthop((struct rt6_info *)dst, daddr); neigh = __ipv6_neigh_lookup_noref(dev, nexthop); - if (unlikely(!neigh)) - neigh = __neigh_create(&nd_tbl, nexthop, dev, false); - if (!IS_ERR(neigh)) { - sock_confirm_neigh(skb, neigh); - ret = neigh_output(neigh, skb, false); - rcu_read_unlock_bh(); - return ret; + + if (unlikely(IS_ERR_OR_NULL(neigh))) { + if (unlikely(!neigh)) + neigh = __neigh_create(&nd_tbl, nexthop, dev, false); + if (IS_ERR(neigh)) { + rcu_read_unlock_bh(); + IP6_INC_STATS(net, idev, IPSTATS_MIB_OUTNOROUTES); + kfree_skb_reason(skb, SKB_DROP_REASON_NEIGH_CREATEFAIL); + return -EINVAL; + } } + sock_confirm_neigh(skb, neigh); + ret = neigh_output(neigh, skb, false); rcu_read_unlock_bh(); - - IP6_INC_STATS(net, idev, IPSTATS_MIB_OUTNOROUTES); - kfree_skb_reason(skb, SKB_DROP_REASON_NEIGH_CREATEFAIL); - return -EINVAL; + return ret; } static int From patchwork Sun Apr 3 13:06:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799554 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16F3BC433F5 for ; Sun, 3 Apr 2022 13:09:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358666AbiDCNK5 (ORCPT ); Sun, 3 Apr 2022 09:10:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358681AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0E71F68 for ; Sun, 3 Apr 2022 06:08:29 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id n63-20020a1c2742000000b0038d0c31db6eso4097198wmn.1 for ; Sun, 03 Apr 2022 06:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zMAqAkJjLHteJHC+8R6JZHKtOKmA+0ISlxrtx+knh+U=; b=Vl3eTujJ3Cm6qQ924Yg6Z7yFygWMi6uozeKs1qVM5XtTR4YLYSpSTHEQ+SsoW5Ig6r DjdBlonHCs93ohwFZ/7GibdEX6/O8FFMPiDfgu70TSr8CS7cEE42kENDQLfZdU3yIGDw WowNwk1okbPBIEWNEOvdBWCRiPvwRi8n40ykcHqjIfhyc0TxHBz6C3/HTKM5mBY0nJX9 0OlOgYLGKynkPSIuBM5enbzJKilgh6M9/u+ZGJ3lqoLCnps6ToK0ylgtXHVnDABwOZj5 WhYXCvntcQDH1CpMHSJIE+Sq80piDuwtBllm+uG+n1CqDANFXjw0lIrNulb0K2T8Smot pHqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zMAqAkJjLHteJHC+8R6JZHKtOKmA+0ISlxrtx+knh+U=; b=yP5IzDKvdRPRGEizb9MTNciqSN6WUT476Hqgw8yBH1gvnTql21gp1uPRP+A+au9evb gltmkdnwQDPRByvoq506ZUiKITk4unK905RZZXZOrNNDdUPhb6cVAYjmPx+zmeWAStWq KuyedqsiqjLh2+BUGTB8fd8EmeEbXDlpU+HGR9YoaRPvGelFjNRwvscw3JhOgjKh6ULv Ss9GKduL1l9aEHcmPDOcHf096CG89Sxl1aTSetvijam+cLVr3oymaxORCWFq65KDVVYc 2oAhgvxdRrdKQ8UfgYLQV7a5TwbzaccWQhXYZ7FAZWgT4Bkv1jJbcVAPpea8HUo3glRJ 0+bw== X-Gm-Message-State: AOAM530GFDWUNDabYE9zu63sVutwTBpRxDNYB7tyZnSmFbg7wVZbXbgk WDjsqWMy6dxO2e/ar3LvAlS2YN7fTvA= X-Google-Smtp-Source: ABdhPJyAHQu8S8Yp9iV/YArIKcWkrDisznt/wYpcIAK0W7tDUV07QgZJKSu4/0ioKMtBuLgu73yyfg== X-Received: by 2002:a7b:ce8e:0:b0:38e:70fb:9778 with SMTP id q14-20020a7bce8e000000b0038e70fb9778mr844746wmj.65.1648991308247; Sun, 03 Apr 2022 06:08:28 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:28 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 15/27] net: inline dev_queue_xmit() Date: Sun, 3 Apr 2022 14:06:27 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Inline dev_queue_xmit() and dev_queue_xmit_accel(), they both are small proxy functions doing nothing but redirecting the control flow to __dev_queue_xmit(). Signed-off-by: Pavel Begunkov --- include/linux/netdevice.h | 14 ++++++++++++-- net/core/dev.c | 15 ++------------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index a4e41f7edc47..6aca1f3b21ff 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2932,10 +2932,20 @@ u16 dev_pick_tx_zero(struct net_device *dev, struct sk_buff *skb, u16 dev_pick_tx_cpu_id(struct net_device *dev, struct sk_buff *skb, struct net_device *sb_dev); -int dev_queue_xmit(struct sk_buff *skb); -int dev_queue_xmit_accel(struct sk_buff *skb, struct net_device *sb_dev); +int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev); int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id); +static inline int dev_queue_xmit(struct sk_buff *skb) +{ + return __dev_queue_xmit(skb, NULL); +} + +static inline int dev_queue_xmit_accel(struct sk_buff *skb, + struct net_device *sb_dev) +{ + return __dev_queue_xmit(skb, sb_dev); +} + static inline int dev_direct_xmit(struct sk_buff *skb, u16 queue_id) { int ret; diff --git a/net/core/dev.c b/net/core/dev.c index 6044b6124edc..ed5459552117 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4084,7 +4084,7 @@ struct netdev_queue *netdev_core_pick_tx(struct net_device *dev, * the BH enable code must have IRQs enabled so that it will not deadlock. * --BLG */ -static int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) +int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) { struct net_device *dev = skb->dev; struct netdev_queue *txq; @@ -4200,18 +4200,7 @@ static int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) rcu_read_unlock_bh(); return rc; } - -int dev_queue_xmit(struct sk_buff *skb) -{ - return __dev_queue_xmit(skb, NULL); -} -EXPORT_SYMBOL(dev_queue_xmit); - -int dev_queue_xmit_accel(struct sk_buff *skb, struct net_device *sb_dev) -{ - return __dev_queue_xmit(skb, sb_dev); -} -EXPORT_SYMBOL(dev_queue_xmit_accel); +EXPORT_SYMBOL(__dev_queue_xmit); int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id) { From patchwork Sun Apr 3 13:06:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799553 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1F48C433F5 for ; Sun, 3 Apr 2022 13:09:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358647AbiDCNKz (ORCPT ); Sun, 3 Apr 2022 09:10:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358685AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D686D1158 for ; Sun, 3 Apr 2022 06:08:30 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id c7so10754674wrd.0 for ; Sun, 03 Apr 2022 06:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CFbFT18LoVBpsWkCx5QJjbFpMIJZnGfm91ftNPGZ6o8=; b=lZG5o7+jtayx39U76t/VSPCW2evRVlkyt6L0YK580aRwadkRD9iHF/hqIk1m/XiMdu HjJZANuV1wnaeV/wF6fYryacodWUOCH1aVPXDSNYN049A6O+QTsUK5FlWAMmdqwHaLt3 xYlbI97dfnKvaPl7r8opP8VYjgUwoqfcFuPprKpZjCn8dB6Rc/oZopeBqL2VCjd7EvDN XQ6zkKCO/pxpfn8+9vkx4xvSK0Hvm31wMmChAGWnIzu4OSgPUOdyOJdcQkLH9Yt2SnBs y+zdTa1hUIua2uEPgnOidOwpC4mzOXXWZMFVQoRMTrnF6ZkY5P4v5pS9xxZ5OL6TYED2 RqUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CFbFT18LoVBpsWkCx5QJjbFpMIJZnGfm91ftNPGZ6o8=; b=YdTKCKrGU01G9UbuvvnBX/oTgMAERy7i9oronM1M8FoQL/K/V2YUVo6ylKFSBE/Jj+ 31XoksKEOXeN8F8od7DMX+qdGkqwCBgp41OBHMPGHE40lmQI72uiMuXOXSkdkBUkh5e2 r3EvbThtAFZ/UGuoTTDydy6Kwk/4sApDXmgjwJnDs0rx2MNCxzEyHxwL3e2YOUFiTtr0 cG7aZ3YRlgSamProOzpdGrsEPV7E3Hd7YDRRkP8wbiNNVoCgjsv74c2t2S9yn4iQhwsH bMGpdItcdNo0/AKQ1G9FPuxmMMB4ZtEsETeoErx2Y2LMnEp6QWFRgH6eaMCM7KndzI3K bkIA== X-Gm-Message-State: AOAM532eQYmf2UcX4yyZcG0NZikQlcL5TFtEsj6gb8GmLwxqQ7Rd+JQa JpRrjKdDFmAKiIxodPZFN9lsTT4CzJM= X-Google-Smtp-Source: ABdhPJz3y+nl2Un9KxPO9j5qe7nBtWSnUfiUwlS/d2lhmPRstm4Y793J0ROCFVlFuex2e5rIOnNFbg== X-Received: by 2002:a05:6000:1acb:b0:204:1ce4:7c2a with SMTP id i11-20020a0560001acb00b002041ce47c2amr13612368wry.234.1648991309351; Sun, 03 Apr 2022 06:08:29 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:29 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 16/27] ipv6: partially inline fl6_update_dst() Date: Sun, 3 Apr 2022 14:06:28 +0100 Message-Id: <1c35d43fdee5ae1de64b1bb1acea48a40eb9a6ba.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org fl6_update_dst() doesn't do anything when there are no opts passed. Inline the null checking part. Signed-off-by: Pavel Begunkov --- include/net/ipv6.h | 15 ++++++++++++--- net/ipv6/exthdrs.c | 15 ++++++--------- 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 0320bea599c9..48a25f663646 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -1114,9 +1114,18 @@ int ipv6_find_hdr(const struct sk_buff *skb, unsigned int *offset, int target, int ipv6_find_tlv(const struct sk_buff *skb, int offset, int type); -struct in6_addr *fl6_update_dst(struct flowi6 *fl6, - const struct ipv6_txoptions *opt, - struct in6_addr *orig); +struct in6_addr *__fl6_update_dst(struct flowi6 *fl6, + const struct ipv6_txoptions *opt, + struct in6_addr *orig); + +static inline struct in6_addr *fl6_update_dst(struct flowi6 *fl6, + const struct ipv6_txoptions *opt, + struct in6_addr *orig) +{ + if (!opt || !opt->srcrt) + return NULL; + return __fl6_update_dst(fl6, opt, orig); +} /* * socket options (ipv6_sockglue.c) diff --git a/net/ipv6/exthdrs.c b/net/ipv6/exthdrs.c index 658d5eabaf7e..0b37b11cd2a9 100644 --- a/net/ipv6/exthdrs.c +++ b/net/ipv6/exthdrs.c @@ -1365,8 +1365,8 @@ struct ipv6_txoptions *__ipv6_fixup_options(struct ipv6_txoptions *opt_space, EXPORT_SYMBOL_GPL(__ipv6_fixup_options); /** - * fl6_update_dst - update flowi destination address with info given - * by srcrt option, if any. + * __fl6_update_dst - update flowi destination address with info given + * by srcrt option. * * @fl6: flowi6 for which daddr is to be updated * @opt: struct ipv6_txoptions in which to look for srcrt opt @@ -1375,13 +1375,10 @@ EXPORT_SYMBOL_GPL(__ipv6_fixup_options); * Returns NULL if no txoptions or no srcrt, otherwise returns orig * and initial value of fl6->daddr set in orig */ -struct in6_addr *fl6_update_dst(struct flowi6 *fl6, - const struct ipv6_txoptions *opt, - struct in6_addr *orig) +struct in6_addr *__fl6_update_dst(struct flowi6 *fl6, + const struct ipv6_txoptions *opt, + struct in6_addr *orig) { - if (!opt || !opt->srcrt) - return NULL; - *orig = fl6->daddr; switch (opt->srcrt->type) { @@ -1403,4 +1400,4 @@ struct in6_addr *fl6_update_dst(struct flowi6 *fl6, return orig; } -EXPORT_SYMBOL_GPL(fl6_update_dst); +EXPORT_SYMBOL_GPL(__fl6_update_dst); From patchwork Sun Apr 3 13:06:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799548 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B851EC433F5 for ; Sun, 3 Apr 2022 13:08:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358517AbiDCNKm (ORCPT ); Sun, 3 Apr 2022 09:10:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358720AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECB8B20187 for ; Sun, 3 Apr 2022 06:08:31 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id d29so4585029wra.10 for ; Sun, 03 Apr 2022 06:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1AwOsomC8j1IsP+7LcAos7c47K6Vxss+ztTv0W2ukEs=; b=LEJfInosH44bRW+Psi+gmdL8Kuypuqiv7y9sjofNwnTfwUD43P3DB5PAHgA2iTYVtY 9kmsMSYhwY3bV9XQDyT5HAi4sw1cdPUnMoAx9O51h5+x9e39ILSpZXo81mPfWQfTB/no rBxgVU4EdPqYtoGGcJzC3w51EEO6z86vFZ+7ARWw2i7001uhv4yz1NG2/t9qXX5+oxeE m2n3AO1z6uNe7HvROj57sLTeEIFxK2THnLxxgQIESRbQEz5J08qR9WeZkBK6UL35rDf6 rN/NmAWAhHUuePMY7MyOgbtvIXEfAFENmR8cyAyB6aQoax7bh/7R9vARn1ewLVWYt75I b9EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1AwOsomC8j1IsP+7LcAos7c47K6Vxss+ztTv0W2ukEs=; b=yXUswIy6arzEQxoC7suN+yxaUzYXjx74hjBLUke4PnSzw3+7dNjAGJQgdu29MOTTY4 ZkGxQd2H9pnHz0Wq1Kw+KuDuLeQ2XQvlmpNPM88FiXZfUdjIaDW1BP5cF+Fc845B2+PB BxIUWhbHqNW7CAmk43tmiDCrI77ho+B+kFv9mXAV8L27AgOFGQRh37h2xSaa88uwegYn xGJOXsZwsC2rb8AVzjjMM0CwpA88Z+IES44jByiB9np3EagY6u/PaAh1aRI+s60DlTj1 zPaZL3GV65VvR2F3legkJwzVusz0GHVjihdtf6+OCZNvkrw4uMG44ggPNlmX2xiEYsnH VvTA== X-Gm-Message-State: AOAM533MHCp8csChHoG1eEq1VLbHET0yekX/yd95osmfmYd0U5BbuxtT OZzT+fr+mCuOZ5LcmOXM9WelnIFXCIQ= X-Google-Smtp-Source: ABdhPJxRX29YlrO0ForaTa48S5g6oGm4F6yQuWQCjytBMPZ4UXlX7Go980LuDFk78v79L2CQu/WUeA== X-Received: by 2002:a5d:6c66:0:b0:204:1175:691c with SMTP id r6-20020a5d6c66000000b002041175691cmr14213620wrz.328.1648991310427; Sun, 03 Apr 2022 06:08:30 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:30 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 17/27] tcp: optimise skb_zerocopy_iter_stream() Date: Sun, 3 Apr 2022 14:06:29 +0100 Message-Id: <0c14446e83b09759c08d9ab12b747e80508de74b.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org It's expensive to make a copy of 40B struct iov_iter to the point it was taking 0.2-0.5% of all cycles in my tests. iov_iter_revert() should be fine as it's a simple case without nested reverts/truncates. Signed-off-by: Pavel Begunkov --- net/core/skbuff.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 65ac779eb5cd..77cbdb02e885 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -1343,7 +1343,6 @@ int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, struct msghdr *msg, int len, struct ubuf_info *uarg) { - struct iov_iter orig_iter = msg->msg_iter; int err, orig_len = skb->len; /* An skb can only point to one uarg. This edge case happens when @@ -1357,7 +1356,7 @@ int skb_zerocopy_iter_stream(struct sock *sk, struct sk_buff *skb, struct sock *save_sk = skb->sk; /* Streams do not free skb on error. Reset to prev state. */ - msg->msg_iter = orig_iter; + iov_iter_revert(&msg->msg_iter, skb->len - orig_len); skb->sk = sk; ___pskb_trim(skb, orig_len); skb->sk = save_sk; From patchwork Sun Apr 3 13:06:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799552 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC31BC433EF for ; Sun, 3 Apr 2022 13:09:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358587AbiDCNKx (ORCPT ); Sun, 3 Apr 2022 09:10:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358727AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45E2F24F1D for ; Sun, 3 Apr 2022 06:08:33 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id u16so10666216wru.4 for ; Sun, 03 Apr 2022 06:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HTOkaQJQEwGkCyutV8MTQiPLjfsdfmVwwswGO/KW+7M=; b=D96me3xUOgLSErFmTBzGLsVo4dugLrvAGWI50PgZhJOZFNolF8SfrkGGkwSKjhe4gY SBIwN9mqiI1zAKiV5YioAJDXkjjWcdZI9XD6JAsadveT2D6E2nTQNet3RLSV2HBhGN+F 8ELsSZFdXqM/8/vtJAaZgc7norUPBfTEl5RhMkcnPXu8jDuyEhF/Oz1mqDR6A7JJMHtn 8vZ9YpsF+s+8WArj3VGg5kb7xI9F1p6G7LOEF1ZY3kBsdLAxAVAYsiG8HTPrRZSCCzFV 4e6wLnywuuLCP6vZQwRuLBORQ9RBb7m3zE54odWtZOYTftPpuSQDrZuKt9dFySkqQnu7 bDqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HTOkaQJQEwGkCyutV8MTQiPLjfsdfmVwwswGO/KW+7M=; b=3NuZfNy5HpvHdwf0yTe5ByHlO/VUdLIHvZYXgHYTYPkFxDJkEy6zqBpO1k6THtj51m XGCVMsD0ig40niF7UdkXdiNWPZXipuJYf/7XOLisJVzL6vo19OqKgquqVLQXmXvjf+U/ XNgLcwF+GbFVTgJaQdu7IAOYsuN49WI6jIGmUiat4aXKlc3DZ47N5bHWdTVojME0tEtF +OfeYRI/HnMfBR4AC1z7Q0hu9vNGF0NTtgX+MtZT8hhThsO07XCyRAPhzUUq6RPtPn6Z 69eyVAagqztbyvR8AVO9C4JJcM6H6C0mfuE9M+fVe3CDKPY2juf58R3cQLN3LlpgEzgG s8Ng== X-Gm-Message-State: AOAM531lLCdha59OiK1tqgMxEW90swX56rhLWa5lOX1FAesno+OXRcG2 TVJGtfXzVcFRUdyreRmXcws+r3RR3/8= X-Google-Smtp-Source: ABdhPJzzpc4WYTcmFDYxmrQACLwJ5AQhOAMi0wEfP9RR5YgURAgyzBETFT9S2OyMtoA5EPQgi7zfWg== X-Received: by 2002:adf:eb81:0:b0:1e3:2bf5:132 with SMTP id t1-20020adfeb81000000b001e32bf50132mr13800044wrn.246.1648991311572; Sun, 03 Apr 2022 06:08:31 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:31 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 18/27] net: optimise ipcm6 cookie init Date: Sun, 3 Apr 2022 14:06:30 +0100 Message-Id: <15066b7067614521d47e679c7cadac044d70bd30.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Users of ipcm6_init() have a somewhat complex post initialisation of ->dontfrag and ->tclass. Not only it adds additional overhead, but also complicates the code. First, replace ipcm6_init() with ipcm6_init_sk(). As it might be not an equivalent change, let's first look at ->dontfrag. The logic was to set it from cmsg if specified and otherwise fallback to np->dontfrag. Now it's initialising to np->dontfrag in the beginning and then potentially overriding with cmsg, which is absolutely the same behaviour. It's a bit more complex with ->tclass as ip6_datagram_send_ctl() might set it to -1, which is a default and not valid value. The solution here is to skip -1's specified in cmsg, so it'll be left with the socket default value getting us to the old behaviour. Signed-off-by: Pavel Begunkov --- include/net/ipv6.h | 9 --------- net/ipv6/datagram.c | 4 ++-- net/ipv6/ip6_output.c | 2 -- net/ipv6/raw.c | 8 +------- net/ipv6/udp.c | 7 +------ net/l2tp/l2tp_ip6.c | 8 +------- 6 files changed, 5 insertions(+), 33 deletions(-) diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 48a25f663646..2f2d9af58f05 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -352,15 +352,6 @@ struct ipcm6_cookie { struct ipv6_txoptions *opt; }; -static inline void ipcm6_init(struct ipcm6_cookie *ipc6) -{ - *ipc6 = (struct ipcm6_cookie) { - .hlimit = -1, - .tclass = -1, - .dontfrag = -1, - }; -} - static inline void ipcm6_init_sk(struct ipcm6_cookie *ipc6, const struct ipv6_pinfo *np) { diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c index 206f66310a88..1b334bc855ae 100644 --- a/net/ipv6/datagram.c +++ b/net/ipv6/datagram.c @@ -1003,9 +1003,9 @@ int ip6_datagram_send_ctl(struct net *net, struct sock *sk, if (tc < -1 || tc > 0xff) goto exit_f; + if (tc != -1) + ipc6->tclass = tc; err = 0; - ipc6->tclass = tc; - break; } diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 4319364a4a8c..bd5de7a5aa8c 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -2003,8 +2003,6 @@ struct sk_buff *ip6_make_skb(struct sock *sk, ip6_cork_release(cork, &v6_cork); return ERR_PTR(err); } - if (ipc6->dontfrag < 0) - ipc6->dontfrag = inet6_sk(sk)->dontfrag; err = __ip6_append_data(sk, &queue, cork, &v6_cork, ¤t->task_frag, getfrag, from, diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index c51d5ce3711c..0e0156938968 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -808,7 +808,7 @@ static int rawv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) fl6.flowi6_mark = sk->sk_mark; fl6.flowi6_uid = sk->sk_uid; - ipcm6_init(&ipc6); + ipcm6_init_sk(&ipc6, np); ipc6.sockc.tsflags = sk->sk_tsflags; ipc6.sockc.mark = sk->sk_mark; @@ -920,9 +920,6 @@ static int rawv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (hdrincl) fl6.flowi6_flags |= FLOWI_FLAG_KNOWN_NH; - if (ipc6.tclass < 0) - ipc6.tclass = np->tclass; - fl6.flowlabel = ip6_make_flowinfo(ipc6.tclass, fl6.flowlabel); dst = ip6_dst_lookup_flow(sock_net(sk), sk, &fl6, final_p); @@ -933,9 +930,6 @@ static int rawv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (ipc6.hlimit < 0) ipc6.hlimit = ip6_sk_dst_hoplimit(np, &fl6, dst); - if (ipc6.dontfrag < 0) - ipc6.dontfrag = np->dontfrag; - if (msg->msg_flags&MSG_CONFIRM) goto do_confirm; diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 7f0fa9bd9ffe..4b15b37fc8f9 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1313,7 +1313,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) int is_udplite = IS_UDPLITE(sk); int (*getfrag)(void *, char *, int, int, int, struct sk_buff *); - ipcm6_init(&ipc6); + ipcm6_init_sk(&ipc6, np); ipc6.gso_size = READ_ONCE(up->gso_size); ipc6.sockc.tsflags = sk->sk_tsflags; ipc6.sockc.mark = sk->sk_mark; @@ -1518,9 +1518,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) security_sk_classify_flow(sk, flowi6_to_flowi_common(fl6)); - if (ipc6.tclass < 0) - ipc6.tclass = np->tclass; - fl6->flowlabel = ip6_make_flowinfo(ipc6.tclass, fl6->flowlabel); dst = ip6_sk_dst_lookup_flow(sk, fl6, final_p, connected); @@ -1566,8 +1563,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) up->pending = AF_INET6; do_append_data: - if (ipc6.dontfrag < 0) - ipc6.dontfrag = np->dontfrag; up->len += ulen; err = ip6_append_data(sk, getfrag, msg, ulen, sizeof(struct udphdr), &ipc6, fl6, (struct rt6_info *)dst, diff --git a/net/l2tp/l2tp_ip6.c b/net/l2tp/l2tp_ip6.c index 96f975777438..4459926f5840 100644 --- a/net/l2tp/l2tp_ip6.c +++ b/net/l2tp/l2tp_ip6.c @@ -521,7 +521,7 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) fl6.flowi6_mark = sk->sk_mark; fl6.flowi6_uid = sk->sk_uid; - ipcm6_init(&ipc6); + ipcm6_init_sk(&ipc6, np); if (lsa) { if (addr_len < SIN6_LEN_RFC2133) @@ -608,9 +608,6 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) security_sk_classify_flow(sk, flowi6_to_flowi_common(&fl6)); - if (ipc6.tclass < 0) - ipc6.tclass = np->tclass; - fl6.flowlabel = ip6_make_flowinfo(ipc6.tclass, fl6.flowlabel); dst = ip6_dst_lookup_flow(sock_net(sk), sk, &fl6, final_p); @@ -622,9 +619,6 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (ipc6.hlimit < 0) ipc6.hlimit = ip6_sk_dst_hoplimit(np, &fl6, dst); - if (ipc6.dontfrag < 0) - ipc6.dontfrag = np->dontfrag; - if (msg->msg_flags & MSG_CONFIRM) goto do_confirm; From patchwork Sun Apr 3 13:06:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799550 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4F69C433EF for ; Sun, 3 Apr 2022 13:09:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358515AbiDCNKt (ORCPT ); Sun, 3 Apr 2022 09:10:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358552AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B81B217A8D for ; Sun, 3 Apr 2022 06:08:34 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id w4so10596416wrg.12 for ; Sun, 03 Apr 2022 06:08:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gQROuweWM/sccPHitCdr191lrVLAeR8d9SPDAu73ZAg=; b=pyoE93zTijQuGGdL+1oQIdwkjWOaN0xSooVn7JV+eEsMbLCpfzRhF/5E+u4J/yRS6K EHWA9taZyO0+ZA0uc0yyn9pb0uy5X2XlIr/L/E74e05kFX4eim5F6p/9RCfGuQ2p39dU P29vwVFOqaI+qbwBKT4BLL23yYsszGC0MRABA6B1xjUTAKsjR3kiyqZIaeDAXwLWm1Q9 ry4blh8Tlnw0DYX0OCB6XwJs3PGG9xA8exq+lmJfgPK+ikq3/tk56+NKEEzyEl6NEM3d P4oTt4UucejAIeUnDvrwAHSY9FkZz7Su7zb7VGzaAxNsffcFJQEVzJVYzBulMcUvvJ4F 3sdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gQROuweWM/sccPHitCdr191lrVLAeR8d9SPDAu73ZAg=; b=ozKAyCeUJ3s3hbkWxMOrIpbO1ZbEUFN73SsWYQUxscZa9oASODVuZX/BqsJeKn9AUG hoiQFUaX0dao/E8q6owRuTBKav9oMme2+ZgCsF2IDvxVQF2T+0sTI1g9F1cV7geqjAju yqlVZ5sYu4JtJNVBfO53KMUFNY/tLByM3o8ibHv6YavwXS/CruIzbxxNJuLuFAMM7EiR W+UEGxKsCjtKrlxkWTUcukb/OJXsyQewcv38D6af3QY3VYqiKFVI6k2hjUzR+s8NEAsy mNMhvcs47ST3eaWnwOdnI5WS3VxETakojXGyncinZA7DPlmRgH73EjeI2miCjmdn+rI4 OOMQ== X-Gm-Message-State: AOAM532DHkdbMF0aHFlnaA5m78aLzn3dtZnfjWXLNd5nixtvBXSols3z zPSLbpErQ/R1s7NSeOTzzcNrlWsKEB0= X-Google-Smtp-Source: ABdhPJxm1FlYXT68QjSdTZIe4HVfMLFopV6TPF7TaBDUTGuCczRYIa3zqmeLFQQMTMX4eR2YWtIvXw== X-Received: by 2002:a5d:54ce:0:b0:205:133d:c152 with SMTP id x14-20020a5d54ce000000b00205133dc152mr13820207wrv.334.1648991312548; Sun, 03 Apr 2022 06:08:32 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:32 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 19/27] udp/ipv6: refactor udpv6_sendmsg udplite checks Date: Sun, 3 Apr 2022 14:06:31 +0100 Message-Id: <06d86a707139f7a0127952bb6dae977cad94e573.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Don't save a IS_UDPLITE() result in advance but do when it's really needed, so it doesn't store/load it from the stack. Same for resolving the getfrag callback pointer. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 4b15b37fc8f9..588bd7e3ebc1 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1310,7 +1310,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) int ulen = len; int corkreq = READ_ONCE(up->corkflag) || msg->msg_flags&MSG_MORE; int err; - int is_udplite = IS_UDPLITE(sk); int (*getfrag)(void *, char *, int, int, int, struct sk_buff *); ipcm6_init_sk(&ipc6, np); @@ -1371,7 +1370,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (len > INT_MAX - sizeof(struct udphdr)) return -EMSGSIZE; - getfrag = is_udplite ? udplite_getfrag : ip_generic_getfrag; if (up->pending) { if (up->pending == AF_INET) return udp_sendmsg(sk, msg, len); @@ -1538,6 +1536,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) if (!corkreq) { struct sk_buff *skb; + getfrag = IS_UDPLITE(sk) ? udplite_getfrag : ip_generic_getfrag; skb = ip6_make_skb(sk, getfrag, msg, ulen, sizeof(struct udphdr), &ipc6, (struct rt6_info *)dst, @@ -1564,6 +1563,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) do_append_data: up->len += ulen; + getfrag = IS_UDPLITE(sk) ? udplite_getfrag : ip_generic_getfrag; err = ip6_append_data(sk, getfrag, msg, ulen, sizeof(struct udphdr), &ipc6, fl6, (struct rt6_info *)dst, corkreq ? msg->msg_flags|MSG_MORE : msg->msg_flags); @@ -1594,7 +1594,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) */ if (err == -ENOBUFS || test_bit(SOCK_NOSPACE, &sk->sk_socket->flags)) { UDP6_INC_STATS(sock_net(sk), - UDP_MIB_SNDBUFERRORS, is_udplite); + UDP_MIB_SNDBUFERRORS, IS_UDPLITE(sk)); } return err; From patchwork Sun Apr 3 13:06:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799549 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D680C433EF for ; Sun, 3 Apr 2022 13:08:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358592AbiDCNKq (ORCPT ); Sun, 3 Apr 2022 09:10:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358765AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69A6624596 for ; Sun, 3 Apr 2022 06:08:35 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id w4so10596458wrg.12 for ; Sun, 03 Apr 2022 06:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=va9zZWjDTCI1oJw3Kv1ZIgWs6fFF0wfjfDRP7pFWDs0=; b=UgV09UQozqN4AXwtXAhtfOvEF6cezzc3WCU/rr7kxaiO9l8rlULljUb1t9Vzeh910l UaZgb9A0cLXa8b93gX0QoX++QzqxxNDfIzh5stw6AwVBhAxo90sKD5ma0XN6Q9tyovRF F87KSWP8QZOnp4hfAO/0AaKEqlzppPPbV5cGZ3AsgZTRFgokoLIirzjK2jtDdVArEj4k OVDT0hTWjGjgOWGStIZApUc4yVywYOi3cjqIU2UdsotnbIAtLE8lH9mhgsrBP8c3O8gS lXLjIwIYtIZM3fguYs3qSz6AFtDrZQ4xtMOojw37MU9nnR4b95ZYA/Rzg8IPfNd0UsyL uMnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=va9zZWjDTCI1oJw3Kv1ZIgWs6fFF0wfjfDRP7pFWDs0=; b=YliJhX2vs6crVDh4Vjsi8qr7pgHbF87mw9w/XvDaxCtnxSfsDiFRITdxHmPc+BRhfA ObBC2FrgogMg+wU4tjK4MhHxjU4pUjsu34cE5bZH70QHDm4TFaiecELh6j5p51WLNbCJ DZMuISIgdkrG4tkmBVulU9B9gESHgbxqezxkxkL0stBs9zdjZePAyBmc4eKEo5bQbOI9 gHYzgNT2zcX6Lr3JU9t9kVosiY21jBDlATXvd5vT15XNcNhJE0BOKo/Flbht9231kMKc pfbIQ5Qn2QhuDSORrJ3+eNtSRqOp4FYZ9vJjpnoGH9Eu5YGKEMKhF0HMW2vbFAACILtX JW4Q== X-Gm-Message-State: AOAM532odVPzOT70P2J0OrzREZePZ8UfceC0Fa2wfd4jXgAKYP4hSRSt tC8z63TsnkdTzcTmIWRn+I+/ijGIvAw= X-Google-Smtp-Source: ABdhPJyC8rIPCbJwhflo1WGJqkVtqx4xECfvBeMnmTQogkqTmQ8lp4bimfEfYOgj6eiX1NA/9O+mxg== X-Received: by 2002:a05:6000:186d:b0:204:110a:d832 with SMTP id d13-20020a056000186d00b00204110ad832mr13936228wri.47.1648991313746; Sun, 03 Apr 2022 06:08:33 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:33 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 20/27] udp/ipv6: move pending section of udpv6_sendmsg Date: Sun, 3 Apr 2022 14:06:32 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Move up->pending section of udpv6_sendmsg() to the beginning of the function. Even though it require some code duplication for sin6 parsing, it clearly localises the pending handling in one place, removes an extra if and more importantly will prepare the code for further patches. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 67 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 588bd7e3ebc1..26832be40f31 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1317,6 +1317,44 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) ipc6.sockc.tsflags = sk->sk_tsflags; ipc6.sockc.mark = sk->sk_mark; + /* Rough check on arithmetic overflow, + better check is made in ip6_append_data(). + */ + if (unlikely(len > INT_MAX - sizeof(struct udphdr))) + return -EMSGSIZE; + + /* There are pending frames. */ + if (up->pending) { + if (up->pending == AF_INET) + return udp_sendmsg(sk, msg, len); + + /* Do a quick destination sanity check before corking. */ + if (sin6) { + if (msg->msg_namelen < offsetof(struct sockaddr, sa_data)) + return -EINVAL; + if (sin6->sin6_family == AF_INET6) { + if (msg->msg_namelen < SIN6_LEN_RFC2133) + return -EINVAL; + if (ipv6_addr_any(&sin6->sin6_addr) && + ipv6_addr_v4mapped(&np->saddr)) + return -EINVAL; + } else if (sin6->sin6_family != AF_UNSPEC) { + return -EINVAL; + } + } + + /* The socket lock must be held while it's corked. */ + lock_sock(sk); + if (unlikely(up->pending != AF_INET6)) { + /* Just now it was seen corked, userspace is buggy */ + err = up->pending ? -EAFNOSUPPORT : -EINVAL; + release_sock(sk); + return err; + } + dst = NULL; + goto do_append_data; + } + /* destination address check */ if (sin6) { if (addr_len < offsetof(struct sockaddr, sa_data)) @@ -1342,12 +1380,11 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) default: return -EINVAL; } - } else if (!up->pending) { + } else { if (sk->sk_state != TCP_ESTABLISHED) return -EDESTADDRREQ; daddr = &sk->sk_v6_daddr; - } else - daddr = NULL; + } if (daddr) { if (ipv6_addr_v4mapped(daddr)) { @@ -1364,30 +1401,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } } - /* Rough check on arithmetic overflow, - better check is made in ip6_append_data(). - */ - if (len > INT_MAX - sizeof(struct udphdr)) - return -EMSGSIZE; - - if (up->pending) { - if (up->pending == AF_INET) - return udp_sendmsg(sk, msg, len); - /* - * There are pending frames. - * The socket lock must be held while it's corked. - */ - lock_sock(sk); - if (likely(up->pending)) { - if (unlikely(up->pending != AF_INET6)) { - release_sock(sk); - return -EAFNOSUPPORT; - } - dst = NULL; - goto do_append_data; - } - release_sock(sk); - } ulen += sizeof(struct udphdr); memset(fl6, 0, sizeof(*fl6)); From patchwork Sun Apr 3 13:06:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799551 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 933E4C433FE for ; Sun, 3 Apr 2022 13:09:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358552AbiDCNKx (ORCPT ); Sun, 3 Apr 2022 09:10:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358767AbiDCNK3 (ORCPT ); Sun, 3 Apr 2022 09:10:29 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74BD713B for ; Sun, 3 Apr 2022 06:08:36 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id d7so10624620wrb.7 for ; Sun, 03 Apr 2022 06:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jkBtV/cBPJqLk8JiVyuTBgO4OMSKymsJTd5NAuhuvMo=; b=l9XcWzk6W9tBWqE6mf8ZDT7U9zlat0PiHJ+uiTmCIh0aSKB9ahLyABVnEzf7zUFQj8 LqhjgsoNYhS+zPeweNFNcjwkv7Z5jVBegpUVvuDVWuY55v3oQile3tNzEadxDOcsJeKK QN3dF9d5G6Fd5+o94dU81SREYCCWXqGrZomMRJ7tRTE3T74ax1mQSdyDDawOO+TH7VG/ 7gV998WhuPUbgXrXgniaTR8aBDzVtxbHnj5NRQMUmQ7JRA0qdlG0GGjOlRueCD00V8A3 emM1AdD+UrRTVZsM0czmv684aqgPNRA956PkX7K5NfnM/cwZBS84xaKVB6XILgU87e7e +ROw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jkBtV/cBPJqLk8JiVyuTBgO4OMSKymsJTd5NAuhuvMo=; b=3Hd54FdpZZuNSPfUvKvIOCUS1y7E1VHgiyv4Al3cTg+F41Vfj7HI9kr4RP3Kxg71Gb CaDpkV7NMNqH/tzMs1Cty/yi76P7DbXQZ9ht+5TYDJ9WOwOqhbBfbFTuT5TfbI2lqvDU cLdtlx41YbKYqmYks8Z0Ph1THrgSejo92xRqNCR28XrZJTvkc1tPmWS4DbWSukqRdlF9 H/8UECyoAJmPx/f7yhjQMq0CA/nUYro5tmAAi04c79SbQbdmZ3Xhw3b4EktTXitsBRIB XiMopoetEsxxfEflBOkqubd7PsLFVdX076fAKPZ6IUj4GavGFuvkAzwRVDtdPvYOq6dY k8Yg== X-Gm-Message-State: AOAM532u3+CdwGhE7+RHTJ0Jt+8SMaTJSG42Tf8y5dLCEGKh1ctFPyox +/hPoiLWElP8SGyNO863SslZL0eyT3M= X-Google-Smtp-Source: ABdhPJwGUChmaVZLKLi9NVGKBAkkLssNISqu9W/lNq42i5YqfHL5cSEsEcrsjfZQs6qgnVy0EHAspA== X-Received: by 2002:adf:df81:0:b0:206:dad:ba4a with SMTP id z1-20020adfdf81000000b002060dadba4amr2233519wrl.657.1648991314954; Sun, 03 Apr 2022 06:08:34 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:34 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 21/27] udp/ipv6: prioritise the ip6 path over ip4 checks Date: Sun, 3 Apr 2022 14:06:33 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org For AF_INET6 sockets we care the most about ipv6 but not ip4 mappings as it's requires some extra hops anyway. Take AF_INET6 case from the address parsing switch and add an explicit path for it. It removes some extra ifs from the path and removes the switch overhead. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 37 +++++++++++++++++-------------------- 1 file changed, 17 insertions(+), 20 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 26832be40f31..707e26ed45a4 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1357,30 +1357,27 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) /* destination address check */ if (sin6) { - if (addr_len < offsetof(struct sockaddr, sa_data)) - return -EINVAL; + if (addr_len < SIN6_LEN_RFC2133 || sin6->sin6_family != AF_INET6) { + if (addr_len < offsetof(struct sockaddr, sa_data)) + return -EINVAL; - switch (sin6->sin6_family) { - case AF_INET6: - if (addr_len < SIN6_LEN_RFC2133) + switch (sin6->sin6_family) { + case AF_INET: + goto do_udp_sendmsg; + case AF_UNSPEC: + msg->msg_name = sin6 = NULL; + msg->msg_namelen = addr_len = 0; + goto no_daddr; + default: return -EINVAL; - daddr = &sin6->sin6_addr; - if (ipv6_addr_any(daddr) && - ipv6_addr_v4mapped(&np->saddr)) - ipv6_addr_set_v4mapped(htonl(INADDR_LOOPBACK), - daddr); - break; - case AF_INET: - goto do_udp_sendmsg; - case AF_UNSPEC: - msg->msg_name = sin6 = NULL; - msg->msg_namelen = addr_len = 0; - daddr = NULL; - break; - default: - return -EINVAL; + } } + + daddr = &sin6->sin6_addr; + if (ipv6_addr_any(daddr) && ipv6_addr_v4mapped(&np->saddr)) + ipv6_addr_set_v4mapped(htonl(INADDR_LOOPBACK), daddr); } else { +no_daddr: if (sk->sk_state != TCP_ESTABLISHED) return -EDESTADDRREQ; daddr = &sk->sk_v6_daddr; From patchwork Sun Apr 3 13:06:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799558 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2800AC433EF for ; Sun, 3 Apr 2022 13:09:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358718AbiDCNLG (ORCPT ); Sun, 3 Apr 2022 09:11:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358533AbiDCNKc (ORCPT ); Sun, 3 Apr 2022 09:10:32 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F01361158 for ; Sun, 3 Apr 2022 06:08:37 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id m30so10675822wrb.1 for ; Sun, 03 Apr 2022 06:08:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KwpUHaq2Ufbfs+a40mjnOExLVtmmwM2j4c3BJ8ZmRlg=; b=kTS5cZCq7+GiSsDmzKbxRyntTfa7midbA4Lua4Ie444m5qSsrCEAY0yV0cG6Fv7NMQ /ys79InODOShzlWuue63nHwy+BFUt9lHemO2ybStKRRMqs4zqomFzu31WHEiAfRUfaOb jxPF7RqN+ZOZIRCJW4aaJ2E793tWvyjhc8BCD1DobIdf7ETKaoIENtUPf77NvGhZKxFh PEIjPtipEm438OG7C2IpHr/SKqu6ZJdSL971uEvijRjVR+RGbCfx6bvoyQCNX0790Ub4 oB1H0dzUQkZz4X5/nGdVoJKt//1prd9Ay51WZbDGYjdxcGb8/WGWtb+gCkxN0RVtm2h1 VenA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KwpUHaq2Ufbfs+a40mjnOExLVtmmwM2j4c3BJ8ZmRlg=; b=M49nhys9uzGpM7U6bXk/mNDR5zrCVysKNvuI42gWdoKXEO1FMlSiXFDSzB/9llLsoP iETwAIBDprdV9cjUOl8W/RqUS2PfPUgTt4h/RK8QEKGVVSaCa0y9nlXn+vgvA0dMc2nL i91G7BVK71HAFKcT9MB0B58eCE/TDSWRftCoS5o3TCTCJoDhEKireUo4+yR2DwP3lPdN 6w+AR+hBIeq4jmRSgoi44yk78+XINwu3agP1Y/E5fhMghC5wn90R1JJhKn0IR/VHCpVF AhHEL5g2s2fBavMaAqiBeJ9Fa6LuI33s5O+0ir7blRVkrs4dR2O+hpf66sep+5YnpqRR QZXg== X-Gm-Message-State: AOAM533IXiMcagWuLzNH0GmYzz2RGpr9sRtfqcoIAYkWkpf+bSatScV7 xwY8X7Vy4NAzj44KX6UX2dSglxe7Yeo= X-Google-Smtp-Source: ABdhPJz98a7Y0UztvBcKPWvaz7l+1ihpEC7/rXRRTWdIRxYPHg0E8N8VrfCeEBemmAnImzb1huwa4A== X-Received: by 2002:a5d:6945:0:b0:206:bd5:bf90 with SMTP id r5-20020a5d6945000000b002060bd5bf90mr3092868wrw.252.1648991316324; Sun, 03 Apr 2022 06:08:36 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:35 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 22/27] udp/ipv6: optimise udpv6_sendmsg() daddr checks Date: Sun, 3 Apr 2022 14:06:34 +0100 Message-Id: <0af7aa19e575c9bd9d8be3bc3e7814af4b3a09b7.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org All paths taking udpv6_sendmsg() to the ipv6_addr_v4mapped() check set a non zero daddr, we can safely kill the NULL check just before it. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 707e26ed45a4..cbb11316a526 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1383,19 +1383,18 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) daddr = &sk->sk_v6_daddr; } - if (daddr) { - if (ipv6_addr_v4mapped(daddr)) { - struct sockaddr_in sin; - sin.sin_family = AF_INET; - sin.sin_port = sin6 ? sin6->sin6_port : inet->inet_dport; - sin.sin_addr.s_addr = daddr->s6_addr32[3]; - msg->msg_name = &sin; - msg->msg_namelen = sizeof(sin); + if (ipv6_addr_v4mapped(daddr)) { + struct sockaddr_in sin; + + sin.sin_family = AF_INET; + sin.sin_port = sin6 ? sin6->sin6_port : inet->inet_dport; + sin.sin_addr.s_addr = daddr->s6_addr32[3]; + msg->msg_name = &sin; + msg->msg_namelen = sizeof(sin); do_udp_sendmsg: - if (__ipv6_only_sock(sk)) - return -ENETUNREACH; - return udp_sendmsg(sk, msg, len); - } + if (__ipv6_only_sock(sk)) + return -ENETUNREACH; + return udp_sendmsg(sk, msg, len); } ulen += sizeof(struct udphdr); From patchwork Sun Apr 3 13:06:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799559 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4425EC433FE for ; Sun, 3 Apr 2022 13:09:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358720AbiDCNLH (ORCPT ); Sun, 3 Apr 2022 09:11:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358550AbiDCNKo (ORCPT ); Sun, 3 Apr 2022 09:10:44 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51AB263B6 for ; Sun, 3 Apr 2022 06:08:39 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id h4so10584783wrc.13 for ; Sun, 03 Apr 2022 06:08:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uKsgDfFrJ6sHu/Q67vDokAer++R96ex3/kqPUqL2P3Y=; b=onKw21DPHB/wMUHDHUpOxOM9DwsKwfIMsIfLBLpaIjY7hE39w6KYf8I1TvXBE0BBrt keWPdtsywbscivuacOUvo39eF7wlJz0SQRgJLBdL4VaBdbfpe8FO7h/kFwQNrch7PedE Hs6qgIngX2hwS5N8MMjsEf97uuGD+g7QzUGbNwQd9kIHHCecpOzK3wr5AbRXqLGh4sU6 r56vi52sQ/rngdeoQ9l1rMfEdMmAjRJeFVGwhVjv7aTpRCDlGM+DKhjjVPFfHFiXOaQW O9Hu9Yc/t42udnH1vYnPsjgHhIrpi51+UFxSL1IYqcbFyqcb6DWrnRjYznoYmI/q1qBP YwBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uKsgDfFrJ6sHu/Q67vDokAer++R96ex3/kqPUqL2P3Y=; b=PMbfBfMKZWuSFeD2r7z6vJRPks24wXPRDTUhr6re7mKKkDfoOZY/jQB7vmoTxHyG0t bbWXqvea6HLNw0anxKZ9AyLoTpEKIEWCJBn2tIcZ73XBLHMcokwjWwHTxxGwDBjPlhpO guTuzaIQEXJZLDGG+ZAl3azA3OtzrfUtfav+/Dc3SMPLP6UwwLhJM2vsNUfMR3XA6nmd gpCYx/PuAODHKWTv0bhnmH+uiWvNgawVpnuBfk35quue9qCk84BBAi9sdLEn1q7N5HjE ovBDr0xEPh2iWMV+Pwwn1yAuNzZywVmpYWQUZ8NYY0j5xrrF9zcrdJ1qjvoN7Jcg7OT6 /McQ== X-Gm-Message-State: AOAM533Ye498mvuZ1iP0DVUFamxa40oSbER2M8sK784zJZ5DcwJU0yJJ FE9D1iVaYw7BBmlr79u7bAUkh5c/me0= X-Google-Smtp-Source: ABdhPJzkbqUTzhXiSpknivlerCj6Nfygtjd6hFDLbTIg2LpEKBPzLJip/qoedpUbN1u2B8pcrgiWXw== X-Received: by 2002:a5d:5704:0:b0:203:f9bb:b969 with SMTP id a4-20020a5d5704000000b00203f9bbb969mr13439886wrv.459.1648991317790; Sun, 03 Apr 2022 06:08:37 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:37 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 23/27] udp/ipv6: optimise out daddr reassignment Date: Sun, 3 Apr 2022 14:06:35 +0100 Message-Id: <24a4aec56b6a94c1d54a0e81bc0af6efd29fd622.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org There is nothing that checks daddr placement in udpv6_sendmsg(), so the check reassigning it to ->sk_v6_daddr looks like a not needed anymore artifact from the past. Remove it. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index cbb11316a526..2b5a3ed3f138 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1417,14 +1417,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } } - /* - * Otherwise it will be difficult to maintain - * sk->sk_dst_cache. - */ - if (sk->sk_state == TCP_ESTABLISHED && - ipv6_addr_equal(daddr, &sk->sk_v6_daddr)) - daddr = &sk->sk_v6_daddr; - if (addr_len >= sizeof(struct sockaddr_in6) && sin6->sin6_scope_id && __ipv6_addr_needs_scope_id(__ipv6_addr_type(daddr))) From patchwork Sun Apr 3 13:06:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799560 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77E61C433EF for ; Sun, 3 Apr 2022 13:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358765AbiDCNLQ (ORCPT ); Sun, 3 Apr 2022 09:11:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358691AbiDCNK6 (ORCPT ); Sun, 3 Apr 2022 09:10:58 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDFE4E08F for ; Sun, 3 Apr 2022 06:08:40 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id i127-20020a1c3b85000000b0038e710da2dcso137593wma.1 for ; Sun, 03 Apr 2022 06:08:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v9yxQTyA86mePd2RU+Ke+BjoJwXYliN0D9JR7+nNx10=; b=eh50WPqcbnMjWNo34j9nSp6EVrW1w7XIppkPs08aA3NB/XEDDXelB5JxemmWh9ay9j TvPSDPXOD8G36aIFPdvQ81hPQRdWZE6N4Xh8Q9cKgaJCVn+nvq5a/Uql39w9ZQ346g5Q zQWbA6lE9IMX5U+yIPdULIiBUDjDYAVV3KzXdjXUPxLmMpuLGKYN63lHxx8UNDKjcCsr iVIS0ymTMCqv39l9J9p1L84Rlpg40ciEQDuJivV8Lm4IUKpnLQsSDtmXN3T6UuXx42rr rXsLQwMtyBd1EHyPDCAD3SEJ2d+/TmN1ZYOekfFWHpoqoagDccGqlRf3WKl8t49udZQf qE6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v9yxQTyA86mePd2RU+Ke+BjoJwXYliN0D9JR7+nNx10=; b=NZ+xmXvVcrd+VqZeb8pZDDprb2RCOE13i9/M5org73T67H2/mj7XNvJ4Wc53Xj5ltQ Y6y5nE+x5BSKfKaHCl2FIrUW3JTjPCvdbVWgMLm4xmU8/hxkfDcZWW37klw2N9cKKnlW rRIRwsZdiI2MneTdXdtkJRGdkYzsy1ALxpT3KBUIF6O2nN63fIxWducPXb+fgjIgNU82 9YxzpxzyL8IEIbNHhVxKbX1SoJ61NjyTdKuhY17MQ0XmereSf+VYlotV54GtecyQXcMe 9R4XBvS+13R3OE9JJZQ9EmiSXxg0NO2fOz6cfECiE2wLzevZ1AfvwxU0Llm2CEOSLs/A Jp5g== X-Gm-Message-State: AOAM532jfFAMjpqrLPzQF5deRIwfcux/ZQt33oFa+QcVWB8IWNLXOUgt Y+wZ06gg7GNd7A7qDmy5FcfHNOo9I0g= X-Google-Smtp-Source: ABdhPJz/nuUIpN2sgzHIcobyL9DNueuSnkK12RPkUNWHhgEv3bz9dKf9cfXD7zgwg3OEHPA9ciF/dg== X-Received: by 2002:a05:600c:4fc4:b0:38c:d622:f445 with SMTP id o4-20020a05600c4fc400b0038cd622f445mr15775436wmq.73.1648991318942; Sun, 03 Apr 2022 06:08:38 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:38 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 24/27] udp/ipv6: clean up udpv6_sendmsg's saddr init Date: Sun, 3 Apr 2022 14:06:36 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org We initialise fl6 in udpv6_sendmsg() to zeroes, that sets saddr to any addr, then it might be changed in by cmsg but only to a non-any addr. After we check again for it left set to "any", which is likely to be so, and try to initialise it from socket saddr. The result of it is that fl6->saddr is set to cmsg's saddr if specified and inet6_sk(sk)->saddr otherwise. We can achieve the same by pre-setting it to the sockets saddr and potentially overriding by cmsg after. This looks a bit cleaner comparing to conditional init and also removes extra checks from the way. Signed-off-by: Pavel Begunkov --- net/ipv6/udp.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 2b5a3ed3f138..0b82447629b7 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1431,14 +1431,15 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) connected = true; } + fl6->flowi6_uid = sk->sk_uid; + fl6->saddr = np->saddr; + fl6->daddr = *daddr; + if (!fl6->flowi6_oif) fl6->flowi6_oif = sk->sk_bound_dev_if; - if (!fl6->flowi6_oif) fl6->flowi6_oif = np->sticky_pktinfo.ipi6_ifindex; - fl6->flowi6_uid = sk->sk_uid; - if (msg->msg_controllen) { opt = &opt_space; memset(opt, 0, sizeof(struct ipv6_txoptions)); @@ -1473,9 +1474,6 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) fl6->flowi6_proto = sk->sk_protocol; fl6->flowi6_mark = ipc6.sockc.mark; - fl6->daddr = *daddr; - if (ipv6_addr_any(&fl6->saddr) && !ipv6_addr_any(&np->saddr)) - fl6->saddr = np->saddr; fl6->fl6_sport = inet->inet_sport; if (cgroup_bpf_enabled(CGROUP_UDP6_SENDMSG) && !connected) { From patchwork Sun Apr 3 13:06:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799561 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD96C433F5 for ; Sun, 3 Apr 2022 13:10:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358806AbiDCNL4 (ORCPT ); Sun, 3 Apr 2022 09:11:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358749AbiDCNLL (ORCPT ); Sun, 3 Apr 2022 09:11:11 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6072A26AD4 for ; Sun, 3 Apr 2022 06:08:43 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id d29so4585482wra.10 for ; Sun, 03 Apr 2022 06:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JSmEvATg3WA4os4xJY73VvZ9JZfZo2VfV1MEBUfftts=; b=eG5JfJsG7lB74ixtQefMRFk8g2WVqGrim5q70OGHklTdz9nLpF7e5BIOKf7/+FWiDv qtuEL8wmfDl6PF/GVgLv3tDsWCcju7qLcKrStoEaVBlVmUnDejxRYz0rY+AdpyqPCaqE ZlEe9daAJQ/UK53rEjASbKTSZricheWc+Qaln4bXmCOR0B62Cm+E96wyNaAP6Qv6lqMy lT16kL3VmH7esX97GSunziYZhJeOZDHTFiAX2lpO7Bz/sjT4qNkXTfxJwiuKEKEpeMvc 1enDzs0bIUEt4YYhU9qaHf+Katjql2YXZBhwCGIsTJqL1SOE1RTK1fJu17JBYjO7l82G lK7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JSmEvATg3WA4os4xJY73VvZ9JZfZo2VfV1MEBUfftts=; b=bU/r0fL3zA2KSaKWNqF2beMqqCVsoem0Z2QC9TOtHNbs6F9x3LqfdJW5n9WvD24cwt ruXicAq2NuGn7JXOHXRTp2PU0xVahj2vzFTse6NORKOty/1nMFLBi4TL3o0KBPGYsQhw Iw5yM5fQXG5C1xS+nuVSnFRSFlAjdWumpQPIpPr9+UwLY1QajxuH5q0srZnE+/Al7r3H 1lBk6HZ2ZYvDufoIkemNgER/q+mpBi03JEer7VK871UTmx/Fc5lRnjM+5xBaGDhnn9WQ KHb+O06P+QOoVsl8rv3siMR5kVGrD7NkFBGp9TDN+HULmMl9scoJKDM/eLYOw3lV2xIv +4hQ== X-Gm-Message-State: AOAM531U4PgKKxmDGJOOh8boTvOBinVwla81SsvbPW10ITCvGlfQORNp d8a2rD3aLQijsPSfMiVXbAKGyzvCjtA= X-Google-Smtp-Source: ABdhPJyEnZ1WMKCWflAe3G/UYe1/rB+bPHZ4yMk+tdnYYzf4S60yzm45aXoklzdotc7CO5nwWWZA4g== X-Received: by 2002:adf:d1eb:0:b0:205:a502:d2ff with SMTP id g11-20020adfd1eb000000b00205a502d2ffmr14000893wrd.125.1648991322258; Sun, 03 Apr 2022 06:08:42 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:41 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 25/27] ipv6: refactor opts push in __ip6_make_skb() Date: Sun, 3 Apr 2022 14:06:37 +0100 Message-Id: <3bfe4b8198720c5538b228fb82706d3f25f8a6b7.1648981571.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Don't preload v6_cork->opt before we actually need it, it likely to be saved on the stack and read again for no good reason. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index bd5de7a5aa8c..3c37b07cbfae 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1857,7 +1857,6 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, struct ipv6_pinfo *np = inet6_sk(sk); struct net *net = sock_net(sk); struct ipv6hdr *hdr; - struct ipv6_txoptions *opt = v6_cork->opt; struct rt6_info *rt = (struct rt6_info *)cork->base.dst; struct flowi6 *fl6 = &cork->fl.u.ip6; unsigned char proto = fl6->flowi6_proto; @@ -1886,10 +1885,14 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, __skb_pull(skb, skb_network_header_len(skb)); final_dst = &fl6->daddr; - if (opt && opt->opt_flen) - ipv6_push_frag_opts(skb, opt, &proto); - if (opt && opt->opt_nflen) - ipv6_push_nfrag_opts(skb, opt, &proto, &final_dst, &fl6->saddr); + if (v6_cork->opt) { + struct ipv6_txoptions *opt = v6_cork->opt; + + if (opt->opt_flen) + ipv6_push_frag_opts(skb, opt, &proto); + if (opt->opt_nflen) + ipv6_push_nfrag_opts(skb, opt, &proto, &final_dst, &fl6->saddr); + } skb_push(skb, sizeof(struct ipv6hdr)); skb_reset_network_header(skb); From patchwork Sun Apr 3 13:06:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799562 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3C87C433F5 for ; Sun, 3 Apr 2022 13:10:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358692AbiDCNL6 (ORCPT ); Sun, 3 Apr 2022 09:11:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358750AbiDCNLL (ORCPT ); Sun, 3 Apr 2022 09:11:11 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C30627172 for ; Sun, 3 Apr 2022 06:08:45 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id r11-20020a1c440b000000b0038ccb70e239so484149wma.3 for ; Sun, 03 Apr 2022 06:08:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PZv4QnAyNuMdX6ZYSNFz/+Z4cnf7iFtgX+JqxbuLIGo=; b=mlyaVkKN2gHHASZQdwWb6MNLgWsxWJOygWcoew6k+SUw9E1O1YTw5fP1Vg0DVDHjnK lyG9+epOBOjT7YCsXwQ3RDJfS8VZyBwtf/bVlYAYcpOogx/YuKiAQAa9b/79hlhqj5bZ 433gtkbVgKRGuz29buB9xYj2SWWa7EpEifCruep5QTIkleaA0dSPJlqpVF6g8k2U7aT5 wS4YhCushzMjxtiTVkvZ8agp5vbWcaWB6KyT1FIr/FMeRPvKeORyIZKC1cbbLwVP6lOD zDc5kB7vGXpvNmTXuiZ6GHxsQCWkAWIPvgLcNA+Dt14gbuaqMOo94gL3p2Jjw8LkJ9rW mKEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PZv4QnAyNuMdX6ZYSNFz/+Z4cnf7iFtgX+JqxbuLIGo=; b=zY6cgNVi2O0+mg23ipUkI5du9YwiGp9gY/Vxwan8Fl9d3/47kmmj5yl5lPO2GUrqcp uqnIVjaBq5WwXa/rNXUeUhIVySPTwMveHTihBL28JucrJtNDJxX3tyxgnJfxaYFMkzKw e+pp9YqTjcYfYxaI0zDZNSwwSmcV8NyJc4q/ZkXf1mXTvbxSvvHNV525ea5V4iz5GAd/ BW41auqoQ4DuFTn5jvjUoklTnKNC/rUofkwkgcM80G5jPQlmkwcJPPy9L4+Hpdhzu+Xj zMI+VyFzrfpG13cspYP5NLjqHE00Qdfa9bmjgDsd08uNDKlIkhXGNoJJgwDyOM8LqEK1 mGlA== X-Gm-Message-State: AOAM5303SGYToIyiV04FxbQFwmVKXQ9FKsCEYJGJmKPNKQzQ7e0hieW0 uodMFwVTvNdd8KxoPK6rhX92Z6Trau0= X-Google-Smtp-Source: ABdhPJwnaAn1hzoNSaTw7b8GZS81qqQ6Lm8uZKSM6VD36PDNv1WpRfXZqsFKOITomwKwcRs7U19jUQ== X-Received: by 2002:a1c:7308:0:b0:38c:7b63:e385 with SMTP id d8-20020a1c7308000000b0038c7b63e385mr15647276wmb.116.1648991323587; Sun, 03 Apr 2022 06:08:43 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:43 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 26/27] ipv6: improve opt-less __ip6_make_skb() Date: Sun, 3 Apr 2022 14:06:38 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org We do a bit of a network header pointer shuffling in __ip6_make_skb() expecting that ipv6_push_*frag_opts() might change the layout. Avoid it with associated overhead when there are no opts. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 3c37b07cbfae..f7c092af64f5 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1882,22 +1882,20 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, /* Allow local fragmentation. */ skb->ignore_df = ip6_sk_ignore_df(sk); - __skb_pull(skb, skb_network_header_len(skb)); - final_dst = &fl6->daddr; if (v6_cork->opt) { struct ipv6_txoptions *opt = v6_cork->opt; + __skb_pull(skb, skb_network_header_len(skb)); if (opt->opt_flen) ipv6_push_frag_opts(skb, opt, &proto); if (opt->opt_nflen) ipv6_push_nfrag_opts(skb, opt, &proto, &final_dst, &fl6->saddr); + skb_push(skb, sizeof(struct ipv6hdr)); + skb_reset_network_header(skb); } - skb_push(skb, sizeof(struct ipv6hdr)); - skb_reset_network_header(skb); hdr = ipv6_hdr(skb); - ip6_flow_hdr(hdr, v6_cork->tclass, ip6_make_flowlabel(net, skb, fl6->flowlabel, ip6_autoflowlabel(net, np), fl6)); From patchwork Sun Apr 3 13:06:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12799563 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58D1CC433EF for ; Sun, 3 Apr 2022 13:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235723AbiDCNMR (ORCPT ); Sun, 3 Apr 2022 09:12:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345979AbiDCNLV (ORCPT ); Sun, 3 Apr 2022 09:11:21 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9059B27B22 for ; Sun, 3 Apr 2022 06:08:46 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id h16so4365004wmd.0 for ; Sun, 03 Apr 2022 06:08:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qR0CNrwB97TJccmTjLshQeiDj/mDiBlEwPLgtdaERnc=; b=QBhYfWDNXisDdliMa3dC1Gc0AwJ0OphUWSNOmJWjKiLeA/Cn60VhLhttluNEsxLl8R Z03Co/2g9O4nUcU95357YVb6us75chYcaB+8EBGxB21kSWDLoWhe+Tqy1grrbpZo3PEU 1N0IPpOO5uc3IVBlzR6lda/BogyJ6vh14WeXrCQm75LPb/jez2DfIMNmQwn1fNhN2U+X 26C3EKENP9b/bwWLxb2iJLiCNzNFutcsIuefDsEns4Q1VlWXnD46fXI3ET4mOONiuBQQ q37v8JAWM346qNACuysbDH7uv2jhjXMF5EtKA4aPs5e+mtv1OI3mFbBxU3PrdEoLgHna IErQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qR0CNrwB97TJccmTjLshQeiDj/mDiBlEwPLgtdaERnc=; b=MUqahmtPH1qGyf1ex6Aj0mwK1YfPc5zW8PJSaHwnu9k/RjQbHLt803HXZuezdlDtNz uX47XXbGdz96sXmsRqfIh/gz5xdSNp36w5ON88l29yAAJlhWFmyrqXDz9ANP5igyo5im 9qwTmFXVV7r23G3D4K++s4bHSS1w3kD12vyS/eLrRwYJfxawYFDF3ohXWLUmBmID7+BG CWm7BqnICWCXGvOFiWS+jBigogUSNcaAWXnxsHQRBhi4mUcswXso7LztYJK/QunRR1fV ejuWyV5Yy7w3PlWh1lu9TTxX3BE1dcnR+Lk1BJz8Y3jo4+sEpE6tmWDHRFJTlcaJpna2 WIVg== X-Gm-Message-State: AOAM532KdVeo5AZrjyaq6OFiR1p+Uh6dFpUZb97VkkcXYb8SExxTF9+X wszST/lMwIsnOWzyPMrNDS+/60KYaPQ= X-Google-Smtp-Source: ABdhPJyJIacOc7HZ9N39EFsW484ZnhMRBFvX24Zhz6pSISKfPa4MNmTsV6PEnhbBQYeqZsfrgfwjtg== X-Received: by 2002:a7b:cb93:0:b0:38e:710e:8934 with SMTP id m19-20020a7bcb93000000b0038e710e8934mr660512wmi.83.1648991324858; Sun, 03 Apr 2022 06:08:44 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-233-133.dab.02.net. [82.132.233.133]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm7866995wmq.35.2022.04.03.06.08.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 06:08:44 -0700 (PDT) From: Pavel Begunkov To: netdev@vger.kernel.org, "David S . Miller" , Jakub Kicinski Cc: Eric Dumazet , Wei Liu , Paul Durrant , Pavel Begunkov Subject: [PATCH net-next 27/27] ipv6: clean up ip6_setup_cork Date: Sun, 3 Apr 2022 14:06:39 +0100 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Do a bit of refactoring for ip6_setup_cork(). Cache a xfrm_dst_path() result to not call it twice, reshuffle ifs to not repeat some parts twice and so. Signed-off-by: Pavel Begunkov --- net/ipv6/ip6_output.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index f7c092af64f5..e10b7f42e493 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1356,15 +1356,13 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, struct ipv6_pinfo *np = inet6_sk(sk); unsigned int mtu; struct ipv6_txoptions *nopt, *opt = ipc6->opt; + struct dst_entry *xrfm_dst; /* callers pass dst together with a reference, set it first so * ip6_cork_release() can put it down even in case of an error. */ cork->base.dst = &rt->dst; - /* - * setup for corking - */ if (opt) { if (WARN_ON(v6_cork->opt)) return -EINVAL; @@ -1397,28 +1395,26 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork, } v6_cork->hop_limit = ipc6->hlimit; v6_cork->tclass = ipc6->tclass; - if (rt->dst.flags & DST_XFRM_TUNNEL) - mtu = np->pmtudisc >= IPV6_PMTUDISC_PROBE ? - READ_ONCE(rt->dst.dev->mtu) : dst_mtu(&rt->dst); + + xrfm_dst = xfrm_dst_path(&rt->dst); + if (dst_allfrag(xrfm_dst)) + cork->base.flags |= IPCORK_ALLFRAG; + + if (np->pmtudisc < IPV6_PMTUDISC_PROBE) + mtu = dst_mtu(rt->dst.flags & DST_XFRM_TUNNEL ? &rt->dst : xrfm_dst); else - mtu = np->pmtudisc >= IPV6_PMTUDISC_PROBE ? - READ_ONCE(rt->dst.dev->mtu) : dst_mtu(xfrm_dst_path(&rt->dst)); - if (np->frag_size < mtu) { - if (np->frag_size) - mtu = np->frag_size; - } + mtu = READ_ONCE(rt->dst.dev->mtu); + + if (np->frag_size < mtu && np->frag_size) + mtu = np->frag_size; + cork->base.fragsize = mtu; cork->base.gso_size = ipc6->gso_size; cork->base.tx_flags = 0; cork->base.mark = ipc6->sockc.mark; sock_tx_timestamp(sk, ipc6->sockc.tsflags, &cork->base.tx_flags); - - if (dst_allfrag(xfrm_dst_path(&rt->dst))) - cork->base.flags |= IPCORK_ALLFRAG; cork->base.length = 0; - cork->base.transmit_time = ipc6->sockc.transmit_time; - return 0; }