From patchwork Fri Mar 12 15:04:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134851 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62AB9C433DB for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3452164FF5 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229959AbhCLPFM (ORCPT ); Fri, 12 Mar 2021 10:05:12 -0500 Received: from mail.kernel.org ([198.145.29.99]:43520 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231717AbhCLPEu (ORCPT ); Fri, 12 Mar 2021 10:04:50 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3A65F64F77; Fri, 12 Mar 2021 15:04:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561489; bh=V8FFZJKr/fjyAn8LXI9YH/ZhAckd3YMLQ0Q0jEu07Ac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f8+LZLzlwirkiSkib1IwNCX5LnE0s8/VtJFdSw+krKtQxMA/PniNug0CN4JSmwhjl DQL1H30Sk5REn+rur6TegwvKK9dMO2bzcUelagv/6t6zubrd762C5PZnJnkyb10Y3n 9LRqF+/bSzLAmrZWOn/mbABuo021gYITWEZD+gItZGT0SG1aFQoKq/EuyMJvxa8v9H 65+BOAQJCuNg/Cfk/6Lul1W5o4DN4CTympCZqAoZcTC2YmovW8jAXzkicLghcq615p kFute+1i0c7028ODlLxkSw+KNuCtG+CqbnnwZnMYJjhWohEjAs7BWmOnyUId4LWRcV GxAOnh4Qw7p1Q== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 01/16] net-sysfs: convert xps_cpus_show to bitmap_zalloc Date: Fri, 12 Mar 2021 16:04:29 +0100 Message-Id: <20210312150444.355207-2-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use bitmap_zalloc instead of zalloc_cpumask_var in xps_cpus_show to align with xps_rxqs_show. This will improve maintenance and allow us to factorize the two functions. The function should behave the same. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 307628fdf380..3a083c0c9dd3 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1367,8 +1367,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, int cpu, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - cpumask_var_t mask; - unsigned long index; + unsigned long *mask, index; if (!netif_is_multiqueue(dev)) return -ENOENT; @@ -1396,7 +1395,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, } } - if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) { + mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); + if (!mask) { ret = -ENOMEM; goto err_rtnl_unlock; } @@ -1414,7 +1414,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, for (i = map->len; i--;) { if (map->queues[i] == index) { - cpumask_set_cpu(cpu, mask); + set_bit(cpu, mask); break; } } @@ -1424,8 +1424,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, rtnl_unlock(); - len = snprintf(buf, PAGE_SIZE, "%*pb\n", cpumask_pr_args(mask)); - free_cpumask_var(mask); + len = bitmap_print_to_pagebuf(false, buf, mask, nr_cpu_ids); + bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; err_rtnl_unlock: From patchwork Fri Mar 12 15:04:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134857 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 728A1C433E6 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C61764FFD for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232009AbhCLPFN (ORCPT ); Fri, 12 Mar 2021 10:05:13 -0500 Received: from mail.kernel.org ([198.145.29.99]:43542 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230136AbhCLPEw (ORCPT ); Fri, 12 Mar 2021 10:04:52 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 150B464F78; Fri, 12 Mar 2021 15:04:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561492; bh=h/BOmfBbTaJBFbxKqdSRFVRIBV/jimnFB8eQIyKuQ0o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f1N7VKyn07N6rM7grOrQvWmh61L6MKZKh4yIHeNhUtW4U651Bg012ED31xN7LXRHT kbNSVndXi1fwJ6b8V5PYNqtQSDedIEynSoUpYhpQj3y62EfmszPzcfeZ1SDzW8XH0s ErYAW17+2TvNJAwlCToleSej2iHo8QPXaU33h+/t57LLl09KGyjWxkfGAgveaZubwt nIaX7aHGSY447gZUkUu1QM6pI0k1ovOKZAYg5vEUBiZdj+W2CAGJZ0LwzvzTAgJB+v KMnjlpeYwhWUaFy2q7museBJwh6e9CuDZp1gbcavRa1saXWFpRXvEBYuSv9AkN5T7X 7NuIE0W7+msrg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 02/16] net-sysfs: store the return of get_netdev_queue_index in an unsigned int Date: Fri, 12 Mar 2021 16:04:30 +0100 Message-Id: <20210312150444.355207-3-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In net-sysfs, get_netdev_queue_index returns an unsigned int. Some of its callers use an unsigned long to store the returned value. Update the code to be consistent, this should only be cosmetic. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 3a083c0c9dd3..5dc4223f6b68 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1367,7 +1367,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, int cpu, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; if (!netif_is_multiqueue(dev)) return -ENOENT; @@ -1437,7 +1438,7 @@ static ssize_t xps_cpus_store(struct netdev_queue *queue, const char *buf, size_t len) { struct net_device *dev = queue->dev; - unsigned long index; + unsigned int index; cpumask_var_t mask; int err; @@ -1479,7 +1480,8 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; index = get_netdev_queue_index(queue); @@ -1541,7 +1543,8 @@ static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, { struct net_device *dev = queue->dev; struct net *net = dev_net(dev); - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; int err; if (!ns_capable(net->user_ns, CAP_NET_ADMIN)) From patchwork Fri Mar 12 15:04:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134855 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90CA3C43381 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6040A64FE2 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232032AbhCLPFO (ORCPT ); Fri, 12 Mar 2021 10:05:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:43566 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231788AbhCLPEz (ORCPT ); Fri, 12 Mar 2021 10:04:55 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id E964464F77; Fri, 12 Mar 2021 15:04:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561495; bh=++YZELZPcLZWiZKJwsBz+YU9mmpmYURGyxz8dkpwo8A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HjILpSZqEyMPrGZth+ApFZVAeH5wlBGHu0BIfJbUKETlxqOR+hwi925zSTE+60rH2 8w7TcY1VTdo+6pLaV1TX1Dlj5CkecxkvUBCb8sVHfF8m8JLy5Pg+MKVaGNBW/ehfV+ 2+QHrZTWjqpjvR4Ks55Pdj/TzuAQymajExu+c836AbB4twQzELIetLdIEpEWxWnHFR ULQ3CLWPHBL8pwxgP0BmMhKO02RRpXZpV3aI8bJ9gDlBeMLY/0O+X401jm/toJ4xA/ ZYXxOd3iKvzQaVkOxvNDk7ZbbWU2asf49idcdugfSGTzpvbpWj84bxhne11fBormXE 0kVXlN9nOxyAg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 03/16] net-sysfs: make xps_cpus_show and xps_rxqs_show consistent Date: Fri, 12 Mar 2021 16:04:31 +0100 Message-Id: <20210312150444.355207-4-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Make the implementations of xps_cpus_show and xps_rxqs_show to converge, as the two share the same logic but diverted over time. This should not modify their behaviour but will help future changes and improve maintenance. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 5dc4223f6b68..5f76183ad5bc 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1364,7 +1364,7 @@ static const struct attribute_group dql_group = { static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) { - int cpu, len, ret, num_tc = 1, tc = 0; + int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; unsigned long *mask; @@ -1404,23 +1404,26 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, rcu_read_lock(); dev_maps = rcu_dereference(dev->xps_cpus_map); - if (dev_maps) { - for_each_possible_cpu(cpu) { - int i, tci = cpu * num_tc + tc; - struct xps_map *map; - - map = rcu_dereference(dev_maps->attr_map[tci]); - if (!map) - continue; - - for (i = map->len; i--;) { - if (map->queues[i] == index) { - set_bit(cpu, mask); - break; - } + if (!dev_maps) + goto out_no_maps; + + for (j = -1; j = netif_attrmask_next(j, NULL, nr_cpu_ids), + j < nr_cpu_ids;) { + int i, tci = j * num_tc + tc; + struct xps_map *map; + + map = rcu_dereference(dev_maps->attr_map[tci]); + if (!map) + continue; + + for (i = map->len; i--;) { + if (map->queues[i] == index) { + set_bit(j, mask); + break; } } } +out_no_maps: rcu_read_unlock(); rtnl_unlock(); From patchwork Fri Mar 12 15:04:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134859 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A52A5C433E9 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87CAE64F80 for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232066AbhCLPFO (ORCPT ); Fri, 12 Mar 2021 10:05:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:43586 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231920AbhCLPE6 (ORCPT ); Fri, 12 Mar 2021 10:04:58 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id BC86064F78; Fri, 12 Mar 2021 15:04:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561498; bh=IkWoeZjlWcKuAH652uOUPC0KWyiO47AOSp7a8uMpdJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FVnDd0yEpk5rirHXEvnAUPV31ssUT7YX4t++JtNiAbUzu1gKzOpI0VOYCrMjqnXkv sLQL8OFSqOHIbmcx8n29XEDFypKijf9C4fdGrKn3o1BEaEqrYI31/yqyOJqjRA+4LE KQHmXX1+JbLnnvcQjRebTQ9DM8aFWyNfECGHifRgDGK+Xk77EhfLerEDf6izYQu3AC X1IPboB/zff3A7NocnXaWyiCqph8Ca4ngslnZsO17CjEOIbTfvxfuFq9iO+JBQVzw3 TRSoxsE7WxeY2g6pQ8hlVKwTE1uXuCgM4+IuzL566Gku2r17fOJvYLDgaKxzCBFyVv XayRvs6tdIN1A== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 04/16] net: embed num_tc in the xps maps Date: Fri, 12 Mar 2021 16:04:32 +0100 Message-Id: <20210312150444.355207-5-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The xps cpus/rxqs map is accessed using dev->num_tc, which is used when allocating the map. But later updates of dev->num_tc can lead to having a mismatch between the maps and how they're accessed. In such cases the map values do not make any sense and out of bound accesses can occur (that can be easily seen using KASAN). This patch aims at fixing this by embedding num_tc into the maps, using the value at the time the map is created. This brings two improvements: - The maps can be accessed using the embedded num_tc, so we know for sure we won't have out of bound accesses. - Checks can be made before accessing the maps so we know the values retrieved will make sense. We also update __netif_set_xps_queue to conditionally copy old maps from dev_maps in the new one only if the number of traffic classes from both maps match. Signed-off-by: Antoine Tenart --- include/linux/netdevice.h | 6 ++++ net/core/dev.c | 63 +++++++++++++++++++++++++-------------- net/core/net-sysfs.c | 45 +++++++++++----------------- 3 files changed, 64 insertions(+), 50 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index b379d08a12ed..aa5c45198785 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -771,9 +771,15 @@ struct xps_map { /* * This structure holds all XPS maps for device. Maps are indexed by CPU. + * + * We keep track of the number of traffic classes used when the struct is + * allocated, in num_tc. This will be used to navigate the maps, to ensure we're + * not crossing its upper bound, as the original dev->num_tc can be updated in + * the meantime. */ struct xps_dev_maps { struct rcu_head rcu; + s16 num_tc; struct xps_map __rcu *attr_map[]; /* Either CPUs map or RXQs map */ }; diff --git a/net/core/dev.c b/net/core/dev.c index 2bfdd528c7c3..3fa96d54654c 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2491,7 +2491,7 @@ static bool remove_xps_queue_cpu(struct net_device *dev, struct xps_dev_maps *dev_maps, int cpu, u16 offset, u16 count) { - int num_tc = dev->num_tc ? : 1; + int num_tc = dev_maps->num_tc; bool active = false; int tci; @@ -2634,10 +2634,10 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, { const unsigned long *online_mask = NULL, *possible_mask = NULL; struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; + bool active = false, copy = false; int i, j, tci, numa_node_id = -2; int maps_sz, num_tc = 1, tc = 0; struct xps_map *map, *new_map; - bool active = false; unsigned int nr_ids; if (dev->num_tc) { @@ -2672,19 +2672,29 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (maps_sz < L1_CACHE_BYTES) maps_sz = L1_CACHE_BYTES; + /* The old dev_maps could be larger or smaller than the one we're + * setting up now, as dev->num_tc could have been updated in between. We + * could try to be smart, but let's be safe instead and only copy + * foreign traffic classes if the two map sizes match. + */ + if (dev_maps && dev_maps->num_tc == num_tc) + copy = true; + /* allocate memory for queue storage */ for (j = -1; j = netif_attrmask_next_and(j, online_mask, mask, nr_ids), j < nr_ids;) { - if (!new_dev_maps) - new_dev_maps = kzalloc(maps_sz, GFP_KERNEL); if (!new_dev_maps) { - mutex_unlock(&xps_map_mutex); - return -ENOMEM; + new_dev_maps = kzalloc(maps_sz, GFP_KERNEL); + if (!new_dev_maps) { + mutex_unlock(&xps_map_mutex); + return -ENOMEM; + } + + new_dev_maps->num_tc = num_tc; } tci = j * num_tc + tc; - map = dev_maps ? xmap_dereference(dev_maps->attr_map[tci]) : - NULL; + map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; map = expand_xps_map(map, j, index, is_rxqs_map); if (!map) @@ -2706,7 +2716,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { /* copy maps belonging to foreign traffic classes */ - for (i = tc, tci = j * num_tc; dev_maps && i--; tci++) { + for (i = tc, tci = j * num_tc; copy && i--; tci++) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); @@ -2736,14 +2746,14 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, numa_node_id = -1; } #endif - } else if (dev_maps) { + } else if (copy) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); } /* copy maps belonging to foreign traffic classes */ - for (i = num_tc - tc, tci++; dev_maps && --i; tci++) { + for (i = num_tc - tc, tci++; copy && --i; tci++) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); @@ -2761,11 +2771,18 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - for (i = num_tc, tci = j * num_tc; i--; tci++) { - new_map = xmap_dereference(new_dev_maps->attr_map[tci]); + for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) { map = xmap_dereference(dev_maps->attr_map[tci]); - if (map && map != new_map) - kfree_rcu(map, rcu); + if (!map) + continue; + + if (copy) { + new_map = xmap_dereference(new_dev_maps->attr_map[tci]); + if (map == new_map) + continue; + } + + kfree_rcu(map, rcu); } } @@ -2789,12 +2806,12 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* removes tx-queue from unused CPUs/rx-queues */ for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - for (i = tc, tci = j * num_tc; i--; tci++) + for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) active |= remove_xps_queue(dev_maps, tci, index); if (!netif_attr_test_mask(j, mask, nr_ids) || !netif_attr_test_online(j, online_mask, nr_ids)) active |= remove_xps_queue(dev_maps, tci, index); - for (i = num_tc - tc, tci++; --i; tci++) + for (i = dev_maps->num_tc - tc, tci++; --i; tci++) active |= remove_xps_queue(dev_maps, tci, index); } @@ -2812,7 +2829,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, j < nr_ids;) { for (i = num_tc, tci = j * num_tc; i--; tci++) { new_map = xmap_dereference(new_dev_maps->attr_map[tci]); - map = dev_maps ? + map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; if (new_map && new_map != map) @@ -3944,13 +3961,15 @@ sch_handle_egress(struct sk_buff *skb, int *ret, struct net_device *dev) static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb, struct xps_dev_maps *dev_maps, unsigned int tci) { + int tc = netdev_get_prio_tc_map(dev, skb->priority); struct xps_map *map; int queue_index = -1; - if (dev->num_tc) { - tci *= dev->num_tc; - tci += netdev_get_prio_tc_map(dev, skb->priority); - } + if (tc >= dev_maps->num_tc) + return queue_index; + + tci *= dev_maps->num_tc; + tci += tc; map = rcu_dereference(dev_maps->attr_map[tci]); if (map) { diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 5f76183ad5bc..1364d0f39cb0 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1364,9 +1364,9 @@ static const struct attribute_group dql_group = { static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) { - int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; + int j, len, ret, tc = 0; unsigned long *mask; unsigned int index; @@ -1378,22 +1378,13 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, if (!rtnl_trylock()) return restart_syscall(); - if (dev->num_tc) { - /* Do not allow XPS on subordinate device directly */ - num_tc = dev->num_tc; - if (num_tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } - - /* If queue belongs to subordinate dev use its map */ - dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; + /* If queue belongs to subordinate dev use its map */ + dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; - tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } + tc = netdev_txq_to_tc(dev, index); + if (tc < 0) { + ret = -EINVAL; + goto err_rtnl_unlock; } mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); @@ -1404,12 +1395,12 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, rcu_read_lock(); dev_maps = rcu_dereference(dev->xps_cpus_map); - if (!dev_maps) + if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; for (j = -1; j = netif_attrmask_next(j, NULL, nr_cpu_ids), j < nr_cpu_ids;) { - int i, tci = j * num_tc + tc; + int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; map = rcu_dereference(dev_maps->attr_map[tci]); @@ -1480,9 +1471,9 @@ static struct netdev_queue_attribute xps_cpus_attribute __ro_after_init static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) { - int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; + int j, len, ret, tc = 0; unsigned long *mask; unsigned int index; @@ -1491,14 +1482,12 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) if (!rtnl_trylock()) return restart_syscall(); - if (dev->num_tc) { - num_tc = dev->num_tc; - tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } + tc = netdev_txq_to_tc(dev, index); + if (tc < 0) { + ret = -EINVAL; + goto err_rtnl_unlock; } + mask = bitmap_zalloc(dev->num_rx_queues, GFP_KERNEL); if (!mask) { ret = -ENOMEM; @@ -1507,12 +1496,12 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) rcu_read_lock(); dev_maps = rcu_dereference(dev->xps_rxqs_map); - if (!dev_maps) + if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; for (j = -1; j = netif_attrmask_next(j, NULL, dev->num_rx_queues), j < dev->num_rx_queues;) { - int i, tci = j * num_tc + tc; + int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; map = rcu_dereference(dev_maps->attr_map[tci]); From patchwork Fri Mar 12 15:04:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134863 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 249F6C4332B for ; Fri, 12 Mar 2021 15:05:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E84E64FFE for ; Fri, 12 Mar 2021 15:05:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232093AbhCLPFQ (ORCPT ); Fri, 12 Mar 2021 10:05:16 -0500 Received: from mail.kernel.org ([198.145.29.99]:43606 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231958AbhCLPFC (ORCPT ); Fri, 12 Mar 2021 10:05:02 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8190F64F77; Fri, 12 Mar 2021 15:05:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561501; bh=Al4qLbv/PdSTK5twqZuWQnIytk1BKT+/FlGdYmeA93k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WBXKbGpkb4GllUQRXEESV6WbKMf9N6VHCjdZZxykGtcY4+kfZnIElJ8aK0E5fDklt a3JniTy7JlDESQ2RGrbfxrsHdSHP/BNmEjLTEf3efMf7pxxfQDhJ128TQN6y3ExL0Z W4mZQYOn4017dHT1I2G1EUtzoTgTwC9+hqWIAZZ0e588sQuYO5mU7lgt7XM6mNKPJo 0o4jsW4/DgeFP8eYk9musmu2L6l7s4Ytw+y/u+vlLCqlAkptnb1qGe7yTDZ3ZLAkeE IqRgYwY5iXESl19FqJnxZzy/CZgcvRPYSzcxYDSEVEZue5W/yBPIPT0lgGE+zfolNP XmO/tJjNVPSoA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 05/16] net: embed nr_ids in the xps maps Date: Fri, 12 Mar 2021 16:04:33 +0100 Message-Id: <20210312150444.355207-6-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Embed nr_ids (the number of cpu for the xps cpus map, and the number of rxqs for the xps cpus map) in dev_maps. That will help not accessing out of bound memory if those values change after dev_maps was allocated. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- include/linux/netdevice.h | 4 ++++ net/core/dev.c | 45 ++++++++++++++++++--------------------- net/core/net-sysfs.c | 38 +++++++++++++++++++-------------- 3 files changed, 47 insertions(+), 40 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index aa5c45198785..5c9e056a0e2d 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -772,6 +772,9 @@ struct xps_map { /* * This structure holds all XPS maps for device. Maps are indexed by CPU. * + * We keep track of the number of cpus/rxqs used when the struct is allocated, + * in nr_ids. This will help not accessing out-of-bound memory. + * * We keep track of the number of traffic classes used when the struct is * allocated, in num_tc. This will be used to navigate the maps, to ensure we're * not crossing its upper bound, as the original dev->num_tc can be updated in @@ -779,6 +782,7 @@ struct xps_map { */ struct xps_dev_maps { struct rcu_head rcu; + unsigned int nr_ids; s16 num_tc; struct xps_map __rcu *attr_map[]; /* Either CPUs map or RXQs map */ }; diff --git a/net/core/dev.c b/net/core/dev.c index 3fa96d54654c..98a9c620f05a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2524,14 +2524,14 @@ static void reset_xps_maps(struct net_device *dev, } static void clean_xps_maps(struct net_device *dev, const unsigned long *mask, - struct xps_dev_maps *dev_maps, unsigned int nr_ids, - u16 offset, u16 count, bool is_rxqs_map) + struct xps_dev_maps *dev_maps, u16 offset, u16 count, + bool is_rxqs_map) { + unsigned int nr_ids = dev_maps->nr_ids; bool active = false; int i, j; - for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), - j < nr_ids;) + for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), j < nr_ids;) active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) @@ -2551,7 +2551,6 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, { const unsigned long *possible_mask = NULL; struct xps_dev_maps *dev_maps; - unsigned int nr_ids; if (!static_key_false(&xps_needed)) return; @@ -2561,11 +2560,9 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (static_key_false(&xps_rxqs_needed)) { dev_maps = xmap_dereference(dev->xps_rxqs_map); - if (dev_maps) { - nr_ids = dev->num_rx_queues; - clean_xps_maps(dev, possible_mask, dev_maps, nr_ids, - offset, count, true); - } + if (dev_maps) + clean_xps_maps(dev, possible_mask, dev_maps, offset, + count, true); } dev_maps = xmap_dereference(dev->xps_cpus_map); @@ -2574,9 +2571,7 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (num_possible_cpus() > 1) possible_mask = cpumask_bits(cpu_possible_mask); - nr_ids = nr_cpu_ids; - clean_xps_maps(dev, possible_mask, dev_maps, nr_ids, offset, count, - false); + clean_xps_maps(dev, possible_mask, dev_maps, offset, count, false); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2673,11 +2668,12 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, maps_sz = L1_CACHE_BYTES; /* The old dev_maps could be larger or smaller than the one we're - * setting up now, as dev->num_tc could have been updated in between. We - * could try to be smart, but let's be safe instead and only copy - * foreign traffic classes if the two map sizes match. + * setting up now, as dev->num_tc or nr_ids could have been updated in + * between. We could try to be smart, but let's be safe instead and only + * copy foreign traffic classes if the two map sizes match. */ - if (dev_maps && dev_maps->num_tc == num_tc) + if (dev_maps && + dev_maps->num_tc == num_tc && dev_maps->nr_ids == nr_ids) copy = true; /* allocate memory for queue storage */ @@ -2690,6 +2686,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, return -ENOMEM; } + new_dev_maps->nr_ids = nr_ids; new_dev_maps->num_tc = num_tc; } @@ -2770,7 +2767,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, goto out_no_old_maps; for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + j < dev_maps->nr_ids;) { for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) { map = xmap_dereference(dev_maps->attr_map[tci]); if (!map) @@ -2804,12 +2801,12 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, goto out_no_maps; /* removes tx-queue from unused CPUs/rx-queues */ - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = -1; j = netif_attrmask_next(j, possible_mask, dev_maps->nr_ids), + j < dev_maps->nr_ids;) { for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) active |= remove_xps_queue(dev_maps, tci, index); - if (!netif_attr_test_mask(j, mask, nr_ids) || - !netif_attr_test_online(j, online_mask, nr_ids)) + if (!netif_attr_test_mask(j, mask, dev_maps->nr_ids) || + !netif_attr_test_online(j, online_mask, dev_maps->nr_ids)) active |= remove_xps_queue(dev_maps, tci, index); for (i = dev_maps->num_tc - tc, tci++; --i; tci++) active |= remove_xps_queue(dev_maps, tci, index); @@ -3965,7 +3962,7 @@ static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb, struct xps_map *map; int queue_index = -1; - if (tc >= dev_maps->num_tc) + if (tc >= dev_maps->num_tc || tci >= dev_maps->nr_ids) return queue_index; tci *= dev_maps->num_tc; @@ -4004,7 +4001,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, if (dev_maps) { int tci = sk_rx_queue_get(sk); - if (tci >= 0 && tci < dev->num_rx_queues) + if (tci >= 0) queue_index = __get_xps_queue_idx(dev, skb, dev_maps, tci); } diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 1364d0f39cb0..bb08bdc88fa9 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1366,9 +1366,9 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, { struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; + unsigned int index, nr_ids; int j, len, ret, tc = 0; unsigned long *mask; - unsigned int index; if (!netif_is_multiqueue(dev)) return -ENOENT; @@ -1387,19 +1387,20 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, goto err_rtnl_unlock; } - mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); + rcu_read_lock(); + dev_maps = rcu_dereference(dev->xps_cpus_map); + nr_ids = dev_maps ? dev_maps->nr_ids : nr_cpu_ids; + + mask = bitmap_zalloc(nr_ids, GFP_KERNEL); if (!mask) { ret = -ENOMEM; - goto err_rtnl_unlock; + goto err_rcu_unlock; } - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_cpus_map); if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; - for (j = -1; j = netif_attrmask_next(j, NULL, nr_cpu_ids), - j < nr_cpu_ids;) { + for (j = -1; j = netif_attrmask_next(j, NULL, nr_ids), j < nr_ids;) { int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; @@ -1419,10 +1420,12 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, rtnl_unlock(); - len = bitmap_print_to_pagebuf(false, buf, mask, nr_cpu_ids); + len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; +err_rcu_unlock: + rcu_read_unlock(); err_rtnl_unlock: rtnl_unlock(); return ret; @@ -1473,9 +1476,9 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) { struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; + unsigned int index, nr_ids; int j, len, ret, tc = 0; unsigned long *mask; - unsigned int index; index = get_netdev_queue_index(queue); @@ -1488,19 +1491,20 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) goto err_rtnl_unlock; } - mask = bitmap_zalloc(dev->num_rx_queues, GFP_KERNEL); + rcu_read_lock(); + dev_maps = rcu_dereference(dev->xps_rxqs_map); + nr_ids = dev_maps ? dev_maps->nr_ids : dev->num_rx_queues; + + mask = bitmap_zalloc(nr_ids, GFP_KERNEL); if (!mask) { ret = -ENOMEM; - goto err_rtnl_unlock; + goto err_rcu_unlock; } - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_rxqs_map); if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; - for (j = -1; j = netif_attrmask_next(j, NULL, dev->num_rx_queues), - j < dev->num_rx_queues;) { + for (j = -1; j = netif_attrmask_next(j, NULL, nr_ids), j < nr_ids;) { int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; @@ -1520,11 +1524,13 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) rtnl_unlock(); - len = bitmap_print_to_pagebuf(false, buf, mask, dev->num_rx_queues); + len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; +err_rcu_unlock: + rcu_read_unlock(); err_rtnl_unlock: rtnl_unlock(); return ret; From patchwork Fri Mar 12 15:04:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134861 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4775EC4332D for ; Fri, 12 Mar 2021 15:05:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E33B65002 for ; Fri, 12 Mar 2021 15:05:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232109AbhCLPFQ (ORCPT ); Fri, 12 Mar 2021 10:05:16 -0500 Received: from mail.kernel.org ([198.145.29.99]:43630 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231968AbhCLPFE (ORCPT ); Fri, 12 Mar 2021 10:05:04 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3333764F78; Fri, 12 Mar 2021 15:05:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561503; bh=5LCORNHhuwZS2ICNdojquekNf91WE/wxlob3/jaJgt0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nbJfG/cUnykzzJF9MsvLWu3AwBE5WVBkdkZvymhsvQGV0kUBm53bC/3+BkTZjwB54 v+VX/vsD6D4dZBnEspoBml0qjUloHQleypmlG18EBVZhnTjsBf2Fk6EfvxtNgLRQdC HEi3dgdXzhzrCX2kPd9gsiqv3W2vrQFIqEYNajeTkWOHQfYlY3CAavuK8kRAilyE/w 2MCZFfwORM7yTFnSgY+lSzEftzXVThufGRRRW+3FiU5ZlIEF7YMfrC+Vm9I2t82zG2 pui15lZvcm2dSV+NcQWcqtHDGhAO0e5DLzHeyC3njkiJd8kuAwgIE104yRhoCZpd9W MhluzF35cSNyg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 06/16] net: assert the rtnl lock is held when calling __netif_set_xps_queue Date: Fri, 12 Mar 2021 16:04:34 +0100 Message-Id: <20210312150444.355207-7-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add ASSERT_RTNL at the top of __netif_set_xps_queue and add a comment about holding the rtnl lock before the function. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- net/core/dev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/core/dev.c b/net/core/dev.c index 98a9c620f05a..24d8f059e2a6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2623,7 +2623,7 @@ static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index, return new_map; } -/* Must be called under cpus_read_lock */ +/* Must be called under rtnl_lock and cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, bool is_rxqs_map) { @@ -2635,6 +2635,8 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, struct xps_map *map, *new_map; unsigned int nr_ids; + ASSERT_RTNL(); + if (dev->num_tc) { /* Do not allow XPS on subordinate device directly */ num_tc = dev->num_tc; From patchwork Fri Mar 12 15:04:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134865 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA5DEC433DB for ; Fri, 12 Mar 2021 15:06:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C944164FD6 for ; Fri, 12 Mar 2021 15:06:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232145AbhCLPFl (ORCPT ); Fri, 12 Mar 2021 10:05:41 -0500 Received: from mail.kernel.org ([198.145.29.99]:43654 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231886AbhCLPFH (ORCPT ); Fri, 12 Mar 2021 10:05:07 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1C68B64F77; Fri, 12 Mar 2021 15:05:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561506; bh=AArPus5Hd7gC84uB/U3C9vk2sbdmMuD5IkZNWRvimDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a46RzIDRB7V3qMnDoz8dfFTpwEqWruWdPHt/qajL7SXLoSF91do22jej6jRkhpLGJ KM9vtr3DBBh8KbeiM8aBbS1asbvX8AaH8ip8FmRI3GqeV4sboRfDjXNgTzr7pbg0GD RQ0aTGbgwI3QLeAKyc/aEL3f9Pp1F3hJBud8TKKPPVUizmDvrS3Dn3cb0CVUoGcG2P 8L55RxGMHgvmKyfHZVeCONAONPOpkhTaNk6SC4noH8NOBWe1jKOJTtxuwXR1bEgE8Q EyxNqpwnkRE4t3RigPEc8daAId6JHgtP8BhWvn4nkhExPLq26y9HQ0tj9a8CbsuaSz H8M+VHJevyXNw== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 07/16] net: remove the xps possible_mask Date: Fri, 12 Mar 2021 16:04:35 +0100 Message-Id: <20210312150444.355207-8-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Remove the xps possible_mask. It was an optimization but we can just loop from 0 to nr_ids now that it is embedded in the xps dev_maps. That simplifies the code a bit. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- net/core/dev.c | 40 +++++++++++++--------------------------- net/core/net-sysfs.c | 4 ++-- 2 files changed, 15 insertions(+), 29 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 24d8f059e2a6..241f440b306a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2523,33 +2523,28 @@ static void reset_xps_maps(struct net_device *dev, kfree_rcu(dev_maps, rcu); } -static void clean_xps_maps(struct net_device *dev, const unsigned long *mask, +static void clean_xps_maps(struct net_device *dev, struct xps_dev_maps *dev_maps, u16 offset, u16 count, bool is_rxqs_map) { - unsigned int nr_ids = dev_maps->nr_ids; bool active = false; int i, j; - for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), j < nr_ids;) - active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, - count); + for (j = 0; j < dev_maps->nr_ids; j++) + active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) reset_xps_maps(dev, dev_maps, is_rxqs_map); if (!is_rxqs_map) { - for (i = offset + (count - 1); count--; i--) { + for (i = offset + (count - 1); count--; i--) netdev_queue_numa_node_write( - netdev_get_tx_queue(dev, i), - NUMA_NO_NODE); - } + netdev_get_tx_queue(dev, i), NUMA_NO_NODE); } } static void netif_reset_xps_queues(struct net_device *dev, u16 offset, u16 count) { - const unsigned long *possible_mask = NULL; struct xps_dev_maps *dev_maps; if (!static_key_false(&xps_needed)) @@ -2561,17 +2556,14 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (static_key_false(&xps_rxqs_needed)) { dev_maps = xmap_dereference(dev->xps_rxqs_map); if (dev_maps) - clean_xps_maps(dev, possible_mask, dev_maps, offset, - count, true); + clean_xps_maps(dev, dev_maps, offset, count, true); } dev_maps = xmap_dereference(dev->xps_cpus_map); if (!dev_maps) goto out_no_maps; - if (num_possible_cpus() > 1) - possible_mask = cpumask_bits(cpu_possible_mask); - clean_xps_maps(dev, possible_mask, dev_maps, offset, count, false); + clean_xps_maps(dev, dev_maps, offset, count, false); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2627,8 +2619,8 @@ static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index, int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, bool is_rxqs_map) { - const unsigned long *online_mask = NULL, *possible_mask = NULL; struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; + const unsigned long *online_mask = NULL; bool active = false, copy = false; int i, j, tci, numa_node_id = -2; int maps_sz, num_tc = 1, tc = 0; @@ -2658,10 +2650,8 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, nr_ids = dev->num_rx_queues; } else { maps_sz = XPS_CPU_DEV_MAPS_SIZE(num_tc); - if (num_possible_cpus() > 1) { + if (num_possible_cpus() > 1) online_mask = cpumask_bits(cpu_online_mask); - possible_mask = cpumask_bits(cpu_possible_mask); - } dev_maps = xmap_dereference(dev->xps_cpus_map); nr_ids = nr_cpu_ids; } @@ -2712,8 +2702,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, static_key_slow_inc_cpuslocked(&xps_rxqs_needed); } - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { /* copy maps belonging to foreign traffic classes */ for (i = tc, tci = j * num_tc; copy && i--; tci++) { /* fill in the new device map from the old device map */ @@ -2768,8 +2757,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (!dev_maps) goto out_no_old_maps; - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < dev_maps->nr_ids;) { + for (j = 0; j < dev_maps->nr_ids; j++) { for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) { map = xmap_dereference(dev_maps->attr_map[tci]); if (!map) @@ -2803,8 +2791,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, goto out_no_maps; /* removes tx-queue from unused CPUs/rx-queues */ - for (j = -1; j = netif_attrmask_next(j, possible_mask, dev_maps->nr_ids), - j < dev_maps->nr_ids;) { + for (j = 0; j < dev_maps->nr_ids; j++) { for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) active |= remove_xps_queue(dev_maps, tci, index); if (!netif_attr_test_mask(j, mask, dev_maps->nr_ids) || @@ -2824,8 +2811,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, return 0; error: /* remove any maps that we added */ - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { for (i = num_tc, tci = j * num_tc; i--; tci++) { new_map = xmap_dereference(new_dev_maps->attr_map[tci]); map = copy ? diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index bb08bdc88fa9..c762c435ff76 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1400,7 +1400,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; - for (j = -1; j = netif_attrmask_next(j, NULL, nr_ids), j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; @@ -1504,7 +1504,7 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) if (!dev_maps || tc >= dev_maps->num_tc) goto out_no_maps; - for (j = -1; j = netif_attrmask_next(j, NULL, nr_ids), j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; From patchwork Fri Mar 12 15:04:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134871 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69461C4332E for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37FA664FDC for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232191AbhCLPFo (ORCPT ); Fri, 12 Mar 2021 10:05:44 -0500 Received: from mail.kernel.org ([198.145.29.99]:43724 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231861AbhCLPFK (ORCPT ); Fri, 12 Mar 2021 10:05:10 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0AFF464F77; Fri, 12 Mar 2021 15:05:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561509; bh=q7O3DGm7YSwdgyPAPW4Ree6m04AlcNdO4tkr4bEDEFY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lfyj/3/Ls0AOcXw1eKoViL19vg7KMSxKshpfftLmXAA07o2FEcGV8zJJIxqe8Gh98 mw80zxXDaFD+/HtS2Ow/VTut+VIgs7gaH20p/aHScyWfUrwaIXTxfn4PPBkshs5jK+ lg5uFmuMGnX9TZZ32P9REEC1KE7XVl49Kul46Au65Dh8dprfZXtl3igsETsvMSRCh5 joucEBwf4lCIajdnOC4MsP+5TLFe07AYxYE0lYi/FrRmDm8UV1lQ+pB1V2BaJ71usC 4RJvhRhdHKmca1Aw182L6/QyaIgeeC15taYc5BPLX03zqLp1B4Br3hxpkUlbKkWJiD tslSPY1TeWoCw== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 08/16] net: move the xps maps to an array Date: Fri, 12 Mar 2021 16:04:36 +0100 Message-Id: <20210312150444.355207-9-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Move the xps maps (xps_cpus_map and xps_rxqs_map) to an array in net_device. That will simplify a lot the code removing the need for lots of if/else conditionals as the correct map will be available using its offset in the array. This should not modify the xps maps behaviour in any way. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- drivers/net/virtio_net.c | 2 +- include/linux/netdevice.h | 17 +++++---- net/core/dev.c | 73 +++++++++++++++++---------------------- net/core/net-sysfs.c | 6 ++-- 4 files changed, 46 insertions(+), 52 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index e97288dd6e5a..dde9bbcc5ff0 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2015,7 +2015,7 @@ static void virtnet_set_affinity(struct virtnet_info *vi) } virtqueue_set_affinity(vi->rq[i].vq, mask); virtqueue_set_affinity(vi->sq[i].vq, mask); - __netif_set_xps_queue(vi->dev, cpumask_bits(mask), i, false); + __netif_set_xps_queue(vi->dev, cpumask_bits(mask), i, XPS_CPUS); cpumask_clear(mask); } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 5c9e056a0e2d..bcd15a2d3ddc 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -754,6 +754,13 @@ struct rx_queue_attribute { const char *buf, size_t len); }; +/* XPS map type and offset of the xps map within net_device->xps_maps[]. */ +enum xps_map_type { + XPS_CPUS = 0, + XPS_RXQS, + XPS_MAPS_MAX, +}; + #ifdef CONFIG_XPS /* * This structure holds an XPS map which can be of variable length. The @@ -1773,8 +1780,7 @@ enum netdev_ml_priv_type { * @tx_queue_len: Max frames per queue allowed * @tx_global_lock: XXX: need comments on this one * @xdp_bulkq: XDP device bulk queue - * @xps_cpus_map: all CPUs map for XPS device - * @xps_rxqs_map: all RXQs map for XPS device + * @xps_maps: all CPUs/RXQs maps for XPS device * * @xps_maps: XXX: need comments on this one * @miniq_egress: clsact qdisc specific data for @@ -2070,8 +2076,7 @@ struct net_device { struct xdp_dev_bulk_queue __percpu *xdp_bulkq; #ifdef CONFIG_XPS - struct xps_dev_maps __rcu *xps_cpus_map; - struct xps_dev_maps __rcu *xps_rxqs_map; + struct xps_dev_maps __rcu *xps_maps[XPS_MAPS_MAX]; #endif #ifdef CONFIG_NET_CLS_ACT struct mini_Qdisc __rcu *miniq_egress; @@ -3701,7 +3706,7 @@ static inline void netif_wake_subqueue(struct net_device *dev, u16 queue_index) int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, u16 index); int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map); + u16 index, enum xps_map_type type); /** * netif_attr_test_mask - Test a CPU or Rx queue set in a mask @@ -3796,7 +3801,7 @@ static inline int netif_set_xps_queue(struct net_device *dev, static inline int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map) + u16 index, enum xps_map_type type) { return 0; } diff --git a/net/core/dev.c b/net/core/dev.c index 241f440b306a..dfdd476a6d67 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2511,31 +2511,34 @@ static bool remove_xps_queue_cpu(struct net_device *dev, static void reset_xps_maps(struct net_device *dev, struct xps_dev_maps *dev_maps, - bool is_rxqs_map) + enum xps_map_type type) { - if (is_rxqs_map) { - static_key_slow_dec_cpuslocked(&xps_rxqs_needed); - RCU_INIT_POINTER(dev->xps_rxqs_map, NULL); - } else { - RCU_INIT_POINTER(dev->xps_cpus_map, NULL); - } static_key_slow_dec_cpuslocked(&xps_needed); + if (type == XPS_RXQS) + static_key_slow_dec_cpuslocked(&xps_rxqs_needed); + + RCU_INIT_POINTER(dev->xps_maps[type], NULL); + kfree_rcu(dev_maps, rcu); } -static void clean_xps_maps(struct net_device *dev, - struct xps_dev_maps *dev_maps, u16 offset, u16 count, - bool is_rxqs_map) +static void clean_xps_maps(struct net_device *dev, enum xps_map_type type, + u16 offset, u16 count) { + struct xps_dev_maps *dev_maps; bool active = false; int i, j; + dev_maps = xmap_dereference(dev->xps_maps[type]); + if (!dev_maps) + return; + for (j = 0; j < dev_maps->nr_ids; j++) active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) - reset_xps_maps(dev, dev_maps, is_rxqs_map); + reset_xps_maps(dev, dev_maps, type); - if (!is_rxqs_map) { + if (type == XPS_CPUS) { for (i = offset + (count - 1); count--; i--) netdev_queue_numa_node_write( netdev_get_tx_queue(dev, i), NUMA_NO_NODE); @@ -2545,27 +2548,17 @@ static void clean_xps_maps(struct net_device *dev, static void netif_reset_xps_queues(struct net_device *dev, u16 offset, u16 count) { - struct xps_dev_maps *dev_maps; - if (!static_key_false(&xps_needed)) return; cpus_read_lock(); mutex_lock(&xps_map_mutex); - if (static_key_false(&xps_rxqs_needed)) { - dev_maps = xmap_dereference(dev->xps_rxqs_map); - if (dev_maps) - clean_xps_maps(dev, dev_maps, offset, count, true); - } - - dev_maps = xmap_dereference(dev->xps_cpus_map); - if (!dev_maps) - goto out_no_maps; + if (static_key_false(&xps_rxqs_needed)) + clean_xps_maps(dev, XPS_RXQS, offset, count); - clean_xps_maps(dev, dev_maps, offset, count, false); + clean_xps_maps(dev, XPS_CPUS, offset, count); -out_no_maps: mutex_unlock(&xps_map_mutex); cpus_read_unlock(); } @@ -2617,7 +2610,7 @@ static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index, /* Must be called under rtnl_lock and cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map) + u16 index, enum xps_map_type type) { struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; const unsigned long *online_mask = NULL; @@ -2644,15 +2637,15 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, } mutex_lock(&xps_map_mutex); - if (is_rxqs_map) { + + dev_maps = xmap_dereference(dev->xps_maps[type]); + if (type == XPS_RXQS) { maps_sz = XPS_RXQ_DEV_MAPS_SIZE(num_tc, dev->num_rx_queues); - dev_maps = xmap_dereference(dev->xps_rxqs_map); nr_ids = dev->num_rx_queues; } else { maps_sz = XPS_CPU_DEV_MAPS_SIZE(num_tc); if (num_possible_cpus() > 1) online_mask = cpumask_bits(cpu_online_mask); - dev_maps = xmap_dereference(dev->xps_cpus_map); nr_ids = nr_cpu_ids; } @@ -2685,7 +2678,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, tci = j * num_tc + tc; map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; - map = expand_xps_map(map, j, index, is_rxqs_map); + map = expand_xps_map(map, j, index, type == XPS_RXQS); if (!map) goto error; @@ -2698,7 +2691,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (!dev_maps) { /* Increment static keys at most once per type */ static_key_slow_inc_cpuslocked(&xps_needed); - if (is_rxqs_map) + if (type == XPS_RXQS) static_key_slow_inc_cpuslocked(&xps_rxqs_needed); } @@ -2727,7 +2720,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (pos == map->len) map->queues[map->len++] = index; #ifdef CONFIG_NUMA - if (!is_rxqs_map) { + if (type == XPS_CPUS) { if (numa_node_id == -2) numa_node_id = cpu_to_node(j); else if (numa_node_id != cpu_to_node(j)) @@ -2748,10 +2741,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, } } - if (is_rxqs_map) - rcu_assign_pointer(dev->xps_rxqs_map, new_dev_maps); - else - rcu_assign_pointer(dev->xps_cpus_map, new_dev_maps); + rcu_assign_pointer(dev->xps_maps[type], new_dev_maps); /* Cleanup old maps */ if (!dev_maps) @@ -2780,12 +2770,11 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, active = true; out_no_new_maps: - if (!is_rxqs_map) { + if (type == XPS_CPUS) /* update Tx queue numa node */ netdev_queue_numa_node_write(netdev_get_tx_queue(dev, index), (numa_node_id >= 0) ? numa_node_id : NUMA_NO_NODE); - } if (!dev_maps) goto out_no_maps; @@ -2803,7 +2792,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* free map if not active */ if (!active) - reset_xps_maps(dev, dev_maps, is_rxqs_map); + reset_xps_maps(dev, dev_maps, type); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2835,7 +2824,7 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, int ret; cpus_read_lock(); - ret = __netif_set_xps_queue(dev, cpumask_bits(mask), index, false); + ret = __netif_set_xps_queue(dev, cpumask_bits(mask), index, XPS_CPUS); cpus_read_unlock(); return ret; @@ -3985,7 +3974,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, if (!static_key_false(&xps_rxqs_needed)) goto get_cpus_map; - dev_maps = rcu_dereference(sb_dev->xps_rxqs_map); + dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_RXQS]); if (dev_maps) { int tci = sk_rx_queue_get(sk); @@ -3996,7 +3985,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, get_cpus_map: if (queue_index < 0) { - dev_maps = rcu_dereference(sb_dev->xps_cpus_map); + dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_CPUS]); if (dev_maps) { unsigned int tci = skb->sender_cpu - 1; diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index c762c435ff76..ca1f3b63cfad 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1388,7 +1388,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, } rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_cpus_map); + dev_maps = rcu_dereference(dev->xps_maps[XPS_CPUS]); nr_ids = dev_maps ? dev_maps->nr_ids : nr_cpu_ids; mask = bitmap_zalloc(nr_ids, GFP_KERNEL); @@ -1492,7 +1492,7 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) } rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_rxqs_map); + dev_maps = rcu_dereference(dev->xps_maps[XPS_RXQS]); nr_ids = dev_maps ? dev_maps->nr_ids : dev->num_rx_queues; mask = bitmap_zalloc(nr_ids, GFP_KERNEL); @@ -1566,7 +1566,7 @@ static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, } cpus_read_lock(); - err = __netif_set_xps_queue(dev, mask, index, true); + err = __netif_set_xps_queue(dev, mask, index, XPS_RXQS); cpus_read_unlock(); rtnl_unlock(); From patchwork Fri Mar 12 15:04:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134867 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D21AC433E6 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2714B65011 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232165AbhCLPFm (ORCPT ); Fri, 12 Mar 2021 10:05:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:43744 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231717AbhCLPFM (ORCPT ); Fri, 12 Mar 2021 10:05:12 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id DD6E364F78; Fri, 12 Mar 2021 15:05:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561512; bh=abzu7yjRwHt6cpL0vf0+1d43bsI36Val4Dxh/GZy/HA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=crw/51TS/5KfJVVa8fCC3KlwU3LG4q7dPE3KinbHU5gn0XEAIAzwb2Uk0Bxf1bPNi yalcNagwBIOEL1fUqKQhBpbVHAmSVTIPScKsUgWigFqQTSdmdy5PSmAk3+tnMWZQAG sd4VdUBMWoq555DpfCL07Cw6/O5thz4ElHUGt0TXCqkl1mgBwdSK5+2shJyGYOdUiC AZCyV60AQHhBpz91O0UOeKGG00hvhg04QxIFH1kgXWFyr9qbWyDqh+n9dtI6T9afAv HK05LiHw63xKfUmvZWlLUyG7APoD0w+qof1CAU/HheZ9iwQ8SqGO46s78JGp9an7Ty iAlKUpA1C3Xbg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 09/16] net: add an helper to copy xps maps to the new dev_maps Date: Fri, 12 Mar 2021 16:04:37 +0100 Message-Id: <20210312150444.355207-10-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch adds an helper, xps_copy_dev_maps, to copy maps from dev_maps to new_dev_maps at a given index. The logic should be the same, with an improved code readability and maintenance. Signed-off-by: Antoine Tenart --- net/core/dev.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index dfdd476a6d67..4d39938417c4 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2608,6 +2608,25 @@ static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index, return new_map; } +/* Copy xps maps at a given index */ +static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, + struct xps_dev_maps *new_dev_maps, int index, + int tc, bool skip_tc) +{ + int i, tci = index * dev_maps->num_tc; + struct xps_map *map; + + /* copy maps belonging to foreign traffic classes */ + for (i = 0; i < dev_maps->num_tc; i++, tci++) { + if (i == tc && skip_tc) + continue; + + /* fill in the new device map from the old device map */ + map = xmap_dereference(dev_maps->attr_map[tci]); + RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); + } +} + /* Must be called under rtnl_lock and cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, enum xps_map_type type) @@ -2696,23 +2715,16 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, } for (j = 0; j < nr_ids; j++) { - /* copy maps belonging to foreign traffic classes */ - for (i = tc, tci = j * num_tc; copy && i--; tci++) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); - } + bool skip_tc = false; - /* We need to explicitly update tci as prevous loop - * could break out early if dev_maps is NULL. - */ tci = j * num_tc + tc; - if (netif_attr_test_mask(j, mask, nr_ids) && netif_attr_test_online(j, online_mask, nr_ids)) { /* add tx-queue to CPU/rx-queue maps */ int pos = 0; + skip_tc = true; + map = xmap_dereference(new_dev_maps->attr_map[tci]); while ((pos < map->len) && (map->queues[pos] != index)) pos++; @@ -2727,18 +2739,11 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, numa_node_id = -1; } #endif - } else if (copy) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); } - /* copy maps belonging to foreign traffic classes */ - for (i = num_tc - tc, tci++; copy && --i; tci++) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); - } + if (copy) + xps_copy_dev_maps(dev_maps, new_dev_maps, j, tc, + skip_tc); } rcu_assign_pointer(dev->xps_maps[type], new_dev_maps); From patchwork Fri Mar 12 15:04:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134869 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64ABEC4332B for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4DCD364FE2 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232223AbhCLPFq (ORCPT ); Fri, 12 Mar 2021 10:05:46 -0500 Received: from mail.kernel.org ([198.145.29.99]:43764 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232067AbhCLPFP (ORCPT ); Fri, 12 Mar 2021 10:05:15 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9783264F78; Fri, 12 Mar 2021 15:05:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561515; bh=t7v9Cnk0wuYPToBbUmRLJDu4jjX6yO2eMC/jsZiBq/k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XLXvDXK96sDsCd4pWqwnqKJRP0jFCMprcOnt8GVAgh1Gbm7DnqTsCB6ht9hMcI08G Es086gL2ON0cTo7L0Ay/4ZK0WlnSTQwj0hH/Q6zyj2pjNCgPAiVXPHkjTZPXf73GIu UlMirfY2Pkn5GSDT6nocx1I13QJvGHLqdOE3nrgFrMmkVANEYOmQP2DhI6eMTkH/n4 0Rc18NYIBWM802+NtBkTugtyueCFrf8Si7y3zdNXRUKGFRt/LQogZEr6+FFiUGjK0a 3j0PqQco6fNn17e3JJReHtVS3KndQT1cPeWbkCb2EV+ps09MFK9jA5Mj4Tlth2ckG5 Qbzkc/YYGtFTg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 10/16] net: improve queue removal readability in __netif_set_xps_queue Date: Fri, 12 Mar 2021 16:04:38 +0100 Message-Id: <20210312150444.355207-11-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Improve the readability of the loop removing tx-queue from unused CPUs/rx-queues in __netif_set_xps_queue. The change should only be cosmetic. Signed-off-by: Antoine Tenart --- net/core/dev.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 4d39938417c4..052797ca65f6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2786,13 +2786,16 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* removes tx-queue from unused CPUs/rx-queues */ for (j = 0; j < dev_maps->nr_ids; j++) { - for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) - active |= remove_xps_queue(dev_maps, tci, index); - if (!netif_attr_test_mask(j, mask, dev_maps->nr_ids) || - !netif_attr_test_online(j, online_mask, dev_maps->nr_ids)) - active |= remove_xps_queue(dev_maps, tci, index); - for (i = dev_maps->num_tc - tc, tci++; --i; tci++) + tci = j * dev_maps->num_tc; + + for (i = 0; i < dev_maps->num_tc; i++, tci++) { + if (i == tc && + netif_attr_test_mask(j, mask, dev_maps->nr_ids) && + netif_attr_test_online(j, online_mask, dev_maps->nr_ids)) + continue; + active |= remove_xps_queue(dev_maps, tci, index); + } } /* free map if not active */ From patchwork Fri Mar 12 15:04:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134873 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 894FCC4332D for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5DBF965011 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232241AbhCLPFq (ORCPT ); Fri, 12 Mar 2021 10:05:46 -0500 Received: from mail.kernel.org ([198.145.29.99]:43784 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232117AbhCLPFS (ORCPT ); Fri, 12 Mar 2021 10:05:18 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 76FA864F77; Fri, 12 Mar 2021 15:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561518; bh=93q1yME4SnPArwRej82dCQACOb9FzkuLHbN6O+/CdeE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FoAqWr3T5pJZEQe7GRfgZc6zpueZcLpw4RqcMKvZrkxl0K7mN5fql2TDU6ZNPyKdd fzU9JEmdPTsvxyqozOyW02fOmgYQaoLzBWVGVpqAlVI2nlH0YSp+q2Y80xeMBfu3+8 NOJhxNmqouoC6PbodJ75hQnX0o4kxKmSNmAbs5reSP2MagZSWAabMiMxIlODNWxQlt CrbblsuiWdCWRbp0EoIe7PWp1bm8F7Kw0ElOHihCi1rBQfcvPcIgn9bJLcO1vdawvi rFVm8YGvzf7WDcM6/iXNbsQ/KzIQEVjJNdIlJEFoOckhwSuWyne1aDK4yxqMmn22I/ fj2ra+PIPAynA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 11/16] net-sysfs: move the rtnl unlock up in the xps show helpers Date: Fri, 12 Mar 2021 16:04:39 +0100 Message-Id: <20210312150444.355207-12-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Now that nr_ids and num_tc are stored in the xps dev_maps, which are RCU protected, we do not have the need to protect the maps in the rtnl lock. Move the rtnl unlock up so we reduce the rtnl locking section. We also increase the reference count on the subordinate device if any, as we don't want this device to be freed while we use it (now that the rtnl lock isn't protecting it in the whole function). Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index ca1f3b63cfad..094fea082649 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1383,10 +1383,14 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, tc = netdev_txq_to_tc(dev, index); if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; + rtnl_unlock(); + return -EINVAL; } + /* Make sure the subordinate device can't be freed */ + get_device(&dev->dev); + rtnl_unlock(); + rcu_read_lock(); dev_maps = rcu_dereference(dev->xps_maps[XPS_CPUS]); nr_ids = dev_maps ? dev_maps->nr_ids : nr_cpu_ids; @@ -1417,8 +1421,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, } out_no_maps: rcu_read_unlock(); - - rtnl_unlock(); + put_device(&dev->dev); len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); bitmap_free(mask); @@ -1426,8 +1429,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, err_rcu_unlock: rcu_read_unlock(); -err_rtnl_unlock: - rtnl_unlock(); + put_device(&dev->dev); return ret; } @@ -1486,10 +1488,9 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) return restart_syscall(); tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } + rtnl_unlock(); + if (tc < 0) + return -EINVAL; rcu_read_lock(); dev_maps = rcu_dereference(dev->xps_maps[XPS_RXQS]); @@ -1522,8 +1523,6 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) out_no_maps: rcu_read_unlock(); - rtnl_unlock(); - len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); bitmap_free(mask); @@ -1531,8 +1530,6 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) err_rcu_unlock: rcu_read_unlock(); -err_rtnl_unlock: - rtnl_unlock(); return ret; } From patchwork Fri Mar 12 15:04:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134877 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CA7CC43331 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8810C64FDC for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232252AbhCLPFr (ORCPT ); Fri, 12 Mar 2021 10:05:47 -0500 Received: from mail.kernel.org ([198.145.29.99]:43808 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231694AbhCLPFV (ORCPT ); Fri, 12 Mar 2021 10:05:21 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D45464F78; Fri, 12 Mar 2021 15:05:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561520; bh=HglvgNnzmFDnsMQHBlOkGPxvrTxsc96RCxg7FlJWmEU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XJbVmTg/6m3n6dlX1EeIKEe0BZXtEmKGlxSUlSl4/gHI9fFeocC1r4T1fVupGMDqr BcrK1femPDUYIIMcs512QJDH6D07wOwTN6N62aQ0ZhNhc2mjS2AfLbkBgtzLhPTzHn pvMAdZB+8x4FszP2gn/+2eOQAOOFISiTzLXqLn/+oMQiCjLchCAsRVRHzGttxbJHZ/ 0Lrm/3q9Zg6/vFyAqC3mwCwYndxbhXcZ5/yya1Cu07n6cW0YSXWfmwg/QAJsfEEQbc OyadaiFomdmBtXJqMWzzQj1aQ2gI+OjHn5/Xsl+xmJQrftegkurlP8z6Mr6MqNX8VV 2/ropg6hPmaPw== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 12/16] net-sysfs: move the xps cpus/rxqs retrieval in a common function Date: Fri, 12 Mar 2021 16:04:40 +0100 Message-Id: <20210312150444.355207-13-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Most of the xps_cpus_show and xps_rxqs_show functions share the same logic. Having it in two different functions does not help maintenance. This patch moves their common logic into a new function, xps_queue_show, to improve this. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 125 +++++++++++++++++-------------------------- 1 file changed, 48 insertions(+), 77 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 094fea082649..562a42fcd437 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1361,44 +1361,27 @@ static const struct attribute_group dql_group = { #endif /* CONFIG_BQL */ #ifdef CONFIG_XPS -static ssize_t xps_cpus_show(struct netdev_queue *queue, - char *buf) +static ssize_t xps_queue_show(struct net_device *dev, unsigned int index, + int tc, char *buf, enum xps_map_type type) { - struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - unsigned int index, nr_ids; - int j, len, ret, tc = 0; unsigned long *mask; - - if (!netif_is_multiqueue(dev)) - return -ENOENT; - - index = get_netdev_queue_index(queue); - - if (!rtnl_trylock()) - return restart_syscall(); - - /* If queue belongs to subordinate dev use its map */ - dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; - - tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - rtnl_unlock(); - return -EINVAL; - } - - /* Make sure the subordinate device can't be freed */ - get_device(&dev->dev); - rtnl_unlock(); + unsigned int nr_ids; + int j, len; rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_maps[XPS_CPUS]); - nr_ids = dev_maps ? dev_maps->nr_ids : nr_cpu_ids; + dev_maps = rcu_dereference(dev->xps_maps[type]); + + /* Default to nr_cpu_ids/dev->num_rx_queues and do not just return 0 + * when dev_maps hasn't been allocated yet, to be backward compatible. + */ + nr_ids = dev_maps ? dev_maps->nr_ids : + (type == XPS_CPUS ? nr_cpu_ids : dev->num_rx_queues); mask = bitmap_zalloc(nr_ids, GFP_KERNEL); if (!mask) { - ret = -ENOMEM; - goto err_rcu_unlock; + rcu_read_unlock(); + return -ENOMEM; } if (!dev_maps || tc >= dev_maps->num_tc) @@ -1421,16 +1404,44 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, } out_no_maps: rcu_read_unlock(); - put_device(&dev->dev); len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); bitmap_free(mask); + return len < PAGE_SIZE ? len : -EINVAL; +} + +static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) +{ + struct net_device *dev = queue->dev; + unsigned int index; + int len, tc; + + if (!netif_is_multiqueue(dev)) + return -ENOENT; + + index = get_netdev_queue_index(queue); + + if (!rtnl_trylock()) + return restart_syscall(); + + /* If queue belongs to subordinate dev use its map */ + dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; + + tc = netdev_txq_to_tc(dev, index); + if (tc < 0) { + rtnl_unlock(); + return -EINVAL; + } + + /* Make sure the subordinate device can't be freed */ + get_device(&dev->dev); + rtnl_unlock(); + + len = xps_queue_show(dev, index, tc, buf, XPS_CPUS); -err_rcu_unlock: - rcu_read_unlock(); put_device(&dev->dev); - return ret; + return len; } static ssize_t xps_cpus_store(struct netdev_queue *queue, @@ -1477,10 +1488,8 @@ static struct netdev_queue_attribute xps_cpus_attribute __ro_after_init static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) { struct net_device *dev = queue->dev; - struct xps_dev_maps *dev_maps; - unsigned int index, nr_ids; - int j, len, ret, tc = 0; - unsigned long *mask; + unsigned int index; + int tc; index = get_netdev_queue_index(queue); @@ -1492,45 +1501,7 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) if (tc < 0) return -EINVAL; - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_maps[XPS_RXQS]); - nr_ids = dev_maps ? dev_maps->nr_ids : dev->num_rx_queues; - - mask = bitmap_zalloc(nr_ids, GFP_KERNEL); - if (!mask) { - ret = -ENOMEM; - goto err_rcu_unlock; - } - - if (!dev_maps || tc >= dev_maps->num_tc) - goto out_no_maps; - - for (j = 0; j < nr_ids; j++) { - int i, tci = j * dev_maps->num_tc + tc; - struct xps_map *map; - - map = rcu_dereference(dev_maps->attr_map[tci]); - if (!map) - continue; - - for (i = map->len; i--;) { - if (map->queues[i] == index) { - set_bit(j, mask); - break; - } - } - } -out_no_maps: - rcu_read_unlock(); - - len = bitmap_print_to_pagebuf(false, buf, mask, nr_ids); - bitmap_free(mask); - - return len < PAGE_SIZE ? len : -EINVAL; - -err_rcu_unlock: - rcu_read_unlock(); - return ret; + return xps_queue_show(dev, index, tc, buf, XPS_RXQS); } static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, From patchwork Fri Mar 12 15:04:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134875 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89D84C43332 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 74EA364FD6 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232262AbhCLPFs (ORCPT ); Fri, 12 Mar 2021 10:05:48 -0500 Received: from mail.kernel.org ([198.145.29.99]:43828 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232119AbhCLPFX (ORCPT ); Fri, 12 Mar 2021 10:05:23 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id EDD2F64F80; Fri, 12 Mar 2021 15:05:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561523; bh=OtJELyvwxwbcqivCTKnOesWBm51y5qBsGR0qwAtSA/M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r/zvwTXlLVqfIa8sQNbIBjRKKH3+9r7E0q3KEoDT+L9PBNaqfyNhqydDtdqwZXNxC 0xzwFiXt62tOptdVzNBjAlsthEqf/tYzq7pn8u8jQmd2Q+51QD+vSjzsjtgJHjhCq/ 2woB6fZ78uzfIzp2vrq9wvF+9RYlnEvQE1JcYqZaJN1GNi2cDJfVUtEDNanaLflk3c ylKKuKGngaDwU3zyVpaNzVXSlmhcRveMTuQ/7jlr8Vs0eyp9FeWs7oqZQDLrjB8ofZ 1y2CFCnmPuCAj3IHHfb8mCOl/qyAZhD2Nk2quKdYz54AhqvcO3obE92swbOIjAW5J/ rGtYUThlK4Xuw== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 13/16] net: fix use after free in xps Date: Fri, 12 Mar 2021 16:04:41 +0100 Message-Id: <20210312150444.355207-14-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org When setting up an new dev_maps in __netif_set_xps_queue, we remove and free maps from unused CPUs/rx-queues near the end of the function; by calling remove_xps_queue. However it's possible those maps are also part of the old not-freed-yet dev_maps, which might be used concurrently. When that happens, a map can be freed while its corresponding entry in the old dev_maps table isn't NULLed, leading to: BUG: KASAN: use-after-free in xps_queue_show+0x469/0x480 This fixes the map freeing logic for unused CPUs/rx-queues, to also NULL the map entries from the old dev_maps table. Signed-off-by: Antoine Tenart --- net/core/dev.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 052797ca65f6..748e377c7fe3 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2460,7 +2460,7 @@ static DEFINE_MUTEX(xps_map_mutex); rcu_dereference_protected((P), lockdep_is_held(&xps_map_mutex)) static bool remove_xps_queue(struct xps_dev_maps *dev_maps, - int tci, u16 index) + struct xps_dev_maps *old_maps, int tci, u16 index) { struct xps_map *map = NULL; int pos; @@ -2479,6 +2479,8 @@ static bool remove_xps_queue(struct xps_dev_maps *dev_maps, break; } + if (old_maps) + RCU_INIT_POINTER(old_maps->attr_map[tci], NULL); RCU_INIT_POINTER(dev_maps->attr_map[tci], NULL); kfree_rcu(map, rcu); return false; @@ -2499,7 +2501,7 @@ static bool remove_xps_queue_cpu(struct net_device *dev, int i, j; for (i = count, j = offset; i--; j++) { - if (!remove_xps_queue(dev_maps, tci, j)) + if (!remove_xps_queue(dev_maps, NULL, tci, j)) break; } @@ -2631,7 +2633,7 @@ static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, enum xps_map_type type) { - struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; + struct xps_dev_maps *dev_maps, *new_dev_maps = NULL, *old_dev_maps = NULL; const unsigned long *online_mask = NULL; bool active = false, copy = false; int i, j, tci, numa_node_id = -2; @@ -2768,7 +2770,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, } } - kfree_rcu(dev_maps, rcu); + old_dev_maps = copy ? dev_maps : NULL; out_no_old_maps: dev_maps = new_dev_maps; @@ -2794,10 +2796,14 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, netif_attr_test_online(j, online_mask, dev_maps->nr_ids)) continue; - active |= remove_xps_queue(dev_maps, tci, index); + active |= remove_xps_queue(dev_maps, old_dev_maps, tci, + index); } } + if (old_dev_maps) + kfree_rcu(old_dev_maps, rcu); + /* free map if not active */ if (!active) reset_xps_maps(dev, dev_maps, type); From patchwork Fri Mar 12 15:04:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134879 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B257BC43333 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9733964F80 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231976AbhCLPFt (ORCPT ); Fri, 12 Mar 2021 10:05:49 -0500 Received: from mail.kernel.org ([198.145.29.99]:43848 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232127AbhCLPF0 (ORCPT ); Fri, 12 Mar 2021 10:05:26 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id A522664F77; Fri, 12 Mar 2021 15:05:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561526; bh=HxjTi4CD5M5CaHKrmYW6GLRrYkRcRRecOXvwvY5SycA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=svR0gzcnVZsBKkm7/hyP5ozparzvN1yR2IXDOh1x6hIJYsOZ1gNBMSFtBTNvd/84F yApj2Bfye1NHwLlvcIho376/FZo5jZD+l6bT9cXqWKJ36lX3f54Xp9tEW+mlOqd41f 0Os3/nWZAUY7k900jzUL5VHd2BG767RzAsLOvfw2BwSZ+bQbyX/kNYHErrI0gXE9aP jS1EgsiXobCIfg8uWjmMyFvy45nAAwXbKqmZG/4ovY+5ALCM79Wx2e6wkLuNAPCdgY U5Wo6xsLCt4JIksxzE/neyLTDVMJ8qepP2jQS3uEC3mkKvZjY0788//pexzENj0O0A ZZCZg3tjXSmdw== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 14/16] net: NULL the old xps map entries when freeing them Date: Fri, 12 Mar 2021 16:04:42 +0100 Message-Id: <20210312150444.355207-15-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In __netif_set_xps_queue, old map entries from the old dev_maps are freed but their corresponding entry in the old dev_maps aren't NULLed. Fix this. Signed-off-by: Antoine Tenart --- net/core/dev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/core/dev.c b/net/core/dev.c index 748e377c7fe3..4f1b38de00ac 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2766,6 +2766,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, continue; } + RCU_INIT_POINTER(dev_maps->attr_map[tci], NULL); kfree_rcu(map, rcu); } } From patchwork Fri Mar 12 15:04:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134883 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04A28C432C3 for ; Fri, 12 Mar 2021 15:06:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBF9264FDC for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232283AbhCLPFt (ORCPT ); Fri, 12 Mar 2021 10:05:49 -0500 Received: from mail.kernel.org ([198.145.29.99]:43868 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231670AbhCLPF3 (ORCPT ); Fri, 12 Mar 2021 10:05:29 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 528A864F77; Fri, 12 Mar 2021 15:05:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561528; bh=9doiaF75Et92SQa+Sn+F7w3D+qlHs7O3DPoohGc9wf4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lGrKdL+yfH0HjCuvKgKQk74Ui+ioB802HXUizPjamyhzeAFq/kDi39K/dumX/vsJ3 ZrMl3cYnwxB0wUmWlkhx7TuOAjX35DnOiMAZhT2g718sBBn2TFksR4wjR9d5kepZqz b6ELd0G3B3qD9JsIOGjhTFoqaBlvJ+5ZC6chBS836cK4Vae2Iequ61jKFxp/359PMi YALCZRmTilbstqDybfTBxhtXGDqTs4u2prTttQywhKd8gW+TGCapFSuD2vcic8yLJs 9W0m1CMXiUmWjIvg6xCJlg01Cxk9ctHTREyaLmICB6j2amQQjSZ2yjnCSoklxOaJWM HFAynA6xsHDdQ== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com, saeedm@nvidia.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 15/16] net/mlx5e: take the rtnl lock when calling netif_set_xps_queue Date: Fri, 12 Mar 2021 16:04:43 +0100 Message-Id: <20210312150444.355207-16-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org netif_set_xps_queue must be called with the rtnl lock taken, and this is now enforced using ASSERT_RTNL(). mlx5e_attach_netdev was taking the lock conditionally, fix this by taking the rtnl lock all the time. Signed-off-by: Antoine Tenart --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index ec2fcb2a2977..96cba86b9f0d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -5557,7 +5557,6 @@ static void mlx5e_update_features(struct net_device *netdev) int mlx5e_attach_netdev(struct mlx5e_priv *priv) { - const bool take_rtnl = priv->netdev->reg_state == NETREG_REGISTERED; const struct mlx5e_profile *profile = priv->profile; int max_nch; int err; @@ -5578,15 +5577,11 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv) * 2. Set our default XPS cpumask. * 3. Build the RQT. * - * rtnl_lock is required by netif_set_real_num_*_queues in case the - * netdev has been registered by this point (if this function was called - * in the reload or resume flow). + * rtnl_lock is required by netif_set_xps_queue. */ - if (take_rtnl) - rtnl_lock(); + rtnl_lock(); err = mlx5e_num_channels_changed(priv); - if (take_rtnl) - rtnl_unlock(); + rtnl_unlock(); if (err) goto out; From patchwork Fri Mar 12 15:04:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12134881 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCAFBC4321A for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ABAA765007 for ; Fri, 12 Mar 2021 15:06:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232289AbhCLPFu (ORCPT ); Fri, 12 Mar 2021 10:05:50 -0500 Received: from mail.kernel.org ([198.145.29.99]:43890 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232130AbhCLPFb (ORCPT ); Fri, 12 Mar 2021 10:05:31 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 18F7F64F78; Fri, 12 Mar 2021 15:05:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615561531; bh=SmBFsh200oLrNKpbhIObeffm9Jh3UMHhD/+jxp/SARk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EwwRWj9cOPgHrQdRmEhFTg0DRd89rv+QzTU6JUClKvD8x+fGy0ragzRH8YcFgtqRO oM5xU7ngsbA3qghCH+s4z1UyZ8cPYG9zUXyAqhQ7F68Xldd8GR54Ewd8h1pQ8B/gth QD5lIHajD5e6kIL6Yqe/53CPWU7bJMgnKqhEe+a4fqNE6j2tm29d8kiPfvZI94S4Rf sHa4OIrS+19rT0+VlLRqZ60V3NQafPRV/LarcIEftSsq/3wOzpk5AEr16MUv6kigOk c5lh0tEpQJCq26NAe6xIbt0fBuCvG74bngz73Yxvq8OtvZrt5rfggrWygGlXL25rfO NExWcq+tsd3qA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com, mst@redhat.com, jasowang@redhat.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next v3 16/16] virtio_net: take the rtnl lock when calling virtnet_set_affinity Date: Fri, 12 Mar 2021 16:04:44 +0100 Message-Id: <20210312150444.355207-17-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210312150444.355207-1-atenart@kernel.org> References: <20210312150444.355207-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org netif_set_xps_queue must be called with the rtnl lock taken, and this is now enforced using ASSERT_RTNL(). In virtio_net, netif_set_xps_queue is called by virtnet_set_affinity. As this function can be called from an ethtool helper, we can't take the rtnl lock directly in it. Instead we take the rtnl lock when calling virtnet_set_affinity when the rtnl lock isn't taken already. Signed-off-by: Antoine Tenart --- drivers/net/virtio_net.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index dde9bbcc5ff0..54d2277f6c98 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2027,7 +2027,9 @@ static int virtnet_cpu_online(unsigned int cpu, struct hlist_node *node) { struct virtnet_info *vi = hlist_entry_safe(node, struct virtnet_info, node); + rtnl_lock(); virtnet_set_affinity(vi); + rtnl_unlock(); return 0; } @@ -2035,7 +2037,9 @@ static int virtnet_cpu_dead(unsigned int cpu, struct hlist_node *node) { struct virtnet_info *vi = hlist_entry_safe(node, struct virtnet_info, node_dead); + rtnl_lock(); virtnet_set_affinity(vi); + rtnl_unlock(); return 0; } @@ -2883,7 +2887,9 @@ static int init_vqs(struct virtnet_info *vi) goto err_free; get_online_cpus(); + rtnl_lock(); virtnet_set_affinity(vi); + rtnl_unlock(); put_online_cpus(); return 0;