From patchwork Thu Jan 28 14:43:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053807 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4F97C433E6 for ; Thu, 28 Jan 2021 14:45:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F18C64DFA for ; Thu, 28 Jan 2021 14:45:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232143AbhA1Op3 (ORCPT ); Thu, 28 Jan 2021 09:45:29 -0500 Received: from mail.kernel.org ([198.145.29.99]:49412 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231489AbhA1Oow (ORCPT ); Thu, 28 Jan 2021 09:44:52 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id DAE6364DE2; Thu, 28 Jan 2021 14:44:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845051; bh=2arO9EYEEjPZwb3AGMNnI3tjt5dOsCgVOQaz+CdTdUk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tIXf7rZM+poV0MNpKEDZbOwPEAjTY8BNQhGSJzjhs4imGU0e+08ST/jcxzjS/laH5 lIkqthUZfTcD8mDqJNfxbji4fYV8cDbxy4+xCxA5CQOoKsiu8iFHEoK5btoSS7W7UO aSRtuAXfhFy3gItq2VMnhh5CR8Zt491PJxWi03B8wM4mijPIS12I1ZHM8WjZTSdKJu 9GZppGK/jdzt7o8O1pnqxPMTK9tYBKl+iwEIjZkJL2ki7f3/untlUEjRsQ0K9HnyR7 pRfUmQnsaIhO5uXMn51oi8sH+6E4HABCp+hlE4CSMvES5K0+8edudW6wcuIAWpb8+q ciaoQxhswFLjQ== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 01/11] net-sysfs: convert xps_cpus_show to bitmap_zalloc Date: Thu, 28 Jan 2021 15:43:55 +0100 Message-Id: <20210128144405.4157244-2-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use bitmap_zalloc instead if zalloc_cpumask_var in xps_cpus_show to align with xps_rxqs_show. This will improve maintenance and allow us to factorize the two functions. The function should behave the same. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index daf502c13d6d..e052fc5f7e94 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1320,8 +1320,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, int cpu, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - cpumask_var_t mask; - unsigned long index; + unsigned long *mask, index; if (!netif_is_multiqueue(dev)) return -ENOENT; @@ -1349,7 +1348,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, } } - if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) { + mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); + if (!mask) { ret = -ENOMEM; goto err_rtnl_unlock; } @@ -1367,7 +1367,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, for (i = map->len; i--;) { if (map->queues[i] == index) { - cpumask_set_cpu(cpu, mask); + set_bit(cpu, mask); break; } } @@ -1377,8 +1377,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, rtnl_unlock(); - len = snprintf(buf, PAGE_SIZE, "%*pb\n", cpumask_pr_args(mask)); - free_cpumask_var(mask); + len = bitmap_print_to_pagebuf(false, buf, mask, nr_cpu_ids); + bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; err_rtnl_unlock: From patchwork Thu Jan 28 14:43:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053839 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D7C3C433E9 for ; Thu, 28 Jan 2021 14:47:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5959464DEC for ; Thu, 28 Jan 2021 14:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232096AbhA1OrC (ORCPT ); Thu, 28 Jan 2021 09:47:02 -0500 Received: from mail.kernel.org ([198.145.29.99]:49438 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231915AbhA1Ooz (ORCPT ); Thu, 28 Jan 2021 09:44:55 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id D329D64DE0; Thu, 28 Jan 2021 14:44:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845054; bh=51ZhoMUTg79iyFfvNiHV8sojQPSnhzYx6O57vmjqBNY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ex7p/J2qUYhRn+D9PTgxpX4tZx7LSNd8kk+XNeYm/tPI4bJHqjAc6+f51yw/PhVuP W9CKBe4EYuGN3qbY8uiNLDS+qQ23V9wKWnlhE2FAiStapgjr0jsX270wUFMXWvPHxo j9mPSBtSQbhaXLSywOmspXt9FdvRAHcSASIR8qyhprwSKd44W9zeSJyi7Z4y4nJGGO zBVKGjiJvkoISKWNpA7mlu3xgYvWs7+yg3EeVJVmMkMVu/cPSBb33arS8NwDsv++/L 06/pblEdPLh3ystGAJ+L8CAEL16bNrXSdh0OV8ZWaG8NGPwRCcidclipb2ZhCrTcej VO+KqJQS/xsWQ== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 02/11] net-sysfs: store the return of get_netdev_queue_index in an unsigned int Date: Thu, 28 Jan 2021 15:43:56 +0100 Message-Id: <20210128144405.4157244-3-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In net-sysfs, get_netdev_queue_index returns an unsigned int. Some of its callers use an unsigned long to store the returned value. Update the code to be consistent, this should only be cosmetic. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index e052fc5f7e94..5a39e9b38e5f 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1320,7 +1320,8 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, int cpu, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; if (!netif_is_multiqueue(dev)) return -ENOENT; @@ -1390,7 +1391,7 @@ static ssize_t xps_cpus_store(struct netdev_queue *queue, const char *buf, size_t len) { struct net_device *dev = queue->dev; - unsigned long index; + unsigned int index; cpumask_var_t mask; int err; @@ -1432,7 +1433,8 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; struct xps_dev_maps *dev_maps; - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; index = get_netdev_queue_index(queue); @@ -1494,7 +1496,8 @@ static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, { struct net_device *dev = queue->dev; struct net *net = dev_net(dev); - unsigned long *mask, index; + unsigned long *mask; + unsigned int index; int err; if (!ns_capable(net->user_ns, CAP_NET_ADMIN)) From patchwork Thu Jan 28 14:43:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053837 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3700AC433E0 for ; Thu, 28 Jan 2021 14:47:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0AB9B64DF8 for ; Thu, 28 Jan 2021 14:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232322AbhA1Oqv (ORCPT ); Thu, 28 Jan 2021 09:46:51 -0500 Received: from mail.kernel.org ([198.145.29.99]:49458 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232035AbhA1Oo6 (ORCPT ); Thu, 28 Jan 2021 09:44:58 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id E452C64DE5; Thu, 28 Jan 2021 14:44:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845057; bh=pLx1XznkaVjYaQOKMNx9wnZaulskH2iom1amrmS2WF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VZqBBoXr/JUCrXHwPoAgvFyJY5YS2T2ZkYnPN7Zso/nSVCF0YYiUKZ66HRcbH20f5 /3GRL44kCJ5e/Yf2msxJ9ofsn3ZrcpugMZGes1dkJrdfEYaVJBdW9+cb3To6/SPyQA BdnP0ra8hJfcfzkOdarT3H54UuFg9vfRDu8HmSbvTo5vfhF67ZebOox3IDhnRWORw2 C4JUTaMwFB8VRYBXj3E+jAD5n6WQ1jC4sZQqafEhEUx76tLFGeuWYX6+NP9a5dFnMg KWjkOTOAClkctzz+EdIVvaJobkT580G1og38Dj9eJWSgKrB7rIcSib53qTjBOUbxfX 5IqU69dmb83PA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 03/11] net-sysfs: move the xps cpus/rxqs retrieval in a common function Date: Thu, 28 Jan 2021 15:43:57 +0100 Message-Id: <20210128144405.4157244-4-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Most of the xps_cpus_show and xps_rxqs_show functions share the same logic. Having it in two different functions does not help maintenance and we can already see small implementation differences. This should not be the case and this patch moves their common logic into a new function, xps_queue_show, to improve maintenance. While the rtnl lock could be held in the new xps_queue_show, it is still held in xps_cpus_show and xps_rxqs_show as this is an important information when looking at those two functions. This does not add complexity. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 168 ++++++++++++++++++++----------------------- 1 file changed, 79 insertions(+), 89 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 5a39e9b38e5f..6e6bc05181f6 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1314,77 +1314,98 @@ static const struct attribute_group dql_group = { #endif /* CONFIG_BQL */ #ifdef CONFIG_XPS -static ssize_t xps_cpus_show(struct netdev_queue *queue, - char *buf) +/* Should be called with the rtnl lock held. */ +static int xps_queue_show(struct net_device *dev, unsigned long **mask, + unsigned int index, bool is_rxqs_map) { - int cpu, len, ret, num_tc = 1, tc = 0; - struct net_device *dev = queue->dev; + const unsigned long *possible_mask = NULL; + int j, num_tc = 0, tc = 0, ret = 0; struct xps_dev_maps *dev_maps; - unsigned long *mask; - unsigned int index; - - if (!netif_is_multiqueue(dev)) - return -ENOENT; - - index = get_netdev_queue_index(queue); - - if (!rtnl_trylock()) - return restart_syscall(); + unsigned int nr_ids; if (dev->num_tc) { /* Do not allow XPS on subordinate device directly */ num_tc = dev->num_tc; - if (num_tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } + if (num_tc < 0) + return -EINVAL; /* If queue belongs to subordinate dev use its map */ dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } + if (tc < 0) + return -EINVAL; } - mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); - if (!mask) { - ret = -ENOMEM; - goto err_rtnl_unlock; + rcu_read_lock(); + + if (is_rxqs_map) { + dev_maps = rcu_dereference(dev->xps_rxqs_map); + nr_ids = dev->num_rx_queues; + } else { + dev_maps = rcu_dereference(dev->xps_cpus_map); + nr_ids = nr_cpu_ids; + if (num_possible_cpus() > 1) + possible_mask = cpumask_bits(cpu_possible_mask); } + if (!dev_maps) + goto rcu_unlock; - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_cpus_map); - if (dev_maps) { - for_each_possible_cpu(cpu) { - int i, tci = cpu * num_tc + tc; - struct xps_map *map; - - map = rcu_dereference(dev_maps->attr_map[tci]); - if (!map) - continue; - - for (i = map->len; i--;) { - if (map->queues[i] == index) { - set_bit(cpu, mask); - break; - } + for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), + j < nr_ids;) { + int i, tci = j * num_tc + tc; + struct xps_map *map; + + map = rcu_dereference(dev_maps->attr_map[tci]); + if (!map) + continue; + + for (i = map->len; i--;) { + if (map->queues[i] == index) { + set_bit(j, *mask); + break; } } } + +rcu_unlock: rcu_read_unlock(); + return ret; +} + +static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) +{ + struct net_device *dev = queue->dev; + unsigned long *mask; + unsigned int index; + int len, ret; + + if (!netif_is_multiqueue(dev)) + return -ENOENT; + + index = get_netdev_queue_index(queue); + + mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); + if (!mask) + return -ENOMEM; + + if (!rtnl_trylock()) { + bitmap_free(mask); + return restart_syscall(); + } + + ret = xps_queue_show(dev, &mask, index, false); rtnl_unlock(); + if (ret) { + bitmap_free(mask); + return ret; + } + len = bitmap_print_to_pagebuf(false, buf, mask, nr_cpu_ids); bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; - -err_rtnl_unlock: - rtnl_unlock(); - return ret; } static ssize_t xps_cpus_store(struct netdev_queue *queue, @@ -1430,65 +1451,34 @@ static struct netdev_queue_attribute xps_cpus_attribute __ro_after_init static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) { - int j, len, ret, num_tc = 1, tc = 0; struct net_device *dev = queue->dev; - struct xps_dev_maps *dev_maps; unsigned long *mask; unsigned int index; + int len, ret; index = get_netdev_queue_index(queue); - if (!rtnl_trylock()) - return restart_syscall(); - - if (dev->num_tc) { - num_tc = dev->num_tc; - tc = netdev_txq_to_tc(dev, index); - if (tc < 0) { - ret = -EINVAL; - goto err_rtnl_unlock; - } - } mask = bitmap_zalloc(dev->num_rx_queues, GFP_KERNEL); - if (!mask) { - ret = -ENOMEM; - goto err_rtnl_unlock; - } - - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_rxqs_map); - if (!dev_maps) - goto out_no_maps; - - for (j = -1; j = netif_attrmask_next(j, NULL, dev->num_rx_queues), - j < dev->num_rx_queues;) { - int i, tci = j * num_tc + tc; - struct xps_map *map; - - map = rcu_dereference(dev_maps->attr_map[tci]); - if (!map) - continue; + if (!mask) + return -ENOMEM; - for (i = map->len; i--;) { - if (map->queues[i] == index) { - set_bit(j, mask); - break; - } - } + if (!rtnl_trylock()) { + bitmap_free(mask); + return restart_syscall(); } -out_no_maps: - rcu_read_unlock(); + ret = xps_queue_show(dev, &mask, index, true); rtnl_unlock(); + if (ret) { + bitmap_free(mask); + return ret; + } + len = bitmap_print_to_pagebuf(false, buf, mask, dev->num_rx_queues); bitmap_free(mask); return len < PAGE_SIZE ? len : -EINVAL; - -err_rtnl_unlock: - rtnl_unlock(); - return ret; } static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, From patchwork Thu Jan 28 14:43:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053811 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0759DC433DB for ; Thu, 28 Jan 2021 14:46:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A583064DEC for ; Thu, 28 Jan 2021 14:46:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232258AbhA1OqH (ORCPT ); Thu, 28 Jan 2021 09:46:07 -0500 Received: from mail.kernel.org ([198.145.29.99]:49480 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232037AbhA1OpB (ORCPT ); Thu, 28 Jan 2021 09:45:01 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id AB61264DE6; Thu, 28 Jan 2021 14:44:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845060; bh=PUEhSX7z6zPjji/GrVsr2le+Z18R4MLuVfRPLwfKX1I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UhSTCoW2zKLZxlt824xPYs8y20qYwHM+RAkrsjAa0blSEClYvC3OHjH9/t46IM1p6 JYqr7ZRVhx/hSuz5Th/UO2g/hKtwb1L2MFkOo8mlFLKNAPAcSUOtIGYYXD/8B8d0dw ZpERwpIeDW/jYSZ7ThXKJ80uX2ynjrqWQBwl/cE//5Wx8NRsp7zNkoIO0eG+jDAOms GPrBUEz43kKuOlYlj133oJqatlwrdhySNJHoe2lpxBP13Ym/d9Bxzn3IYfHjnYSzIC H7+/wJhPaQTsbXBykFlrHuG0pYL/yeWk+Aky0bQp/AkXEyCV/xG02kA6xI90l3b+Je WWi7rfpsq8A2w== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 04/11] net: embed num_tc in the xps maps Date: Thu, 28 Jan 2021 15:43:58 +0100 Message-Id: <20210128144405.4157244-5-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The xps cpus/rxqs map is accessed using dev->num_tc, which is used when allocating the map. But later updates of dev->num_tc can lead to having a mismatch between the maps and how they're accessed. In such cases the map values do not make any sense and out of bound accesses can occur (that can be easily seen using KASAN). This patch aims at fixing this by embedding num_tc into the maps, using the value at the time the map is created. This brings two improvements: - The maps can be accessed using the embedded num_tc, so we know for sure we won't have out of bound accesses. - Checks can be made before accessing the maps so we know the values retrieved will make sense. We also update __netif_set_xps_queue to conditionally copy old maps from dev_maps in the new one only if the number of traffic classes from both maps match. Signed-off-by: Antoine Tenart --- include/linux/netdevice.h | 6 ++++ net/core/dev.c | 63 +++++++++++++++++++++++++-------------- net/core/net-sysfs.c | 25 ++++++---------- 3 files changed, 56 insertions(+), 38 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 9e8572533d8e..481307de6983 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -779,9 +779,15 @@ struct xps_map { /* * This structure holds all XPS maps for device. Maps are indexed by CPU. + * + * We keep track of the number of traffic classes used when the struct is + * allocated, in num_tc. This will be used to navigate the maps, to ensure we're + * not crossing its upper bound, as the original dev->num_tc can be updated in + * the meantime. */ struct xps_dev_maps { struct rcu_head rcu; + s16 num_tc; struct xps_map __rcu *attr_map[]; /* Either CPUs map or RXQs map */ }; diff --git a/net/core/dev.c b/net/core/dev.c index 6df3f1bcdc68..f43281a7367c 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2472,7 +2472,7 @@ static bool remove_xps_queue_cpu(struct net_device *dev, struct xps_dev_maps *dev_maps, int cpu, u16 offset, u16 count) { - int num_tc = dev->num_tc ? : 1; + int num_tc = dev_maps->num_tc; bool active = false; int tci; @@ -2615,10 +2615,10 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, { const unsigned long *online_mask = NULL, *possible_mask = NULL; struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; + bool active = false, copy = false; int i, j, tci, numa_node_id = -2; int maps_sz, num_tc = 1, tc = 0; struct xps_map *map, *new_map; - bool active = false; unsigned int nr_ids; if (dev->num_tc) { @@ -2653,19 +2653,29 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (maps_sz < L1_CACHE_BYTES) maps_sz = L1_CACHE_BYTES; + /* The old dev_maps could be larger or smaller than the one we're + * setting up now, as dev->num_tc could have been updated in between. We + * could try to be smart, but let's be safe instead and only copy + * foreign traffic classes if the two map sizes match. + */ + if (dev_maps && dev_maps->num_tc == num_tc) + copy = true; + /* allocate memory for queue storage */ for (j = -1; j = netif_attrmask_next_and(j, online_mask, mask, nr_ids), j < nr_ids;) { - if (!new_dev_maps) - new_dev_maps = kzalloc(maps_sz, GFP_KERNEL); if (!new_dev_maps) { - mutex_unlock(&xps_map_mutex); - return -ENOMEM; + new_dev_maps = kzalloc(maps_sz, GFP_KERNEL); + if (!new_dev_maps) { + mutex_unlock(&xps_map_mutex); + return -ENOMEM; + } + + new_dev_maps->num_tc = num_tc; } tci = j * num_tc + tc; - map = dev_maps ? xmap_dereference(dev_maps->attr_map[tci]) : - NULL; + map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; map = expand_xps_map(map, j, index, is_rxqs_map); if (!map) @@ -2687,7 +2697,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { /* copy maps belonging to foreign traffic classes */ - for (i = tc, tci = j * num_tc; dev_maps && i--; tci++) { + for (i = tc, tci = j * num_tc; copy && i--; tci++) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); @@ -2717,14 +2727,14 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, numa_node_id = -1; } #endif - } else if (dev_maps) { + } else if (copy) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); } /* copy maps belonging to foreign traffic classes */ - for (i = num_tc - tc, tci++; dev_maps && --i; tci++) { + for (i = num_tc - tc, tci++; copy && --i; tci++) { /* fill in the new device map from the old device map */ map = xmap_dereference(dev_maps->attr_map[tci]); RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); @@ -2742,11 +2752,18 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - for (i = num_tc, tci = j * num_tc; i--; tci++) { - new_map = xmap_dereference(new_dev_maps->attr_map[tci]); + for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) { map = xmap_dereference(dev_maps->attr_map[tci]); - if (map && map != new_map) - kfree_rcu(map, rcu); + if (!map) + continue; + + if (copy) { + new_map = xmap_dereference(new_dev_maps->attr_map[tci]); + if (map == new_map) + continue; + } + + kfree_rcu(map, rcu); } } @@ -2770,12 +2787,12 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* removes tx-queue from unused CPUs/rx-queues */ for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - for (i = tc, tci = j * num_tc; i--; tci++) + for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) active |= remove_xps_queue(dev_maps, tci, index); if (!netif_attr_test_mask(j, mask, nr_ids) || !netif_attr_test_online(j, online_mask, nr_ids)) active |= remove_xps_queue(dev_maps, tci, index); - for (i = num_tc - tc, tci++; --i; tci++) + for (i = dev_maps->num_tc - tc, tci++; --i; tci++) active |= remove_xps_queue(dev_maps, tci, index); } @@ -2793,7 +2810,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, j < nr_ids;) { for (i = num_tc, tci = j * num_tc; i--; tci++) { new_map = xmap_dereference(new_dev_maps->attr_map[tci]); - map = dev_maps ? + map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; if (new_map && new_map != map) @@ -3914,13 +3931,15 @@ sch_handle_egress(struct sk_buff *skb, int *ret, struct net_device *dev) static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb, struct xps_dev_maps *dev_maps, unsigned int tci) { + int tc = netdev_get_prio_tc_map(dev, skb->priority); struct xps_map *map; int queue_index = -1; - if (dev->num_tc) { - tci *= dev->num_tc; - tci += netdev_get_prio_tc_map(dev, skb->priority); - } + if (tc >= dev_maps->num_tc) + return queue_index; + + tci *= dev_maps->num_tc; + tci += tc; map = rcu_dereference(dev_maps->attr_map[tci]); if (map) { diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 6e6bc05181f6..f606f3556ad7 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1319,23 +1319,13 @@ static int xps_queue_show(struct net_device *dev, unsigned long **mask, unsigned int index, bool is_rxqs_map) { const unsigned long *possible_mask = NULL; - int j, num_tc = 0, tc = 0, ret = 0; struct xps_dev_maps *dev_maps; + int j, tc = 0, ret = 0; unsigned int nr_ids; - if (dev->num_tc) { - /* Do not allow XPS on subordinate device directly */ - num_tc = dev->num_tc; - if (num_tc < 0) - return -EINVAL; - - /* If queue belongs to subordinate dev use its map */ - dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; - - tc = netdev_txq_to_tc(dev, index); - if (tc < 0) - return -EINVAL; - } + tc = netdev_txq_to_tc(dev, index); + if (tc < 0) + return -EINVAL; rcu_read_lock(); @@ -1348,12 +1338,12 @@ static int xps_queue_show(struct net_device *dev, unsigned long **mask, if (num_possible_cpus() > 1) possible_mask = cpumask_bits(cpu_possible_mask); } - if (!dev_maps) + if (!dev_maps || tc >= dev_maps->num_tc) goto rcu_unlock; for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - int i, tci = j * num_tc + tc; + int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; map = rcu_dereference(dev_maps->attr_map[tci]); @@ -1386,6 +1376,9 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) index = get_netdev_queue_index(queue); + /* If queue belongs to subordinate dev use its map */ + dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev; + mask = bitmap_zalloc(nr_cpu_ids, GFP_KERNEL); if (!mask) return -ENOMEM; From patchwork Thu Jan 28 14:43:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053845 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E635CC433E0 for ; Thu, 28 Jan 2021 14:49:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE2AE64D99 for ; Thu, 28 Jan 2021 14:49:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231916AbhA1OtL (ORCPT ); Thu, 28 Jan 2021 09:49:11 -0500 Received: from mail.kernel.org ([198.145.29.99]:49946 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232242AbhA1OqG (ORCPT ); Thu, 28 Jan 2021 09:46:06 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8ADAD64DE7; Thu, 28 Jan 2021 14:44:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845063; bh=guh8Y3D5WpBxBXIW36Sv2E+rYAoUFJg7h/uMzWpFCe8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Aw/xqM8VYA+49LNcocD/GfQlEeNCKU91ZeGNVtYbzSsA3nKRpZuPMnx2rC1JcH8el xQVrssrEGBeEjvmt0EtJVJqOIN0adZGeRa85bjtbtxk0M3XlFjWclQJ0ZE2l1rnXZI CJ/OuYM4dcelh6aXuJ4oy1QBp60dpO14AiXcVFgs/cSfOB9Jzx4Liesw8XkfUde8F1 b3u0V7FI/2HN8daM2lF95p0sKbm3efiHdA2yiQPkMDaA0xHtM2wu8yNG81L2pZ1aBL pzCJVHNecHu3CZsBxON0lJSoUo95hGIP9WFUWh9+zmc3wXdAxWDL4J1xxMzAMMlGR3 rWnhjVxEIerew== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 05/11] net: add an helper to copy xps maps to the new dev_maps Date: Thu, 28 Jan 2021 15:43:59 +0100 Message-Id: <20210128144405.4157244-6-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch adds an helper, xps_copy_dev_maps, to copy maps from dev_maps to new_dev_maps at a given index. The logic should be the same, with an improved code readability and maintenance. Signed-off-by: Antoine Tenart --- net/core/dev.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index f43281a7367c..7e5b1a4ae4a5 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2609,6 +2609,25 @@ static struct xps_map *expand_xps_map(struct xps_map *map, int attr_index, return new_map; } +/* Copy xps maps at a given index */ +static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, + struct xps_dev_maps *new_dev_maps, int index, + int tc, bool skip_tc) +{ + int i, tci = index * dev_maps->num_tc; + struct xps_map *map; + + /* copy maps belonging to foreign traffic classes */ + for (i = 0; i < dev_maps->num_tc; i++, tci++) { + if (i == tc && skip_tc) + continue; + + /* fill in the new device map from the old device map */ + map = xmap_dereference(dev_maps->attr_map[tci]); + RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); + } +} + /* Must be called under cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, bool is_rxqs_map) @@ -2696,23 +2715,16 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - /* copy maps belonging to foreign traffic classes */ - for (i = tc, tci = j * num_tc; copy && i--; tci++) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); - } + bool skip_tc = false; - /* We need to explicitly update tci as prevous loop - * could break out early if dev_maps is NULL. - */ tci = j * num_tc + tc; - if (netif_attr_test_mask(j, mask, nr_ids) && netif_attr_test_online(j, online_mask, nr_ids)) { /* add tx-queue to CPU/rx-queue maps */ int pos = 0; + skip_tc = true; + map = xmap_dereference(new_dev_maps->attr_map[tci]); while ((pos < map->len) && (map->queues[pos] != index)) pos++; @@ -2727,18 +2739,11 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, numa_node_id = -1; } #endif - } else if (copy) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); } - /* copy maps belonging to foreign traffic classes */ - for (i = num_tc - tc, tci++; copy && --i; tci++) { - /* fill in the new device map from the old device map */ - map = xmap_dereference(dev_maps->attr_map[tci]); - RCU_INIT_POINTER(new_dev_maps->attr_map[tci], map); - } + if (copy) + xps_copy_dev_maps(dev_maps, new_dev_maps, j, tc, + skip_tc); } if (is_rxqs_map) From patchwork Thu Jan 28 14:44:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053851 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4585DC433DB for ; Thu, 28 Jan 2021 14:51:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E9E4664DE8 for ; Thu, 28 Jan 2021 14:51:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232076AbhA1OvP (ORCPT ); Thu, 28 Jan 2021 09:51:15 -0500 Received: from mail.kernel.org ([198.145.29.99]:49948 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232240AbhA1OqF (ORCPT ); Thu, 28 Jan 2021 09:46:05 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 845E864DDE; Thu, 28 Jan 2021 14:44:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845066; bh=mNXZyQYKYjQhJsHDYG8cSo3dbMx0jEQhqLiErGBf0V0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZNbnkwKnIUaXgXQeGYMR1oUtithhJIApHaQBAtbR9U0IacIZvTtvC4n5CdZ+tD9D5 /dgPL13rQ6uC8wN8VeYBD29t7Kw3xbGp+bvEoAruOhH7wfdxk444IvJ2GHbLIUtZZ+ qYayM15T3y8D5DsfFZN8XQb+uwtyKL7pBDDuiPTAhfFmlXm0l9BEMh9V20k3GlYPjH 4VxxEgm+Pim1i+9/76aemFNIfb3xdveiiKnD2KlI+Xt4/b3YYAK8z1035aYtgT3Ulu kY9Id6P3duy4KdN6y7A6d+SH6Cp2ZK7zwHXtlJXIBYps/IEXIp8KfAHCrd1BxO8rup kKVfE8jiW9gHg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 06/11] net: improve queue removal readability in __netif_set_xps_queue Date: Thu, 28 Jan 2021 15:44:00 +0100 Message-Id: <20210128144405.4157244-7-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Improve the readability of the loop removing tx-queue from unused CPUs/rx-queues in __netif_set_xps_queue. The change should only be cosmetic. Signed-off-by: Antoine Tenart --- net/core/dev.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 7e5b1a4ae4a5..118cc0985ff1 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2792,13 +2792,16 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* removes tx-queue from unused CPUs/rx-queues */ for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { - for (i = tc, tci = j * dev_maps->num_tc; i--; tci++) - active |= remove_xps_queue(dev_maps, tci, index); - if (!netif_attr_test_mask(j, mask, nr_ids) || - !netif_attr_test_online(j, online_mask, nr_ids)) - active |= remove_xps_queue(dev_maps, tci, index); - for (i = dev_maps->num_tc - tc, tci++; --i; tci++) + tci = j * dev_maps->num_tc; + + for (i = 0; i < dev_maps->num_tc; i++, tci++) { + if (i == tc && + netif_attr_test_mask(j, mask, nr_ids) && + netif_attr_test_online(j, online_mask, nr_ids)) + continue; + active |= remove_xps_queue(dev_maps, tci, index); + } } /* free map if not active */ From patchwork Thu Jan 28 14:44:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053849 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C946CC433DB for ; Thu, 28 Jan 2021 14:51:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7514564DE8 for ; Thu, 28 Jan 2021 14:51:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231782AbhA1OvM (ORCPT ); Thu, 28 Jan 2021 09:51:12 -0500 Received: from mail.kernel.org ([198.145.29.99]:49950 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232243AbhA1OqF (ORCPT ); Thu, 28 Jan 2021 09:46:05 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 77EB864DE8; Thu, 28 Jan 2021 14:44:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845069; bh=LYclJsgU7zLfnHvgaNEw4T/k0RU1A2lux24BYUCkn3Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bBqDLW5IEEMm339OWJeZU+u53MIzqEmN7VrU8YgGxLsuEPZDZvxvgIRKBP3sW6NPr Uy0WgVccnrcYbA/KnR/nnN0JgAYKbn5B+zZ5xfL//n75hBo/Y9JAM6zd4GgksX8RcT tGUVZAouBzKa7dDfzHW4C3wlsEeeby9G3kIUEMkTWnXLEPMHQaU2ih341VMtXdtAgy eK53Djlgp6FUDDi8UNzA3/+vXixPcJ90+6OvMAhBEkgc76v23lRhScfikYpErvCFOl hcY8Psz1pf4ASWKSSaRdjwFmbLoefdAr4X6Eop+KBV5NkBWYDSvNrgNqG1G//eElbI id3q9hyRugyIg== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 07/11] net: xps: embed nr_ids in dev_maps Date: Thu, 28 Jan 2021 15:44:01 +0100 Message-Id: <20210128144405.4157244-8-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Embed nr_ids (the number of cpu for the xps cpus map, and the number of rxqs for the xps cpus map) in dev_maps. That will help not accessing out of bound memory if those values change after dev_maps was allocated. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- include/linux/netdevice.h | 4 ++++ net/core/dev.c | 26 +++++++++++--------------- net/core/net-sysfs.c | 4 ++-- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 481307de6983..f923eb97c446 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -780,6 +780,9 @@ struct xps_map { /* * This structure holds all XPS maps for device. Maps are indexed by CPU. * + * We keep track of the number of cpus/rxqs used when the struct is allocated, + * in nr_ids. This will help not accessing out-of-bound memory. + * * We keep track of the number of traffic classes used when the struct is * allocated, in num_tc. This will be used to navigate the maps, to ensure we're * not crossing its upper bound, as the original dev->num_tc can be updated in @@ -787,6 +790,7 @@ struct xps_map { */ struct xps_dev_maps { struct rcu_head rcu; + unsigned int nr_ids; s16 num_tc; struct xps_map __rcu *attr_map[]; /* Either CPUs map or RXQs map */ }; diff --git a/net/core/dev.c b/net/core/dev.c index 118cc0985ff1..2a0a777390c6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2505,14 +2505,14 @@ static void reset_xps_maps(struct net_device *dev, } static void clean_xps_maps(struct net_device *dev, const unsigned long *mask, - struct xps_dev_maps *dev_maps, unsigned int nr_ids, - u16 offset, u16 count, bool is_rxqs_map) + struct xps_dev_maps *dev_maps, u16 offset, u16 count, + bool is_rxqs_map) { + unsigned int nr_ids = dev_maps->nr_ids; bool active = false; int i, j; - for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), - j < nr_ids;) + for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), j < nr_ids;) active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) @@ -2532,7 +2532,6 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, { const unsigned long *possible_mask = NULL; struct xps_dev_maps *dev_maps; - unsigned int nr_ids; if (!static_key_false(&xps_needed)) return; @@ -2542,11 +2541,9 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (static_key_false(&xps_rxqs_needed)) { dev_maps = xmap_dereference(dev->xps_rxqs_map); - if (dev_maps) { - nr_ids = dev->num_rx_queues; - clean_xps_maps(dev, possible_mask, dev_maps, nr_ids, - offset, count, true); - } + if (dev_maps) + clean_xps_maps(dev, possible_mask, dev_maps, offset, + count, true); } dev_maps = xmap_dereference(dev->xps_cpus_map); @@ -2555,9 +2552,7 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (num_possible_cpus() > 1) possible_mask = cpumask_bits(cpu_possible_mask); - nr_ids = nr_cpu_ids; - clean_xps_maps(dev, possible_mask, dev_maps, nr_ids, offset, count, - false); + clean_xps_maps(dev, possible_mask, dev_maps, offset, count, false); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2690,6 +2685,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, return -ENOMEM; } + new_dev_maps->nr_ids = nr_ids; new_dev_maps->num_tc = num_tc; } @@ -3943,7 +3939,7 @@ static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb, struct xps_map *map; int queue_index = -1; - if (tc >= dev_maps->num_tc) + if (tc >= dev_maps->num_tc || tci >= dev_maps->nr_ids) return queue_index; tci *= dev_maps->num_tc; @@ -3982,7 +3978,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, if (dev_maps) { int tci = sk_rx_queue_get(sk); - if (tci >= 0 && tci < dev->num_rx_queues) + if (tci >= 0) queue_index = __get_xps_queue_idx(dev, skb, dev_maps, tci); } diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index f606f3556ad7..5b7123d3b66f 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1331,16 +1331,16 @@ static int xps_queue_show(struct net_device *dev, unsigned long **mask, if (is_rxqs_map) { dev_maps = rcu_dereference(dev->xps_rxqs_map); - nr_ids = dev->num_rx_queues; } else { dev_maps = rcu_dereference(dev->xps_cpus_map); - nr_ids = nr_cpu_ids; if (num_possible_cpus() > 1) possible_mask = cpumask_bits(cpu_possible_mask); } if (!dev_maps || tc >= dev_maps->num_tc) goto rcu_unlock; + nr_ids = dev_maps->nr_ids; + for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), j < nr_ids;) { int i, tci = j * dev_maps->num_tc + tc; From patchwork Thu Jan 28 14:44:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053853 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F6C1C433E0 for ; Thu, 28 Jan 2021 14:51:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 013A264DE7 for ; Thu, 28 Jan 2021 14:51:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232243AbhA1OvU (ORCPT ); Thu, 28 Jan 2021 09:51:20 -0500 Received: from mail.kernel.org ([198.145.29.99]:49952 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232244AbhA1OqF (ORCPT ); Thu, 28 Jan 2021 09:46:05 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 7493E64DE9; Thu, 28 Jan 2021 14:44:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845072; bh=DsA1S6viX+MkR70qVehutTDiq1qcSOgjTCBHV3VbLJs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uLhxjunM6FWszOhTRftQp6LC4nn6uMcFkAWD4Ij+eHE+ulYYWTr4zHHKgfBYf5pRy zgzCIH5gcfP7zdDrDqyicJhhzufX+1/kHTfVQiPGuRIjeVsXIriWGuKAqoqk2S0+zi XAufajV8FUa/Z2751ZqmicGJA4e87rBI7hj2m1VNXBhGQB5ysbp4qAtoU2M8p5/g1v 4VyesWUjv8l/Oe6JALlPJoPnkh3F4WGUz4D/6Da5SN5N9YRhU2/xArF2rVPhUzP/0r cWmZe9gy7KkJUfEkEi5y2CeH5Zy3Se7d8meIlM8iy1syl1rjVZYDjuqugHWxTwmIMy OL3JoMsiDZsnA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 08/11] net: assert the rtnl lock is held when calling __netif_set_xps_queue Date: Thu, 28 Jan 2021 15:44:02 +0100 Message-Id: <20210128144405.4157244-9-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add ASSERT_RTNL at the top of __netif_set_xps_queue and add a comment about holding the rtnl lock before the function. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- net/core/dev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/core/dev.c b/net/core/dev.c index 2a0a777390c6..c639761ddb5e 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2623,7 +2623,7 @@ static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, } } -/* Must be called under cpus_read_lock */ +/* Must be called under rtnl_lock and cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, bool is_rxqs_map) { @@ -2635,6 +2635,8 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, struct xps_map *map, *new_map; unsigned int nr_ids; + ASSERT_RTNL(); + if (dev->num_tc) { /* Do not allow XPS on subordinate device directly */ num_tc = dev->num_tc; From patchwork Thu Jan 28 14:44:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053841 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B603C433E0 for ; Thu, 28 Jan 2021 14:48:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E3F6964D99 for ; Thu, 28 Jan 2021 14:48:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232418AbhA1Os2 (ORCPT ); Thu, 28 Jan 2021 09:48:28 -0500 Received: from mail.kernel.org ([198.145.29.99]:49960 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232266AbhA1OqJ (ORCPT ); Thu, 28 Jan 2021 09:46:09 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 54A6464DEA; Thu, 28 Jan 2021 14:44:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845075; bh=GNswaViKwyujKTrV8gLmpv1zifrWZbj9jDY/O3FJqnE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OIYBPYKwqvKqAcfPXDpFUTTbcXYJAiA8olacctEIfV639HHmbJgyNaWJyxrTE+iAY VWuM8NB74/d/o7fbLm1ZTWRwgX0E7MQsNZULd5SOZrFcNUjw9KeDRGV7K9UtvLFgaA B8sjLc9mCnr/Q/Q2N7djjCvTjxChiTXnT2heAYY+ZlJZrYI7D9XeMW6HZnHLG+/Z/M DQjl85d1E1YA+ylBBtPpEEyw7R91L9jhe3TSBeKsn6bfBDEZORfxyCgssDKr8B9HfF hIj4BQienkqgINZ0TgOJVhcsbL+Vi2DGuw2c/ckNRA43iJnzHkkVNGDszbOgvwG3yT sf9lrKBUt0YTA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 09/11] net: remove the xps possible_mask Date: Thu, 28 Jan 2021 15:44:03 +0100 Message-Id: <20210128144405.4157244-10-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Remove the xps possible_mask. It was an optimization but we can just loop from 0 to nr_ids now that it is embedded in the xps dev_maps. That simplifies the code a bit. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- net/core/dev.c | 43 ++++++++++++++----------------------------- net/core/net-sysfs.c | 14 +++----------- 2 files changed, 17 insertions(+), 40 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index c639761ddb5e..d487605d6992 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2504,33 +2504,27 @@ static void reset_xps_maps(struct net_device *dev, kfree_rcu(dev_maps, rcu); } -static void clean_xps_maps(struct net_device *dev, const unsigned long *mask, +static void clean_xps_maps(struct net_device *dev, struct xps_dev_maps *dev_maps, u16 offset, u16 count, bool is_rxqs_map) { - unsigned int nr_ids = dev_maps->nr_ids; bool active = false; int i, j; - for (j = -1; j = netif_attrmask_next(j, mask, nr_ids), j < nr_ids;) - active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, - count); + for (j = 0; j < dev_maps->nr_ids; j++) + active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) reset_xps_maps(dev, dev_maps, is_rxqs_map); - if (!is_rxqs_map) { - for (i = offset + (count - 1); count--; i--) { + if (!is_rxqs_map) + for (i = offset + (count - 1); count--; i--) netdev_queue_numa_node_write( - netdev_get_tx_queue(dev, i), - NUMA_NO_NODE); - } - } + netdev_get_tx_queue(dev, i), NUMA_NO_NODE); } static void netif_reset_xps_queues(struct net_device *dev, u16 offset, u16 count) { - const unsigned long *possible_mask = NULL; struct xps_dev_maps *dev_maps; if (!static_key_false(&xps_needed)) @@ -2542,17 +2536,14 @@ static void netif_reset_xps_queues(struct net_device *dev, u16 offset, if (static_key_false(&xps_rxqs_needed)) { dev_maps = xmap_dereference(dev->xps_rxqs_map); if (dev_maps) - clean_xps_maps(dev, possible_mask, dev_maps, offset, - count, true); + clean_xps_maps(dev, dev_maps, offset, count, true); } dev_maps = xmap_dereference(dev->xps_cpus_map); if (!dev_maps) goto out_no_maps; - if (num_possible_cpus() > 1) - possible_mask = cpumask_bits(cpu_possible_mask); - clean_xps_maps(dev, possible_mask, dev_maps, offset, count, false); + clean_xps_maps(dev, dev_maps, offset, count, false); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2627,8 +2618,8 @@ static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, u16 index, bool is_rxqs_map) { - const unsigned long *online_mask = NULL, *possible_mask = NULL; struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; + const unsigned long *online_mask = NULL; bool active = false, copy = false; int i, j, tci, numa_node_id = -2; int maps_sz, num_tc = 1, tc = 0; @@ -2658,10 +2649,8 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, nr_ids = dev->num_rx_queues; } else { maps_sz = XPS_CPU_DEV_MAPS_SIZE(num_tc); - if (num_possible_cpus() > 1) { + if (num_possible_cpus() > 1) online_mask = cpumask_bits(cpu_online_mask); - possible_mask = cpumask_bits(cpu_possible_mask); - } dev_maps = xmap_dereference(dev->xps_cpus_map); nr_ids = nr_cpu_ids; } @@ -2711,8 +2700,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, static_key_slow_inc_cpuslocked(&xps_rxqs_needed); } - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { bool skip_tc = false; tci = j * num_tc + tc; @@ -2753,8 +2741,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (!dev_maps) goto out_no_old_maps; - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { for (i = num_tc, tci = j * dev_maps->num_tc; i--; tci++) { map = xmap_dereference(dev_maps->attr_map[tci]); if (!map) @@ -2788,8 +2775,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, goto out_no_maps; /* removes tx-queue from unused CPUs/rx-queues */ - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { tci = j * dev_maps->num_tc; for (i = 0; i < dev_maps->num_tc; i++, tci++) { @@ -2812,8 +2798,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, return 0; error: /* remove any maps that we added */ - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < nr_ids; j++) { for (i = num_tc, tci = j * num_tc; i--; tci++) { new_map = xmap_dereference(new_dev_maps->attr_map[tci]); map = copy ? diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 5b7123d3b66f..0c564f288460 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1318,10 +1318,8 @@ static const struct attribute_group dql_group = { static int xps_queue_show(struct net_device *dev, unsigned long **mask, unsigned int index, bool is_rxqs_map) { - const unsigned long *possible_mask = NULL; struct xps_dev_maps *dev_maps; int j, tc = 0, ret = 0; - unsigned int nr_ids; tc = netdev_txq_to_tc(dev, index); if (tc < 0) @@ -1329,20 +1327,14 @@ static int xps_queue_show(struct net_device *dev, unsigned long **mask, rcu_read_lock(); - if (is_rxqs_map) { + if (is_rxqs_map) dev_maps = rcu_dereference(dev->xps_rxqs_map); - } else { + else dev_maps = rcu_dereference(dev->xps_cpus_map); - if (num_possible_cpus() > 1) - possible_mask = cpumask_bits(cpu_possible_mask); - } if (!dev_maps || tc >= dev_maps->num_tc) goto rcu_unlock; - nr_ids = dev_maps->nr_ids; - - for (j = -1; j = netif_attrmask_next(j, possible_mask, nr_ids), - j < nr_ids;) { + for (j = 0; j < dev_maps->nr_ids; j++) { int i, tci = j * dev_maps->num_tc + tc; struct xps_map *map; From patchwork Thu Jan 28 14:44:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053843 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09E34C433DB for ; Thu, 28 Jan 2021 14:49:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B891E64DEC for ; Thu, 28 Jan 2021 14:49:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231868AbhA1Os4 (ORCPT ); Thu, 28 Jan 2021 09:48:56 -0500 Received: from mail.kernel.org ([198.145.29.99]:49962 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232265AbhA1OqI (ORCPT ); Thu, 28 Jan 2021 09:46:08 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4C67864DED; Thu, 28 Jan 2021 14:44:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845077; bh=a8WqvuXs37iaOzjpCKR/WWfzIPAFUVRSP/CDa4D6agA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U9+zNP+ChywRUuOUreyITvwVN7fllHG35Q+NmsBjVqmYBZifrpoCk310ro2NK3XFx p48qpY7b1yiu2hzAjkddD40RM19ey1k2Wjkx/dwY6b5AUcHABHBgHci3jlC1M1DlYK 2MtzrEbRK/1DUe6zduoOhiOfrYyoO9Thrp8ulPrgjCROCTR131hFij8BPazNluVNoB 3MFdnGRffFY8veNQpblaSXd2ZBfhW0atXNdRnT1IeX7aqfpYsBCYHSaaUn9yqEESLU m24NOGu8EQVwaYgTLDBCsSqOMZj5kXAE3dqzo1NJERkFa2NkNcQh0V4SnKj7y+1lDz UEKwR9pkYhYuA== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 10/11] net-sysfs: remove the rtnl lock when accessing the xps maps Date: Thu, 28 Jan 2021 15:44:04 +0100 Message-Id: <20210128144405.4157244-11-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Now that nr_ids and num_tc are stored in the xps dev_maps, which are RCU protected, we do not have the need to protect the xps_queue_show function with the rtnl lock. Signed-off-by: Antoine Tenart --- net/core/net-sysfs.c | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 0c564f288460..08c7a494d0e1 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1314,7 +1314,6 @@ static const struct attribute_group dql_group = { #endif /* CONFIG_BQL */ #ifdef CONFIG_XPS -/* Should be called with the rtnl lock held. */ static int xps_queue_show(struct net_device *dev, unsigned long **mask, unsigned int index, bool is_rxqs_map) { @@ -1375,14 +1374,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) if (!mask) return -ENOMEM; - if (!rtnl_trylock()) { - bitmap_free(mask); - return restart_syscall(); - } - ret = xps_queue_show(dev, &mask, index, false); - rtnl_unlock(); - if (ret) { bitmap_free(mask); return ret; @@ -1447,14 +1439,7 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) if (!mask) return -ENOMEM; - if (!rtnl_trylock()) { - bitmap_free(mask); - return restart_syscall(); - } - ret = xps_queue_show(dev, &mask, index, true); - rtnl_unlock(); - if (ret) { bitmap_free(mask); return ret; From patchwork Thu Jan 28 14:44:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antoine Tenart X-Patchwork-Id: 12053847 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55B6DC433DB for ; Thu, 28 Jan 2021 14:49:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E30FB64DE8 for ; Thu, 28 Jan 2021 14:49:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232331AbhA1OtX (ORCPT ); Thu, 28 Jan 2021 09:49:23 -0500 Received: from mail.kernel.org ([198.145.29.99]:50014 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229627AbhA1OqS (ORCPT ); Thu, 28 Jan 2021 09:46:18 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2BE1164DEB; Thu, 28 Jan 2021 14:44:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611845080; bh=95wGOrI3J411y0TDjWelWEYM3mT3BEoMgqm54aU2jfk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SFBu2lhW1eAEXflU4rhziDacuyxe4wbPiBuAOdVoPRl+truAQkD1TGWVvdigjBXsF Tm2YKXMGfeXFv5m++7cJXlfxP6QvGgukBGthhoX/7lAiMPY/2Yo6QkJLk4Skq2WGC+ Cbvxn+OiR4B2HVJwVYaAKQ64RQIkACtnb3+QQhs4zrsBX7diSKEJMxey2yGPR3V8F5 YMdTJronaZd3dN34OcYWbtaD6xzeqs67g7C+sisiEfFZYPKfmFYMJ76+wAzNnB20N9 TaA+NGKnhtSbJNZ7xcgiq827qUm1RtQQt4IrrakSqJ55D5h5ki49LRUQmXW4/XW+y4 r9UV+qS9Zfx3w== From: Antoine Tenart To: davem@davemloft.net, kuba@kernel.org, alexander.duyck@gmail.com Cc: Antoine Tenart , netdev@vger.kernel.org Subject: [PATCH net-next 11/11] net: move the xps maps to an array Date: Thu, 28 Jan 2021 15:44:05 +0100 Message-Id: <20210128144405.4157244-12-atenart@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210128144405.4157244-1-atenart@kernel.org> References: <20210128144405.4157244-1-atenart@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Move the xps maps (xps_cpus_map and xps_rxqs_map) to an array in net_device. That will simplify a lot the code removing the need for lots of if/else conditionals as the correct map will be available using its offset in the array. This should not modify the xps maps behaviour in any way. Suggested-by: Alexander Duyck Signed-off-by: Antoine Tenart --- drivers/net/virtio_net.c | 2 +- include/linux/netdevice.h | 17 +++++---- net/core/dev.c | 73 +++++++++++++++++---------------------- net/core/net-sysfs.c | 13 +++---- 4 files changed, 48 insertions(+), 57 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index ba8e63792549..1c98ef44c6a1 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1980,7 +1980,7 @@ static void virtnet_set_affinity(struct virtnet_info *vi) } virtqueue_set_affinity(vi->rq[i].vq, mask); virtqueue_set_affinity(vi->sq[i].vq, mask); - __netif_set_xps_queue(vi->dev, cpumask_bits(mask), i, false); + __netif_set_xps_queue(vi->dev, cpumask_bits(mask), i, XPS_CPUS); cpumask_clear(mask); } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index f923eb97c446..e14fc0f13e5f 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -762,6 +762,13 @@ struct rx_queue_attribute { const char *buf, size_t len); }; +/* XPS map type and offset of the xps map within net_device->xps_maps[]. */ +enum xps_map_type { + XPS_CPUS = 0, + XPS_RXQS, + XPS_MAPS_MAX, +}; + #ifdef CONFIG_XPS /* * This structure holds an XPS map which can be of variable length. The @@ -1770,8 +1777,7 @@ enum netdev_priv_flags { * @tx_queue_len: Max frames per queue allowed * @tx_global_lock: XXX: need comments on this one * @xdp_bulkq: XDP device bulk queue - * @xps_cpus_map: all CPUs map for XPS device - * @xps_rxqs_map: all RXQs map for XPS device + * @xps_maps: all CPUs/RXQs maps for XPS device * * @xps_maps: XXX: need comments on this one * @miniq_egress: clsact qdisc specific data for @@ -2063,8 +2069,7 @@ struct net_device { struct xdp_dev_bulk_queue __percpu *xdp_bulkq; #ifdef CONFIG_XPS - struct xps_dev_maps __rcu *xps_cpus_map; - struct xps_dev_maps __rcu *xps_rxqs_map; + struct xps_dev_maps __rcu *xps_maps[XPS_MAPS_MAX]; #endif #ifdef CONFIG_NET_CLS_ACT struct mini_Qdisc __rcu *miniq_egress; @@ -3668,7 +3673,7 @@ static inline void netif_wake_subqueue(struct net_device *dev, u16 queue_index) int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, u16 index); int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map); + u16 index, enum xps_map_type type); /** * netif_attr_test_mask - Test a CPU or Rx queue set in a mask @@ -3763,7 +3768,7 @@ static inline int netif_set_xps_queue(struct net_device *dev, static inline int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map) + u16 index, enum xps_map_type type) { return 0; } diff --git a/net/core/dev.c b/net/core/dev.c index d487605d6992..a51106b8e1ac 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2492,31 +2492,34 @@ static bool remove_xps_queue_cpu(struct net_device *dev, static void reset_xps_maps(struct net_device *dev, struct xps_dev_maps *dev_maps, - bool is_rxqs_map) + enum xps_map_type type) { - if (is_rxqs_map) { - static_key_slow_dec_cpuslocked(&xps_rxqs_needed); - RCU_INIT_POINTER(dev->xps_rxqs_map, NULL); - } else { - RCU_INIT_POINTER(dev->xps_cpus_map, NULL); - } static_key_slow_dec_cpuslocked(&xps_needed); + if (type == XPS_RXQS) + static_key_slow_dec_cpuslocked(&xps_rxqs_needed); + + RCU_INIT_POINTER(dev->xps_maps[type], NULL); + kfree_rcu(dev_maps, rcu); } -static void clean_xps_maps(struct net_device *dev, - struct xps_dev_maps *dev_maps, u16 offset, u16 count, - bool is_rxqs_map) +static void clean_xps_maps(struct net_device *dev, enum xps_map_type type, + u16 offset, u16 count) { + struct xps_dev_maps *dev_maps; bool active = false; int i, j; + dev_maps = xmap_dereference(dev->xps_maps[type]); + if (!dev_maps) + return; + for (j = 0; j < dev_maps->nr_ids; j++) active |= remove_xps_queue_cpu(dev, dev_maps, j, offset, count); if (!active) - reset_xps_maps(dev, dev_maps, is_rxqs_map); + reset_xps_maps(dev, dev_maps, type); - if (!is_rxqs_map) + if (type == XPS_CPUS) for (i = offset + (count - 1); count--; i--) netdev_queue_numa_node_write( netdev_get_tx_queue(dev, i), NUMA_NO_NODE); @@ -2525,27 +2528,17 @@ static void clean_xps_maps(struct net_device *dev, static void netif_reset_xps_queues(struct net_device *dev, u16 offset, u16 count) { - struct xps_dev_maps *dev_maps; - if (!static_key_false(&xps_needed)) return; cpus_read_lock(); mutex_lock(&xps_map_mutex); - if (static_key_false(&xps_rxqs_needed)) { - dev_maps = xmap_dereference(dev->xps_rxqs_map); - if (dev_maps) - clean_xps_maps(dev, dev_maps, offset, count, true); - } - - dev_maps = xmap_dereference(dev->xps_cpus_map); - if (!dev_maps) - goto out_no_maps; + if (static_key_false(&xps_rxqs_needed)) + clean_xps_maps(dev, XPS_RXQS, offset, count); - clean_xps_maps(dev, dev_maps, offset, count, false); + clean_xps_maps(dev, XPS_CPUS, offset, count); -out_no_maps: mutex_unlock(&xps_map_mutex); cpus_read_unlock(); } @@ -2616,7 +2609,7 @@ static void xps_copy_dev_maps(struct xps_dev_maps *dev_maps, /* Must be called under rtnl_lock and cpus_read_lock */ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, - u16 index, bool is_rxqs_map) + u16 index, enum xps_map_type type) { struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; const unsigned long *online_mask = NULL; @@ -2643,15 +2636,15 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, } mutex_lock(&xps_map_mutex); - if (is_rxqs_map) { + + dev_maps = xmap_dereference(dev->xps_maps[type]); + if (type == XPS_RXQS) { maps_sz = XPS_RXQ_DEV_MAPS_SIZE(num_tc, dev->num_rx_queues); - dev_maps = xmap_dereference(dev->xps_rxqs_map); nr_ids = dev->num_rx_queues; } else { maps_sz = XPS_CPU_DEV_MAPS_SIZE(num_tc); if (num_possible_cpus() > 1) online_mask = cpumask_bits(cpu_online_mask); - dev_maps = xmap_dereference(dev->xps_cpus_map); nr_ids = nr_cpu_ids; } @@ -2683,7 +2676,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, tci = j * num_tc + tc; map = copy ? xmap_dereference(dev_maps->attr_map[tci]) : NULL; - map = expand_xps_map(map, j, index, is_rxqs_map); + map = expand_xps_map(map, j, index, type == XPS_RXQS); if (!map) goto error; @@ -2696,7 +2689,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (!dev_maps) { /* Increment static keys at most once per type */ static_key_slow_inc_cpuslocked(&xps_needed); - if (is_rxqs_map) + if (type == XPS_RXQS) static_key_slow_inc_cpuslocked(&xps_rxqs_needed); } @@ -2718,7 +2711,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, if (pos == map->len) map->queues[map->len++] = index; #ifdef CONFIG_NUMA - if (!is_rxqs_map) { + if (type == XPS_CPUS) { if (numa_node_id == -2) numa_node_id = cpu_to_node(j); else if (numa_node_id != cpu_to_node(j)) @@ -2732,10 +2725,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, skip_tc); } - if (is_rxqs_map) - rcu_assign_pointer(dev->xps_rxqs_map, new_dev_maps); - else - rcu_assign_pointer(dev->xps_cpus_map, new_dev_maps); + rcu_assign_pointer(dev->xps_maps[type], new_dev_maps); /* Cleanup old maps */ if (!dev_maps) @@ -2764,12 +2754,11 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, active = true; out_no_new_maps: - if (!is_rxqs_map) { + if (type == XPS_CPUS) /* update Tx queue numa node */ netdev_queue_numa_node_write(netdev_get_tx_queue(dev, index), (numa_node_id >= 0) ? numa_node_id : NUMA_NO_NODE); - } if (!dev_maps) goto out_no_maps; @@ -2790,7 +2779,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask, /* free map if not active */ if (!active) - reset_xps_maps(dev, dev_maps, is_rxqs_map); + reset_xps_maps(dev, dev_maps, type); out_no_maps: mutex_unlock(&xps_map_mutex); @@ -2822,7 +2811,7 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, int ret; cpus_read_lock(); - ret = __netif_set_xps_queue(dev, cpumask_bits(mask), index, false); + ret = __netif_set_xps_queue(dev, cpumask_bits(mask), index, XPS_CPUS); cpus_read_unlock(); return ret; @@ -3961,7 +3950,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, if (!static_key_false(&xps_rxqs_needed)) goto get_cpus_map; - dev_maps = rcu_dereference(sb_dev->xps_rxqs_map); + dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_RXQS]); if (dev_maps) { int tci = sk_rx_queue_get(sk); @@ -3972,7 +3961,7 @@ static int get_xps_queue(struct net_device *dev, struct net_device *sb_dev, get_cpus_map: if (queue_index < 0) { - dev_maps = rcu_dereference(sb_dev->xps_cpus_map); + dev_maps = rcu_dereference(sb_dev->xps_maps[XPS_CPUS]); if (dev_maps) { unsigned int tci = skb->sender_cpu - 1; diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 08c7a494d0e1..41c296708011 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -1315,7 +1315,7 @@ static const struct attribute_group dql_group = { #ifdef CONFIG_XPS static int xps_queue_show(struct net_device *dev, unsigned long **mask, - unsigned int index, bool is_rxqs_map) + unsigned int index, enum xps_map_type type) { struct xps_dev_maps *dev_maps; int j, tc = 0, ret = 0; @@ -1326,10 +1326,7 @@ static int xps_queue_show(struct net_device *dev, unsigned long **mask, rcu_read_lock(); - if (is_rxqs_map) - dev_maps = rcu_dereference(dev->xps_rxqs_map); - else - dev_maps = rcu_dereference(dev->xps_cpus_map); + dev_maps = rcu_dereference(dev->xps_maps[type]); if (!dev_maps || tc >= dev_maps->num_tc) goto rcu_unlock; @@ -1374,7 +1371,7 @@ static ssize_t xps_cpus_show(struct netdev_queue *queue, char *buf) if (!mask) return -ENOMEM; - ret = xps_queue_show(dev, &mask, index, false); + ret = xps_queue_show(dev, &mask, index, XPS_CPUS); if (ret) { bitmap_free(mask); return ret; @@ -1439,7 +1436,7 @@ static ssize_t xps_rxqs_show(struct netdev_queue *queue, char *buf) if (!mask) return -ENOMEM; - ret = xps_queue_show(dev, &mask, index, true); + ret = xps_queue_show(dev, &mask, index, XPS_RXQS); if (ret) { bitmap_free(mask); return ret; @@ -1481,7 +1478,7 @@ static ssize_t xps_rxqs_store(struct netdev_queue *queue, const char *buf, } cpus_read_lock(); - err = __netif_set_xps_queue(dev, mask, index, true); + err = __netif_set_xps_queue(dev, mask, index, XPS_RXQS); cpus_read_unlock(); rtnl_unlock();