From patchwork Tue May 10 08:44:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= X-Patchwork-Id: 12844673 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82805C433F5 for ; Tue, 10 May 2022 08:45:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238173AbiEJItE (ORCPT ); Tue, 10 May 2022 04:49:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232706AbiEJItB (ORCPT ); Tue, 10 May 2022 04:49:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 16FB42A28DF for ; Tue, 10 May 2022 01:45:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652172303; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o0lRJEI5xRVDCzoq7NyFraR7DjcCFB8ST4qUSGCXOLk=; b=F9x3N0FfbCWCAIzyri2K36E1OVPbN5kyipAd93sDfHI9WlmP+HISlN9RawjyG+9fcYNuQF QSmvbY/LmaM5ZMJsgCE+oXEKURq3bq6alSBB03q/TuKpPY5qUk4I0YKk7dybmgnya4ZbXY mlJ1pzoIgQnvkFV087IxOn+HU5pw+Xs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-618-KEemUh1rO422wjobinvFmA-1; Tue, 10 May 2022 04:44:59 -0400 X-MC-Unique: KEemUh1rO422wjobinvFmA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8FC6D185A7BA; Tue, 10 May 2022 08:44:58 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9240413721; Tue, 10 May 2022 08:44:56 +0000 (UTC) From: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= To: ecree.xilinx@gmail.com, habetsm.xilinx@gmail.com, ap420073@gmail.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= Subject: [PATCH net-next 1/5] sfc: add new helper macros to iterate channels by type Date: Tue, 10 May 2022 10:44:39 +0200 Message-Id: <20220510084443.14473-2-ihuguet@redhat.com> In-Reply-To: <20220510084443.14473-1-ihuguet@redhat.com> References: <20220510084443.14473-1-ihuguet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Sometimes in the driver it's needed to iterate a subset of the channels depending on whether it is an rx, tx or xdp channel. Now it's done iterating over all channels and checking if it's of the desired type, leading to too much nested and a bit complex to understand code. Add new iterator macros to allow iterating only over a single type of channel. Signed-off-by: Íñigo Huguet --- drivers/net/ethernet/sfc/net_driver.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 318db906a154..7f665ba6a082 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -1501,6 +1501,27 @@ efx_get_channel(struct efx_nic *efx, unsigned index) _channel = (_channel->channel + 1 < (_efx)->n_channels) ? \ (_efx)->channel[_channel->channel + 1] : NULL) +#define efx_for_each_rx_channel(_channel, _efx) \ + for (_channel = (_efx)->channel[0]; \ + _channel; \ + _channel = (_channel->channel + 1 < (_efx)->n_rx_channels) ? \ + (_efx)->channel[_channel->channel + 1] : NULL) + +#define efx_for_each_tx_channel(_channel, _efx) \ + for (_channel = (_efx)->channel[efx->tx_channel_offset]; \ + _channel; \ + _channel = (_channel->channel + 1 < \ + (_efx)->tx_channel_offset + (_efx)->n_tx_channels) ? \ + (_efx)->channel[_channel->channel + 1] : NULL) + +#define efx_for_each_xdp_channel(_channel, _efx) \ + for (_channel = ((_efx)->n_xdp_channels > 0) ? \ + (_efx)->channel[efx->xdp_channel_offset] : NULL; \ + _channel; \ + _channel = (_channel->channel + 1 < \ + (_efx)->xdp_channel_offset + (_efx)->n_xdp_channels) ? \ + (_efx)->channel[_channel->channel + 1] : NULL) + /* Iterate over all used channels in reverse */ #define efx_for_each_channel_rev(_channel, _efx) \ for (_channel = (_efx)->channel[(_efx)->n_channels - 1]; \ From patchwork Tue May 10 08:44:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= X-Patchwork-Id: 12844675 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E42A2C433F5 for ; Tue, 10 May 2022 08:45:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238178AbiEJItG (ORCPT ); Tue, 10 May 2022 04:49:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238172AbiEJItD (ORCPT ); Tue, 10 May 2022 04:49:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 56EF02A2F7B for ; Tue, 10 May 2022 01:45:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652172306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Imw52/oNdQSoSXzpgws3ep9t+R90M7Leua7GRrni830=; b=Tjcn1DBxlLRXQnzRopKuRko44YXuX81PevShhI00hsOBvsBW4cB0f6c+DFIzHC1nVnnOBx oZkf8mTZ/tS1abg2/58rFdBV/ipK3mA4mtMkgum5HAlAbvxRccgz+uE+2JacurP7uGkH9Z mtRvISuBKZe677kot8I3R2AlNGrRzYM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-PtKLByhAN3y2BKEJvP3mZg-1; Tue, 10 May 2022 04:45:01 -0400 X-MC-Unique: PtKLByhAN3y2BKEJvP3mZg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8B2DD86B8AC; Tue, 10 May 2022 08:45:00 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id D90FF413721; Tue, 10 May 2022 08:44:58 +0000 (UTC) From: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= To: ecree.xilinx@gmail.com, habetsm.xilinx@gmail.com, ap420073@gmail.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= Subject: [PATCH net-next 2/5] sfc: separate channel->tx_queue and efx->xdp_tx_queue mappings Date: Tue, 10 May 2022 10:44:40 +0200 Message-Id: <20220510084443.14473-3-ihuguet@redhat.com> In-Reply-To: <20220510084443.14473-1-ihuguet@redhat.com> References: <20220510084443.14473-1-ihuguet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Channels that contains tx queues need to determine the mapping of this queue structures to hw queue numbers. This applies both to all tx queues, no matter if they are normal tx queues, xdp_tx queues or both at the same time. Also, a lookup table to map each cpu to a xdp_tx queue is created, containing pointers to the xdp_tx queues, that should already be allocated in one or more channels. This lookup table is global to all efx_nic structure. Mappings to hw queues and xdp lookup table creation were done at the same time in efx_set_channels, but it had a bit messy and not very clear code. Then, commit 059a47f1da93 ("net: sfc: add missing xdp queue reinitialization") moved part of that initialization to a separate function to fix a bug produced because the xdp_tx queues lookup table was not reinitialized after channels reallocation, leaving it pointing to deallocated queues. Not all of that initialization needs to be redone, but only the xdp_tx queues lookup table, and not the mappings to hw queues. So this resulted in even less clear code. This patch moves back the part of that code that doesn't need to be reinitialized. That is, the mapping of tx queues with hw queues numbers. As a result, xdp queues lookup table creation and this are done in different places, conforming to single responsibility principle and resulting in more clear code. Signed-off-by: Íñigo Huguet --- drivers/net/ethernet/sfc/efx_channels.c | 69 +++++++++++++------------ 1 file changed, 37 insertions(+), 32 deletions(-) diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c index 3f28f9861dfa..8feba80f0a34 100644 --- a/drivers/net/ethernet/sfc/efx_channels.c +++ b/drivers/net/ethernet/sfc/efx_channels.c @@ -767,6 +767,19 @@ void efx_remove_channels(struct efx_nic *efx) kfree(efx->xdp_tx_queues); } +static inline int efx_alloc_xdp_tx_queues(struct efx_nic *efx) +{ + if (efx->xdp_tx_queue_count) { + EFX_WARN_ON_PARANOID(efx->xdp_tx_queues); + efx->xdp_tx_queues = kcalloc(efx->xdp_tx_queue_count, + sizeof(*efx->xdp_tx_queues), + GFP_KERNEL); + if (!efx->xdp_tx_queues) + return -ENOMEM; + } + return 0; +} + static int efx_set_xdp_tx_queue(struct efx_nic *efx, int xdp_queue_number, struct efx_tx_queue *tx_queue) { @@ -789,44 +802,29 @@ static void efx_set_xdp_channels(struct efx_nic *efx) int xdp_queue_number = 0; int rc; - /* We need to mark which channels really have RX and TX - * queues, and adjust the TX queue numbers if we have separate - * RX-only and TX-only channels. - */ efx_for_each_channel(channel, efx) { if (channel->channel < efx->tx_channel_offset) continue; if (efx_channel_is_xdp_tx(channel)) { efx_for_each_channel_tx_queue(tx_queue, channel) { - tx_queue->queue = next_queue++; rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, tx_queue); if (rc == 0) xdp_queue_number++; } - } else { - efx_for_each_channel_tx_queue(tx_queue, channel) { - tx_queue->queue = next_queue++; - netif_dbg(efx, drv, efx->net_dev, - "Channel %u TXQ %u is HW %u\n", - channel->channel, tx_queue->label, - tx_queue->queue); - } + } else if (efx->xdp_txq_queues_mode == EFX_XDP_TX_QUEUES_BORROWED) { /* If XDP is borrowing queues from net stack, it must * use the queue with no csum offload, which is the * first one of the channel * (note: tx_queue_by_type is not initialized yet) */ - if (efx->xdp_txq_queues_mode == - EFX_XDP_TX_QUEUES_BORROWED) { - tx_queue = &channel->tx_queue[0]; - rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, - tx_queue); - if (rc == 0) - xdp_queue_number++; - } + tx_queue = &channel->tx_queue[0]; + rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, + tx_queue); + if (rc == 0) + xdp_queue_number++; } } WARN_ON(efx->xdp_txq_queues_mode == EFX_XDP_TX_QUEUES_DEDICATED && @@ -952,31 +950,38 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries) int efx_set_channels(struct efx_nic *efx) { + struct efx_tx_queue *tx_queue; struct efx_channel *channel; + unsigned int queue_num = 0; int rc; efx->tx_channel_offset = efx_separate_tx_channels ? efx->n_channels - efx->n_tx_channels : 0; - if (efx->xdp_tx_queue_count) { - EFX_WARN_ON_PARANOID(efx->xdp_tx_queues); - - /* Allocate array for XDP TX queue lookup. */ - efx->xdp_tx_queues = kcalloc(efx->xdp_tx_queue_count, - sizeof(*efx->xdp_tx_queues), - GFP_KERNEL); - if (!efx->xdp_tx_queues) - return -ENOMEM; - } - + /* We need to mark which channels really have RX and TX queues, and + * adjust the TX queue numbers if we have separate RX/TX only channels. + */ efx_for_each_channel(channel, efx) { if (channel->channel < efx->n_rx_channels) channel->rx_queue.core_index = channel->channel; else channel->rx_queue.core_index = -1; + + if (channel->channel >= efx->tx_channel_offset) { + efx_for_each_channel_tx_queue(tx_queue, channel) { + tx_queue->queue = queue_num++; + netif_dbg(efx, drv, efx->net_dev, + "Channel %u TXQ %u is HW %u\n", + channel->channel, tx_queue->label, + tx_queue->queue); + } + } } + rc = efx_alloc_xdp_tx_queues(efx); + if (rc) + return rc; efx_set_xdp_channels(efx); rc = netif_set_real_num_tx_queues(efx->net_dev, efx->n_tx_channels); From patchwork Tue May 10 08:44:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= X-Patchwork-Id: 12844674 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39940C433EF for ; Tue, 10 May 2022 08:45:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238176AbiEJItF (ORCPT ); Tue, 10 May 2022 04:49:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238150AbiEJItD (ORCPT ); Tue, 10 May 2022 04:49:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E45FE2A2F71 for ; Tue, 10 May 2022 01:45:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652172306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HSTF3Y32hwg9VhtSm1YWx5yoGh71K2SCy1twxLuciQ4=; b=PtCyX7WJCq+Eu+F4gANqUxCk3sJt306N0vKNy6ho+JZ/NZFE4gyHzSyqX0qxM3pBteJHZz 3RbwOnCMn70+axnD+X4yzbKLqte5JJTbjUhp3HZP9x9tMPbNlLz7aEhyVzlCIxXsJDTZhp 4XyTwKdtLOdpK0NBI4xH+nYE28A6CmY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-62-VhP2wT9_Ply6d99KpFBirA-1; Tue, 10 May 2022 04:45:03 -0400 X-MC-Unique: VhP2wT9_Ply6d99KpFBirA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8BE76185A7A4; Tue, 10 May 2022 08:45:02 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id D28C7413721; Tue, 10 May 2022 08:45:00 +0000 (UTC) From: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= To: ecree.xilinx@gmail.com, habetsm.xilinx@gmail.com, ap420073@gmail.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= Subject: [PATCH net-next 3/5] sfc: rename set_channels to set_queues and document it Date: Tue, 10 May 2022 10:44:41 +0200 Message-Id: <20220510084443.14473-4-ihuguet@redhat.com> In-Reply-To: <20220510084443.14473-1-ihuguet@redhat.com> References: <20220510084443.14473-1-ihuguet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Function efx_set_channels had a bit missleading name because it was only setting some parameters related to queues, but the name suggests that much more things related to channels are being configured. Also, function efx_set_xdp_channels has been renamed to efx_xdp_tx_queues. It was even more missleading because there are cases where XDP dedicated channels might not exist, but there are still some xdp_tx queues parameters to configure, for example when sharing tx queues from normal channels for XDP. Finally, added some comments as documentation for these functions. Signed-off-by: Íñigo Huguet --- drivers/net/ethernet/sfc/ef100_netdev.c | 2 +- drivers/net/ethernet/sfc/efx.c | 2 +- drivers/net/ethernet/sfc/efx_channels.c | 16 ++++++++++++---- drivers/net/ethernet/sfc/efx_channels.h | 2 +- 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/sfc/ef100_netdev.c b/drivers/net/ethernet/sfc/ef100_netdev.c index 67fe44db6b61..fe8253d59782 100644 --- a/drivers/net/ethernet/sfc/ef100_netdev.c +++ b/drivers/net/ethernet/sfc/ef100_netdev.c @@ -118,7 +118,7 @@ static int ef100_net_open(struct net_device *net_dev) if (rc) goto fail; - rc = efx_set_channels(efx); + rc = efx_set_queues(efx); if (rc) goto fail; diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index 5a772354da83..3d8a83b94e0a 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -289,7 +289,7 @@ static int efx_probe_nic(struct efx_nic *efx) if (rc) goto fail1; - rc = efx_set_channels(efx); + rc = efx_set_queues(efx); if (rc) goto fail1; diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c index 8feba80f0a34..1c05063a7215 100644 --- a/drivers/net/ethernet/sfc/efx_channels.c +++ b/drivers/net/ethernet/sfc/efx_channels.c @@ -780,6 +780,7 @@ static inline int efx_alloc_xdp_tx_queues(struct efx_nic *efx) return 0; } +/* Assign a tx queue to one CPU for XDP_TX action */ static int efx_set_xdp_tx_queue(struct efx_nic *efx, int xdp_queue_number, struct efx_tx_queue *tx_queue) { @@ -794,7 +795,11 @@ static int efx_set_xdp_tx_queue(struct efx_nic *efx, int xdp_queue_number, return 0; } -static void efx_set_xdp_channels(struct efx_nic *efx) +/* Create the cpu to tx_queue mappings for XDP_TX. Depending on the value of + * efx->xdp_txq_queues_mode, it may use dedicated XDP channels or shared queues + * with netdev core + */ +static void efx_set_xdp_tx_queues(struct efx_nic *efx) { struct efx_tx_queue *tx_queue; struct efx_channel *channel; @@ -915,7 +920,7 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries) efx_init_napi_channel(efx->channel[i]); } - efx_set_xdp_channels(efx); + efx_set_xdp_tx_queues(efx); out: /* Destroy unused channel structures */ for (i = 0; i < efx->n_channels; i++) { @@ -948,7 +953,10 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries) goto out; } -int efx_set_channels(struct efx_nic *efx) +/* Assign hw queues to each RX and TX queue, create the cpu->tx_queue mapping + * for XDP_TX and register the RX and TX queues with netdev core + */ +int efx_set_queues(struct efx_nic *efx) { struct efx_tx_queue *tx_queue; struct efx_channel *channel; @@ -982,7 +990,7 @@ int efx_set_channels(struct efx_nic *efx) rc = efx_alloc_xdp_tx_queues(efx); if (rc) return rc; - efx_set_xdp_channels(efx); + efx_set_xdp_tx_queues(efx); rc = netif_set_real_num_tx_queues(efx->net_dev, efx->n_tx_channels); if (rc) diff --git a/drivers/net/ethernet/sfc/efx_channels.h b/drivers/net/ethernet/sfc/efx_channels.h index 64abb99a56b8..148e22415b05 100644 --- a/drivers/net/ethernet/sfc/efx_channels.h +++ b/drivers/net/ethernet/sfc/efx_channels.h @@ -35,7 +35,7 @@ int efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries); void efx_set_channel_names(struct efx_nic *efx); int efx_init_channels(struct efx_nic *efx); int efx_probe_channels(struct efx_nic *efx); -int efx_set_channels(struct efx_nic *efx); +int efx_set_queues(struct efx_nic *efx); void efx_remove_channel(struct efx_channel *channel); void efx_remove_channels(struct efx_nic *efx); void efx_fini_channels(struct efx_nic *efx); From patchwork Tue May 10 08:44:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= X-Patchwork-Id: 12844677 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4188C433F5 for ; Tue, 10 May 2022 08:45:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238172AbiEJItS (ORCPT ); Tue, 10 May 2022 04:49:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238181AbiEJItI (ORCPT ); Tue, 10 May 2022 04:49:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4A64F2A2F7B for ; Tue, 10 May 2022 01:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652172309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5nF7F3ee/5LY1wxs6YA0/kK8PunSr1381xwJrQb044o=; b=fVdAW/b+PeS6+H9/OFIhmfhQOoE6OJhy96FM82uKzWtY64hxR5owITyOq3zQVrvCvxrTw8 5uDUKi/nPpr6jRROnPaBJd/n/srOGDo3Wq8RT1Ot0LPxGFA7Q8td5k553IhhYTJ5IPWsU/ Qv0Ia1S7nnB01SWPb04TCpb2TLzRldo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-73-Lh9ibqnGPzeyA_J3MYlXJw-1; Tue, 10 May 2022 04:45:05 -0400 X-MC-Unique: Lh9ibqnGPzeyA_J3MYlXJw-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8BC331C08962; Tue, 10 May 2022 08:45:04 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id D47FB413721; Tue, 10 May 2022 08:45:02 +0000 (UTC) From: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= To: ecree.xilinx@gmail.com, habetsm.xilinx@gmail.com, ap420073@gmail.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= Subject: [PATCH net-next 4/5] sfc: refactor efx_set_xdp_tx_queues Date: Tue, 10 May 2022 10:44:42 +0200 Message-Id: <20220510084443.14473-5-ihuguet@redhat.com> In-Reply-To: <20220510084443.14473-1-ihuguet@redhat.com> References: <20220510084443.14473-1-ihuguet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Refactor this code to make easier to follow what's going on there and to show the intent of the code more clearly. No functional changes. Signed-off-by: Íñigo Huguet --- drivers/net/ethernet/sfc/efx_channels.c | 65 ++++++++++--------------- 1 file changed, 27 insertions(+), 38 deletions(-) diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c index 1c05063a7215..f6634faa1ec4 100644 --- a/drivers/net/ethernet/sfc/efx_channels.c +++ b/drivers/net/ethernet/sfc/efx_channels.c @@ -781,17 +781,18 @@ static inline int efx_alloc_xdp_tx_queues(struct efx_nic *efx) } /* Assign a tx queue to one CPU for XDP_TX action */ -static int efx_set_xdp_tx_queue(struct efx_nic *efx, int xdp_queue_number, - struct efx_tx_queue *tx_queue) +static inline int efx_set_xdp_tx_queue(struct efx_nic *efx, int cpu, + struct efx_tx_queue *tx_queue) { - if (xdp_queue_number >= efx->xdp_tx_queue_count) + if (cpu >= efx->xdp_tx_queue_count) return -EINVAL; netif_dbg(efx, drv, efx->net_dev, "Channel %u TXQ %u is XDP %u, HW %u\n", tx_queue->channel->channel, tx_queue->label, - xdp_queue_number, tx_queue->queue); - efx->xdp_tx_queues[xdp_queue_number] = tx_queue; + cpu, tx_queue->queue); + + efx->xdp_tx_queues[cpu] = tx_queue; return 0; } @@ -803,49 +804,37 @@ static void efx_set_xdp_tx_queues(struct efx_nic *efx) { struct efx_tx_queue *tx_queue; struct efx_channel *channel; - unsigned int next_queue = 0; - int xdp_queue_number = 0; - int rc; - - efx_for_each_channel(channel, efx) { - if (channel->channel < efx->tx_channel_offset) - continue; - - if (efx_channel_is_xdp_tx(channel)) { + unsigned int queue_num, cpu; + + cpu = 0; + if (efx->xdp_txq_queues_mode == EFX_XDP_TX_QUEUES_BORROWED) { + efx_for_each_tx_channel(channel, efx) { + /* borrow first channel's queue, with no csum offload */ + if (efx_set_xdp_tx_queue(efx, cpu, &channel->tx_queue[0]) == 0) + cpu++; + } + } else { + efx_for_each_xdp_channel(channel, efx) { efx_for_each_channel_tx_queue(tx_queue, channel) { - rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, - tx_queue); - if (rc == 0) - xdp_queue_number++; + if (efx_set_xdp_tx_queue(efx, cpu, tx_queue) == 0) + cpu++; } - } else if (efx->xdp_txq_queues_mode == EFX_XDP_TX_QUEUES_BORROWED) { - - /* If XDP is borrowing queues from net stack, it must - * use the queue with no csum offload, which is the - * first one of the channel - * (note: tx_queue_by_type is not initialized yet) - */ - tx_queue = &channel->tx_queue[0]; - rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, - tx_queue); - if (rc == 0) - xdp_queue_number++; } } + WARN_ON(efx->xdp_txq_queues_mode == EFX_XDP_TX_QUEUES_DEDICATED && - xdp_queue_number != efx->xdp_tx_queue_count); + cpu != efx->xdp_tx_queue_count); WARN_ON(efx->xdp_txq_queues_mode != EFX_XDP_TX_QUEUES_DEDICATED && - xdp_queue_number > efx->xdp_tx_queue_count); + cpu > efx->xdp_tx_queue_count); /* If we have more CPUs than assigned XDP TX queues, assign the already * existing queues to the exceeding CPUs */ - next_queue = 0; - while (xdp_queue_number < efx->xdp_tx_queue_count) { - tx_queue = efx->xdp_tx_queues[next_queue++]; - rc = efx_set_xdp_tx_queue(efx, xdp_queue_number, tx_queue); - if (rc == 0) - xdp_queue_number++; + queue_num = 0; + while (cpu < efx->xdp_tx_queue_count) { + tx_queue = efx->xdp_tx_queues[queue_num++]; + if (efx_set_xdp_tx_queue(efx, cpu, tx_queue) == 0) + cpu++; } } From patchwork Tue May 10 08:44:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= X-Patchwork-Id: 12844676 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36A4CC433EF for ; Tue, 10 May 2022 08:45:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238202AbiEJItR (ORCPT ); Tue, 10 May 2022 04:49:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238203AbiEJItP (ORCPT ); Tue, 10 May 2022 04:49:15 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4133C2A3778 for ; Tue, 10 May 2022 01:45:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652172312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Fmj5zsMWUjo4ZEcl7lSpNPy9bf2T7j+Knl6kLajk+k=; b=SM9aeLj87jZFgOHeT/5lAbfoM2NC2/lVNyJXO7zP4FEkI5iCEH+Ofy8Ljib1qkKa72oRP/ t6xiw+bwuXfKWMLZ/qJcfHH7kUqjxmJhnARDv9V4+IZ0TDIYe3eo96CDMK8uotOdyPuVsk RL6ODWW2y+LEPf7egR40mwqjAmRv0vI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-333-a-use5GQNhK5KDmEkg21mA-1; Tue, 10 May 2022 04:45:07 -0400 X-MC-Unique: a-use5GQNhK5KDmEkg21mA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8FC2A3C11A03; Tue, 10 May 2022 08:45:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.180]) by smtp.corp.redhat.com (Postfix) with ESMTP id D513F413721; Tue, 10 May 2022 08:45:04 +0000 (UTC) From: =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= To: ecree.xilinx@gmail.com, habetsm.xilinx@gmail.com, ap420073@gmail.com Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, =?utf-8?b?w43DsWlnbyBIdWd1ZXQ=?= Subject: [PATCH net-next 5/5] sfc: move tx_channel_offset calculation to interrupts probe Date: Tue, 10 May 2022 10:44:43 +0200 Message-Id: <20220510084443.14473-6-ihuguet@redhat.com> In-Reply-To: <20220510084443.14473-1-ihuguet@redhat.com> References: <20220510084443.14473-1-ihuguet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org All parameters related to what channels are used for RX, TX and/or XDP are calculated in efx_probe_interrupts or its called function efx_allocate_msix_channels. tx_channel_offset was recalculated needlessly in efx_set_queues. Remove this from here since it's more coherent to calculate it only once, in the same place than the rest of channels parameters. If MSIX is not used, this value was not set in efx_probe_interrupts, so let's do it now. The value calculated in efx_set_queues was wrong anyway, because with the addition of the support for XDP, additional channels had been added after the TX channels, and efx->n_channels - efx->n_tx_channels didn't point to the beginning of the TX channels any more. Signed-off-by: Íñigo Huguet --- drivers/net/ethernet/sfc/efx_channels.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c index f6634faa1ec4..b9bbef07bb5e 100644 --- a/drivers/net/ethernet/sfc/efx_channels.c +++ b/drivers/net/ethernet/sfc/efx_channels.c @@ -220,14 +220,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, n_channels -= efx->n_xdp_channels; if (efx_separate_tx_channels) { - efx->n_tx_channels = - min(max(n_channels / 2, 1U), - efx->max_tx_channels); - efx->tx_channel_offset = - n_channels - efx->n_tx_channels; - efx->n_rx_channels = - max(n_channels - - efx->n_tx_channels, 1U); + efx->n_tx_channels = min(max(n_channels / 2, 1U), efx->max_tx_channels); + efx->tx_channel_offset = n_channels - efx->n_tx_channels; + efx->n_rx_channels = max(n_channels - efx->n_tx_channels, 1U); } else { efx->n_tx_channels = min(n_channels, efx->max_tx_channels); efx->tx_channel_offset = 0; @@ -303,6 +298,7 @@ int efx_probe_interrupts(struct efx_nic *efx) efx->n_channels = 1; efx->n_rx_channels = 1; efx->n_tx_channels = 1; + efx->tx_channel_offset = 0; efx->n_xdp_channels = 0; efx->xdp_channel_offset = efx->n_channels; rc = pci_enable_msi(efx->pci_dev); @@ -323,6 +319,7 @@ int efx_probe_interrupts(struct efx_nic *efx) efx->n_channels = 1 + (efx_separate_tx_channels ? 1 : 0); efx->n_rx_channels = 1; efx->n_tx_channels = 1; + efx->tx_channel_offset = 1; efx->n_xdp_channels = 0; efx->xdp_channel_offset = efx->n_channels; efx->legacy_irq = efx->pci_dev->irq; @@ -952,10 +949,6 @@ int efx_set_queues(struct efx_nic *efx) unsigned int queue_num = 0; int rc; - efx->tx_channel_offset = - efx_separate_tx_channels ? - efx->n_channels - efx->n_tx_channels : 0; - /* We need to mark which channels really have RX and TX queues, and * adjust the TX queue numbers if we have separate RX/TX only channels. */