| Message ID | 20240930195617.37369-3-jdamato@fastly.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | ena: Link IRQs, queues, and NAPI instances | expand |
> Link queues to NAPIs using the netdev-genl API so this information is > queryable. > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > --dump queue-get --json='{"ifindex": 2}' > > [{'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'rx'}, > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'rx'}, > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'rx'}, > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'rx'}, > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'rx'}, > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'rx'}, > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'rx'}, > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'rx'}, > {'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'tx'}, > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'tx'}, > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'tx'}, > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'tx'}, > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'tx'}, > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'tx'}, > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'tx'}, > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'tx'}] > > Signed-off-by: Joe Damato <jdamato@fastly.com> > --- > drivers/net/ethernet/amazon/ena/ena_netdev.c | 26 > +++++++++++++++++--- > 1 file changed, 22 insertions(+), 4 deletions(-) > > diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c > b/drivers/net/ethernet/amazon/ena/ena_netdev.c > index e88de5e426ef..1c59aedaa5d5 100644 > --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c > +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c > @@ -1821,20 +1821,38 @@ static void ena_napi_disable_in_range(struct > ena_adapter *adapter, > int first_index, > int count) { > + struct napi_struct *napi; Is this variable necessary? It has been defined in the enable function because it is needed in netif_queue_set_napi() API as well as in napi_enable(), and it makes sense in order to avoid long lines In here, the variable is only used in a call to napi_disable(), can the code be kept as it is? don't see a need to shorten the napi_disable() call line. > int i; > > - for (i = first_index; i < first_index + count; i++) > - napi_disable(&adapter->ena_napi[i].napi); > + for (i = first_index; i < first_index + count; i++) { > + napi = &adapter->ena_napi[i].napi; > + if (!ENA_IS_XDP_INDEX(adapter, i)) { > + netif_queue_set_napi(adapter->netdev, i, > + NETDEV_QUEUE_TYPE_TX, NULL); > + netif_queue_set_napi(adapter->netdev, i, > + NETDEV_QUEUE_TYPE_RX, NULL); > + } > + napi_disable(napi); > + } > } > > static void ena_napi_enable_in_range(struct ena_adapter *adapter, > int first_index, > int count) { > + struct napi_struct *napi; > int i; > > - for (i = first_index; i < first_index + count; i++) > - napi_enable(&adapter->ena_napi[i].napi); > + for (i = first_index; i < first_index + count; i++) { > + napi = &adapter->ena_napi[i].napi; > + napi_enable(napi); > + if (!ENA_IS_XDP_INDEX(adapter, i)) { Can you share some info on why you decided to set the queue to napi association only in non-xdp case? In XDP, there's a napi poll function that's executed and it runs on the TX queue. I am assuming that it's because XDP is not yet supported in the framework? If so, there's a need to add an explicit comment above if (!ENA_IS_XDP_INDEX(adapter, i)) { explaining this in order to avoid confusion with the rest of the code. > + netif_queue_set_napi(adapter->netdev, i, > + NETDEV_QUEUE_TYPE_RX, napi); > + netif_queue_set_napi(adapter->netdev, i, > + NETDEV_QUEUE_TYPE_TX, napi); > + } > + } > } > > /* Configure the Rx forwarding */ > -- > 2.43.0 Thank you for uploading this patch.
On Tue, Oct 01, 2024 at 09:06:16AM +0000, Arinzon, David wrote: > > Link queues to NAPIs using the netdev-genl API so this information is > > queryable. > > > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > > --dump queue-get --json='{"ifindex": 2}' > > > > [{'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'rx'}, > > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'rx'}, > > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'rx'}, > > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'rx'}, > > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'rx'}, > > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'rx'}, > > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'rx'}, > > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'rx'}, > > {'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'tx'}, > > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'tx'}, > > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'tx'}, > > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'tx'}, > > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'tx'}, > > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'tx'}, > > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'tx'}, > > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'tx'}] > > > > Signed-off-by: Joe Damato <jdamato@fastly.com> > > --- > > drivers/net/ethernet/amazon/ena/ena_netdev.c | 26 > > +++++++++++++++++--- > > 1 file changed, 22 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c > > b/drivers/net/ethernet/amazon/ena/ena_netdev.c > > index e88de5e426ef..1c59aedaa5d5 100644 > > --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c > > +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c > > @@ -1821,20 +1821,38 @@ static void ena_napi_disable_in_range(struct > > ena_adapter *adapter, > > int first_index, > > int count) { > > + struct napi_struct *napi; > > Is this variable necessary? It has been defined in the enable function because > it is needed in netif_queue_set_napi() API as well as in napi_enable(), > and it makes sense in order to avoid long lines > In here, the variable is only used in a call to napi_disable(), can the code > be kept as it is? don't see a need to shorten the napi_disable() call line. It is true that its only used to call napi_disable so if you prefer to have it removed let me know? I think it looks nicer with the variable, but it's your driver. > > int i; > > > > - for (i = first_index; i < first_index + count; i++) > > - napi_disable(&adapter->ena_napi[i].napi); > > + for (i = first_index; i < first_index + count; i++) { > > + napi = &adapter->ena_napi[i].napi; > > + if (!ENA_IS_XDP_INDEX(adapter, i)) { > > + netif_queue_set_napi(adapter->netdev, i, > > + NETDEV_QUEUE_TYPE_TX, NULL); > > + netif_queue_set_napi(adapter->netdev, i, > > + NETDEV_QUEUE_TYPE_RX, NULL); > > + } > > + napi_disable(napi); > > + } > > } > > > > static void ena_napi_enable_in_range(struct ena_adapter *adapter, > > int first_index, > > int count) { > > + struct napi_struct *napi; > > int i; > > > > - for (i = first_index; i < first_index + count; i++) > > - napi_enable(&adapter->ena_napi[i].napi); > > + for (i = first_index; i < first_index + count; i++) { > > + napi = &adapter->ena_napi[i].napi; > > + napi_enable(napi); > > + if (!ENA_IS_XDP_INDEX(adapter, i)) { > > Can you share some info on why you decided to set the queue to napi association > only in non-xdp case? > In XDP, there's a napi poll function that's executed and it runs on the TX queue. > I am assuming that it's because XDP is not yet supported in the framework? If so, > there's a need to add an explicit comment above if (!ENA_IS_XDP_INDEX(adapter, i)) { > explaining this in order to avoid confusion with the rest of the code. Yes; it is skipped for XDP queues, but they could be supported in the future. Other drivers that support this API work similarly (see also: bnxt, ice, mlx4, etc). > > + netif_queue_set_napi(adapter->netdev, i, > > + NETDEV_QUEUE_TYPE_RX, napi); > > + netif_queue_set_napi(adapter->netdev, i, > > + NETDEV_QUEUE_TYPE_TX, napi); > > + } > > + } > > } > > > > /* Configure the Rx forwarding */ > > -- > > 2.43.0 > > Thank you for uploading this patch. Can you please let me know (explicitly) if you want me to send a second revision (when net-next allows for it) to remove the 'struct napi_struct' and add a comment as described above?
> > > Link queues to NAPIs using the netdev-genl API so this information > > > is queryable. > > > > > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml > \ > > > --dump queue-get --json='{"ifindex": 2}' > > > > > > [{'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'rx'}, > > > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'rx'}, > > > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'rx'}, > > > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'rx'}, > > > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'rx'}, > > > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'rx'}, > > > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'rx'}, > > > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'rx'}, > > > {'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'tx'}, > > > {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'tx'}, > > > {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'tx'}, > > > {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'tx'}, > > > {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'tx'}, > > > {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'tx'}, > > > {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'tx'}, > > > {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'tx'}] > > > > > > Signed-off-by: Joe Damato <jdamato@fastly.com> > > > --- > > > drivers/net/ethernet/amazon/ena/ena_netdev.c | 26 > > > +++++++++++++++++--- > > > 1 file changed, 22 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c > > > b/drivers/net/ethernet/amazon/ena/ena_netdev.c > > > index e88de5e426ef..1c59aedaa5d5 100644 > > > --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c > > > +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c > > > @@ -1821,20 +1821,38 @@ static void ena_napi_disable_in_range(struct > > > ena_adapter *adapter, > > > int first_index, > > > int count) { > > > + struct napi_struct *napi; > > > > Is this variable necessary? It has been defined in the enable function > > because it is needed in netif_queue_set_napi() API as well as in > > napi_enable(), and it makes sense in order to avoid long lines In > > here, the variable is only used in a call to napi_disable(), can the > > code be kept as it is? don't see a need to shorten the napi_disable() call > line. > > It is true that its only used to call napi_disable so if you prefer to have it > removed let me know? > > I think it looks nicer with the variable, but it's your driver. > > > > int i; > > > > > > - for (i = first_index; i < first_index + count; i++) > > > - napi_disable(&adapter->ena_napi[i].napi); > > > + for (i = first_index; i < first_index + count; i++) { > > > + napi = &adapter->ena_napi[i].napi; > > > + if (!ENA_IS_XDP_INDEX(adapter, i)) { > > > + netif_queue_set_napi(adapter->netdev, i, > > > + NETDEV_QUEUE_TYPE_TX, NULL); > > > + netif_queue_set_napi(adapter->netdev, i, > > > + NETDEV_QUEUE_TYPE_RX, NULL); > > > + } > > > + napi_disable(napi); > > > + } > > > } > > > > > > static void ena_napi_enable_in_range(struct ena_adapter *adapter, > > > int first_index, > > > int count) { > > > + struct napi_struct *napi; > > > int i; > > > > > > - for (i = first_index; i < first_index + count; i++) > > > - napi_enable(&adapter->ena_napi[i].napi); > > > + for (i = first_index; i < first_index + count; i++) { > > > + napi = &adapter->ena_napi[i].napi; > > > + napi_enable(napi); > > > + if (!ENA_IS_XDP_INDEX(adapter, i)) { > > > > Can you share some info on why you decided to set the queue to napi > > association only in non-xdp case? > > In XDP, there's a napi poll function that's executed and it runs on the TX > queue. > > I am assuming that it's because XDP is not yet supported in the > > framework? If so, there's a need to add an explicit comment above if > > (!ENA_IS_XDP_INDEX(adapter, i)) { explaining this in order to avoid > confusion with the rest of the code. > > Yes; it is skipped for XDP queues, but they could be supported in the future. > > Other drivers that support this API work similarly (see also: bnxt, ice, mlx4, > etc). > > > > + netif_queue_set_napi(adapter->netdev, i, > > > + NETDEV_QUEUE_TYPE_RX, napi); > > > + netif_queue_set_napi(adapter->netdev, i, > > > + NETDEV_QUEUE_TYPE_TX, napi); > > > + } > > > + } > > > } > > > > > > /* Configure the Rx forwarding */ > > > -- > > > 2.43.0 > > > > Thank you for uploading this patch. > > Can you please let me know (explicitly) if you want me to send a second > revision (when net-next allows for it) to remove the 'struct napi_struct' and > add a comment as described above? Yes, I would appreciate that. I guess the `struct napi_struct` is OK, this way both functions will look the same. Regarding the comment, yes please, something like /* This API is supported for non-XDP queues only */ in both places. I also added a small request to preserve reverse christmas tree order on patch 1/2 in the series. Thank you again for the patches in the driver.
On Tue, Oct 01, 2024 at 03:50:24PM +0000, Arinzon, David wrote: [...] > > > > > > Thank you for uploading this patch. > > > > Can you please let me know (explicitly) if you want me to send a second > > revision (when net-next allows for it) to remove the 'struct napi_struct' and > > add a comment as described above? > > Yes, I would appreciate that. > I guess the `struct napi_struct` is OK, this way both functions will look the same. > Regarding the comment, yes please, something like /* This API is supported for non-XDP queues only */ in both places. > I also added a small request to preserve reverse christmas tree order on patch 1/2 in the series. Thanks for mentioning the nit about reverse christmas tree order; I missed that. I will: - Fix the ordering of the variables in 1/2 - Add 2 comments in 2/2 I'll have to wait the ~48hr timeout before I can post the v2, but I'll be sure to CC you on the next revision. > Thank you again for the patches in the driver. No worries, thanks for the review. BTW: Since neither of the changes you've asked me to make are functional changes, would you mind testing the driver changes on your side just to make sure? I tested them myself on an ec2 instance with an ENA driver, but I am not an expert on ENA :) - Joe
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index e88de5e426ef..1c59aedaa5d5 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -1821,20 +1821,38 @@ static void ena_napi_disable_in_range(struct ena_adapter *adapter, int first_index, int count) { + struct napi_struct *napi; int i; - for (i = first_index; i < first_index + count; i++) - napi_disable(&adapter->ena_napi[i].napi); + for (i = first_index; i < first_index + count; i++) { + napi = &adapter->ena_napi[i].napi; + if (!ENA_IS_XDP_INDEX(adapter, i)) { + netif_queue_set_napi(adapter->netdev, i, + NETDEV_QUEUE_TYPE_TX, NULL); + netif_queue_set_napi(adapter->netdev, i, + NETDEV_QUEUE_TYPE_RX, NULL); + } + napi_disable(napi); + } } static void ena_napi_enable_in_range(struct ena_adapter *adapter, int first_index, int count) { + struct napi_struct *napi; int i; - for (i = first_index; i < first_index + count; i++) - napi_enable(&adapter->ena_napi[i].napi); + for (i = first_index; i < first_index + count; i++) { + napi = &adapter->ena_napi[i].napi; + napi_enable(napi); + if (!ENA_IS_XDP_INDEX(adapter, i)) { + netif_queue_set_napi(adapter->netdev, i, + NETDEV_QUEUE_TYPE_RX, napi); + netif_queue_set_napi(adapter->netdev, i, + NETDEV_QUEUE_TYPE_TX, napi); + } + } } /* Configure the Rx forwarding */
Link queues to NAPIs using the netdev-genl API so this information is queryable. $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ --dump queue-get --json='{"ifindex": 2}' [{'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'rx'}, {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'rx'}, {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'rx'}, {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'rx'}, {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'rx'}, {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'rx'}, {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'rx'}, {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'rx'}, {'id': 0, 'ifindex': 2, 'napi-id': 8201, 'type': 'tx'}, {'id': 1, 'ifindex': 2, 'napi-id': 8202, 'type': 'tx'}, {'id': 2, 'ifindex': 2, 'napi-id': 8203, 'type': 'tx'}, {'id': 3, 'ifindex': 2, 'napi-id': 8204, 'type': 'tx'}, {'id': 4, 'ifindex': 2, 'napi-id': 8205, 'type': 'tx'}, {'id': 5, 'ifindex': 2, 'napi-id': 8206, 'type': 'tx'}, {'id': 6, 'ifindex': 2, 'napi-id': 8207, 'type': 'tx'}, {'id': 7, 'ifindex': 2, 'napi-id': 8208, 'type': 'tx'}] Signed-off-by: Joe Damato <jdamato@fastly.com> --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 26 +++++++++++++++++--- 1 file changed, 22 insertions(+), 4 deletions(-)