From patchwork Mon Sep 18 15:36:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13389878 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C208171BF; Mon, 18 Sep 2023 15:46:16 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4AD019BD; Mon, 18 Sep 2023 08:45:11 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1695051378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5Oxv1BgCX/PwGf84sb7NyYKUruLb3J69ciez1lO5rJM=; b=CSGR/fuU4O2gJGeO8+uh8huBwlhMEbFyvmuxtvRQD0zl6K7Ij0VK/qW0Eq7EebwS4TNKhM /Edgu2YSoIcijHOOuf/NnfikIdK0u+6GeuLc9LrgiysTj/KVEqm/YRmGjepfGQ3wi355Fz Wm8xUsmmNtYRJfZSUJL158PgEtyQhkhZ1oj4R0U9XPzFhQTlSSRK5MmhteCl8GYS4qdtRB CDHcuulPPNlic3veRMaN+pUWxenBYJHPK1MUyZXn3EGBCllf2Yi927WGNnm6yUt0oquRsV ObQnag2k9fR1R/ZLBI1gCTxZeRiM6p6a9xNnjBctnvt30a8kL7SphtnwurVOiw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1695051378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5Oxv1BgCX/PwGf84sb7NyYKUruLb3J69ciez1lO5rJM=; b=KuE1krbypCgqooKbfYdliSujdUWjORcYDOenUc6oq1f4c+4ytXNWXKo609H5/M9WqhXZl8 ayE9kyqod5hLAyBg== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Arthur Kiyanovski , David Arinzon , Noam Dagan , Saeed Bishara , Shay Agroskin Subject: [PATCH net v2 1/3] net: ena: Flush XDP packets on error. Date: Mon, 18 Sep 2023 17:36:09 +0200 Message-Id: <20230918153611.165722-2-bigeasy@linutronix.de> In-Reply-To: <20230918153611.165722-1-bigeasy@linutronix.de> References: <20230918153611.165722-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org xdp_do_flush() should be invoked before leaving the NAPI poll function after a XDP-redirect. This is not the case if the driver leaves via the error path (after having a redirect in one of its previous iterations). Invoke xdp_do_flush() also in the error path. Cc: Arthur Kiyanovski Cc: David Arinzon Cc: Noam Dagan Cc: Saeed Bishara Cc: Shay Agroskin Fixes: a318c70ad152b ("net: ena: introduce XDP redirect implementation") Acked-by: Arthur Kiyanovski Signed-off-by: Sebastian Andrzej Siewior --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index ad32ca81f7ef4..f955bde10cf90 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -1833,6 +1833,9 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi, return work_done; error: + if (xdp_flags & ENA_XDP_REDIRECT) + xdp_do_flush(); + adapter = netdev_priv(rx_ring->netdev); if (rc == -ENOSPC) { From patchwork Mon Sep 18 15:36:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13389856 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEC4A1D6AF; Mon, 18 Sep 2023 15:41:49 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0018E2722; Mon, 18 Sep 2023 08:40:22 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1695051378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=caRkeBcleE2vGm9+Tw42z34WPv/eNtNADjbS+meETfc=; b=E2yR630LOM2H6ub4VYl3yRsGl8uHXYbqAmyHKE8EOfJn5yBe0WP8IMVp7fFNnO/LeHnEm4 21G39QAjsqfjlSMbEOMGX8gbOoSykcN5l0KOpwWhIjs8oAWJP2GIuHHGgfXLytITm5W3xy c04yzxlqbCs/cCk/13m/Njbgi0D5TxQdSaaKfWwR0oVGhqGuKZsv9hmMZ79R6DEiAm+TKr UfhP7WZIVNCydBSjK73G4q7m0PS3dG/T6YzijGxbxhwbpbY7H38eDZTm/kPVp1pMnl8S2g XMNYoYimVu8XsGuK6gEdv3rKpve8etq3ZpyI62z/UWNrqEv/4t3aS71MKBxyvA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1695051378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=caRkeBcleE2vGm9+Tw42z34WPv/eNtNADjbS+meETfc=; b=fsprMuZxsQAoCgdXkEjuPh/9FNE2+704BaWnvGGjoPi3p4Me9zdB03uFDUTltePJVEAfwt QMzxrsFkWl2bATAA== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Michael Chan , Andy Gospodarek Subject: [PATCH net v2 2/3] bnxt_en: Flush XDP for bnxt_poll_nitroa0()'s NAPI Date: Mon, 18 Sep 2023 17:36:10 +0200 Message-Id: <20230918153611.165722-3-bigeasy@linutronix.de> In-Reply-To: <20230918153611.165722-1-bigeasy@linutronix.de> References: <20230918153611.165722-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org bnxt_poll_nitroa0() invokes bnxt_rx_pkt() which can run a XDP program which in turn can return XDP_REDIRECT. bnxt_rx_pkt() is also used by __bnxt_poll_work() which flushes (xdp_do_flush()) the packets after each round. bnxt_poll_nitroa0() lacks this feature. xdp_do_flush() should be invoked before leaving the NAPI callback. Invoke xdp_do_flush() after a redirect in bnxt_poll_nitroa0() NAPI. Cc: Michael Chan Fixes: f18c2b77b2e4e ("bnxt_en: optimized XDP_REDIRECT support") Reviewed-by: Andy Gospodarek Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 5cc0dbe121327..7551aa8068f8f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2614,6 +2614,7 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) struct rx_cmp_ext *rxcmp1; u32 cp_cons, tmp_raw_cons; u32 raw_cons = cpr->cp_raw_cons; + bool flush_xdp = false; u32 rx_pkts = 0; u8 event = 0; @@ -2648,6 +2649,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) rx_pkts++; else if (rc == -EBUSY) /* partial completion */ break; + if (event & BNXT_REDIRECT_EVENT) + flush_xdp = true; } else if (unlikely(TX_CMP_TYPE(txcmp) == CMPL_BASE_TYPE_HWRM_DONE)) { bnxt_hwrm_handler(bp, txcmp); @@ -2667,6 +2670,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) if (event & BNXT_AGG_EVENT) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + if (flush_xdp) + xdp_do_flush(); if (!bnxt_has_work(bp, cpr) && rx_pkts < budget) { napi_complete_done(napi, rx_pkts); From patchwork Mon Sep 18 15:36:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13389857 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 287BC1D6AF; Mon, 18 Sep 2023 15:42:18 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2A13115; Mon, 18 Sep 2023 08:40:23 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1695051379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5uItVMOEWNOMCKaEwLwvaBU6x/Cu3Qyaf2heiEbofrg=; b=Be5pNv3292IXVjkqNhDEVwevzotC4Yyle9R988mOxRd3j+fLUSaC0+/uJU6vvcUrWn/Q3K K5YCpdCUkozlKXuxxXCJBlx1pIzfXP/SxmAn/OFkfQeLNiisYzFSK4u3LgtRjZZqa1xEX8 zuEgyxdXjQj9Kf1uqfVERd3HEu0HJ2GQR8hdfpElCstC/1sDyubTHV6XD0Nh2MAbNNWIzy dxTeq7nMAydejD7+uEgxRxmctEq2IBzdgZQ680d/jXVQqu8OV9EDhC7gfNCk09vSXhlkpl i9ZM+Qb4cydBfB7Pg2xdX5sEnZd9CWfBdXzi1B1hcOt3ZpjhmoYdnkSoT03IeQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1695051379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5uItVMOEWNOMCKaEwLwvaBU6x/Cu3Qyaf2heiEbofrg=; b=e5KTDeUUKfw5/aBGh8iahBp8Okdv9ax1G6Q5g8Q0/k5yLD9MpFpWqouJcEohHZUtqYZlzO 8iIn/3MPb6kvKwAA== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Geetha sowjanya , Subbaraya Sundeep , Sunil Goutham , hariprasad Subject: [PATCH net v2 3/3] octeontx2-pf: Do xdp_do_flush() after redirects. Date: Mon, 18 Sep 2023 17:36:11 +0200 Message-Id: <20230918153611.165722-4-bigeasy@linutronix.de> In-Reply-To: <20230918153611.165722-1-bigeasy@linutronix.de> References: <20230918153611.165722-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org xdp_do_flush() should be invoked before leaving the NAPI poll function if XDP-redirect has been performed. Invoke xdp_do_flush() before leaving NAPI. Cc: Geetha sowjanya Cc: Subbaraya Sundeep Cc: Sunil Goutham Cc: hariprasad Fixes: 06059a1a9a4a5 ("octeontx2-pf: Add XDP support to netdev PF") Signed-off-by: Sebastian Andrzej Siewior Acked-by: Geethasowjanya Akula --- .../marvell/octeontx2/nic/otx2_txrx.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index e77d438489557..53b2a4ef52985 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -29,7 +29,8 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq); + struct otx2_cq_queue *cq, + bool *need_xdp_flush); static int otx2_nix_cq_op_status(struct otx2_nic *pfvf, struct otx2_cq_queue *cq) @@ -337,7 +338,7 @@ static bool otx2_check_rcv_errors(struct otx2_nic *pfvf, static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, - struct nix_cqe_rx_s *cqe) + struct nix_cqe_rx_s *cqe, bool *need_xdp_flush) { struct nix_rx_parse_s *parse = &cqe->parse; struct nix_rx_sg_s *sg = &cqe->sg; @@ -353,7 +354,7 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, } if (pfvf->xdp_prog) - if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq)) + if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq, need_xdp_flush)) return; skb = napi_get_frags(napi); @@ -388,6 +389,7 @@ static int otx2_rx_napi_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, int budget) { + bool need_xdp_flush = false; struct nix_cqe_rx_s *cqe; int processed_cqe = 0; @@ -409,13 +411,15 @@ static int otx2_rx_napi_handler(struct otx2_nic *pfvf, cq->cq_head++; cq->cq_head &= (cq->cqe_cnt - 1); - otx2_rcv_pkt_handler(pfvf, napi, cq, cqe); + otx2_rcv_pkt_handler(pfvf, napi, cq, cqe, &need_xdp_flush); cqe->hdr.cqe_type = NIX_XQE_TYPE_INVALID; cqe->sg.seg_addr = 0x00; processed_cqe++; cq->pend_cqe--; } + if (need_xdp_flush) + xdp_do_flush(); /* Free CQEs to HW */ otx2_write64(pfvf, NIX_LF_CQ_OP_DOOR, @@ -1354,7 +1358,8 @@ bool otx2_xdp_sq_append_pkt(struct otx2_nic *pfvf, u64 iova, int len, u16 qidx) static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq) + struct otx2_cq_queue *cq, + bool *need_xdp_flush) { unsigned char *hard_start, *data; int qidx = cq->cq_idx; @@ -1391,8 +1396,10 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, otx2_dma_unmap_page(pfvf, iova, pfvf->rbsize, DMA_FROM_DEVICE); - if (!err) + if (!err) { + *need_xdp_flush = true; return true; + } put_page(page); break; default: