From patchwork Fri Sep 8 13:57:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13377486 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6776E63D3; Fri, 8 Sep 2023 13:58:00 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 285EB1BF5; Fri, 8 Sep 2023 06:57:59 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1694181477; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NqXfRFQG7Ol4PjIe25MGz86mhUAqsw77YCDv1lV1K9I=; b=Xj1mbrhpRee9s95uD+GI0GuBDkIg1dW1l44jPGqAvYn8DIfUDblh6PsiRGCndceYffMZVi KnQz/0YS86b8u6QixGCBJ8CNUYKMgYjhsgaO8aET7orocwaokw9HRORzFglBO69vroxQ/j gXx3oleMauA1H5sqx83je+AAoUodd89pwlWFUUOnonvoRZSyvkpdddG413G1AyjmugDdW+ cMCWpSmQNJSly+pXqpa/rzHUn0oiXT5S5zUeqdcWbLuQqbUxslzRWvQkubVMqhlPEepQ31 vzxDHdK5on+pqGCgtRb1QAXQHNzrhA41oqVqITRjeEt9EZWTmu4sAFQRJuiLLg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1694181477; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NqXfRFQG7Ol4PjIe25MGz86mhUAqsw77YCDv1lV1K9I=; b=ueM6BUs15VfaGtZfQAuWUtRS49vP6y5/g61jWw+TAbgXX2Jc82eoHAzTQNmbH3T4R+Gfug PeuzYFnJHhLHITAA== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Arthur Kiyanovski , David Arinzon , Noam Dagan , Saeed Bishara , Shay Agroskin Subject: [PATCH net 1/4] net: ena: Flush XDP packets on error. Date: Fri, 8 Sep 2023 15:57:45 +0200 Message-Id: <20230908135748.794163-2-bigeasy@linutronix.de> In-Reply-To: <20230908135748.794163-1-bigeasy@linutronix.de> References: <20230908135748.794163-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org xdp_do_flush() should be invoked before leaving the NAPI poll function after a XDP-redirect. This is not the case if the driver leaves via the error path (after having a redirect in one of its previous iterations). Invoke xdp_do_flush() also in the error path. Cc: Arthur Kiyanovski Cc: David Arinzon Cc: Noam Dagan Cc: Saeed Bishara Cc: Shay Agroskin Fixes: a318c70ad152b ("net: ena: introduce XDP redirect implementation") Signed-off-by: Sebastian Andrzej Siewior Acked-by: Arthur Kiyanovski --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index ad32ca81f7ef4..f955bde10cf90 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -1833,6 +1833,9 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi, return work_done; error: + if (xdp_flags & ENA_XDP_REDIRECT) + xdp_do_flush(); + adapter = netdev_priv(rx_ring->netdev); if (rc == -ENOSPC) { From patchwork Fri Sep 8 13:57:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13377487 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3A1FE563; Fri, 8 Sep 2023 13:58:00 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8994E1BF6; Fri, 8 Sep 2023 06:57:59 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1694181478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UEnWQjN8yKLvp2C1VP+XA3VBPqY8Xx7vebay2Q8ZdmA=; b=tGgfq7nSwBbr3kiGTCCnybDdL8XowQCXdBt1BIrT1rbZZSt7VKgfsXaRDQT5LNtpg3MIvI UMR8QsEs4kWYPgulqhj5Mqg2X897oaMo40vIf+VOsB2pWQuKOEPVvul7pQqIfJqCE5GDv5 4L20LUBv/wz8Ut/SEFiB71GB++nsfZCf1qKlvIkjWdWT1e5ak01U3Bqqd7CKFyNGDBwtRo w3/mQ8qMTYZB2BoY8pzJeg4B80YtcMI0NlTGUok5kblCOY9NHL+jRFg+TntvgYjKmfZT2u ErbuiGruWgkj471Aum3SIoGhEgqnai9BBlwygXP/mWNxGGum3IoIsp+CCkoPog== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1694181478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UEnWQjN8yKLvp2C1VP+XA3VBPqY8Xx7vebay2Q8ZdmA=; b=esKwYOdFMGQ+TG/I1j7x25XLVS7hRQc/KMWHmfxZZjgl8gZ4PDWfqmR7KRJljxa8s9W/mN m18XCy3dwO4YTdDg== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Michael Chan Subject: [PATCH net 2/4] bnxt_en: Flush XDP for bnxt_poll_nitroa0()'s NAPI Date: Fri, 8 Sep 2023 15:57:46 +0200 Message-Id: <20230908135748.794163-3-bigeasy@linutronix.de> In-Reply-To: <20230908135748.794163-1-bigeasy@linutronix.de> References: <20230908135748.794163-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org bnxt_poll_nitroa0() invokes bnxt_rx_pkt() which can run a XDP program which in turn can return XDP_REDIRECT. bnxt_rx_pkt() is also used by __bnxt_poll_work() which flushes (xdp_do_flush()) the packets after each round. bnxt_poll_nitroa0() lacks this feature. xdp_do_flush() should be invoked before leaving the NAPI callback. Invoke xdp_do_flush() after a redirect in bnxt_poll_nitroa0() NAPI. Cc: Michael Chan Fixes: f18c2b77b2e4e ("bnxt_en: optimized XDP_REDIRECT support") Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Andy Gospodarek --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 5cc0dbe121327..7551aa8068f8f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2614,6 +2614,7 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) struct rx_cmp_ext *rxcmp1; u32 cp_cons, tmp_raw_cons; u32 raw_cons = cpr->cp_raw_cons; + bool flush_xdp = false; u32 rx_pkts = 0; u8 event = 0; @@ -2648,6 +2649,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) rx_pkts++; else if (rc == -EBUSY) /* partial completion */ break; + if (event & BNXT_REDIRECT_EVENT) + flush_xdp = true; } else if (unlikely(TX_CMP_TYPE(txcmp) == CMPL_BASE_TYPE_HWRM_DONE)) { bnxt_hwrm_handler(bp, txcmp); @@ -2667,6 +2670,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) if (event & BNXT_AGG_EVENT) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + if (flush_xdp) + xdp_do_flush(); if (!bnxt_has_work(bp, cpr) && rx_pkts < budget) { napi_complete_done(napi, rx_pkts); From patchwork Fri Sep 8 13:57:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13377488 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F96AF9C1; Fri, 8 Sep 2023 13:58:01 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1FAC1BF8; Fri, 8 Sep 2023 06:57:59 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1694181478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ML+aEaZjiCZB3YKWe6Q0J9RcruG1KCXhm2H7e9PKSwA=; b=sWabV3FKQaoxEe69YJv6V0jL4iixxdMqgvPo1LpCwEeOYYKMwmFOmS8yG+SbMQlGbdKtaL hl5Ro6UqjOzlTACohCoMh2D+dTfUAMwFMUvqpPfd8JpNKVKYwRzoKhV4AXTNeOP+xXRgbZ 8lyFVWkkC+gODb6AUoKO5UJtN/Ilu3fP7bpz67FrHwflvNb//3RbLRjKcvUbAzIzTNp4td JFAwgMjxvyIjpnFZA+4xKsW7KSgTN6oOkQqIlJGr8woBxHf0s3yKzOLVrFO9riV+3yw1pA BuIW8MVATqKv34/jXdNYVyXM/1bmzVFpJ4pRzssOzvR62j//3MGCWpnEhwjvXA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1694181478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ML+aEaZjiCZB3YKWe6Q0J9RcruG1KCXhm2H7e9PKSwA=; b=ugh6CUPcA2GukpKuTy2aLzTnzAuNFrqicKct5s0UpxKmFYIh8nqYMr5bgk1ooalltzGX/q 15QTG9Pemkt6KSDw== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Geetha sowjanya , Subbaraya Sundeep , Sunil Goutham , hariprasad Subject: [PATCH net 3/4] octeontx2-pf: Do xdp_do_flush() after redirects. Date: Fri, 8 Sep 2023 15:57:47 +0200 Message-Id: <20230908135748.794163-4-bigeasy@linutronix.de> In-Reply-To: <20230908135748.794163-1-bigeasy@linutronix.de> References: <20230908135748.794163-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org xdp_do_flush() should be invoked before leaving the NAPI poll function if XDP-redirect has been performed. Invoke xdp_do_flush() before leaving NAPI. Cc: Geetha sowjanya Cc: Subbaraya Sundeep Cc: Sunil Goutham Cc: hariprasad Fixes: 06059a1a9a4a5 ("octeontx2-pf: Add XDP support to netdev PF") Signed-off-by: Sebastian Andrzej Siewior --- .../marvell/octeontx2/nic/otx2_txrx.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index e369baf115301..6c02eaa460277 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -29,7 +29,8 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq); + struct otx2_cq_queue *cq, + bool *need_xdp_flush); static int otx2_nix_cq_op_status(struct otx2_nic *pfvf, struct otx2_cq_queue *cq) @@ -337,7 +338,7 @@ static bool otx2_check_rcv_errors(struct otx2_nic *pfvf, static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, - struct nix_cqe_rx_s *cqe) + struct nix_cqe_rx_s *cqe, bool *need_xdp_flush) { struct nix_rx_parse_s *parse = &cqe->parse; struct nix_rx_sg_s *sg = &cqe->sg; @@ -353,7 +354,7 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, } if (pfvf->xdp_prog) - if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq)) + if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq, need_xdp_flush)) return; skb = napi_get_frags(napi); @@ -388,6 +389,7 @@ static int otx2_rx_napi_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, int budget) { + bool need_xdp_flush = false; struct nix_cqe_rx_s *cqe; int processed_cqe = 0; @@ -409,13 +411,15 @@ static int otx2_rx_napi_handler(struct otx2_nic *pfvf, cq->cq_head++; cq->cq_head &= (cq->cqe_cnt - 1); - otx2_rcv_pkt_handler(pfvf, napi, cq, cqe); + otx2_rcv_pkt_handler(pfvf, napi, cq, cqe, &need_xdp_flush); cqe->hdr.cqe_type = NIX_XQE_TYPE_INVALID; cqe->sg.seg_addr = 0x00; processed_cqe++; cq->pend_cqe--; } + if (need_xdp_flush) + xdp_do_flush(); /* Free CQEs to HW */ otx2_write64(pfvf, NIX_LF_CQ_OP_DOOR, @@ -1334,7 +1338,8 @@ bool otx2_xdp_sq_append_pkt(struct otx2_nic *pfvf, u64 iova, int len, u16 qidx) static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq) + struct otx2_cq_queue *cq, + bool *need_xdp_flush) { unsigned char *hard_start, *data; int qidx = cq->cq_idx; @@ -1371,8 +1376,10 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, otx2_dma_unmap_page(pfvf, iova, pfvf->rbsize, DMA_FROM_DEVICE); - if (!err) + if (!err) { + *need_xdp_flush = true; return true; + } put_page(page); break; default: From patchwork Fri Sep 8 13:57:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 13377489 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAABE125A6; Fri, 8 Sep 2023 13:58:01 +0000 (UTC) Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78F971BF5; Fri, 8 Sep 2023 06:58:00 -0700 (PDT) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1694181479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDmUPAeW4CUFwvGNZZf+Y/8buEalFOTqf/KOWGRWSAs=; b=0yDMz0gir+qi/hNhG31yIvHtnYwsrI5/toPSLQEyHKG4Xi6k4rlnTnTM6vQtOBUmlMAPdl +lyZ1/dUNVMe9TKdpZX6vbFs2zlFIawjr0bBcpF2aB5pxe+mpYHEJU+/3F/0RS1emKmlCF PTxhpt2eAqXiODymfUO/UhuMQiPtPEIDFMQUIhGGFTR7flBC7kY/bm9fnP4d4pXBG7rGzU SmC77qnKDS3GYwU/jqlyXeAVyLtikIw2XDZ/h5ikxw9QILttUxJ9b2kdZ99/RXHOMcLRoS vD/wg6pj5t5LrZSTlarBmACtA5hcI+MlqBKvYpJ4lf8ib5m3Y0/mhStMAhu4rw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1694181479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDmUPAeW4CUFwvGNZZf+Y/8buEalFOTqf/KOWGRWSAs=; b=0iLvnp+f+za9kQ0fZ7x9NKxPirY4+8PCAWEj2PubSOrnXR0MNNMEIdvFZk8cUG+EOgbyKG 48L9GR7pv45HN4Dw== To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: "David S. Miller" , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Paolo Abeni , Thomas Gleixner , Sebastian Andrzej Siewior , Andrii Nakryiko , Hao Luo , Jiri Olsa , KP Singh , Martin KaFai Lau , Song Liu , Stanislav Fomichev , Yonghong Song Subject: [PATCH net 4/4] bpf, cpumap: Flush xdp after cpu_map_bpf_prog_run_skb(). Date: Fri, 8 Sep 2023 15:57:48 +0200 Message-Id: <20230908135748.794163-5-bigeasy@linutronix.de> In-Reply-To: <20230908135748.794163-1-bigeasy@linutronix.de> References: <20230908135748.794163-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org xdp_do_flush() should be invoked before leaving the NAPI poll function if XDP-redirect has been performed. There are two possible XDP invocations in cpu_map_bpf_prog_run(): - cpu_map_bpf_prog_run_xdp() - cpu_map_bpf_prog_run_skb() Both functions update stats->redirect if a redirect is performed but xdp_do_flush() is only invoked after the former. Invoke xdp_do_flush() after both functions run and a redirect was performed. Cc: Andrii Nakryiko Cc: Hao Luo Cc: Jiri Olsa Cc: KP Singh Cc: Martin KaFai Lau Cc: Song Liu Cc: Stanislav Fomichev Cc: Yonghong Song Fixes: 11941f8a85362 ("bpf: cpumap: Implement generic cpumap") Signed-off-by: Sebastian Andrzej Siewior --- kernel/bpf/cpumap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index e42a1bdb7f536..f3ba11b4a732b 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -248,12 +248,12 @@ static int cpu_map_bpf_prog_run(struct bpf_cpu_map_entry *rcpu, void **frames, nframes = cpu_map_bpf_prog_run_xdp(rcpu, frames, xdp_n, stats); - if (stats->redirect) - xdp_do_flush(); - if (unlikely(!list_empty(list))) cpu_map_bpf_prog_run_skb(rcpu, list, stats); + if (stats->redirect) + xdp_do_flush(); + rcu_read_unlock_bh(); /* resched point, may call do_softirq() */ return nframes;