| Message ID | 1678263432-17329-1-git-send-email-alibuda@linux.alibaba.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | [net,v2] net/smc: fix NULL sndbuf_desc in smc_cdc_tx_handler() | expand |
On Wed, Mar 08, 2023 at 04:17:12PM +0800, D. Wythe wrote: > From: "D. Wythe" <alibuda@linux.alibaba.com> > > When performing a stress test on SMC-R by rmmod mlx5_ib driver > during the wrk/nginx test, we found that there is a probability > of triggering a panic while terminating all link groups. > > This issue dues to the race between smc_smcr_terminate_all() > and smc_buf_create(). > > smc_smcr_terminate_all > > smc_buf_create > /* init */ > conn->sndbuf_desc = NULL; > ... > > __smc_lgr_terminate > smc_conn_kill > smc_close_abort > smc_cdc_get_slot_and_msg_send > > __softirqentry_text_start > smc_wr_tx_process_cqe > smc_cdc_tx_handler > READ(conn->sndbuf_desc->len); > /* panic dues to NULL sndbuf_desc */ > > conn->sndbuf_desc = xxx; > > This patch tries to fix the issue by always to check the sndbuf_desc > before send any cdc msg, to make sure that no null pointer is > seen during cqe processing. > > Fixes: 0b29ec643613 ("net/smc: immediate termination for SMCR link groups") > Signed-off-by: D. Wythe <alibuda@linux.alibaba.com> Reviewed-by: Tony Lu <tonylu@linux.alibaba.com> Thanks. > > v2 -> v1: change retval from EINVAL to ENOBUFS Better to put the changelog before Fixes tag. > > --- > net/smc/smc_cdc.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c > index 53f63bf..89105e9 100644 > --- a/net/smc/smc_cdc.c > +++ b/net/smc/smc_cdc.c > @@ -114,6 +114,9 @@ int smc_cdc_msg_send(struct smc_connection *conn, > union smc_host_cursor cfed; > int rc; > > + if (unlikely(!READ_ONCE(conn->sndbuf_desc))) > + return -ENOBUFS; > + > smc_cdc_add_pending_send(conn, pend); > > conn->tx_cdc_seq++; > -- > 1.8.3.1
On 08.03.23 09:17, D. Wythe wrote: > From: "D. Wythe" <alibuda@linux.alibaba.com> > > When performing a stress test on SMC-R by rmmod mlx5_ib driver > during the wrk/nginx test, we found that there is a probability > of triggering a panic while terminating all link groups. > > This issue dues to the race between smc_smcr_terminate_all() > and smc_buf_create(). > > smc_smcr_terminate_all > > smc_buf_create > /* init */ > conn->sndbuf_desc = NULL; > ... > > __smc_lgr_terminate > smc_conn_kill > smc_close_abort > smc_cdc_get_slot_and_msg_send > > __softirqentry_text_start > smc_wr_tx_process_cqe > smc_cdc_tx_handler > READ(conn->sndbuf_desc->len); > /* panic dues to NULL sndbuf_desc */ > > conn->sndbuf_desc = xxx; > > This patch tries to fix the issue by always to check the sndbuf_desc > before send any cdc msg, to make sure that no null pointer is > seen during cqe processing. > > Fixes: 0b29ec643613 ("net/smc: immediate termination for SMCR link groups") > Signed-off-by: D. Wythe <alibuda@linux.alibaba.com> > Thanks for the fix! Reviewed-by: Wenjia Zhang <wenjia@linux.ibm.com> > v2 -> v1: change retval from EINVAL to ENOBUFS > > --- > net/smc/smc_cdc.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c > index 53f63bf..89105e9 100644 > --- a/net/smc/smc_cdc.c > +++ b/net/smc/smc_cdc.c > @@ -114,6 +114,9 @@ int smc_cdc_msg_send(struct smc_connection *conn, > union smc_host_cursor cfed; > int rc; > > + if (unlikely(!READ_ONCE(conn->sndbuf_desc))) > + return -ENOBUFS; > + > smc_cdc_add_pending_send(conn, pend); > > conn->tx_cdc_seq++;
diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c index 53f63bf..89105e9 100644 --- a/net/smc/smc_cdc.c +++ b/net/smc/smc_cdc.c @@ -114,6 +114,9 @@ int smc_cdc_msg_send(struct smc_connection *conn, union smc_host_cursor cfed; int rc; + if (unlikely(!READ_ONCE(conn->sndbuf_desc))) + return -ENOBUFS; + smc_cdc_add_pending_send(conn, pend); conn->tx_cdc_seq++;