| Message ID | 20240528075226.94255-3-hengqi@linux.alibaba.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | virtio_net: fix race on control_buf | expand |
> Refactored the handling of control_buf to be within the cvq_lock critical > section, mitigating race conditions between reading device responses and > new command submissions. > > Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> > --- > drivers/net/virtio_net.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index > 6b0512a628e0..3d8407d9e3d2 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct > virtnet_info *vi, u8 class, u8 cmd { > struct scatterlist *sgs[5], hdr, stat; > u32 out_num = 0, tmp, in_num = 0; > + bool ret; > int err; > > /* Caller should know better */ > @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct > virtnet_info *vi, u8 class, u8 cmd > } > > unlock: > + ret = vi->ctrl->status == VIRTIO_NET_OK; > mutex_unlock(&vi->cvq_lock); > - return vi->ctrl->status == VIRTIO_NET_OK; > + return ret; > } > > static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, > -- > 2.32.0.3.g01195cf9f > Reviewed-by: Hariprasad Kelam <hkelam@marvell.com>
On Tue, May 28, 2024 at 03:52:26PM +0800, Heng Qi wrote: > Refactored the handling of control_buf to be within the cvq_lock > critical section, mitigating race conditions between reading device > responses and new command submissions. > > Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> I don't get what does this change. status can change immediately after you drop the mutex, can it not? what exactly is the race conditions you are worried about? > --- > drivers/net/virtio_net.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index 6b0512a628e0..3d8407d9e3d2 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > { > struct scatterlist *sgs[5], hdr, stat; > u32 out_num = 0, tmp, in_num = 0; > + bool ret; > int err; > > /* Caller should know better */ > @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > } > > unlock: > + ret = vi->ctrl->status == VIRTIO_NET_OK; > mutex_unlock(&vi->cvq_lock); > - return vi->ctrl->status == VIRTIO_NET_OK; > + return ret; > } > > static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, > -- > 2.32.0.3.g01195cf9f
On Tue, 28 May 2024 11:46:28 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote: > On Tue, May 28, 2024 at 03:52:26PM +0800, Heng Qi wrote: > > Refactored the handling of control_buf to be within the cvq_lock > > critical section, mitigating race conditions between reading device > > responses and new command submissions. > > > > Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> > > > I don't get what does this change. status can change immediately > after you drop the mutex, can it not? what exactly is the > race conditions you are worried about? See the following case: 1. Command A is acknowledged and successfully executed by the device. 2. After releasing the mutex (mutex_unlock), process P1 gets preempted before it can read vi->ctrl->status, *which should be VIRTIO_NET_OK*. 3. A new command B (like the DIM command) is issued. 4. Post vi->ctrl->status being set to VIRTIO_NET_ERR by virtnet_send_command_reply(), process P2 gets preempted. 5. Process P1 resumes, reads *vi->ctrl->status as VIRTIO_NET_ERR*, and reports this error back for Command A. <-- Race causes incorrect results to be read. Thanks. > > > --- > > drivers/net/virtio_net.c | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > index 6b0512a628e0..3d8407d9e3d2 100644 > > --- a/drivers/net/virtio_net.c > > +++ b/drivers/net/virtio_net.c > > @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > { > > struct scatterlist *sgs[5], hdr, stat; > > u32 out_num = 0, tmp, in_num = 0; > > + bool ret; > > int err; > > > > /* Caller should know better */ > > @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > } > > > > unlock: > > + ret = vi->ctrl->status == VIRTIO_NET_OK; > > mutex_unlock(&vi->cvq_lock); > > - return vi->ctrl->status == VIRTIO_NET_OK; > > + return ret; > > } > > > > static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, > > -- > > 2.32.0.3.g01195cf9f >
On Wed, May 29, 2024 at 12:01:45AM +0800, Heng Qi wrote: > On Tue, 28 May 2024 11:46:28 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote: > > On Tue, May 28, 2024 at 03:52:26PM +0800, Heng Qi wrote: > > > Refactored the handling of control_buf to be within the cvq_lock > > > critical section, mitigating race conditions between reading device > > > responses and new command submissions. > > > > > > Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> > > > > > > I don't get what does this change. status can change immediately > > after you drop the mutex, can it not? what exactly is the > > race conditions you are worried about? > > See the following case: > > 1. Command A is acknowledged and successfully executed by the device. > 2. After releasing the mutex (mutex_unlock), process P1 gets preempted before > it can read vi->ctrl->status, *which should be VIRTIO_NET_OK*. > 3. A new command B (like the DIM command) is issued. > 4. Post vi->ctrl->status being set to VIRTIO_NET_ERR by > virtnet_send_command_reply(), process P2 gets preempted. > 5. Process P1 resumes, reads *vi->ctrl->status as VIRTIO_NET_ERR*, and reports > this error back for Command A. <-- Race causes incorrect results to be read. > > Thanks. Why is it important that P1 gets VIRTIO_NET_OK? After all it is no longer the state. > > > > > --- > > > drivers/net/virtio_net.c | 4 +++- > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > index 6b0512a628e0..3d8407d9e3d2 100644 > > > --- a/drivers/net/virtio_net.c > > > +++ b/drivers/net/virtio_net.c > > > @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > > { > > > struct scatterlist *sgs[5], hdr, stat; > > > u32 out_num = 0, tmp, in_num = 0; > > > + bool ret; > > > int err; > > > > > > /* Caller should know better */ > > > @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > > } > > > > > > unlock: > > > + ret = vi->ctrl->status == VIRTIO_NET_OK; > > > mutex_unlock(&vi->cvq_lock); > > > - return vi->ctrl->status == VIRTIO_NET_OK; > > > + return ret; > > > } > > > > > > static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, > > > -- > > > 2.32.0.3.g01195cf9f > >
On Tue, 28 May 2024 12:45:32 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote: > On Wed, May 29, 2024 at 12:01:45AM +0800, Heng Qi wrote: > > On Tue, 28 May 2024 11:46:28 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote: > > > On Tue, May 28, 2024 at 03:52:26PM +0800, Heng Qi wrote: > > > > Refactored the handling of control_buf to be within the cvq_lock > > > > critical section, mitigating race conditions between reading device > > > > responses and new command submissions. > > > > > > > > Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> > > > > > > > > > I don't get what does this change. status can change immediately > > > after you drop the mutex, can it not? what exactly is the > > > race conditions you are worried about? > > > > See the following case: > > > > 1. Command A is acknowledged and successfully executed by the device. > > 2. After releasing the mutex (mutex_unlock), process P1 gets preempted before > > it can read vi->ctrl->status, *which should be VIRTIO_NET_OK*. > > 3. A new command B (like the DIM command) is issued. > > 4. Post vi->ctrl->status being set to VIRTIO_NET_ERR by > > virtnet_send_command_reply(), process P2 gets preempted. > > 5. Process P1 resumes, reads *vi->ctrl->status as VIRTIO_NET_ERR*, and reports > > this error back for Command A. <-- Race causes incorrect results to be read. > > > > Thanks. > > > Why is it important that P1 gets VIRTIO_NET_OK? > After all it is no longer the state. The driver needs to know whether the command actually executed success. Thanks. > > > > > > > > --- > > > > drivers/net/virtio_net.c | 4 +++- > > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > index 6b0512a628e0..3d8407d9e3d2 100644 > > > > --- a/drivers/net/virtio_net.c > > > > +++ b/drivers/net/virtio_net.c > > > > @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > > > { > > > > struct scatterlist *sgs[5], hdr, stat; > > > > u32 out_num = 0, tmp, in_num = 0; > > > > + bool ret; > > > > int err; > > > > > > > > /* Caller should know better */ > > > > @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd > > > > } > > > > > > > > unlock: > > > > + ret = vi->ctrl->status == VIRTIO_NET_OK; > > > > mutex_unlock(&vi->cvq_lock); > > > > - return vi->ctrl->status == VIRTIO_NET_OK; > > > > + return ret; > > > > } > > > > > > > > static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, > > > > -- > > > > 2.32.0.3.g01195cf9f > > > >
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 6b0512a628e0..3d8407d9e3d2 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd { struct scatterlist *sgs[5], hdr, stat; u32 out_num = 0, tmp, in_num = 0; + bool ret; int err; /* Caller should know better */ @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd } unlock: + ret = vi->ctrl->status == VIRTIO_NET_OK; mutex_unlock(&vi->cvq_lock); - return vi->ctrl->status == VIRTIO_NET_OK; + return ret; } static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd,
Refactored the handling of control_buf to be within the cvq_lock critical section, mitigating race conditions between reading device responses and new command submissions. Fixes: 6f45ab3e0409 ("virtio_net: Add a lock for the command VQ.") Signed-off-by: Heng Qi <hengqi@linux.alibaba.com> --- drivers/net/virtio_net.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)