| Message ID | 20240412000246.1167600-5-bvanassche@acm.org (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Fix a rare crash in the UFS driver | expand |
On Thu, 2024-04-11 at 17:02 -0700, Bart Van Assche wrote: > > External email : Please do not click links or open attachments until > you have verified the sender or the content. > If ufshcd_abort() returns SUCCESS for an already completed command > then > that command is completed twice. This results in a crash. Prevent > this by > checking whether a command has completed without completion interrupt > from > the timeout handler. This CL fixes the following kernel crash: > > Unable to handle kernel NULL pointer dereference at virtual address > 0000000000000000 > Call trace: > dma_direct_map_sg+0x70/0x274 > scsi_dma_map+0x84/0x124 > ufshcd_queuecommand+0x3fc/0x880 > scsi_queue_rq+0x7d0/0x111c > blk_mq_dispatch_rq_list+0x440/0xebc > blk_mq_do_dispatch_sched+0x5a4/0x6b8 > __blk_mq_sched_dispatch_requests+0x150/0x220 > __blk_mq_run_hw_queue+0xf0/0x218 > __blk_mq_delay_run_hw_queue+0x8c/0x18c > blk_mq_run_hw_queue+0x1a4/0x360 > blk_mq_sched_insert_requests+0x130/0x334 > blk_mq_flush_plug_list+0x138/0x234 > blk_flush_plug_list+0x118/0x164 > blk_finish_plug() > read_pages+0x38c/0x408 > page_cache_ra_unbounded+0x230/0x2f8 > do_sync_mmap_readahead+0x1a4/0x208 > filemap_fault+0x27c/0x8f4 > f2fs_filemap_fault+0x28/0xfc > __do_fault+0xc4/0x208 > handle_pte_fault+0x290/0xe04 > do_handle_mm_fault+0x52c/0x858 > do_page_fault+0x5dc/0x798 > do_translation_fault+0x40/0x54 > do_mem_abort+0x60/0x134 > el0_da+0x40/0xb8 > el0t_64_sync_handler+0xc4/0xe4 > el0t_64_sync+0x1b4/0x1b8 > > Signed-off-by: Bart Van Assche <bvanassche@acm.org> > --- > drivers/ufs/core/ufshcd.c | 19 +++++++++++++++++++ > 1 file changed, 19 insertions(+) > > diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c > index 08abdd763c51..98b14623317f 100644 > --- a/drivers/ufs/core/ufshcd.c > +++ b/drivers/ufs/core/ufshcd.c > @@ -9022,6 +9022,25 @@ static void ufshcd_async_scan(void *data, > async_cookie_t cookie) > static enum scsi_timeout_action ufshcd_eh_timed_out(struct scsi_cmnd > *scmd) > { > struct ufs_hba *hba = shost_priv(scmd->device->host); > +struct scsi_cmnd *cmd2 = scmd; > + > +WARN_ON_ONCE(!scmd); > + > +if (is_mcq_enabled(hba)) { > +struct ufs_hw_queue *hwq = > +ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(scmd)); > + > +ufshcd_mcq_poll_cqe_lock(hba, hwq, &cmd2); > +} else { > +__ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT, > + &cmd2); > may need check __ufshcd_poll return value? > +} > +if (cmd2 == NULL) { > +sdev_printk(KERN_INFO, scmd->device, > + "%s: cmd with tag %#x has already been completed\n", > + __func__, blk_mq_unique_tag(scsi_cmd_to_rq(scmd))); > +return SCSI_EH_DONE; > +} > > if (!hba->system_suspending) { > /* Activate the error handler in the SCSI core. */
On 4/12/24 02:34, Peter Wang (王信友) wrote: >> +if (is_mcq_enabled(hba)) { >> +struct ufs_hw_queue *hwq = >> +ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(scmd)); >> + >> +ufshcd_mcq_poll_cqe_lock(hba, hwq, &cmd2); >> +} else { >> +__ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT, >> + &cmd2); > > may need check __ufshcd_poll return value? I don't think we should do that. __ufshcd_poll() returns a value > 0 if it has completed any SCSI command and may complete commands other than @scmd. In this function we need to know whether or not @scmd has been completed. Thanks, Bart.
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 08abdd763c51..98b14623317f 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -9022,6 +9022,25 @@ static void ufshcd_async_scan(void *data, async_cookie_t cookie) static enum scsi_timeout_action ufshcd_eh_timed_out(struct scsi_cmnd *scmd) { struct ufs_hba *hba = shost_priv(scmd->device->host); + struct scsi_cmnd *cmd2 = scmd; + + WARN_ON_ONCE(!scmd); + + if (is_mcq_enabled(hba)) { + struct ufs_hw_queue *hwq = + ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(scmd)); + + ufshcd_mcq_poll_cqe_lock(hba, hwq, &cmd2); + } else { + __ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT, + &cmd2); + } + if (cmd2 == NULL) { + sdev_printk(KERN_INFO, scmd->device, + "%s: cmd with tag %#x has already been completed\n", + __func__, blk_mq_unique_tag(scsi_cmd_to_rq(scmd))); + return SCSI_EH_DONE; + } if (!hba->system_suspending) { /* Activate the error handler in the SCSI core. */
If ufshcd_abort() returns SUCCESS for an already completed command then that command is completed twice. This results in a crash. Prevent this by checking whether a command has completed without completion interrupt from the timeout handler. This CL fixes the following kernel crash: Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 Call trace: dma_direct_map_sg+0x70/0x274 scsi_dma_map+0x84/0x124 ufshcd_queuecommand+0x3fc/0x880 scsi_queue_rq+0x7d0/0x111c blk_mq_dispatch_rq_list+0x440/0xebc blk_mq_do_dispatch_sched+0x5a4/0x6b8 __blk_mq_sched_dispatch_requests+0x150/0x220 __blk_mq_run_hw_queue+0xf0/0x218 __blk_mq_delay_run_hw_queue+0x8c/0x18c blk_mq_run_hw_queue+0x1a4/0x360 blk_mq_sched_insert_requests+0x130/0x334 blk_mq_flush_plug_list+0x138/0x234 blk_flush_plug_list+0x118/0x164 blk_finish_plug() read_pages+0x38c/0x408 page_cache_ra_unbounded+0x230/0x2f8 do_sync_mmap_readahead+0x1a4/0x208 filemap_fault+0x27c/0x8f4 f2fs_filemap_fault+0x28/0xfc __do_fault+0xc4/0x208 handle_pte_fault+0x290/0xe04 do_handle_mm_fault+0x52c/0x858 do_page_fault+0x5dc/0x798 do_translation_fault+0x40/0x54 do_mem_abort+0x60/0x134 el0_da+0x40/0xb8 el0t_64_sync_handler+0xc4/0xe4 el0t_64_sync+0x1b4/0x1b8 Signed-off-by: Bart Van Assche <bvanassche@acm.org> --- drivers/ufs/core/ufshcd.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)