| Message ID | 20170208222507.25715-10-bart.vanassche@sandisk.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
On Wed, 2017-02-08 at 14:24 -0800, Bart Van Assche wrote: > target_submit_tmr() only supports the I_T_L nexus for SCSI abort > and other task management functions. Make it possible for target > drivers to specify I_T nexus for SCSI abort by passing the > TARGET_SCF_IGNORE_TMR_LUN flag to target_submit_tmr(). > > Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> > Reviewed-by: Hannes Reinecke <hare@suse.com> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Cc: Himanshu Madhani <himanshu.madhani@cavium.com> > Cc: Giridhar Malavali <giridhar.malavali@cavium.com> > --- > drivers/target/target_core_tmr.c | 16 +++++++++++++++- > drivers/target/target_core_transport.c | 30 +++++++++++++++++++++++------- > include/target/target_core_base.h | 1 + > 3 files changed, 39 insertions(+), 8 deletions(-) > > diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c > index 311dc3c2f1dc..367799b4dde1 100644 > --- a/drivers/target/target_core_tmr.c > +++ b/drivers/target/target_core_tmr.c > @@ -149,6 +149,13 @@ static bool __target_check_io_state(struct se_cmd *se_cmd, > return kref_get_unless_zero(&se_cmd->cmd_kref); > } > > +/** > + * core_tmr_abort_task - abort a SCSI command > + * @dev: LUN specified in task management function or NULL if no LUN has been > + * specified. > + * @tmr: Task management function. > + * @se_sess: Session a.k.a. I_T nexus. > + */ > void core_tmr_abort_task( > struct se_device *dev, > struct se_tmr_req *tmr, > @@ -161,7 +168,7 @@ void core_tmr_abort_task( > spin_lock_irqsave(&se_sess->sess_cmd_lock, flags); > list_for_each_entry(se_cmd, &se_sess->sess_cmd_list, se_cmd_list) { > > - if (dev != se_cmd->se_dev) > + if (dev && dev != se_cmd->se_dev) > continue; > > /* skip task management functions, including tmr->task_cmd */ > @@ -178,6 +185,13 @@ void core_tmr_abort_task( > if (!__target_check_io_state(se_cmd, se_sess, 0)) > continue; > > + if (!tmr->tmr_dev && > + transport_lookup_tmr_lun(tmr->task_cmd, > + se_cmd->orig_fe_lun) < 0) { > + target_put_sess_cmd(se_cmd); > + continue; > + } > + There is a deadlock here. While se_sess->sess_cmd_lock is held and transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is also taken. However, core_tmr_drain_tmr_list() holds se_device->se_tmr_lock while walking se_device->se_tmr_list and taking se_sess->sess_cmd_lock on each se_tmr_req->task_cmd. Dropping this patch for now. -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote: > There is a deadlock here. While se_sess->sess_cmd_lock is held and > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is > also taken. I'll move the transport_lookup_tmr_lun() further down. Bart.-- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote: > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote: > > There is a deadlock here. While se_sess->sess_cmd_lock is held and > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is > > also taken. > > I'll move the transport_lookup_tmr_lun() further down. > It should be done in transport_generic_handle_tmr(). -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote: > On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote: > > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote: > > > There is a deadlock here. While se_sess->sess_cmd_lock is held and > > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is > > > also taken. > > > > I'll move the transport_lookup_tmr_lun() further down. > > It should be done in transport_generic_handle_tmr(). Please explain why you think that's how it should be done. I'm not convinced. Bart.-- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 2017-02-10 at 01:13 +0000, Bart Van Assche wrote: > On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote: > > On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote: > > > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote: > > > > There is a deadlock here. While se_sess->sess_cmd_lock is held and > > > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is > > > > also taken. > > > > > > I'll move the transport_lookup_tmr_lun() further down. > > > > It should be done in transport_generic_handle_tmr(). > > Please explain why you think that's how it should be done. I'm not convinced. Doing it 'further down' is totally broken for LUN != 0. transport_lookup_tmr_lun() is responsible for adding se_tmr to se_device->tmr_list, but it's also responsible for se_cmd->orig_fe_lun assignment based on incoming unpacked_lun from fabric driver code. Since this patch tries to call transport_lookup_tmr_lun() from within tmr_wq context using a se_cmd->orig_fe_lun that is always zero, it's always going to be issuing all TMRs to LUN=0. -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2017-02-09 at 23:20 -0800, Nicholas A. Bellinger wrote: > On Fri, 2017-02-10 at 01:13 +0000, Bart Van Assche wrote: > > On Thu, 2017-02-09 at 17:07 -0800, Nicholas A. Bellinger wrote: > > > On Thu, 2017-02-09 at 17:24 +0000, Bart Van Assche wrote: > > > > On Thu, 2017-02-09 at 01:39 -0800, Nicholas A. Bellinger wrote: > > > > > There is a deadlock here. While se_sess->sess_cmd_lock is held and > > > > > transport_lookup_tmr_lun() is called above, se_device->se_tmr_lock is > > > > > also taken. > > > > > > > > I'll move the transport_lookup_tmr_lun() further down. > > > > > > It should be done in transport_generic_handle_tmr(). > > > > Please explain why you think that's how it should be done. I'm not convinced. > > Doing it 'further down' is totally broken for LUN != 0. > > transport_lookup_tmr_lun() is responsible for adding se_tmr to > se_device->tmr_list, but it's also responsible for se_cmd->orig_fe_lun > assignment based on incoming unpacked_lun from fabric driver code. > > Since this patch tries to call transport_lookup_tmr_lun() from within > tmr_wq context using a se_cmd->orig_fe_lun that is always zero, it's > always going to be issuing all TMRs to LUN=0. Your comment seems to apply to the se_cmd structure of the task management function. But that's not what is passed to transport_lookup_tmr_lun(). The LUN passed in my patch to that function is the LUN of the command that is being aborted. Had you perhaps misinterpret my patch? Bart.-- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 311dc3c2f1dc..367799b4dde1 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -149,6 +149,13 @@ static bool __target_check_io_state(struct se_cmd *se_cmd, return kref_get_unless_zero(&se_cmd->cmd_kref); } +/** + * core_tmr_abort_task - abort a SCSI command + * @dev: LUN specified in task management function or NULL if no LUN has been + * specified. + * @tmr: Task management function. + * @se_sess: Session a.k.a. I_T nexus. + */ void core_tmr_abort_task( struct se_device *dev, struct se_tmr_req *tmr, @@ -161,7 +168,7 @@ void core_tmr_abort_task( spin_lock_irqsave(&se_sess->sess_cmd_lock, flags); list_for_each_entry(se_cmd, &se_sess->sess_cmd_list, se_cmd_list) { - if (dev != se_cmd->se_dev) + if (dev && dev != se_cmd->se_dev) continue; /* skip task management functions, including tmr->task_cmd */ @@ -178,6 +185,13 @@ void core_tmr_abort_task( if (!__target_check_io_state(se_cmd, se_sess, 0)) continue; + if (!tmr->tmr_dev && + transport_lookup_tmr_lun(tmr->task_cmd, + se_cmd->orig_fe_lun) < 0) { + target_put_sess_cmd(se_cmd); + continue; + } + list_del_init(&se_cmd->se_cmd_list); spin_unlock_irqrestore(&se_sess->sess_cmd_lock, flags); diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 084a7fbfc72e..72fd0b430380 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -1593,18 +1593,18 @@ static void target_complete_tmr_failure(struct work_struct *work) } /** - * target_submit_tmr - lookup unpacked lun and submit uninitialized se_cmd - * for TMR CDBs + * target_submit_tmr - submit a SCSI task management function to the target core * * @se_cmd: command descriptor to submit * @se_sess: associated se_sess for endpoint * @sense: pointer to SCSI sense buffer - * @unpacked_lun: unpacked LUN to reference for struct se_lun + * @unpacked_lun: LUN the TMR applies to. Ignored if TARGET_SCF_IGNORE_TMR_LUN + * has been set in @flags. * @fabric_context: fabric context for TMR req * @tm_type: Type of TM request * @gfp: gfp type for caller * @tag: referenced task tag for TMR_ABORT_TASK - * @flags: submit cmd flags + * @flags: submit cmd flags (TARGET_SCF_*). * * Callable from all contexts. **/ @@ -1640,7 +1640,14 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, return ret; } - ret = transport_lookup_tmr_lun(se_cmd, unpacked_lun); + if (flags & TARGET_SCF_IGNORE_TMR_LUN) { + WARN_ON_ONCE(tm_type != TMR_ABORT_TASK); + se_cmd->se_lun = NULL; + se_cmd->se_dev = NULL; + se_cmd->se_tmr_req->tmr_dev = NULL; + } else { + ret = transport_lookup_tmr_lun(se_cmd, unpacked_lun); + } if (ret) { /* * For callback during failure handling, push this work off @@ -1650,6 +1657,7 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, schedule_work(&se_cmd->work); return 0; } + WARN_ON_ONCE(tm_type != TMR_ABORT_TASK && !se_cmd->se_dev); transport_generic_handle_tmr(se_cmd); return 0; } @@ -3128,8 +3136,16 @@ static void target_tmr_work(struct work_struct *work) transport_cmd_check_stop_to_fabric(cmd); } -int transport_generic_handle_tmr( - struct se_cmd *cmd) +/** + * transport_generic_handle_tmr - queue a task management function + * + * Note: task management functions for which flag TARGET_SCF_IGNORE_TMR_LUN + * has been set are queued on a separate workqueue. This may cause out-of-order + * execution of task management functions. That is fine though since the + * SCSI Architecture Manual does not require that task management functions + * are executed in order. + */ +int transport_generic_handle_tmr(struct se_cmd *cmd) { unsigned long flags; diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 5cb7583ad701..fb87f0e5d0d6 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -186,6 +186,7 @@ enum target_sc_flags_table { TARGET_SCF_ACK_KREF = 0x02, TARGET_SCF_UNKNOWN_SIZE = 0x04, TARGET_SCF_USE_CPUID = 0x08, + TARGET_SCF_IGNORE_TMR_LUN = 0x10, }; /* fabric independent task management function values */