From patchwork Wed Apr 21 07:55:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12215649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BBC3C43460 for ; Wed, 21 Apr 2021 08:02:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D692E61158 for ; Wed, 21 Apr 2021 08:02:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234116AbhDUIDP (ORCPT ); Wed, 21 Apr 2021 04:03:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32114 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233932AbhDUICp (ORCPT ); Wed, 21 Apr 2021 04:02:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618992130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a1hD/3rjKkdxv5OoEpCwDI/vC5R7zg8GHDnJJ7oJclY=; b=SHxKuONqdk0ycNJA7O7ucLFvcFwg17YQzNIC0utMMiTeHGI7nzYGHhe4iSok3rNTg4dlpp C3lp9AIKn0z7N/Pa3KSbJ9LEt42X6oIAF2WFa0DCdXc1ymGs9zJq0FgZfc6GQdWOV2AWOu 2Ofh8GOzHWgzgDyRDPGwipp/UHYBvU4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-365-7JcF-WwPNS-n2JbZnKuwlQ-1; Wed, 21 Apr 2021 04:02:06 -0400 X-MC-Unique: 7JcF-WwPNS-n2JbZnKuwlQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 232F919253F9; Wed, 21 Apr 2021 07:57:09 +0000 (UTC) Received: from localhost (ovpn-13-15.pek2.redhat.com [10.72.13.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6649E60916; Wed, 21 Apr 2021 07:57:07 +0000 (UTC) From: Ming Lei To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Satish Kharat , Karan Tilak Kumar , David Jeffery Subject: [PATCH 1/5] scsi: fnic: use blk_mq_tagset_busy_iter() to walk scsi commands in fnic_terminate_rport_io Date: Wed, 21 Apr 2021 15:55:39 +0800 Message-Id: <20210421075543.1919826-2-ming.lei@redhat.com> In-Reply-To: <20210421075543.1919826-1-ming.lei@redhat.com> References: <20210421075543.1919826-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org So far, scsi_host_find_tag() is supposed to use in fast path and the passed tag should be active. Convert the scsi command walking into blk_mq_tagset_busy_iter(), which has been one common pattern for handling failure. Cc: Satish Kharat Cc: Karan Tilak Kumar Cc: David Jeffery Signed-off-by: Ming Lei --- drivers/scsi/fnic/fnic_scsi.c | 223 +++++++++++++++++----------------- 1 file changed, 113 insertions(+), 110 deletions(-) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index 36744968378f..522e1b43409d 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -1678,23 +1678,123 @@ static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id) } -void fnic_terminate_rport_io(struct fc_rport *rport) +static bool fnic_terminate_cmd(struct request *req, void *data, bool reserved) { - int tag; - int abt_tag; - int term_cnt = 0; + int abt_tag, tag; struct fnic_io_req *io_req; spinlock_t *io_lock; unsigned long flags; - struct scsi_cmnd *sc; struct scsi_lun fc_lun; - struct fc_rport_libfc_priv *rdata; - struct fc_lport *lport; - struct fnic *fnic; struct fc_rport *cmd_rport; + enum fnic_ioreq_state old_ioreq_state; + struct scsi_cmnd *sc = blk_mq_rq_to_pdu(req); + struct fc_lport *lp = shost_priv(sc->device->host); + struct fnic *fnic = lport_priv(lp); + struct fc_rport *rport; struct reset_stats *reset_stats; struct terminate_stats *term_stats; - enum fnic_ioreq_state old_ioreq_state; + int *term_cnt = data; + + term_stats = &fnic->fnic_stats.term_stats; + reset_stats = &fnic->fnic_stats.reset_stats; + tag = req->tag; + io_lock = fnic_io_lock_tag(fnic, tag); + + spin_lock_irqsave(io_lock, flags); + rport = starget_to_rport(scsi_target(sc->device)); + abt_tag = tag; + io_req = (struct fnic_io_req *)CMD_SP(sc); + if (!io_req) + goto unlock; + + cmd_rport = starget_to_rport(scsi_target(sc->device)); + if (rport != cmd_rport) + goto unlock; + + if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && + (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "fnic_terminate_rport_io dev rst not pending sc 0x%p\n", + sc); + goto unlock; + } + /* + * Found IO that is still pending with firmware and + * belongs to rport that went away + */ + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + goto unlock; + + if (io_req->abts_done) { + shost_printk(KERN_ERR, fnic->lport->host, + "fnic_terminate_rport_io: io_req->abts_done is set " + "state is %s\n", + fnic_ioreq_state_to_str(CMD_STATE(sc))); + } + if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) { + FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, + "fnic_terminate_rport_io " + "IO not yet issued %p tag 0x%x flags " + "%x state %d\n", + sc, tag, CMD_FLAGS(sc), CMD_STATE(sc)); + } + old_ioreq_state = CMD_STATE(sc); + CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; + CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { + atomic64_inc(&reset_stats->device_reset_terminates); + abt_tag = (tag | FNIC_TAG_DEV_RST); + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "fnic_terminate_rport_io dev rst sc 0x%p\n", sc); + } + + BUG_ON(io_req->abts_done); + + FNIC_SCSI_DBG(KERN_DEBUG, + fnic->lport->host, + "fnic_terminate_rport_io: Issuing abts\n"); + + spin_unlock_irqrestore(io_lock, flags); + + /* Now queue the abort command to firmware */ + int_to_scsilun(sc->device->lun, &fc_lun); + + if (fnic_queue_abort_io_req(fnic, abt_tag, + FCPIO_ITMF_ABT_TASK_TERM, + fc_lun.scsi_lun, io_req)) { + /* + * Revert the cmd state back to old state, if + * it hasn't changed in between. This cmd will get + * aborted later by scsi_eh, or cleaned up during + * lun reset + */ + spin_lock_irqsave(io_lock, flags); + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + CMD_STATE(sc) = old_ioreq_state; + spin_unlock_irqrestore(io_lock, flags); + } else { + spin_lock_irqsave(io_lock, flags); + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) + CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; + else + CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; + spin_unlock_irqrestore(io_lock, flags); + atomic64_inc(&term_stats->terminates); + (*term_cnt)++; + } + + return true; + unlock: + spin_unlock_irqrestore(io_lock, flags); + return true; +} + +void fnic_terminate_rport_io(struct fc_rport *rport) +{ + int term_cnt = 0; + struct fc_rport_libfc_priv *rdata; + struct fc_lport *lport; + struct fnic *fnic; if (!rport) { printk(KERN_ERR "fnic_terminate_rport_io: rport is NULL\n"); @@ -1722,108 +1822,11 @@ void fnic_terminate_rport_io(struct fc_rport *rport) if (fnic->in_remove) return; - reset_stats = &fnic->fnic_stats.reset_stats; - term_stats = &fnic->fnic_stats.term_stats; - - for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) { - abt_tag = tag; - io_lock = fnic_io_lock_tag(fnic, tag); - spin_lock_irqsave(io_lock, flags); - sc = scsi_host_find_tag(fnic->lport->host, tag); - if (!sc) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - - io_req = (struct fnic_io_req *)CMD_SP(sc); - if (!io_req) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - - cmd_rport = starget_to_rport(scsi_target(sc->device)); - if (rport != cmd_rport) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - - if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && - (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "fnic_terminate_rport_io dev rst not pending sc 0x%p\n", - sc); - spin_unlock_irqrestore(io_lock, flags); - continue; - } - /* - * Found IO that is still pending with firmware and - * belongs to rport that went away - */ - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - if (io_req->abts_done) { - shost_printk(KERN_ERR, fnic->lport->host, - "fnic_terminate_rport_io: io_req->abts_done is set " - "state is %s\n", - fnic_ioreq_state_to_str(CMD_STATE(sc))); - } - if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) { - FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, - "fnic_terminate_rport_io " - "IO not yet issued %p tag 0x%x flags " - "%x state %d\n", - sc, tag, CMD_FLAGS(sc), CMD_STATE(sc)); - } - old_ioreq_state = CMD_STATE(sc); - CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; - CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { - atomic64_inc(&reset_stats->device_reset_terminates); - abt_tag = (tag | FNIC_TAG_DEV_RST); - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "fnic_terminate_rport_io dev rst sc 0x%p\n", sc); - } - - BUG_ON(io_req->abts_done); - - FNIC_SCSI_DBG(KERN_DEBUG, - fnic->lport->host, - "fnic_terminate_rport_io: Issuing abts\n"); - - spin_unlock_irqrestore(io_lock, flags); - - /* Now queue the abort command to firmware */ - int_to_scsilun(sc->device->lun, &fc_lun); - - if (fnic_queue_abort_io_req(fnic, abt_tag, - FCPIO_ITMF_ABT_TASK_TERM, - fc_lun.scsi_lun, io_req)) { - /* - * Revert the cmd state back to old state, if - * it hasn't changed in between. This cmd will get - * aborted later by scsi_eh, or cleaned up during - * lun reset - */ - spin_lock_irqsave(io_lock, flags); - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) - CMD_STATE(sc) = old_ioreq_state; - spin_unlock_irqrestore(io_lock, flags); - } else { - spin_lock_irqsave(io_lock, flags); - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) - CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; - else - CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; - spin_unlock_irqrestore(io_lock, flags); - atomic64_inc(&term_stats->terminates); - term_cnt++; - } - } - if (term_cnt > atomic64_read(&term_stats->max_terminates)) - atomic64_set(&term_stats->max_terminates, term_cnt); + blk_mq_tagset_busy_iter(&fnic->lport->host->tag_set, + fnic_terminate_cmd, &term_cnt); + if (term_cnt > atomic64_read(&fnic->fnic_stats.term_stats.max_terminates)) + atomic64_set(&fnic->fnic_stats.term_stats.max_terminates, term_cnt); } /* From patchwork Wed Apr 21 07:55:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12215645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 232D5C433B4 for ; Wed, 21 Apr 2021 08:02:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0110B61435 for ; Wed, 21 Apr 2021 08:02:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234193AbhDUIDS (ORCPT ); Wed, 21 Apr 2021 04:03:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50073 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234202AbhDUICz (ORCPT ); Wed, 21 Apr 2021 04:02:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618992142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f2IAoIByH3ev2kKwV7r9bhguSfcT9NvZEm5wcfvwITc=; b=gQTxzJ/rrDriyL2zXJQAmFdCDtJWTqX/J3Y5j/h5xXTubB+Bic3YIDihnqOgm91GRhj/Kr tdqU8Mg2LObe44xbeZag5eJr7BOwCyoOYBOV3lRZeogzlbyPsK9ENHn/YTz9UGuK9Z2Cuw 8iRKL+ltfMCQCdq7+uEgzdZ2jJghhcA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-1-LoBfzvIHOnyHHTKu7QM2eg-1; Wed, 21 Apr 2021 04:02:09 -0400 X-MC-Unique: LoBfzvIHOnyHHTKu7QM2eg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 185DE8E2CD; Wed, 21 Apr 2021 07:57:15 +0000 (UTC) Received: from localhost (ovpn-13-15.pek2.redhat.com [10.72.13.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AB105D9D0; Wed, 21 Apr 2021 07:57:10 +0000 (UTC) From: Ming Lei To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Satish Kharat , Karan Tilak Kumar , David Jeffery Subject: [PATCH 2/5] scsi: fnic: use blk_mq_tagset_busy_iter() to walk scsi commands in fnic_clean_pending_aborts Date: Wed, 21 Apr 2021 15:55:40 +0800 Message-Id: <20210421075543.1919826-3-ming.lei@redhat.com> In-Reply-To: <20210421075543.1919826-1-ming.lei@redhat.com> References: <20210421075543.1919826-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org So far, scsi_host_find_tag() is supposed to use in fast path and the passed tag should be active. Convert the scsi command walking into blk_mq_tagset_busy_iter() in fnic_terminate_rport_io, which has been one common pattern for handling failure. Cc: Satish Kharat Cc: Karan Tilak Kumar Cc: David Jeffery Signed-off-by: Ming Lei --- drivers/scsi/fnic/fnic_scsi.c | 283 ++++++++++++++++++---------------- 1 file changed, 153 insertions(+), 130 deletions(-) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index 522e1b43409d..12503b59f42c 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -2121,172 +2121,195 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic, return ret; } -/* - * Clean up any pending aborts on the lun - * For each outstanding IO on this lun, whose abort is not completed by fw, - * issue a local abort. Wait for abort to complete. Return 0 if all commands - * successfully aborted, 1 otherwise - */ -static int fnic_clean_pending_aborts(struct fnic *fnic, - struct scsi_cmnd *lr_sc, - bool new_sc) +struct fnic_clean_pending_aborts_data { + struct fnic *fnic; + struct scsi_cmnd *lr_sc; + bool new_sc; + int ret; +}; +static bool __fnic_clean_pending_aborts(struct request *req, void *data, + bool reserved) { - int tag, abt_tag; + struct fnic_clean_pending_aborts_data *aborts_data = data; + struct fnic *fnic = aborts_data->fnic; + struct scsi_cmnd *lr_sc = aborts_data->lr_sc; + bool new_sc = aborts_data->new_sc; + struct scsi_device *lun_dev = lr_sc->device; + struct scsi_cmnd *sc = blk_mq_rq_to_pdu(req); + int tag = req->tag; + int abt_tag; struct fnic_io_req *io_req; spinlock_t *io_lock; unsigned long flags; - int ret = 0; - struct scsi_cmnd *sc; struct scsi_lun fc_lun; - struct scsi_device *lun_dev = lr_sc->device; DECLARE_COMPLETION_ONSTACK(tm_done); enum fnic_ioreq_state old_ioreq_state; - for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) { - io_lock = fnic_io_lock_tag(fnic, tag); - spin_lock_irqsave(io_lock, flags); - sc = scsi_host_find_tag(fnic->lport->host, tag); - /* - * ignore this lun reset cmd if issued using new SC - * or cmds that do not belong to this lun - */ - if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - - io_req = (struct fnic_io_req *)CMD_SP(sc); + io_lock = fnic_io_lock_tag(fnic, tag); + spin_lock_irqsave(io_lock, flags); + sc = scsi_host_find_tag(fnic->lport->host, tag); + /* + * ignore this lun reset cmd if issued using new SC + * or cmds that do not belong to this lun + */ + if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) + goto unlock; - if (!io_req || sc->device != lun_dev) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + io_req = (struct fnic_io_req *)CMD_SP(sc); - /* - * Found IO that is still pending with firmware and - * belongs to the LUN that we are resetting - */ - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "Found IO in %s on lun\n", - fnic_ioreq_state_to_str(CMD_STATE(sc))); + if (!io_req || sc->device != lun_dev) + goto unlock; - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && - (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { - FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, - "%s dev rst not pending sc 0x%p\n", __func__, - sc); - spin_unlock_irqrestore(io_lock, flags); - continue; - } + /* + * Found IO that is still pending with firmware and + * belongs to the LUN that we are resetting + */ + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "Found IO in %s on lun\n", + fnic_ioreq_state_to_str(CMD_STATE(sc))); - if (io_req->abts_done) - shost_printk(KERN_ERR, fnic->lport->host, - "%s: io_req->abts_done is set state is %s\n", - __func__, fnic_ioreq_state_to_str(CMD_STATE(sc))); - old_ioreq_state = CMD_STATE(sc); - /* - * Any pending IO issued prior to reset is expected to be - * in abts pending state, if not we need to set - * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending. - * When IO is completed, the IO will be handed over and - * handled in this function. - */ - CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + goto unlock; - BUG_ON(io_req->abts_done); + if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && + (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { + FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, + "%s dev rst not pending sc 0x%p\n", __func__, + sc); + goto unlock; + } - abt_tag = tag; - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { - abt_tag |= FNIC_TAG_DEV_RST; - FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, - "%s: dev rst sc 0x%p\n", __func__, sc); - } + if (io_req->abts_done) + shost_printk(KERN_ERR, fnic->lport->host, + "%s: io_req->abts_done is set state is %s\n", + __func__, fnic_ioreq_state_to_str(CMD_STATE(sc))); + old_ioreq_state = CMD_STATE(sc); + /* + * Any pending IO issued prior to reset is expected to be + * in abts pending state, if not we need to set + * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending. + * When IO is completed, the IO will be handed over and + * handled in this function. + */ + CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; - CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; - io_req->abts_done = &tm_done; - spin_unlock_irqrestore(io_lock, flags); + BUG_ON(io_req->abts_done); - /* Now queue the abort command to firmware */ - int_to_scsilun(sc->device->lun, &fc_lun); + abt_tag = tag; + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { + abt_tag |= FNIC_TAG_DEV_RST; + FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, + "%s: dev rst sc 0x%p\n", __func__, sc); + } - if (fnic_queue_abort_io_req(fnic, abt_tag, - FCPIO_ITMF_ABT_TASK_TERM, - fc_lun.scsi_lun, io_req)) { - spin_lock_irqsave(io_lock, flags); - io_req = (struct fnic_io_req *)CMD_SP(sc); - if (io_req) - io_req->abts_done = NULL; - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) - CMD_STATE(sc) = old_ioreq_state; - spin_unlock_irqrestore(io_lock, flags); - ret = 1; - goto clean_pending_aborts_end; - } else { - spin_lock_irqsave(io_lock, flags); - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) - CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; - spin_unlock_irqrestore(io_lock, flags); - } - CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; + CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; + io_req->abts_done = &tm_done; + spin_unlock_irqrestore(io_lock, flags); - wait_for_completion_timeout(&tm_done, - msecs_to_jiffies - (fnic->config.ed_tov)); + /* Now queue the abort command to firmware */ + int_to_scsilun(sc->device->lun, &fc_lun); - /* Recheck cmd state to check if it is now aborted */ + if (fnic_queue_abort_io_req(fnic, abt_tag, + FCPIO_ITMF_ABT_TASK_TERM, + fc_lun.scsi_lun, io_req)) { spin_lock_irqsave(io_lock, flags); io_req = (struct fnic_io_req *)CMD_SP(sc); - if (!io_req) { - spin_unlock_irqrestore(io_lock, flags); - CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL; - continue; - } + if (io_req) + io_req->abts_done = NULL; + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + CMD_STATE(sc) = old_ioreq_state; + spin_unlock_irqrestore(io_lock, flags); + goto fail; + } else { + spin_lock_irqsave(io_lock, flags); + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) + CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; + spin_unlock_irqrestore(io_lock, flags); + } + CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; - io_req->abts_done = NULL; + wait_for_completion_timeout(&tm_done, + msecs_to_jiffies + (fnic->config.ed_tov)); - /* if abort is still pending with fw, fail */ - if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) { - spin_unlock_irqrestore(io_lock, flags); - CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE; - ret = 1; - goto clean_pending_aborts_end; - } - CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE; + /* Recheck cmd state to check if it is now aborted */ + spin_lock_irqsave(io_lock, flags); + io_req = (struct fnic_io_req *)CMD_SP(sc); + if (!io_req) { + CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL; + goto unlock; + } - /* original sc used for lr is handled by dev reset code */ - if (sc != lr_sc) - CMD_SP(sc) = NULL; + io_req->abts_done = NULL; + + /* if abort is still pending with fw, fail */ + if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) { spin_unlock_irqrestore(io_lock, flags); + CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE; + goto fail; + } + CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE; - /* original sc used for lr is handled by dev reset code */ - if (sc != lr_sc) { - fnic_release_ioreq_buf(fnic, io_req, sc); - mempool_free(io_req, fnic->io_req_pool); - } + /* original sc used for lr is handled by dev reset code */ + if (sc != lr_sc) + CMD_SP(sc) = NULL; + spin_unlock_irqrestore(io_lock, flags); - /* - * Any IO is returned during reset, it needs to call scsi_done - * to return the scsi_cmnd to upper layer. - */ - if (sc->scsi_done) { - /* Set result to let upper SCSI layer retry */ - sc->result = DID_RESET << 16; - sc->scsi_done(sc); - } + /* original sc used for lr is handled by dev reset code */ + if (sc != lr_sc) { + fnic_release_ioreq_buf(fnic, io_req, sc); + mempool_free(io_req, fnic->io_req_pool); } + /* + * Any IO is returned during reset, it needs to call scsi_done + * to return the scsi_cmnd to upper layer. + */ + if (sc->scsi_done) { + /* Set result to let upper SCSI layer retry */ + sc->result = DID_RESET << 16; + sc->scsi_done(sc); + } + return true; + unlock: + spin_unlock_irqrestore(io_lock, flags); + return true; + fail: + aborts_data->ret = 1; + return false; +} + +/* + * Clean up any pending aborts on the lun + * For each outstanding IO on this lun, whose abort is not completed by fw, + * issue a local abort. Wait for abort to complete. Return 0 if all commands + * successfully aborted, 1 otherwise + */ +static int fnic_clean_pending_aborts(struct fnic *fnic, + struct scsi_cmnd *lr_sc, + bool new_sc) + +{ + int ret = 0; + struct fnic_clean_pending_aborts_data data = { + .fnic = fnic, + .lr_sc = lr_sc, + .new_sc = new_sc, + .ret = 0, + }; + + blk_mq_tagset_busy_iter(&fnic->lport->host->tag_set, + __fnic_clean_pending_aborts, &data); + if (data.ret) + return data.ret; + schedule_timeout(msecs_to_jiffies(2 * fnic->config.ed_tov)); /* walk again to check, if IOs are still pending in fw */ if (fnic_is_abts_pending(fnic, lr_sc)) ret = FAILED; -clean_pending_aborts_end: return ret; } From patchwork Wed Apr 21 07:55:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12215641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 209F6C433ED for ; Wed, 21 Apr 2021 08:02:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E17D261158 for ; Wed, 21 Apr 2021 08:02:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232140AbhDUIDO (ORCPT ); Wed, 21 Apr 2021 04:03:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43209 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234182AbhDUICd (ORCPT ); Wed, 21 Apr 2021 04:02:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618992120; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I20lH1NKD9pMUYhKPg9cB5HE4ySbNgtDoQh38ubAJ38=; b=agYx+FnNtDqS6lo8EcTWOSKEaLp2iDtKj+EcXpOZkDsoqZNd0AzSPBs99+HhzEqRLagmPB rFlf6A7x+K+y5cIVTGhKB6pozfm2G4TASWLrJW+gqxjIpkf/EAYRQTa1h1SQfs6rOkVDWR 4MIXNKLQ9ALRNE6T9w3U0HaXgWXLeuU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-78-DW6EQPUpMnGQB96LBAyYPQ-1; Wed, 21 Apr 2021 04:01:58 -0400 X-MC-Unique: DW6EQPUpMnGQB96LBAyYPQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A579E10978D8; Wed, 21 Apr 2021 07:57:21 +0000 (UTC) Received: from localhost (ovpn-13-15.pek2.redhat.com [10.72.13.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8012D19D80; Wed, 21 Apr 2021 07:57:17 +0000 (UTC) From: Ming Lei To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Satish Kharat , Karan Tilak Kumar , David Jeffery Subject: [PATCH 3/5] scsi: fnic: use blk_mq_tagset_busy_iter() to walk scsi commands in fnic_cleanup_io Date: Wed, 21 Apr 2021 15:55:41 +0800 Message-Id: <20210421075543.1919826-4-ming.lei@redhat.com> In-Reply-To: <20210421075543.1919826-1-ming.lei@redhat.com> References: <20210421075543.1919826-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org So far, scsi_host_find_tag() is supposed to use in fast path and the passed tag should be active. Convert the scsi command walking into blk_mq_tagset_busy_iter(), which has been one common pattern for handling failure. Cc: Satish Kharat Cc: Karan Tilak Kumar Cc: David Jeffery Signed-off-by: Ming Lei --- drivers/scsi/fnic/fnic_scsi.c | 139 +++++++++++++++++----------------- 1 file changed, 70 insertions(+), 69 deletions(-) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index 12503b59f42c..f052526066c1 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -1361,93 +1361,94 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do) return wq_work_done; } -static void fnic_cleanup_io(struct fnic *fnic, int exclude_id) +static bool fnic_cleanup_cmd(struct request *req, void *data, bool reserved) { - int i; struct fnic_io_req *io_req; unsigned long flags = 0; - struct scsi_cmnd *sc; spinlock_t *io_lock; unsigned long start_time = 0; + struct scsi_cmnd *sc = blk_mq_rq_to_pdu(req); + struct fc_lport *lp = shost_priv(sc->device->host); + struct fnic *fnic = lport_priv(lp); struct fnic_stats *fnic_stats = &fnic->fnic_stats; + int *exclude_id = data; - for (i = 0; i < fnic->fnic_max_tag_id; i++) { - if (i == exclude_id) - continue; + if (*exclude_id == req->tag) + return true; - io_lock = fnic_io_lock_tag(fnic, i); - spin_lock_irqsave(io_lock, flags); - sc = scsi_host_find_tag(fnic->lport->host, i); - if (!sc) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + io_lock = fnic_io_lock_tag(fnic, req->tag); + spin_lock_irqsave(io_lock, flags); - io_req = (struct fnic_io_req *)CMD_SP(sc); - if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && - !(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) { - /* - * We will be here only when FW completes reset - * without sending completions for outstanding ios. - */ - CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE; - if (io_req && io_req->dr_done) - complete(io_req->dr_done); - else if (io_req && io_req->abts_done) - complete(io_req->abts_done); - spin_unlock_irqrestore(io_lock, flags); - continue; - } else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - if (!io_req) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + io_req = (struct fnic_io_req *)CMD_SP(sc); + if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && + !(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) { + /* + * We will be here only when FW completes reset + * without sending completions for outstanding ios. + */ + CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE; + if (io_req && io_req->dr_done) + complete(io_req->dr_done); + else if (io_req && io_req->abts_done) + complete(io_req->abts_done); + goto unlock; + } else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { + goto unlock; + } + if (!io_req) + goto unlock; - CMD_SP(sc) = NULL; + CMD_SP(sc) = NULL; - spin_unlock_irqrestore(io_lock, flags); + spin_unlock_irqrestore(io_lock, flags); - /* - * If there is a scsi_cmnd associated with this io_req, then - * free the corresponding state - */ - start_time = io_req->start_time; - fnic_release_ioreq_buf(fnic, io_req, sc); - mempool_free(io_req, fnic->io_req_pool); + /* + * If there is a scsi_cmnd associated with this io_req, then + * free the corresponding state + */ + start_time = io_req->start_time; + fnic_release_ioreq_buf(fnic, io_req, sc); + mempool_free(io_req, fnic->io_req_pool); - sc->result = DID_TRANSPORT_DISRUPTED << 16; - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "%s: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n", - __func__, sc->request->tag, sc, - (jiffies - start_time)); + sc->result = DID_TRANSPORT_DISRUPTED << 16; + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "%s: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n", + __func__, sc->request->tag, sc, + (jiffies - start_time)); - if (atomic64_read(&fnic->io_cmpl_skip)) - atomic64_dec(&fnic->io_cmpl_skip); - else - atomic64_inc(&fnic_stats->io_stats.io_completions); + if (atomic64_read(&fnic->io_cmpl_skip)) + atomic64_dec(&fnic->io_cmpl_skip); + else + atomic64_inc(&fnic_stats->io_stats.io_completions); - /* Complete the command to SCSI */ - if (sc->scsi_done) { - if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) - shost_printk(KERN_ERR, fnic->lport->host, - "Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n", - sc->request->tag, sc); + /* Complete the command to SCSI */ + if (sc->scsi_done) { + if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) + shost_printk(KERN_ERR, fnic->lport->host, + "Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n", + sc->request->tag, sc); - FNIC_TRACE(fnic_cleanup_io, - sc->device->host->host_no, i, sc, - jiffies_to_msecs(jiffies - start_time), - 0, ((u64)sc->cmnd[0] << 32 | - (u64)sc->cmnd[2] << 24 | - (u64)sc->cmnd[3] << 16 | - (u64)sc->cmnd[4] << 8 | sc->cmnd[5]), - (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc))); + FNIC_TRACE(fnic_cleanup_io, + sc->device->host->host_no, req->tag, sc, + jiffies_to_msecs(jiffies - start_time), + 0, ((u64)sc->cmnd[0] << 32 | + (u64)sc->cmnd[2] << 24 | + (u64)sc->cmnd[3] << 16 | + (u64)sc->cmnd[4] << 8 | sc->cmnd[5]), + (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc))); - sc->scsi_done(sc); - } + sc->scsi_done(sc); } + return true; + unlock: + spin_unlock_irqrestore(io_lock, flags); + return true; +} + +static void fnic_cleanup_io(struct fnic *fnic, int exclude_id) +{ + blk_mq_tagset_busy_iter(&fnic->lport->host->tag_set, + fnic_cleanup_cmd, &exclude_id); } void fnic_wq_copy_cleanup_handler(struct vnic_wq_copy *wq, From patchwork Wed Apr 21 07:55:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12215643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53B64C43470 for ; Wed, 21 Apr 2021 08:02:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DFA8613ED for ; Wed, 21 Apr 2021 08:02:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234154AbhDUIDR (ORCPT ); Wed, 21 Apr 2021 04:03:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49757 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234193AbhDUICp (ORCPT ); Wed, 21 Apr 2021 04:02:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618992131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X4HexoTyGHWd89nZ9hhiPgbKCQdbnM1+vT1JnkCOLJM=; b=OXHwjVrAIX4bfPiOUnisfUvRGjJ1a5YQWJWrovv+RPRUxcDFj7rjPE67vzQxCBNXATnByR ZGHoCdVX9pUuwN9GV3k8y+hk17qfkC2fyRJ+l4bMsMIrLRKQjn9Eh3H5drR+nYsIzKkVHD jbWm8hUTiNDbROXzzADNROFi2bEGi6s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-101-Aw2qmczbMByhmGgnq3K6Ww-1; Wed, 21 Apr 2021 04:02:10 -0400 X-MC-Unique: Aw2qmczbMByhmGgnq3K6Ww-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1BF3B8E2E8; Wed, 21 Apr 2021 07:57:28 +0000 (UTC) Received: from localhost (ovpn-13-15.pek2.redhat.com [10.72.13.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 123A860C05; Wed, 21 Apr 2021 07:57:23 +0000 (UTC) From: Ming Lei To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Satish Kharat , Karan Tilak Kumar , David Jeffery Subject: [PATCH 4/5] scsi: fnic: use blk_mq_tagset_busy_iter() to walk scsi commands in fnic_rport_exch_reset Date: Wed, 21 Apr 2021 15:55:42 +0800 Message-Id: <20210421075543.1919826-5-ming.lei@redhat.com> In-Reply-To: <20210421075543.1919826-1-ming.lei@redhat.com> References: <20210421075543.1919826-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org So far, scsi_host_find_tag() is supposed to use in fast path and the passed tag should be active. Convert the scsi command walking into blk_mq_tagset_busy_iter(), which has been one common pattern for handling failure. Cc: Satish Kharat Cc: Karan Tilak Kumar Cc: David Jeffery Signed-off-by: Ming Lei --- drivers/scsi/fnic/fnic_scsi.c | 202 ++++++++++++++++++---------------- 1 file changed, 107 insertions(+), 95 deletions(-) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index f052526066c1..ecbf4f5c5a07 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -1559,124 +1559,136 @@ static inline int fnic_queue_abort_io_req(struct fnic *fnic, int tag, return 0; } -static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id) +struct fnic_reset_data { + u32 port_id; + int term_cnt; +}; + +static bool fnic_reset_cmd(struct request *req, void *data, bool reserved) { + struct fnic_reset_data *reset_data = data; int tag; int abt_tag; - int term_cnt = 0; struct fnic_io_req *io_req; spinlock_t *io_lock; unsigned long flags; - struct scsi_cmnd *sc; - struct reset_stats *reset_stats = &fnic->fnic_stats.reset_stats; - struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats; struct scsi_lun fc_lun; enum fnic_ioreq_state old_ioreq_state; + struct scsi_cmnd *sc = blk_mq_rq_to_pdu(req); + struct fc_lport *lp = shost_priv(sc->device->host); + struct fnic *fnic = lport_priv(lp); + struct reset_stats *reset_stats = &fnic->fnic_stats.reset_stats; + struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats; - FNIC_SCSI_DBG(KERN_DEBUG, - fnic->lport->host, - "fnic_rport_exch_reset called portid 0x%06x\n", - port_id); + tag = req->tag; + abt_tag = tag; + io_lock = fnic_io_lock_tag(fnic, tag); + spin_lock_irqsave(io_lock, flags); - if (fnic->in_remove) - return; + io_req = (struct fnic_io_req *)CMD_SP(sc); - for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) { - abt_tag = tag; - io_lock = fnic_io_lock_tag(fnic, tag); - spin_lock_irqsave(io_lock, flags); - sc = scsi_host_find_tag(fnic->lport->host, tag); - if (!sc) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + if (!io_req || io_req->port_id != reset_data->port_id) + goto unlock; - io_req = (struct fnic_io_req *)CMD_SP(sc); + if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && + (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "fnic_rport_exch_reset dev rst not pending sc 0x%p\n", + sc); + goto unlock; + } - if (!io_req || io_req->port_id != port_id) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + /* + * Found IO that is still pending with firmware and + * belongs to rport that went away + */ + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + goto unlock; - if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) && - (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) { - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "fnic_rport_exch_reset dev rst not pending sc 0x%p\n", - sc); - spin_unlock_irqrestore(io_lock, flags); - continue; - } + if (io_req->abts_done) { + shost_printk(KERN_ERR, fnic->lport->host, + "fnic_rport_exch_reset: io_req->abts_done is set " + "state is %s\n", + fnic_ioreq_state_to_str(CMD_STATE(sc))); + } - /* - * Found IO that is still pending with firmware and - * belongs to rport that went away - */ - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } - if (io_req->abts_done) { - shost_printk(KERN_ERR, fnic->lport->host, - "fnic_rport_exch_reset: io_req->abts_done is set " - "state is %s\n", - fnic_ioreq_state_to_str(CMD_STATE(sc))); - } + if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) { + shost_printk(KERN_ERR, fnic->lport->host, + "rport_exch_reset " + "IO not yet issued %p tag 0x%x flags " + "%x state %d\n", + sc, tag, CMD_FLAGS(sc), CMD_STATE(sc)); + } + old_ioreq_state = CMD_STATE(sc); + CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; + CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { + atomic64_inc(&reset_stats->device_reset_terminates); + abt_tag = (tag | FNIC_TAG_DEV_RST); + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "fnic_rport_exch_reset dev rst sc 0x%p\n", + sc); + } - if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) { - shost_printk(KERN_ERR, fnic->lport->host, - "rport_exch_reset " - "IO not yet issued %p tag 0x%x flags " - "%x state %d\n", - sc, tag, CMD_FLAGS(sc), CMD_STATE(sc)); - } - old_ioreq_state = CMD_STATE(sc); - CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING; - CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE; - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) { - atomic64_inc(&reset_stats->device_reset_terminates); - abt_tag = (tag | FNIC_TAG_DEV_RST); - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "fnic_rport_exch_reset dev rst sc 0x%p\n", - sc); - } + BUG_ON(io_req->abts_done); - BUG_ON(io_req->abts_done); + FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, + "fnic_rport_reset_exch: Issuing abts\n"); - FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, - "fnic_rport_reset_exch: Issuing abts\n"); + spin_unlock_irqrestore(io_lock, flags); + /* Now queue the abort command to firmware */ + int_to_scsilun(sc->device->lun, &fc_lun); + + if (fnic_queue_abort_io_req(fnic, abt_tag, + FCPIO_ITMF_ABT_TASK_TERM, + fc_lun.scsi_lun, io_req)) { + /* + * Revert the cmd state back to old state, if + * it hasn't changed in between. This cmd will get + * aborted later by scsi_eh, or cleaned up during + * lun reset + */ + spin_lock_irqsave(io_lock, flags); + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + CMD_STATE(sc) = old_ioreq_state; spin_unlock_irqrestore(io_lock, flags); + } else { + spin_lock_irqsave(io_lock, flags); + if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) + CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; + else + CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; + spin_unlock_irqrestore(io_lock, flags); + atomic64_inc(&term_stats->terminates); + reset_data->term_cnt++; + } + return true; +unlock: + spin_unlock_irqrestore(io_lock, flags); + return true; +} - /* Now queue the abort command to firmware */ - int_to_scsilun(sc->device->lun, &fc_lun); +static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id) +{ + struct fnic_reset_data data = { + .port_id = port_id, + .term_cnt = 0, + }; - if (fnic_queue_abort_io_req(fnic, abt_tag, - FCPIO_ITMF_ABT_TASK_TERM, - fc_lun.scsi_lun, io_req)) { - /* - * Revert the cmd state back to old state, if - * it hasn't changed in between. This cmd will get - * aborted later by scsi_eh, or cleaned up during - * lun reset - */ - spin_lock_irqsave(io_lock, flags); - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) - CMD_STATE(sc) = old_ioreq_state; - spin_unlock_irqrestore(io_lock, flags); - } else { - spin_lock_irqsave(io_lock, flags); - if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) - CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED; - else - CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED; - spin_unlock_irqrestore(io_lock, flags); - atomic64_inc(&term_stats->terminates); - term_cnt++; - } - } - if (term_cnt > atomic64_read(&term_stats->max_terminates)) - atomic64_set(&term_stats->max_terminates, term_cnt); + FNIC_SCSI_DBG(KERN_DEBUG, + fnic->lport->host, + "fnic_rport_exch_reset called portid 0x%06x\n", + port_id); + + if (fnic->in_remove) + return; + + blk_mq_tagset_busy_iter(&fnic->lport->host->tag_set, + fnic_reset_cmd, &data); + if (data.term_cnt > atomic64_read(&fnic->fnic_stats.term_stats.max_terminates)) + atomic64_set(&fnic->fnic_stats.term_stats.max_terminates, data.term_cnt); } static bool fnic_terminate_cmd(struct request *req, void *data, bool reserved) From patchwork Wed Apr 21 07:55:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12215647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C86ECC43461 for ; Wed, 21 Apr 2021 08:02:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91C2161433 for ; Wed, 21 Apr 2021 08:02:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234202AbhDUIDU (ORCPT ); Wed, 21 Apr 2021 04:03:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26166 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234160AbhDUIDG (ORCPT ); Wed, 21 Apr 2021 04:03:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618992153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5O/1WT/nJmCgZV1GTGTm/Hp3TOh6bWTc8vjsZOKmgZM=; b=LXc1YXViJ8Cujeyct8Lg751KoVn157j6cYhtpSgQl/izz5zeKf5AiiCNjybDs7k2zRVHON CsYVnlPo4HlkMB5ciXvuf86S6/mNR5B3FiHyfXM1rguF6uBpYDgWwYcU4FKCfClETZxjjY AxhUXMRpH8ReQr6oVVTOBbD8ZXFJF/I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-137-abXYSjNbN2m_j3-jKdcYPg-1; Wed, 21 Apr 2021 04:02:24 -0400 X-MC-Unique: abXYSjNbN2m_j3-jKdcYPg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 10A021087FEB; Wed, 21 Apr 2021 07:57:35 +0000 (UTC) Received: from localhost (ovpn-13-15.pek2.redhat.com [10.72.13.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id B39935C1A1; Wed, 21 Apr 2021 07:57:30 +0000 (UTC) From: Ming Lei To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Satish Kharat , Karan Tilak Kumar , David Jeffery Subject: [PATCH 5/5] scsi: fnic: use blk_mq_tagset_busy_iter() to walk scsi commands in fnic_is_abts_pending Date: Wed, 21 Apr 2021 15:55:43 +0800 Message-Id: <20210421075543.1919826-6-ming.lei@redhat.com> In-Reply-To: <20210421075543.1919826-1-ming.lei@redhat.com> References: <20210421075543.1919826-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org So far, scsi_host_find_tag() is supposed to use in fast path and the passed tag should be active. Convert the scsi command walking into blk_mq_tagset_busy_iter(), which has been one common pattern for handling failure. Cc: Satish Kharat Cc: Karan Tilak Kumar Cc: David Jeffery Signed-off-by: Ming Lei --- drivers/scsi/fnic/fnic_scsi.c | 96 ++++++++++++++++++++--------------- 1 file changed, 55 insertions(+), 41 deletions(-) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index ecbf4f5c5a07..af8e860f488e 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -2814,58 +2814,72 @@ void fnic_exch_mgr_reset(struct fc_lport *lp, u32 sid, u32 did) } -/* - * fnic_is_abts_pending() is a helper function that - * walks through tag map to check if there is any IOs pending,if there is one, - * then it returns 1 (true), otherwise 0 (false) - * if @lr_sc is non NULL, then it checks IOs specific to particular LUN, - * otherwise, it checks for all IOs. - */ -int fnic_is_abts_pending(struct fnic *fnic, struct scsi_cmnd *lr_sc) +struct fnic_is_abts_pending_data { + struct scsi_cmnd *lr_sc; + int ret; +}; + +static bool __fnic_is_abts_pending(struct request *req, void *data, + bool reserved) { - int tag; + struct fnic_is_abts_pending_data *pdata = data; struct fnic_io_req *io_req; spinlock_t *io_lock; unsigned long flags; - int ret = 0; - struct scsi_cmnd *sc; struct scsi_device *lun_dev = NULL; + struct scsi_cmnd *sc = blk_mq_rq_to_pdu(req); + struct fc_lport *lp = shost_priv(sc->device->host); + struct fnic *fnic = lport_priv(lp); - if (lr_sc) - lun_dev = lr_sc->device; + if (pdata->lr_sc) + lun_dev = pdata->lr_sc->device; - /* walk again to check, if IOs are still pending in fw */ - for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) { - sc = scsi_host_find_tag(fnic->lport->host, tag); - /* - * ignore this lun reset cmd or cmds that do not belong to - * this lun - */ - if (!sc || (lr_sc && (sc->device != lun_dev || sc == lr_sc))) - continue; + /* + * ignore this lun reset cmd or cmds that do not belong to + * this lun + */ + if (!sc || (pdata->lr_sc && (sc->device != lun_dev || + sc == pdata->lr_sc))) + return true; - io_lock = fnic_io_lock_hash(fnic, sc); - spin_lock_irqsave(io_lock, flags); + io_lock = fnic_io_lock_hash(fnic, sc); + spin_lock_irqsave(io_lock, flags); - io_req = (struct fnic_io_req *)CMD_SP(sc); + io_req = (struct fnic_io_req *)CMD_SP(sc); - if (!io_req || sc->device != lun_dev) { - spin_unlock_irqrestore(io_lock, flags); - continue; - } + if (!io_req || sc->device != lun_dev) + goto unlock; - /* - * Found IO that is still pending with firmware and - * belongs to the LUN that we are resetting - */ - FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, - "Found IO in %s on lun\n", - fnic_ioreq_state_to_str(CMD_STATE(sc))); + /* + * Found IO that is still pending with firmware and + * belongs to the LUN that we are resetting + */ + FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, + "Found IO in %s on lun\n", + fnic_ioreq_state_to_str(CMD_STATE(sc))); - if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) - ret = 1; - spin_unlock_irqrestore(io_lock, flags); - } + if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) + pdata->ret = 1; + unlock: + spin_unlock_irqrestore(io_lock, flags); + return true; +} - return ret; +/* + * fnic_is_abts_pending() is a helper function that + * walks through tag map to check if there is any IOs pending,if there is one, + * then it returns 1 (true), otherwise 0 (false) + * if @lr_sc is non NULL, then it checks IOs specific to particular LUN, + * otherwise, it checks for all IOs. + */ +int fnic_is_abts_pending(struct fnic *fnic, struct scsi_cmnd *lr_sc) +{ + struct fnic_is_abts_pending_data data = { + .lr_sc = lr_sc, + .ret = 0, + }; + + blk_mq_tagset_busy_iter(&fnic->lport->host->tag_set, + __fnic_is_abts_pending, &data); + return data.ret; }