From patchwork Fri Sep 22 06:17:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 9965247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1EA38602D8 for ; Fri, 22 Sep 2017 06:18:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00D9429707 for ; Fri, 22 Sep 2017 06:18:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA10A2970B; Fri, 22 Sep 2017 06:18:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78E5A29707 for ; Fri, 22 Sep 2017 06:18:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751880AbdIVGS0 (ORCPT ); Fri, 22 Sep 2017 02:18:26 -0400 Received: from mail-pg0-f66.google.com ([74.125.83.66]:34868 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751868AbdIVGSL (ORCPT ); Fri, 22 Sep 2017 02:18:11 -0400 Received: by mail-pg0-f66.google.com with SMTP id j16so123281pga.2 for ; Thu, 21 Sep 2017 23:18:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=GXZ+WWwNP9YxIKF3e4PAojxxdiOAgFvY3knyd34qFdk=; b=DDianxRyssvwx7AR+WgQOmxc1P92GvjXgOyY0MwfU8UXUko+cFCwCkzvqhnUOTojf7 FWQa5ba3W5bStPrSg0/sABMHqTLFeAvtSqLKm3DVT4bOc8broKxSewOBxEJDLbuv4cF8 L8op7pMLNQxtXYyRD+SLUIu/LZqoQ+g2G8rZlCl0eNI2bA18iWaS+9PKkXii2lCxaLLr K8ypmKrmrPgKf6f0G06gWZecjc6m+PHwgjIk3jDSYuS+PG48feo41kIm36fy4UiEExi3 C3ZmKQg/hDaJiVYecV1S28nNerXpkL8x+1Fo8k4HvHqDUUaNk38DQTAFU0DjQxsVduZs Ppzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GXZ+WWwNP9YxIKF3e4PAojxxdiOAgFvY3knyd34qFdk=; b=AANJc+qDzzeNXx1wx/50UR39FCYyJtrl9WXBOzNExHJaCONbO3nCkvXQS/lhdmzarK mpxmEW6R21Z9z9jUfPEKbc1IWIN9PCPwNuW1P7zjlSXFI0Zyvy7fyv/ndVv6V7C2o19/ n9t4/i1dXSUHUhsI4ZRvtc+ZN2+QwaSsUQd68/3agVfi/kJywgjXXDnKIb0UaYeaefcy /AQpFMRmCRdrCJhDAiV3sFLBrhf0Pw9fNyPPrOfQI/F5BUteBWkLLbPihDWMIOzBcNyQ 7LmFYstKUXznG/r8XJA2JYCoyERhVBnbmk4wlUhIDRIdpxK6wtonYMV08+Io/KeDyfpK nTFw== X-Gm-Message-State: AHPjjUgxBgoMhWHEfAGipoCGXZlMZXg4r8YYnOUqspqH5UJg1pTRduFP LCgu/2XagudqCgQucYw7bJaWvQ== X-Google-Smtp-Source: AOwi7QAgMP1OtZnmbeg3ydHdKmALigypj+gbTCQ+sz8Xk0DuNBjvnbhduiTyGshRWsVHwf6PzGZkCw== X-Received: by 10.99.67.193 with SMTP id q184mr8103278pga.56.1506061091141; Thu, 21 Sep 2017 23:18:11 -0700 (PDT) Received: from os42.localdomain (ip68-4-115-207.oc.oc.cox.net. [68.4.115.207]) by smtp.gmail.com with ESMTPSA id e87sm5911355pfl.46.2017.09.21.23.18.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 21 Sep 2017 23:18:10 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: Dick Kennedy , James Smart Subject: [PATCH 16/19] lpfc: Ensure io aborts interlocked with the target. Date: Thu, 21 Sep 2017 23:17:38 -0700 Message-Id: <20170922061741.12347-17-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20170922061741.12347-1-jsmart2021@gmail.com> References: <20170922061741.12347-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dick Kennedy Before releasing nvme io back to the io stack for possible retry on other paths, ensure the io termination is interlocked with the target device by ensuring the entire ABTS-LS protocol is complete. Additionally, FC-NVME ABTS-LS protocol does not use RRQ. Remove RRQ behavior from ABTS-LS. Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reviewed-by: Johannes Thumshirn --- drivers/scsi/lpfc/lpfc_nvme.c | 59 +++++++++++++++++++++++++------------------ 1 file changed, 34 insertions(+), 25 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 02ba06f364c4..e3642c1890ea 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -850,7 +850,7 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, } else { lpfc_ncmd->status = (bf_get(lpfc_wcqe_c_status, wcqe) & LPFC_IOCB_STATUS_MASK); - lpfc_ncmd->result = wcqe->parameter; + lpfc_ncmd->result = (wcqe->parameter & IOERR_PARAM_MASK); /* For NVME, the only failure path that results in an * IO error is when the adapter rejects it. All other @@ -884,6 +884,17 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, lpfc_ncmd->status, lpfc_ncmd->result, wcqe->total_data_placed); break; + case IOSTAT_LOCAL_REJECT: + /* Let fall through to set command final state. */ + if (lpfc_ncmd->result == IOERR_ABORT_REQUESTED) + lpfc_printf_vlog(vport, KERN_INFO, + LOG_NVME_IOERR, + "6032 Delay Aborted cmd %p " + "nvme cmd %p, xri x%x, " + "xb %d\n", + lpfc_ncmd, nCmd, + lpfc_ncmd->cur_iocbq.sli4_xritag, + bf_get(lpfc_wcqe_c_xb, wcqe)); default: out_err: lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME_IOERR, @@ -930,12 +941,18 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, #endif freqpriv = nCmd->private; freqpriv->nvme_buf = NULL; - nCmd->done(nCmd); + + /* NVME targets need completion held off until the abort exchange + * completes. + */ + if (!lpfc_ncmd->flags & LPFC_SBUF_XBUSY) + nCmd->done(nCmd); spin_lock_irqsave(&phba->hbalock, flags); lpfc_ncmd->nrport = NULL; spin_unlock_irqrestore(&phba->hbalock, flags); + /* Call release with XB=1 to queue the IO into the abort list. */ lpfc_release_nvme_buf(phba, lpfc_ncmd); } @@ -2064,9 +2081,6 @@ lpfc_get_nvme_buf(struct lpfc_hba *phba, struct lpfc_nodelist *ndlp) spin_lock_irqsave(&phba->nvme_buf_list_get_lock, iflag); list_for_each_entry_safe(lpfc_ncmd, lpfc_ncmd_next, &phba->lpfc_nvme_buf_list_get, list) { - if (lpfc_test_rrq_active(phba, ndlp, - lpfc_ncmd->cur_iocbq.sli4_lxritag)) - continue; list_del_init(&lpfc_ncmd->list); found = 1; break; @@ -2079,9 +2093,6 @@ lpfc_get_nvme_buf(struct lpfc_hba *phba, struct lpfc_nodelist *ndlp) spin_unlock(&phba->nvme_buf_list_put_lock); list_for_each_entry_safe(lpfc_ncmd, lpfc_ncmd_next, &phba->lpfc_nvme_buf_list_get, list) { - if (lpfc_test_rrq_active( - phba, ndlp, lpfc_ncmd->cur_iocbq.sli4_lxritag)) - continue; list_del_init(&lpfc_ncmd->list); found = 1; break; @@ -2118,7 +2129,6 @@ lpfc_release_nvme_buf(struct lpfc_hba *phba, struct lpfc_nvme_buf *lpfc_ncmd) spin_lock_irqsave(&phba->sli4_hba.abts_nvme_buf_list_lock, iflag); - lpfc_ncmd->nvmeCmd = NULL; list_add_tail(&lpfc_ncmd->list, &phba->sli4_hba.lpfc_abts_nvme_buf_list); spin_unlock_irqrestore(&phba->sli4_hba.abts_nvme_buf_list_lock, @@ -2486,18 +2496,18 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) * @axri: pointer to the fcp xri abort wcqe structure. * * This routine is invoked by the worker thread to process a SLI4 fast-path - * FCP aborted xri. + * NVME aborted xri. Aborted NVME IO commands are completed to the transport + * here. **/ void lpfc_sli4_nvme_xri_aborted(struct lpfc_hba *phba, struct sli4_wcqe_xri_aborted *axri) { uint16_t xri = bf_get(lpfc_wcqe_xa_xri, axri); - uint16_t rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); struct lpfc_nvme_buf *lpfc_ncmd, *next_lpfc_ncmd; + struct nvmefc_fcp_req *nvme_cmd = NULL; struct lpfc_nodelist *ndlp; unsigned long iflag = 0; - int rrq_empty = 0; if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME)) return; @@ -2513,25 +2523,24 @@ lpfc_sli4_nvme_xri_aborted(struct lpfc_hba *phba, spin_unlock( &phba->sli4_hba.abts_nvme_buf_list_lock); - rrq_empty = list_empty(&phba->active_rrq_list); spin_unlock_irqrestore(&phba->hbalock, iflag); ndlp = lpfc_ncmd->ndlp; - if (ndlp) { - lpfc_set_rrq_active( - phba, ndlp, - lpfc_ncmd->cur_iocbq.sli4_lxritag, - rxid, 1); + if (ndlp) lpfc_sli4_abts_err_handler(phba, ndlp, axri); - } lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6311 XRI Aborted xri x%x tag x%x " - "released\n", - xri, lpfc_ncmd->cur_iocbq.iotag); - + "6311 nvme_cmd %p xri x%x tag x%x " + "abort complete and xri released\n", + lpfc_ncmd->nvmeCmd, xri, + lpfc_ncmd->cur_iocbq.iotag); + + /* Aborted NVME commands are required to not complete + * before the abort exchange command fully completes. + * Once completed, it is available via the put list. + */ + nvme_cmd = lpfc_ncmd->nvmeCmd; + nvme_cmd->done(nvme_cmd); lpfc_release_nvme_buf(phba, lpfc_ncmd); - if (rrq_empty) - lpfc_worker_wake_up(phba); return; } }