From patchwork Wed May 22 00:48:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5513924 for ; Wed, 22 May 2019 00:49:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 993D02882F for ; Wed, 22 May 2019 00:49:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BB5F28A0F; Wed, 22 May 2019 00:49:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D89F2882F for ; Wed, 22 May 2019 00:49:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728130AbfEVAtX (ORCPT ); Tue, 21 May 2019 20:49:23 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:42954 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbfEVAtX (ORCPT ); Tue, 21 May 2019 20:49:23 -0400 Received: by mail-pl1-f194.google.com with SMTP id go2so167851plb.9 for ; Tue, 21 May 2019 17:49:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MIgeez25auR/8LV5piBQP5AWXOBRFasx1/Awg7vqAz4=; b=Yde+4yHVN58KM2l7ZLLM6YAnjePGaB0udRDVsy+H2mWi1ZKVaf27YRWJPS5akTiuuH rznyBWjgKEaa0wvoWXbbxUnSu398L4LgquEwdCNr0KCauKtUI8CwS9ag2Q40fHYYRQam JykhqEcrlkA3P6miZi2NxwIoKIo8YpxSA4Hyqovk+EtpG/1Dv7GBh8DXsvWaqZk9if6t 9mIGPb3zdsDozHE/hhfygoT7sh0Ui1xSXXylIE6YONzK42iliaykr9dYPo4mOXqSY0qZ 0dBaDTo43rykY9rzV4rxuN3+zRSmWeXKHPVRvcnFTtWSDUBN8X60RqvaoMF4t71BvCrW JQmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MIgeez25auR/8LV5piBQP5AWXOBRFasx1/Awg7vqAz4=; b=EFhxcUnvhOZZrrcJl27jZ7p6o756VSoqh61GBOa+tT23KtbPi0bF7R/aRg5GeHP1ka NVxTSRPHTmovUlsELuqsK1NgjQmHPWlCu3F9KYISetq5mhNdNTVtZwCvVIkc7nzhZ2Sa KHv65fAz3BUHwSNwA5mP4Y2cv9B+l06kF6eiyu1DXeSclISd5WxjlvpuIS32PUXAHZEt 2kWGyXuKWrF6oN1UaTyoEnKJH5FgPRdXVAd1rz/ZS5q5enFJELs6uyQ6Xb1iuDd2oCLH KjSI9dpPHJlXHyFPXzCpO09SoghC9mDeWoHX7Pr3cgzwKX3vcACt2YF/bFY+0Sld4OcV gTyw== X-Gm-Message-State: APjAAAVts0MG/wMYVnRHtAYWhDyBBoDdeEptxGVCW/zmih16zgQqgRc9 db771WpLN3R3pLyhmmagLWK3Iac3 X-Google-Smtp-Source: APXvYqzhzvyKHpdEd8EcOU+9+b0fzPr2qvKUCwgdD5PqqPgXuG4QXGnc5Y94WjwYEp26A5RL1fBO1w== X-Received: by 2002:a17:902:8c82:: with SMTP id t2mr79086138plo.256.1558486162882; Tue, 21 May 2019 17:49:22 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:22 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 01/21] lpfc: Fix alloc context on oas lun creations Date: Tue, 21 May 2019 17:48:51 -0700 Message-Id: <20190522004911.573-2-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Softlockups are seen in low memory situations. They are due to doing oas_lun allocation with GFP_KERNEL in atomic contexts. Change the calls to oas_lun to indicate atomic context so that GFP_ATOMIC is used. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_scsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index ba996fbde89b..3873d5b97bc6 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -5741,7 +5741,7 @@ lpfc_enable_oas_lun(struct lpfc_hba *phba, struct lpfc_name *vport_wwpn, /* Create an lun info structure and add to list of luns */ lun_info = lpfc_create_device_data(phba, vport_wwpn, target_wwpn, lun, - pri, false); + pri, true); if (lun_info) { lun_info->oas_enabled = true; lun_info->priority = pri; From patchwork Wed May 22 00:48:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC11A924 for ; Wed, 22 May 2019 00:49:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEF271FE6B for ; Wed, 22 May 2019 00:49:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A2FE228875; Wed, 22 May 2019 00:49:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47B052897D for ; Wed, 22 May 2019 00:49:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728142AbfEVAtZ (ORCPT ); Tue, 21 May 2019 20:49:25 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:44265 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbfEVAtY (ORCPT ); Tue, 21 May 2019 20:49:24 -0400 Received: by mail-pg1-f195.google.com with SMTP id n2so356649pgp.11 for ; Tue, 21 May 2019 17:49:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yqgMSY9XAdmgbcg91K4UDyttzUETrPM5p9LWDQpbA7k=; b=B1mbj72E/5FTs9NJxKrQf1WGK/vfsNJp9ijfHzz8HJbpwLyo4d0LzKKRSm/91BMx1v 5gwtjbZ2qCkKRUtmpJh3CozX05LMdJNoZHcfxmKXNGAH8zGuhBPqeLj7ashJeuq2CwKX 2BKnICNmVz7Lx+WkG9WoahQ+kpYojxKmjSddmlgCHtOCBwQDTRK1Q6XM/303jYkVAcYV GCaTpLxfCYdb0qdLccWQEohIj48z94KwZQdu+hpa+EJE5zqX4GIWZUr8JzN20PdLg5Mt tLgEfSZBza1FJup0JOF7o2ArnWfg5NqEHtuo6QULdwBCo4fCa69GGzlRubtSd1OYD5Bo oFGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yqgMSY9XAdmgbcg91K4UDyttzUETrPM5p9LWDQpbA7k=; b=cl0sEk40uylOZPKFYG1JGw78zzG6H0bUk42Mdg0e14VYJp03Zi6JCitqFyV6mz+jVX YBUt2Cr0CQu4lEMQuDquHkY+bE+6NKsXCynz3slaGna4DDwCh1zgwTzCbZiaKtgDDON7 jCjx6JvWCvRVe8Dfe53MZy1n3+FQp6pGQqeXSisXCmuIhn0dF7ASP41PdP8Daj82o0kJ PIo+sKQDK99pONdDWtAjtyFnrbeBvN1oQN2Nps8TN4n1TRJL1xdwkia4WQ57efl50lG6 YAxAOvTsfZbfyOlBqS5ZHuEMYXQLsUCKOxx8kaveKCAGh4FAqluVGtO+FDboN6q3FTz9 AYvA== X-Gm-Message-State: APjAAAUFveMehWFRMO8U+BqecCOIknwgEB00h1UTSH9lL0ZqCwjTZtJw iRKlTsx22QFVutQBBrLF3YcpUsML X-Google-Smtp-Source: APXvYqxhX4vPmoWno/4v0sols97gqdE5LwvVyZQ+mSBwHQ3QMaj/pXfoTPqwNapknQglmqcEvAL6sw== X-Received: by 2002:a63:ff0f:: with SMTP id k15mr85096157pgi.407.1558486163819; Tue, 21 May 2019 17:49:23 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:23 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 02/21] lpfc: Fix nvmet target abort cmd matching Date: Tue, 21 May 2019 17:48:52 -0700 Message-Id: <20190522004911.573-3-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After receiving an unsolicited ABTS (meaning rxid is 0xFFFF), the driver used the oxid from the initiator to match against a local xri which may have been allocated for the io. The xri would be the rxid - it's an invalid check resulting in the command not being matched or erroneously matched. Change the lookup to use the oxid and the SID to match against received IO's original values. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvmet.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index d74bfd264495..c9011579aa0f 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -1497,6 +1497,7 @@ void lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, struct sli4_wcqe_xri_aborted *axri) { +#if (IS_ENABLED(CONFIG_NVME_TARGET_FC)) uint16_t xri = bf_get(lpfc_wcqe_xa_xri, axri); uint16_t rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); struct lpfc_nvmet_rcv_ctx *ctxp, *next_ctxp; @@ -1562,6 +1563,7 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, } spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); spin_unlock_irqrestore(&phba->hbalock, iflag); +#endif } int @@ -1572,19 +1574,23 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, struct lpfc_hba *phba = vport->phba; struct lpfc_nvmet_rcv_ctx *ctxp, *next_ctxp; struct nvmefc_tgt_fcp_req *rsp; + uint32_t sid; uint16_t xri; unsigned long iflag = 0; xri = be16_to_cpu(fc_hdr->fh_ox_id); + sid = sli4_sid_from_fc_hdr(fc_hdr); spin_lock_irqsave(&phba->hbalock, iflag); spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); list_for_each_entry_safe(ctxp, next_ctxp, &phba->sli4_hba.lpfc_abts_nvmet_ctx_list, list) { - if (ctxp->ctxbuf->sglq->sli4_xritag != xri) + if (ctxp->oxid != xri || ctxp->sid != sid) continue; + xri = ctxp->ctxbuf->sglq->sli4_xritag; + spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); spin_unlock_irqrestore(&phba->hbalock, iflag); @@ -1613,7 +1619,7 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, xri, raw_smp_processor_id(), 1); lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6320 NVMET Rcv ABTS:rjt xri x%x\n", xri); + "6320 NVMET Rcv ABTS:rjt xid x%x\n", xri); /* Respond with BA_RJT accordingly */ lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 0); From patchwork Wed May 22 00:48:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A940924 for ; Wed, 22 May 2019 00:49:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5BFC11FE6B for ; Wed, 22 May 2019 00:49:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4486628A0F; Wed, 22 May 2019 00:49:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E59EC1FE6B for ; Wed, 22 May 2019 00:49:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728145AbfEVAtZ (ORCPT ); Tue, 21 May 2019 20:49:25 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:44265 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbfEVAtZ (ORCPT ); Tue, 21 May 2019 20:49:25 -0400 Received: by mail-pg1-f194.google.com with SMTP id n2so356655pgp.11 for ; Tue, 21 May 2019 17:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HwHpllSGXEnnagT/c/OmgyMimlVroFXqaZCeTvxYnG4=; b=I6KLCha2KxD0UmhdBV/50m5Br4UJv0IwlFnPSL2uZHzA6vc+I6SuYTVNlmpjfRUyXr nOtB0WMrQhlaYFkDBRo17dtlwxLB7gmYCrFgYLyxWE/P+X23cf7nuuluLqm6v9+Lh/Yw k55j2/5tv6ktj80jxfzzOGte98GqPDU0Yc60E58UqJB2EVMwL4RP/QteHu0bwwHhycq7 E6ECOCAXzAM+nlNKsQAn7Vt6pF2glMKAv0UvMA6PlwbnCgNBG4QDVn1/Z4rKgzhRvKdT PogIviqcC0bTljhw48ykyYrTgZ0hhbAa/FIn9VPJc43lrNMoHvWzpzwaV+lEDvmdKwXe 1yuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HwHpllSGXEnnagT/c/OmgyMimlVroFXqaZCeTvxYnG4=; b=iPUqEQqKR7K3cVu8h6oH/2TRPHikF0NYyR612I/MUyYIDQARGw89YE2tMe2AtrTJox VS/NysA116E/KcaxnxWYNa1Do30PsvZJk5FM2qog0IH4jw3Aus54b7ZaMdJRpOTqMyv9 3ER4f7SQlcwAp5lA7ZC1xeXXkQhNSK+sXYr34c/h0O8L0GdRzHltBUqdPn/oLqR7dgXf VAkubu4UsEXchvjvev4PocB9CP8zE6UY7IkdUTpEZWl9W5krpZwGawMiXaHLlG0Q/fOO /EhLOoBdRblqsDNlTVjh22wz9jstgD/OXwhZlKtVNdg3HI68hZGwO4H0szDZd6briptk Xq8w== X-Gm-Message-State: APjAAAWgBGedYgZdslwcQaPBsLbbOyMEbAxvD7hJTa8uyxlZpSQBoU1W lHlB5K8C6ik63vYGPK4+zvF7hsY9 X-Google-Smtp-Source: APXvYqxe7xp+CKDc6J9yl76CsV0B1qfvTir3zc0+2QQsBdQnVtbEnT32bl4a3RHmpetUeIlWdPmKRg== X-Received: by 2002:a65:44c8:: with SMTP id g8mr87373150pgs.443.1558486164593; Tue, 21 May 2019 17:49:24 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:24 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 03/21] lpfc: Correct nvmet buffer free race condition Date: Tue, 21 May 2019 17:48:53 -0700 Message-Id: <20190522004911.573-4-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A race condition resulted in receive buffers being placed in the free list twice. Change the locking and handling to check whether the "other" path will be freeing the entry in a later thread and skip it if it is. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvmet.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index c9011579aa0f..3a11861b7ad6 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -343,16 +343,23 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) } if (ctxp->rqb_buffer) { - nvmebuf = ctxp->rqb_buffer; spin_lock_irqsave(&ctxp->ctxlock, iflag); - ctxp->rqb_buffer = NULL; - if (ctxp->flag & LPFC_NVMET_CTX_REUSE_WQ) { - ctxp->flag &= ~LPFC_NVMET_CTX_REUSE_WQ; - spin_unlock_irqrestore(&ctxp->ctxlock, iflag); - nvmebuf->hrq->rqbp->rqb_free_buffer(phba, nvmebuf); + nvmebuf = ctxp->rqb_buffer; + /* check if freed in another path whilst acquiring lock */ + if (nvmebuf) { + ctxp->rqb_buffer = NULL; + if (ctxp->flag & LPFC_NVMET_CTX_REUSE_WQ) { + ctxp->flag &= ~LPFC_NVMET_CTX_REUSE_WQ; + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + nvmebuf->hrq->rqbp->rqb_free_buffer(phba, + nvmebuf); + } else { + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + /* repost */ + lpfc_rq_buf_free(phba, &nvmebuf->hbuf); + } } else { spin_unlock_irqrestore(&ctxp->ctxlock, iflag); - lpfc_rq_buf_free(phba, &nvmebuf->hbuf); /* repost */ } } ctxp->state = LPFC_NVMET_STE_FREE; From patchwork Wed May 22 00:48:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53E15924 for ; Wed, 22 May 2019 00:49:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43B6F2882F for ; Wed, 22 May 2019 00:49:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 356AD28895; Wed, 22 May 2019 00:49:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D93662882F for ; Wed, 22 May 2019 00:49:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728150AbfEVAt0 (ORCPT ); Tue, 21 May 2019 20:49:26 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:45315 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbfEVAt0 (ORCPT ); Tue, 21 May 2019 20:49:26 -0400 Received: by mail-pg1-f194.google.com with SMTP id i21so355064pgi.12 for ; Tue, 21 May 2019 17:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wB7NADiRW3dEmTOyxx6TXaa0Cw9K2E6Iq3U4WtpT5gg=; b=Z2KQq+0oX83Rg4SqNAhZDl0PVs32Bjf7k4V1FGaSIHqj80Fu/UfftKUiFAvFSbpnwx GTLl6HqjwWn0GJc1zYKZ28NiTJMoH9uzi81HIRHhQ2QPnJRZDDTvj7eEq7tX8vUgOcMJ Vu15mDqF7XHpfCnQuazXt2gOb1mRHqqedgSBDhQoUxHsAoK8i4EUlGEai7JWMk/Yzd9E 6YvbOOGnQcEKoGAX7YPpI8IXgdJHC6KHEJfPGHh1XZ+I2Nr6Jqv5BC7Km2UrbU4nAHWK LptBi70mCy8iwm+d093cL0gHzeq3W77fSZ8aTtTRgMeSUFn5BzpuQ5v8F6QR2XlEYu9i ar8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wB7NADiRW3dEmTOyxx6TXaa0Cw9K2E6Iq3U4WtpT5gg=; b=iLMEjhLqSY9reQuN7bmPkmirxdinS695cRLgxGUl3kxQpX1dvu9M/g/FrlIN4mtpEp jsugP+N53bGns1HkG8RDaCRzJ42Lfz3xZlmn4Z/PeVfWO5ZO1j3OQb4lSmqtH6dSBy4C CjxDHGhJID+3oWwBRdkvTA+SvqexHa5M6EDhGTB1fxogoJxzJ89TM5ZX9ifF9KCGMyEB ULCaMk7J/1xECgvhOvFXdDGTgh6ukLyhfanu46ZTAQnnZFeDGlIRXqQJD9ZnwgIA8l6x +RZrXsRMCeHnrX5PMQj57bE1jXZv4Hx05+Ks8ysGruHdMAzAQMFy9EUZTIQdODJSrSwz kysA== X-Gm-Message-State: APjAAAXukkh48aFYmr3uLDX4YE6fVniKM6ybDTllQaws++j3ePPVMBYU kUxLGkpbnX94CCAmDce88rMx9Pkl X-Google-Smtp-Source: APXvYqw/VXWQmVlkyIrAAiFwjZo4YAI7nYrjc4E3/1LkB0sPzZ4lhAV8+8vFoFsse1S2yUVTvmx+qQ== X-Received: by 2002:a63:f146:: with SMTP id o6mr86503001pgk.179.1558486165417; Tue, 21 May 2019 17:49:25 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:25 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 04/21] lpfc: Revise message when stuck due to unresponsive adapter Date: Tue, 21 May 2019 17:48:54 -0700 Message-Id: <20190522004911.573-5-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Revise a stalled adapter message to also include the number of jobs that are stalling the thread. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvme.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 9d99cb915390..39514d4c279d 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -2143,7 +2143,9 @@ lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, struct completion *lport_unreg_cmp) { u32 wait_tmo; - int ret; + int ret, i, pending = 0; + struct lpfc_sli_ring *pring; + struct lpfc_hba *phba = vport->phba; /* Host transport has to clean up and confirm requiring an indefinite * wait. Print a message if a 10 second wait expires and renew the @@ -2153,10 +2155,18 @@ lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, while (true) { ret = wait_for_completion_timeout(lport_unreg_cmp, wait_tmo); if (unlikely(!ret)) { + pending = 0; + for (i = 0; i < phba->cfg_hdw_queue; i++) { + pring = phba->sli4_hba.hdwq[i].nvme_wq->pring; + if (!pring) + continue; + if (pring->txcmplq_cnt) + pending += pring->txcmplq_cnt; + } lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME_IOERR, "6176 Lport %p Localport %p wait " - "timed out. Renewing.\n", - lport, vport->localport); + "timed out. Pending %d. Renewing.\n", + lport, vport->localport, pending); continue; } break; From patchwork Wed May 22 00:48:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24AE714B6 for ; Wed, 22 May 2019 00:49:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 182C32882F for ; Wed, 22 May 2019 00:49:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CD4128A0F; Wed, 22 May 2019 00:49:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 481742897D for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728162AbfEVAt3 (ORCPT ); Tue, 21 May 2019 20:49:29 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:34942 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbfEVAt2 (ORCPT ); Tue, 21 May 2019 20:49:28 -0400 Received: by mail-pl1-f194.google.com with SMTP id p1so182991plo.2 for ; Tue, 21 May 2019 17:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nU9bGb+ZDTI7rIQtHz9KYet0VysucJK8jffF6oR+4jk=; b=KeTo/TEVB5GDLqq95a80Z7PyoHCr5Cj3NuH2qXPtXxGFEBdDJErul+q685MXltwmiW Bc2uey2xIxtGOpJFyflm/URMu2VmpDXa70w3WeSIuDsourQpFkycj6/0G+MsJF6Kx0eG F+YTmyUma0VTcnK2niNtGG9T4DZ5LfTlGAsuPHiBy3rsJgBq9RnKIvS3NbPjzDAwXHj+ KEQ8Sn9PRcpEih54Qcyou0zDvlTSTt7FkqrkNxxvMZkIxrvcUIFv0cmXn7y4GOm3cpdP 2zkGtJrRnIEs385tpSylyvBxV1bicXIZBKHyZhCRgrUEHpDc7WnIOZWov1sIhdAosR9Y cPSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nU9bGb+ZDTI7rIQtHz9KYet0VysucJK8jffF6oR+4jk=; b=PU8GT6hvyAjFkpo8rCtou+cRWw6K7yWpoLu5HsZzDoqBzYI/eYYeX8dzAQpc2Zwx36 VdERmkiQfBTgBxwkRTiUq+SVt3By4lfXQZN6cuwhqdUboFY07SIJVR9cogiiHNIgDjts 9VrVFJms6v41oKIMXKKO6A2vhFoJ2UBZNT/emiAMGGR4uXnsDmHskMBQHYcpud0YIITE VuLSW4V9vhhgN35MC27C24AxS64XpLdtIvSnMsXnsePPKMfAI/ZHKJBhOIxGgkQ0agop 18y5t3IrEA1PvrVFKlvmeenww2+Iwt2sgzLjpa5IqzFHxFWEX5t4by9J5M85V3G2DeBH BkDw== X-Gm-Message-State: APjAAAWSS9PE3xBUR39R/eIGfIeegeNouxZTXF9ihp6wZANZEQ4nVX/j Wd6mgutsU7MLPKAxGfk+3/odlXwA X-Google-Smtp-Source: APXvYqxZ6PnotmRAwAoczk7IucUS/O3MWdv9CSt+VYo9xYK6GeQ0VScXlTSE8l0hIdkt0D6Nuf8xCA== X-Received: by 2002:a17:902:e683:: with SMTP id cn3mr64003746plb.86.1558486166237; Tue, 21 May 2019 17:49:26 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:25 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 05/21] lpfc: Separate CQ processing for nvmet_fc upcalls Date: Tue, 21 May 2019 17:48:55 -0700 Message-Id: <20190522004911.573-6-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently the driver is notified of new command frame receipt by CQE's. As part of the CQE processing, the driver upcalls the nvmet_fc transport to deliver the command. nvmet_fc, as part of receiving the command builds out a context for it, where one of the first steps is to allocate memory for the io. When running with tests that do large ios (1MB), it was found on some systems, the total number of outstanding I/O's, at 1MB per, completely consumed the systems memory. Thus additional ios were getting blocked in the memory allocator. Given that this blocked the lpfc thread processing cqes, there were lots of other commands that were received and which are then held up, and given CQEs are serially processed, the aggregate delays for an IO waiting behind the others became cummulative - enough so that the initiator hit timeouts for the ios. The basic fix is to avoid the direct upcall and instead schedule a work item for each io as it is received. This allows the cq processing to complete very quickly, and each io can then run or block on it's own. However, this general solution hurts latency when there are few ios. As such, implemented the fix such that the driver watches how many cqes it has processed sequentially in one run. As long as the count is below a threshold, the direct nvmet_fc upcall will be made. Only when the count is exceeded will it revert to work scheduling. Given that debug of this showed a surprisingly long delay in cq processing, the io timer stats were updated to better reflect the processing of the different points. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_crtn.h | 3 +- drivers/scsi/lpfc/lpfc_nvmet.c | 67 +++++++++++++++++++++++++++++------------- drivers/scsi/lpfc/lpfc_sli.c | 22 ++++++++------ drivers/scsi/lpfc/lpfc_sli4.h | 2 ++ 4 files changed, 64 insertions(+), 30 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index e0b14d791b8c..97f0ef236661 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -568,7 +568,8 @@ void lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba); void lpfc_nvmet_unsol_ls_event(struct lpfc_hba *phba, struct lpfc_sli_ring *pring, struct lpfc_iocbq *piocb); void lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba, uint32_t idx, - struct rqb_dmabuf *nvmebuf, uint64_t isr_ts); + struct rqb_dmabuf *nvmebuf, uint64_t isr_ts, + uint8_t cqflag); void lpfc_nvme_mod_param_dep(struct lpfc_hba *phba); void lpfc_nvme_abort_fcreq_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 3a11861b7ad6..95386f90a874 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -395,8 +395,9 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) spin_lock_init(&ctxp->ctxlock); #ifdef CONFIG_SCSI_LPFC_DEBUG_FS - if (ctxp->ts_cmd_nvme) { - ctxp->ts_cmd_nvme = ktime_get_ns(); + /* NOTE: isr time stamp is stale when context is re-assigned*/ + if (ctxp->ts_isr_cmd) { + ctxp->ts_cmd_nvme = 0; ctxp->ts_nvme_data = 0; ctxp->ts_data_wqput = 0; ctxp->ts_isr_data = 0; @@ -1877,6 +1878,10 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf) payload = (uint32_t *)(nvmebuf->dbuf.virt); tgtp = (struct lpfc_nvmet_tgtport *)phba->targetport->private; +#ifdef CONFIG_SCSI_LPFC_DEBUG_FS + if (ctxp->ts_isr_cmd) + ctxp->ts_cmd_nvme = ktime_get_ns(); +#endif /* * The calling sequence should be: * nvmet_fc_rcv_fcp_req->lpfc_nvmet_xmt_fcp_op/cmp- req->done @@ -2015,6 +2020,8 @@ lpfc_nvmet_replenish_context(struct lpfc_hba *phba, * @phba: pointer to lpfc hba data structure. * @idx: relative index of MRQ vector * @nvmebuf: pointer to lpfc nvme command HBQ data structure. + * @isr_timestamp: in jiffies. + * @cqflag: cq processing information regarding workload. * * This routine is used for processing the WQE associated with a unsolicited * event. It first determines whether there is an existing ndlp that matches @@ -2027,7 +2034,8 @@ static void lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, uint32_t idx, struct rqb_dmabuf *nvmebuf, - uint64_t isr_timestamp) + uint64_t isr_timestamp, + uint8_t cqflag) { struct lpfc_nvmet_rcv_ctx *ctxp; struct lpfc_nvmet_tgtport *tgtp; @@ -2136,24 +2144,41 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, spin_lock_init(&ctxp->ctxlock); #ifdef CONFIG_SCSI_LPFC_DEBUG_FS - if (isr_timestamp) { + if (isr_timestamp) ctxp->ts_isr_cmd = isr_timestamp; - ctxp->ts_cmd_nvme = ktime_get_ns(); - ctxp->ts_nvme_data = 0; - ctxp->ts_data_wqput = 0; - ctxp->ts_isr_data = 0; - ctxp->ts_data_nvme = 0; - ctxp->ts_nvme_status = 0; - ctxp->ts_status_wqput = 0; - ctxp->ts_isr_status = 0; - ctxp->ts_status_nvme = 0; - } else { - ctxp->ts_cmd_nvme = 0; - } + ctxp->ts_cmd_nvme = 0; + ctxp->ts_nvme_data = 0; + ctxp->ts_data_wqput = 0; + ctxp->ts_isr_data = 0; + ctxp->ts_data_nvme = 0; + ctxp->ts_nvme_status = 0; + ctxp->ts_status_wqput = 0; + ctxp->ts_isr_status = 0; + ctxp->ts_status_nvme = 0; #endif atomic_inc(&tgtp->rcv_fcp_cmd_in); - lpfc_nvmet_process_rcv_fcp_req(ctx_buf); + /* check for cq processing load */ + if (!cqflag) { + lpfc_nvmet_process_rcv_fcp_req(ctx_buf); + return; + } + + if (!queue_work(phba->wq, &ctx_buf->defer_work)) { + atomic_inc(&tgtp->rcv_fcp_cmd_drop); + lpfc_printf_log(phba, KERN_ERR, LOG_NVME, + "6325 Unable to queue work for oxid x%x. " + "FCP Drop IO [x%x x%x x%x]\n", + ctxp->oxid, + atomic_read(&tgtp->rcv_fcp_cmd_in), + atomic_read(&tgtp->rcv_fcp_cmd_out), + atomic_read(&tgtp->xmt_fcp_release)); + + spin_lock_irqsave(&ctxp->ctxlock, iflag); + lpfc_nvmet_defer_release(phba, ctxp); + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + lpfc_nvmet_unsol_fcp_issue_abort(phba, ctxp, sid, oxid); + } } /** @@ -2190,6 +2215,8 @@ lpfc_nvmet_unsol_ls_event(struct lpfc_hba *phba, struct lpfc_sli_ring *pring, * @phba: pointer to lpfc hba data structure. * @idx: relative index of MRQ vector * @nvmebuf: pointer to received nvme data structure. + * @isr_timestamp: in jiffies. + * @cqflag: cq processing information regarding workload. * * This routine is used to process an unsolicited event received from a SLI * (Service Level Interface) ring. The actual processing of the data buffer @@ -2201,14 +2228,14 @@ void lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba, uint32_t idx, struct rqb_dmabuf *nvmebuf, - uint64_t isr_timestamp) + uint64_t isr_timestamp, + uint8_t cqflag) { if (phba->nvmet_support == 0) { lpfc_rq_buf_free(phba, &nvmebuf->hbuf); return; } - lpfc_nvmet_unsol_fcp_buffer(phba, idx, nvmebuf, - isr_timestamp); + lpfc_nvmet_unsol_fcp_buffer(phba, idx, nvmebuf, isr_timestamp, cqflag); } /** diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 2acda188b0dc..946f3024d4db 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -13582,14 +13582,9 @@ __lpfc_sli4_process_cq(struct lpfc_hba *phba, struct lpfc_queue *cq, goto rearm_and_exit; /* Process all the entries to the CQ */ + cq->q_flag = 0; cqe = lpfc_sli4_cq_get(cq); while (cqe) { -#if defined(CONFIG_SCSI_LPFC_DEBUG_FS) && defined(BUILD_NVME) - if (phba->ktime_on) - cq->isr_timestamp = ktime_get_ns(); - else - cq->isr_timestamp = 0; -#endif workposted |= handler(phba, cq, cqe); __lpfc_sli4_consume_cqe(phba, cq, cqe); @@ -13603,6 +13598,9 @@ __lpfc_sli4_process_cq(struct lpfc_hba *phba, struct lpfc_queue *cq, consumed = 0; } + if (count == LPFC_NVMET_CQ_NOTIFY) + cq->q_flag |= HBA_NVMET_CQ_NOTIFY; + cqe = lpfc_sli4_cq_get(cq); } if (count >= phba->cfg_cq_poll_threshold) { @@ -13918,10 +13916,10 @@ lpfc_sli4_nvmet_handle_rcqe(struct lpfc_hba *phba, struct lpfc_queue *cq, goto drop; if (fc_hdr->fh_type == FC_TYPE_FCP) { - dma_buf->bytes_recv = bf_get(lpfc_rcqe_length, rcqe); + dma_buf->bytes_recv = bf_get(lpfc_rcqe_length, rcqe); lpfc_nvmet_unsol_fcp_event( - phba, idx, dma_buf, - cq->isr_timestamp); + phba, idx, dma_buf, cq->isr_timestamp, + cq->q_flag & HBA_NVMET_CQ_NOTIFY); return false; } drop: @@ -14087,6 +14085,12 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq, } work_cq: +#if defined(CONFIG_SCSI_LPFC_DEBUG_FS) + if (phba->ktime_on) + cq->isr_timestamp = ktime_get_ns(); + else + cq->isr_timestamp = 0; +#endif if (!queue_work_on(cq->chann, phba->wq, &cq->irqwork)) lpfc_printf_log(phba, KERN_ERR, LOG_SLI, "0363 Cannot schedule soft IRQ " diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 8e4fd1a98023..fbc1f1880d53 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -197,6 +197,8 @@ struct lpfc_queue { #define LPFC_DB_LIST_FORMAT 0x02 uint8_t q_flag; #define HBA_NVMET_WQFULL 0x1 /* We hit WQ Full condition for NVMET */ +#define HBA_NVMET_CQ_NOTIFY 0x1 /* LPFC_NVMET_CQ_NOTIFY CQEs this EQE */ +#define LPFC_NVMET_CQ_NOTIFY 4 void __iomem *db_regaddr; uint16_t dpp_enable; uint16_t dpp_id; From patchwork Wed May 22 00:48:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33B5914B6 for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 263F828895 for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1A4172897D; Wed, 22 May 2019 00:49:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC5512886F for ; Wed, 22 May 2019 00:49:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728022AbfEVAt3 (ORCPT ); Tue, 21 May 2019 20:49:29 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:37663 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728152AbfEVAt2 (ORCPT ); Tue, 21 May 2019 20:49:28 -0400 Received: by mail-pl1-f193.google.com with SMTP id p15so179136pll.4 for ; Tue, 21 May 2019 17:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7P7rAcRLVwcT//Zl7ym7fSP8NHGnFsvFpulcejwWEU8=; b=ok/lhFCbTbRxx7gA6ZNESQM/6wHUhx8OaXqWgSpuKs8v5k3jqEJkY6quA/rAB+LIfp yxxmtNFvMUADblP7s3atnc1uefubmi6oBYlyBTd/7urd3vznWkj86MnPqDtLbEMZJr9T 4r5M3j7ak9ynDsVeRebzdB0wFzWHfUgzmk3Xsabf1xyFYCIeYd/nJJzI88BlsZIcT/e9 j+jwMNBL2V/eyJuYN4PgJvlOjsaKk0SAp6n8PURMOO246EnpKCYqgv/quHin2DoloQ47 S+PnilRrp+FtP9Dlkc/o/LacZUEg42UwVqufOP+aIrdH6Ka4Z95w27kglmv94qaOtI7k vSKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7P7rAcRLVwcT//Zl7ym7fSP8NHGnFsvFpulcejwWEU8=; b=qSD58+jGCWtOliV1WaMOMgGPIEkcB1CnGpzXLpNHZQcf5FRF3rYvk+RoBjKg2cQnpI QwWjTZ/auviugUJCHN8EsIk5AU8YrW1ThNa6vEIeYHDGKKiHC6wOFWOMeivGXj3oxdqJ Ejou/E/C9qEEHSPltxAV+y/K1ldglOPUdTsctVRsnh6bsn3r1vfnKW1sOTgBu0UV0Cbe 2NvyCplXBgv0eOFqRjdQG8CHaLi+qSBNgYYtWI7E73hMzBjD3/ZLJKpQP7zbMS5T3nvs r5BH1Pczng+QCpKzobqmSm1S3WsDEVSb7HXEcRZ4rZle+18TEeFqkyWFpRLtKBTVGsVl NNXg== X-Gm-Message-State: APjAAAV0j9cw1kGdGy650UXbnkw7knZj/K/E3WrKDOIs6t/cWVJk/Mya BD/eU8QABTixgoug72N5cQJZFeRh X-Google-Smtp-Source: APXvYqz4tAsyYLn8bQldTc16FX4acgRRkdJlZ3Y52GifiOTwnHm9aN5jKNTe0/IYg+Y2+lJ1LuOSfQ== X-Received: by 2002:a17:902:7406:: with SMTP id g6mr85170351pll.328.1558486166984; Tue, 21 May 2019 17:49:26 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:26 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 06/21] lpfc: Fix nvmet handling of received ABTS for unmapped frames Date: Tue, 21 May 2019 17:48:56 -0700 Message-Id: <20190522004911.573-7-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver currently is relying on firmware to match ABTS's to existing exchanges. This works fine as long as an exchange has been assigned to the io and work posted to it. However, for unmapped frames (rxid=0xFFFF), the driver has yet to assign an xri. The driver was blindly saying it couldn't match the ABTS and sending the BA_xxx. However, the command frame may have been in queues waiting on xri's before posting to the nvmet_fc layer. When xri's became available, the command frame would still be pushed to the transport and that io would execute, even though the io had been killed by ABTS. The initiator, seeing the io ABTS'd, would reuse the exchange for a different io which would be received on the target and pushed up. If the "zombie" io then came back down and started transmitting, the initiator would match the oxid and accept erroneous data. Bad things happened. Add tracking of active exchanges in the target to allow matching of a received ABTS against active or pending IO requests. If the ABTS is matched to a pending or active IO, the drive initiates cleanup and conditionally notifies the transport. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 2 + drivers/scsi/lpfc/lpfc_nvmet.c | 234 ++++++++++++++++++++++++++++++++++++----- drivers/scsi/lpfc/lpfc_nvmet.h | 1 + drivers/scsi/lpfc/lpfc_sli4.h | 2 + 4 files changed, 214 insertions(+), 25 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index eaaef682de25..70afa585b027 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -6551,6 +6551,8 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) spin_lock_init(&phba->sli4_hba.abts_nvmet_buf_list_lock); INIT_LIST_HEAD(&phba->sli4_hba.lpfc_abts_nvmet_ctx_list); INIT_LIST_HEAD(&phba->sli4_hba.lpfc_nvmet_io_wait_list); + spin_lock_init(&phba->sli4_hba.t_active_list_lock); + INIT_LIST_HEAD(&phba->sli4_hba.t_active_ctx_list); } /* This abort list used by worker thread */ diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 95386f90a874..a943b2a20001 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -220,19 +220,66 @@ lpfc_nvmet_cmd_template(void) /* Word 12, 13, 14, 15 - is zero */ } +struct lpfc_nvmet_rcv_ctx * +lpfc_nvmet_get_ctx_for_xri(struct lpfc_hba *phba, u16 xri) +{ + struct lpfc_nvmet_rcv_ctx *ctxp; + unsigned long iflag; + bool found = false; + + spin_lock_irqsave(&phba->sli4_hba.t_active_list_lock, iflag); + list_for_each_entry(ctxp, &phba->sli4_hba.t_active_ctx_list, list) { + if (ctxp->ctxbuf->sglq->sli4_xritag != xri) + continue; + + found = true; + break; + } + spin_unlock_irqrestore(&phba->sli4_hba.t_active_list_lock, iflag); + if (found) + return ctxp; + + return NULL; +} + +struct lpfc_nvmet_rcv_ctx * +lpfc_nvmet_get_ctx_for_oxid(struct lpfc_hba *phba, u16 oxid, u32 sid) +{ + struct lpfc_nvmet_rcv_ctx *ctxp; + unsigned long iflag; + bool found = false; + + spin_lock_irqsave(&phba->sli4_hba.t_active_list_lock, iflag); + list_for_each_entry(ctxp, &phba->sli4_hba.t_active_ctx_list, list) { + if (ctxp->oxid != oxid || ctxp->sid != sid) + continue; + + found = true; + break; + } + spin_unlock_irqrestore(&phba->sli4_hba.t_active_list_lock, iflag); + if (found) + return ctxp; + + return NULL; +} + static void lpfc_nvmet_defer_release(struct lpfc_hba *phba, struct lpfc_nvmet_rcv_ctx *ctxp) { lockdep_assert_held(&ctxp->ctxlock); lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6313 NVMET Defer ctx release xri x%x flg x%x\n", + "6313 NVMET Defer ctx release oxid x%x flg x%x\n", ctxp->oxid, ctxp->flag); if (ctxp->flag & LPFC_NVMET_CTX_RLS) return; ctxp->flag |= LPFC_NVMET_CTX_RLS; + spin_lock(&phba->sli4_hba.t_active_list_lock); + list_del(&ctxp->list); + spin_unlock(&phba->sli4_hba.t_active_list_lock); spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); list_add_tail(&ctxp->list, &phba->sli4_hba.lpfc_abts_nvmet_ctx_list); spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); @@ -410,9 +457,7 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) #endif atomic_inc(&tgtp->rcv_fcp_cmd_in); - /* flag new work queued, replacement buffer has already - * been reposted - */ + /* Indicate that a replacement buffer has been posted */ spin_lock_irqsave(&ctxp->ctxlock, iflag); ctxp->flag |= LPFC_NVMET_CTX_REUSE_WQ; spin_unlock_irqrestore(&ctxp->ctxlock, iflag); @@ -441,6 +486,9 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf) * Use the CPU context list, from the MRQ the IO was received on * (ctxp->idx), to save context structure. */ + spin_lock_irqsave(&phba->sli4_hba.t_active_list_lock, iflag); + list_del_init(&ctxp->list); + spin_unlock_irqrestore(&phba->sli4_hba.t_active_list_lock, iflag); cpu = raw_smp_processor_id(); infop = lpfc_get_ctx_list(phba, cpu, ctxp->idx); spin_lock_irqsave(&infop->nvmet_ctx_list_lock, iflag); @@ -708,8 +756,10 @@ lpfc_nvmet_xmt_fcp_op_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe, } lpfc_printf_log(phba, KERN_INFO, logerr, - "6315 IO Error Cmpl xri x%x: %x/%x XBUSY:x%x\n", - ctxp->oxid, status, result, ctxp->flag); + "6315 IO Error Cmpl oxid: x%x xri: x%x %x/%x " + "XBUSY:x%x\n", + ctxp->oxid, ctxp->ctxbuf->sglq->sli4_xritag, + status, result, ctxp->flag); } else { rsp->fcp_error = NVME_SC_SUCCESS; @@ -930,7 +980,7 @@ lpfc_nvmet_xmt_fcp_op(struct nvmet_fc_target_port *tgtport, (ctxp->state == LPFC_NVMET_STE_ABORT)) { atomic_inc(&lpfc_nvmep->xmt_fcp_drop); lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, - "6102 IO xri x%x aborted\n", + "6102 IO oxid x%x aborted\n", ctxp->oxid); rc = -ENXIO; goto aerr; @@ -1030,7 +1080,7 @@ lpfc_nvmet_xmt_fcp_abort(struct nvmet_fc_target_port *tgtport, ctxp->hdwq = &phba->sli4_hba.hdwq[0]; lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6103 NVMET Abort op: oxri x%x flg x%x ste %d\n", + "6103 NVMET Abort op: oxid x%x flg x%x ste %d\n", ctxp->oxid, ctxp->flag, ctxp->state); lpfc_nvmeio_data(phba, "NVMET FCP ABRT: xri x%x flg x%x ste x%x\n", @@ -1043,7 +1093,7 @@ lpfc_nvmet_xmt_fcp_abort(struct nvmet_fc_target_port *tgtport, /* Since iaab/iaar are NOT set, we need to check * if the firmware is in process of aborting IO */ - if (ctxp->flag & LPFC_NVMET_XBUSY) { + if (ctxp->flag & (LPFC_NVMET_XBUSY | LPFC_NVMET_ABORT_OP)) { spin_unlock_irqrestore(&ctxp->ctxlock, flags); return; } @@ -1106,6 +1156,7 @@ lpfc_nvmet_xmt_fcp_release(struct nvmet_fc_target_port *tgtport, ctxp->state, aborting); atomic_inc(&lpfc_nvmep->xmt_fcp_release); + ctxp->flag &= ~LPFC_NVMET_TNOTIFY; if (aborting) return; @@ -1130,7 +1181,7 @@ lpfc_nvmet_defer_rcv(struct nvmet_fc_target_port *tgtport, if (!nvmebuf) { lpfc_printf_log(phba, KERN_INFO, LOG_NVME_IOERR, - "6425 Defer rcv: no buffer xri x%x: " + "6425 Defer rcv: no buffer oxid x%x: " "flg %x ste %x\n", ctxp->oxid, ctxp->flag, ctxp->state); return; @@ -1510,6 +1561,7 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, uint16_t rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); struct lpfc_nvmet_rcv_ctx *ctxp, *next_ctxp; struct lpfc_nvmet_tgtport *tgtp; + struct nvmefc_tgt_fcp_req *req = NULL; struct lpfc_nodelist *ndlp; unsigned long iflag = 0; int rrq_empty = 0; @@ -1540,7 +1592,7 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, */ if (ctxp->flag & LPFC_NVMET_CTX_RLS && !(ctxp->flag & LPFC_NVMET_ABORT_OP)) { - list_del(&ctxp->list); + list_del_init(&ctxp->list); released = true; } ctxp->flag &= ~LPFC_NVMET_XBUSY; @@ -1560,7 +1612,7 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, } lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6318 XB aborted oxid %x flg x%x (%x)\n", + "6318 XB aborted oxid x%x flg x%x (%x)\n", ctxp->oxid, ctxp->flag, released); if (released) lpfc_nvmet_ctxbuf_post(phba, ctxp->ctxbuf); @@ -1571,6 +1623,32 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, } spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); spin_unlock_irqrestore(&phba->hbalock, iflag); + + ctxp = lpfc_nvmet_get_ctx_for_xri(phba, xri); + if (ctxp) { + /* + * Abort already done by FW, so BA_ACC sent. + * However, the transport may be unaware. + */ + lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, + "6323 NVMET Rcv ABTS xri x%x ctxp state x%x " + "flag x%x oxid x%x rxid x%x\n", + xri, ctxp->state, ctxp->flag, ctxp->oxid, + rxid); + + spin_lock_irqsave(&ctxp->ctxlock, iflag); + ctxp->flag |= LPFC_NVMET_ABTS_RCV; + ctxp->state = LPFC_NVMET_STE_ABORT; + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + + lpfc_nvmeio_data(phba, + "NVMET ABTS RCV: xri x%x CPU %02x rjt %d\n", + xri, smp_processor_id(), 0); + + req = &ctxp->ctx.fcp_req; + if (req) + nvmet_fc_rcv_fcp_abort(phba->targetport, req); + } #endif } @@ -1583,18 +1661,18 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, struct lpfc_nvmet_rcv_ctx *ctxp, *next_ctxp; struct nvmefc_tgt_fcp_req *rsp; uint32_t sid; - uint16_t xri; + uint16_t oxid, xri; unsigned long iflag = 0; - xri = be16_to_cpu(fc_hdr->fh_ox_id); sid = sli4_sid_from_fc_hdr(fc_hdr); + oxid = be16_to_cpu(fc_hdr->fh_ox_id); spin_lock_irqsave(&phba->hbalock, iflag); spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); list_for_each_entry_safe(ctxp, next_ctxp, &phba->sli4_hba.lpfc_abts_nvmet_ctx_list, list) { - if (ctxp->oxid != xri || ctxp->sid != sid) + if (ctxp->oxid != oxid || ctxp->sid != sid) continue; xri = ctxp->ctxbuf->sglq->sli4_xritag; @@ -1623,11 +1701,92 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); spin_unlock_irqrestore(&phba->hbalock, iflag); - lpfc_nvmeio_data(phba, "NVMET ABTS RCV: xri x%x CPU %02x rjt %d\n", - xri, raw_smp_processor_id(), 1); + /* check the wait list */ + if (phba->sli4_hba.nvmet_io_wait_cnt) { + struct rqb_dmabuf *nvmebuf; + struct fc_frame_header *fc_hdr_tmp; + u32 sid_tmp; + u16 oxid_tmp; + bool found = false; + + spin_lock_irqsave(&phba->sli4_hba.nvmet_io_wait_lock, iflag); + + /* match by oxid and s_id */ + list_for_each_entry(nvmebuf, + &phba->sli4_hba.lpfc_nvmet_io_wait_list, + hbuf.list) { + fc_hdr_tmp = (struct fc_frame_header *) + (nvmebuf->hbuf.virt); + oxid_tmp = be16_to_cpu(fc_hdr_tmp->fh_ox_id); + sid_tmp = sli4_sid_from_fc_hdr(fc_hdr_tmp); + if (oxid_tmp != oxid || sid_tmp != sid) + continue; + + lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, + "6321 NVMET Rcv ABTS oxid x%x from x%x " + "is waiting for a ctxp\n", + oxid, sid); + + list_del_init(&nvmebuf->hbuf.list); + phba->sli4_hba.nvmet_io_wait_cnt--; + found = true; + break; + } + spin_unlock_irqrestore(&phba->sli4_hba.nvmet_io_wait_lock, + iflag); + + /* free buffer since already posted a new DMA buffer to RQ */ + if (found) { + nvmebuf->hrq->rqbp->rqb_free_buffer(phba, nvmebuf); + /* Respond with BA_ACC accordingly */ + lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 1); + return 0; + } + } + + /* check active list */ + ctxp = lpfc_nvmet_get_ctx_for_oxid(phba, oxid, sid); + if (ctxp) { + xri = ctxp->ctxbuf->sglq->sli4_xritag; + + spin_lock_irqsave(&ctxp->ctxlock, iflag); + ctxp->flag |= (LPFC_NVMET_ABTS_RCV | LPFC_NVMET_ABORT_OP); + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + + lpfc_nvmeio_data(phba, + "NVMET ABTS RCV: xri x%x CPU %02x rjt %d\n", + xri, raw_smp_processor_id(), 0); + + lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, + "6322 NVMET Rcv ABTS:acc oxid x%x xri x%x " + "flag x%x state x%x\n", + ctxp->oxid, xri, ctxp->flag, ctxp->state); + + if (ctxp->flag & LPFC_NVMET_TNOTIFY) { + /* Notify the transport */ + nvmet_fc_rcv_fcp_abort(phba->targetport, + &ctxp->ctx.fcp_req); + } else { + spin_lock_irqsave(&ctxp->ctxlock, iflag); + lpfc_nvmet_defer_release(phba, ctxp); + spin_unlock_irqrestore(&ctxp->ctxlock, iflag); + } + if (ctxp->state == LPFC_NVMET_STE_RCV) + lpfc_nvmet_unsol_fcp_issue_abort(phba, ctxp, ctxp->sid, + ctxp->oxid); + else + lpfc_nvmet_sol_fcp_issue_abort(phba, ctxp, ctxp->sid, + ctxp->oxid); + + lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 0); + return 0; + } + + lpfc_nvmeio_data(phba, "NVMET ABTS RCV: oxid x%x CPU %02x rjt %d\n", + oxid, raw_smp_processor_id(), 1); lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6320 NVMET Rcv ABTS:rjt xid x%x\n", xri); + "6320 NVMET Rcv ABTS:rjt oxid x%x\n", oxid); /* Respond with BA_RJT accordingly */ lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 0); @@ -1711,6 +1870,18 @@ lpfc_nvmet_wqfull_process(struct lpfc_hba *phba, spin_unlock_irqrestore(&pring->ring_lock, iflags); return; } + if (rc == WQE_SUCCESS) { +#ifdef CONFIG_SCSI_LPFC_DEBUG_FS + if (ctxp->ts_cmd_nvme) { + if (ctxp->ctx.fcp_req.op == NVMET_FCOP_RSP) + ctxp->ts_status_wqput = ktime_get_ns(); + else + ctxp->ts_data_wqput = ktime_get_ns(); + } +#endif + } else { + WARN_ON(rc); + } } wq->q_flag &= ~HBA_NVMET_WQFULL; spin_unlock_irqrestore(&pring->ring_lock, iflags); @@ -1876,8 +2047,16 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf) return; } + if (ctxp->flag & LPFC_NVMET_ABTS_RCV) { + lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, + "6324 IO oxid x%x aborted\n", + ctxp->oxid); + return; + } + payload = (uint32_t *)(nvmebuf->dbuf.virt); tgtp = (struct lpfc_nvmet_tgtport *)phba->targetport->private; + ctxp->flag |= LPFC_NVMET_TNOTIFY; #ifdef CONFIG_SCSI_LPFC_DEBUG_FS if (ctxp->ts_isr_cmd) ctxp->ts_cmd_nvme = ktime_get_ns(); @@ -1931,6 +2110,7 @@ lpfc_nvmet_process_rcv_fcp_req(struct lpfc_nvmet_ctxbuf *ctx_buf) phba->sli4_hba.nvmet_mrq_data[qno], 1, qno); return; } + ctxp->flag &= ~LPFC_NVMET_TNOTIFY; atomic_inc(&tgtp->rcv_fcp_cmd_drop); lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, "2582 FCP Drop IO x%x: err x%x: x%x x%x x%x\n", @@ -2122,6 +2302,9 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba, sid = sli4_sid_from_fc_hdr(fc_hdr); ctxp = (struct lpfc_nvmet_rcv_ctx *)ctx_buf->context; + spin_lock_irqsave(&phba->sli4_hba.t_active_list_lock, iflag); + list_add_tail(&ctxp->list, &phba->sli4_hba.t_active_ctx_list); + spin_unlock_irqrestore(&phba->sli4_hba.t_active_list_lock, iflag); if (ctxp->state != LPFC_NVMET_STE_FREE) { lpfc_printf_log(phba, KERN_ERR, LOG_NVME_IOERR, "6414 NVMET Context corrupt %d %d oxid x%x\n", @@ -2773,7 +2956,7 @@ lpfc_nvmet_sol_fcp_abort_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe, if ((ctxp->flag & LPFC_NVMET_CTX_RLS) && !(ctxp->flag & LPFC_NVMET_XBUSY)) { spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); - list_del(&ctxp->list); + list_del_init(&ctxp->list); spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); released = true; } @@ -2782,7 +2965,7 @@ lpfc_nvmet_sol_fcp_abort_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe, atomic_inc(&tgtp->xmt_abort_rsp); lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6165 ABORT cmpl: xri x%x flg x%x (%d) " + "6165 ABORT cmpl: oxid x%x flg x%x (%d) " "WCQE: %08x %08x %08x %08x\n", ctxp->oxid, ctxp->flag, released, wcqe->word0, wcqe->total_data_placed, @@ -2857,7 +3040,7 @@ lpfc_nvmet_unsol_fcp_abort_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe, if ((ctxp->flag & LPFC_NVMET_CTX_RLS) && !(ctxp->flag & LPFC_NVMET_XBUSY)) { spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); - list_del(&ctxp->list); + list_del_init(&ctxp->list); spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); released = true; } @@ -2866,7 +3049,7 @@ lpfc_nvmet_unsol_fcp_abort_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe, atomic_inc(&tgtp->xmt_abort_rsp); lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, - "6316 ABTS cmpl xri x%x flg x%x (%x) " + "6316 ABTS cmpl oxid x%x flg x%x (%x) " "WCQE: %08x %08x %08x %08x\n", ctxp->oxid, ctxp->flag, released, wcqe->word0, wcqe->total_data_placed, @@ -3237,7 +3420,7 @@ lpfc_nvmet_unsol_fcp_issue_abort(struct lpfc_hba *phba, spin_lock_irqsave(&ctxp->ctxlock, flags); if (ctxp->flag & LPFC_NVMET_CTX_RLS) { spin_lock(&phba->sli4_hba.abts_nvmet_buf_list_lock); - list_del(&ctxp->list); + list_del_init(&ctxp->list); spin_unlock(&phba->sli4_hba.abts_nvmet_buf_list_lock); released = true; } @@ -3246,8 +3429,9 @@ lpfc_nvmet_unsol_fcp_issue_abort(struct lpfc_hba *phba, atomic_inc(&tgtp->xmt_abort_rsp_error); lpfc_printf_log(phba, KERN_ERR, LOG_NVME_ABTS, - "6135 Failed to Issue ABTS for oxid x%x. Status x%x\n", - ctxp->oxid, rc); + "6135 Failed to Issue ABTS for oxid x%x. Status x%x " + "(%x)\n", + ctxp->oxid, rc, released); if (released) lpfc_nvmet_ctxbuf_post(phba, ctxp->ctxbuf); return 1; diff --git a/drivers/scsi/lpfc/lpfc_nvmet.h b/drivers/scsi/lpfc/lpfc_nvmet.h index 2f3f603d94c4..8ff67deac10a 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.h +++ b/drivers/scsi/lpfc/lpfc_nvmet.h @@ -140,6 +140,7 @@ struct lpfc_nvmet_rcv_ctx { #define LPFC_NVMET_ABTS_RCV 0x10 /* ABTS received on exchange */ #define LPFC_NVMET_CTX_REUSE_WQ 0x20 /* ctx reused via WQ */ #define LPFC_NVMET_DEFER_WQFULL 0x40 /* Waiting on a free WQE */ +#define LPFC_NVMET_TNOTIFY 0x80 /* notify transport of abts */ struct rqb_dmabuf *rqb_buffer; struct lpfc_nvmet_ctxbuf *ctxbuf; struct lpfc_sli4_hdw_queue *hdwq; diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index fbc1f1880d53..12ffe5736921 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -845,6 +845,8 @@ struct lpfc_sli4_hba { struct list_head lpfc_nvmet_sgl_list; spinlock_t abts_nvmet_buf_list_lock; /* list of aborted NVMET IOs */ struct list_head lpfc_abts_nvmet_ctx_list; + spinlock_t t_active_list_lock; /* list of active NVMET IOs */ + struct list_head t_active_ctx_list; struct list_head lpfc_nvmet_io_wait_list; struct lpfc_nvmet_ctx_info *nvmet_ctx_info; struct lpfc_sglq **lpfc_sglq_active_list; From patchwork Wed May 22 00:48:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D54D14B6 for ; Wed, 22 May 2019 00:49:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8120728A0F for ; Wed, 22 May 2019 00:49:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7560E2886F; Wed, 22 May 2019 00:49:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2845B2882F for ; Wed, 22 May 2019 00:49:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728155AbfEVAt2 (ORCPT ); Tue, 21 May 2019 20:49:28 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:41020 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728153AbfEVAt2 (ORCPT ); Tue, 21 May 2019 20:49:28 -0400 Received: by mail-pl1-f195.google.com with SMTP id f12so171025plt.8 for ; Tue, 21 May 2019 17:49:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/417Jjd7m4X8lFU69iO0S8Ju3qoLuM52ru+/43vM4hA=; b=sF6a+XDEalu4+4tB3My4xT+UD/FD9sDy5Wxn9nnPANwgJcuKZ6JVuKRV77yimKDsny lI7w0fWovFWXwt0Vd6GOlr+IkgZqk48ASo3R8gMIaqq9u5a0epXMpESvQdPpUrlIhemk AMrEy7YBWgTCAhXaq60n/IvJnSuiCerpFsJYt9qlexCVR9rKIgC0s3XZtmDlQKhq/UqU i5DR5RPmWton5cR1TGXY8Qwq5xcW1/HrCP2WbFqX2VrLsBzYTEMDI5v3xi9Irp1xhQKf dmwoNCmcQHO/oOEKTNtrZVcZ/37qLoBK7DjNvB2SeC+m91dn8uzudJ5fL92IQDLoa5CU RhZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/417Jjd7m4X8lFU69iO0S8Ju3qoLuM52ru+/43vM4hA=; b=oK1l/TO33KguxA5vDTcxQnGuIurIpvJMJPtrC9WM6f0hTZ+9MR7ZhqW9VCCECKbGIb dV+BTEioluTwiIvXIHdubTmC6o6h5feFlbkfQuQG6t6n7A/6Kow3GmDiLR4HkUXEpE2t KbItNc/3++HE+drBR46h+xWs4pgSL5RHhMH/pFxF6ZGT2YIZOUt0J2FslZkBNA2dTJuH vUnpE7w+SYTaF4k/6QIl4aObxmRx9OIMp1ldMHAPtRAlVlqgRW1vXrJIkGWNI4MrXM2/ 9E31okz/y8nE6rWAq8pt0M9Zx/N1kk+gf+wqaWxQS+hStTj92ehr1ySJrQTwbtaCJqzG m4jg== X-Gm-Message-State: APjAAAXORwWUDRJDGr0tHOM+J/4h610/zdovqxvYaKXK2EY4jaf+mSK9 nyBvWSLDJbd7iAqeqJHh0FJqkupT X-Google-Smtp-Source: APXvYqzmxiDXOT/FbWF4u2lu+/dgqmu6+MXCMqRVdzcNWkettGiSNhIpuQYXIhV41HVR6tbh/l345w== X-Received: by 2002:a17:902:a613:: with SMTP id u19mr72109139plq.42.1558486167682; Tue, 21 May 2019 17:49:27 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:27 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 07/21] lpfc: Revert message logging on unsupported topology Date: Tue, 21 May 2019 17:48:57 -0700 Message-Id: <20190522004911.573-8-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Turns out the message change in 12.2.0.1 for unsupported topology makes the linux driver out of sync with other products. Revert the message back to the prior content for product consistency. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- As this is not a technical issue, I've not added a Fixes line --- drivers/scsi/lpfc/lpfc_attr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index e9adb3f1961d..aabd42c4c6f8 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -4090,9 +4090,9 @@ lpfc_topology_store(struct device *dev, struct device_attribute *attr, } if ((phba->pcidev->device == PCI_DEVICE_ID_LANCER_G6_FC || phba->pcidev->device == PCI_DEVICE_ID_LANCER_G7_FC) && - val != FLAGS_TOPOLOGY_MODE_PT_PT) { + val == 4) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, - "3114 Only non-FC-AL mode is supported\n"); + "3114 Loop mode not supported\n"); return -EINVAL; } phba->cfg_topology = val; From patchwork Wed May 22 00:48:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954625 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DABB924 for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 629412882F for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 576BA28A24; Wed, 22 May 2019 00:49:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD9032882F for ; Wed, 22 May 2019 00:49:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728174AbfEVAt3 (ORCPT ); Tue, 21 May 2019 20:49:29 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:45942 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728153AbfEVAt3 (ORCPT ); Tue, 21 May 2019 20:49:29 -0400 Received: by mail-pf1-f193.google.com with SMTP id s11so328908pfm.12 for ; Tue, 21 May 2019 17:49:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=de13heP8GurAC0k8VaKozAE/7LTqjlixmb+w3JaByBM=; b=lCwRt0Ti5+hAZ4VcH3gqgdS97vhRGp5YeU1wHKBRwpiBCrgxXhzryea7eG0SG+eVUg 7k/RsDwXQJ53TaI5Y/hfxaOH1XJirO0o5wx+0rkesdohst4tSQWTT8iLj3E0S28gE5Fk UkHF9l0nCRwZQ2jzek4EsGjuJTogaC52IBs5kNz/SgeTRBnYEUeMkfR3XHgEzmSOWRIL DvGz73rk+D6VQYnXXJDKy7Lo/u9dfb1mI4zaslO4mgGIdRGK1OzV2oD3x1xXWaJn9mHG V9X2j6cKI391dLvaWqi/5CrzA26y3UZckJzUrjt8KVM0p5hoPqjqdTXS/4wywEHiqpR8 QDdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=de13heP8GurAC0k8VaKozAE/7LTqjlixmb+w3JaByBM=; b=S7qhJeIu4am9W6rtqQtzFv6KPSFWq1dJljYCan1ogWuN9w6kIXz3TF+d7agqRXnye8 qgk1gr3nIg9JLSfATcW/EbZQCu9ylfjbgKxG4q00hfqBB4ZxFWNeyFJPyx6ckLgkFJvb UNZAz8vOHCSWuAiyy382DtouK6VWndxoVxudmLuvGwvii/DEGR/ULmWpJ2cQALHhH1bl ps2ekeByaAlhGKPSdasKAQSE3E4sBIoznPVmmsIemzbr3/PgdCZrxU6aHacv3lGZs6Ys 0wGk1tPA0YUUem+xEMNcpSC04vTA87ltqTUbbRKcXvS0xyMjLr6FQ9nPezsWE7ATAzcu JDhQ== X-Gm-Message-State: APjAAAXYamr9htsAMg4pWReLWi+K3Qq7l3vjQgQgqzuE9vn3XR0D+Kra 2TK31qNWYlXJ16m+NOQ74qmK3e4L X-Google-Smtp-Source: APXvYqyNpUFqzB3KVednXjDmi6Cf5uV0BuCxui0j5ZcCxWMqupErzhBlM20yNiCmpSCR69/NwwzWCQ== X-Received: by 2002:a65:62d8:: with SMTP id m24mr85852278pgv.141.1558486168426; Tue, 21 May 2019 17:49:28 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:28 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 08/21] lpfc: Fix PT2PT PLOGI collison stopping discovery Date: Tue, 21 May 2019 17:48:58 -0700 Message-Id: <20190522004911.573-9-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Under heavy load the target stops responding, the drivers aborts timeout and we start recovery by logging out of the target, but the target is never logged into again. In a point-to-point scenario, there were battling PLOGI's. When we received a PLOGI request after having sent one, the driver cancels the processing of the original plogi. However, the completion path of the remaining plogi was coded to skip the reg_rpi that should be happening on the 2nd plogi. Correct by adding a simple pt2pt check such that the 2nd plogi isn't skipped and the reg_login occurs. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index c8fb0b455f2a..532728ee1f95 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -4196,6 +4196,7 @@ lpfc_cmpl_els_rsp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, if ((rspiocb->iocb.ulpStatus == 0) && (ndlp->nlp_flag & NLP_ACC_REGLOGIN)) { if (!lpfc_unreg_rpi(vport, ndlp) && + (!(vport->fc_flag & FC_PT2PT)) && (ndlp->nlp_state == NLP_STE_PLOGI_ISSUE || ndlp->nlp_state == NLP_STE_REG_LOGIN_ISSUE)) { lpfc_printf_vlog(vport, KERN_INFO, From patchwork Wed May 22 00:48:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1ED4D14B6 for ; Wed, 22 May 2019 00:49:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 136E81FE6B for ; Wed, 22 May 2019 00:49:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 085D528A0F; Wed, 22 May 2019 00:49:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABEA51FE6B for ; Wed, 22 May 2019 00:49:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728179AbfEVAtb (ORCPT ); Tue, 21 May 2019 20:49:31 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:36535 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728165AbfEVAt3 (ORCPT ); Tue, 21 May 2019 20:49:29 -0400 Received: by mail-pl1-f195.google.com with SMTP id d21so180397plr.3 for ; Tue, 21 May 2019 17:49:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ch1GpZn4+x/spa9si74gDQsnT6AMvKZB+KZkFHXzv0A=; b=eThR142ug9oZviGCXQv4uU8/eHA7VrKe1XLNdx/+SpNg0iHRkDf8NWpigWg7xm4C+R kZWhE5Hj4K8URmhRMwu53ZCd4GkVuwH4+lA3zUeVqCoERxWt5SOrtTAyn00R/iGzpI1Q pt/rRHpUl152tLS2ZseiArUrky47h7+WWTcYpfhsZgLgkz9T6/ID3TFJ9vAPrTPVAykS pwcDSXibCJAb11zNbQSwGXvQgRZZPn5m7it9FFWLDPjWM+DJKVvZNY8SkhnoOkZF1DAT YWy/9uu2/ZEKhr/5Ospngxi7Qj4LT5Vid/VuiK1qikz/BIXboCU/NTLyZhiLvzPyeLc0 zNyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ch1GpZn4+x/spa9si74gDQsnT6AMvKZB+KZkFHXzv0A=; b=MiHd5DPBr/FbgdWgXAj7VgbsK8Ylz48Bh523EmH4cREqdJ0FZnwGMiCzeUw3vn0XFz AYmlfbEudwd/Nw9qat5b7Usjj2Vgd5Qs6zI2vkHH+f2yaHx+r1FwT/idRei3W9Z+/lVb xNmBacuFF993XUI/XqBRi+Yf22HeLOeqp0a7SMlPNy+kTjiId4ibYq0PDkisTL6EsBZh CjbeAe1LIGBcWLnuJ2O/OyfdzfJ5lFAhi7Z+A+den0WrNvuAfZRH09ASllD8RY/Vl8YN b0TGNkZwKnANHxAW4j7gcJT3r04vsTfBC507bhUvz8+qFyd+7kt1LIjgr8qu9izSWBsC ln1g== X-Gm-Message-State: APjAAAVKKlXk1OQLebdhbJ0HpDcj775fn8j7XtiBV9r4iV5HDcjmHb6a b8gxN5AVQeuf9E3qh/9hNvufomKq X-Google-Smtp-Source: APXvYqxu0jq1bXAgsupBdIS5DccZm7K8412WXs61QiT/W7UP3WVsDU7ydKGHvoHSsTYDW9XxHjoz4Q== X-Received: by 2002:a17:902:704c:: with SMTP id h12mr31590563plt.65.1558486169181; Tue, 21 May 2019 17:49:29 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:28 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 09/21] lpfc: Prevent 'use after free' memory overwrite in nvmet LS handling. Date: Tue, 21 May 2019 17:48:59 -0700 Message-Id: <20190522004911.573-10-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use-after-free memory overwrite detected. Problem reported by Ewan Milne at Red Hat after running lpfc target with additional memory checking enabled. Race condition when lpfc_nvmet_xmt_ls_rsp_cmp frees the ctxp memory in interrupt context before lpfc_nvmet_xmt_ls_rsp clears a field in the ctxp after successfully issuing the wqe. Remove the unnecessary ctxp write after reposting the rq buffer. The ctxp->rqb_buffer field is not checked in LS handling after the wqe is submitted. Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reported-by: Ewan Milne --- drivers/scsi/lpfc/lpfc_nvmet.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index a943b2a20001..08c2c4e3515b 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -907,7 +907,6 @@ lpfc_nvmet_xmt_ls_rsp(struct nvmet_fc_target_port *tgtport, * before freeing ctxp and iocbq. */ lpfc_in_buf_free(phba, &nvmebuf->dbuf); - ctxp->rqb_buffer = 0; atomic_inc(&nvmep->xmt_ls_rsp); return 0; } From patchwork Wed May 22 00:49:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64BDC924 for ; Wed, 22 May 2019 00:49:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 580BA1FE6B for ; Wed, 22 May 2019 00:49:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C0E528875; Wed, 22 May 2019 00:49:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5FD21FE6B for ; Wed, 22 May 2019 00:49:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728186AbfEVAtd (ORCPT ); Tue, 21 May 2019 20:49:33 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:36537 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728153AbfEVAtb (ORCPT ); Tue, 21 May 2019 20:49:31 -0400 Received: by mail-pl1-f195.google.com with SMTP id d21so180409plr.3 for ; Tue, 21 May 2019 17:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xmLUUzk8AUHFUinh2BQF66VpDpUdo6v5ekPIei2arX4=; b=qqj9v4XpstD7S3RwDNY8kc1oSlhjXnVoA02XK34vcv4wcf+KCl7o0aMoICU4NoVUdZ TaTlr1Kw5Wa7EM2NMZadokPmkBCsW55yeP1bkjnICz6/GIw/NyJyjlhKWw37FVECcLdh flWRlNNQVAJ6zgCyCuxjZ2Oh6U+vrU5wUeygJDABWS51aAqfUUeClvJ84jyS8ZvF0wHU zml0pMaa3eIka7G2WuVOlE5a4XO/iRu9DwpaoponIP1Gx+2EtDX667eRUWGfPaPT/qTR 3mQJ71o6Wo0OrpUqdxiH7q5Wu5AZuqepGu46/WH5ZDhb7hEixHTAToaEc81p4PRw4mgy 8sKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xmLUUzk8AUHFUinh2BQF66VpDpUdo6v5ekPIei2arX4=; b=U+waYKrWfIJuCffrh6gqAuF7dqWSC/rUsxdeW6nfQqFPq0WHUQHv8MoiuaV/Drg+FU FfiQDPJiH5PNtogrK46pZyHsvIifAT4GmHd/rVVr8KTenZNI0bHoVP8uDEF4nRaUfm5j XLczntkkO9KJHW9Fsd/tou82gPMl89PCN1MlPd6z+CGpdygIpZc/qwsdb7U7OHDzsoy5 9Mdl2dpDpbdFZe8yV89ZLlY3cK7GGnZTZVDEC7WdwXIKt7hWPlp1Gk4fp0lS+klyCJdU PvMe/dsT1uwpR4puaDovfBLLdwSU3rZOl6aN4YJot/9ls6Jw64zdSWvQCB3li6XroayQ gbkw== X-Gm-Message-State: APjAAAVAMD+iXe03SaAWlU2u0JM4lc/NcOzoKV/BQ/vLKg7YZi10lNMn TmIhLLLyWbKTHCDtWIpLTKTRJkWX X-Google-Smtp-Source: APXvYqxsmBwJCOUwENOL9/D6OO6F21vyZWZ+Zk28WYm0JXYmJNuc5rugk/KVrSBP2ui6ZKMLYupi+A== X-Received: by 2002:a17:902:b695:: with SMTP id c21mr87146086pls.160.1558486170033; Tue, 21 May 2019 17:49:30 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:29 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 10/21] lpfc: Cancel queued work for an IO when processing a received ABTS. Date: Tue, 21 May 2019 17:49:00 -0700 Message-Id: <20190522004911.573-11-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When queued work is executed posting a new command to the transport the driver is reporting a null buffer. The driver had received an ABTS which matched a command that had been scheduled for delivery to the transport. The driver proceeded to cancel the command, but the work item was never cancelled. Fix by cancelling the queued work item. Also turns out the ABTS response was not properly sending a BA_ACC, so set the flag to send the ACC. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvmet.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 08c2c4e3515b..36e8d842d973 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -1766,6 +1766,7 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, nvmet_fc_rcv_fcp_abort(phba->targetport, &ctxp->ctx.fcp_req); } else { + cancel_work_sync(&ctxp->ctxbuf->defer_work); spin_lock_irqsave(&ctxp->ctxlock, iflag); lpfc_nvmet_defer_release(phba, ctxp); spin_unlock_irqrestore(&ctxp->ctxlock, iflag); @@ -1777,7 +1778,7 @@ lpfc_nvmet_rcv_unsol_abort(struct lpfc_vport *vport, lpfc_nvmet_sol_fcp_issue_abort(phba, ctxp, ctxp->sid, ctxp->oxid); - lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 0); + lpfc_sli4_seq_abort_rsp(vport, fc_hdr, 1); return 0; } From patchwork Wed May 22 00:49:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DEC4914C0 for ; Wed, 22 May 2019 00:49:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D30D22882F for ; Wed, 22 May 2019 00:49:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C801728A0F; Wed, 22 May 2019 00:49:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E8ED628895 for ; Wed, 22 May 2019 00:49:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728202AbfEVAte (ORCPT ); Tue, 21 May 2019 20:49:34 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:37667 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728165AbfEVAtb (ORCPT ); Tue, 21 May 2019 20:49:31 -0400 Received: by mail-pl1-f194.google.com with SMTP id p15so179188pll.4 for ; Tue, 21 May 2019 17:49:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=hE7RU2gTsXp7a1/zfkhwqyxBxkgBabYLcwxgJQRKJpY=; b=M51J2vWSe8UBrUK+DSt0wThoiN3+BtevwLO849zO2EiDSrUOjEdtVMg9OWQPoKr090 SBJdpGIUjh/0kbsYFhtv7Ghn3++izIpy9FZ4PGazOesIe+3BNoQTpjfeG7tEaY+FSU5A STXT0WWyLipjwvTV5+iqB76s0svJUmvFtddvREMeDUCpMcUiSkWy8wQwTXX2zHtjpxHz jVsofjC9B0c6bfd6itEmykXkNZulhMHMX1RDaIu2YXICW73gxWYRguydC3K2AmpB+/j7 SXAmP1rgqCd1D5gw94JWowATU2GSukD6SHGnSo8DZEpy0jzPbfB+hmA+Ncx2ZOtFIdo1 IBDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=hE7RU2gTsXp7a1/zfkhwqyxBxkgBabYLcwxgJQRKJpY=; b=W34Yb+1DOW68oI8/ziHp3w+wbj/k4SZnKr6UWUuIuNFJgPZIq1+2nbYLHDBc3Ypejc nUY6LHhMCOAAEy5LAoaakwQ+WTunRoTcu9Sj9f2kVrJogksTIfmkPOjPgXcocd5epx9g m5DVBiPFy4XdM/IVgYcpoiT38gk6dhLWHfbjwFleGSwAfIpfZ8Kl03XS6MBe8rRhMcdH KqUS/sQ40mos7oZmdaSPF56sLjXEqOGwABmcf07un/Ihug9wAhvQjz2oDxuLUrk5xLJQ F45CxYPPXFUwqw8WQT5w9tiStSS4EuW+/Mz9xiFZqafyp6Zr9Fpr6JJeTcSA5Pyb0AaR m1LA== X-Gm-Message-State: APjAAAVB31wgJl+m+CMQuv65QXMzVQ0xI0QrhuWfbCE7ZJ6oIS1nO/Q7 USqgdTmJy54dfN5mi4q57nOqrpfw X-Google-Smtp-Source: APXvYqzE84H09EvWnQ73qSjqVdhw/Mk6YCjzXnB6HAlclSbbk7H8lN0VsKoDjdSuu4dIgsPz1TJQ1Q== X-Received: by 2002:a17:902:f081:: with SMTP id go1mr39658997plb.211.1558486170897; Tue, 21 May 2019 17:49:30 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:30 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 11/21] lpfc: Fix hardlockup in scsi_cmd_iocb_cmpl Date: Tue, 21 May 2019 17:49:01 -0700 Message-Id: <20190522004911.573-12-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There is a race condition with the abort handler declaring a waitq item on it's stack, followed by a timeout in the abort handler that has it give up on the abort return to its caller. When the io is finally aborted and its completion handler called, it references the waitq element that the abort_handler set up, which is no longer valid resulting in a deadlock. Fix by clearing the waitq reference, under lock, when the abort handler timeout gives up. Have the completion handler validate the waitq before referencing it. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_scsi.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index 3873d5b97bc6..5a5a9bbe6023 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -3879,10 +3879,8 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn, */ spin_lock(&lpfc_cmd->buf_lock); lpfc_cmd->cur_iocbq.iocb_flag &= ~LPFC_DRIVER_ABORTED; - if (lpfc_cmd->waitq) { + if (lpfc_cmd->waitq) wake_up(lpfc_cmd->waitq); - lpfc_cmd->waitq = NULL; - } spin_unlock(&lpfc_cmd->buf_lock); lpfc_release_scsi_buf(phba, lpfc_cmd); @@ -4718,6 +4716,9 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd) iocb->sli4_xritag, ret, cmnd->device->id, cmnd->device->lun); } + + lpfc_cmd->waitq = NULL; + spin_unlock(&lpfc_cmd->buf_lock); goto out; From patchwork Wed May 22 00:49:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62462924 for ; Wed, 22 May 2019 00:49:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5385428A26 for ; Wed, 22 May 2019 00:49:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 444F328875; Wed, 22 May 2019 00:49:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA30228875 for ; Wed, 22 May 2019 00:49:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728189AbfEVAtd (ORCPT ); Tue, 21 May 2019 20:49:33 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:40457 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728180AbfEVAtc (ORCPT ); Tue, 21 May 2019 20:49:32 -0400 Received: by mail-pf1-f196.google.com with SMTP id u17so339542pfn.7 for ; Tue, 21 May 2019 17:49:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KrMD7sQ5G+C5wnNN4XWbWIzy+HEyYIkwcK0jX9ZMEkc=; b=eWh8PIkPASvY0ZzKiHfyGxhE7f6RyEItxXe2tTx4AAvBkDzTGwHLNqeJt8ylCie8qn hIfELP485FmSptqLSKSDuvefl2eT6hVvGuF4IHAgQaUuiY4e2gG7GFNmyrfFcCm7LnUy piJ9kgeWpofEw1xHJEn0o9msNUZBVexW5LvGaiShMQGFgJIrgHsVw4Sbw7lg/qvWYd0a /A7vQ0R/spILizQrp+HH7HV6x3XySfDETEpT8sM0fJwuGUKiryfFm2OTOHS9hg/TUhyO EnYpzHIyf14OfaISPdOtBtuE/zZRYhc1BN8Re3EFWLA+98KBgBeOjOiwKxZoc/t/ebyO HIDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KrMD7sQ5G+C5wnNN4XWbWIzy+HEyYIkwcK0jX9ZMEkc=; b=lGYmm1swQZX1Uavi/4dOH777bXw8IvgzSjQjDEKqP/0XAYrJXFYmlNjYNaG9tthpOC bdCuhKuQuhBULfIx8xcenGlukbwv4aQXe336W9uy59g66mnUF8dyODno8XuqNqr66z6w K9tIn+mb4QNx1H/wNzKi1/xRIL0cdvU6QGmaE1ferXSW+o3C1yPDALdXu+2VVF9HhR8U XNvWfaKAHaaCmBuEaDPeLr0c+k5DVtn+THPmxBeCTOzP+eQDkBe4ZuyOU4aYSqC/1DN/ lnMdkcK7OZZdb+ZGMRrE2QaF2CYCL1Ak9mofQ8HyepBx3NP6cJPe2zCyK6layQfTqBcy 82/w== X-Gm-Message-State: APjAAAUfRrok2HAhUXu9pJbfn2JF/piyher/YNG8gL5hb8QN2J73TUnB sOkS5apAgKe5UdXKNJvegedrlasq X-Google-Smtp-Source: APXvYqxEv8uRN78Sop2EiwufBCV7M4T//6eyzcs0TKUlNUigVB2GPpkaNcVdSmBGDFdsNkxrAYfVwg== X-Received: by 2002:a63:950d:: with SMTP id p13mr86873284pgd.269.1558486171684; Tue, 21 May 2019 17:49:31 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:31 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 12/21] lpfc: Rework misleading nvme not supported in firmware message Date: Tue, 21 May 2019 17:49:02 -0700 Message-Id: <20190522004911.573-13-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver unconditionally says fw doesn't support nvme when in truth it was a driver parameter settings that disabled nvme support. Rework the code validating nvme support to accurately report what condition is disabling nvme support. Save state on whether nvme fw supports nvme in case sysfs attributes change dynamically. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 55 +++++++++++++++++++++++++++++-------------- drivers/scsi/lpfc/lpfc_sli4.h | 1 + 2 files changed, 38 insertions(+), 18 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 70afa585b027..1468a4d7c501 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -11333,24 +11333,43 @@ lpfc_get_sli4_parameters(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq) mbx_sli4_parameters); phba->sli4_hba.extents_in_use = bf_get(cfg_ext, mbx_sli4_parameters); phba->sli4_hba.rpi_hdrs_in_use = bf_get(cfg_hdrr, mbx_sli4_parameters); - phba->nvme_support = (bf_get(cfg_nvme, mbx_sli4_parameters) && - bf_get(cfg_xib, mbx_sli4_parameters)); - - if ((phba->cfg_enable_fc4_type == LPFC_ENABLE_FCP) || - !phba->nvme_support) { - phba->nvme_support = 0; - phba->nvmet_support = 0; - phba->cfg_nvmet_mrq = 0; - lpfc_printf_log(phba, KERN_ERR, LOG_INIT | LOG_NVME, - "6101 Disabling NVME support: " - "Not supported by firmware: %d %d\n", - bf_get(cfg_nvme, mbx_sli4_parameters), - bf_get(cfg_xib, mbx_sli4_parameters)); - - /* If firmware doesn't support NVME, just use SCSI support */ - if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP)) - return -ENODEV; - phba->cfg_enable_fc4_type = LPFC_ENABLE_FCP; + + /* Check for firmware nvme support */ + rc = (bf_get(cfg_nvme, mbx_sli4_parameters) && + bf_get(cfg_xib, mbx_sli4_parameters)); + + if (rc) { + /* Save this to indicate the Firmware supports NVME */ + sli4_params->nvme = 1; + + /* Firmware NVME support, check driver FC4 NVME support */ + if (phba->cfg_enable_fc4_type == LPFC_ENABLE_FCP) { + lpfc_printf_log(phba, KERN_INFO, LOG_INIT | LOG_NVME, + "6133 Disabling NVME support: " + "FC4 type not supported: x%x\n", + phba->cfg_enable_fc4_type); + goto fcponly; + } + } else { + /* No firmware NVME support, check driver FC4 NVME support */ + sli4_params->nvme = 0; + if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT | LOG_NVME, + "6101 Disabling NVME support: Not " + "supported by firmware (%d %d) x%x\n", + bf_get(cfg_nvme, mbx_sli4_parameters), + bf_get(cfg_xib, mbx_sli4_parameters), + phba->cfg_enable_fc4_type); +fcponly: + phba->nvme_support = 0; + phba->nvmet_support = 0; + phba->cfg_nvmet_mrq = 0; + + /* If no FC4 type support, move to just SCSI support */ + if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP)) + return -ENODEV; + phba->cfg_enable_fc4_type = LPFC_ENABLE_FCP; + } } /* Only embed PBDE for if_type 6, PBDE support requires xib be set */ diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 12ffe5736921..8b28a55c73bb 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -514,6 +514,7 @@ struct lpfc_pc_sli4_params { #define LPFC_WQ_SZ64_SUPPORT 1 #define LPFC_WQ_SZ128_SUPPORT 2 uint8_t wqpcnt; + uint8_t nvme; }; #define LPFC_CQ_4K_PAGE_SZ 0x1 From patchwork Wed May 22 00:49:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1E0214B6 for ; Wed, 22 May 2019 00:49:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 964492882F for ; Wed, 22 May 2019 00:49:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8AFCF28A0F; Wed, 22 May 2019 00:49:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8070828875 for ; Wed, 22 May 2019 00:49:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728198AbfEVAte (ORCPT ); Tue, 21 May 2019 20:49:34 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:43744 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728183AbfEVAtd (ORCPT ); Tue, 21 May 2019 20:49:33 -0400 Received: by mail-pf1-f196.google.com with SMTP id c6so333064pfa.10 for ; Tue, 21 May 2019 17:49:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=SUnKxYhAN4+6nBCP66ac/DR2LVbWpgldXGaWm9XnYVc=; b=aVKtql943TpBlUIy5qkdhDePyep7hSJUEpbuSPA5h5G5Tg6bmaMOI8hDx2gMW0NTNb 4us5U2eCGCQfYraTpdBi/AvmXPVFlAttPn3iKERrvL160gOyw69nncfPqsJhr1djcmHj XWO8NqV7rALLOaj9XiLNqqZ7lRNzrjZfrWAXYy0+g5i4Uv8jbG4U6yNssVHTy+MVr2GA jXpAFh5H0HIPlpOkGPspCqHayPIxajQGw6lezDNhi6yT/i14MtRZgYwMdzvbeOLDynTj WVHxc5OexiTV94KAHf6P6BlVTgGPV7o9s0kcbvYgJKX5xXMv0BSEMVHkKZVjzwBnbxOf rUBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=SUnKxYhAN4+6nBCP66ac/DR2LVbWpgldXGaWm9XnYVc=; b=uPyMEZqQJiNXQLkeMWqiJ04QSwEsvnsNLAaTic1onf9pn71IlfdGOHpieDUwj1ba0Q se87+YVP9Qg+FaRDNZkh5GwqZFjqrbZ1kMs5hFLUeR5wZvtxMoLfxPxGtnvmkvhM0NMc Bbs2KMJjbzEssv2wEDb2B1z1OWU+jzcTJhuU4vy6ZF1fCz2I/3iQOKwZqB4syzE+RprW Ces7dZxCgCW+w/U6/8s8obzOdEuPmLsDyf8PXLVY3oL3msaCbJnTe4y5Pc2eSTpXlaio lJZe5dydExoVrNFS8Ip3qIb3IfU1E9qIZS3FD08tpjZKjdNe4YdQ6E9EhX7wFZ/i2/1D +94w== X-Gm-Message-State: APjAAAV9HBNnAvxelimCLoh9WTuJrgEoYHll4DUzhzja5KaWkHQ0JqoZ wAtNiF4bXqgRKohKf8wvSjzautVd X-Google-Smtp-Source: APXvYqzDBdP8in6oWYiykKLWCwUKzUQ/tu1JKyRAYHyMqwLfVuMmLviiwCByOnGjDSooBHn1gLH2sg== X-Received: by 2002:a62:e201:: with SMTP id a1mr91429127pfi.67.1558486172375; Tue, 21 May 2019 17:49:32 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:32 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 13/21] lpfc: Fix memory leak in abnormal exit path from lpfc_eq_create. Date: Tue, 21 May 2019 17:49:03 -0700 Message-Id: <20190522004911.573-14-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP eq create is leaking mailbox memory if it encounters an error. rework error path to free the memory. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_sli.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 946f3024d4db..4432060f7315 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -14641,8 +14641,10 @@ lpfc_eq_create(struct lpfc_hba *phba, struct lpfc_queue *eq, uint32_t imax) lpfc_printf_log(phba, KERN_ERR, LOG_SLI, "0360 Unsupported EQ count. (%d)\n", eq->entry_count); - if (eq->entry_count < 256) - return -EINVAL; + if (eq->entry_count < 256) { + status = -EINVAL; + goto out; + } /* fall through - otherwise default to smallest count */ case 256: bf_set(lpfc_eq_context_count, &eq_create->u.request.context, @@ -14694,7 +14696,7 @@ lpfc_eq_create(struct lpfc_hba *phba, struct lpfc_queue *eq, uint32_t imax) eq->host_index = 0; eq->notify_interval = LPFC_EQ_NOTIFY_INTRVL; eq->max_proc_limit = LPFC_EQ_MAX_PROC_LIMIT; - +out: mempool_free(mbox, phba->mbox_mem_pool); return status; } From patchwork Wed May 22 00:49:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 761BB924 for ; Wed, 22 May 2019 00:49:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A55128875 for ; Wed, 22 May 2019 00:49:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F0F928895; Wed, 22 May 2019 00:49:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2E4028A24 for ; Wed, 22 May 2019 00:49:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728215AbfEVAtg (ORCPT ); Tue, 21 May 2019 20:49:36 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:46506 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728184AbfEVAtd (ORCPT ); Tue, 21 May 2019 20:49:33 -0400 Received: by mail-pl1-f196.google.com with SMTP id r18so159594pls.13 for ; Tue, 21 May 2019 17:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5vkbR7OsLuKbrSI2TPtp/d7+DOzMcDSb1ztVyrbXNRg=; b=SWGccQJ/3+Qa6JgUeFiM+8ff8z289nyK5sSl4jpQmX8UykuUEzGjt8sgA2Q1PBr4K7 35jatL4FzL9r+0prSQQz2ueEyBWFFvd63MUJh7Btpxn4Hm7eN7iQHs7GZFqyInrJFR6y Sk5XsC6m3nvWJKotZDGTzjrKws5SiAzEP+RHEeq5EhfdsZm0FP/lRe4boyxjuiHas59d Yi3XERsrPEMJyoKwR5VLmbjsOxVNI3ldj2xsC3NifOmanBJ8WmiOyXvjt1GBadiZdtUC wpDrT8DjFqhVnb9vmAlMeLfbKV9evMmcHmEl9pHlH3E7J6zrCgfErZzsS7pIrYU6NDMk KhcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5vkbR7OsLuKbrSI2TPtp/d7+DOzMcDSb1ztVyrbXNRg=; b=AQMkqiIrCbApNtJIXbwBz3szcgqWgX63u+FB/9ifoaiC0YfVSk+WIQOigss/HOs0FL vRmxArdIWbWP3QwYKF0Q1xMx8Cs4BKhWj0p4+dH4Ryz15J+GnP94BGOf8VgY97Vl7H4T XyfEOcI3QvONUOA8IddPwaE8Z3d12Cq415XfHZPIXGyoYIeykNmGLnyv5xKK60TSktSZ 0WbjXHu9AoFkfnspg5KhF0Qba9MFsies0FjboB/INDfHyuOt2JpHtfUHiS9AcQ1XIBCj zDvzWV3taSoNrQixTasUf1nYI32rxabdysJmLzXPfrFNBtUPXoDE1q2ux53OJb8lBnFE LfiA== X-Gm-Message-State: APjAAAXq3DrNjSr/uzan2xPsRnZy9UrxagO4xp2/VN5eh9U1B1BG4b6E zLc5v/DebuNRdzO6c7WkKxQ+p1B+ X-Google-Smtp-Source: APXvYqzfINsFBTDBLaECzkpSffLf5fRbM7qCvqCtWsJ1vny6zpIqeDRqwwKzxVpuj55JLCVokfLnhg== X-Received: by 2002:a17:902:12d:: with SMTP id 42mr79584201plb.4.1558486173056; Tue, 21 May 2019 17:49:33 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:32 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 14/21] lpfc: Fix incorrect logical link speed on trunks when links down Date: Tue, 21 May 2019 17:49:04 -0700 Message-Id: <20190522004911.573-15-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Invalid logical speed is displayed for trunk enabled ports when all ports are down. Also noted that link speed is incorrectly reported for the units when links are up. Current code is returning the logical link speed from the last event from the adapter. In cases where the last link went down, the link speed in the event was not valid - meaning that although the links where down the field had a bogus value. Rework the event handling to qualify the trunk link state before using the event speed data. Also correct units on other areas where the logical link speed was taken from a link event. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_bsg.c | 2 +- drivers/scsi/lpfc/lpfc_init.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_bsg.c b/drivers/scsi/lpfc/lpfc_bsg.c index b0202bc0aa62..b7216d694bff 100644 --- a/drivers/scsi/lpfc/lpfc_bsg.c +++ b/drivers/scsi/lpfc/lpfc_bsg.c @@ -5741,7 +5741,7 @@ lpfc_get_trunk_info(struct bsg_job *job) event_reply->port_speed = phba->sli4_hba.link_state.speed / 1000; event_reply->logical_speed = - phba->sli4_hba.link_state.logical_speed / 100; + phba->sli4_hba.link_state.logical_speed / 1000; job_error: bsg_reply->result = rc; bsg_job_done(job, bsg_reply->result, diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 1468a4d7c501..73b77aaf7135 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -5055,7 +5055,7 @@ lpfc_update_trunk_link_status(struct lpfc_hba *phba, bf_get(lpfc_acqe_fc_la_speed, acqe_fc)); phba->sli4_hba.link_state.logical_speed = - bf_get(lpfc_acqe_fc_la_llink_spd, acqe_fc); + bf_get(lpfc_acqe_fc_la_llink_spd, acqe_fc) * 10; /* We got FC link speed, convert to fc_linkspeed (READ_TOPOLOGY) */ phba->fc_linkspeed = lpfc_async_link_speed_to_read_top( @@ -5158,8 +5158,14 @@ lpfc_sli4_async_fc_evt(struct lpfc_hba *phba, struct lpfc_acqe_fc_la *acqe_fc) bf_get(lpfc_acqe_fc_la_port_number, acqe_fc); phba->sli4_hba.link_state.fault = bf_get(lpfc_acqe_link_fault, acqe_fc); - phba->sli4_hba.link_state.logical_speed = + + if (bf_get(lpfc_acqe_fc_la_att_type, acqe_fc) == + LPFC_FC_LA_TYPE_LINK_DOWN) + phba->sli4_hba.link_state.logical_speed = 0; + else if (!phba->sli4_hba.conf_trunk) + phba->sli4_hba.link_state.logical_speed = bf_get(lpfc_acqe_fc_la_llink_spd, acqe_fc) * 10; + lpfc_printf_log(phba, KERN_INFO, LOG_SLI, "2896 Async FC event - Speed:%dGBaud Topology:x%x " "LA Type:x%x Port Type:%d Port Number:%d Logical speed:" From patchwork Wed May 22 00:49:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0507614C0 for ; Wed, 22 May 2019 00:49:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAD3B28875 for ; Wed, 22 May 2019 00:49:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE7FE28895; Wed, 22 May 2019 00:49:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25DF828A24 for ; Wed, 22 May 2019 00:49:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728127AbfEVAtf (ORCPT ); Tue, 21 May 2019 20:49:35 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:46508 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728165AbfEVAte (ORCPT ); Tue, 21 May 2019 20:49:34 -0400 Received: by mail-pl1-f196.google.com with SMTP id r18so159610pls.13 for ; Tue, 21 May 2019 17:49:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pNz02KWu9MY5I80lfuOTL/E+wprRSQEPXW1e5pknxwY=; b=k9LVFo+0bcJEm7a/LYP0IKC8IU8PX/XZ+3cNbRTJ446haQkwUTD++BclBXlVCX/XRz yblpoZuXyUM7tJJGK4fzfa3qRhWAc5Rsb81TATVFSoE5BoLxOegM4dgyib60ZcHzxDmH r8S8+SqLQewnkYD0SCDJxoCd7jrwTlGNGoLN15XIid7PawhwKaHgwOcylvKIiCRVNOar onG3e0wj9St2AuzdXV2nJjicFcLv40Fc4C96Hod0WbyALYmS+BsPVvEMtpSVcK/CcIao cprYgm6DrHxD0j+kgG8SUJM1bzV7HjnJ7AaP7PIEysTX7C37IRJBhal/dFgc7WCdADRf R4nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pNz02KWu9MY5I80lfuOTL/E+wprRSQEPXW1e5pknxwY=; b=i49rexujmjp1hqGBGN6rSEotB55QBQ3EtatEnLseBgio6vDTgDMl2lFimThziubmBz fFcswfaCtBbDyAkZdfi70vewlfyavSivon/qfGRP/ufRKELIYfMfwAnIhYGzPRX8oxjL eOfp0kvg3B4pjsZCAce3O+MGA1UHGvAbHgGmxefNzcfOTb/9g1RQfujAQgwicLf9Ow7o jprjrfykhy6nd23BFzR8IyX4xhS3lug8rhxBQ4kvbbCEiQ0PyTZXLA8lQg1N1Es7D3WS MAsWBbuMRyRz51ev/B4INyqFxtgG1VEPRc9kVFDj2piF/bsT0ToaeUCG0estY63CPtSF Ib1Q== X-Gm-Message-State: APjAAAX4rY+YIqgaNGViHfR+SyHpvBf+0opuXRoYQydOXOHZwIuBQfHR wZorMlS4R1AbAhTlqi3ozLKEiNTq X-Google-Smtp-Source: APXvYqyHzbOMcevs0c2JmzNB3vHwDpD6DmONVKwFGbg/CdT+koXCHeBuZlXg70BGs1roHc2QkuFnsw== X-Received: by 2002:a17:902:184:: with SMTP id b4mr60417396plb.2.1558486173853; Tue, 21 May 2019 17:49:33 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:33 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 15/21] lpfc: Fix oops when driver is loaded with 1 interrupt vector Date: Tue, 21 May 2019 17:49:05 -0700 Message-Id: <20190522004911.573-16-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver was coded expecting enough hardware queues and interrupt vectors such that at least there was one per socket. In the case where there were fewer than sockets, cpus were left unassigned thus null pointers. Rework the affinity mappings. Map settings for the cpu's that are in the irq cpu mask. For each cpu not in the mask, map to another cpu that does have a mask. Choice of the "other" cpu will attempt to map to the same cpu but differing hyperthread, or cpu within in same core, or cpu within same socket, or finally cpu in the base socket. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_attr.c | 27 +++++--- drivers/scsi/lpfc/lpfc_init.c | 143 +++++++++++++++++++++++++++++++++++++++--- drivers/scsi/lpfc/lpfc_sli4.h | 4 +- 3 files changed, 155 insertions(+), 19 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index aabd42c4c6f8..58f26e5f3a59 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -5236,35 +5236,44 @@ lpfc_fcp_cpu_map_show(struct device *dev, struct device_attribute *attr, len += scnprintf( buf + len, PAGE_SIZE - len, "CPU %02d hdwq None " - "physid %d coreid %d ht %d\n", + "physid %d coreid %d ht %d ua %d\n", phba->sli4_hba.curr_disp_cpu, - cpup->phys_id, - cpup->core_id, cpup->hyper); + cpup->phys_id, cpup->core_id, + (cpup->flag & LPFC_CPU_MAP_HYPER), + (cpup->flag & LPFC_CPU_MAP_UNASSIGN)); else len += scnprintf( buf + len, PAGE_SIZE - len, "CPU %02d EQ %04d hdwq %04d " - "physid %d coreid %d ht %d\n", + "physid %d coreid %d ht %d ua %d\n", phba->sli4_hba.curr_disp_cpu, cpup->eq, cpup->hdwq, cpup->phys_id, - cpup->core_id, cpup->hyper); + cpup->core_id, + (cpup->flag & LPFC_CPU_MAP_HYPER), + (cpup->flag & LPFC_CPU_MAP_UNASSIGN)); } else { if (cpup->hdwq == LPFC_VECTOR_MAP_EMPTY) len += scnprintf( buf + len, PAGE_SIZE - len, "CPU %02d hdwq None " - "physid %d coreid %d ht %d IRQ %d\n", + "physid %d coreid %d ht %d ua %d IRQ %d\n", phba->sli4_hba.curr_disp_cpu, cpup->phys_id, - cpup->core_id, cpup->hyper, cpup->irq); + cpup->core_id, + (cpup->flag & LPFC_CPU_MAP_HYPER), + (cpup->flag & LPFC_CPU_MAP_UNASSIGN), + cpup->irq); else len += scnprintf( buf + len, PAGE_SIZE - len, "CPU %02d EQ %04d hdwq %04d " - "physid %d coreid %d ht %d IRQ %d\n", + "physid %d coreid %d ht %d ua %d IRQ %d\n", phba->sli4_hba.curr_disp_cpu, cpup->eq, cpup->hdwq, cpup->phys_id, - cpup->core_id, cpup->hyper, cpup->irq); + cpup->core_id, + (cpup->flag & LPFC_CPU_MAP_HYPER), + (cpup->flag & LPFC_CPU_MAP_UNASSIGN), + cpup->irq); } phba->sli4_hba.curr_disp_cpu++; diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 73b77aaf7135..021b01561597 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -10653,24 +10653,31 @@ lpfc_find_hyper(struct lpfc_hba *phba, int cpu, static void lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) { - int i, cpu, idx; + int i, cpu, idx, new_cpu, start_cpu, first_cpu; int max_phys_id, min_phys_id; int max_core_id, min_core_id; struct lpfc_vector_map_info *cpup; + struct lpfc_vector_map_info *new_cpup; const struct cpumask *maskp; #ifdef CONFIG_X86 struct cpuinfo_x86 *cpuinfo; #endif /* Init cpu_map array */ - memset(phba->sli4_hba.cpu_map, 0xff, - (sizeof(struct lpfc_vector_map_info) * - phba->sli4_hba.num_possible_cpu)); + for_each_possible_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + cpup->phys_id = LPFC_VECTOR_MAP_EMPTY; + cpup->core_id = LPFC_VECTOR_MAP_EMPTY; + cpup->hdwq = LPFC_VECTOR_MAP_EMPTY; + cpup->eq = LPFC_VECTOR_MAP_EMPTY; + cpup->irq = LPFC_VECTOR_MAP_EMPTY; + cpup->flag = 0; + } max_phys_id = 0; - min_phys_id = 0xffff; + min_phys_id = LPFC_VECTOR_MAP_EMPTY; max_core_id = 0; - min_core_id = 0xffff; + min_core_id = LPFC_VECTOR_MAP_EMPTY; /* Update CPU map with physical id and core id of each CPU */ for_each_present_cpu(cpu) { @@ -10679,13 +10686,12 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) cpuinfo = &cpu_data(cpu); cpup->phys_id = cpuinfo->phys_proc_id; cpup->core_id = cpuinfo->cpu_core_id; - cpup->hyper = lpfc_find_hyper(phba, cpu, - cpup->phys_id, cpup->core_id); + if (lpfc_find_hyper(phba, cpu, cpup->phys_id, cpup->core_id)) + cpup->flag |= LPFC_CPU_MAP_HYPER; #else /* No distinction between CPUs for other platforms */ cpup->phys_id = 0; cpup->core_id = cpu; - cpup->hyper = 0; #endif lpfc_printf_log(phba, KERN_INFO, LOG_INIT, @@ -10711,6 +10717,12 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) eqi->icnt = 0; } + /* This loop sets up all CPUs that are affinitized with a + * irq vector assigned to the driver. All affinitized CPUs + * will get a link to that vectors IRQ and EQ. For now we + * are assuming all CPUs using the same EQ will all share + * the same hardware queue. + */ for (idx = 0; idx < phba->cfg_irq_chann; idx++) { maskp = pci_irq_get_affinity(phba->pcidev, idx); if (!maskp) @@ -10728,6 +10740,119 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) cpu, cpup->hdwq, cpup->irq); } } + + /* After looking at each irq vector assigned to this pcidev, its + * possible to see that not ALL CPUs have been accounted for. + * Next we will set any unassigned cpu map entries to a IRQ + * on the same phys_id + */ + first_cpu = cpumask_first(cpu_present_mask); + start_cpu = first_cpu; + + for_each_present_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Is this CPU entry unassigned */ + if (cpup->eq == LPFC_VECTOR_MAP_EMPTY) { + /* Mark CPU as IRQ not assigned by the kernel */ + cpup->flag |= LPFC_CPU_MAP_UNASSIGN; + + /* If so, find a new_cpup thats on the the same + * phys_id as cpup. start_cpu will start where we + * left off so all unassigned entries don't get assgined + * the IRQ of the first entry. + */ + new_cpu = start_cpu; + for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) { + new_cpup = &phba->sli4_hba.cpu_map[new_cpu]; + if (!(new_cpup->flag & LPFC_CPU_MAP_UNASSIGN) && + (new_cpup->irq != LPFC_VECTOR_MAP_EMPTY) && + (new_cpup->phys_id == cpup->phys_id)) + goto found_same; + new_cpu = cpumask_next( + new_cpu, cpu_present_mask); + if (new_cpu == nr_cpumask_bits) + new_cpu = first_cpu; + } + /* At this point, we leave the CPU as unassigned */ + continue; +found_same: + /* We found a matching phys_id, so copy the IRQ info */ + cpup->eq = new_cpup->eq; + cpup->hdwq = new_cpup->hdwq; + cpup->irq = new_cpup->irq; + + /* Bump start_cpu to the next slot to minmize the + * chance of having multiple unassigned CPU entries + * selecting the same IRQ. + */ + start_cpu = cpumask_next(new_cpu, cpu_present_mask); + if (start_cpu == nr_cpumask_bits) + start_cpu = first_cpu; + + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "3337 Set Affinity: CPU %d " + "hdwq %d irq %d from id %d same " + "phys_id (%d)\n", + cpu, cpup->hdwq, cpup->irq, + new_cpu, cpup->phys_id); + } + } + + /* Set any unassigned cpu map entries to a IRQ on any phys_id */ + start_cpu = first_cpu; + + for_each_present_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Is this entry unassigned */ + if (cpup->eq == LPFC_VECTOR_MAP_EMPTY) { + /* Mark it as IRQ not assigned by the kernel */ + cpup->flag |= LPFC_CPU_MAP_UNASSIGN; + + /* If so, find a new_cpup thats on any phys_id + * as the cpup. start_cpu will start where we + * left off so all unassigned entries don't get + * assigned the IRQ of the first entry. + */ + new_cpu = start_cpu; + for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) { + new_cpup = &phba->sli4_hba.cpu_map[new_cpu]; + if (!(new_cpup->flag & LPFC_CPU_MAP_UNASSIGN) && + (new_cpup->irq != LPFC_VECTOR_MAP_EMPTY)) + goto found_any; + new_cpu = cpumask_next( + new_cpu, cpu_present_mask); + if (new_cpu == nr_cpumask_bits) + new_cpu = first_cpu; + } + /* We should never leave an entry unassigned */ + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "3339 Set Affinity: CPU %d " + "hdwq %d irq %d UNASSIGNED\n", + cpu, cpup->hdwq, cpup->irq); + continue; +found_any: + /* We found an available entry, copy the IRQ info */ + cpup->eq = new_cpup->eq; + cpup->hdwq = new_cpup->hdwq; + cpup->irq = new_cpup->irq; + + /* Bump start_cpu to the next slot to minmize the + * chance of having multiple unassigned CPU entries + * selecting the same IRQ. + */ + start_cpu = cpumask_next(new_cpu, cpu_present_mask); + if (start_cpu == nr_cpumask_bits) + start_cpu = first_cpu; + + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "3338 Set Affinity: CPU %d " + "hdwq %d irq %d from id %d (%d/%d)\n", + cpu, cpup->hdwq, cpup->irq, new_cpu, + new_cpup->phys_id, new_cpup->core_id); + } + } return; } diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 8b28a55c73bb..69c6dba77dce 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -549,7 +549,9 @@ struct lpfc_vector_map_info { uint16_t irq; uint16_t eq; uint16_t hdwq; - uint16_t hyper; + uint16_t flag; +#define LPFC_CPU_MAP_HYPER 0x1 +#define LPFC_CPU_MAP_UNASSIGN 0x2 }; #define LPFC_VECTOR_MAP_EMPTY 0xffff From patchwork Wed May 22 00:49:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954645 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB1E614B6 for ; Wed, 22 May 2019 00:49:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CAA6028875 for ; Wed, 22 May 2019 00:49:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE9E128B27; Wed, 22 May 2019 00:49:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0714128875 for ; Wed, 22 May 2019 00:49:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728228AbfEVAth (ORCPT ); Tue, 21 May 2019 20:49:37 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:39616 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728203AbfEVAtg (ORCPT ); Tue, 21 May 2019 20:49:36 -0400 Received: by mail-pf1-f193.google.com with SMTP id z26so343819pfg.6 for ; Tue, 21 May 2019 17:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AB2LPDeIKZ9AUorIfwFGUJw/j2iW9D3eupoHtt/9gyU=; b=fpHp18ANP+uGKdaMYf/xDZy0pxrkLS4eADh5CAmgXPccSlWuURBqmSf1RY09dJghUf vIFd10kg1YBfDTWfyW25JdDsAnGNi5Ulk8VKQx3W/DXFgFd5LYsGS/z8i62XUc190mBw K1AmvtrK8gAekd847kGeYexCOXLTh5UiMNydZVXG9Gq+zZwc9bnHv2S/YqoszPAoqFex s4HRdt6exQtr+jAtgXi9D8UlKGaifNyHzKauDsf6w3L3n61X5rNd2xm6JdCKCE5HwLts oLlMJ4UCVG2qj/MUrgYkQOtUqwCR+4B3sPA+iuEhuXSGxS3rcTvP+Z5V/PNqh4YqoeF0 pQhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AB2LPDeIKZ9AUorIfwFGUJw/j2iW9D3eupoHtt/9gyU=; b=X2yTJHrPvKVMzR+3iI49qseLfTA1MCODqAm8seRFaOBM3af8XkNfALKZUz3mZpVhTY s0IPnePNuq4Qu/GWmgLM8+OkDdZfNQShbLc+5qCqeRhzbOhqHQ8ozgQZ8xodcCh64ekv TV/dVHbMWs+1oLrz6jqBjcKf22VLJVdH4D6lNjj56fkO2GXPrEkH3QklCfYOPQxKJT/n cfPHJLET1m0A7Gw37AniFJMXSP+dmX+r1dQOCLiT3kOE+tIRid+uGb4LadQsGZiT0HnQ w5dJq4wKe0D9K8du2xt+LPbcP/jDlyTR5CyRilUMP61L8DPza41J7I+dKzIKfcUcw5Uj izug== X-Gm-Message-State: APjAAAV9vduGHp3E0BgxzsU6HjgJ9vTjGHNVc7IqAyglPmKCeSgcaCbF WT24f2BpjTt94G1rytS+KzyHSEIy X-Google-Smtp-Source: APXvYqxbODzZ5YLs1OcUtucXDNB6mNP11ii+F9FfruK/7WyKTPk8ErEXDZ2rOvqWzFgqSZIrWeOptA== X-Received: by 2002:a62:d286:: with SMTP id c128mr92664650pfg.159.1558486174716; Tue, 21 May 2019 17:49:34 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:34 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 16/21] lpfc: Fix poor use of hardware queues if fewer irq vectors Date: Tue, 21 May 2019 17:49:06 -0700 Message-Id: <20190522004911.573-17-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While fixing the resources per socket, realized the driver was not using hardware queues (up to 1 per cpu) if there were fewer interrupt vectors. The driver was only using the hardware queue assigned to the cpu with the vector. Rework the affinity map check to use the additional hardware queue elements that had been allocated. If the cpu count exceeds the hardware queue count - share, but choose what is shared with by: hyperthread peer, core peer, socket peer, or finally similar cpu in a different socket. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_attr.c | 3 +- drivers/scsi/lpfc/lpfc_init.c | 321 +++++++++++++++++++++++++++++------------- drivers/scsi/lpfc/lpfc_sli.c | 42 +++--- drivers/scsi/lpfc/lpfc_sli4.h | 2 + 4 files changed, 255 insertions(+), 113 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 58f26e5f3a59..065c526218b2 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -5115,7 +5115,8 @@ lpfc_cq_max_proc_limit_store(struct device *dev, struct device_attribute *attr, /* set the values on the cq's */ for (i = 0; i < phba->cfg_irq_chann; i++) { - eq = phba->sli4_hba.hdwq[i].hba_eq; + /* Get the EQ corresponding to the IRQ vector */ + eq = phba->sli4_hba.hba_eq_hdl[i].eq; if (!eq) continue; diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 021b01561597..416f0fb155f5 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -93,7 +93,6 @@ static void lpfc_sli4_cq_event_release_all(struct lpfc_hba *); static void lpfc_sli4_disable_intr(struct lpfc_hba *); static uint32_t lpfc_sli4_enable_intr(struct lpfc_hba *, uint32_t); static void lpfc_sli4_oas_verify(struct lpfc_hba *phba); -static uint16_t lpfc_find_eq_handle(struct lpfc_hba *, uint16_t); static uint16_t lpfc_find_cpu_handle(struct lpfc_hba *, uint16_t, int); static struct scsi_transport_template *lpfc_transport_template = NULL; @@ -1274,8 +1273,10 @@ lpfc_hb_eq_delay_work(struct work_struct *work) if (!eqcnt) goto requeue; + /* Loop thru all IRQ vectors */ for (i = 0; i < phba->cfg_irq_chann; i++) { - eq = phba->sli4_hba.hdwq[i].hba_eq; + /* Get the EQ corresponding to the IRQ vector */ + eq = phba->sli4_hba.hba_eq_hdl[i].eq; if (eq && eqcnt[eq->last_cpu] < 2) eqcnt[eq->last_cpu]++; continue; @@ -8748,8 +8749,10 @@ int lpfc_sli4_queue_create(struct lpfc_hba *phba) { struct lpfc_queue *qdesc; - int idx, eqidx, cpu; + int idx, cpu, eqcpu; struct lpfc_sli4_hdw_queue *qp; + struct lpfc_vector_map_info *cpup; + struct lpfc_vector_map_info *eqcpup; struct lpfc_eq_intr_info *eqi; /* @@ -8834,40 +8837,60 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) INIT_LIST_HEAD(&phba->sli4_hba.lpfc_wq_list); /* Create HBA Event Queues (EQs) */ - for (idx = 0; idx < phba->cfg_hdw_queue; idx++) { - /* determine EQ affinity */ - eqidx = lpfc_find_eq_handle(phba, idx); - cpu = lpfc_find_cpu_handle(phba, eqidx, LPFC_FIND_BY_EQ); - /* - * If there are more Hardware Queues than available - * EQs, multiple Hardware Queues may share a common EQ. + for_each_present_cpu(cpu) { + /* We only want to create 1 EQ per vector, even though + * multiple CPUs might be using that vector. so only + * selects the CPUs that are LPFC_CPU_FIRST_IRQ. */ - if (idx >= phba->cfg_irq_chann) { - /* Share an existing EQ */ - phba->sli4_hba.hdwq[idx].hba_eq = - phba->sli4_hba.hdwq[eqidx].hba_eq; + cpup = &phba->sli4_hba.cpu_map[cpu]; + if (!(cpup->flag & LPFC_CPU_FIRST_IRQ)) continue; - } - /* Create an EQ */ + + /* Get a ptr to the Hardware Queue associated with this CPU */ + qp = &phba->sli4_hba.hdwq[cpup->hdwq]; + + /* Allocate an EQ */ qdesc = lpfc_sli4_queue_alloc(phba, LPFC_DEFAULT_PAGE_SIZE, phba->sli4_hba.eq_esize, phba->sli4_hba.eq_ecount, cpu); if (!qdesc) { lpfc_printf_log(phba, KERN_ERR, LOG_INIT, - "0497 Failed allocate EQ (%d)\n", idx); + "0497 Failed allocate EQ (%d)\n", + cpup->hdwq); goto out_error; } qdesc->qe_valid = 1; - qdesc->hdwq = idx; - - /* Save the CPU this EQ is affinitised to */ - qdesc->chann = cpu; - phba->sli4_hba.hdwq[idx].hba_eq = qdesc; + qdesc->hdwq = cpup->hdwq; + qdesc->chann = cpu; /* First CPU this EQ is affinitised to */ qdesc->last_cpu = qdesc->chann; + + /* Save the allocated EQ in the Hardware Queue */ + qp->hba_eq = qdesc; + eqi = per_cpu_ptr(phba->sli4_hba.eq_info, qdesc->last_cpu); list_add(&qdesc->cpu_list, &eqi->list); } + /* Now we need to populate the other Hardware Queues, that share + * an IRQ vector, with the associated EQ ptr. + */ + for_each_present_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Check for EQ already allocated in previous loop */ + if (cpup->flag & LPFC_CPU_FIRST_IRQ) + continue; + + /* Check for multiple CPUs per hdwq */ + qp = &phba->sli4_hba.hdwq[cpup->hdwq]; + if (qp->hba_eq) + continue; + + /* We need to share an EQ for this hdwq */ + eqcpu = lpfc_find_cpu_handle(phba, cpup->eq, LPFC_FIND_BY_EQ); + eqcpup = &phba->sli4_hba.cpu_map[eqcpu]; + qp->hba_eq = phba->sli4_hba.hdwq[eqcpup->hdwq].hba_eq; + } /* Allocate SCSI SLI4 CQ/WQs */ for (idx = 0; idx < phba->cfg_hdw_queue; idx++) { @@ -9130,23 +9153,31 @@ static inline void lpfc_sli4_release_hdwq(struct lpfc_hba *phba) { struct lpfc_sli4_hdw_queue *hdwq; + struct lpfc_queue *eq; uint32_t idx; hdwq = phba->sli4_hba.hdwq; - for (idx = 0; idx < phba->cfg_hdw_queue; idx++) { - if (idx < phba->cfg_irq_chann) - lpfc_sli4_queue_free(hdwq[idx].hba_eq); - hdwq[idx].hba_eq = NULL; + /* Loop thru all Hardware Queues */ + for (idx = 0; idx < phba->cfg_hdw_queue; idx++) { + /* Free the CQ/WQ corresponding to the Hardware Queue */ lpfc_sli4_queue_free(hdwq[idx].fcp_cq); lpfc_sli4_queue_free(hdwq[idx].nvme_cq); lpfc_sli4_queue_free(hdwq[idx].fcp_wq); lpfc_sli4_queue_free(hdwq[idx].nvme_wq); + hdwq[idx].hba_eq = NULL; hdwq[idx].fcp_cq = NULL; hdwq[idx].nvme_cq = NULL; hdwq[idx].fcp_wq = NULL; hdwq[idx].nvme_wq = NULL; } + /* Loop thru all IRQ vectors */ + for (idx = 0; idx < phba->cfg_irq_chann; idx++) { + /* Free the EQ corresponding to the IRQ vector */ + eq = phba->sli4_hba.hba_eq_hdl[idx].eq; + lpfc_sli4_queue_free(eq); + phba->sli4_hba.hba_eq_hdl[idx].eq = NULL; + } } /** @@ -9330,10 +9361,13 @@ lpfc_setup_cq_lookup(struct lpfc_hba *phba) qp = phba->sli4_hba.hdwq; memset(phba->sli4_hba.cq_lookup, 0, (sizeof(struct lpfc_queue *) * (phba->sli4_hba.cq_max + 1))); + /* Loop thru all IRQ vectors */ for (qidx = 0; qidx < phba->cfg_irq_chann; qidx++) { - eq = qp[qidx].hba_eq; + /* Get the EQ corresponding to the IRQ vector */ + eq = phba->sli4_hba.hba_eq_hdl[qidx].eq; if (!eq) continue; + /* Loop through all CQs associated with that EQ */ list_for_each_entry(childq, &eq->child_list, list) { if (childq->queue_id > phba->sli4_hba.cq_max) continue; @@ -9362,9 +9396,10 @@ lpfc_sli4_queue_setup(struct lpfc_hba *phba) { uint32_t shdr_status, shdr_add_status; union lpfc_sli4_cfg_shdr *shdr; + struct lpfc_vector_map_info *cpup; struct lpfc_sli4_hdw_queue *qp; LPFC_MBOXQ_t *mboxq; - int qidx; + int qidx, cpu; uint32_t length, usdelay; int rc = -ENOMEM; @@ -9425,32 +9460,55 @@ lpfc_sli4_queue_setup(struct lpfc_hba *phba) rc = -ENOMEM; goto out_error; } + + /* Loop thru all IRQ vectors */ for (qidx = 0; qidx < phba->cfg_irq_chann; qidx++) { - if (!qp[qidx].hba_eq) { - lpfc_printf_log(phba, KERN_ERR, LOG_INIT, - "0522 Fast-path EQ (%d) not " - "allocated\n", qidx); - rc = -ENOMEM; - goto out_destroy; - } - rc = lpfc_eq_create(phba, qp[qidx].hba_eq, - phba->cfg_fcp_imax); - if (rc) { - lpfc_printf_log(phba, KERN_ERR, LOG_INIT, - "0523 Failed setup of fast-path EQ " - "(%d), rc = 0x%x\n", qidx, - (uint32_t)rc); - goto out_destroy; + /* Create HBA Event Queues (EQs) in order */ + for_each_present_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Look for the CPU thats using that vector with + * LPFC_CPU_FIRST_IRQ set. + */ + if (!(cpup->flag & LPFC_CPU_FIRST_IRQ)) + continue; + if (qidx != cpup->eq) + continue; + + /* Create an EQ for that vector */ + rc = lpfc_eq_create(phba, qp[cpup->hdwq].hba_eq, + phba->cfg_fcp_imax); + if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "0523 Failed setup of fast-path" + " EQ (%d), rc = 0x%x\n", + cpup->eq, (uint32_t)rc); + goto out_destroy; + } + + /* Save the EQ for that vector in the hba_eq_hdl */ + phba->sli4_hba.hba_eq_hdl[cpup->eq].eq = + qp[cpup->hdwq].hba_eq; + + lpfc_printf_log(phba, KERN_INFO, LOG_INIT, + "2584 HBA EQ setup: queue[%d]-id=%d\n", + cpup->eq, + qp[cpup->hdwq].hba_eq->queue_id); } - lpfc_printf_log(phba, KERN_INFO, LOG_INIT, - "2584 HBA EQ setup: queue[%d]-id=%d\n", qidx, - qp[qidx].hba_eq->queue_id); } + /* Loop thru all Hardware Queues */ if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) { for (qidx = 0; qidx < phba->cfg_hdw_queue; qidx++) { + cpu = lpfc_find_cpu_handle(phba, qidx, + LPFC_FIND_BY_HDWQ); + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Create the CQ/WQ corresponding to the + * Hardware Queue + */ rc = lpfc_create_wq_cq(phba, - qp[qidx].hba_eq, + phba->sli4_hba.hdwq[cpup->hdwq].hba_eq, qp[qidx].nvme_cq, qp[qidx].nvme_wq, &phba->sli4_hba.hdwq[qidx].nvme_cq_map, @@ -9466,8 +9524,12 @@ lpfc_sli4_queue_setup(struct lpfc_hba *phba) } for (qidx = 0; qidx < phba->cfg_hdw_queue; qidx++) { + cpu = lpfc_find_cpu_handle(phba, qidx, LPFC_FIND_BY_HDWQ); + cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* Create the CQ/WQ corresponding to the Hardware Queue */ rc = lpfc_create_wq_cq(phba, - qp[qidx].hba_eq, + phba->sli4_hba.hdwq[cpup->hdwq].hba_eq, qp[qidx].fcp_cq, qp[qidx].fcp_wq, &phba->sli4_hba.hdwq[qidx].fcp_cq_map, @@ -9719,6 +9781,7 @@ void lpfc_sli4_queue_unset(struct lpfc_hba *phba) { struct lpfc_sli4_hdw_queue *qp; + struct lpfc_queue *eq; int qidx; /* Unset mailbox command work queue */ @@ -9770,14 +9833,20 @@ lpfc_sli4_queue_unset(struct lpfc_hba *phba) /* Unset fast-path SLI4 queues */ if (phba->sli4_hba.hdwq) { + /* Loop thru all Hardware Queues */ for (qidx = 0; qidx < phba->cfg_hdw_queue; qidx++) { + /* Destroy the CQ/WQ corresponding to Hardware Queue */ qp = &phba->sli4_hba.hdwq[qidx]; lpfc_wq_destroy(phba, qp->fcp_wq); lpfc_wq_destroy(phba, qp->nvme_wq); lpfc_cq_destroy(phba, qp->fcp_cq); lpfc_cq_destroy(phba, qp->nvme_cq); - if (qidx < phba->cfg_irq_chann) - lpfc_eq_destroy(phba, qp->hba_eq); + } + /* Loop thru all IRQ vectors */ + for (qidx = 0; qidx < phba->cfg_irq_chann; qidx++) { + /* Destroy the EQ corresponding to the IRQ vector */ + eq = phba->sli4_hba.hba_eq_hdl[qidx].eq; + lpfc_eq_destroy(phba, eq); } } @@ -10567,11 +10636,12 @@ lpfc_sli_disable_intr(struct lpfc_hba *phba) } /** - * lpfc_find_cpu_handle - Find the CPU that corresponds to the specified EQ + * lpfc_find_cpu_handle - Find the CPU that corresponds to the specified Queue * @phba: pointer to lpfc hba data structure. * @id: EQ vector index or Hardware Queue index * @match: LPFC_FIND_BY_EQ = match by EQ * LPFC_FIND_BY_HDWQ = match by Hardware Queue + * Return the CPU that matches the selection criteria */ static uint16_t lpfc_find_cpu_handle(struct lpfc_hba *phba, uint16_t id, int match) @@ -10579,40 +10649,27 @@ lpfc_find_cpu_handle(struct lpfc_hba *phba, uint16_t id, int match) struct lpfc_vector_map_info *cpup; int cpu; - /* Find the desired phys_id for the specified EQ */ + /* Loop through all CPUs */ for_each_present_cpu(cpu) { cpup = &phba->sli4_hba.cpu_map[cpu]; + + /* If we are matching by EQ, there may be multiple CPUs using + * using the same vector, so select the one with + * LPFC_CPU_FIRST_IRQ set. + */ if ((match == LPFC_FIND_BY_EQ) && + (cpup->flag & LPFC_CPU_FIRST_IRQ) && (cpup->irq != LPFC_VECTOR_MAP_EMPTY) && (cpup->eq == id)) return cpu; + + /* If matching by HDWQ, select the first CPU that matches */ if ((match == LPFC_FIND_BY_HDWQ) && (cpup->hdwq == id)) return cpu; } return 0; } -/** - * lpfc_find_eq_handle - Find the EQ that corresponds to the specified - * Hardware Queue - * @phba: pointer to lpfc hba data structure. - * @hdwq: Hardware Queue index - */ -static uint16_t -lpfc_find_eq_handle(struct lpfc_hba *phba, uint16_t hdwq) -{ - struct lpfc_vector_map_info *cpup; - int cpu; - - /* Find the desired phys_id for the specified EQ */ - for_each_present_cpu(cpu) { - cpup = &phba->sli4_hba.cpu_map[cpu]; - if (cpup->hdwq == hdwq) - return cpup->eq; - } - return 0; -} - #ifdef CONFIG_X86 /** * lpfc_find_hyper - Determine if the CPU map entry is hyper-threaded @@ -10719,32 +10776,40 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) /* This loop sets up all CPUs that are affinitized with a * irq vector assigned to the driver. All affinitized CPUs - * will get a link to that vectors IRQ and EQ. For now we - * are assuming all CPUs using the same EQ will all share - * the same hardware queue. + * will get a link to that vectors IRQ and EQ. */ for (idx = 0; idx < phba->cfg_irq_chann; idx++) { + /* Get a CPU mask for all CPUs affinitized to this vector */ maskp = pci_irq_get_affinity(phba->pcidev, idx); if (!maskp) continue; + i = 0; + /* Loop through all CPUs associated with vector idx */ for_each_cpu_and(cpu, maskp, cpu_present_mask) { + /* Set the EQ index and IRQ for that vector */ cpup = &phba->sli4_hba.cpu_map[cpu]; cpup->eq = idx; - cpup->hdwq = idx; cpup->irq = pci_irq_vector(phba->pcidev, idx); - lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + lpfc_printf_log(phba, KERN_INFO, LOG_INIT, "3336 Set Affinity: CPU %d " - "hdwq %d irq %d\n", - cpu, cpup->hdwq, cpup->irq); + "irq %d eq %d\n", + cpu, cpup->irq, cpup->eq); + + /* If this is the first CPU thats assigned to this + * vector, set LPFC_CPU_FIRST_IRQ. + */ + if (!i) + cpup->flag |= LPFC_CPU_FIRST_IRQ; + i++; } } /* After looking at each irq vector assigned to this pcidev, its * possible to see that not ALL CPUs have been accounted for. - * Next we will set any unassigned cpu map entries to a IRQ - * on the same phys_id + * Next we will set any unassigned (unaffinitized) cpu map + * entries to a IRQ on the same phys_id. */ first_cpu = cpumask_first(cpu_present_mask); start_cpu = first_cpu; @@ -10757,7 +10822,7 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) /* Mark CPU as IRQ not assigned by the kernel */ cpup->flag |= LPFC_CPU_MAP_UNASSIGN; - /* If so, find a new_cpup thats on the the same + /* If so, find a new_cpup thats on the the SAME * phys_id as cpup. start_cpu will start where we * left off so all unassigned entries don't get assgined * the IRQ of the first entry. @@ -10779,7 +10844,6 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) found_same: /* We found a matching phys_id, so copy the IRQ info */ cpup->eq = new_cpup->eq; - cpup->hdwq = new_cpup->hdwq; cpup->irq = new_cpup->irq; /* Bump start_cpu to the next slot to minmize the @@ -10790,12 +10854,11 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) if (start_cpu == nr_cpumask_bits) start_cpu = first_cpu; - lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + lpfc_printf_log(phba, KERN_INFO, LOG_INIT, "3337 Set Affinity: CPU %d " - "hdwq %d irq %d from id %d same " + "irq %d from id %d same " "phys_id (%d)\n", - cpu, cpup->hdwq, cpup->irq, - new_cpu, cpup->phys_id); + cpu, cpup->irq, new_cpu, cpup->phys_id); } } @@ -10810,7 +10873,7 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) /* Mark it as IRQ not assigned by the kernel */ cpup->flag |= LPFC_CPU_MAP_UNASSIGN; - /* If so, find a new_cpup thats on any phys_id + /* If so, find a new_cpup thats on ANY phys_id * as the cpup. start_cpu will start where we * left off so all unassigned entries don't get * assigned the IRQ of the first entry. @@ -10829,13 +10892,12 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) /* We should never leave an entry unassigned */ lpfc_printf_log(phba, KERN_ERR, LOG_INIT, "3339 Set Affinity: CPU %d " - "hdwq %d irq %d UNASSIGNED\n", - cpu, cpup->hdwq, cpup->irq); + "irq %d UNASSIGNED\n", + cpup->hdwq, cpup->irq); continue; found_any: /* We found an available entry, copy the IRQ info */ cpup->eq = new_cpup->eq; - cpup->hdwq = new_cpup->hdwq; cpup->irq = new_cpup->irq; /* Bump start_cpu to the next slot to minmize the @@ -10846,13 +10908,82 @@ lpfc_cpu_affinity_check(struct lpfc_hba *phba, int vectors) if (start_cpu == nr_cpumask_bits) start_cpu = first_cpu; - lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + lpfc_printf_log(phba, KERN_INFO, LOG_INIT, "3338 Set Affinity: CPU %d " - "hdwq %d irq %d from id %d (%d/%d)\n", - cpu, cpup->hdwq, cpup->irq, new_cpu, + "irq %d from id %d (%d/%d)\n", + cpu, cpup->irq, new_cpu, new_cpup->phys_id, new_cpup->core_id); } } + + /* Finally we need to associate a hdwq with each cpu_map entry + * This will be 1 to 1 - hdwq to cpu, unless there are less + * hardware queues then CPUs. For that case we will just round-robin + * the available hardware queues as they get assigned to CPUs. + */ + idx = 0; + start_cpu = 0; + for_each_present_cpu(cpu) { + cpup = &phba->sli4_hba.cpu_map[cpu]; + if (idx >= phba->cfg_hdw_queue) { + /* We need to reuse a Hardware Queue for another CPU, + * so be smart about it and pick one that has its + * IRQ/EQ mapped to the same phys_id (CPU package). + * and core_id. + */ + new_cpu = start_cpu; + for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) { + new_cpup = &phba->sli4_hba.cpu_map[new_cpu]; + if ((new_cpup->hdwq != LPFC_VECTOR_MAP_EMPTY) && + (new_cpup->phys_id == cpup->phys_id) && + (new_cpup->core_id == cpup->core_id)) + goto found_hdwq; + new_cpu = cpumask_next( + new_cpu, cpu_present_mask); + if (new_cpu == nr_cpumask_bits) + new_cpu = first_cpu; + } + + /* If we can't match both phys_id and core_id, + * settle for just a phys_id match. + */ + new_cpu = start_cpu; + for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) { + new_cpup = &phba->sli4_hba.cpu_map[new_cpu]; + if ((new_cpup->hdwq != LPFC_VECTOR_MAP_EMPTY) && + (new_cpup->phys_id == cpup->phys_id)) + goto found_hdwq; + new_cpu = cpumask_next( + new_cpu, cpu_present_mask); + if (new_cpu == nr_cpumask_bits) + new_cpu = first_cpu; + } + + /* Otherwise just round robin on cfg_hdw_queue */ + cpup->hdwq = idx % phba->cfg_hdw_queue; + goto logit; +found_hdwq: + /* We found an available entry, copy the IRQ info */ + start_cpu = cpumask_next(new_cpu, cpu_present_mask); + if (start_cpu == nr_cpumask_bits) + start_cpu = first_cpu; + cpup->hdwq = new_cpup->hdwq; + } else { + /* 1 to 1, CPU to hdwq */ + cpup->hdwq = idx; + } +logit: + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "3335 Set Affinity: CPU %d (phys %d core %d): " + "hdwq %d eq %d irq %d flg x%x\n", + cpu, cpup->phys_id, cpup->core_id, + cpup->hdwq, cpup->eq, cpup->irq, cpup->flag); + idx++; + } + + /* The cpu_map array will be used later during initialization + * when EQ / CQ / WQs are allocated and configured. + */ return; } diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 4432060f7315..d55259fc0af5 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -5548,6 +5548,7 @@ lpfc_sli4_arm_cqeq_intr(struct lpfc_hba *phba) int qidx; struct lpfc_sli4_hba *sli4_hba = &phba->sli4_hba; struct lpfc_sli4_hdw_queue *qp; + struct lpfc_queue *eq; sli4_hba->sli4_write_cq_db(phba, sli4_hba->mbx_cq, 0, LPFC_QUEUE_REARM); sli4_hba->sli4_write_cq_db(phba, sli4_hba->els_cq, 0, LPFC_QUEUE_REARM); @@ -5555,18 +5556,24 @@ lpfc_sli4_arm_cqeq_intr(struct lpfc_hba *phba) sli4_hba->sli4_write_cq_db(phba, sli4_hba->nvmels_cq, 0, LPFC_QUEUE_REARM); - qp = sli4_hba->hdwq; if (sli4_hba->hdwq) { + /* Loop thru all Hardware Queues */ for (qidx = 0; qidx < phba->cfg_hdw_queue; qidx++) { - sli4_hba->sli4_write_cq_db(phba, qp[qidx].fcp_cq, 0, + qp = &sli4_hba->hdwq[qidx]; + /* ARM the corresponding CQ */ + sli4_hba->sli4_write_cq_db(phba, qp->fcp_cq, 0, LPFC_QUEUE_REARM); - sli4_hba->sli4_write_cq_db(phba, qp[qidx].nvme_cq, 0, + sli4_hba->sli4_write_cq_db(phba, qp->nvme_cq, 0, LPFC_QUEUE_REARM); } - for (qidx = 0; qidx < phba->cfg_irq_chann; qidx++) - sli4_hba->sli4_write_eq_db(phba, qp[qidx].hba_eq, - 0, LPFC_QUEUE_REARM); + /* Loop thru all IRQ vectors */ + for (qidx = 0; qidx < phba->cfg_irq_chann; qidx++) { + eq = sli4_hba->hba_eq_hdl[qidx].eq; + /* ARM the corresponding EQ */ + sli4_hba->sli4_write_eq_db(phba, eq, + 0, LPFC_QUEUE_REARM); + } } if (phba->nvmet_support) { @@ -7858,20 +7865,22 @@ lpfc_sli4_process_missed_mbox_completions(struct lpfc_hba *phba) struct lpfc_sli4_hba *sli4_hba = &phba->sli4_hba; uint32_t eqidx; struct lpfc_queue *fpeq = NULL; + struct lpfc_queue *eq; bool mbox_pending; if (unlikely(!phba) || (phba->sli_rev != LPFC_SLI_REV4)) return false; - /* Find the eq associated with the mcq */ - - if (sli4_hba->hdwq) - for (eqidx = 0; eqidx < phba->cfg_irq_chann; eqidx++) - if (sli4_hba->hdwq[eqidx].hba_eq->queue_id == - sli4_hba->mbx_cq->assoc_qid) { - fpeq = sli4_hba->hdwq[eqidx].hba_eq; + /* Find the EQ associated with the mbox CQ */ + if (sli4_hba->hdwq) { + for (eqidx = 0; eqidx < phba->cfg_irq_chann; eqidx++) { + eq = phba->sli4_hba.hba_eq_hdl[eqidx].eq; + if (eq->queue_id == sli4_hba->mbx_cq->assoc_qid) { + fpeq = eq; break; } + } + } if (!fpeq) return false; @@ -14217,7 +14226,7 @@ lpfc_sli4_hba_intr_handler(int irq, void *dev_id) return IRQ_NONE; /* Get to the EQ struct associated with this vector */ - fpeq = phba->sli4_hba.hdwq[hba_eqidx].hba_eq; + fpeq = phba->sli4_hba.hba_eq_hdl[hba_eqidx].eq; if (unlikely(!fpeq)) return IRQ_NONE; @@ -14502,7 +14511,7 @@ lpfc_modify_hba_eq_delay(struct lpfc_hba *phba, uint32_t startq, /* set values by EQ_DELAY register if supported */ if (phba->sli.sli_flag & LPFC_SLI_USE_EQDR) { for (qidx = startq; qidx < phba->cfg_irq_chann; qidx++) { - eq = phba->sli4_hba.hdwq[qidx].hba_eq; + eq = phba->sli4_hba.hba_eq_hdl[qidx].eq; if (!eq) continue; @@ -14511,7 +14520,6 @@ lpfc_modify_hba_eq_delay(struct lpfc_hba *phba, uint32_t startq, if (++cnt >= numq) break; } - return; } @@ -14539,7 +14547,7 @@ lpfc_modify_hba_eq_delay(struct lpfc_hba *phba, uint32_t startq, dmult = LPFC_DMULT_MAX; for (qidx = startq; qidx < phba->cfg_irq_chann; qidx++) { - eq = phba->sli4_hba.hdwq[qidx].hba_eq; + eq = phba->sli4_hba.hba_eq_hdl[qidx].eq; if (!eq) continue; eq->q_mode = usdelay; diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 69c6dba77dce..3aeca387b22a 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -452,6 +452,7 @@ struct lpfc_hba_eq_hdl { uint32_t idx; char handler_name[LPFC_SLI4_HANDLER_NAME_SZ]; struct lpfc_hba *phba; + struct lpfc_queue *eq; }; /*BB Credit recovery value*/ @@ -552,6 +553,7 @@ struct lpfc_vector_map_info { uint16_t flag; #define LPFC_CPU_MAP_HYPER 0x1 #define LPFC_CPU_MAP_UNASSIGN 0x2 +#define LPFC_CPU_FIRST_IRQ 0x4 }; #define LPFC_VECTOR_MAP_EMPTY 0xffff From patchwork Wed May 22 00:49:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03CD414B6 for ; Wed, 22 May 2019 00:49:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA26B28895 for ; Wed, 22 May 2019 00:49:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DEC5428875; Wed, 22 May 2019 00:49:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8AE581FE6B for ; Wed, 22 May 2019 00:49:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728224AbfEVAtg (ORCPT ); Tue, 21 May 2019 20:49:36 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:46509 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728165AbfEVAtg (ORCPT ); Tue, 21 May 2019 20:49:36 -0400 Received: by mail-pl1-f196.google.com with SMTP id r18so159624pls.13 for ; Tue, 21 May 2019 17:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UPAcGN2LwzH3hCrG35HVFdEySeJoW3EL3RwTklAowHc=; b=f1+f/5RIpao1RWNcTw5enm8saYFulr9cL1POgRzpgiaxs0UeIoAGB+8HJWY2mUx1gv MEfXlY148aoEGeb+lJst4fNigNZjouGtdM4DLZvOanfZaL2i0YUFvNn4j4kfHVR20118 DdJChRtKstgI0oyLC/e74CFr+JBSTdrS49KkkidjW1/I1QJJum0l7KpKoy9VFhFszSE5 KlHUjp7R7Og6zb0DoW00UERj2kdL1+MQl6BrjTk4Mw5d1oIoxgF0jnv3vl3MlpfUwJwU k0aHrYK+zvog0HzsWQjotURDFtMjQgt3wtmb3H2i928ped4S9pyyC6HMoVZECRPFydxG t8WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UPAcGN2LwzH3hCrG35HVFdEySeJoW3EL3RwTklAowHc=; b=UpKAIty8G+wKzOD0gdpsZxTeNINxt+P66iD1H/0awgEmp8IMKv0i2Kaa40UUSbgK2u atR1qJso2+DZerNds4xq1LCeFpUjCb1h4bpuhUHHp7CSTGej91HEQ7KSI8/jHrvJaqdY PToCi4lrvsakhW1SvR3WPSfABiPYHTUeUFUnTfwFs3zFEsUCmQFFo3cHEES9ZDSQbD6G sRXGQp0a4wU/Hedrb10Uvpxh3NGlnZWzTuR24NxFW/bWL9HS2O//tPghJFUFgtFlxf7f HqpuirxD5+y3GNLvsTzO1zuy7CU5iX67RJVRdj1Sl8iM7Ila5DtegvHYJhWVTnGW1ac7 9ZLw== X-Gm-Message-State: APjAAAVHrmu1Hb+DeSzdGvlwD57OPTaxyZ+fNVAmU9Nfo9y5aWQYXY5U Vuyt4fysxk0nRqKSNDOQqyiYnt2U X-Google-Smtp-Source: APXvYqzyL8kTjGtcR++pPcs9ANuGnxDHT0tAGHOgrtlwfwi9WMj4o9ypMd4zkiIiErehXEddYM8iaA== X-Received: by 2002:a17:902:9a07:: with SMTP id v7mr87847826plp.180.1558486175438; Tue, 21 May 2019 17:49:35 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:35 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 17/21] lpfc: Fix fcp_rsp_len checking on lun reset Date: Tue, 21 May 2019 17:49:07 -0700 Message-Id: <20190522004911.573-18-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Issuing a LUN reset was resulting in a command failure which then escalated to a host reset. The FCP-4 spec allows fcp_rsp_len field to specify the number of valid bytes of FCP_RSP_INFO, and the value could be 4 or 8. The driver is allowing only a value of 8, thus it failed the command. Revise the driver to allow 4 or 8 Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_scsi.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index 5a5a9bbe6023..f9df800e7067 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -4798,7 +4798,12 @@ lpfc_check_fcp_rsp(struct lpfc_vport *vport, struct lpfc_io_buf *lpfc_cmd) rsp_info, rsp_len, rsp_info_code); - if ((fcprsp->rspStatus2&RSP_LEN_VALID) && (rsp_len == 8)) { + /* If FCP_RSP_LEN_VALID bit is one, then the FCP_RSP_LEN + * field specifies the number of valid bytes of FCP_RSP_INFO. + * The FCP_RSP_LEN field shall be set to 0x04 or 0x08 + */ + if ((fcprsp->rspStatus2 & RSP_LEN_VALID) && + ((rsp_len == 8) || (rsp_len == 4))) { switch (rsp_info_code) { case RSP_NO_FAILURE: lpfc_printf_vlog(vport, KERN_INFO, LOG_FCP, From patchwork Wed May 22 00:49:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0FED924 for ; Wed, 22 May 2019 00:49:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A49A128875 for ; Wed, 22 May 2019 00:49:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9925D28A26; Wed, 22 May 2019 00:49:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A3782897D for ; Wed, 22 May 2019 00:49:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728210AbfEVAti (ORCPT ); Tue, 21 May 2019 20:49:38 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:33003 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728184AbfEVAtg (ORCPT ); Tue, 21 May 2019 20:49:36 -0400 Received: by mail-pg1-f195.google.com with SMTP id h17so388550pgv.0 for ; Tue, 21 May 2019 17:49:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8vlxPfdjqDnpKOFwQDs/sHb8PGm43i0NxnYIMcUCHf8=; b=RFlJJF/TN6uhVaf0zWcpKnuaW2gU6qT/DlBNldofv5kSjQwYO1hQLrJPYM/cXZ3VTu Y5O8QYqJPjEosCN7xL2yujr98ugFn/hnsNBsEn2LQbKElnfEOiYci4ty+jc77+Va9HNT jNVGflG1fPBTbYeizOAb/Uq6ieuVslqU2c5rs5znrx0W7XRK1CNWN6mik+1/t0fTdv35 6MtbcXD8stdqVfOT8YxxLktmPyrriG69bM2osSTG+zZ0Ki6ExwCA874vl4rUdr8usI9z YnqZejdFtlC++Xsy300rhlSOvCOH1ICpUOmxucMoJA2ED6hYalgLOYgWWfeVqytH04tq jbLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8vlxPfdjqDnpKOFwQDs/sHb8PGm43i0NxnYIMcUCHf8=; b=qdvE3Yqc/bVxbwQReNpagyWM1s93Vcl6Wc9vTR0ss5ie0OHN2+HAI5C+/IdaPe1CAK CdSdyxPa9+979mjc7VHYIrhpYOj39ylvc1t2nXMZl55z/vkx61qDxjHdZZofhJKXN56X Qr0waadNqLiey2g99om9mcH5PU3X1a5w3wFmnYWFjpOqXKCNAiyru6nhjSVgg3RWBk61 wRJIy2jza/F2sl2cnYIjuQVTYCB1f+VLoK+G7FjXOIjn3K8XHz7nCbnKsBhCTgiuvzXd pDfDHEB+OSaEjNQ8HrcQmTJhhfiC5lJ6L/E/h5l1v0DW8X+rQE8W/yBz1VxDyRDVCDjU vsLQ== X-Gm-Message-State: APjAAAWkwIFYPjpl0JGNU1Ory+z8FKT+CT4qpf8N9YnQZhIxTVOmxHQu tAbqnZUzRhuQJ1MrYyu0NDN63mrn X-Google-Smtp-Source: APXvYqwrGVfES7gDMUyJ0ZAohIkkuGmuVlCstIWaUWFRG0XYVKT/+oP2XlkBlBWEGdsKAjTtFskX7A== X-Received: by 2002:a62:2f87:: with SMTP id v129mr18587238pfv.9.1558486176127; Tue, 21 May 2019 17:49:36 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:35 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 18/21] lpfc: Fix FDMI fc4type for nvme support Date: Tue, 21 May 2019 17:49:08 -0700 Message-Id: <20190522004911.573-19-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP FDMI protocol support registration was not accurately showing nvme support. The fcponly-path clears the parameter object. Move the code out of the fcponly code path. Fix the FDMI registration data to properly check for nvme support. Commonize the manner in which the fdmi routines set protocol support. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_ct.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c index 4812bbbf43cc..ec72c39997d2 100644 --- a/drivers/scsi/lpfc/lpfc_ct.c +++ b/drivers/scsi/lpfc/lpfc_ct.c @@ -2358,6 +2358,7 @@ static int lpfc_fdmi_port_attr_fc4type(struct lpfc_vport *vport, struct lpfc_fdmi_attr_def *ad) { + struct lpfc_hba *phba = vport->phba; struct lpfc_fdmi_attr_entry *ae; uint32_t size; @@ -2366,9 +2367,13 @@ lpfc_fdmi_port_attr_fc4type(struct lpfc_vport *vport, ae->un.AttrTypes[3] = 0x02; /* Type 0x1 - ELS */ ae->un.AttrTypes[2] = 0x01; /* Type 0x8 - FCP */ - if (vport->nvmei_support || vport->phba->nvmet_support) - ae->un.AttrTypes[6] = 0x01; /* Type 0x28 - NVME */ ae->un.AttrTypes[7] = 0x01; /* Type 0x20 - CT */ + + /* Check to see if Firmware supports NVME and on physical port */ + if ((phba->sli_rev == LPFC_SLI_REV4) && (vport == phba->pport) && + phba->sli4_hba.pc_sli4_params.nvme) + ae->un.AttrTypes[6] = 0x01; /* Type 0x28 - NVME */ + size = FOURBYTES + 32; ad->AttrLen = cpu_to_be16(size); ad->AttrType = cpu_to_be16(RPRT_SUPPORTED_FC4_TYPES); @@ -2680,9 +2685,12 @@ lpfc_fdmi_port_attr_active_fc4type(struct lpfc_vport *vport, ae->un.AttrTypes[3] = 0x02; /* Type 0x1 - ELS */ ae->un.AttrTypes[2] = 0x01; /* Type 0x8 - FCP */ + ae->un.AttrTypes[7] = 0x01; /* Type 0x20 - CT */ + + /* Check to see if NVME is configured or not */ if (vport->phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) ae->un.AttrTypes[6] = 0x1; /* Type 0x28 - NVME */ - ae->un.AttrTypes[7] = 0x01; /* Type 0x20 - CT */ + size = FOURBYTES + 32; ad->AttrLen = cpu_to_be16(size); ad->AttrType = cpu_to_be16(RPRT_ACTIVE_FC4_TYPES); From patchwork Wed May 22 00:49:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE84514C0 for ; Wed, 22 May 2019 00:49:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C28FE2897D for ; Wed, 22 May 2019 00:49:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B75E028875; Wed, 22 May 2019 00:49:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B10C62882F for ; Wed, 22 May 2019 00:49:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728231AbfEVAti (ORCPT ); Tue, 21 May 2019 20:49:38 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:36546 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728227AbfEVAth (ORCPT ); Tue, 21 May 2019 20:49:37 -0400 Received: by mail-pl1-f195.google.com with SMTP id d21so180514plr.3 for ; Tue, 21 May 2019 17:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xRt4kHIXH2tkalnzLg5rnVq4ZDA5QXvmA0Ae1wNV6O0=; b=mAebVITF0mOD1thE+TkeoCxro9u2Ph4e3kHpQ6CwnuLDY1gpZc8WOyBKX7ztVi3Q6A kWu4n/cM6EJFfL7okkbR2UOugsI7Q/gvpQkISXOp/xilZf1IeQ0KoJCa7Vqxf6AQI7or NvivV3kASvwJyov41Yb+P7QYWFFrEcoN/nNcTcIRR1FoQYLP7dztbJlUaGNIQhb1ZPa5 Jqozt+UbthtCFzk5HUAQ98buxctxoM5GbhojLn87z8E+I+TAmgXbi6sRp5TC/rK1Lc2B PUZVWmWM4zGeWURHRlF86pzwQM4wiUvGjYH+LqaeQbwtgsgzx03oQvwbvB9pVhszgvQZ PIgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xRt4kHIXH2tkalnzLg5rnVq4ZDA5QXvmA0Ae1wNV6O0=; b=GXLJorGoxZLyTy+z4zpPb72xSNL1fdSTRxcsFP3qpO6w2KS8hP8YibAwnGdkgNntW0 7DlZbOHCJDNIF2vOS5NgK9vRv2XEN7u+kHVhjBDA2gzlQvjWQK+7HVc1I3PZihnVZT9n EU8P0rMMjKRk+zW9xZOexKHmFVqgL1g+jPt9OpRE1PwX7zq2oUDPw6sU4pmkMo6f8NdA 1hlW17ey1z/1SrJFX8GfqBh25UM9OtB8kovoRqBt3/xsgvz9eDpdpC//jyxOe6y0+J2v 8eewvOfSSHRhtwgyA48zZnztzQTHs56as3o2QTHPUDWH9LvUUVZtlOx8ZKIEUVl2bJpf 6mpQ== X-Gm-Message-State: APjAAAX68UcioIhrFBtaVdBGzeoqzNzBCa/6CJpbO7Dggyg9IA6mfss8 b/LgBt0xnmVbkXC4X1H03qhmedkd X-Google-Smtp-Source: APXvYqz9i9JJtumbm+fQjXz9inGntyp4QlBKQGaDFBEUbvVZlLw+uytXYf/ROz0MUffpLz/0pZ1hWg== X-Received: by 2002:a17:902:8303:: with SMTP id bd3mr38671029plb.240.1558486176934; Tue, 21 May 2019 17:49:36 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:36 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 19/21] lpfc: Fix BFS crash with t10-dif enabled. Date: Tue, 21 May 2019 17:49:09 -0700 Message-Id: <20190522004911.573-20-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Crashes in scsi_queue_rq or in dma_unmap_direct_sg during BFS when lpfc has lpfc_enable_bg=1. lpfc is setting the t10-dif and prot sg after scsi_add_host_with_dma() has been called. The scsi_host_set_prot() and scsi_host_set_guard() routines need to be called before scsi_add_host_with_dma(). Revise the calling sequence to set the protection/guard data before calling scsi_add_host_with_dma(). Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reviewed-by: Ewan D. Milne --- drivers/scsi/lpfc/lpfc_init.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 416f0fb155f5..a2b827dd36ff 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -94,6 +94,7 @@ static void lpfc_sli4_disable_intr(struct lpfc_hba *); static uint32_t lpfc_sli4_enable_intr(struct lpfc_hba *, uint32_t); static void lpfc_sli4_oas_verify(struct lpfc_hba *phba); static uint16_t lpfc_find_cpu_handle(struct lpfc_hba *, uint16_t, int); +static void lpfc_setup_bg(struct lpfc_hba *, struct Scsi_Host *); static struct scsi_transport_template *lpfc_transport_template = NULL; static struct scsi_transport_template *lpfc_vport_transport_template = NULL; @@ -4348,6 +4349,9 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev) timer_setup(&vport->delayed_disc_tmo, lpfc_delayed_disc_tmo, 0); + if (phba->sli3_options & LPFC_SLI3_BG_ENABLED) + lpfc_setup_bg(phba, shost); + error = scsi_add_host_with_dma(shost, dev, &phba->pcidev->dev); if (error) goto out_put_shost; @@ -7669,8 +7673,6 @@ lpfc_post_init_setup(struct lpfc_hba *phba) */ shost = pci_get_drvdata(phba->pcidev); shost->can_queue = phba->cfg_hba_queue_depth - 10; - if (phba->sli3_options & LPFC_SLI3_BG_ENABLED) - lpfc_setup_bg(phba, shost); lpfc_host_attrib_init(shost); From patchwork Wed May 22 00:49:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D93C214C0 for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDFD92882F for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C2D5328875; Wed, 22 May 2019 00:49:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E4EF28B25 for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728227AbfEVAtl (ORCPT ); Tue, 21 May 2019 20:49:41 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:36545 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728203AbfEVAti (ORCPT ); Tue, 21 May 2019 20:49:38 -0400 Received: by mail-pl1-f193.google.com with SMTP id d21so180522plr.3 for ; Tue, 21 May 2019 17:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1gozswPOHnJaHYUH69o0H90nCpLdM+EdT9uH9W6Ys7c=; b=VVU8TWSK5AXDLpuJe9JlLMchE1RBfOjCLpdE38e8dUmautVoRKSrWcAnV82DQRGZM5 5gAA5wTGgN1wmeUlDxgEnqLLaCGTbZXDc2yUV0DgRJu8tkfSqyLY5A0zQa60IYti+A/3 4SKKdl3ypASqPEpzBFFB4SY1NXE1lR6oBfuoelrIu+oiyRttqeH7ZHd2fLhHJaNrqUat r0Eon78VwJRgWS3vQKyIu1zmZiUKvRPdERiItH+WN/RO7ck5y3CNQRSnYKmS7j6AbAhd Rz/dE0GDOukHIDQ2Dy2GeGTb4x2ZkSIJl4jaUS+B8sAmK6SREEGpOcC+UPYsHU+rqkZt LE4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1gozswPOHnJaHYUH69o0H90nCpLdM+EdT9uH9W6Ys7c=; b=huhq7NLrpfkVDFgEnByzKE8s0oeVPOA0k/yoomvK3S/gqfq9oC3TpAc58+vuKSQcT7 uXlzldVUw/hGFUjwGTAH9yTRH0jYShzTqKrEmeNED4Z2qN2W1Vp0WaoSP9vZr4RSTwyp NwcR7LGJdQmmRzupbRyZlvxHEFqENCjTYO09l4LIMQ3pX/YmoG6SetAXlM1HrPhkJsc/ jlif1orSwjRPETbWd1ELm+pEGIMQgixwV29a2V2EWXLbVq6E76T0OXKGiC5fUXoMMf2a D5ObtqQPK1d51ZbYsL/le5703LhYnRR0Kho7X9YGvh8uFPA0B14LRMb7gFGyOt5snw+q Sl7w== X-Gm-Message-State: APjAAAUtqFMK/MxyUAhFXVVg3YdhLl3DEdyaQwxJvZke9TJn9xnVhWCQ tsnDKkeZjPvb46XAfmnuhUb9ES1M X-Google-Smtp-Source: APXvYqwRHNJpPmygllDmbKFZDOBtTYGNUy0/9zstLIP8P6MyXHboPtR5OV1dv01TQbJ8wZZahXCH4w== X-Received: by 2002:a17:902:5983:: with SMTP id p3mr84498929pli.224.1558486177784; Tue, 21 May 2019 17:49:37 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:37 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 20/21] lpfc: Fix kernel warnings related to smp_processor_id() Date: Tue, 21 May 2019 17:49:10 -0700 Message-Id: <20190522004911.573-21-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Kernel warnings may be seen with preempt debugging enabled. Replace smp_processor_id calls with raw_smp_processor_id or cpu information stored in hdwq structures. Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvmet.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 36e8d842d973..eb93189f4544 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -1642,7 +1642,7 @@ lpfc_sli4_nvmet_xri_aborted(struct lpfc_hba *phba, lpfc_nvmeio_data(phba, "NVMET ABTS RCV: xri x%x CPU %02x rjt %d\n", - xri, smp_processor_id(), 0); + xri, raw_smp_processor_id(), 0); req = &ctxp->ctx.fcp_req; if (req) From patchwork Wed May 22 00:49:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10954651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86D06924 for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BA5A2882F for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 703E228B1F; Wed, 22 May 2019 00:49:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 188FA2882F for ; Wed, 22 May 2019 00:49:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728243AbfEVAtk (ORCPT ); Tue, 21 May 2019 20:49:40 -0400 Received: from mail-pl1-f175.google.com ([209.85.214.175]:42460 "EHLO mail-pl1-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728236AbfEVAtj (ORCPT ); Tue, 21 May 2019 20:49:39 -0400 Received: by mail-pl1-f175.google.com with SMTP id go2so168056plb.9 for ; Tue, 21 May 2019 17:49:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vjbieRIJLeqeCax1MXZpL9yhGeOMw3FLIQHHtfwCpe8=; b=bScRWNpLjchx5/KWf0W4jDOc9MREm0pU+IyxrhG78CSSbZtAKFsgcOTpu0DimCRK8z fHtoHRTxYDWvSnAj+sJXny3zvKyu9kKuIZROfDjZ6MSn/vOC/MmcOn0BDc9vMA+ATQwY MFVhtLD29SAbwDu/hV4GXFGOtqU/Fi4dLpwXDdz7ZLLDPYzYd/XTaTj5gj5Hb++hvsUU Y9ENkWYYpeppsfU/9RKfxOFzQumXMFrvcoPDFaR6d95AhEZL3LEsa8p/iwpnI5v6RG6I tFhTapSwclI6d0iqT2r4gw239GmS57SBsxfWJ5SGCItvf/C/bTIfbWWvGd9Qn9F/oNme t19g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vjbieRIJLeqeCax1MXZpL9yhGeOMw3FLIQHHtfwCpe8=; b=RgryrMBuWRcTgSrWB+RcSrXPDf25ipodrcXAkMf9ApwE1xGQVdF+EI3QbqGGpvMm40 LooXuPz79cSuqhgx336sBMIiiE1H3F6W4CooPYxRdKSNYyu/ipMZxzaKOHTOcbL3A0wg tSdm9BkO/G4DnI1acyDzfWKWmclgacXbj/UyCMOUVKgmxNBH379KVI6NYd9gMGEUIiHA mMeKMhNGgVUyMoK9V45QdS+NmlZxTzviIABE1oY0J8ZKhB7sfNhqiZorKO3iegYksJo3 Q5r5L+vq9nO8CcNkxAEfMk60fyhYdHFccG9LBijOy7lAyZ+2OfZVfxeLyb4UPWSXIGcL 0qUQ== X-Gm-Message-State: APjAAAWqmDYcKdf4Sajowagc++tUrwydOZXiL7MbtTZImNrJ47dNne+z edixKy0NV3MzFT937JHZ8MGWVgK6 X-Google-Smtp-Source: APXvYqz342yjuf6axlKfKkpErvEDXLyJ+CMLjmJTaK57rp0Z/dPH/kvO/EYzlyPmWebtRBlpQKDrlw== X-Received: by 2002:a17:902:b58f:: with SMTP id a15mr43193182pls.201.1558486178581; Tue, 21 May 2019 17:49:38 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id j184sm22550121pge.83.2019.05.21.17.49.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 21 May 2019 17:49:38 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy Subject: [PATCH 21/21] lpfc: Update lpfc version to 12.2.0.3 Date: Tue, 21 May 2019 17:49:11 -0700 Message-Id: <20190522004911.573-22-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190522004911.573-1-jsmart2021@gmail.com> References: <20190522004911.573-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Update lpfc version to 12.2.0.3 Signed-off-by: Dick Kennedy Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h index f7d9ef428417..f7e93aaf1e00 100644 --- a/drivers/scsi/lpfc/lpfc_version.h +++ b/drivers/scsi/lpfc/lpfc_version.h @@ -20,7 +20,7 @@ * included with this package. * *******************************************************************/ -#define LPFC_DRIVER_VERSION "12.2.0.1" +#define LPFC_DRIVER_VERSION "12.2.0.3" #define LPFC_DRIVER_NAME "lpfc" /* Used for SLI 2/3 */