From patchwork Mon Apr 9 21:24:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 10332039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0F7376053B for ; Mon, 9 Apr 2018 21:24:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2B2828BEC for ; Mon, 9 Apr 2018 21:24:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E76C028C05; Mon, 9 Apr 2018 21:24:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77ED828BEC for ; Mon, 9 Apr 2018 21:24:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753210AbeDIVYv (ORCPT ); Mon, 9 Apr 2018 17:24:51 -0400 Received: from mail-pl0-f66.google.com ([209.85.160.66]:33508 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753056AbeDIVYs (ORCPT ); Mon, 9 Apr 2018 17:24:48 -0400 Received: by mail-pl0-f66.google.com with SMTP id s10-v6so5982554plp.0 for ; Mon, 09 Apr 2018 14:24:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=o7V65yMF5+QE7RAqx9I8Q6S0j4l1+Vp0zZ9Nt8SS3lY=; b=r8cWDLpiBK6faMfodaIKrbN1oQEwAokSg60+x9G7P8veXztUvKYrciKoO60GTSB3uk B9XZw8y7Nv4f/vS8fDBzAygNf3yXFfse2764wqb207WKEd6YvSemdQ3VHOB2EFTBWdCh XFIJd5POijnksEHABqtJL+vpGU1zaa+E7piBI90A025f1rtdejwBEaFPRWe9KtVjb9iQ zmTbIp7jOjaO61B8VjSWk2ZYhkwelVpNHzMoc4YlHBQWgCjdEa8xb+Bwi2vhdEEorEZT ZZlG+Lqr1/shF+yeRzm50Ad0+ktOYmy/Gw0lobI1oDAPJhedDNSRk7mgFoV53tWA+H3Y ZkBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=o7V65yMF5+QE7RAqx9I8Q6S0j4l1+Vp0zZ9Nt8SS3lY=; b=QzvEV574sHkIRlabLD1MTHhPNbZhCaq2tuk9mClbe5y+fjP7LjqtmuQR3ZWjEeB6JL ushCuFRPe3gwlZPwoXfVcc10Sc7WTOmRh5MBga4WvZgdrAzBJ2TDA173hElaypkhP0VW e+MovZxra2pKC+wqT5VKZXbCRObfqx/DYXnxYJsWZHUlTpe+kWE9WjYk2xX1oLbY0nJw PrP6daIHgpb4XkXyuWMYEvi80dJ3xfwQ2nZtz4FQdydj+yM1NQmRWisEvkWDJ0agSMyB hqCpL9ffQea0tQlZg3YRxeQHn25N3kwZr8WTnE00iuI1DSRqWX64JOtPNxGfONJBsV/D t/0A== X-Gm-Message-State: AElRT7Gf+YRci7CPPY/DIOlgvDFeA5u3sZnhjykLZmZJ4GXnx4rfae+a uzsdHh4yz2COlrvyLW3NM6woaw== X-Google-Smtp-Source: AIpwx4/FlMaMkurnd5TuTbzwPvXqIFRiqNZhF5PDo0Zcx5ayFEKCUQbvMnzOdTzIO487Yuoj6Lm8hg== X-Received: by 2002:a17:902:8ec8:: with SMTP id x8-v6mr41277285plo.179.1523309087519; Mon, 09 Apr 2018 14:24:47 -0700 (PDT) Received: from pallmd1.broadcom.com ([192.19.223.250]) by smtp.gmail.com with ESMTPSA id w88sm2153103pfa.150.2018.04.09.14.24.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 09 Apr 2018 14:24:47 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Dick Kennedy , James Smart Subject: [PATCH v2 04/13] lpfc: Enlarge nvmet asynchronous receive buffer counts Date: Mon, 9 Apr 2018 14:24:24 -0700 Message-Id: <20180409212433.31076-5-jsmart2021@gmail.com> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20180409212433.31076-1-jsmart2021@gmail.com> References: <20180409212433.31076-1-jsmart2021@gmail.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Under large io load, the current sizing of asynchronous buffer counts could be exceeded, indicated by a 2885 log message: 2885 Port Status Event: port status reg 0x81800000, port smphr reg 0xc000, error 1=0x52004a01, error 2=0x0 Enlarge the async receive queue size. Allow for a configurable number of buffers to be posted to each RQ, using the new attribute lpfc_nvmet_mrq_post. Signed-off-by: Dick Kennedy Signed-off-by: James Smart Reviewed-by: Hannes Reinecke --- drivers/scsi/lpfc/lpfc.h | 1 + drivers/scsi/lpfc/lpfc_attr.c | 11 +++++++++++ drivers/scsi/lpfc/lpfc_nvmet.h | 6 ++++-- drivers/scsi/lpfc/lpfc_sli.c | 2 +- 4 files changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index 2b47c69c1732..20b249a649dd 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -782,6 +782,7 @@ struct lpfc_hba { uint32_t cfg_nvme_oas; uint32_t cfg_nvme_embed_cmd; uint32_t cfg_nvme_io_channel; + uint32_t cfg_nvmet_mrq_post; uint32_t cfg_nvmet_mrq; uint32_t cfg_enable_nvmet; uint32_t cfg_nvme_enable_fb; diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 3a6b1be18886..15f921d8ea56 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -3424,6 +3424,15 @@ LPFC_ATTR_R(nvmet_mrq, "Specify number of RQ pairs for processing NVMET cmds"); /* + * lpfc_nvmet_mrq_post: Specify number of RQ buffer to initially post + * to each NVMET RQ. Range 64 to 2048, default is 512. + */ +LPFC_ATTR_R(nvmet_mrq_post, + LPFC_NVMET_RQE_DEF_POST, LPFC_NVMET_RQE_MIN_POST, + LPFC_NVMET_RQE_DEF_COUNT, + "Specify number of RQ buffers to initially post"); + +/* * lpfc_enable_fc4_type: Defines what FC4 types are supported. * Supported Values: 1 - register just FCP * 3 - register both FCP and NVME @@ -5353,6 +5362,7 @@ struct device_attribute *lpfc_hba_attrs[] = { &dev_attr_lpfc_suppress_rsp, &dev_attr_lpfc_nvme_io_channel, &dev_attr_lpfc_nvmet_mrq, + &dev_attr_lpfc_nvmet_mrq_post, &dev_attr_lpfc_nvme_enable_fb, &dev_attr_lpfc_nvmet_fb_size, &dev_attr_lpfc_enable_bg, @@ -6403,6 +6413,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba) lpfc_enable_fc4_type_init(phba, lpfc_enable_fc4_type); lpfc_nvmet_mrq_init(phba, lpfc_nvmet_mrq); + lpfc_nvmet_mrq_post_init(phba, lpfc_nvmet_mrq_post); /* Initialize first burst. Target vs Initiator are different. */ lpfc_nvme_enable_fb_init(phba, lpfc_nvme_enable_fb); diff --git a/drivers/scsi/lpfc/lpfc_nvmet.h b/drivers/scsi/lpfc/lpfc_nvmet.h index c1bcef3f103c..81f520abfd64 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.h +++ b/drivers/scsi/lpfc/lpfc_nvmet.h @@ -22,8 +22,10 @@ ********************************************************************/ #define LPFC_NVMET_DEFAULT_SEGS (64 + 1) /* 256K IOs */ -#define LPFC_NVMET_RQE_DEF_COUNT 512 -#define LPFC_NVMET_SUCCESS_LEN 12 +#define LPFC_NVMET_RQE_MIN_POST 128 +#define LPFC_NVMET_RQE_DEF_POST 512 +#define LPFC_NVMET_RQE_DEF_COUNT 2048 +#define LPFC_NVMET_SUCCESS_LEN 12 #define LPFC_NVMET_MRQ_OFF 0xffff #define LPFC_NVMET_MRQ_AUTO 0 diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index cb17e2b2be81..e0a8c8008195 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -7199,7 +7199,7 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) lpfc_post_rq_buffer( phba, phba->sli4_hba.nvmet_mrq_hdr[i], phba->sli4_hba.nvmet_mrq_data[i], - LPFC_NVMET_RQE_DEF_COUNT, i); + phba->cfg_nvmet_mrq_post, i); } }