From patchwork Wed Jul 26 09:40:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDF14C001DE for ; Wed, 26 Jul 2023 09:41:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233429AbjGZJla (ORCPT ); Wed, 26 Jul 2023 05:41:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233621AbjGZJl2 (ORCPT ); Wed, 26 Jul 2023 05:41:28 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B63ADC1 for ; Wed, 26 Jul 2023 02:40:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QvKrmms9fUHhxbhcGxDz7QDKl8B/hBW7QeIiIj8phvY=; b=COaS5kKNUHqm/rRYnWqwd9ur4+S/hxxWmcO01AbI9k+7CBWKTuRQicGOYmGtXNjliizSuy v9RHR3y80Tv2MO99AhFQfrSJtnaNoxNc2sBqtZtXluGkpL1olrlluXKxQYpzvorIrAHzrd kFpjBuwYx4z5b8xKUHPdO3gE4HxHIfI= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-113-vtSBVKQAMs6GQr_RlH85qQ-1; Wed, 26 Jul 2023 05:40:40 -0400 X-MC-Unique: vtSBVKQAMs6GQr_RlH85qQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 773753C01BA1; Wed, 26 Jul 2023 09:40:39 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB50740C2063; Wed, 26 Jul 2023 09:40:38 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 1/9] blk-mq: add blk_mq_max_nr_hw_queues() Date: Wed, 26 Jul 2023 17:40:19 +0800 Message-Id: <20230726094027.535126-2-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blk_mq_alloc_tag_set() may override set->nr_hw_queues as 1 in case of kdump kernel. This way causes trouble for driver, because blk-mq and driver see different queue mapping. Especially the only online CPU may not be 1 for kdump kernel, in which 'maxcpus=1' is passed from kernel command line, then driver may map hctx0 into one inactive real hw queue which cpu affinity is 0(offline). The issue exists on all drivers which use managed irq and support multiple hw queue. Prepare for fixing this kind of issue by applying the added helper, so driver can take blk-mq max nr_hw_queues knowledge into account when calculating io queues. Signed-off-by: Ming Lei --- block/blk-mq.c | 16 ++++++++++++++++ include/linux/blk-mq.h | 1 + 2 files changed, 17 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index b04ff6f56926..617d6f849a7b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -140,6 +140,22 @@ void blk_mq_freeze_queue_wait(struct request_queue *q) } EXPORT_SYMBOL_GPL(blk_mq_freeze_queue_wait); +/* + * Return the max supported nr_hw_queues for each hw queue type + * + * blk_mq_alloc_tag_set() may change nr_hw_queues for kdump kernel, so + * driver has to take blk-mq max supported nr_hw_queues into account + * when figuring out nr_hw_queues from hardware info, for avoiding + * inconsistency between driver and blk-mq. + */ +unsigned int blk_mq_max_nr_hw_queues(void) +{ + if (is_kdump_kernel()) + return 1; + return nr_cpu_ids; +} +EXPORT_SYMBOL_GPL(blk_mq_max_nr_hw_queues); + int blk_mq_freeze_queue_wait_timeout(struct request_queue *q, unsigned long timeout) { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 495ca198775f..4c0cfd1f9e52 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -711,6 +711,7 @@ int blk_mq_alloc_sq_tag_set(struct blk_mq_tag_set *set, const struct blk_mq_ops *ops, unsigned int queue_depth, unsigned int set_flags); void blk_mq_free_tag_set(struct blk_mq_tag_set *set); +unsigned int blk_mq_max_nr_hw_queues(void); void blk_mq_free_request(struct request *rq); int blk_rq_poll(struct request *rq, struct io_comp_batch *iob, From patchwork Wed Jul 26 09:40:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD71EC04A6A for ; Wed, 26 Jul 2023 09:41:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232968AbjGZJlp (ORCPT ); Wed, 26 Jul 2023 05:41:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233626AbjGZJlj (ORCPT ); Wed, 26 Jul 2023 05:41:39 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1012BF5 for ; Wed, 26 Jul 2023 02:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k/XiItvuETzuFaV0q2/0Ml9SkHL0l2Lbxykjfon98Ww=; b=RNEAQ0cYqZ4TqxSdpO1USA3ef5WxVH1bsyWyiYSsjK22bgnHEu9FR/SajvDVbewYWlbY8s OL+ZrYYgPS/9Rb094bMhyuTSfwLo+LSxyCCLsn6lIen5Ibh4Tr7hjz7AKKA79xp1xR8czM wAMv0Acz5a9KmyD9Om43vUmN+AFij+Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-351-Ad3sbx7uMbeI3W7dnB5SwQ-1; Wed, 26 Jul 2023 05:40:43 -0400 X-MC-Unique: Ad3sbx7uMbeI3W7dnB5SwQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3B3F4803FDF; Wed, 26 Jul 2023 09:40:43 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C6F3F7835; Wed, 26 Jul 2023 09:40:42 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 2/9] nvme-pci: use blk_mq_max_nr_hw_queues() to calculate io queues Date: Wed, 26 Jul 2023 17:40:20 +0800 Message-Id: <20230726094027.535126-3-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Reported-by: Wen Xiong Signed-off-by: Ming Lei --- drivers/nvme/host/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index baf69af7ea78..a1227ae7eb39 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2251,7 +2251,7 @@ static unsigned int nvme_max_io_queues(struct nvme_dev *dev) */ if (dev->ctrl.quirks & NVME_QUIRK_SHARED_TAGS) return 1; - return num_possible_cpus() + dev->nr_write_queues + dev->nr_poll_queues; + return blk_mq_max_nr_hw_queues() + dev->nr_write_queues + dev->nr_poll_queues; } static int nvme_setup_io_queues(struct nvme_dev *dev) From patchwork Wed Jul 26 09:40:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AD7FC00528 for ; Wed, 26 Jul 2023 09:41:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230379AbjGZJln (ORCPT ); Wed, 26 Jul 2023 05:41:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233663AbjGZJlh (ORCPT ); Wed, 26 Jul 2023 05:41:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07B03F3 for ; Wed, 26 Jul 2023 02:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XUAbwUholcdP/E78pjIWjWfXWvuCT6ODN3CqmEOBnlY=; b=XJlt3rRAOnTCfNlEVs7s0XFS4BKFj9LgX25QMLUur2qvC48xtyYWMyMJmU6ZvmskOaIDtV eoqN526FxVNfLDZv5Yk8wE6gjRn8d9im1HSERtz0jlHa4v1LXVa/W6w/2/HSAUw1bLUo6B Z42Y5oaq4uafuePqZufah11t/4e2wG8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-294-oQFhGE4cMQWWMo61YsirzA-1; Wed, 26 Jul 2023 05:40:46 -0400 X-MC-Unique: oQFhGE4cMQWWMo61YsirzA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4EB1888D583; Wed, 26 Jul 2023 09:40:46 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CF031401C2E; Wed, 26 Jul 2023 09:40:45 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 3/9] scsi: core: add helper of scsi_max_nr_hw_queues() Date: Wed, 26 Jul 2023 17:40:21 +0800 Message-Id: <20230726094027.535126-4-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blk_mq_alloc_tag_set() may change nr_hw_queues for kdump kernel, so driver has to take blk-mq max supported nr_hw_queues into account when figuring out nr_hw_queues from hardware info, for avoiding inconsistency between scsi driver and blk-mq. Add helper of scsi_max_nr_hw_queues() for avoiding nr_hw_queues inconsistency between scsi driver and blk-mq. Signed-off-by: Ming Lei --- include/scsi/scsi_host.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 70b7475dcf56..2273f855940f 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -803,6 +803,11 @@ extern int scsi_host_unblock(struct Scsi_Host *shost, int new_state); void scsi_host_busy_iter(struct Scsi_Host *, bool (*fn)(struct scsi_cmnd *, void *), void *priv); +static inline unsigned int scsi_max_nr_hw_queues(void) +{ + return blk_mq_max_nr_hw_queues(); +} + struct class_container; /* From patchwork Wed Jul 26 09:40:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C85DCC001DE for ; Wed, 26 Jul 2023 09:41:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233496AbjGZJln (ORCPT ); Wed, 26 Jul 2023 05:41:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233633AbjGZJlg (ORCPT ); Wed, 26 Jul 2023 05:41:36 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C73CBF for ; Wed, 26 Jul 2023 02:40:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJmMz6GY1NqtDQ+LEnNxukeD0cgVej3lcgCePHgOvCg=; b=C9ZP5Eh6ldWatcWc26VHq3ISsY3tZNsZOeLDDNVcDHmApmP7KKCtKamatph5HnwDR4CPEJ dDR8tSBRG1qGt20Q8z2fZr9EGoQSXNM6BgE3w//UGWZGSLX/q6Fd3k/ifvxrqbyLNUzNID LUAs01pVUgIwHRq8ddnt1cp7jsglYf8= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-bqkQtEHjOTuEnIH8CcLJ7g-1; Wed, 26 Jul 2023 05:40:50 -0400 X-MC-Unique: bqkQtEHjOTuEnIH8CcLJ7g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BADB129AB3E6; Wed, 26 Jul 2023 09:40:49 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0041A40C2063; Wed, 26 Jul 2023 09:40:48 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Justin Tee , James Smart Subject: [PATCH V2 4/9] scsi: lpfc: use blk_mq_max_nr_hw_queues() to calculate io vectors Date: Wed, 26 Jul 2023 17:40:22 +0800 Message-Id: <20230726094027.535126-5-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Justin Tee Cc: James Smart Signed-off-by: Ming Lei Reviewed-by: Justin Tee --- drivers/scsi/lpfc/lpfc_init.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 3221a934066b..c546e5275108 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -13022,6 +13022,8 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba) cpu = cpumask_first(aff_mask); cpu_select = lpfc_next_online_cpu(aff_mask, cpu); } else { + vectors = min_t(unsigned int, vectors, + scsi_max_nr_hw_queues()); flags |= PCI_IRQ_AFFINITY; } From patchwork Wed Jul 26 09:40:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCD27C00528 for ; Wed, 26 Jul 2023 09:41:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233714AbjGZJly (ORCPT ); Wed, 26 Jul 2023 05:41:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233665AbjGZJls (ORCPT ); Wed, 26 Jul 2023 05:41:48 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD9C7FA for ; Wed, 26 Jul 2023 02:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tpdmXVrSFhq/BTLlYN4vy8lpJH7u5Vwwe/yeAm4djqM=; b=d2qDBKJIIfLEsYpU04mP2iJCn5f3YrM1xMuv/rLYN89NMIRGUZjKiv9aeOaZaIKPImDQxM rEVdeMUWLhLvG5ZT3Qfc8rBSFQWxBeUfx46XLCcDY1Tpw1FKz9LzG9wMeOJRC3cFGu0DXE +xYPlz6iQprlNNL50EHuDCPlbznHk7E= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-0nPrqDWGMH-CodtW9dOuMg-1; Wed, 26 Jul 2023 05:40:53 -0400 X-MC-Unique: 0nPrqDWGMH-CodtW9dOuMg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 36142104458B; Wed, 26 Jul 2023 09:40:53 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21C23200B41D; Wed, 26 Jul 2023 09:40:51 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Xiang Chen Subject: [PATCH V2 5/9] scsi: hisi: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:23 +0800 Message-Id: <20230726094027.535126-6-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Xiang Chen Signed-off-by: Ming Lei --- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index 20e1607c6282..60d2301e7f9d 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -2550,6 +2550,9 @@ static int interrupt_preinit_v3_hw(struct hisi_hba *hisi_hba) hisi_hba->cq_nvecs = vectors - BASE_VECTORS_V3_HW - hisi_hba->iopoll_q_cnt; + if (hisi_hba->cq_nvecs > scsi_max_nr_hw_queues()) + hisi_hba->cq_nvecs = scsi_max_nr_hw_queues(); + shost->nr_hw_queues = hisi_hba->cq_nvecs + hisi_hba->iopoll_q_cnt; return devm_add_action(&pdev->dev, hisi_sas_v3_free_vectors, pdev); From patchwork Wed Jul 26 09:40:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7BD7C0015E for ; Wed, 26 Jul 2023 09:41:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233057AbjGZJly (ORCPT ); Wed, 26 Jul 2023 05:41:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233627AbjGZJlo (ORCPT ); Wed, 26 Jul 2023 05:41:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0E09FD for ; Wed, 26 Jul 2023 02:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2DxJS8RxHnnXfV+djc15xnTM9KjPIA9NavQgadLTDR8=; b=Z8xkdFlhD+euo9+mJQxzwLt+GiTUP9mDzHqRNfGli8p7tJa5woIHgtR/yJrs+i+oWqie4Q Qw5dPCNDL2MANB5IXEpCeZBtu9LRJN4xt2DLSE2x2jM7B3V6cALl7nEm0a5hUpvlQr5zcP mWosyEvcZwDFUx0sJj2WRiaFIj/Gnks= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-315-w_AlwBYUO3Sf2SeTYlwxOw-1; Wed, 26 Jul 2023 05:40:57 -0400 X-MC-Unique: w_AlwBYUO3Sf2SeTYlwxOw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 04DE285A58A; Wed, 26 Jul 2023 09:40:57 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 356B0F782E; Wed, 26 Jul 2023 09:40:55 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sreekanth Reddy , Sathya Prakash Veerichetty , Kashyap Desai , Sumit Saxena Subject: [PATCH V2 6/9] scsi: mpi3mr: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:24 +0800 Message-Id: <20230726094027.535126-7-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sreekanth Reddy Cc: Sathya Prakash Veerichetty Cc: Kashyap Desai Cc: Sumit Saxena Signed-off-by: Ming Lei --- drivers/scsi/mpi3mr/mpi3mr_fw.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index 5fa07d6ee5b8..8fe14e728af6 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -815,6 +815,9 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one) desc.post_vectors = mrioc->requested_poll_qcount; min_vec = desc.pre_vectors + desc.post_vectors; + if (max_vectors - min_vec > scsi_max_nr_hw_queues()) + max_vectors = min_vec + scsi_max_nr_hw_queues(); + irq_flags |= PCI_IRQ_AFFINITY | PCI_IRQ_ALL_TYPES; retval = pci_alloc_irq_vectors_affinity(mrioc->pdev, From patchwork Wed Jul 26 09:40:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02305C0015E for ; Wed, 26 Jul 2023 09:42:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233753AbjGZJmN (ORCPT ); Wed, 26 Jul 2023 05:42:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232700AbjGZJly (ORCPT ); Wed, 26 Jul 2023 05:41:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61507128 for ; Wed, 26 Jul 2023 02:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364466; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ketWVMOak+Jss00Ni7DMSRSVvfoPSxigoCA6y+QxH1E=; b=O1rZcirrwg1Nq1/Tj8XGkV72zmz2e4n0GkTJaXkGdneqRZJJ+lqY7aYk568IlBU/O8NnkR qjDRetuN5rIlmTfEu6aiI7Zyj2lFq+eL9zYFUd8CV34akMRZ+tJ8iNbsTgjctlr23+xgAZ tlDdLykWm6Im/twloSrh1e5PUDTHHGo= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-88-QbZBECAnMwK9prNNVi7zDQ-1; Wed, 26 Jul 2023 05:41:01 -0400 X-MC-Unique: QbZBECAnMwK9prNNVi7zDQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1E55D29AB3E3; Wed, 26 Jul 2023 09:41:00 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B13A200B41D; Wed, 26 Jul 2023 09:40:58 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sreekanth Reddy , Sathya Prakash Veerichetty , Kashyap Desai , Sumit Saxena Subject: [PATCH V2 7/9] scsi: megaraid: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:25 +0800 Message-Id: <20230726094027.535126-8-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sreekanth Reddy Cc: Sathya Prakash Veerichetty Cc: Kashyap Desai Cc: Sumit Saxena Signed-off-by: Ming Lei --- drivers/scsi/megaraid/megaraid_sas_base.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index 050eed8e2684..587ccbc1ec92 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -5921,6 +5921,10 @@ __megasas_alloc_irq_vectors(struct megasas_instance *instance) int i, irq_flags; struct irq_affinity desc = { .pre_vectors = instance->low_latency_index_start }; struct irq_affinity *descp = &desc; + unsigned max_vecs = instance->msix_vectors - instance->iopoll_q_count; + + if (max_vecs > scsi_max_nr_hw_queues()) + max_vecs = scsi_max_nr_hw_queues(); irq_flags = PCI_IRQ_MSIX; @@ -5934,7 +5938,7 @@ __megasas_alloc_irq_vectors(struct megasas_instance *instance) */ i = pci_alloc_irq_vectors_affinity(instance->pdev, instance->low_latency_index_start, - instance->msix_vectors - instance->iopoll_q_count, irq_flags, descp); + max_vecs, irq_flags, descp); return i; } From patchwork Wed Jul 26 09:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 926B5C0015E for ; Wed, 26 Jul 2023 09:42:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232802AbjGZJmR (ORCPT ); Wed, 26 Jul 2023 05:42:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233728AbjGZJl7 (ORCPT ); Wed, 26 Jul 2023 05:41:59 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F0FF12C for ; Wed, 26 Jul 2023 02:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jmP+yqN1CMl4seuzyOdC4zCzUZWNFx4ZDmn6s0/7WWk=; b=IyjQwog0BosDhpjUoTgSYjckrLdMUjkHM+picU2vm4rC+18mTZtI0adedMaKwTB5nz0W37 qLproKOpK6e4azhjJr2jaVSxfzsMKwopC+s28R501gzPn1M1t+BuwFf1tZh6U29krRlrBY bWVJBfAAnp4zZUjyG2oNhwM12uD7uuk= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-492-8sZ-sCqpOy-Gs1sNG80Ajw-1; Wed, 26 Jul 2023 05:41:04 -0400 X-MC-Unique: 8sZ-sCqpOy-Gs1sNG80Ajw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 80B303815F65; Wed, 26 Jul 2023 09:41:03 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF7D9200B41D; Wed, 26 Jul 2023 09:41:02 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sathya Prakash , Sreekanth Reddy , Suganath Prabu Subramani Subject: [PATCH V2 8/9] scsi: mpt3sas: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:26 +0800 Message-Id: <20230726094027.535126-9-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sathya Prakash Cc: Sreekanth Reddy Cc: Suganath Prabu Subramani Signed-off-by: Ming Lei --- drivers/scsi/mpt3sas/mpt3sas_base.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 53f5492579cb..d238e0275edd 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -3332,8 +3332,8 @@ _base_alloc_irq_vectors(struct MPT3SAS_ADAPTER *ioc) * Don't allocate msix vectors for poll_queues. * msix_vectors is always within a range of FW supported reply queue. */ - int nr_msix_vectors = ioc->iopoll_q_start_index; - + int nr_msix_vectors = min_t(unsigned int, ioc->iopoll_q_start_index, + scsi_max_nr_hw_queues()); if (ioc->smp_affinity_enable) irq_flags |= PCI_IRQ_AFFINITY | PCI_IRQ_ALL_TYPES; From patchwork Wed Jul 26 09:40:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 13327773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBB05C0015E for ; Wed, 26 Jul 2023 09:42:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233696AbjGZJmW (ORCPT ); Wed, 26 Jul 2023 05:42:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233691AbjGZJmC (ORCPT ); Wed, 26 Jul 2023 05:42:02 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA828E69 for ; Wed, 26 Jul 2023 02:41:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=991prPOr3CC0OYGragucunFfWE1ASbB8T8+eCTuUx4g=; b=VHY2DQwkupPaZpjlyvWNieKFLV4s4SuYZCicd2bk6Krovt60q2/tQLuiD+6K5umjv+1sqL 8Ci+RNPxauGO4Oed+68ISBRTAyYYFT7F994EaN3faj7oFtEPGp+hDdf7LQeoFUWB/Was6H eQjSg69inRZgYY1PypX6FQYVPVgECCA= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-438-06SQxYjyMtOj-FMRXjmHLA-1; Wed, 26 Jul 2023 05:41:07 -0400 X-MC-Unique: 06SQxYjyMtOj-FMRXjmHLA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F1EF53815F60; Wed, 26 Jul 2023 09:41:06 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2ED79492C13; Wed, 26 Jul 2023 09:41:05 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Jack Wang Subject: [PATCH V2 9/9] scsi: pm8001: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:27 +0800 Message-Id: <20230726094027.535126-10-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Acked-by: Jack Wang Signed-off-by: Ming Lei --- drivers/scsi/pm8001/pm8001_init.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index 2e886c1d867d..3e769f4fe26d 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -965,6 +965,8 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) rc = pci_alloc_irq_vectors(pm8001_ha->pdev, 1, 1, PCI_IRQ_MSIX); } else { + unsigned int max_vecs = min_t(unsigned int, PM8001_MAX_MSIX_VEC, + scsi_max_nr_hw_queues() + 1); /* * Queue index #0 is used always for housekeeping, so don't * include in the affinity spreading. @@ -973,7 +975,7 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) .pre_vectors = 1, }; rc = pci_alloc_irq_vectors_affinity( - pm8001_ha->pdev, 2, PM8001_MAX_MSIX_VEC, + pm8001_ha->pdev, 2, max_vecs, PCI_IRQ_MSIX | PCI_IRQ_AFFINITY, &desc); }