From patchwork Fri Jan 11 17:40:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Himanshu Madhani X-Patchwork-Id: 10760429 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC2FE13B4 for ; Fri, 11 Jan 2019 17:41:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3D322A1ED for ; Fri, 11 Jan 2019 17:41:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 954EF2A200; Fri, 11 Jan 2019 17:41:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11E0E2A1ED for ; Fri, 11 Jan 2019 17:41:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731767AbfAKRl6 (ORCPT ); Fri, 11 Jan 2019 12:41:58 -0500 Received: from mail-eopbgr820083.outbound.protection.outlook.com ([40.107.82.83]:33751 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731586AbfAKRl6 (ORCPT ); Fri, 11 Jan 2019 12:41:58 -0500 Received: from DM6PR07CA0005.namprd07.prod.outlook.com (2603:10b6:5:94::18) by DM6PR07MB4380.namprd07.prod.outlook.com (2603:10b6:5:bf::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.18; Fri, 11 Jan 2019 17:41:55 +0000 Received: from DM3NAM05FT041.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e51::207) by DM6PR07CA0005.outlook.office365.com (2603:10b6:5:94::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.13 via Frontend Transport; Fri, 11 Jan 2019 17:41:55 +0000 Received-SPF: Fail (protection.outlook.com: domain of marvell.com does not designate 199.233.58.38 as permitted sender) receiver=protection.outlook.com; client-ip=199.233.58.38; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (199.233.58.38) by DM3NAM05FT041.mail.protection.outlook.com (10.152.98.155) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.20.1537.10 via Frontend Transport; Fri, 11 Jan 2019 17:41:54 +0000 Received: from dut1171.mv.qlogic.com (10.112.88.18) by CAEXCH02.caveonetworks.com (10.67.98.110) with Microsoft SMTP Server (TLS) id 14.2.347.0; Fri, 11 Jan 2019 09:40:47 -0800 Received: from dut1171.mv.qlogic.com (localhost [127.0.0.1]) by dut1171.mv.qlogic.com (8.14.7/8.14.7) with ESMTP id x0BHelM2023029; Fri, 11 Jan 2019 09:40:47 -0800 Received: (from root@localhost) by dut1171.mv.qlogic.com (8.14.7/8.14.7/Submit) id x0BHelf9023028; Fri, 11 Jan 2019 09:40:47 -0800 From: Himanshu Madhani To: , CC: , Subject: [PATCH] qla2xxx: Use correct number of vectors for online CPUs Date: Fri, 11 Jan 2019 09:40:47 -0800 Message-ID: <20190111174047.22994-1-hmadhani@marvell.com> X-Mailer: git-send-email 2.12.0 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131917021151311470;(abac79dc-c90b-41ba-8033-08d666125e47);(abac79dc-c90b-41ba-8033-08d666125e47) X-Forefront-Antispam-Report: CIP:199.233.58.38;IPV:CAL;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(396003)(136003)(39860400002)(376002)(346002)(2980300002)(1110001)(1109001)(339900001)(189003)(199004)(51416003)(47776003)(14444005)(53936002)(97736004)(69596002)(126002)(2906002)(498600001)(87636003)(476003)(26826003)(1076003)(486006)(80596001)(2616005)(356004)(54906003)(110136005)(316002)(575784001)(50226002)(50466002)(68736007)(305945005)(36906005)(5660300001)(105606002)(8676002)(8936002)(81166006)(86362001)(42186006)(4326008)(81156014)(26005)(16586007)(336012)(36756003)(48376002)(106466001)(6346003);DIR:OUT;SFP:1101;SCL:1;SRVR:DM6PR07MB4380;H:CAEXCH02.caveonetworks.com;FPR:;SPF:Fail;LANG:en;PTR:InfoDomainNonexistent;MX:1;A:1; X-Microsoft-Exchange-Diagnostics: 1;DM3NAM05FT041;1:7AOnttTHbVLkYef3d04m4HcI2H5HfeZiH3u28KvJDUtsxWzrFK/R3jLd4jAC1WyahKr0IDENc0tGb7U0sJhLOH/YvitolORXB50sxcPNLT+NykEPN1ksskYrmGbjjaXdI2pKGEwrn/b/Vfr5GzX2Uw== X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 27403cfa-620b-4eea-71e1-08d677ec13ff X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(5600109)(711020)(2017052603328);SRVR:DM6PR07MB4380; X-Microsoft-Exchange-Diagnostics: 1;DM6PR07MB4380;3:ZOaRuF+9lrRlE4YT18fXrgcG/kDymMOBXhGjGkhHsyLY1um2E5i5uWrivOUalG68qxamh81xo0xOJANwrIIue2U4xELrWGvyB5Kk/7ub8p1nduueHT81FPj3dXgNwjh1TpO5wRFDI65JqI0KizbiCjEfZ/CN7a8E7A1RSGgwtfewr639xNN9PxRv61l1P26F2iUBjTCjy38+YGdIkS8lwMiOxDNo9EpjYBXl/BnoVmxbiLTnbaYNXePlgKv1xoFFP15Y9bSJE0R5cgWje1xVlLFklXqJZCY04fIUN6PENWTOeQvDv9AY1ZCIqKMIw8fYTOIwpLJxE3QZuMPK3DDbQ1fZxBIYilD7lb4R+QKR5XNgLPV+YBhN9X/r2ggu8ray;25:SiI5HyJ9MIN0RSnFOHPem2WWI6iXLQ/Ru6dmd6TlpsTlPyuLJh/nKoAlqpVlBU8Rcw6VL1Lyov+5HibjZ+t6BXJIrNxz7y//UPDjAIbiJjz2OJYxkcVyN9T+p0iNkifSw9la2J8tcj9Vc7TOUn70LGSfPgf38Fw4QR7BsnsIv83HZHwB94ZZ9jrHIogm+dlJpE3GKj2/Daj5d4ZhaeBYL3k9SHmi0/O+pzlxUwe0IeUEjOYauLIHTiELbqzGHhobqyQamT74LVDbqMXfP+cBvknQpsgAIg0z1zpWS+LkVEaCJCYVxxD3SXPB53NHbWHpXZMgU7gN3JZIkfvJxrTd8A== X-MS-TrafficTypeDiagnostic: DM6PR07MB4380: X-Microsoft-Exchange-Diagnostics: 1;DM6PR07MB4380;31:a+V8htcI/BPkhIPerBMZOfRhju4mSkJxYZL+chPaJDcrI5Q2btPnGuNen1MjbDX0FQ3eLJY5nUC4gOhCg5s2vR0G3sUF2ndrfsbL3Z+w0Yrq+q79070fmjBGoVkkNoQPoB3WNvfZ9y1zUFXVcI/AHmPHvqO79sQ6cgkD6BHvcA2iRfagP2Yt9gH/HuEKIyy3LfRfRKHknyoMBjSz80B+755mpk4ckVsK+AgnqVL9wnY=;4:ry6sjP3ChkrdmpjaQnRn+tAxIVHyFmJySnWPWjSHJN/0WMwd++fVBM5hBYSF+R9W4OCP2m7oM4gNE27XQTlGNymb2k2sJualSeS9YTOWRKSGYuKysE/1shAahV5SVaYr0mBgfEenabnR6XVzi0c48/025ukw+na4eyTHCq+MFf5JRmdkPLfVLKnkTQPaHpjyzZr2wqU7uez/0CbZZv3OBig2gJssHufcGk+UczFZQnz+jRESC8pZJrK5+Kpb6U4fdq9hHfY5olydDO4NElbqx5J08k25XYOgF2AIMm/d3hPNP4rCHKg4IziDr2gbrs36 X-Microsoft-Antispam-PRVS: X-Forefront-PRVS: 09144DB0F7 X-Microsoft-Exchange-Diagnostics: 1;DM6PR07MB4380;23:8q6F7kntTis0RyRdDxB7djhhf7OZxRKS87Jhl3ErVNG41Y7dI6rG9f2p8lni+XlN13pok+M7R07878rfjxQQKyqkS6oaAei3S/YESK3z4WPdxhqqUNw2Cd0vKxaJ5lU7ipRgrsREKNF0H7VINMT6JzjpGxIFIn4y5wKimMTFupGYsbIDFmrj9z8nKIAlwPDxURHsie6ay9cNaCTdvXyE5L6Or7MWoapg/pXYCrYkRp45kh6UMQh3fzFBpjJxhAlCf5P5J+t5MTbO3MAFWwTLVVxZr4EOySgGL/Ns9un3R1nu7j7swHfNcIZZnZ0djv0DehFrbuoMfk2S/2b3AAM2tHPbhCIPOuMgkWuL/0lEsrk4tesW9b3RWKZiIDOhHcu31k0DWZZGzQjvcAJRUYhNvssTok15HIwVox4m8TICqoXGUonCIkbwhKKTVHDJvzUToYLykPSLdwggpDl49JkMxOZcfBhdIAQsUecenE3qt8r2nUKSLgtJR0dbuB+utCVZOgwXenlRG4P73qb8ra1G26I7yLuKtWfSbjj6PUEcenECVfr6urBzempCph/kXVr7SSkfn0PBHwpdAp8lGx1o1eU1csrwBJ+o3pkJXI5c1U/NXEmu3iceQQz6eODcPfsWQ/MEDF6l0R5w3tZuk1MvDBzpz2ECKNjz59wWBMmhLEHA1dDvFJUr1kWWG/Jiq12ESr+M8G963KaIUsLD7pKtLJhXJAYyuNGrCKlorx1W12z8Zsn3y74tNuj4Kw8Wjc76X3QcEv2x2gBtssnVfMoA/9pBA04AhgUCEx5yEQ00JwcEuQXgLJezZIBb3p/OTd3LPlModqoeByidg9QYMYgt1+8EYCAjSCrwkfkqSt9aZSyzMqsYjh7UK83h2IF0GYUtfMxyMeFNJFQdmq8KG+wKf45JSUMaxzILUBTGyScpuqMFdlPoIbdt6TDVoRqcYIGwZaH5OQ/VRPfwrGqXxG+ot8uVsu65v3pHW07Dy24rMy0UWfWETzC5BE9cu21dfr62Ry+HmjWXK/oEFAW9lxhCcKuKWe0BbMvG7yyDl0vpfyN2lwsTSEfAHbqBAU08SqpaTb19cjKKTHlizPDQEdjs8KMLpuLiYEA4VfttDU/BuFsymvf1Ui6gD7k9AymB9NoeHzknrHMsqajQ2pp95XqulOyViQC9gU6FhRYIlKwoVdnkE1pNBqkM/Kh9HsxGUKvt X-Microsoft-Antispam-Message-Info: EQHdzRdfs/iPS97SrqWfCDsj6OnyUasSc9UFeM9Ae59OULc8L1ElLfKYlYrhbD0IAVRJ/Hzx+QwYLpf4V5/vkoC1wLiclkG9kTHpi8mKLsR3Obn1q8ZCiF7lchn7hCorjf3BhPBrl3nZE70HiK/DeELxkGvyh8i2hDrzRpQCtNRZjjQCGrLfaj1NFXZrcJQbpzzm3b8966q/rRAFkuySyY2X+yHYNcxR/76D8x64ztvHiniWLsldpon7UfRC43Te4G845m9lpxaY8g0NEoHO/acq3XJa2/OXECD4vq3/4niz8JdW3TwNtajHl21eFOuDRfarggRarMi28zTbtW7IVyz3SJiAKIwVZEkfQmt4xrCCSg8luJ1C2JZFkukPZ0oQDGlF+0fiatuNjHqY/4psAUaMVN/ZVFdYllAR8pxMza0= X-Microsoft-Exchange-Diagnostics: 1;DM6PR07MB4380;6:Ai9cEEcDTMc9mpV1mMtNfmJDBqHvGT73UvWfF5cgxuJcO1nMwW0HeU0cx2vjdek2UMd7j4wv0PBNjYrjBMjR86ljze3ug4UdbD6HXA5F0ZYcUmE9lNG3FXyWnN+2YkcPp3wNTHbDH5rbq9xCZwdcY3orrZoHPqAzQsRmEJnKM0AeTilFuzKHhJaHiD9fvg8IroSoyVvUUcm6ds4X/Cb0UfX1b16D1ij6yLy2orsFTIzFO2UmMdFvopUMji6VHimFMkpKfY02HKcRfBbbkaqZHQyJGmoeKVqXBhIAzarNdnnqre3F7w7oX8s7kCCJS1UGBRq16gOqch6HYS2NrKDNZhdCbxFM1vPIRE8iQ75uNdYlyt7I2jVIUN/sWy4QBNVv36aKWJb1fo7waCbqzukDSKeUVMW974HvOrQN56kDIp6AUONnxvnAmaeQwsMP9OJ1oliJbc6+Us0Ue5lpuxkA4A==;5:aiGFfHSi0i0OG0wSSaZ1m1ypoIAzp/PSkUy6bh2TgzrTQMVhR8aeMq+9wodGQbJonLOoFzzzy4nAiWK0Iagj/+a+rWc7CRk2F8aa5w4uZS7VFJAET/a7cqDQhRLrADlDhaBNcoKHrCAncw4vsglgEKMUUErHIJ+61JSuiuxww9HMlFNPzQAKUIwRnCzGiEBiXOde4cx/NQ5wNhXArF4i1A==;7:joP4aRcrKFiMh2WWLynOlT7TdVtlVLe0fVzILOp97cYxt84xuS0pvbOjMEN6KXt6L4z8EnG3fu+6D5Cusv0m2idQS/vpW6qia5voZ7IckrNm+0A1MtC8GTCiiyk/LaAyoebmYoPneeedU3HBLmz9nA== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2019 17:41:54.7561 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 27403cfa-620b-4eea-71e1-08d677ec13ff X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e;Ip=[199.233.58.38];Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR07MB4380 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ming Lei When SCSI-MQ is enabled, in some case system would present nr_possible_cpus() which is greater than requested vectors by the driver. This results into driver being able to get larger number of MSI-X vectors than actual online CPUs. Driver then uses pci_alloc_irq_vectors_affinity() to assign 1:1 mapping and affinity for each MSI-x vector to CPUs. When the command is submitted using MSI-x vector, assigned to offline CPU, it results in an ABTS and system hang. This hang is result of a driver not being able to process interrupt on a vector assigned to an Off-line CPUs This patch fixes this issue by setting irq_offset value for the blk_mq_pci_map_queues() to use only those CPUs which has CPU mask affinity assigned and are online. By using the irq_offset value, driver will allow online cpumask to decide which vectors are used in blk_mq_pci_map_queues(). Fixes: 5601236b6f794 ("scsi: qla2xxx: Add Block Multi Queue functionality.") Cc: #4.19 Signed-off-by: Ming Lei Reviewed-by: Himanshu Madhani Tested-by: Himanshu Madhani Signed-off-by: Himanshu Madhani Reviewed-by: Ewan D. Milne --- Hi Martin, With SCSI-MQ set as default starting 4.19, we found regression where systems with lower online CPUs but higer possible CPUs in a system resulted in the driver assigning MSIx vectors to offline CPU causing system hang. Please apply this patch to 5.0/fixes branch at your earliest convenience for inclusion in 5.0-rc3. Thanks, Himanshu --- drivers/scsi/qla2xxx/qla_def.h | 2 ++ drivers/scsi/qla2xxx/qla_isr.c | 1 + drivers/scsi/qla2xxx/qla_os.c | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/qla2xxx/qla_def.h b/drivers/scsi/qla2xxx/qla_def.h index 26b93c563f92..d1fc4958222a 100644 --- a/drivers/scsi/qla2xxx/qla_def.h +++ b/drivers/scsi/qla2xxx/qla_def.h @@ -4394,6 +4394,8 @@ typedef struct scsi_qla_host { uint16_t n2n_id; struct list_head gpnid_list; struct fab_scan scan; + + unsigned int irq_offset; } scsi_qla_host_t; struct qla27xx_image_status { diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c index 30d3090842f8..8507c43b918c 100644 --- a/drivers/scsi/qla2xxx/qla_isr.c +++ b/drivers/scsi/qla2xxx/qla_isr.c @@ -3446,6 +3446,7 @@ qla24xx_enable_msix(struct qla_hw_data *ha, struct rsp_que *rsp) "Adjusted Max no of queues pairs: %d.\n", ha->max_qpairs); } } + vha->irq_offset = desc.pre_vectors; ha->msix_entries = kcalloc(ha->msix_count, sizeof(struct qla_msix_entry), GFP_KERNEL); diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c index ea69dafc9774..c6ef83d0d99b 100644 --- a/drivers/scsi/qla2xxx/qla_os.c +++ b/drivers/scsi/qla2xxx/qla_os.c @@ -6939,7 +6939,7 @@ static int qla2xxx_map_queues(struct Scsi_Host *shost) if (USER_CTRL_IRQ(vha->hw)) rc = blk_mq_map_queues(qmap); else - rc = blk_mq_pci_map_queues(qmap, vha->hw->pdev, 0); + rc = blk_mq_pci_map_queues(qmap, vha->hw->pdev, vha->irq_offset); return rc; }