From patchwork Mon May 18 11:23:57 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 6428121 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id EFCF69F318 for ; Mon, 18 May 2015 11:24:09 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CFD19205EB for ; Mon, 18 May 2015 11:24:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7E93A205DC for ; Mon, 18 May 2015 11:24:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753669AbbERLYG (ORCPT ); Mon, 18 May 2015 07:24:06 -0400 Received: from mail-by2on0075.outbound.protection.outlook.com ([207.46.100.75]:44386 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753893AbbERLYF (ORCPT ); Mon, 18 May 2015 07:24:05 -0400 Received: from BLUPR02CA034.namprd02.prod.outlook.com (25.160.23.152) by BN3PR0201MB1027.namprd02.prod.outlook.com (25.161.207.149) with Microsoft SMTP Server (TLS) id 15.1.160.19; Mon, 18 May 2015 11:24:01 +0000 Received: from BY2FFO11FD009.protection.gbl (2a01:111:f400:7c0c::151) by BLUPR02CA034.outlook.office365.com (2a01:111:e400:8ad::24) with Microsoft SMTP Server (TLS) id 15.1.166.22 via Frontend Transport; Mon, 18 May 2015 11:24:01 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.172) smtp.mailfrom=sandisk.com; vger.kernel.org; dkim=none (message not signed) header.d=none; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.172 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.172; helo=milsmgep11.sandisk.com; Received: from milsmgep11.sandisk.com (63.163.107.172) by BY2FFO11FD009.mail.protection.outlook.com (10.1.14.73) with Microsoft SMTP Server id 15.1.172.14 via Frontend Transport; Mon, 18 May 2015 11:24:00 +0000 Received: from MILHUBIP04.sdcorp.global.sandisk.com ( [172.22.12.162]) by milsmgep11.sandisk.com (Symantec Messaging Gateway) with SMTP id 81.C3.04667.FCBC9555; Mon, 18 May 2015 04:23:59 -0700 (PDT) Received: from milsmgip11.sandisk.com (10.177.8.100) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.224.2; Mon, 18 May 2015 04:23:59 -0700 X-AuditID: ac160a68-f790b6d00000123b-6d-5559cbcfb99c Received: from [10.50.231.56] ( [10.177.8.100]) by milsmgip11.sandisk.com (Symantec Messaging Gateway) with SMTP id B4.E7.03643.ECBC9555; Mon, 18 May 2015 04:23:59 -0700 (PDT) Message-ID: <5559CBCD.6070905@sandisk.com> Date: Mon, 18 May 2015 13:23:57 +0200 From: Bart Van Assche User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Doug Ledford CC: James Bottomley , Sagi Grimberg , Sebastian Parschauer , linux-rdma , "linux-scsi@vger.kernel.org" Subject: [PATCH v2 05/12] IB/srp: Fix connection state tracking References: <5559CB3A.6060102@sandisk.com> In-Reply-To: <5559CB3A.6060102@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpjkeLIzCtJLcpLzFFi42JZI8azSPf86chQgxPrLC1env/AavF//W0W i2eHelksuq/vYLO40faAzeLkbX4HNo9n0w8zeRz+8YPZ48+fz8we7/ddZfP4vEkugDWKyyYl NSezLLVI3y6BK+NSK0fBH4OKi0vlGhgXaXQxcnJICJhIrOi4xgphi0lcuLeerYuRi0NI4ASj xLsfd1khnB2MEhe3rGCB6Th5axNUYjOjxNrPs8DaeQW0JNY3nWQGsVkEVCUubr/DDmKzCRhJ fHs/E6xZVCBMYtrv51D1ghInZz4Bi4sIqElserWIHWQos8BHRokdS26wgSSEBewlet+tBRsq BLRgwZ7lYDangLbEjg/3gAZxADVoSqzfpQ8SZhaQl9j+dg4zyBwJgZOsEp+ebWOE6FWXOLlk PtMERpFZSHbPQmifhaR9ASPzKkax3Myc4tz01AJDQ73ixLyUzOJsveT83E2M4NjhytjBuHWS +SFGAQ5GJR7eCe8iQoVYE8uKK3MPMUpwMCuJ8G7fFxkqxJuSWFmVWpQfX1Sak1p8iFGag0VJ nLc3VydUSCA9sSQ1OzW1ILUIJsvEwSnVwPjoYkEr3zu2osJXZzc/XOJx8Obuvbr9fscF/Tb4 HFhfNeWXVvbSLu+yiWHatnYMh2Sn2SjbTv/1ccuJ1PU7/995N1/iw8vgc0u+LWCc1MoistTP duau7/6v817H/agsE/Y1WWza8Sc7i29VR/uqmce+HmLLFT94QKOL6+GPI9vWvZr97Mp77W0f lFiKMxINtZiLihMBknlUZZkCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLJMWRmVeSWpSXmKPExsXCtZEjRff86chQg0d31Sxenv/AavF//W0W i2eHelksuq/vYLO40faAzeLkbX4HNo9n0w8zeRz+8YPZ48+fz8we7/ddZfP4vEkugDWKyyYl NSezLLVI3y6BK+NSK0fBH4OKi0vlGhgXaXQxcnJICJhInLy1iRXCFpO4cG89WxcjF4eQwEZG icOb+9hAErwCWhLrm04yg9gsAqoSF7ffYQex2QSMJL69n8kCYosKhElM+/2cFaJeUOLkzCdg cREBNYlNrxaxgwxlFvjKKDHj+n5GkISwgL1E77u1YEOFgBYs2LMczOYU0JbY8eEe2CBmAXWJ P/MuMUPY8hLb385hnsDIPwvJjllIymYhKVvAyLyKUSw3M6c4Nz2zwNBQrzgxLyWzOFsvOT93 EyM4gDkjdzA+nWh+iJGJg1OqgXHHqdPObf2qrs45eptDvXXWHlVP+sE8cUJBdm+bc+9E1bIE 06eH5dlmSaRk662NfZ+10EBvyruiM3wRSsrnW23cX2v+aD+bFlL3c9EKxt0VO3T/eOi9kTE9 1bX218Kbs9M/OhhELyholti7sHvP6vm24QkfOEOibzM5vHP7v+pQbdyGSvHf55VYijMSDbWY i4oTASwu6X0QAgAA X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD009; 1:cs+/SXqbAieaLapp4OtIYvZHQBb1RQnlJJXDIx0HoesIGsPjBPPhA4BfQzXcKOnD0KSQvmZ/stRNssZDGvtV5EESUoSOEKwrba7EXC0vGJDXLlbeHXaxv13AEcteiTzXFH+BWJTIOojtuBKB8Vt5XbY+DZEko+wq4CqCm774+iZPvz32uNHk3sPX8birux9LroKfCNVF2OpARdnEjzHS1AyvwGkPNP6zYV4SIweFwizH6b5IUvivoSTkHKxrbuMKrkat3TxbPgIzXEYmJcw/wkA60xxYq2atxNJmoJU83b23Ip874Vu3sv1pMTV8kHgCoEVO7/tLZFi14DyMJbFCbg== X-Forefront-Antispam-Report: CIP:63.163.107.172; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(438002)(199003)(189002)(87936001)(92566002)(33656002)(229853001)(46102003)(83506001)(59896002)(64126003)(23676002)(106466001)(86362001)(110136002)(4001350100001)(189998001)(5001830100001)(5001960100002)(5001920100001)(50986999)(76176999)(64706001)(87266999)(54356999)(47776003)(65956001)(65816999)(65806001)(5001860100001)(77096005)(4001540100001)(19580395003)(19580405001)(80316001)(77156002)(62966003)(69596002)(50466002)(68736005)(81156007)(97736004)(36756003)(2950100001)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0201MB1027; H:milsmgep11.sandisk.com; FPR:; SPF:Pass; PTR:ErrorRetry; A:1; MX:1; LANG:en; X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0201MB1027; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(3002001); SRVR:BN3PR0201MB1027; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0201MB1027; X-Forefront-PRVS: 058043A388 X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2015 11:24:00.2225 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.172]; Helo=[milsmgep11.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0201MB1027 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Reception of a DREQ message only causes the state of a single channel to change. Hence move the 'connected' member variable from the target to the channel data structure. This patch avoids that following false positive warning can be reported by srp_destroy_qp(): WARNING: at drivers/infiniband/ulp/srp/ib_srp.c:617 srp_destroy_qp+0xa6/0x120 [ib_srp]() Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] warn_slowpath_null+0x1a/0x20 [] srp_destroy_qp+0xa6/0x120 [ib_srp] [] srp_free_ch_ib+0x82/0x1e0 [ib_srp] [] srp_create_target+0x7ab/0x998 [ib_srp] [] dev_attr_store+0x20/0x30 [] sysfs_write_file+0xef/0x170 [] vfs_write+0xc8/0x190 [] sys_write+0x51/0x90 Signed-off-by: Bart Van Assche Cc: Sagi Grimberg Cc: Sebastian Parschauer Cc: #v3.19 --- drivers/infiniband/ulp/srp/ib_srp.c | 60 ++++++++++++++++++------------------- drivers/infiniband/ulp/srp/ib_srp.h | 2 +- 2 files changed, 30 insertions(+), 32 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 034d755..83e4d8a 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -465,14 +465,13 @@ static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target) */ static void srp_destroy_qp(struct srp_rdma_ch *ch) { - struct srp_target_port *target = ch->target; static struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR }; static struct ib_recv_wr wr = { .wr_id = SRP_LAST_WR_ID }; struct ib_recv_wr *bad_wr; int ret; /* Destroying a QP and reusing ch->done is only safe if not connected */ - WARN_ON_ONCE(target->connected); + WARN_ON_ONCE(ch->connected); ret = ib_modify_qp(ch->qp, &attr, IB_QP_STATE); WARN_ONCE(ret, "ib_cm_init_qp_attr() returned %d\n", ret); @@ -827,35 +826,19 @@ static bool srp_queue_remove_work(struct srp_target_port *target) return changed; } -static bool srp_change_conn_state(struct srp_target_port *target, - bool connected) -{ - bool changed = false; - - spin_lock_irq(&target->lock); - if (target->connected != connected) { - target->connected = connected; - changed = true; - } - spin_unlock_irq(&target->lock); - - return changed; -} - static void srp_disconnect_target(struct srp_target_port *target) { struct srp_rdma_ch *ch; int i; - if (srp_change_conn_state(target, false)) { - /* XXX should send SRP_I_LOGOUT request */ + /* XXX should send SRP_I_LOGOUT request */ - for (i = 0; i < target->ch_count; i++) { - ch = &target->ch[i]; - if (ch->cm_id && ib_send_cm_dreq(ch->cm_id, NULL, 0)) { - shost_printk(KERN_DEBUG, target->scsi_host, - PFX "Sending CM DREQ failed\n"); - } + for (i = 0; i < target->ch_count; i++) { + ch = &target->ch[i]; + ch->connected = false; + if (ch->cm_id && ib_send_cm_dreq(ch->cm_id, NULL, 0)) { + shost_printk(KERN_DEBUG, target->scsi_host, + PFX "Sending CM DREQ failed\n"); } } } @@ -1002,12 +985,26 @@ static void srp_rport_delete(struct srp_rport *rport) srp_queue_remove_work(target); } +/** + * srp_connected_ch() - number of connected channels + * @target: SRP target port. + */ +static int srp_connected_ch(struct srp_target_port *target) +{ + int i, c = 0; + + for (i = 0; i < target->ch_count; i++) + c += target->ch[i].connected; + + return c; +} + static int srp_connect_ch(struct srp_rdma_ch *ch, bool multich) { struct srp_target_port *target = ch->target; int ret; - WARN_ON_ONCE(!multich && target->connected); + WARN_ON_ONCE(!multich && srp_connected_ch(target) > 0); ret = srp_lookup_path(ch); if (ret) @@ -1030,7 +1027,7 @@ static int srp_connect_ch(struct srp_rdma_ch *ch, bool multich) */ switch (ch->status) { case 0: - srp_change_conn_state(target, true); + ch->connected = true; return 0; case SRP_PORT_REDIRECT: @@ -1946,7 +1943,7 @@ static void srp_handle_qp_err(u64 wr_id, enum ib_wc_status wc_status, return; } - if (target->connected && !target->qp_in_error) { + if (ch->connected && !target->qp_in_error) { if (wr_id & LOCAL_INV_WR_ID_MASK) { shost_printk(KERN_ERR, target->scsi_host, PFX "LOCAL_INV failed with status %d\n", @@ -2384,7 +2381,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event) case IB_CM_DREQ_RECEIVED: shost_printk(KERN_WARNING, target->scsi_host, PFX "DREQ received - connection closed\n"); - srp_change_conn_state(target, false); + ch->connected = false; if (ib_send_cm_drep(cm_id, NULL, 0)) shost_printk(KERN_ERR, target->scsi_host, PFX "Sending CM DREP failed\n"); @@ -2440,7 +2437,7 @@ static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag, struct srp_iu *iu; struct srp_tsk_mgmt *tsk_mgmt; - if (!target->connected || target->qp_in_error) + if (!ch->connected || target->qp_in_error) return -1; init_completion(&ch->tsk_mgmt_done); @@ -2814,7 +2811,8 @@ static int srp_add_target(struct srp_host *host, struct srp_target_port *target) scsi_scan_target(&target->scsi_host->shost_gendev, 0, target->scsi_id, SCAN_WILD_CARD, 0); - if (!target->connected || target->qp_in_error) { + if (srp_connected_ch(target) < target->ch_count || + target->qp_in_error) { shost_printk(KERN_INFO, target->scsi_host, PFX "SCSI scan failed - removing SCSI host\n"); srp_queue_remove_work(target); diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index a611556..e690847 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -170,6 +170,7 @@ struct srp_rdma_ch { struct completion tsk_mgmt_done; u8 tsk_mgmt_status; + bool connected; }; /** @@ -214,7 +215,6 @@ struct srp_target_port { __be16 pkey; u32 rq_tmo_jiffies; - bool connected; int zero_req_lim;