From patchwork Wed Mar 14 13:02:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 10282095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8122A602BD for ; Wed, 14 Mar 2018 13:03:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 716952887A for ; Wed, 14 Mar 2018 13:03:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 663C3288E4; Wed, 14 Mar 2018 13:03:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC7A82887A for ; Wed, 14 Mar 2018 13:03:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751696AbeCNNDa (ORCPT ); Wed, 14 Mar 2018 09:03:30 -0400 Received: from mail-sn1nam02on0080.outbound.protection.outlook.com ([104.47.36.80]:6273 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751480AbeCNND2 (ORCPT ); Wed, 14 Mar 2018 09:03:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=icQsQGYAZQj0dZtd0FwxyA3dNL63/RltkOV7JK5fDC0=; b=DNVCh6emzuq/nOmwXslAC57cA9g3V5MzmK6M+FisdT6rgm1RBRAMK8FVabxvNcUUCKBEo+3tjnut3ho/Qx+RM3Fvs5Z33FIpipIDvBTnO++UBMnKeXUHVLg7tn9wgynwsskQCYu4yPNkzN4aX2eC3XpW6bYp6vb57jK3Ju4iidc= Received: from CY1PR07CA0009.namprd07.prod.outlook.com (2a01:111:e400:c60a::19) by BN7PR07MB4211.namprd07.prod.outlook.com (2603:10b6:406:b2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.567.14; Wed, 14 Mar 2018 13:03:26 +0000 Received: from BN1BFFO11FD012.protection.gbl (2a01:111:f400:7c10::1:195) by CY1PR07CA0009.outlook.office365.com (2a01:111:e400:c60a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.588.14 via Frontend Transport; Wed, 14 Mar 2018 13:03:26 +0000 Authentication-Results: spf=fail (sender IP is 50.232.66.26) smtp.mailfrom=cavium.com; vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=cavium.com; Received-SPF: Fail (protection.outlook.com: domain of cavium.com does not designate 50.232.66.26 as permitted sender) receiver=protection.outlook.com; client-ip=50.232.66.26; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BN1BFFO11FD012.mail.protection.outlook.com (10.58.144.75) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.527.18 via Frontend Transport; Wed, 14 Mar 2018 13:03:26 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Wed, 14 Mar 2018 06:03:23 -0700 From: Michal Kalderon To: , , CC: , , Michal Kalderon , Ariel Elior Subject: [PATCH for-rc 3/3] RDMA/qedr: Fix QP state initialization race Date: Wed, 14 Mar 2018 15:02:35 +0200 Message-ID: <1521032555-19708-4-git-send-email-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1521032555-19708-1-git-send-email-Michal.Kalderon@cavium.com> References: <1521032555-19708-1-git-send-email-Michal.Kalderon@cavium.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(396003)(376002)(39380400002)(39860400002)(2980300002)(1109001)(1110001)(339900001)(189003)(199004)(305945005)(8676002)(8936002)(72206003)(356003)(5660300001)(2906002)(81156014)(81166006)(76176011)(69596002)(106466001)(51416003)(105606002)(316002)(2950100002)(47776003)(54906003)(110136005)(186003)(77096007)(16586007)(6666003)(26005)(336012)(36756003)(4326008)(107886003)(50226002)(59450400001)(85426001)(2201001)(48376002)(50466002)(478600001)(86362001)(104016004); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR07MB4211; H:CAEXCH02.caveonetworks.com; FPR:; SPF:Fail; PTR:50-232-66-26-static.hfc.comcastbusiness.net; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD012; 1:BYqBRmDPVJlVQ4RwxqPLYiqv5j22xXyFSIG8k06pehGcX4pdn8hRowYgfr9T6507uLODMRMQPzRZshW5w65DCYbbCdi2PhsPOss4zYMMuWtROwVR7M4huBQXQtyy9YuR X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6f962c9a-30d5-4a14-bbe2-08d589abf9ae X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060); SRVR:BN7PR07MB4211; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 3:rw4RovkbW2Us+AoSUHVw76GuEWX0wK/3QKQYuEOL779Tps3mAI7WN3wKZAA3A3wooXjOJVLwNtUahuk6qTL4hdSsKCiJKnz1oPDMq0yGU34DiHtXxq+mb3JfQwSep2IpXCCNHNoqqMDJ7GsDnRhLCOZfzcNZ7JfC4y9PVYLMjF0EaFZDa0qObizG08+e9SN0eoWjMlb5hzxgWvxQBkzayzhOx5DHzFndRPz0lezX1yrOSXvFa0iKh1Vot3Twsd0UpxlP2sz1ZE/3tAb3radwdy0C5FvEBdLOqvjPYg8sWVL8nbYgjtbGJeKkOjOWaHt5CNHPJtvJpPflgZnAIgs47T4zHOnApBUqCdaJSF8mmRA=; 25:cq4NPR7W8UcNtMD4p/KwchDxtUdcewLLzP6C5oVPtKtSjAUomT6MIdzkl/f7abT72vaE+tl390IKPM2o1lywfjIaU4xf893raBNw5d1Ad7jPSx1f4shx2eribuxhesZKX3CieWn90kOr/O9kCfPP8//3uBMzwDdRMmrsjcG9lSQO35zE9d3Z60kYr0Rxmhcw0ipX0iTbXMW3Cb0vu/TJc78HP7XrIlIRbmuV6YC1U6C7HOtvIwtc5vEUl7jjheXOr2IkkHhxSYMPdYrgqeGvkmj2/jde8pq9VUZ64fuFjclmzEPJxujJ3K9xKO5hECi+IxIBKROIFiF2eeROgSs6NQ== X-MS-TrafficTypeDiagnostic: BN7PR07MB4211: X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 31:ncT8kueQZZyPKUCvrdpljcq1PosEqjg+FVXavSZNMEnQ7or4bMTTyLli3V2AT9rwh5LlLhbIGHCMAJs8erL2+GNGPzxhzMURH/fFktYV/uZZx9BFOXKyiJRAXmSOZx759/kL8IjHxmgjrXExTkqG/dIE+hW0YNq6T7OvE51Z4og4ES63ZYEZ7PdkP3w2ekv7QHmsJsJybGRaKmYYub8P77sytMs356YjNt5h3FnTKOs=; 20:/5ffMjHzcTNstef8k+2WlkzfCQCeimjLCtyl3Xf5kbHGEHtXGLzOAJ4biHzhKxQM4bRHF9MiB+x5EdaGoFENK1shU+qvCzvNV4pHWr5+z6g+FfLMxTZZ6rkKgC3b6pUJgyimgrkg9O8dwJNuXW2FfEsacP4lS+zita9yNeyoPrvNe/NzpVa1/+L+NMPZg4LxYqXT84+zki337bqXEI3uRDsUnAAb3HeOOJauc5nmEAi7DCc3w3X5SUKi/nExdb6pZXupARiRGOSBuBh2H1Ev/fZXpv8epNP5tU9I5ftkYO0JnTaIcKxxpwmVdPj6oKjUkT5YCPG54RoZJqJbR7eTNbcAoDtTvu0Vd9c9DYyLfvnhvrMdrbj0Nmqi3fsxEY+9cR3aWsq4/EG6nua5YwKi7JIicNWHz/GhGoo1D54tA2m5N0IpxfhSF7S4N1KEzpUoarheCxBH7HJEdkA4T2hsNyEwX6UKwoRyFsNL7Cgf+33Vuo6okP1L72yMJebrT8Og X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231221)(944501244)(52105095)(93006095)(93001095)(3002001)(10201501046)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:BN7PR07MB4211; BCL:0; PCL:0; RULEID:; SRVR:BN7PR07MB4211; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 4:nLzbPcRhS63NYiEsqtz2KeqaBrL9Xr4a1UgVZRRG2hdCv/jBgfLHo6bGJdHR96rDOToGadf2yR8TBHVhpYLzf2dS71b7kmL2I6OOkeV+PMmrdm3BbNAxf32GJhUv3bMjrPEuKwHTh1cU31qDcx+5P5yHpjC4IjE/v/dx7FqGckfi2+kKi9VUa6bZjZEuqbnCtT3fT++v9p3AjNY+Yd4KHXu8Q9MpcPG8q9th1JwKYffqiisVSKpvrT+WzDzOQ/94apkeFYJZ8zWc4alBvqCttg== X-Forefront-PRVS: 0611A21987 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN7PR07MB4211; 23:cv0dMJWE43LwG7YR78hIYyo1sPil0sjJEnx7DNDPI?= =?us-ascii?Q?Cqnt/Llauhcl1TKS/dZ8tMcpH/B5zu1exvyCk+34Nk7OAsz/p9w5+9YvLPnI?= =?us-ascii?Q?v63qpuJqQBw3xunkzM0k3UdgmrZjwHn8hffDi3jpU7VJbmxrvAiDUuy+rVXX?= =?us-ascii?Q?2xn/VZn/rR+U1m7ZJ0ha0qH9GegGl5n18K4zcJ7i7l7jsGqEzMeeVVNad6kn?= =?us-ascii?Q?/V6yAScYSod6ZSMq+4xwjq5tWog9j5h94y1qQG/jQqK6BM08T76QbmkdS0si?= =?us-ascii?Q?nakF/nr4ouV16Y6nF8xvavE9shgfdw+o9MMAnZr00Kx1AO3efFbng7Ttae58?= =?us-ascii?Q?aV6oD2Mw4e9uxUXMkvK7dhZjRRoockUuWFyxu7yWcZCooqCHUo7zvYpCbnVe?= =?us-ascii?Q?Q9U4JRYzpJR+FD+evHkiSwQmiWnbX6o7B8ezVGutCPhrVjHkSKifBvPegsm0?= =?us-ascii?Q?WLw6QEUqOqCBZC+8hldc8eHI5VUrvkGvhQwxiVWVbCKTl7gHpoeOgkwQkzVG?= =?us-ascii?Q?Sn1BIPi0C5vSOYnxt+4/7o6k/vA/cJtpiMG9AIHya584HBUfhtP75wZ+4g02?= =?us-ascii?Q?QsGSvv9Vyu4n/bbawVlapWJBm1pIU013CLN2/gkhxDmT438uCdrB/7aCfUsR?= =?us-ascii?Q?qCMPPjMdB8k7hBW9oBYgaNvX9moPT5Ap2tlaOxl399x80L+1U8fZZV7HJQRP?= =?us-ascii?Q?sjiydS4fGlzGSiafoxYhwian3bXY4u9TgLCCrFczjJfDfVxfz6P0WdiYYPi/?= =?us-ascii?Q?JuLWLybUjjfieauyRFP9LY+mUISnri8d3/4Vf8lWl2NSo8P3q42CuYXQ2rch?= =?us-ascii?Q?3OVl5HIJGVNL14D2Idmt+ooVVxCC7bDvR1n5Xt3WzGdI7GwPh0HlJLDeusDl?= =?us-ascii?Q?BwfvM6/cj7o8kDLrhW0KllqbWwwUm3sfs/iYjnWsCaYA1EIElQuld6lFWVmA?= =?us-ascii?Q?qJ0YZx2wC/cmcYJ0MGFrSkWnOTOHbETs6GAx+eR3TTs1FJQ4bFdqPJi2CzBT?= =?us-ascii?Q?XLIbi4xtb1nCn/1hH62qTarZuMGwnFRoznSfsG4OpqP8tAjkzZiUGpkaaczs?= =?us-ascii?Q?vq4sKwfC5r3mVyL0hBEgntH+H+ftzjDXdsJZSrrE2PcenAKsAkv8yf7gHvsb?= =?us-ascii?Q?ZmBjCUO3ng=3D?= X-Microsoft-Antispam-Message-Info: A+ZWg7O3JlnGlSqc5aMt+COxCGcpZPwhlQc06L+uyEX4C0fWgraGdXwwFalMKhUXA0X8IdIKI/q1RZUa40OtrEvhUz/TjtYCnkRELtA4JOgo/dmSyv4Pl5Xs1+krLFb3P40eHVjqEqYlOc/hjiyTGmLrjPk5s9+srtigAWB/OUwKXpjyBA/fDgaxmdKVnJ1E X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4211; 6:iSie0vZbqEit+IJWnIhMaWqtYzxh0eqqjNXQ68e/ZP0AfwO1XKvCCX7maWNTa92LRkzirQ821C3b0pn4pCUKJ8skzmWqzX+O8g1Xdp6KqmJFmPnGUP5bfpu5/Ha7a3y3rUVU6lqtizvLvA0MgfrMu0jqrOjieLvHPnxqAza+ZtlRuVaMeLgmMT6aFAJoiBQ8JtrjxCvZkRa7gk/crmHk1YRwGCkeHM0uQVkNsoR1FOQCnOXiaIF3M2KXOaYDH/rq6MDw1t0UdJb35A8c5ALh7qrZUdldDUjbcgugpcrjk76JSRBvB+xiym2e2p0I6TBdKauFDYRLvwYFSqjsKJT9nKtDg7ww2L05/r4KQnYq5JM=; 5:qU2SA61bCFNBBtFHfNVpJznhcJdCGhlxd+Pq5Rq7bV3PkfV1J43gHDDrfcfWac+o9HxSDE5OELQZYVLzQlSMIrQm+rgI7Htm53cPwCyDMQlMYb1DuUcUseA/w0cNRyLEOirWyyejlgFb5Te1gTLDAwTxYoCQswXf++cle+GQpo0=; 24:yxmxUJ+q2MNFsVG52dkQqUStvHPuHMDKj0PT7ZDkE+of9URknXJnGe8GWqfVvkTiWIMAE/3VHV423tAqGEQzHRH60bgHeDxTpufWN+kQeXY=; 7:MDHzj3ewoJuwYOaMIovA0ZWT3GzH/+cdBddrkJJtNOJAbl9lmCJSpZgLAbxYTAvt0lpYytG+Kp84dRopciTvgBW8ssU5F3DpmclRVaGcDcD1J444bBRcGaFI5odUMZS6vv1VQLdM7AEaliFvaQGzhS1CTqhN/qIMOukKIlk/ygoc13vCCA8J1nPgpPqWF5IFLwtt86LpOt56k5o+CkY+YiXLzozTcONADw4e62frklzKYcmVS5MhFcMghUXKaiQ9 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2018 13:03:26.0596 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6f962c9a-30d5-4a14-bbe2-08d589abf9ae X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR07MB4211 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Once the FW is transitioned to error, FLUSH cqes can be received. We want the driver to be aware of the fact that QP is already in error. Fixes: cecbcddf ("qedr: Add support for QP verbs") Signed-off-by: Michal Kalderon Signed-off-by: Ariel Elior --- drivers/infiniband/hw/qedr/verbs.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c index 716fae0..419a158 100644 --- a/drivers/infiniband/hw/qedr/verbs.c +++ b/drivers/infiniband/hw/qedr/verbs.c @@ -1841,14 +1841,15 @@ static void qedr_reset_qp_hwq_info(struct qedr_qp_hwq_info *qph) static int qedr_update_qp_state(struct qedr_dev *dev, struct qedr_qp *qp, + enum qed_roce_qp_state cur_state, enum qed_roce_qp_state new_state) { int status = 0; - if (new_state == qp->state) + if (new_state == cur_state) return 0; - switch (qp->state) { + switch (cur_state) { case QED_ROCE_QP_STATE_RESET: switch (new_state) { case QED_ROCE_QP_STATE_INIT: @@ -1955,6 +1956,7 @@ int qedr_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, struct qedr_dev *dev = get_qedr_dev(&qp->dev->ibdev); const struct ib_global_route *grh = rdma_ah_read_grh(&attr->ah_attr); enum ib_qp_state old_qp_state, new_qp_state; + enum qed_roce_qp_state cur_state; int rc = 0; DP_DEBUG(dev, QEDR_MSG_QP, @@ -2175,13 +2177,25 @@ int qedr_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, qp->dest_qp_num = attr->dest_qp_num; } + cur_state = qp->state; + + /* Update the QP state before the actual ramrod to prevent a race with + * fast path. Modifying the QP state to error will cause the device to + * flush the CQEs and while polling the flushed CQEs will considered as + * a potential issue if the QP isn't in error state. + */ + if ((attr_mask & IB_QP_STATE) && qp->qp_type != IB_QPT_GSI && + !udata && qp_params.new_state == QED_ROCE_QP_STATE_ERR) + qp->state = QED_ROCE_QP_STATE_ERR; + if (qp->qp_type != IB_QPT_GSI) rc = dev->ops->rdma_modify_qp(dev->rdma_ctx, qp->qed_qp, &qp_params); if (attr_mask & IB_QP_STATE) { if ((qp->qp_type != IB_QPT_GSI) && (!udata)) - rc = qedr_update_qp_state(dev, qp, qp_params.new_state); + rc = qedr_update_qp_state(dev, qp, cur_state, + qp_params.new_state); qp->state = qp_params.new_state; }