From patchwork Thu Aug 10 19:05:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9894533 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F24F360348 for ; Thu, 10 Aug 2017 19:06:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E398528B13 for ; Thu, 10 Aug 2017 19:06:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D7A1128B18; Thu, 10 Aug 2017 19:06:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 205CF28B13 for ; Thu, 10 Aug 2017 19:06:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752756AbdHJTGX (ORCPT ); Thu, 10 Aug 2017 15:06:23 -0400 Received: from mail-by2nam01on0088.outbound.protection.outlook.com ([104.47.34.88]:1920 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752751AbdHJTGW (ORCPT ); Thu, 10 Aug 2017 15:06:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=nHN62QEVkanbqRX3ffrnRc/s9rmuwmU0qahx+5XSfEY=; b=T0lia/63qO88dkZOEDQeieOWRh5wSdZRe00BYVIB+iEqcV3GLcLpnPzkUAZCRr2CgPKzqz299mDS4BgHKXyICNuTaXOBiDWBK+4Z6vbwCEPdC6J/gACNKPfK0KPlfQoKaNl6Od8FJCrW6E7sUSFqmFbI8615iqqSAHsK7pt7DzE= Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by CY4PR05MB3509.namprd05.prod.outlook.com (10.171.247.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1341.9; Thu, 10 Aug 2017 19:05:23 +0000 From: Adit Ranadive To: dledford@redhat.com, linux-rdma@vger.kernel.org, pv-drivers@vmware.com Cc: Bryan Tan , Jorgen Hansen , Aditya Sarwade , Adit Ranadive Subject: [PATCH v1 for-rc] RDMA/vmw_pvrdma: Report CQ missed events Date: Thu, 10 Aug 2017 12:05:02 -0700 Message-Id: <1502391902-5674-1-git-send-email-aditr@vmware.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: BN6PR16CA0022.namprd16.prod.outlook.com (10.172.212.160) To CY4PR05MB3509.namprd05.prod.outlook.com (10.171.247.18) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 47e9e0a2-f9dd-4dc5-1db4-08d4e022c19f X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CY4PR05MB3509; X-Microsoft-Exchange-Diagnostics: 1; CY4PR05MB3509; 3:7OOOeSlA8zuggNv1AeC+d/PtC3uWZ7w/CWdhls65nv+B134aGO487iRG0iGwqI1pEhc5s1Us8AM5KDcHtmHhrFXNyjxN6ZfTTPoZplRdyqTb1UrIHs6KbzlDR7GRy/iZ3C2AH0F6sj04rgEZnV1iR8+UVnFa4guImput1kBqWtlVq62e5c1d/Ylp7zZWEU1q2JQ6NRU2EJx6Ee3WdjGVyJ2WidMw79AHg0OFfZrQIJea0UzprgQzal5a/Fn0kSHn; 25:WGIfLQP4Fc2GeYofVyPt4fIVs9Ehh2gly6RBQq87o9TKYgHop9KdwGtT8gH332Fl3qsRpw8Z5JdqDzUHyLNxe1EnPcsATN9fAcpPFVnjvqGMy6ahHni2C9bms69oGKhKnspXuWAZUJUh2ZMiLipGDumR8Ej14EzVbilWz8WnmomyVhuod/TaS3qktxIF4a/9pNiKopxnQ+486onssd+JTYUOfnbNuYWbcP/CceQ9i0HY9dHkIk+wKGvT+CH1qiivDnv+PSowKizpqPcjv3EUd5rl/EUQ4Fk1JlR5maxXqeT4H+PlH86RFnA5zUUz1xGN/HhS57GZPqTYOgb+oRhzeQ==; 31:aHaEC4q7SQUlbYGgAtg3AFHQOOIbbIvPRbjyT+rOa0j8gsDtp8N5cewF8NdXgGZB7Rzy0WTjd/E7GfghuAaYAHgwYix+n4veLJW+IabZwrFO07OuNE0A8PprIoNECjJthY1cX+1YYScvKmjXNeCna8+SiGtLQNsJfHQrRhyG8AY5JbzaBg++RDcFli576gndT3NdY3YY9iXLRiKjPKY9FQDH3A8QE/mfqsQdnes/5/0= X-MS-TrafficTypeDiagnostic: CY4PR05MB3509: X-Microsoft-Exchange-Diagnostics: 1; CY4PR05MB3509; 20:RY+A09KqsBwuqXYn1MsRGbPnMeV2ZvgIUdygGlFp6MgccMgGrx3VmI1rkAGNtP5J3pf2/nX3EEItMAEB2xUwnoG+7uFYcEyq7macMXpgCVvKvAbC7qdXUavYrcs42X5BFwRwD6JTV2aVinhXLDpZ2s7jw3i3nB8w2C30Xnk7by8y678FVfqPuRkQ86xNRFL0CYxSDgnt6miaQQCAQQOZnS1GJRUMAI0GN016fzV9QXV7QXVAc0CyX7hVfTqSBPbGE7QAsBAtoaDP5qXXkMImCn37oVtzQo6jwc07SGBTW5ZFyIrGR9CgsQYhJy8Su4NuGYzAREPoMl7vYsUP9epGWXArO0pQB3SyXsxmzakAke44JZtyQn53l7cXVJQKykhnWvKZV6jM10rPal7OZ7uLBmFsDYNeB20d0nelFiu8yjb9zpbMgV9mIWU/PbVqjQuBDTSsRPITzNuzdvbpUL2iNzpUTozlHDlfkFLZRzqPJyQxv5S3pjcXQiaBA5MZQIeg; 4:dpNchQmdFHuTOt9FXrPXU9bG/rfICFALaz5GkQsXK4bz3YYDWr1WKl5FWQ9jYbS2QiRerpVNPFFTWRysCJuVwDVJLsNM3nRBHZOa/5Xiti1aHMdFeLbUrfpPhZw3diOLtgie+YTEk1tn6rnI4McnBVKd4tNrBgTNG0HByDprk6N1nHsNSol2wOzjrcMDussMyETOa/GnI4HswCRqyMhFUdsfg7u18ObiOZJwl9p1QR5lsjoaSpGN0l8ECdn6RBfuYUYQE650o1EjekE0bXdUnKiZ80gt5/0Nxl8loAcX+mGt5WKbKk2Ti1xNLHcvo6xyQNBH/bOQcAiCyRRjOSmP2Q== X-Exchange-Antispam-Report-Test: UriScan:(61668805478150)(211171220733660); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(10201501046)(100000703101)(100105400095)(6041248)(20161123558100)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR05MB3509; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR05MB3509; X-Forefront-PRVS: 03950F25EC X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7370300001)(4630300001)(6009001)(39860400002)(199003)(189002)(81156014)(53936002)(6116002)(3846002)(54906002)(7350300001)(47776003)(81166006)(8676002)(50226002)(7736002)(42186005)(110136004)(305945005)(106356001)(50466002)(33646002)(105586002)(66066001)(101416001)(48376002)(107886003)(5660300001)(4326008)(36756003)(478600001)(5003940100001)(189998001)(6486002)(50986999)(6666003)(68736007)(6636002)(25786009)(86362001)(97736004)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR05MB3509; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR05MB3509; 23:xk/ugsTmwLP21w06j0NrIC67EUDbDcPBR7FwbX+rF?= =?us-ascii?Q?oH5dlrzdNYEmcWpHglFAsU0w3HbFkzRDnr88DCqDzpELD2xDk8JuLkailsUd?= =?us-ascii?Q?Y8YbwmKV2S6CVGte/Z2Fm6xg/jPUPdo2p3aLHsl7Br77k6AvDpce+cgr1Xvo?= =?us-ascii?Q?23DgWHVNGgklrcP2ZuSEodCOMegNlhlsOjXFUqDqT5TtvoL8rYUM8HHzq93d?= =?us-ascii?Q?l2O7vBKchJpKvSI1139FjOKpQo7L/EBmqILMc9I617OhEPrtXC8OYSfK9s0C?= =?us-ascii?Q?Ly/SSksjnp60R5KlK2aClpNSGkAVu2rx3u0GpDts3xgpb7cgbpQWRJSLNwoW?= =?us-ascii?Q?KoQuZMfDxP6FMJKvqjeTUJMQIAfosVjkgj3Ls0fhJ+gVyR60juvf5WdSo06d?= =?us-ascii?Q?g/UDiDMNVg8nbLxv72NM7EiRkKcGWABoOIYnKIp2zRktSYj4l0daJbXmkl3f?= =?us-ascii?Q?lL4145stXNNiBuU7qFBy8o6BjzGnrRMRdu831zCYgZa097fCOWaZhVd2k9Xg?= =?us-ascii?Q?lqxmZZ2cnRslnOIkxaphY7skR18M9l/C3IUh6nHnLnTbBqzLD/6w7pQBHleP?= =?us-ascii?Q?q13h1SEc/QG5pdg8qsdRPj0JazG5Pve7Pi8wTBwV1T06geyqXV1zahTj20Hk?= =?us-ascii?Q?fYgP10wsYKC0WoLzdP1gjTVQNBqjI2SYmbAiDXXJj44p2snRyv/IRogaI9Iv?= =?us-ascii?Q?ASU6AbbSB5dEiGDSj+CVdgR9lECXnGnt2IvsTU3EL0y2dQU+5OOOf4pwGqjO?= =?us-ascii?Q?fDESbT7S3QWu2T2U/cOXRrUNC02ifp5GJeHdvCI9jOa1s4K0i9a8ZWGXUG4D?= =?us-ascii?Q?8UNznfLGmBqT59mqWmTz8j9xoeAaiZGHsYQd1PFCyD1Lk4agJ58y8mHVtf/w?= =?us-ascii?Q?x7c3XFYPHQ5x9fCOTEuCnKm/YNwbabKBmq8mEJRIHjUiSNrypc3x6OfcF1Nv?= =?us-ascii?Q?FETFhm1K4/rwTdQ7K2uyESfov9HGvt7BO7Iz5lPasjkMqh5WYfTQ9XaBDNRi?= =?us-ascii?Q?sQ/ZzNlDxgt1aRf6CzMctacW/KNxbOR25aSjv+NE5/ifX+R+TaiMRWcA2Zw8?= =?us-ascii?Q?HCrFpFebmqSOEbYnvl+0iZSQQ0J?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR05MB3509; 6:6qvOPzrFBioorrJRHYjJGSkWZbI/uNNov7bNDwvls3O/mTPji8ODGh2P69Q0LTowRdtJbiRPXd8XwEjdELxDbLKYhFn7fQwo9Op9g16nCGBXRDw5zhXgqGAOYKEvurauIx0dx28avlUk9ON4t/px6tifosU5ku+nWmVIlZBhT6Jk5rT0CLjmiPwXW+FYt5wdsmgbtZGJfRIeTy+BKLGZ9KKS+hDs2ITzgYBtqtrkmWocZocvkFuCNxdDg5zsmPUNIBBZF5PLMNehnRWVFRW0ddq9noA9Fms2vir3w7RcbYPsDA0hwWzI4X/8/44yuteB7ioRWTXMKzVK1xJPK0ZYJg==; 5:bbuorXQbyEuQvkGkJU1JqMxmvBPzHlCrS0+kzxRbfchSN4SAscu+m8V9dZcV/7fheIDTZAzMAuwsvd0uCBdEsAVxJAQRWG1l4WMUT0SErqwhzobmglHwPYdxwQU+D9mzP+DQj39Dm9upFxWBI7ciNg==; 24:zBRuf13TJCOFa9rgXYLv2X/pMS/uNGJoNtsI+Phd1AN4IEzg6mEzPP3DQptJl54rHc48ckSdY4g0VJJOFeGYh/zeZjAjZ0Yi3DoNHvtqe0Y=; 7:XQoBkpxmB9j+xlQTr+vsn+337IBAxLzmOyt4qh9lAI2w0vpJsOWMYHRWeuuDCn3IZ/JJZi65mj3XhQDiEezta5sR6R0H3POSAbO3a4WJrTRXVpDo7AHV88sN46LuwdtCK/9e7FCnFqSBIG+zkyh9WzDy8V/mNodtN1HDDYbL6bNa46COSIg8NzD4GuV/DV9Z/HshafwmWEYAF9IaGS7FD1wDjIdL9iPWlN7uqxjl258= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR05MB3509; 20:gjdQ3hIHctPnBkkDTybxDicBJjLpmpHxtXRBXvNC+hCLBasUzUwMz3IRs4N1e4i2b/rHio67qhM0kFXoLEPEFv+oel81zMn97spP63j+k61zIsCdLe0ZSUWhljRHndW+Z0bBc9Lfc96hNBmCy2jvXkubq452FHF6lBKkP0yhBxk= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Aug 2017 19:05:23.3662 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR05MB3509 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bryan Tan There is a chance of a race between arming the CQ and receiving completions. By reporting CQ missed events any ULPs should poll again to get the completions. Fixes: 29c8d9eba550 ("IB: Add vmw_pvrdma driver") Acked-by: Aditya Sarwade Signed-off-by: Bryan Tan Signed-off-by: Adit Ranadive Reviewed-by: Yuval Shaia --- v0 -> v1: - Check for invalid ring index. --- drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c index 69bda61..90aa326 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c @@ -65,13 +65,28 @@ int pvrdma_req_notify_cq(struct ib_cq *ibcq, struct pvrdma_dev *dev = to_vdev(ibcq->device); struct pvrdma_cq *cq = to_vcq(ibcq); u32 val = cq->cq_handle; + unsigned long flags; + int has_data = 0; val |= (notify_flags & IB_CQ_SOLICITED_MASK) == IB_CQ_SOLICITED ? PVRDMA_UAR_CQ_ARM_SOL : PVRDMA_UAR_CQ_ARM; + spin_lock_irqsave(&cq->cq_lock, flags); + pvrdma_write_uar_cq(dev, val); - return 0; + if (notify_flags & IB_CQ_REPORT_MISSED_EVENTS) { + unsigned int head; + + has_data = pvrdma_idx_ring_has_data(&cq->ring_state->rx, + cq->ibcq.cqe, &head); + if (unlikely(has_data == PVRDMA_INVALID_IDX)) + dev_err(&dev->pdev->dev, "CQ ring state invalid\n"); + } + + spin_unlock_irqrestore(&cq->cq_lock, flags); + + return has_data; } /**