From patchwork Sat Sep 24 23:21:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9349333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CC090601C2 for ; Sat, 24 Sep 2016 23:22:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB33B290C3 for ; Sat, 24 Sep 2016 23:22:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF3FE290C6; Sat, 24 Sep 2016 23:22:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D4293290D3 for ; Sat, 24 Sep 2016 23:22:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030357AbcIXXWb (ORCPT ); Sat, 24 Sep 2016 19:22:31 -0400 Received: from ex13-edg-ou-002.vmware.com ([208.91.0.190]:3077 "EHLO EX13-EDG-OU-002.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965526AbcIXXWW (ORCPT ); Sat, 24 Sep 2016 19:22:22 -0400 Received: from sc9-mailhost2.vmware.com (10.113.161.72) by EX13-EDG-OU-002.vmware.com (10.113.208.156) with Microsoft SMTP Server id 15.0.1156.6; Sat, 24 Sep 2016 16:21:53 -0700 Received: from EX13-CAS-005.vmware.com (smtp-inbound.vmware.com [10.113.191.55]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id D3927B031E; Sat, 24 Sep 2016 16:22:16 -0700 (PDT) Received: from EX13-MBX-037.vmware.com (10.113.191.78) by EX13-MBX-014.vmware.com (10.113.191.34) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Sat, 24 Sep 2016 16:22:16 -0700 Received: from EX13-CAS-003.vmware.com (10.113.191.53) by EX13-MBX-037.vmware.com (10.113.191.78) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Sat, 24 Sep 2016 16:22:16 -0700 Received: from NAM01-BN3-obe.outbound.protection.outlook.com (10.113.170.11) by EX13-CAS-003.vmware.com (10.113.191.53) with Microsoft SMTP Server (TLS) id 15.0.1156.6 via Frontend Transport; Sat, 24 Sep 2016 16:22:15 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=NHVcxJ4ZeDb8CaSBOeoyxIJdzADeseG9zjcGE9PGstQ=; b=T6QtYFC/p6OsVrDa4F4SjMT8GmX+7CP1MR33YD7mhhW47zfFW1V/+ik2TwMWtsDgGIkD/qVXceJGSaP025Qzo3yYcYZj7R+bH/ajnnMJHUbx3FskVq6Nx4MnZ911J3lbSPIKVGtWgHarcfFFkXf7wZkZhBFtqhgOKCz9V/pNifY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by CO2PR0501MB837.namprd05.prod.outlook.com (10.141.244.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.639.2; Sat, 24 Sep 2016 23:22:13 +0000 From: Adit Ranadive To: , , CC: Adit Ranadive , , , , , , Subject: [PATCH v5 04/16] IB/pvrdma: Add the paravirtual RDMA device specification Date: Sat, 24 Sep 2016 16:21:28 -0700 Message-ID: <21a1dfdd8ce9c43e8b4df148482d05ae876eaeda.1474759181.git.aditr@vmware.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: CY1PR18CA0027.namprd18.prod.outlook.com (10.163.31.37) To CO2PR0501MB837.namprd05.prod.outlook.com (10.141.244.147) X-MS-Office365-Filtering-Correlation-Id: 176d33f2-e685-4677-5095-08d3e4d19e4b X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 2:h+3m9BNPcJppF2vVigt4O9L2CVkY1vZMGiMe7C4sxgzfpR+UQ73tZEF3VxU4zn1lHyT+v3vn5n/HpYVZRYJSNP7izq+QV0qIVbIFNZP0p2sTbyP3MivrG22FX4bNxBFFIulRRWqj66xYAoZKRWZjsLW8jD2F+YtrOfABkaNQwKuJMIRgvmcre0EfXwajtuOC; 3:tIVbD7NdJTwElI53LtohOHkmGyqqvBed8DRM6ek3YKQp3u9L+/TitO+7f12vOqigeENwdNhp6194bJ6oG04+4LWRPdR+wfk21RoDOPpc/OH28z1kI5m5EhD5gA0oxfaA; 25:RJR7peUM4720znaJVlY2cb8on/8h7t6YDW/87FEb/Np7Yh4NiegCZWSYT1n94pzZ6iyKRgFnBYDR5ul702W9cihUtmr9bF9ZV3vPW048yj10AMpHEmyx4tOKGN33TtS8e9BsATNDuDk4m7c39IZ4XiUsKLhC5MmT4MQjYx9r6kS6mlx7WsGAYHBCx9mk2G9zZ2D49zDiztBvRJ0MaE5yfqEkaNxl5uQO1qIfWey8qVc/IAiPwQdlej2Eeilk6ULSg32Tqe6Ni2+6LssyKfKKd5gU/giwqX1yh0WEgXMRpH3QtAel9cWxIo5wKYzpDRwjlllucFoZKQRu2ymnxZZuABTCLsLzzZmo/WLAKtBrV2XjEZ6qRwIQ+MCEF5ONxz7ywvPvCeGTiaPaMd8pkAiepky5rori03ySWPX11nOEgcQ= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CO2PR0501MB837; X-LD-Processed: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 31:Vp0pWput7169rQfrfy3JV6I9btQZIxc6+E2I9inmQ9FAgkuNnbUqNDucwIdvxmnIuUAZpc1WC32YbGYCY5G6Y4O7l7RKElW3zSeCsv7UaJvio59gXIL840dgK7EryzMdcG5aJfTWSLDQhotORk6J4/FgrJqoOKI1fpk1caEqH5sp81YNfGkk38jZMZiSX0ua35aSlu+OG/6BsKC81nxH9y8UbNKcjn6Who/Wz9She/M=; 20:O/T6PtP7euY2uq/7sNEfTBHExsFBwZKdssl8oBkHP8ARl3DV9/lJ6b4zv7uNOjkhU6rBa+r0u49q7IhSUZSzflQi6/S/D7/t9+4THqUplb7ikG6+GOxyBIRISq21e8hmspBlkIosnSe3fii2EsvKgA9vKd6LpL6/K0xGBAhLo6iJckZe/qgpr9WcCrIDpJ4TYba4jz3YEynzlyWpSIu3HS+mCW1zFQHhYSbNSO0+e9SEqnwexoy3mjafNuRUeJD49UdQZ5SkxvFLkHHV4i1PEJy0kbTIq/S00itxD01j9b6RKShRijp9mskboPi23XmDNXoJX6jpqOh3UtUofTtYCInZ0Adi+trUQr4t2aCVs8tmwST2qyTaNLx6r+i4GTTFuA0+riBwAFZk9227E/E2oiUPfkdic1t0EzkrJSdWLKyfYAkb8B1ppKIxfS69ovxlOKYqDINZJanttEOyqf2XQ3CpOrl5iiIED15FxNgCGC11mITa4eMCVUYJmG/c0nCK X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(250305191791016)(61668805478150)(22074186197030)(146099531331640); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:CO2PR0501MB837; BCL:0; PCL:0; RULEID:; SRVR:CO2PR0501MB837; X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 4:C+ZO8KEBsAmDx4zssu9DU4dPKc15WS2enbqHnHYg39eemu4utDk3gyG6Y2+tjjSUsicTdhisvlt3kfbNrB9T/S6NwVgjVnRG+F4S+xd1ws3FifoahXi8Lp9WEZi7M9c68ol0rWfPXUR1/SgQXr4/9xyW8W1kDMT8Hv44a3rKIFe+hVqv9CWDRwwgjd3ongPaGsVw94ZRul0Y2AYD+23lk6e8hxbyn7CWbWkQq59eUo/vBLlOmz/irMjumiKOdp6qgDMWcaSO2m4Vm4aNltHkovfyiLnuQ0pupsMHm9tSJ9Y1nUzZsGov4vg1hmkY18SM0qqiV9vx6zcJRtW9iTrFDBwV96KKq6vnalhTc28nqGciZMRqhU/7Rq/CXsZ4oP9+vawshalIt4Fpf2uhrsfWuPZps4A9O06rlAEjjiV8z/DZf+vcDnsP0/11kxavHe/8oOAPObMT2WOJ6wzpvHYghwK1NXqLh8y7BedZv8jzCWM2A5sDflXLqNgP6ANmZZ2kMRC5t4shgjMlI65lMuB+X8lTB3PPl9usc6h0SaqpzUg= X-Forefront-PRVS: 0075CB064E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(107886002)(77096005)(97736004)(5890100001)(4326007)(106356001)(105586002)(15975445007)(76176999)(586003)(6862003)(118296001)(50986999)(2950100002)(33646002)(6116002)(5001770100001)(19580395003)(5660300001)(19580405001)(92566002)(2906002)(3846002)(47776003)(189998001)(50466002)(86362001)(48376002)(575784001)(2201001)(81156014)(81166006)(101416001)(8676002)(50226002)(5003940100001)(36756003)(7736002)(7846002)(305945005)(229853001)(42186005)(68736007)(66066001)(4001430100002)(7099028)(21314002)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:CO2PR0501MB837; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (EX13-EDG-OU-002.vmware.com: aditr@vmware.com does not designate permitted sender hosts) Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CO2PR0501MB837; 23:IEwLr0SN3l78deYmXleeFPe8m2F7SxSKcCrStZZq?= =?us-ascii?Q?1VYfNDPoeZf1pCWDBawfOKZjssUb4WuYngsFoAwz7gILId475Yu9fX1Mrxal?= =?us-ascii?Q?0O78C6BHr5f+2mYTvxl0JucTKSbzLyzTr2x2Ez2ppz4943BrHohRlb9mioaC?= =?us-ascii?Q?CUhYaPGPC2/lONCx9+2Dy3erih5TBhdQpPHvcrtJaJeoqfGfVfNgxW3+FaHF?= =?us-ascii?Q?8VczTPFnsNplTN1REG6mYS3Jy8Gkq83VkA03C5PQF68D5GJ/LQVDBs9CxFhU?= =?us-ascii?Q?TdGBWfbltGz+olpvSuHnNoyO3eS3qTfi0fz1fdm41tZcKfQ08z66x4gBv33t?= =?us-ascii?Q?6NU4oqn+46uK+tgfmAUQzzDsbQeDpf5FE7rMA05P80VeCxALKksUQW99vqu1?= =?us-ascii?Q?bSWfUO8XcBwOt/iNgKWlI8L0D8ysH9PGAuoSh/BmtfvgSvrjLcDfm6X5btC2?= =?us-ascii?Q?UwMxzOubnbo7ZP6PHSWpzsrClxvuADxgxSkIkVDbnL79cFtqMg+u8e6/Vadu?= =?us-ascii?Q?D6gM6gjAH4Smuk7WvjgfV++X9ks+wj15VCoq1szX1BzsxvuDKmkNamapR7SV?= =?us-ascii?Q?JRW3uHW4rgYWBPUp+j7sauq7INUH+2GwlcA4XIpt85z9QWqMNT69WIIO1laC?= =?us-ascii?Q?ucWqBYM+JZQUve08t7V9W8xWeMmurpEkwIIy3wvcTtfvIRBsIwfKzZLsAsrE?= =?us-ascii?Q?MIif6BLTbwOIQtbzPSi8MAE0xPkKkxM2Bb0TYihLcyVfJ5iA7AMQIKJDYCmY?= =?us-ascii?Q?Iax4FpIw4cbhmNf1qlBwx3G02mO2ZCTtE7v4Q2FqtkQ1GImAhLOhne6+V6HB?= =?us-ascii?Q?YIuIMKKzo8q1Kx4ldF4t6VSuTmBC+Kur/NkXvGxdQwEJ0jonJBym0SbWz3E8?= =?us-ascii?Q?Xj4Z4UV4QHfHG/+uAKPml2qMw0+hvvNKMiSPltiPRvH7ycwEG1yxHap/CHJZ?= =?us-ascii?Q?yx2iBZO5HTc/xVA97WpRrm5QzTSbrQVNnibZHbjF0ROBK1bPXpK5c2bSV2Dz?= =?us-ascii?Q?KkdeR8lSvTtEO7c0qig8QIzD7w9cA1VqWYxIWKs+4NN8SUaJqFuudiY2D1/4?= =?us-ascii?Q?i5Z6RbDWxJl5LkX6mUV69zFlIt9U56kKF9Hj6mVQCV0J1Mtt7WAnxN0wikjP?= =?us-ascii?Q?fR7cPlBnsSSy99HDI2Ssbsj1zQy2xfGjfH3DAF2SSHFpQaaZPkn9mwNlYzBP?= =?us-ascii?Q?Cdkar0nSbdwHsPPxp66ynRSFw1f6PTZNW0EyFPkqI3SHIiabs7i8hRHsogLZ?= =?us-ascii?Q?fvAEkTVgB8cvyswiVlyMlQN9ZBKSxz+DntJFStED32Hp2qchgOMq1q/VQg49?= =?us-ascii?Q?VA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 6:+cGBGdiZO12iiDYLMB33Kol5ohX0wTKlTiZlpnZcHRz2T4DAu7k46sU3a/spH8r8hanZn+cZfEmHFU4DYRhA8eymmVurK3bmX3tCtBkxmBPTnajB1yQ4b+jaCrvA4je7dUuJFG5UY1Sh/4VCDHHMYLmVQEdvEowAV5brfc9lKZ1k97wOglEAbLuEM4Bpnd3UCNwX3hywRv4OY3NWbm5fB6PSZl31ds4UL1/R4zMDxUoq9mMNiabWqg4UTi2IC7fXjuXQHj3k8xSRFwkK3ZTPlF1qvBut2O9czE0LdHB9txg=; 5:oC90u+24I5SCDFY2EkhHKmrcGSAekNSulroUTTtQ/KrUHK+K3p8mp1lMD5KfcUKsA271XM7LYTCr8Ay6Dwp6aMSJcpBQQcj/Hp6CVjlzrOj8zH/3hhsfBIDnW1cqibxZNFPDlVAM3cf/wbmSqV8Dhg==; 24:RHsPac8QVd7QwZ7h2DD9j0L9cg8xIUh5oLC3lk+qQ9CYssEQ1TYC9O12/xUs5xp88l1jRcxhJNQFBsWs4rkZJVxwn3Hqmgeb+C98+FyQ7yg=; 7:FBCshRr2t4sIWYkzQa9yFGDK8PVSolLc5SAvPmOS4fEPc951jmysYLBkrWqjR9VistJmy00LtcHkMf9vXUTbfvBrnimHI14bAaL4pqfyjgpr+gfrY5XY6UGsvbfwYKNe+EF4LkB4/IZhOmlDo9Kdk35QlbqdXoNconIBrCwEpCeDjjU3B6hmESBTROR6M2PJO41hNBbGC9+PMhWiWWMfC65DHtpFcP/XuJMz4PgNVpDfXdQ8hM6IIAWeqm14d0zcemshKrwEhsyWAUxeCNcOlFbBFeicoN9jUo1MBoJdoGrJsK8pJc2BKQPC/QDdGllI SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 20:vFWJmxKDpfl3rt/Ped0qbKB9Edke0YdC2IqzM0HNsv7KZvEak5EiiGfPOOuWgLNIKvqnh7+G8Ccvi4Xj9t9bsKt4RoMDtBVB0jgfg/Uv5WJ2SOldCtLafDk61XFvsKqTaCjf2StIRvMh/n8lIL7NkcPVLj9Ie8UpAt+3str4psE= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2016 23:22:13.2075 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR0501MB837 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch describes the main specification of the underlying virtual RDMA device. The pvrdma_dev_api header file defines the Verbs commands and their parameters that can be issued to the device backend. Reviewed-by: Yuval Shaia Reviewed-by: Jorgen Hansen Reviewed-by: George Zhang Reviewed-by: Aditya Sarwade Reviewed-by: Bryan Tan Signed-off-by: Adit Ranadive --- Changes v4->v5: - Update include for headers moved to UAPI. - Removed __ prefix for unsigned vars. Changes v3->v4: - Removed explicit enum values. Changes v2->v3: - Defined 9 and 18 for page directory. - Stripped spaces in some comments. --- drivers/infiniband/hw/pvrdma/pvrdma_defs.h | 301 +++++++++++++++++++++++ drivers/infiniband/hw/pvrdma/pvrdma_dev_api.h | 342 ++++++++++++++++++++++++++ 2 files changed, 643 insertions(+) create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_defs.h create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_dev_api.h diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_defs.h b/drivers/infiniband/hw/pvrdma/pvrdma_defs.h new file mode 100644 index 0000000..8105b01 --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_defs.h @@ -0,0 +1,301 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_DEFS_H__ +#define __PVRDMA_DEFS_H__ + +#include +#include +#include "pvrdma_ib_verbs.h" + +/* + * Masks and accessors for page directory, which is a two-level lookup: + * page directory -> page table -> page. Only one directory for now, but we + * could expand that easily. 9 bits for tables, 9 bits for pages, gives one + * gigabyte for memory regions and so forth. + */ + +#define PVRDMA_PDIR_SHIFT 18 +#define PVRDMA_PTABLE_SHIFT 9 +#define PVRDMA_PAGE_DIR_DIR(x) (((x) >> PVRDMA_PDIR_SHIFT) & 0x1) +#define PVRDMA_PAGE_DIR_TABLE(x) (((x) >> PVRDMA_PTABLE_SHIFT) & 0x1ff) +#define PVRDMA_PAGE_DIR_PAGE(x) ((x) & 0x1ff) +#define PVRDMA_PAGE_DIR_MAX_PAGES (1 * 512 * 512) +#define PVRDMA_MAX_FAST_REG_PAGES 128 + +/* + * Max MSI-X vectors. + */ + +#define PVRDMA_MAX_INTERRUPTS 3 + +/* Register offsets within PCI resource on BAR1. */ +#define PVRDMA_REG_VERSION 0x00 /* R: Version of device. */ +#define PVRDMA_REG_DSRLOW 0x04 /* W: Device shared region low PA. */ +#define PVRDMA_REG_DSRHIGH 0x08 /* W: Device shared region high PA. */ +#define PVRDMA_REG_CTL 0x0c /* W: PVRDMA_DEVICE_CTL */ +#define PVRDMA_REG_REQUEST 0x10 /* W: Indicate device request. */ +#define PVRDMA_REG_ERR 0x14 /* R: Device error. */ +#define PVRDMA_REG_ICR 0x18 /* R: Interrupt cause. */ +#define PVRDMA_REG_IMR 0x1c /* R/W: Interrupt mask. */ +#define PVRDMA_REG_MACL 0x20 /* R/W: MAC address low. */ +#define PVRDMA_REG_MACH 0x24 /* R/W: MAC address high. */ + +/* Object flags. */ +#define PVRDMA_CQ_FLAG_ARMED_SOL BIT(0) /* Armed for solicited-only. */ +#define PVRDMA_CQ_FLAG_ARMED BIT(1) /* Armed. */ +#define PVRDMA_MR_FLAG_DMA BIT(0) /* DMA region. */ +#define PVRDMA_MR_FLAG_FRMR BIT(1) /* Fast reg memory region. */ + +/* + * Atomic operation capability (masked versions are extended atomic + * operations. + */ + +#define PVRDMA_ATOMIC_OP_COMP_SWAP BIT(0) /* Compare and swap. */ +#define PVRDMA_ATOMIC_OP_FETCH_ADD BIT(1) /* Fetch and add. */ +#define PVRDMA_ATOMIC_OP_MASK_COMP_SWAP BIT(2) /* Masked compare and swap. */ +#define PVRDMA_ATOMIC_OP_MASK_FETCH_ADD BIT(3) /* Masked fetch and add. */ + +/* + * Base Memory Management Extension flags to support Fast Reg Memory Regions + * and Fast Reg Work Requests. Each flag represents a verb operation and we + * must support all of them to qualify for the BMME device cap. + */ + +#define PVRDMA_BMME_FLAG_LOCAL_INV BIT(0) /* Local Invalidate. */ +#define PVRDMA_BMME_FLAG_REMOTE_INV BIT(1) /* Remote Invalidate. */ +#define PVRDMA_BMME_FLAG_FAST_REG_WR BIT(2) /* Fast Reg Work Request. */ + +/* + * GID types. The interpretation of the gid_types bit field in the device + * capabilities will depend on the device mode. For now, the device only + * supports RoCE as mode, so only the different GID types for RoCE are + * defined. + */ + +#define PVRDMA_GID_TYPE_FLAG_ROCE_V1 BIT(0) +#define PVRDMA_GID_TYPE_FLAG_ROCE_V2 BIT(1) + +enum pvrdma_pci_resource { + PVRDMA_PCI_RESOURCE_MSIX, /* BAR0: MSI-X, MMIO. */ + PVRDMA_PCI_RESOURCE_REG, /* BAR1: Registers, MMIO. */ + PVRDMA_PCI_RESOURCE_UAR, /* BAR2: UAR pages, MMIO, 64-bit. */ + PVRDMA_PCI_RESOURCE_LAST, /* Last. */ +}; + +enum pvrdma_device_ctl { + PVRDMA_DEVICE_CTL_ACTIVATE, /* Activate device. */ + PVRDMA_DEVICE_CTL_QUIESCE, /* Quiesce device. */ + PVRDMA_DEVICE_CTL_RESET, /* Reset device. */ +}; + +enum pvrdma_intr_vector { + PVRDMA_INTR_VECTOR_RESPONSE, /* Command response. */ + PVRDMA_INTR_VECTOR_ASYNC, /* Async events. */ + PVRDMA_INTR_VECTOR_CQ, /* CQ notification. */ + /* Additional CQ notification vectors. */ +}; + +enum pvrdma_intr_cause { + PVRDMA_INTR_CAUSE_RESPONSE = (1 << PVRDMA_INTR_VECTOR_RESPONSE), + PVRDMA_INTR_CAUSE_ASYNC = (1 << PVRDMA_INTR_VECTOR_ASYNC), + PVRDMA_INTR_CAUSE_CQ = (1 << PVRDMA_INTR_VECTOR_CQ), +}; + +enum pvrdma_intr_type { + PVRDMA_INTR_TYPE_INTX, /* Legacy. */ + PVRDMA_INTR_TYPE_MSI, /* MSI. */ + PVRDMA_INTR_TYPE_MSIX, /* MSI-X. */ +}; + +enum pvrdma_gos_bits { + PVRDMA_GOS_BITS_UNK, /* Unknown. */ + PVRDMA_GOS_BITS_32, /* 32-bit. */ + PVRDMA_GOS_BITS_64, /* 64-bit. */ +}; + +enum pvrdma_gos_type { + PVRDMA_GOS_TYPE_UNK, /* Unknown. */ + PVRDMA_GOS_TYPE_LINUX, /* Linux. */ +}; + +enum pvrdma_device_mode { + PVRDMA_DEVICE_MODE_ROCE, /* RoCE. */ + PVRDMA_DEVICE_MODE_IWARP, /* iWarp. */ + PVRDMA_DEVICE_MODE_IB, /* InfiniBand. */ +}; + +struct pvrdma_gos_info { + u32 gos_bits:2; /* W: PVRDMA_GOS_BITS_ */ + u32 gos_type:4; /* W: PVRDMA_GOS_TYPE_ */ + u32 gos_ver:16; /* W: Guest OS version. */ + u32 gos_misc:10; /* W: Other. */ + u32 pad; /* Pad to 8-byte alignment. */ +}; + +struct pvrdma_device_caps { + u64 fw_ver; /* R: Query device. */ + __be64 node_guid; + __be64 sys_image_guid; + u64 max_mr_size; + u64 page_size_cap; + u64 atomic_arg_sizes; /* EXP verbs. */ + u32 exp_comp_mask; /* EXP verbs. */ + u32 device_cap_flags2; /* EXP verbs. */ + u32 max_fa_bit_boundary; /* EXP verbs. */ + u32 log_max_atomic_inline_arg; /* EXP verbs. */ + u32 vendor_id; + u32 vendor_part_id; + u32 hw_ver; + u32 max_qp; + u32 max_qp_wr; + u32 device_cap_flags; + u32 max_sge; + u32 max_sge_rd; + u32 max_cq; + u32 max_cqe; + u32 max_mr; + u32 max_pd; + u32 max_qp_rd_atom; + u32 max_ee_rd_atom; + u32 max_res_rd_atom; + u32 max_qp_init_rd_atom; + u32 max_ee_init_rd_atom; + u32 max_ee; + u32 max_rdd; + u32 max_mw; + u32 max_raw_ipv6_qp; + u32 max_raw_ethy_qp; + u32 max_mcast_grp; + u32 max_mcast_qp_attach; + u32 max_total_mcast_qp_attach; + u32 max_ah; + u32 max_fmr; + u32 max_map_per_fmr; + u32 max_srq; + u32 max_srq_wr; + u32 max_srq_sge; + u32 max_uar; + u32 gid_tbl_len; + u16 max_pkeys; + u8 local_ca_ack_delay; + u8 phys_port_cnt; + u8 mode; /* PVRDMA_DEVICE_MODE_ */ + u8 atomic_ops; /* PVRDMA_ATOMIC_OP_* bits */ + u8 bmme_flags; /* FRWR Mem Mgmt Extensions */ + u8 gid_types; /* PVRDMA_GID_TYPE_FLAG_ */ + u8 reserved[4]; +}; + +struct pvrdma_ring_page_info { + u32 num_pages; /* Num pages incl. header. */ + u32 reserved; /* Reserved. */ + u64 pdir_dma; /* Page directory PA. */ +}; + +#pragma pack(push, 1) + +struct pvrdma_device_shared_region { + u32 driver_version; /* W: Driver version. */ + u32 pad; /* Pad to 8-byte align. */ + struct pvrdma_gos_info gos_info; /* W: Guest OS information. */ + u64 cmd_slot_dma; /* W: Command slot address. */ + u64 resp_slot_dma; /* W: Response slot address. */ + struct pvrdma_ring_page_info async_ring_pages; + /* W: Async ring page info. */ + struct pvrdma_ring_page_info cq_ring_pages; + /* W: CQ ring page info. */ + u32 uar_pfn; /* W: UAR pageframe. */ + u32 pad2; /* Pad to 8-byte align. */ + struct pvrdma_device_caps caps; /* R: Device capabilities. */ +}; + +#pragma pack(pop) + + +/* Event types. Currently a 1:1 mapping with enum ib_event. */ +enum pvrdma_eqe_type { + PVRDMA_EVENT_CQ_ERR, + PVRDMA_EVENT_QP_FATAL, + PVRDMA_EVENT_QP_REQ_ERR, + PVRDMA_EVENT_QP_ACCESS_ERR, + PVRDMA_EVENT_COMM_EST, + PVRDMA_EVENT_SQ_DRAINED, + PVRDMA_EVENT_PATH_MIG, + PVRDMA_EVENT_PATH_MIG_ERR, + PVRDMA_EVENT_DEVICE_FATAL, + PVRDMA_EVENT_PORT_ACTIVE, + PVRDMA_EVENT_PORT_ERR, + PVRDMA_EVENT_LID_CHANGE, + PVRDMA_EVENT_PKEY_CHANGE, + PVRDMA_EVENT_SM_CHANGE, + PVRDMA_EVENT_SRQ_ERR, + PVRDMA_EVENT_SRQ_LIMIT_REACHED, + PVRDMA_EVENT_QP_LAST_WQE_REACHED, + PVRDMA_EVENT_CLIENT_REREGISTER, + PVRDMA_EVENT_GID_CHANGE, +}; + +/* Event queue element. */ +struct pvrdma_eqe { + u32 type; /* Event type. */ + u32 info; /* Handle, other. */ +}; + +/* CQ notification queue element. */ +struct pvrdma_cqne { + u32 info; /* Handle */ +}; + +static inline void pvrdma_init_cqe(struct pvrdma_cqe *cqe, u64 wr_id, u64 qp) +{ + memset(cqe, 0, sizeof(*cqe)); + cqe->status = PVRDMA_WC_GENERAL_ERR; + cqe->wr_id = wr_id; + cqe->qp = qp; +} + +#endif /* __PVRDMA_DEFS_H__ */ diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_dev_api.h b/drivers/infiniband/hw/pvrdma/pvrdma_dev_api.h new file mode 100644 index 0000000..66682cf --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_dev_api.h @@ -0,0 +1,342 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_DEV_API_H__ +#define __PVRDMA_DEV_API_H__ + +#include +#include "pvrdma_ib_verbs.h" + +enum { + PVRDMA_CMD_FIRST, + PVRDMA_CMD_QUERY_PORT = PVRDMA_CMD_FIRST, + PVRDMA_CMD_QUERY_PKEY, + PVRDMA_CMD_CREATE_PD, + PVRDMA_CMD_DESTROY_PD, + PVRDMA_CMD_CREATE_MR, + PVRDMA_CMD_DESTROY_MR, + PVRDMA_CMD_CREATE_CQ, + PVRDMA_CMD_RESIZE_CQ, + PVRDMA_CMD_DESTROY_CQ, + PVRDMA_CMD_CREATE_QP, + PVRDMA_CMD_MODIFY_QP, + PVRDMA_CMD_QUERY_QP, + PVRDMA_CMD_DESTROY_QP, + PVRDMA_CMD_CREATE_UC, + PVRDMA_CMD_DESTROY_UC, + PVRDMA_CMD_CREATE_BIND, + PVRDMA_CMD_DESTROY_BIND, + PVRDMA_CMD_MAX, +}; + +enum { + PVRDMA_CMD_FIRST_RESP = (1 << 31), + PVRDMA_CMD_QUERY_PORT_RESP = PVRDMA_CMD_FIRST_RESP, + PVRDMA_CMD_QUERY_PKEY_RESP, + PVRDMA_CMD_CREATE_PD_RESP, + PVRDMA_CMD_DESTROY_PD_RESP_NOOP, + PVRDMA_CMD_CREATE_MR_RESP, + PVRDMA_CMD_DESTROY_MR_RESP_NOOP, + PVRDMA_CMD_CREATE_CQ_RESP, + PVRDMA_CMD_RESIZE_CQ_RESP, + PVRDMA_CMD_DESTROY_CQ_RESP_NOOP, + PVRDMA_CMD_CREATE_QP_RESP, + PVRDMA_CMD_MODIFY_QP_RESP, + PVRDMA_CMD_QUERY_QP_RESP, + PVRDMA_CMD_DESTROY_QP_RESP, + PVRDMA_CMD_CREATE_UC_RESP, + PVRDMA_CMD_DESTROY_UC_RESP_NOOP, + PVRDMA_CMD_CREATE_BIND_RESP_NOOP, + PVRDMA_CMD_DESTROY_BIND_RESP_NOOP, + PVRDMA_CMD_MAX_RESP, +}; + +struct pvrdma_cmd_hdr { + u64 response; /* Key for response lookup. */ + u32 cmd; /* PVRDMA_CMD_ */ + u32 reserved; /* Reserved. */ +}; + +struct pvrdma_cmd_resp_hdr { + u64 response; /* From cmd hdr. */ + u32 ack; /* PVRDMA_CMD_XXX_RESP */ + u8 err; /* Error. */ + u8 reserved[3]; /* Reserved. */ +}; + +struct pvrdma_cmd_query_port { + struct pvrdma_cmd_hdr hdr; + u8 port_num; + u8 reserved[7]; +}; + +struct pvrdma_cmd_query_port_resp { + struct pvrdma_cmd_resp_hdr hdr; + struct pvrdma_port_attr attrs; +}; + +struct pvrdma_cmd_query_pkey { + struct pvrdma_cmd_hdr hdr; + u8 port_num; + u8 index; + u8 reserved[6]; +}; + +struct pvrdma_cmd_query_pkey_resp { + struct pvrdma_cmd_resp_hdr hdr; + u16 pkey; + u8 reserved[6]; +}; + +struct pvrdma_cmd_create_uc { + struct pvrdma_cmd_hdr hdr; + u32 pfn; /* UAR page frame number */ + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_uc_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 ctx_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_destroy_uc { + struct pvrdma_cmd_hdr hdr; + u32 ctx_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_pd { + struct pvrdma_cmd_hdr hdr; + u32 ctx_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_pd_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 pd_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_destroy_pd { + struct pvrdma_cmd_hdr hdr; + u32 pd_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_mr { + struct pvrdma_cmd_hdr hdr; + u64 start; + u64 length; + u64 pdir_dma; + u32 pd_handle; + u32 access_flags; + u32 flags; + u32 nchunks; +}; + +struct pvrdma_cmd_create_mr_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 mr_handle; + u32 lkey; + u32 rkey; + u8 reserved[4]; +}; + +struct pvrdma_cmd_destroy_mr { + struct pvrdma_cmd_hdr hdr; + u32 mr_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_cq { + struct pvrdma_cmd_hdr hdr; + u64 pdir_dma; + u32 ctx_handle; + u32 cqe; + u32 nchunks; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_cq_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 cq_handle; + u32 cqe; +}; + +struct pvrdma_cmd_resize_cq { + struct pvrdma_cmd_hdr hdr; + u32 cq_handle; + u32 cqe; +}; + +struct pvrdma_cmd_resize_cq_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 cqe; + u8 reserved[4]; +}; + +struct pvrdma_cmd_destroy_cq { + struct pvrdma_cmd_hdr hdr; + u32 cq_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_qp { + struct pvrdma_cmd_hdr hdr; + u64 pdir_dma; + u32 pd_handle; + u32 send_cq_handle; + u32 recv_cq_handle; + u32 srq_handle; + u32 max_send_wr; + u32 max_recv_wr; + u32 max_send_sge; + u32 max_recv_sge; + u32 max_inline_data; + u32 lkey; + u32 access_flags; + u16 total_chunks; + u16 send_chunks; + u16 max_atomic_arg; + u8 sq_sig_all; + u8 qp_type; + u8 is_srq; + u8 reserved[3]; +}; + +struct pvrdma_cmd_create_qp_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 qpn; + u32 max_send_wr; + u32 max_recv_wr; + u32 max_send_sge; + u32 max_recv_sge; + u32 max_inline_data; +}; + +struct pvrdma_cmd_modify_qp { + struct pvrdma_cmd_hdr hdr; + u32 qp_handle; + u32 attr_mask; + struct pvrdma_qp_attr attrs; +}; + +struct pvrdma_cmd_query_qp { + struct pvrdma_cmd_hdr hdr; + u32 qp_handle; + u32 attr_mask; +}; + +struct pvrdma_cmd_query_qp_resp { + struct pvrdma_cmd_resp_hdr hdr; + struct pvrdma_qp_attr attrs; +}; + +struct pvrdma_cmd_destroy_qp { + struct pvrdma_cmd_hdr hdr; + u32 qp_handle; + u8 reserved[4]; +}; + +struct pvrdma_cmd_destroy_qp_resp { + struct pvrdma_cmd_resp_hdr hdr; + u32 events_reported; + u8 reserved[4]; +}; + +struct pvrdma_cmd_create_bind { + struct pvrdma_cmd_hdr hdr; + u32 mtu; + u32 vlan; + u32 index; + u8 new_gid[16]; + u8 gid_type; + u8 reserved[3]; +}; + +struct pvrdma_cmd_destroy_bind { + struct pvrdma_cmd_hdr hdr; + u32 index; + u8 dest_gid[16]; + u8 reserved[4]; +}; + +union pvrdma_cmd_req { + struct pvrdma_cmd_hdr hdr; + struct pvrdma_cmd_query_port query_port; + struct pvrdma_cmd_query_pkey query_pkey; + struct pvrdma_cmd_create_uc create_uc; + struct pvrdma_cmd_destroy_uc destroy_uc; + struct pvrdma_cmd_create_pd create_pd; + struct pvrdma_cmd_destroy_pd destroy_pd; + struct pvrdma_cmd_create_mr create_mr; + struct pvrdma_cmd_destroy_mr destroy_mr; + struct pvrdma_cmd_create_cq create_cq; + struct pvrdma_cmd_resize_cq resize_cq; + struct pvrdma_cmd_destroy_cq destroy_cq; + struct pvrdma_cmd_create_qp create_qp; + struct pvrdma_cmd_modify_qp modify_qp; + struct pvrdma_cmd_query_qp query_qp; + struct pvrdma_cmd_destroy_qp destroy_qp; + struct pvrdma_cmd_create_bind create_bind; + struct pvrdma_cmd_destroy_bind destroy_bind; +}; + +union pvrdma_cmd_resp { + struct pvrdma_cmd_resp_hdr hdr; + struct pvrdma_cmd_query_port_resp query_port_resp; + struct pvrdma_cmd_query_pkey_resp query_pkey_resp; + struct pvrdma_cmd_create_uc_resp create_uc_resp; + struct pvrdma_cmd_create_pd_resp create_pd_resp; + struct pvrdma_cmd_create_mr_resp create_mr_resp; + struct pvrdma_cmd_create_cq_resp create_cq_resp; + struct pvrdma_cmd_resize_cq_resp resize_cq_resp; + struct pvrdma_cmd_create_qp_resp create_qp_resp; + struct pvrdma_cmd_query_qp_resp query_qp_resp; + struct pvrdma_cmd_destroy_qp_resp destroy_qp_resp; +}; + +#endif /* __PVRDMA_DEV_API_H__ */