From patchwork Thu Nov 3 23:44:30 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9411643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9F4E460585 for ; Thu, 3 Nov 2016 23:49:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F5F62AF77 for ; Thu, 3 Nov 2016 23:49:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 83D3A2AF9C; Thu, 3 Nov 2016 23:49:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 511272AF77 for ; Thu, 3 Nov 2016 23:49:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933536AbcKCXtN (ORCPT ); Thu, 3 Nov 2016 19:49:13 -0400 Received: from ex13-edg-ou-001.vmware.com ([208.91.0.189]:4115 "EHLO EX13-EDG-OU-001.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933483AbcKCXtL (ORCPT ); Thu, 3 Nov 2016 19:49:11 -0400 Received: from sc9-mailhost1.vmware.com (10.113.161.71) by EX13-EDG-OU-001.vmware.com (10.113.208.155) with Microsoft SMTP Server id 15.0.1156.6; Thu, 3 Nov 2016 16:48:55 -0700 Received: from EX13-CAS-005.vmware.com (ex13-cas-005.vmware.com [10.113.191.55]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id 148E9184A9; Thu, 3 Nov 2016 16:49:11 -0700 (PDT) Received: from EX13-MBX-036.vmware.com (10.113.191.77) by EX13-MBX-006.vmware.com (10.113.191.26) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Thu, 3 Nov 2016 16:48:28 -0700 Received: from EX13-CAS-003.vmware.com (10.113.191.53) by EX13-MBX-036.vmware.com (10.113.191.77) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Thu, 3 Nov 2016 16:48:28 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (10.113.170.11) by EX13-CAS-003.vmware.com (10.113.191.53) with Microsoft SMTP Server (TLS) id 15.0.1156.6 via Frontend Transport; Thu, 3 Nov 2016 16:48:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=g/IpG31RrOk2UA6mVlDl6ZgDVJdzcYu8xFtn91u6e44=; b=Wg2rNguozA82Pf4ZFqHeJvd6fUXJLRIdNKTKQkRROZf7PnUu5K4y/J6Dxn+oh5n98q9S0JmBxGQJlBI7BBfOfdJTMO75RcEqJU+xwknjzC4mAt89TE/Tf+NGSl6qlBd3ZCcsRfm4m1Fuz47QNhfI63s1cOOK3NbmfDmOjGJNGog= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; Received: from gandalf.eng.vmware.com (208.91.1.34) by BLUPR0501MB836.namprd05.prod.outlook.com (10.141.251.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.707.1; Thu, 3 Nov 2016 23:48:20 +0000 From: Adit Ranadive To: , , CC: Adit Ranadive Subject: [PATCH 1/8] libpvrdma: Add ABI and main header files Date: Thu, 3 Nov 2016 16:44:30 -0700 Message-ID: <1478216677-6150-2-git-send-email-aditr@vmware.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1478216677-6150-1-git-send-email-aditr@vmware.com> References: <1478216677-6150-1-git-send-email-aditr@vmware.com> MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: BY2PR04CA052.namprd04.prod.outlook.com (10.141.249.170) To BLUPR0501MB836.namprd05.prod.outlook.com (10.141.251.150) X-MS-Office365-Filtering-Correlation-Id: 8ab09507-b6a7-46ac-f447-08d40443e49c X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 2:7dFtlf83c61WyiJJ/UuEZd60BBV3gpXrcezxdcvpM1ZJjZn5r1q75EVraBevdF/P0mlQQu1tUEi1virgfOpLvpcoNii6LBAXRKRC4/vxLkBxAXuxWFKh0dtwE7ZB9KcE2i8EeRxq2JvFhplv01nMPUQ2I+9xmkkMFFgGASsi9ykeO01nmyDOfbXeeu8C3sLP0W+8GnSe+jFMUS2vWUupgQ==; 3:zE7mmEsTIdWg4cWdw+g9Y7NZEpCJp+EiM/sfiFAZUrhlOXZ4HH3Lyzerx2dIYqn3fNOg6aoKMgZyNhUzS1zFut0d/9WSjVqvQaAR0ez4DNk07g+bvhNkgmgvrWluvfK5NcBghg9UZ5JHHQ2Yu/vKdg== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0501MB836; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 25:HH4TK1qw4hSyrjSI2KyKjKzO/VfJlStqfZDuScoMMKHzClLlQG2t4wWjajEVlsq2VgM5YSR6NoQipa0Y9HrT6gN4AVP7mrbwRwwoIl5I/MkDUXBeZLKGRlBOAgyWIfL/E8NPmPpJMu2MNKYmySfFEEFKmO5FHpLDSFvG9aMMSfPT73c1+xw0usG5RkE5u1yHxHMRARcMpf/BZdWP+xJAchIIfGlu6OIGDxNfAMBHi8nZFWFVsaIQBWWqT74b4BP+zXIp4xYy2ML9Wuf5hgvfHh5eJ1uIqFk3QDWQMrZoYBGwurfQvfk5GzxRrCxl+ayUJMcPbuXeKyZtF99VwOYl+kxKZ2YJDBGDrzruS6oIcJp0TyAmyGvgHiOEEG2BpiHYqqBsS/xF1gcKA/CWyXLld7iP3RojADmFhpMIT/70NMf17hWwTiTRvkK/eBrFpgjtiO+vLD0h42WioNddwfOFtgwhHRiv5pSqzi5wjAcGQTxi1VAfRnDRAwqsmfLbmwZ3Rog0iimMhrGNrvej/mZ8+D4nI/0DCVYU/jDEl+mgYcdcG1fIWMkce9Ld0zEDBbffR7IyM6hkAHmRUaTD5LK9edK4giTAIu/rgpe2cPPA3B6Du7RfDdeqk6qMbl0/y+DAoBLV64/mGmGJgxoPYATKB+KcD2lS/lPFiJX7nhS3SH+gdaupf6tVi+NxRJP3SJOMg2fGxQdKwxJJ74M+mZ3eEVIWyMIh95F+WTlLjkDm51ArQ1UlgEei7LW7kuN7nFz+t3krHLrx/LzilDENLQQA3tCxCdfTZufMapS2K8KsxZM4CuXctZ++l1HuGal/3HV3RYA8IFskXEQh4b+XRJMoXw== X-LD-Processed: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 31:14qNlgSq+x196ihrJfghPk1n+Zu3QAFcWYXLFZJGGzG0plJpMXEipX3ZKhykbiTV6eWFd5TRT4EEqdn0nLPL6KvI8HlkRWzsuiZ6BrbKHhE6IZR9b4+v06UuSaIzcDTLb79gu0PM0zb1u+GDCsHUgOqhyGGVRBRp7+pwRiTL/9TkmVz3CeyHO8jHZdxueO9ZxYakUifIIciz12/TuwdYGfaMI/RMlN8UUQL6D4tWvv3psvVoHh5ojDHFdt7ClsbA; 20:bh0OksUWostd0Ox2ioNVl04Sz/VgqyBoDjWIO3qUXqiMk1s2TTMogrO6/15jSIuO2bOevzkF3j5pfIBZqtau/xGF7jtxfC8Bzu1tfK4PhGan0vIAVwDOlGEX/b8f4+b1Hw/rvvt8zXPb9lG6rTaKbVHDYG9XcvOelglzKkAzN0K6SkU5e8qG3uTv6D4zR+wn6oe92fqbCeabhn/RDXIT2NnSY8CwEBLakkk/CN94ygoIMSC8UcXvKidxx5805tZvOBazPziGb4lMFhqmP+PzPO1bIvAd1Fq1dLK437MV0iVLtvRocXCliiDj1BxqZZtCIYXA69VMqqVLpqVFdNsDXwAjrUJiW5RLueoCWoE5OCyAKshHL0OqometSW3K1QS6SkhQOLf5oTM9xfUY5gIMSuGFTjs1+aZpWh4CBMKkSQTtIZHRwkPoWBNR3N7SQezNHQKywLrRIINclhgvY+5idj+bM4zkV1GcHk3KB5viBzZVkMpM2GBdo2Rwr1RnBt0Q X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(250305191791016)(61668805478150)(22074186197030)(35073007944872); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:BLUPR0501MB836; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0501MB836; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 4:CxTAJd51+HCYnAAgwmMCwBbR1nLrxCsPFJxV9C/dGQCrshCvo6DHOhZkX1hs+0MnXiVGrBEobPvIPeScWqwIC2jwHn3ovDX3VejiQ8+Z1u9OJjhez2ebGE1PxQJvHJHMgvHO37k2hAxnJd3rEaJNLUCRh90P8uwk9sVzh1aKV3nmYS5KkAjV275hT8bKuG6gKVLrS00dT+URenNxdl5/52Rlm0QAF+9m3HkODw+wANHx3TKdZu7O+Silf9wG2SbizDsjcKBwqMaIqDJFMrRJuWAA780zE1nvvVFHiNBBj5tALKVSz5R6eM3rwWiH0YhryVOjlYB5cAc/rhvPzOeqajrvTreQVE5TkeQoFw34JpL1nZRHzH2bAu1E3dF4t9TnH0jzAOHmwE2sTd1UQzJ5ysyn5y2jv5O0oaAtXz+Stx3D/ebq51oAw7OQuSmlyVg2nLizliAlzIENwzwvcBI4SIJuF5GwlBoeuIK3w7W9rEOtuHu5VRs4wkQ3Vip7BhakTQIJc8u6J884xKKgjpWk3M8Ac5jnjjtGTTYsRcbpRts= X-Forefront-PRVS: 011579F31F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(15975445007)(8676002)(6116002)(5003940100001)(106356001)(5001770100001)(97736004)(3846002)(586003)(107886002)(19580395003)(19580405001)(81156014)(81166006)(189998001)(66066001)(47776003)(4001430100002)(50226002)(7846002)(92566002)(7736002)(305945005)(575784001)(2201001)(101416001)(6666003)(42186005)(50986999)(6636002)(86362001)(77096005)(6862003)(48376002)(50466002)(105586002)(76176999)(2950100002)(2906002)(4326007)(68736007)(36756003)(5660300001)(229853001)(33646002)(7099028)(2004002)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0501MB836; H:gandalf.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (EX13-EDG-OU-001.vmware.com: aditr@vmware.com does not designate permitted sender hosts) Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0501MB836; 23:QLJ2ywijLcREMOjUaTepAFOady8Lo1flhwqI4fd2?= =?us-ascii?Q?pCHOAzb6mzSOt6+roao02ETkoMRuw3GR5T4xP5XMgJYPBo0TCmqbaOQNjW70?= =?us-ascii?Q?tzxCOHmZ/zdUEXbdiNwGkg752ooszbNph5YaeK9un4UNHkV8Ttmyjmkf/WLx?= =?us-ascii?Q?c2A31vZUoKAXHgRxRrIaXHyhAu0WQnf4C6DowYEgrfIlqQGV9sx0+GWgZKRy?= =?us-ascii?Q?Uds5gvHn588cmuMD/3MYnuEYBv7YXA6/ymJ62itC6jloJYoAsUrKRCJKZ83O?= =?us-ascii?Q?vtu3g2oTMw2TZLbQAtc/EGeXrVVgHFxSQRu40TIFFbBLuJJihoTGh1i68Sxn?= =?us-ascii?Q?/d3ltYchah1SKHdi4x8qnJiYjphtX8Cw0SpeM3zX++hdnUrAgKsoktuvbX+u?= =?us-ascii?Q?D/5Sa6+I/mno4szbKb9rfotoyyrRmz72n4Yu1kY/wS02/ALwAEvBMF07R3vH?= =?us-ascii?Q?xtPE8UC5v2kmIWMJTais2ligixgS8vZsSDdJGt29/bc9yf4Yo5MiK3Z3tPbe?= =?us-ascii?Q?khyD0+7usWh56ySo0DV05v60OjoC3DJhNxGzSEPSR2rA6GgF+GVIRz54qxIU?= =?us-ascii?Q?DIPcsPzFLWHNFHl88DnPxwWNnb42o1d2xHCc4yYuWPCsRHiKxcui+vwmmMk8?= =?us-ascii?Q?eLoM947D9TjAX7GKTZjOyQgOcpPKxy2/2gnDx7+DNG8S4iNLzjqr8nc/v0Mm?= =?us-ascii?Q?cuNPpw2fCkLLfcdJlwyMA3lHH66uHWUvEYptmiGnBNDFGUlJWiaEy+lVJYte?= =?us-ascii?Q?BOwPu2JGokD1ejyu0af4i0wmNBCFQarbWHFGsCAdwWylJ69IlNrdK7Fy9EUj?= =?us-ascii?Q?Y0NkMFCz6PjbBnL3Kh9/QU5YXKkV/uBWMyOS7ylC6r5JNCv1QGLInnTRUBiz?= =?us-ascii?Q?yqG1tJaql2kAIwnchiTAmznWEGB15cDVS+Zue3K63SGEVXqwxURmysioNU2X?= =?us-ascii?Q?0+yM6zJdZ/oOc7CBIfUljHuvzizD8jiBfM3qkh0ojzX3Zlkss+gt6OuYAOoS?= =?us-ascii?Q?X8JLcMw+UdKzel4x5hPkDNhA3mWrx9kLP0vOmD/DMZMLk2HMEjD9Gk34g1n/?= =?us-ascii?Q?pOK9TOhLwoDXv8SfNlf1zyYM7aHFRzFDtWJSBbhlX8w8lVVVqcdqcDE4ody1?= =?us-ascii?Q?j7ThndWPr12rMrt7pMakXjqy5oet7KgE/U80AQGQrqUSdA4QnST8oORPJrrA?= =?us-ascii?Q?yi0WGFXUG7vb2PZlIbnMfIaG0BKShPC//H73gglgelShnwwnj0+xoN91Qs9A?= =?us-ascii?Q?eeVIMxiUz4npw/UUywfQkIM8+wppG5TQyNq3dusmz8hCKFqp8Gvg1nDVVEJ0?= =?us-ascii?Q?0A=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 6:nKctEaBxUBJ1BXRaoAZ9I8vzQcr4z+4SFoBSF0prP1APDZCOIL3L2tJzoYj7SteCjXfSrow43TrydbfrQPN5qkjMhtGboTkkEaNt0Rd3iCEOMzn9RdF6GnXpP3gkgIrqWnPIuDly2tQR9Yz5c5YnXeq3nDzewdaoD0gVm5IZjuhgM1QJCgXuU1NtChW1wc5j3Q6roJxgNzkQYVk5fK6DLg2s3KF7dPFq/mDYLGE6EbZXzFH3vgVZVQjHR7RzHJzues/D5ccS1vhj39xQ6khDdZpmOGDDWxTGTXgi0nhPzJBQNxvEUH7RKPx74sqc4/SR; 5:I6CcOrAnpvE0vXPkkp2gnDC3R8N6Y8ZR/f3KcO2ina4ajGvn0VQ8YR5hTudfLPeTCh8hnJuLy3MaB0cgn68OWdrN0VOXL+H6+tvi6EXpVZWLIAmaHA4kCuQq0PcLwB0rjtTe7krZGkaGYBDs5G4DWMBEYyySBbrzMVeyTrnLtBg=; 24:0WfPFabyQU3T67gSDkxizsZfPKNvfyBoNGtl6QTh46QOJPsFTtlw7a38ZTmWtV4Tb/NmqLN44daDR+2LXEo1/IiDc+d+ARVyBBHX9nzM9zI= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BLUPR0501MB836; 7:bZ2oKRi2x2i+gCe1WzAoihc+Qp3rMA9qSOCPfx6d+rRFE4kL6G8SFYk8zzoJCQV4JRWTVnjZqpqv0vnX0lt8kg03M978a6aHlvEhDaPNFCSa7PJoWfwwG8Bpvdxg1KPqGyZrdevH7XoTgr2H9bve7r0CwK0xjURaAxaJ/3STwHE/lCS+IbqgbDZQmz/6cjeqdXDDziKZmwpxGURSFQcplg51vwadpY9PCdKYMrWaBy1DUNL4LJ/u3GlAUzLCnp4TGnlfVM73OerrdVQ4T63dpRih1PjwOO8l2/n7D8vVEJ9eeXLFFr6HUtXFE85w7bpQcuq7JowFTnNVxe30SZ5Enl48T31DMZ5AjyHIjyIcZf8=; 20:MzUH3X7fDBwHU/LwmEWlyCDmqFD8q9VMthMbS0JCh9m0j6MP1SZnT9K13kHwaHMhbXl4aYGyOYODUMzck131b2tdI8o877Kqbrg1NLWjWPVJmmxK2lqyQOHyjZta5JW/P2Mlc3DF6i3AyFfVsapoz4qzcNJNCrGww5wGuvvLmSs= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2016 23:48:20.5029 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0501MB836 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add all the structures and types to interact with kernel driver. Signed-off-by: Adit Ranadive --- providers/pvrdma/pvrdma-abi.h | 297 ++++++++++++++++++++++++++++++++++++ providers/pvrdma/pvrdma.h | 347 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 644 insertions(+) create mode 100644 providers/pvrdma/pvrdma-abi.h create mode 100644 providers/pvrdma/pvrdma.h diff --git a/providers/pvrdma/pvrdma-abi.h b/providers/pvrdma/pvrdma-abi.h new file mode 100644 index 0000000..c7a38c5 --- /dev/null +++ b/providers/pvrdma/pvrdma-abi.h @@ -0,0 +1,297 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_ABI_H__ +#define __PVRDMA_ABI_H__ + +#include + +#define PVRDMA_UVERBS_ABI_VERSION 3 +#define PVRDMA_UAR_HANDLE_MASK 0x00FFFFFF /* Bottom 24 bits. */ +#define PVRDMA_UAR_QP_OFFSET 0 /* QP doorbell offset. */ +#define PVRDMA_UAR_QP_SEND BIT(30) /* Send bit. */ +#define PVRDMA_UAR_QP_RECV BIT(31) /* Recv bit. */ +#define PVRDMA_UAR_CQ_OFFSET 4 /* CQ doorbell offset. */ +#define PVRDMA_UAR_CQ_ARM_SOL BIT(29) /* Arm solicited bit. */ +#define PVRDMA_UAR_CQ_ARM BIT(30) /* Arm bit. */ +#define PVRDMA_UAR_CQ_POLL BIT(31) /* Poll bit. */ + +enum pvrdma_wr_opcode { + PVRDMA_WR_RDMA_WRITE, + PVRDMA_WR_RDMA_WRITE_WITH_IMM, + PVRDMA_WR_SEND, + PVRDMA_WR_SEND_WITH_IMM, + PVRDMA_WR_RDMA_READ, + PVRDMA_WR_ATOMIC_CMP_AND_SWP, + PVRDMA_WR_ATOMIC_FETCH_AND_ADD, + PVRDMA_WR_LSO, + PVRDMA_WR_SEND_WITH_INV, + PVRDMA_WR_RDMA_READ_WITH_INV, + PVRDMA_WR_LOCAL_INV, + PVRDMA_WR_FAST_REG_MR, + PVRDMA_WR_MASKED_ATOMIC_CMP_AND_SWP, + PVRDMA_WR_MASKED_ATOMIC_FETCH_AND_ADD, + PVRDMA_WR_BIND_MW, + PVRDMA_WR_REG_SIG_MR, +}; + +enum pvrdma_wc_status { + PVRDMA_WC_SUCCESS, + PVRDMA_WC_LOC_LEN_ERR, + PVRDMA_WC_LOC_QP_OP_ERR, + PVRDMA_WC_LOC_EEC_OP_ERR, + PVRDMA_WC_LOC_PROT_ERR, + PVRDMA_WC_WR_FLUSH_ERR, + PVRDMA_WC_MW_BIND_ERR, + PVRDMA_WC_BAD_RESP_ERR, + PVRDMA_WC_LOC_ACCESS_ERR, + PVRDMA_WC_REM_INV_REQ_ERR, + PVRDMA_WC_REM_ACCESS_ERR, + PVRDMA_WC_REM_OP_ERR, + PVRDMA_WC_RETRY_EXC_ERR, + PVRDMA_WC_RNR_RETRY_EXC_ERR, + PVRDMA_WC_LOC_RDD_VIOL_ERR, + PVRDMA_WC_REM_INV_RD_REQ_ERR, + PVRDMA_WC_REM_ABORT_ERR, + PVRDMA_WC_INV_EECN_ERR, + PVRDMA_WC_INV_EEC_STATE_ERR, + PVRDMA_WC_FATAL_ERR, + PVRDMA_WC_RESP_TIMEOUT_ERR, + PVRDMA_WC_GENERAL_ERR, +}; + +enum pvrdma_wc_opcode { + PVRDMA_WC_SEND, + PVRDMA_WC_RDMA_WRITE, + PVRDMA_WC_RDMA_READ, + PVRDMA_WC_COMP_SWAP, + PVRDMA_WC_FETCH_ADD, + PVRDMA_WC_BIND_MW, + PVRDMA_WC_LSO, + PVRDMA_WC_LOCAL_INV, + PVRDMA_WC_FAST_REG_MR, + PVRDMA_WC_MASKED_COMP_SWAP, + PVRDMA_WC_MASKED_FETCH_ADD, + PVRDMA_WC_RECV = 1 << 7, + PVRDMA_WC_RECV_RDMA_WITH_IMM, +}; + +enum pvrdma_wc_flags { + PVRDMA_WC_GRH = 1 << 0, + PVRDMA_WC_WITH_IMM = 1 << 1, + PVRDMA_WC_WITH_INVALIDATE = 1 << 2, + PVRDMA_WC_IP_CSUM_OK = 1 << 3, + PVRDMA_WC_WITH_SMAC = 1 << 4, + PVRDMA_WC_WITH_VLAN = 1 << 5, + PVRDMA_WC_FLAGS_MAX = PVRDMA_WC_WITH_VLAN, +}; + +struct pvrdma_alloc_ucontext_resp { + struct ibv_get_context_resp ibv_resp; + __u32 qp_tab_size; + __u32 reserved; +}; + +struct pvrdma_alloc_pd_resp { + struct ibv_alloc_pd_resp ibv_resp; + __u32 pdn; + __u32 reserved; +}; + +struct pvrdma_create_cq { + struct ibv_create_cq ibv_cmd; + __u64 buf_addr; + __u32 buf_size; + __u32 reserved; +}; + +struct pvrdma_create_cq_resp { + struct ibv_create_cq_resp ibv_resp; + __u32 cqn; + __u32 reserved; +}; + +struct pvrdma_resize_cq { + struct ibv_resize_cq ibv_cmd; + __u64 buf_addr; + __u32 buf_size; + __u32 reserved; +}; + +struct pvrdma_create_srq { + struct ibv_create_srq ibv_cmd; + __u64 buf_addr; +}; + +struct pvrdma_create_srq_resp { + struct ibv_create_srq_resp ibv_resp; + __u32 srqn; + __u32 reserved; +}; + +struct pvrdma_create_qp { + struct ibv_create_qp ibv_cmd; + __u64 rbuf_addr; + __u64 sbuf_addr; + __u32 rbuf_size; + __u32 sbuf_size; + __u64 qp_addr; +}; + +/* PVRDMA masked atomic compare and swap */ +struct pvrdma_ex_cmp_swap { + __u64 swap_val; + __u64 compare_val; + __u64 swap_mask; + __u64 compare_mask; +}; + +/* PVRDMA masked atomic fetch and add */ +struct pvrdma_ex_fetch_add { + __u64 add_val; + __u64 field_boundary; +}; + +/* PVRDMA address vector. */ +struct pvrdma_av { + __u32 port_pd; + __u32 sl_tclass_flowlabel; + __u8 dgid[16]; + __u8 src_path_bits; + __u8 gid_index; + __u8 stat_rate; + __u8 hop_limit; + __u8 dmac[6]; + __u8 reserved[6]; +}; + +/* PVRDMA scatter/gather entry */ +struct pvrdma_sge { + __u64 addr; + __u32 length; + __u32 lkey; +}; + +/* PVRDMA receive queue work request */ +struct pvrdma_rq_wqe_hdr { + __u64 wr_id; /* wr id */ + __u32 num_sge; /* size of s/g array */ + __u32 total_len; /* reserved */ +}; +/* Use pvrdma_sge (ib_sge) for receive queue s/g array elements. */ + +/* PVRDMA send queue work request */ +struct pvrdma_sq_wqe_hdr { + __u64 wr_id; /* wr id */ + __u32 num_sge; /* size of s/g array */ + __u32 total_len; /* reserved */ + __u32 opcode; /* operation type */ + __u32 send_flags; /* wr flags */ + union { + __u32 imm_data; + __u32 invalidate_rkey; + } ex; + __u32 reserved; + union { + struct { + __u64 remote_addr; + __u32 rkey; + __u8 reserved[4]; + } rdma; + struct { + __u64 remote_addr; + __u64 compare_add; + __u64 swap; + __u32 rkey; + __u32 reserved; + } atomic; + struct { + __u64 remote_addr; + __u32 log_arg_sz; + __u32 rkey; + union { + struct pvrdma_ex_cmp_swap cmp_swap; + struct pvrdma_ex_fetch_add fetch_add; + } wr_data; + } masked_atomics; + struct { + __u64 iova_start; + __u64 pl_pdir_dma; + __u32 page_shift; + __u32 page_list_len; + __u32 length; + __u32 access_flags; + __u32 rkey; + } fast_reg; + struct { + __u32 remote_qpn; + __u32 remote_qkey; + struct pvrdma_av av; + } ud; + } wr; +}; +/* Use pvrdma_sge (ib_sge) for send queue s/g array elements. */ + +/* Completion queue element. */ +struct pvrdma_cqe { + __u64 wr_id; + __u64 qp; + __u32 opcode; + __u32 status; + __u32 byte_len; + __u32 imm_data; + __u32 src_qp; + __u32 wc_flags; + __u32 vendor_err; + __u16 pkey_index; + __u16 slid; + __u8 sl; + __u8 dlid_path_bits; + __u8 port_num; + __u8 smac[6]; + __u8 reserved2[7]; /* Pad to next power of 2 (64). */ +}; + +#endif /* __PVRDMA_ABI_H__ */ diff --git a/providers/pvrdma/pvrdma.h b/providers/pvrdma/pvrdma.h new file mode 100644 index 0000000..d3df07d --- /dev/null +++ b/providers/pvrdma/pvrdma.h @@ -0,0 +1,347 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_H__ +#define __PVRDMA_H__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define BIT(nr) (1UL << (nr)) + +#include "pvrdma-abi.h" +#include "pvrdma_ring.h" + +#ifndef rmb +# define rmb() mb() +#endif + +#ifndef wmb +# define wmb() mb() +#endif + +#ifndef likely +#define likely(x) __builtin_expect(!!(x), 1) +#else +#define likely(x) (x) +#endif + +#ifndef unlikely +#define unlikely(x) __builtin_expect(!!(x), 0) +#else +#define unlikely(x) (x) +#endif + +#ifndef max +#define max(a, b) \ + ({ typeof (a) _a = (a); \ + typeof (b) _b = (b); \ + _a > _b ? _a : _b; }) +#endif + +#ifndef min +#define min(a,b) \ + ({ typeof (a) _a = (a); \ + typeof (b) _b = (b); \ + _a < _b ? _a : _b; }) +#endif + +#define PFX "pvrdma: " + +enum { + PVRDMA_OPCODE_NOP = 0x00, + PVRDMA_OPCODE_SEND_INVAL = 0x01, + PVRDMA_OPCODE_RDMA_WRITE = 0x08, + PVRDMA_OPCODE_RDMA_WRITE_IMM = 0x09, + PVRDMA_OPCODE_SEND = 0x0a, + PVRDMA_OPCODE_SEND_IMM = 0x0b, + PVRDMA_OPCODE_LSO = 0x0e, + PVRDMA_OPCODE_RDMA_READ = 0x10, + PVRDMA_OPCODE_ATOMIC_CS = 0x11, + PVRDMA_OPCODE_ATOMIC_FA = 0x12, + PVRDMA_OPCODE_ATOMIC_MASK_CS = 0x14, + PVRDMA_OPCODE_ATOMIC_MASK_FA = 0x15, + PVRDMA_OPCODE_BIND_MW = 0x18, + PVRDMA_OPCODE_FMR = 0x19, + PVRDMA_OPCODE_LOCAL_INVAL = 0x1b, + PVRDMA_OPCODE_CONFIG_CMD = 0x1f, + + PVRDMA_RECV_OPCODE_RDMA_WRITE_IMM = 0x00, + PVRDMA_RECV_OPCODE_SEND = 0x01, + PVRDMA_RECV_OPCODE_SEND_IMM = 0x02, + PVRDMA_RECV_OPCODE_SEND_INVAL = 0x03, + + PVRDMA_CQE_OPCODE_ERROR = 0x1e, + PVRDMA_CQE_OPCODE_RESIZE = 0x16, +}; + +enum { + PVRDMA_WQE_CTRL_FENCE = 1 << 6, + PVRDMA_WQE_CTRL_CQ_UPDATE = 3 << 2, + PVRDMA_WQE_CTRL_SOLICIT = 1 << 1, +}; + +struct pvrdma_device { + struct ibv_device ibv_dev; + int page_size; + int abi_version; +}; + +struct pvrdma_context { + struct ibv_context ibv_ctx; + void *uar; + pthread_spinlock_t uar_lock; + int max_qp_wr; + int max_sge; + int max_cqe; + struct pvrdma_qp **qp_tbl; +}; + +struct pvrdma_buf { + void *buf; + size_t length; +}; + +struct pvrdma_pd { + struct ibv_pd ibv_pd; + uint32_t pdn; +}; + +struct pvrdma_cq { + struct ibv_cq ibv_cq; + struct pvrdma_buf buf; + struct pvrdma_buf resize_buf; + pthread_spinlock_t lock; + struct pvrdma_ring_state *ring_state; + uint32_t cqe_cnt; + uint32_t offset; + uint32_t cqn; +}; + +struct pvrdma_wq { + uint64_t *wrid; + pthread_spinlock_t lock; + int wqe_cnt; + int wqe_size; + struct pvrdma_ring *ring_state; + int max_gs; + int wqe_shift; + int offset; +}; + +struct pvrdma_qp { + struct ibv_qp ibv_qp; + struct pvrdma_buf rbuf; + struct pvrdma_buf sbuf; + int max_inline_data; + int buf_size; + uint32_t sq_signal_bits; + int sq_spare_wqes; + struct pvrdma_wq sq; + struct pvrdma_wq rq; +}; + +struct pvrdma_ah { + struct ibv_ah ibv_ah; + struct pvrdma_av av; +}; + +static inline unsigned long align(unsigned long val, unsigned long align) +{ + return (val + align - 1) & ~(align - 1); +} + +static inline int align_next_power2(int size) +{ + int val = 1; + + while (val < size) + val <<= 1; + + return val; +} + +#define to_vxxx(xxx, type) \ + ((struct pvrdma_##type *) \ + ((void *) ib##xxx - offsetof(struct pvrdma_##type, ibv_##xxx))) + +static inline struct pvrdma_device *to_vdev(struct ibv_device *ibdev) +{ + return to_vxxx(dev, device); +} + +static inline struct pvrdma_context *to_vctx(struct ibv_context *ibctx) +{ + return to_vxxx(ctx, context); +} + +static inline struct pvrdma_pd *to_vpd(struct ibv_pd *ibpd) +{ + return to_vxxx(pd, pd); +} + +static inline struct pvrdma_cq *to_vcq(struct ibv_cq *ibcq) +{ + return to_vxxx(cq, cq); +} + +static inline struct pvrdma_qp *to_vqp(struct ibv_qp *ibqp) +{ + return to_vxxx(qp, qp); +} + +static inline struct pvrdma_ah *to_vah(struct ibv_ah *ibah) +{ + return to_vxxx(ah, ah); +} + +static inline void pvrdma_write_uar_qp(void *uar, unsigned value) +{ + *(uint32_t *)(uar + PVRDMA_UAR_QP_OFFSET) = htole32(value); +} + +static inline void pvrdma_write_uar_cq(void *uar, unsigned value) +{ + *(uint32_t *)(uar + PVRDMA_UAR_CQ_OFFSET) = htole32(value); +} + +static inline int ibv_send_flags_to_pvrdma(int flags) +{ + return flags; +} + +static inline enum pvrdma_wr_opcode ibv_wr_opcode_to_pvrdma( + enum ibv_wr_opcode op) +{ + return (enum pvrdma_wr_opcode)op; +} + +static inline enum ibv_wc_status pvrdma_wc_status_to_ibv( + enum pvrdma_wc_status status) +{ + return (enum ibv_wc_status)status; +} + +static inline enum ibv_wc_opcode pvrdma_wc_opcode_to_ibv( + enum pvrdma_wc_opcode op) +{ + return (enum ibv_wc_opcode)op; +} + +static inline int pvrdma_wc_flags_to_ibv(int flags) +{ + return flags; +} + +int pvrdma_alloc_buf(struct pvrdma_buf *buf, size_t size, int page_size); +void pvrdma_free_buf(struct pvrdma_buf *buf); + +int pvrdma_query_device(struct ibv_context *context, + struct ibv_device_attr *attr); +int pvrdma_query_port(struct ibv_context *context, uint8_t port, + struct ibv_port_attr *attr); + +struct ibv_pd *pvrdma_alloc_pd(struct ibv_context *context); +int pvrdma_free_pd(struct ibv_pd *pd); + +struct ibv_mr *pvrdma_reg_mr(struct ibv_pd *pd, void *addr, + size_t length, int access); +int pvrdma_dereg_mr(struct ibv_mr *mr); + +struct ibv_cq *pvrdma_create_cq(struct ibv_context *context, int cqe, + struct ibv_comp_channel *channel, + int comp_vector); +int pvrdma_alloc_cq_buf(struct pvrdma_device *dev, struct pvrdma_cq *cq, + struct pvrdma_buf *buf, int nent); +int pvrdma_destroy_cq(struct ibv_cq *cq); +int pvrdma_req_notify_cq(struct ibv_cq *cq, int solicited); +int pvrdma_poll_cq(struct ibv_cq *cq, int ne, struct ibv_wc *wc); +void pvrdma_cq_event(struct ibv_cq *cq); +void __pvrdma_cq_clean(struct pvrdma_cq *cq, uint32_t qpn); +void pvrdma_cq_clean(struct pvrdma_cq *cq, uint32_t qpn); +int pvrdma_get_outstanding_cqes(struct pvrdma_cq *cq); +void pvrdma_cq_resize_copy_cqes(struct pvrdma_cq *cq, void *buf, + int new_cqe); + +struct ibv_qp *pvrdma_create_qp(struct ibv_pd *pd, + struct ibv_qp_init_attr *attr); +int pvrdma_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, + int attr_mask, struct ibv_qp_init_attr *init_attr); +int pvrdma_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, + int attr_mask); +int pvrdma_destroy_qp(struct ibv_qp *qp); +void pvrdma_init_qp_indices(struct pvrdma_qp *qp); +void pvrdma_qp_init_sq_ownership(struct pvrdma_qp *qp); +int pvrdma_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, + struct ibv_send_wr **bad_wr); +int pvrdma_post_recv(struct ibv_qp *ibqp, struct ibv_recv_wr *wr, + struct ibv_recv_wr **bad_wr); +void pvrdma_calc_sq_wqe_size(struct ibv_qp_cap *cap, enum ibv_qp_type type, + struct pvrdma_qp *qp); +int pvrdma_alloc_qp_buf(struct pvrdma_device *dev, struct ibv_qp_cap *cap, + enum ibv_qp_type type, struct pvrdma_qp *qp); +void pvrdma_set_sq_sizes(struct pvrdma_qp *qp, struct ibv_qp_cap *cap, + enum ibv_qp_type type); +struct pvrdma_qp *pvrdma_find_qp(struct pvrdma_context *ctx, + uint32_t qpn); +int pvrdma_store_qp(struct pvrdma_context *ctx, uint32_t qpn, + struct pvrdma_qp *qp); +void pvrdma_clear_qp(struct pvrdma_context *ctx, uint32_t qpn); + +struct ibv_ah *pvrdma_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr); +int pvrdma_destroy_ah(struct ibv_ah *ah); + +int pvrdma_alloc_av(struct pvrdma_pd *pd, struct ibv_ah_attr *attr, + struct pvrdma_ah *ah); +void pvrdma_free_av(struct pvrdma_ah *ah); + +#endif /* __PVRDMA_H__ */