From patchwork Thu Oct 24 06:00:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11208325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8917D913 for ; Thu, 24 Oct 2019 06:00:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5CAC420856 for ; Thu, 24 Oct 2019 06:00:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="quwuPs78" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437661AbfJXGAw (ORCPT ); Thu, 24 Oct 2019 02:00:52 -0400 Received: from mail-eopbgr50068.outbound.protection.outlook.com ([40.107.5.68]:21413 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2437663AbfJXGAv (ORCPT ); Thu, 24 Oct 2019 02:00:51 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hK9yWNh1gDYC29oHyp8kKgaayNZW6bZ2FyWhzByKlLbNkf8fxpgkIqIiN3fof3Gk5BrJRHdzixkjnbeQyk9pu//SqQ7Wz0ErjNeji3o3rhIUbjcd1AUBRpEhpwYgt9VEs+tn2N3GiOpOP/+NpHdkaIBo5WjzWgLPROGS+Ft2TRjj0fp4AnogHGgZc5k/FSCcbfHEu/6Ap/kG+l1EwTmGNJ17wPI6qR7WqfRVeppCiDab1s5PJbf0G8r73jJoNKHIRC1jyaVnzKC2VitRp/QzpkhzKPEzRcuKvFvPTWZsaZjlVYjNMKuHWBJN2YbGQgj0DK+7ff9UUYWPm1GbrXF9MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LMzC2j8w0nMWxoSUxWSkRNp7TbYN6wWoeK7RW+TdxvA=; b=haorfkDigvphUD2jywPCXVhYo90Wj6vmi5KBAHsyarDK2eeE+cYDlcJ1ItcaDSj0vE5e8gYN6A93sVkNanB7wQQRzsHN86u1icgaJ3XU+4uFGOC9HQX7u1alvm2lFzBQH2Ze3+aJ3iBopY5ZTKosuI/jwdkQVJJHFV91QTZ1nWVRW0V9o8AVAN7tGrEK3n/y9E9LCfWu3zUxOt7lrvpGPPCnaNIO8KHMSzrcyzgIbJEdUC2j8eCmbxyaPWfHpZuVz68TviWVGvquzgZuM657bBSvB9LgTjlr3jBi8gotiqNvOAJ0tTemRj/yJTdDMTN+ROsFB+/+v6/SXgI8aQJm/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LMzC2j8w0nMWxoSUxWSkRNp7TbYN6wWoeK7RW+TdxvA=; b=quwuPs78dDtsnx4Z9hOtZnC6hMwrJ+n9D5qaZG//qWrhkT3aqfrz2uFwlDYMT01Bqv6l3pzlPRI2pQBqVfJrjXnD/hhvzqcahNRwgdjUDlf4C3IkXIOxnNZqNZeu6gv/bvYo9z+3UZEDCohqRLqGubHGIjSHbHz5DPrYWKbc6TE= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB4182.eurprd05.prod.outlook.com (52.135.164.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Thu, 24 Oct 2019 06:00:47 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a%5]) with mapi id 15.20.2347.029; Thu, 24 Oct 2019 06:00:47 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich Subject: [PATCH rdma-core 1/4] pyverbs: Add support for providers' context Thread-Topic: [PATCH rdma-core 1/4] pyverbs: Add support for providers' context Thread-Index: AQHVijBh7O5I5ByACUyBUhUUPcx13g== Date: Thu, 24 Oct 2019 06:00:47 +0000 Message-ID: <20191024060027.8696-2-noaos@mellanox.com> References: <20191024060027.8696-1-noaos@mellanox.com> In-Reply-To: <20191024060027.8696-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: AM0PR05CA0081.eurprd05.prod.outlook.com (2603:10a6:208:136::21) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 34f912d1-d275-44c6-1a94-08d758478388 x-ms-traffictypediagnostic: AM6PR05MB4182:|AM6PR05MB4182: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0200DDA8BE x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(39860400002)(376002)(366004)(346002)(396003)(199004)(189003)(478600001)(71200400001)(86362001)(486006)(71190400001)(2501003)(11346002)(2616005)(446003)(52116002)(476003)(4326008)(76176011)(110136005)(54906003)(66946007)(66476007)(66556008)(66446008)(64756008)(6636002)(14454004)(99286004)(107886003)(26005)(186003)(305945005)(7736002)(316002)(66066001)(6506007)(386003)(3846002)(36756003)(14444005)(256004)(102836004)(8936002)(50226002)(1076003)(6116002)(2906002)(81156014)(81166006)(6436002)(6512007)(6486002)(5660300002)(25786009)(8676002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB4182;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4flK3L7Jh8oAMayaLHsAXERArxVqRNQ52OshnMi9+nP0XzZP7PlTZpBK8rM99cIIZXCF8l+h2DJSzuorIgp2zp6KSxSMbSqNkv+9mXXxcujhvMvyngCdDz+Ci9u/GLeh9opRmxOtBAQi234l4mJnNi2BIQgERvx2bgRRYE87hwIDbV+DunZdFfPHcK/DWgZwS3l8YCEgNQL7Bw6js10ITLEyI7qYoQw443deOIOnXSXUofFw1zPjVdvTxx7X9aM4IIJ1B0jZUCC+EUHEwN/Nj+gPVuOkptwrtYJ1JsN5tW4O3TViDN1a/YwY4TORQMCcGv8FVFT+u1HWx5RrCdkCt3CWGwIcaUxuQ+kyt0iDpZ/WntjSYOM74bWq3e8jKxSlx7fjkJ2RxAKDs2jDkAc8ahkaDjc2xgk85NUQW7mS4Bz5q0aw56aVQe3WRhSMiR/O MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 34f912d1-d275-44c6-1a94-08d758478388 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2019 06:00:47.2239 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ny93vdmwUkCvd9kEjtOTAkYNjNicN3/g4PRMm9oca12zFL9Cvh3s6Mj/kcOL3SaIVNPk67qBT4a+T+MW3okglA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4182 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Providers can supply an API for a driver-specific context, extending ibv_context. To allow the provider to open the device given provider-specific attributes, Context class will only locate the right ibv_device but will not open it, leaving the provider to open it in any way it finds fit. Signed-off-by: Noa Osherovich --- pyverbs/device.pxd | 1 + pyverbs/device.pyx | 17 +++++++++++++---- pyverbs/qp.pyx | 2 +- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/pyverbs/device.pxd b/pyverbs/device.pxd index 5cc13acb4331..a701a3deeabd 100755 --- a/pyverbs/device.pxd +++ b/pyverbs/device.pxd @@ -9,6 +9,7 @@ cimport pyverbs.libibverbs as v cdef class Context(PyverbsCM): cdef v.ibv_context *context + cdef v.ibv_device *device cdef object name cdef add_ref(self, obj) cdef object pds diff --git a/pyverbs/device.pyx b/pyverbs/device.pyx index 084086bdbc69..58a2aca27fcc 100755 --- a/pyverbs/device.pyx +++ b/pyverbs/device.pyx @@ -78,9 +78,13 @@ cdef class Context(PyverbsCM): """ Initializes a Context object. The function searches the IB devices list for a device with the name provided by the user. If such a device is - found, it is opened. - :param kwargs: Currently supports 'name' argument only, the IB device's - name. + found, it is opened (unless provider attributes were given). + :param kwargs: Arguments: + * *name* (str) + The RDMA device's name + * *attr* (object) + Device-specific attributes, meaning that the device is to be + opened by the provider :return: None """ cdef int count @@ -94,7 +98,7 @@ cdef class Context(PyverbsCM): self.xrcds = weakref.WeakSet() dev_name = kwargs.get('name') - + provider_attr = kwargs.get('attr') if dev_name is not None: self.name = dev_name else: @@ -106,6 +110,11 @@ cdef class Context(PyverbsCM): try: for i in range(count): if dev_list[i].name.decode() == self.name: + if provider_attr is not None: + # A provider opens its own context, we're just + # setting its IB device + self.device = dev_list[i] + return self.context = v.ibv_open_device(dev_list[i]) if self.context == NULL: raise PyverbsRDMAErrno('Failed to open device {dev}'. diff --git a/pyverbs/qp.pyx b/pyverbs/qp.pyx index 961e0d59c589..9167e5f2bd01 100755 --- a/pyverbs/qp.pyx +++ b/pyverbs/qp.pyx @@ -812,7 +812,7 @@ cdef class QP(PyverbsCM): self.update_cqs(init_attr) # In order to use cdef'd methods, a proper casting must be done, let's # infer the type. - if type(creator) == Context: + if issubclass(type(creator), Context): self._create_qp_ex(creator, init_attr) ctx = creator self.context = ctx From patchwork Thu Oct 24 06:00:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11208327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B20C913 for ; Thu, 24 Oct 2019 06:00:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 514E62166E for ; Thu, 24 Oct 2019 06:00:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="X5vdN/Bp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437665AbfJXGAz (ORCPT ); Thu, 24 Oct 2019 02:00:55 -0400 Received: from mail-eopbgr50068.outbound.protection.outlook.com ([40.107.5.68]:21413 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2437466AbfJXGAy (ORCPT ); Thu, 24 Oct 2019 02:00:54 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fSxKf+CmSuxxZAMpF9fWS8rPFCTrOOFoyPv+IzuVixnMRtskjvbyjoqa1gZFFJxdRP6H2zWx7l3Aokhz+HPiwPzpizr+2hWhGM3gcnruhMcHOL3m3IXhPvQcgR+EXeqT/npi9Oj/SbnZxUtrBZFwXjATwQIQT8TYpOKQCVCJafKAKZbm2ZHp28M3JkPUrwFXP6kKDGsYPBYCcjXPoJJ5ssIoGcI/SqLZ7M5T2VTDWvVRuTfGGYGC8PkC59tm3JXJ0Puubx7p2SuyQgLm6BWw5y8O9HAsgDrGPnVOSTtTrx25vVElXbzj6VXrLadxC6+0OI6lXHSq/VkrGum8RmMkOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KEDcWfJAxbJAXmZR4CgGqIByR23UW7rzxEwOwopPDBs=; b=IBgkqarscMhvifepL/BKoawMwet09Ma3MAe+118pPbfs47tJsZ3y9pgU/MnFIxGcn+Yv27aT0sRi8eaxgxbYaSpifWxUULYN77ojnVkDdo+mGojvI1xGBfdWlk4tysVPCEXnvmy6guFdG2j4sgNbRYeikLqy/2T0kAR/Sp3a/SfU/cZTaWVEQSypf+2/aUBZJtep00qQZ2kR8j9eN8ZAlPfaZgCjF9rLUEUmgcVNthjYF9iLRSNjv+OX2OKh6kxMYdLnEHnTw/TFXnyHPXVUF1Rl7ch9VobjRdKC3rJBLhLVfXvIRCYObvS5u5L+j0KW+0av5fxJS80T8B0nxuozhA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KEDcWfJAxbJAXmZR4CgGqIByR23UW7rzxEwOwopPDBs=; b=X5vdN/BpHFd5jCPR335JnNYUS7tQjaT4F0UkVtvIfMsmli8RpC9mT7nfNH3vq2g6UF/Unjj1Q1guuw2dc65p2sOCRDEbf79B/X3WCKldBV2EhPssMuWZ7p3o4VL0AelpsS43CKd1jbVDTftsGQncfBvF5EbmO3rwPUUgeQ9l7fM= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB4182.eurprd05.prod.outlook.com (52.135.164.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Thu, 24 Oct 2019 06:00:48 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a%5]) with mapi id 15.20.2347.029; Thu, 24 Oct 2019 06:00:48 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich Subject: [PATCH rdma-core 2/4] pyverbs/mlx5: Add support for driver-specific context Thread-Topic: [PATCH rdma-core 2/4] pyverbs/mlx5: Add support for driver-specific context Thread-Index: AQHVijBheEMLLjx7j0K4/7pLDXTysA== Date: Thu, 24 Oct 2019 06:00:48 +0000 Message-ID: <20191024060027.8696-3-noaos@mellanox.com> References: <20191024060027.8696-1-noaos@mellanox.com> In-Reply-To: <20191024060027.8696-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: AM0PR05CA0081.eurprd05.prod.outlook.com (2603:10a6:208:136::21) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 50b83d3e-c0ef-41e4-fea0-08d758478428 x-ms-traffictypediagnostic: AM6PR05MB4182:|AM6PR05MB4182: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-forefront-prvs: 0200DDA8BE x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(39860400002)(376002)(366004)(346002)(396003)(199004)(189003)(478600001)(71200400001)(86362001)(486006)(71190400001)(2501003)(11346002)(2616005)(446003)(52116002)(476003)(4326008)(76176011)(110136005)(54906003)(66946007)(66476007)(66556008)(66446008)(64756008)(6636002)(14454004)(99286004)(107886003)(26005)(186003)(305945005)(7736002)(316002)(66066001)(6506007)(386003)(3846002)(36756003)(256004)(102836004)(8936002)(50226002)(1076003)(6116002)(2906002)(81156014)(81166006)(6436002)(6512007)(6486002)(5660300002)(25786009)(8676002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB4182;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZvbSvtPbaMCjyBPXvUipWjgimVwBm6ZTEl067akC6R34sfuRLOD/BgSPiGv8S41vm8PuLcz1jXWjz8lVwqgm+96aO33w7vUCYXi7tH2g9IYAZkxujrlgid/oBNOXIQGkmgTF/e8hu/iSoH34a5swiZdKB8oMAcQ84Jst9kQgwYJzjFs3u63zWAHTMRa9l9bFk7vTFopa1vwGFPgkBl/7wM6pnOr6sAIiyG3iRJrV4U3mlhmR2LhF0ntL9Gl4vGLBAF/L/ey9+xiWyaMJ6myFbRW/u6kjf3sWuZD5EuR3sWuauHpg3yly3FrptaaaDriLed9pl6BEM+uLqgh/0VxgWZY8oM3RjUVD/8k90bKbNuPRDQG0W+vIyFIo7lP0pGcz3p8taOTzdZIqt5LIzk2LP6V1vc7KJVqz7FBP9M1ShVlwAiZ1QZyDwAOxd4t4iHiH MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 50b83d3e-c0ef-41e4-fea0-08d758478428 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2019 06:00:48.1994 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: wULXfycjp/1mQ/D8asnK0NwPqT0XKmiVoK3w2eeJbJo1gA9rYju9vQw0z+FPJwXUMxKe69zKMYzC8qkcDLn4VA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4182 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Allow users to open a device using mlx5dv_open_device. For this, two new classes are introduced: - Mlx5Context which inherits from Context. - Mlx5DVContextAttr, which represents mlx5dv_context_attr struct, used by users to define driver-specific options for the device opening. Signed-off-by: Noa Osherovich --- pyverbs/providers/__init__.pxd | 0 pyverbs/providers/__init__.py | 0 pyverbs/providers/mlx5/__init__.pxd | 0 pyverbs/providers/mlx5/__init__.py | 0 pyverbs/providers/mlx5/libmlx5.pxd | 17 +++++++++ pyverbs/providers/mlx5/mlx5dv.pxd | 14 +++++++ pyverbs/providers/mlx5/mlx5dv.pyx | 57 +++++++++++++++++++++++++++++ 7 files changed, 88 insertions(+) create mode 100644 pyverbs/providers/__init__.pxd create mode 100644 pyverbs/providers/__init__.py create mode 100644 pyverbs/providers/mlx5/__init__.pxd create mode 100644 pyverbs/providers/mlx5/__init__.py create mode 100644 pyverbs/providers/mlx5/libmlx5.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv.pxd create mode 100644 pyverbs/providers/mlx5/mlx5dv.pyx diff --git a/pyverbs/providers/__init__.pxd b/pyverbs/providers/__init__.pxd new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/pyverbs/providers/__init__.py b/pyverbs/providers/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/pyverbs/providers/mlx5/__init__.pxd b/pyverbs/providers/mlx5/__init__.pxd new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/pyverbs/providers/mlx5/__init__.py b/pyverbs/providers/mlx5/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/pyverbs/providers/mlx5/libmlx5.pxd b/pyverbs/providers/mlx5/libmlx5.pxd new file mode 100644 index 000000000000..54d91e288590 --- /dev/null +++ b/pyverbs/providers/mlx5/libmlx5.pxd @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file + +from libcpp cimport bool + +cimport pyverbs.libibverbs as v + + +cdef extern from 'infiniband/mlx5dv.h': + + cdef struct mlx5dv_context_attr: + unsigned int flags + unsigned long comp_mask + + bool mlx5dv_is_supported(v.ibv_device *device) + v.ibv_context* mlx5dv_open_device(v.ibv_device *device, + mlx5dv_context_attr *attr) diff --git a/pyverbs/providers/mlx5/mlx5dv.pxd b/pyverbs/providers/mlx5/mlx5dv.pxd new file mode 100644 index 000000000000..6ab94b6484b0 --- /dev/null +++ b/pyverbs/providers/mlx5/mlx5dv.pxd @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file + +#cython: language_level=3 + +cimport pyverbs.providers.mlx5.libmlx5 as dv +from pyverbs.base cimport PyverbsObject +from pyverbs.device cimport Context + +cdef class Mlx5Context(Context): + pass + +cdef class Mlx5DVContextAttr(PyverbsObject): + cdef dv.mlx5dv_context_attr attr diff --git a/pyverbs/providers/mlx5/mlx5dv.pyx b/pyverbs/providers/mlx5/mlx5dv.pyx new file mode 100644 index 000000000000..0c6b28be1d5a --- /dev/null +++ b/pyverbs/providers/mlx5/mlx5dv.pyx @@ -0,0 +1,57 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file + +from pyverbs.pyverbs_error import PyverbsUserError +cimport pyverbs.providers.mlx5.libmlx5 as dv + + +cdef class Mlx5DVContextAttr(PyverbsObject): + """ + Represent mlx5dv_context_attr struct. This class is used to open an mlx5 + device. + """ + def __cinit__(self, flags=0, comp_mask=0): + self.attr.flags = flags + self.attr.comp_mask = comp_mask + + def __str__(self): + print_format = '{:20}: {:<20}\n' + return print_format.format('flags', self.attr.flags) +\ + print_format.format('comp_mask', self.attr.comp_mask) + + @property + def flags(self): + return self.attr.flags + @flags.setter + def flags(self, val): + self.attr.flags = val + + @property + def comp_mask(self): + return self.attr.comp_mask + @comp_mask.setter + def comp_mask(self, val): + self.attr.comp_mask = val + + +cdef class Mlx5Context(Context): + """ + Represent mlx5 context, which extends Context. + """ + def __cinit__(self, **kwargs): + """ + Open an mlx5 device using the given attributes + :param kwargs: Arguments: + * *name* (str) + The RDMA device's name (used by parent class) + * *attr* (Mlx5DVContextAttr) + mlx5-specific device attributes + :return: None + """ + cdef Mlx5DVContextAttr attr + attr = kwargs.get('attr') + if not attr or not isinstance(attr, Mlx5DVContextAttr): + raise PyverbsUserError('Missing provider attributes') + if not dv.mlx5dv_is_supported(self.device): + raise PyverbsUserError('This is not an MLX5 device') + self.context = dv.mlx5dv_open_device(self.device, &attr.attr) From patchwork Thu Oct 24 06:00:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11208329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 073AD913 for ; Thu, 24 Oct 2019 06:00:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D0BB92166E for ; Thu, 24 Oct 2019 06:00:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="jwn8yx4E" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437596AbfJXGA6 (ORCPT ); Thu, 24 Oct 2019 02:00:58 -0400 Received: from mail-eopbgr50068.outbound.protection.outlook.com ([40.107.5.68]:21413 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2437663AbfJXGA5 (ORCPT ); Thu, 24 Oct 2019 02:00:57 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Cy67AUs3CMD6MkpQx/f/VPfBNYDmsdIYlbneFNEz84pzg1K11u+GvMhuN5J6YuneIIivFOkK1/6sHbHFhE7fRDOYLbwpnR4lB5+qsv0Px602WS4yvkzd00a5UCn1sW0HVS7agR/O3FWE+1cKLNFfn6VAkx6X6VgDWSUNkfdpbXd768jga282lt9vrhXm/DVPBz02XnDo1pmcLiSRfda3pCq7x3Zm3868gabjUbBwA1OEHhlodfc5QEU03qsUV5EccTYP44458G8/refDmLsd09xxWDo6F0v1LprjRMpD1o1WPBbRUvFOsdL/0o2g877H0dBWKyHmZ6CP5GuDbBvnMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z/D0im2QpL6zyzVDoUoXR3iX6byHwXojxBDZbq6WRE4=; b=MLtBuImiWh4yF4Ge9iIQpPO2h4uWyYCHVXQBQOYMqE+htdMajOcjcK3tCYL3dNjIDly/aTTwqQz3T28DSSQ3+kJltiVeQR4yXkMFtfU78BtN7bSMktGTFGzF/vR862Lp5LKuW1vFFrQXVqu+DWV8TxVclrUiHucoPZinFT0Ur8wHMAuwUYKtW57r82udQYiNJLLOOVOPbNP2ja/snwvKeU6GzIIFqIxORHPDJfvwlux4U0q1uVLcBGFPiJ9allcw8Pt1mxBVBr8/8yPlpMMHFtQBbD2OKWa4S+mqJ82UbESRafnPqyhn92AZpXMM6ULKM+0NRnAixMXROrB3RBRLLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z/D0im2QpL6zyzVDoUoXR3iX6byHwXojxBDZbq6WRE4=; b=jwn8yx4EXCfYUzk7I0P1Sfs9F5JBE6iR777wa+GMUq8ePz7EiFNlnYyU/RxGDZ4GXg3KT27Kp8Yfoyl9qbuw+i6lq5Ae3HuAdsi0F0gzcwCI8FpRYFcC6r3EAK95/iNQV2FPig34Kv/NB0sqCvahF3GpGVXzequ812CIfsb/96M= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB4182.eurprd05.prod.outlook.com (52.135.164.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Thu, 24 Oct 2019 06:00:49 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a%5]) with mapi id 15.20.2347.029; Thu, 24 Oct 2019 06:00:49 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich Subject: [PATCH rdma-core 3/4] pyverbs: Add providers to cmake build Thread-Topic: [PATCH rdma-core 3/4] pyverbs: Add providers to cmake build Thread-Index: AQHVijBicvr9tvojhUqvetli8muUrQ== Date: Thu, 24 Oct 2019 06:00:49 +0000 Message-ID: <20191024060027.8696-4-noaos@mellanox.com> References: <20191024060027.8696-1-noaos@mellanox.com> In-Reply-To: <20191024060027.8696-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: AM0PR05CA0081.eurprd05.prod.outlook.com (2603:10a6:208:136::21) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: c01cf6e1-471d-437d-db85-08d7584784b2 x-ms-traffictypediagnostic: AM6PR05MB4182:|AM6PR05MB4182: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2201; x-forefront-prvs: 0200DDA8BE x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(39860400002)(376002)(366004)(346002)(396003)(199004)(189003)(478600001)(71200400001)(86362001)(486006)(71190400001)(2501003)(11346002)(2616005)(446003)(52116002)(476003)(4326008)(76176011)(110136005)(54906003)(66946007)(66476007)(66556008)(66446008)(64756008)(6636002)(14454004)(99286004)(19627235002)(107886003)(26005)(186003)(305945005)(7736002)(316002)(66066001)(6506007)(386003)(3846002)(36756003)(14444005)(256004)(102836004)(8936002)(50226002)(1076003)(6116002)(2906002)(81156014)(81166006)(6436002)(6512007)(6486002)(5660300002)(25786009)(8676002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB4182;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Hhth1woTKuarffSzh0UNMbskqMremziPK9xael6uabVAiGgwTC0U9zyLqqvZbQ5SJa3BPZwyocW1Xyybpkn6cwijYTIbH4z4NdPdeRD+kYGgg9IZFMXret7tLwr+X1dceSoKGfg6zVBrD0akCo73wZpr04zeH1ug+rtefkOLhpl51Gz86Wg9jN5NPdA7wwDdP/QsNuuSxxiYBoa98w1yurvGt/zs/99PGIRAyiOfKF9+QhC6Rb5ntUAfMRnpuGyDNEqQ4uP0nECz/98A1dXkh5O5nI53OmNTqXHZ2v7B433C1j+QcTZ/R8UGIprZjlhmHDSfgTfVC2Hr8SgEZY2E+Naj+nHST+WSZnMDsY0cJGQyovn+zsXGarm3+JJxg1rQvv4EB/i1Ke8ejPIJr2+CAPZ3eRmJdi6a7V2jfVIU0lKiQMw0JPGZmHtJ88gpY3Sz MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: c01cf6e1-471d-437d-db85-08d7584784b2 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2019 06:00:49.1399 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: wIiQynS0Rn/5hNPQyQtDCjiluAXJWoYemXxC1xsViiOTA/4NSL1IPabAhRlkVqUX9R3lOrka98W8WPjxZfyXsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4182 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Adapt the building function to support subdirectories under pyverbs. Add the needed CMakeLists file to pyverbs/providers. Update pyverbs' CMakeLists to include the providers sub directory. Signed-off-by: Noa Osherovich --- buildlib/pyverbs_functions.cmake | 11 ++++++++--- pyverbs/CMakeLists.txt | 7 ++++++- pyverbs/providers/mlx5/CMakeLists.txt | 6 ++++++ 3 files changed, 20 insertions(+), 4 deletions(-) create mode 100644 pyverbs/providers/mlx5/CMakeLists.txt diff --git a/buildlib/pyverbs_functions.cmake b/buildlib/pyverbs_functions.cmake index 8ea5dc0df7de..4c255054fe94 100644 --- a/buildlib/pyverbs_functions.cmake +++ b/buildlib/pyverbs_functions.cmake @@ -1,10 +1,15 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2018, Mellanox Technologies. All rights reserved. See COPYING file -function(rdma_cython_module PY_MODULE) +function(rdma_cython_module PY_MODULE LINKER_FLAGS) foreach(PYX_FILE ${ARGN}) get_filename_component(FILENAME ${PYX_FILE} NAME_WE) - set(PYX "${CMAKE_CURRENT_SOURCE_DIR}/${FILENAME}.pyx") + get_filename_component(DIR ${PYX_FILE} DIRECTORY) + if (DIR) + set(PYX "${CMAKE_CURRENT_SOURCE_DIR}/${DIR}/${FILENAME}.pyx") + else() + set(PYX "${CMAKE_CURRENT_SOURCE_DIR}/${FILENAME}.pyx") + endif() set(CFILE "${CMAKE_CURRENT_BINARY_DIR}/${FILENAME}.c") include_directories(${PYTHON_INCLUDE_DIRS}) add_custom_command( @@ -20,7 +25,7 @@ function(rdma_cython_module PY_MODULE) COMPILE_FLAGS "${CMAKE_C_FLAGS} -fPIC -fno-strict-aliasing -Wno-unused-function -Wno-redundant-decls -Wno-shadow -Wno-cast-function-type -Wno-implicit-fallthrough -Wno-unknown-warning -Wno-unknown-warning-option ${NO_VAR_TRACKING_FLAGS}" LIBRARY_OUTPUT_DIRECTORY "${BUILD_PYTHON}/${PY_MODULE}" PREFIX "") - target_link_libraries(${SONAME} LINK_PRIVATE ${PYTHON_LIBRARIES} ibverbs) + target_link_libraries(${SONAME} LINK_PRIVATE ${PYTHON_LIBRARIES} ibverbs ${LINKER_FLAGS}) install(TARGETS ${SONAME} DESTINATION ${CMAKE_INSTALL_PYTHON_ARCH_LIB}/${PY_MODULE}) endforeach() diff --git a/pyverbs/CMakeLists.txt b/pyverbs/CMakeLists.txt index 90293982b280..7bbb5fc841c0 100755 --- a/pyverbs/CMakeLists.txt +++ b/pyverbs/CMakeLists.txt @@ -1,7 +1,7 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2019, Mellanox Technologies. All rights reserved. See COPYING file -rdma_cython_module(pyverbs +rdma_cython_module(pyverbs "" addr.pyx base.pyx cq.pyx @@ -20,3 +20,8 @@ rdma_python_module(pyverbs pyverbs_error.py utils.py ) + +# mlx5 provider is not built without coherent DMA, e.g. ARM32 build. +if (HAVE_COHERENT_DMA) +add_subdirectory(providers/mlx5) +endif() diff --git a/pyverbs/providers/mlx5/CMakeLists.txt b/pyverbs/providers/mlx5/CMakeLists.txt new file mode 100644 index 000000000000..f6536de8a932 --- /dev/null +++ b/pyverbs/providers/mlx5/CMakeLists.txt @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2019, Mellanox Technologies. All rights reserved. See COPYING file + +rdma_cython_module(pyverbs/providers/mlx5 mlx5 + mlx5dv.pyx +) From patchwork Thu Oct 24 06:00:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11208331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60942112C for ; Thu, 24 Oct 2019 06:01:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2C29520856 for ; Thu, 24 Oct 2019 06:01:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="PTSswJIO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437663AbfJXGBU (ORCPT ); Thu, 24 Oct 2019 02:01:20 -0400 Received: from mail-bgr052101136076.outbound.protection.outlook.com ([52.101.136.76]:57230 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2437466AbfJXGBT (ORCPT ); Thu, 24 Oct 2019 02:01:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f7Y4SBKsgKQErzT1/AGRAtH8Qn+2gWihCHiQ4mHCd+uRDw3xgeVrDHpMRpZvKZPQ4WHTnbQLwCbbZXr65LDpoymwv3uLyPmVpufUlGI6zlwlIe/941pwO1ZkLrT5d+0+cSg38Qf40ukbiwS1Jk1dkW1cZtz8u15itWLM/fqYOaIZKcDnwp9/+5GMm4Z+6Mw+k2Nh+QuhhwlJRRHx0mwtNA6liyWVLpWvYYcs4lBscIM8JQ/jb411G/wQU+rqkSfoMK3N+tx7nBX1pZOnGH6eZ5Z2/yer/rOE/KBJDxg2oukdoYihudlldYcAo2KbUoVFLRqfi57in36EH/CeoyttHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zia2IuMcFxEl3YU5RLOgDR5oZjZ55MXudN4nOr3HlhU=; b=URbYNvTC0Mz2yLLY56P5OjO6jHMTcHCOom06aLuPfleTGkqCMsyU/HnID68/HLGY3rbaO2fwDGUi3uIEtqvX18qtynFinJbZfKBEteMgJBE2uo8BdyZvG4A1Z+hYv6pRwGUGC0/kRT3X90Xlnl6o8c9d01nIEEPm68HagAizWQlXcD/3iK/2BrEYoiM08n6OyLhhyU+0GRHjsxy71DZii9nmDUl8MGxNNlGPgeon+QjEPWe46mXlDXpbjD4lF//xznCxaJh2UXSdmC7BsmkBkG/7z+N2jrVWd/XCf5P+9lQQgyDJKiclT7edquGk+rDpqvepku5bzcLAE8lZgxJaPA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zia2IuMcFxEl3YU5RLOgDR5oZjZ55MXudN4nOr3HlhU=; b=PTSswJIORQQo8FFEqBKvwe3uqSlrff1c9bq0I7ySyzJbY/gq/z8ntg9g03HmZv7aaVNmaVd30NAmABeKKrR9uTGIAQ50MrmnSGAUpujy2uAIb8NCDgqrDH3gkrQiMt+opEtFSEZU9kZMPUTMpu4COou4ov6LPa9Tgl/6CWFEjSs= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB4182.eurprd05.prod.outlook.com (52.135.164.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.21; Thu, 24 Oct 2019 06:00:50 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::ecbd:11b3:e4e9:fa1a%5]) with mapi id 15.20.2347.029; Thu, 24 Oct 2019 06:00:50 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich Subject: [PATCH rdma-core 4/4] pyverbs/mlx5: Add query device capability Thread-Topic: [PATCH rdma-core 4/4] pyverbs/mlx5: Add query device capability Thread-Index: AQHVijBiXOVVE4M5R0SRpwb7EsH5AA== Date: Thu, 24 Oct 2019 06:00:50 +0000 Message-ID: <20191024060027.8696-5-noaos@mellanox.com> References: <20191024060027.8696-1-noaos@mellanox.com> In-Reply-To: <20191024060027.8696-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: AM0PR05CA0081.eurprd05.prod.outlook.com (2603:10a6:208:136::21) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 7dd8e1d2-dbfb-44b4-8ce8-08d75847853e x-ms-traffictypediagnostic: AM6PR05MB4182:|AM6PR05MB4182:|AM6PR05MB4182:|AM6PR05MB4182: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0200DDA8BE x-forefront-antispam-report: SFV:SPM;SFS:(10009020)(4636009)(136003)(39860400002)(376002)(366004)(346002)(396003)(199004)(189003)(478600001)(71200400001)(86362001)(486006)(71190400001)(2501003)(11346002)(2616005)(446003)(52116002)(476003)(4326008)(76176011)(110136005)(54906003)(66946007)(66476007)(66556008)(66446008)(64756008)(6636002)(14454004)(99286004)(19627235002)(107886003)(26005)(186003)(305945005)(7736002)(316002)(66066001)(6506007)(386003)(3846002)(36756003)(14444005)(256004)(102836004)(8936002)(50226002)(1076003)(6116002)(2906002)(81156014)(81166006)(6436002)(6512007)(6486002)(30864003)(5660300002)(25786009)(8676002)(98050200001);DIR:OUT;SFP:1501;SCL:5;SRVR:AM6PR05MB4182;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zfSFsMjRhQk+wzInlj5e53hJDqLm9O9mpeeeGPMl3j/sSi6ZqtMnqxVil7gwuiVrXhUGa0TTdjlPf7vBva9ARfWc57es4u/Ms7RA0AoUB10XKq6c7UqpJonXBcYt54XSn7I1xH1flfwQiKW/KYdz3kjDKmKg6+tAXro/1SY/DHRWx52pXlIvS4Wjgmy8yRx53DTqO3ud+NEJv1moi0blg0xuyX0HkS5nMD+IRMX6FYbUPyfuiCAkI/3HrKBFOBH2Qhoy7aPa6CNrhDBmpubRGTHAXVC10RL516buse2zfzV9GQvcx+n5vmhOMGr7PM9bG97kQe9n5QwU+4UgRdqBTuvfYp3AvQdTCyi2Y66BV8iPyYaqnGTobACLXr/xDAVSahYZces27DRzj4BD5UJ4ByDFeDXkXPLSyfCmD5AqwzLI+XDvgVnAO4iML27tfn8xeEpSBA+ufhj5LBKAwSJ8OG22P9ZbXVVfkdzqf8ksVdwzdcOLWTAYLEmPdL+hXmmnIstHS3xYPmnahZPAGkZPiA== MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7dd8e1d2-dbfb-44b4-8ce8-08d75847853e X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2019 06:00:50.0344 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sEt1sUvyQz3Qxws8bbJg7kGuOlT4eLzW1Em5vQTCHQKQUwai5vYsLNo2P/8+pqvh8bmwa3fWU6bUoHtwEFMVrg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4182 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Allow users to use the mlx5-specific query_device() function via Mlx5Context's query_mlx5_device method. This device querying exposes device properties that are not available via legacy query such as software parsing and CQE compression capabilities. This method returns an Mlx5DVContext object which supports a user-friendly print format. Signed-off-by: Noa Osherovich --- pyverbs/providers/mlx5/CMakeLists.txt | 1 + pyverbs/providers/mlx5/libmlx5.pxd | 29 ++++ pyverbs/providers/mlx5/mlx5_enums.pyx | 1 + pyverbs/providers/mlx5/mlx5dv.pxd | 3 + pyverbs/providers/mlx5/mlx5dv.pyx | 196 ++++++++++++++++++++++++ pyverbs/providers/mlx5/mlx5dv_enums.pxd | 47 ++++++ 6 files changed, 277 insertions(+) create mode 120000 pyverbs/providers/mlx5/mlx5_enums.pyx create mode 100644 pyverbs/providers/mlx5/mlx5dv_enums.pxd diff --git a/pyverbs/providers/mlx5/CMakeLists.txt b/pyverbs/providers/mlx5/CMakeLists.txt index f6536de8a932..d9b0849f02dc 100644 --- a/pyverbs/providers/mlx5/CMakeLists.txt +++ b/pyverbs/providers/mlx5/CMakeLists.txt @@ -3,4 +3,5 @@ rdma_cython_module(pyverbs/providers/mlx5 mlx5 mlx5dv.pyx + mlx5_enums.pyx ) diff --git a/pyverbs/providers/mlx5/libmlx5.pxd b/pyverbs/providers/mlx5/libmlx5.pxd index 54d91e288590..aaf75620a9bb 100644 --- a/pyverbs/providers/mlx5/libmlx5.pxd +++ b/pyverbs/providers/mlx5/libmlx5.pxd @@ -12,6 +12,35 @@ cdef extern from 'infiniband/mlx5dv.h': unsigned int flags unsigned long comp_mask + cdef struct mlx5dv_cqe_comp_caps: + unsigned int max_num + unsigned int supported_format + + cdef struct mlx5dv_sw_parsing_caps: + unsigned int sw_parsing_offloads + unsigned int supported_qpts + + cdef struct mlx5dv_striding_rq_caps: + unsigned int min_single_stride_log_num_of_bytes + unsigned int max_single_stride_log_num_of_bytes + unsigned int min_single_wqe_log_num_of_strides + unsigned int max_single_wqe_log_num_of_strides + unsigned int supported_qpts + + cdef struct mlx5dv_context: + unsigned char version + unsigned long flags + unsigned long comp_mask + mlx5dv_cqe_comp_caps cqe_comp_caps + mlx5dv_sw_parsing_caps sw_parsing_caps + mlx5dv_striding_rq_caps striding_rq_caps + unsigned int tunnel_offloads_caps + unsigned int max_dynamic_bfregs + unsigned long max_clock_info_update_nsec + unsigned int flow_action_flags + unsigned int dc_odp_caps + bool mlx5dv_is_supported(v.ibv_device *device) v.ibv_context* mlx5dv_open_device(v.ibv_device *device, mlx5dv_context_attr *attr) + int mlx5dv_query_device(v.ibv_context *ctx, mlx5dv_context *attrs_out) diff --git a/pyverbs/providers/mlx5/mlx5_enums.pyx b/pyverbs/providers/mlx5/mlx5_enums.pyx new file mode 120000 index 000000000000..ba0e916f9e62 --- /dev/null +++ b/pyverbs/providers/mlx5/mlx5_enums.pyx @@ -0,0 +1 @@ +mlx5dv_enums.pxd \ No newline at end of file diff --git a/pyverbs/providers/mlx5/mlx5dv.pxd b/pyverbs/providers/mlx5/mlx5dv.pxd index 6ab94b6484b0..d9fea82af894 100644 --- a/pyverbs/providers/mlx5/mlx5dv.pxd +++ b/pyverbs/providers/mlx5/mlx5dv.pxd @@ -12,3 +12,6 @@ cdef class Mlx5Context(Context): cdef class Mlx5DVContextAttr(PyverbsObject): cdef dv.mlx5dv_context_attr attr + +cdef class Mlx5DVContext(PyverbsObject): + cdef dv.mlx5dv_context dv diff --git a/pyverbs/providers/mlx5/mlx5dv.pyx b/pyverbs/providers/mlx5/mlx5dv.pyx index 0c6b28be1d5a..dadc9cdcceee 100644 --- a/pyverbs/providers/mlx5/mlx5dv.pyx +++ b/pyverbs/providers/mlx5/mlx5dv.pyx @@ -2,7 +2,10 @@ # Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file from pyverbs.pyverbs_error import PyverbsUserError +cimport pyverbs.providers.mlx5.mlx5dv_enums as dve cimport pyverbs.providers.mlx5.libmlx5 as dv +from pyverbs.base import PyverbsRDMAErrno +cimport pyverbs.libibverbs_enums as e cdef class Mlx5DVContextAttr(PyverbsObject): @@ -55,3 +58,196 @@ cdef class Mlx5Context(Context): if not dv.mlx5dv_is_supported(self.device): raise PyverbsUserError('This is not an MLX5 device') self.context = dv.mlx5dv_open_device(self.device, &attr.attr) + + def query_mlx5_device(self, comp_mask=-1): + """ + Queries the provider for device-specific attributes. + :param comp_mask: Which attributes to query. Default value is -1. If + not changed by user, pyverbs will pass a bitwise OR + of all available enum entries. + :return: A Mlx5DVContext containing the attributes. + """ + dv_attr = Mlx5DVContext() + if comp_mask == -1: + dv_attr.comp_mask = \ + dve.MLX5DV_CONTEXT_MASK_CQE_COMPRESION |\ + dve.MLX5DV_CONTEXT_MASK_SWP |\ + dve.MLX5DV_CONTEXT_MASK_STRIDING_RQ |\ + dve.MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS |\ + dve.MLX5DV_CONTEXT_MASK_DYN_BFREGS |\ + dve.MLX5DV_CONTEXT_MASK_CLOCK_INFO_UPDATE |\ + dve.MLX5DV_CONTEXT_MASK_FLOW_ACTION_FLAGS + else: + dv_attr.comp_mask = comp_mask + rc = dv.mlx5dv_query_device(self.context, &dv_attr.dv) + if rc != 0: + raise PyverbsRDMAErrno('Failed to query mlx5 device {name}, got {rc}'. + format(name=self.name, rc=rc)) + return dv_attr + + +cdef class Mlx5DVContext(PyverbsObject): + """ + Represents mlx5dv_context struct, which exposes mlx5-specific capabilities, + reported by mlx5dv_query_device. + """ + @property + def version(self): + return self.dv.version + + @property + def flags(self): + return self.dv.flags + + @property + def comp_mask(self): + return self.dv.comp_mask + @comp_mask.setter + def comp_mask(self, val): + self.dv.comp_mask = val + + @property + def cqe_comp_caps(self): + return self.dv.cqe_comp_caps + + @property + def sw_parsing_caps(self): + return self.dv.sw_parsing_caps + + @property + def striding_rq_caps(self): + return self.dv.striding_rq_caps + + @property + def tunnel_offload_caps(self): + return self.dv.tunnel_offloads_caps + + @property + def max_dynamic_bfregs(self): + return self.dv.max_dynamic_bfregs + + @property + def max_clock_info_update_nsec(self): + return self.dv.max_clock_info_update_nsec + + @property + def flow_action_flags(self): + return self.dv.flow_action_flags + + @property + def dc_odp_caps(self): + return self.dv.dc_odp_caps + + def __str__(self): + print_format = '{:20}: {:<20}\n' + ident_format = ' {:20}: {:<20}\n' + cqe = 'CQE compression caps:\n' +\ + ident_format.format('max num', + self.dv.cqe_comp_caps.max_num) +\ + ident_format.format('supported formats', + cqe_comp_to_str(self.dv.cqe_comp_caps.supported_format)) + swp = 'SW parsing caps:\n' +\ + ident_format.format('SW parsing offloads', + swp_to_str(self.dv.sw_parsing_caps.sw_parsing_offloads)) +\ + ident_format.format('supported QP types', + qpts_to_str(self.dv.sw_parsing_caps.supported_qpts)) + strd = 'Striding RQ caps:\n' +\ + ident_format.format('min single stride log num of bytes', + self.dv.striding_rq_caps.min_single_stride_log_num_of_bytes) +\ + ident_format.format('max single stride log num of bytes', + self.dv.striding_rq_caps.max_single_stride_log_num_of_bytes) +\ + ident_format.format('min single wqe log num of strides', + self.dv.striding_rq_caps.min_single_wqe_log_num_of_strides) +\ + ident_format.format('max single wqe log num of strides', + self.dv.striding_rq_caps.max_single_wqe_log_num_of_strides) +\ + ident_format.format('supported QP types', + qpts_to_str(self.dv.striding_rq_caps.supported_qpts)) + return print_format.format('Version', self.dv.version) +\ + print_format.format('Flags', + context_flags_to_str(self.dv.flags)) +\ + print_format.format('comp mask', + context_comp_mask_to_str(self.dv.comp_mask)) +\ + cqe + swp + strd +\ + print_format.format('Tunnel offloads caps', + tunnel_offloads_to_str(self.dv.tunnel_offloads_caps)) +\ + print_format.format('Max dynamic BF registers', + self.dv.max_dynamic_bfregs) +\ + print_format.format('Max clock info update [nsec]', + self.dv.max_clock_info_update_nsec) +\ + print_format.format('Flow action flags', + self.dv.flow_action_flags) +\ + print_format.format('DC ODP caps', self.dv.dc_odp_caps) + + +def qpts_to_str(qp_types): + numberic_types = qp_types + qpts_str = '' + qpts = {e.IBV_QPT_RC: 'RC', e.IBV_QPT_UC: 'UC', e.IBV_QPT_UD: 'UD', + e.IBV_QPT_RAW_PACKET: 'Raw Packet', e.IBV_QPT_XRC_SEND: 'XRC Send', + e.IBV_QPT_XRC_RECV: 'XRC Recv', e.IBV_QPT_DRIVER: 'Driver QPT'} + for t in qpts.keys(): + if (1 << t) & qp_types: + qpts_str += qpts[t] + ', ' + qp_types -= t + if qp_types == 0: + break + return qpts_str[:-2] + ' ({})'.format(numberic_types) + + +def bitmask_to_str(bits, values): + numeric_bits = bits + res = '' + for t in values.keys(): + if t & bits: + res += values[t] + ', ' + bits -= t + if bits == 0: + break + return res[:-2] + ' ({})'.format(numeric_bits) # Remove last comma and space + + +def context_comp_mask_to_str(mask): + l = {dve.MLX5DV_CONTEXT_MASK_CQE_COMPRESION: 'CQE compression', + dve.MLX5DV_CONTEXT_MASK_SWP: 'SW parsing', + dve.MLX5DV_CONTEXT_MASK_STRIDING_RQ: 'Striding RQ', + dve.MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS: 'Tunnel offloads', + dve.MLX5DV_CONTEXT_MASK_DYN_BFREGS: 'Dynamic BF regs', + dve.MLX5DV_CONTEXT_MASK_CLOCK_INFO_UPDATE: 'Clock info update', + dve.MLX5DV_CONTEXT_MASK_FLOW_ACTION_FLAGS: 'Flow action flags'} + return bitmask_to_str(mask, l) + + +def context_flags_to_str(flags): + l = {dve.MLX5DV_CONTEXT_FLAGS_CQE_V1: 'CQE v1', + dve.MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED: 'Multi packet WQE allowed', + dve.MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW: 'Enhanced multi packet WQE', + dve.MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP: 'Support CQE 128B compression', + dve.MLX5DV_CONTEXT_FLAGS_CQE_128B_PAD: 'Support CQE 128B padding', + dve.MLX5DV_CONTEXT_FLAGS_PACKET_BASED_CREDIT_MODE: + 'Support packet based credit mode (in RC QP)'} + return bitmask_to_str(flags, l) + + +def swp_to_str(swps): + l = {dve.MLX5DV_SW_PARSING: 'SW Parsing', + dve.MLX5DV_SW_PARSING_CSUM: 'SW Parsing CSUM', + dve.MLX5DV_SW_PARSING_LSO: 'SW Parsing LSO'} + return bitmask_to_str(swps, l) + + +def cqe_comp_to_str(cqe): + l = {dve.MLX5DV_CQE_RES_FORMAT_HASH: 'with hash', + dve.MLX5DV_CQE_RES_FORMAT_CSUM: 'with RX checksum CSUM', + dve.MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX: 'with stride index'} + return bitmask_to_str(cqe, l) + + +def tunnel_offloads_to_str(tun): + l = {dve.MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_VXLAN: 'VXLAN', + dve.MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_GRE: 'GRE', + dve.MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_GENEVE: 'Geneve', + dve.MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_GRE:\ + 'Ctrl word + MPLS over GRE', + dve.MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_UDP:\ + 'Ctrl word + MPLS over UDP'} + return bitmask_to_str(tun, l) diff --git a/pyverbs/providers/mlx5/mlx5dv_enums.pxd b/pyverbs/providers/mlx5/mlx5dv_enums.pxd new file mode 100644 index 000000000000..038a49111a3b --- /dev/null +++ b/pyverbs/providers/mlx5/mlx5dv_enums.pxd @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) +# Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file + +#cython: language_level=3 + +cdef extern from 'infiniband/mlx5dv.h': + + cpdef enum mlx5dv_context_comp_mask: + MLX5DV_CONTEXT_MASK_CQE_COMPRESION = 1 << 0 + MLX5DV_CONTEXT_MASK_SWP = 1 << 1 + MLX5DV_CONTEXT_MASK_STRIDING_RQ = 1 << 2 + MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS = 1 << 3 + MLX5DV_CONTEXT_MASK_DYN_BFREGS = 1 << 4 + MLX5DV_CONTEXT_MASK_CLOCK_INFO_UPDATE = 1 << 5 + MLX5DV_CONTEXT_MASK_FLOW_ACTION_FLAGS = 1 << 6 + + cpdef enum mlx5dv_context_flags: + MLX5DV_CONTEXT_FLAGS_CQE_V1 = 1 << 0 + MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED = 1 << 2 + MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW = 1 << 3 + MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP = 1 << 4 + MLX5DV_CONTEXT_FLAGS_CQE_128B_PAD = 1 << 5 + MLX5DV_CONTEXT_FLAGS_PACKET_BASED_CREDIT_MODE = 1 << 6 + + cpdef enum mlx5dv_sw_parsing_offloads: + MLX5DV_SW_PARSING = 1 << 0 + MLX5DV_SW_PARSING_CSUM = 1 << 1 + MLX5DV_SW_PARSING_LSO = 1 << 2 + + cpdef enum mlx5dv_cqe_comp_res_format: + MLX5DV_CQE_RES_FORMAT_HASH = 1 << 0 + MLX5DV_CQE_RES_FORMAT_CSUM = 1 << 1 + MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX = 1 << 2 + + cpdef enum mlx5dv_tunnel_offloads: + MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_VXLAN = 1 << 0 + MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_GRE = 1 << 1 + MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_GENEVE = 1 << 2 + MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_GRE = 1 << 3 + MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_UDP = 1 << 4 + + cpdef enum mlx5dv_flow_action_cap_flags: + MLX5DV_FLOW_ACTION_FLAGS_ESP_AES_GCM = 1 << 0 + MLX5DV_FLOW_ACTION_FLAGS_ESP_AES_GCM_REQ_METADATA = 1 << 1 + MLX5DV_FLOW_ACTION_FLAGS_ESP_AES_GCM_SPI_STEERING = 1 << 2 + MLX5DV_FLOW_ACTION_FLAGS_ESP_AES_GCM_FULL_OFFLOAD = 1 << 3 + MLX5DV_FLOW_ACTION_FLAGS_ESP_AES_GCM_TX_IV_IS_ESN = 1 << 4