From patchwork Wed Dec 18 13:02:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11300643 X-Patchwork-Delegate: leon@leon.nu Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47A10921 for ; Wed, 18 Dec 2019 13:02:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1229F218AC for ; Wed, 18 Dec 2019 13:02:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="g/q/gHsw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726774AbfLRNCg (ORCPT ); Wed, 18 Dec 2019 08:02:36 -0500 Received: from mail-eopbgr30089.outbound.protection.outlook.com ([40.107.3.89]:39813 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726723AbfLRNCg (ORCPT ); Wed, 18 Dec 2019 08:02:36 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VZZ72V7ZlnB1TtdWfezyV31YVQ5PuQbwjyhgOy5L3RptUhfI9NKOMDQC3GwRol16YnYAK3X6HWA7w+dcsSulRTLMWj5qfF10LZRZ973OktOSWFWLfobfBPAM0sSbE/gpawBiay+zMeJ6UuVN7x+nlXF0wCWvtPJZjyEM3NEbRY3giWiaARZC5GOctN5HQl/4fM2k0ZVtx5pi2Vi5/E1vExcazaDjVdUh3I/75jkxkThzOuFcpr4DJ8ju2MMpk6dt6O4O5XSQsD62jS3S3SRy4VX7llkZSKY0sPpPCfCPa2RBIEiYMs56jbU2ruWzorGiSl6EirvG+8p5w+irmpz1Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g48Dua4AAFz6oqOM0TFX9iO8B7JeUSjFgPeMhKm1JOY=; b=JRtfvXsfyL0iaq1ylUjqpDQIMRIkX84dMfLXesNi8s19OVx9GvGffbMF45wof0UFWznkW9DM8J4OrE2szrqm1Y4+O1FF6PzsjLvbpw70k0OPkXHu+bpDo9jtfGnKgozHF+nGAMWThHIxV3ODTT416MypY962uIdZEGrnb0BZPKbUO+uL33wBWWI1it9hst/YGnDCVkfYRQ9OSB7IPiKUZUBofXKvoBkUzey4vBqq+s2EinZAi2YEK8JBuS2iTT/5/POWkrx4n1vPSWFA1RShNL2BgSXS99xAX091vXVe81RWIGxif4G13kditSc3hi7fzve+8DSHKrd70CZpt+5Mkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g48Dua4AAFz6oqOM0TFX9iO8B7JeUSjFgPeMhKm1JOY=; b=g/q/gHswtGC4/+adlj7Fvtxl4ELWWaUTVFzU6zJC6VnzD1MJafCv/b9Xgdvh/4sjSGqa5ohgjqQDbgvyGCrPWkDSEXSEgP1ki+DzMxiyXIGm2uMhlvYTesdsYHSvj/f+1ssUu8YGG22Go9Rad7OX3nHYPn11kByeq0RyPS+GHWQ= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB5093.eurprd05.prod.outlook.com (20.177.35.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.20; Wed, 18 Dec 2019 13:02:29 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca%4]) with mapi id 15.20.2538.019; Wed, 18 Dec 2019 13:02:29 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich , Edward Srouji Subject: [PATCH rdma-core 1/3] pyverbs: Move close_weakrefs() out of the base object Thread-Topic: [PATCH rdma-core 1/3] pyverbs: Move close_weakrefs() out of the base object Thread-Index: AQHVtaNnC4QwGv1I4UyJa9/HDBVqAg== Date: Wed, 18 Dec 2019 13:02:29 +0000 Message-ID: <20191218130216.503-2-noaos@mellanox.com> References: <20191218130216.503-1-noaos@mellanox.com> In-Reply-To: <20191218130216.503-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: ZR0P278CA0009.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:16::19) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 48b5d9bf-b358-44e7-a213-08d783ba899b x-ms-traffictypediagnostic: AM6PR05MB5093:|AM6PR05MB5093: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3044; x-forefront-prvs: 0255DF69B9 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(396003)(346002)(376002)(39860400002)(366004)(136003)(199004)(189003)(86362001)(1076003)(316002)(6486002)(478600001)(6506007)(5660300002)(66446008)(64756008)(66556008)(66946007)(186003)(6636002)(2906002)(26005)(81166006)(107886003)(8676002)(81156014)(52116002)(66476007)(71200400001)(6512007)(54906003)(36756003)(2616005)(110136005)(4326008)(8936002)(6606295002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB5093;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6pTHVMHFxrz3HGvez40UeNA8+zj0jGdfU+CgINMur9+BJv8P3BDk06XDEWGE7auuk6Xv/Zpa56/YbRXqp7Pi+Eonyzn3eW8wEi4HcdekOsVH5G3Ix2qRrsU0UU+B/+vzzVHUyHpT2EJD8hSoFmYmNMZFcAr3smcd1nSOC248o+rQpU4bcPlqlJ+gcr/a/AgRf9ifooEcskeAkU2sejvJvlId8M4Ay4zgCde9yuPb5DclSPv/F5AzD31OtEKUMqFWuaA5y2axJIfiwDzo3YYSbcVK8t+nw3f5dF8NHnSmU4W4st+oUaFYTasZkh4otpKWtlJNr+t6atsGoJxOFKYSYO2Iw0uNz6u4xsYUuiKla2xvK9yNp5T7RGiMty9tJSBzdrdnb02Pw/RJbZ3WyIxOP6wwo05sBVO2w3vMtbHZ6rJ7kOkJxuYz6oc+T6LIryR3gYwimszU3cJqgV+nEgPjBo5wDg/hFHV8abjxqHyzAZUSweNVlyalQ7tE8714YZRQ MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48b5d9bf-b358-44e7-a213-08d783ba899b X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2019 13:02:29.4287 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8QHi1d+WfbIYpsJ2ZNzmqvRqHWDTVQI2padjpg8+WlrtSFaSMhbehe2Hy0vPCW8UPIGRk2H2t2MfDJd+hwJ2sA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB5093 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org close_weakrefs() is called during __dealloc__(). During that time, the object might be partially destroyed. Since close_weakrefs() doesn't use 'self', there's no prevention to move it outside of the object. Signed-off-by: Noa Osherovich Reviewed-by: Edward Srouji --- pyverbs/base.pxd | 2 ++ pyverbs/base.pyx | 46 ++++++++++++++++++++++++---------------------- pyverbs/cq.pyx | 7 ++++--- pyverbs/device.pyx | 7 ++++--- pyverbs/pd.pyx | 5 +++-- pyverbs/xrcd.pyx | 3 ++- 6 files changed, 39 insertions(+), 31 deletions(-) diff --git a/pyverbs/base.pxd b/pyverbs/base.pxd index e85f7c020e1c..efa63236810c 100644 --- a/pyverbs/base.pxd +++ b/pyverbs/base.pxd @@ -9,3 +9,5 @@ cdef class PyverbsObject(object): cdef class PyverbsCM(PyverbsObject): cpdef close(self) + +cdef close_weakrefs(iterables) diff --git a/pyverbs/base.pyx b/pyverbs/base.pyx index 8b3e6741ae19..d69516285ece 100644 --- a/pyverbs/base.pyx +++ b/pyverbs/base.pyx @@ -12,6 +12,30 @@ LOG_LEVEL=logging.INFO LOG_FORMAT='[%(levelname)s] %(asctime)s %(filename)s:%(lineno)s: %(message)s' logging.basicConfig(format=LOG_FORMAT, level=LOG_LEVEL, datefmt='%d %b %Y %H:%M:%S') + +cdef close_weakrefs(iterables): + """ + For each iterable element of iterables, pop each element and + call its close() method. This method is used when an object is being + closed while other objects still hold C references to it; the object + holds weakrefs to such other object, and closes them before trying to + teardown the C resources. + :param iterables: an array of WeakSets + :return: None + """ + # None elements can be present if an object's close() was called more + # than once (e.g. GC and by another object) + for it in iterables: + if it is None: + continue + while True: + try: + tmp = it.pop() + tmp.close() + except KeyError: # popping an empty set + break + + cdef class PyverbsObject(object): def __cinit__(self): @@ -20,28 +44,6 @@ cdef class PyverbsObject(object): def set_log_level(self, val): self.logger.setLevel(val) - def close_weakrefs(self, iterables): - """ - For each iterable element of iterables, pop each element and - call its close() method. This method is used when an object is being - closed while other objects still hold C references to it; the object - holds weakrefs to such other object, and closes them before trying to - teardown the C resources. - :param iterables: an array of WeakSets - :return: None - """ - # None elements can be present if an object's close() was called more - # than once (e.g. GC and by another object) - for it in iterables: - if it is None: - continue - while True: - try: - tmp = it.pop() - tmp.close() - except KeyError: # popping an empty set - break - cdef class PyverbsCM(PyverbsObject): """ diff --git a/pyverbs/cq.pyx b/pyverbs/cq.pyx index ac6baa618c3d..1ea443fc6966 100755 --- a/pyverbs/cq.pyx +++ b/pyverbs/cq.pyx @@ -4,6 +4,7 @@ import weakref from pyverbs.pyverbs_error import PyverbsError from pyverbs.base import PyverbsRDMAErrno +from pyverbs.base cimport close_weakrefs cimport pyverbs.libibverbs_enums as e from pyverbs.device cimport Context from pyverbs.srq cimport SRQ @@ -35,7 +36,7 @@ cdef class CompChannel(PyverbsCM): cpdef close(self): self.logger.debug('Closing completion channel') - self.close_weakrefs([self.cqs]) + close_weakrefs([self.cqs]) if self.cc != NULL: rc = v.ibv_destroy_comp_channel(self.cc) if rc != 0: @@ -108,7 +109,7 @@ cdef class CQ(PyverbsCM): cpdef close(self): self.logger.debug('Closing CQ') - self.close_weakrefs([self.qps, self.srqs]) + close_weakrefs([self.qps, self.srqs]) if self.cq != NULL: rc = v.ibv_destroy_cq(self.cq) if rc != 0: @@ -292,7 +293,7 @@ cdef class CQEX(PyverbsCM): cpdef close(self): self.logger.debug('Closing CQEx') - self.close_weakrefs([self.srqs, self.qps]) + close_weakrefs([self.srqs, self.qps]) if self.cq != NULL: rc = v.ibv_destroy_cq(self.cq) if rc != 0: diff --git a/pyverbs/device.pyx b/pyverbs/device.pyx index 33d133fd24da..56d7540ceb6c 100755 --- a/pyverbs/device.pyx +++ b/pyverbs/device.pyx @@ -12,6 +12,7 @@ from .pyverbs_error import PyverbsRDMAError, PyverbsError from pyverbs.cq cimport CQEX, CQ, CompChannel from .pyverbs_error import PyverbsUserError from pyverbs.base import PyverbsRDMAErrno +from pyverbs.base cimport close_weakrefs cimport pyverbs.libibverbs_enums as e cimport pyverbs.libibverbs as v from pyverbs.cmid cimport CMID @@ -144,8 +145,8 @@ cdef class Context(PyverbsCM): cpdef close(self): self.logger.debug('Closing Context') - self.close_weakrefs([self.qps, self.ccs, self.cqs, self.dms, self.pds, - self.xrcds]) + close_weakrefs([self.qps, self.ccs, self.cqs, self.dms, self.pds, + self.xrcds]) if self.context != NULL: rc = v.ibv_close_device(self.context) if rc != 0: @@ -647,7 +648,7 @@ cdef class DM(PyverbsCM): cpdef close(self): self.logger.debug('Closing DM') - self.close_weakrefs([self.dm_mrs]) + close_weakrefs([self.dm_mrs]) if self.dm != NULL: rc = v.ibv_free_dm(self.dm) if rc != 0: diff --git a/pyverbs/pd.pyx b/pyverbs/pd.pyx index 4a33587b7df0..8c17ffcba7e8 100755 --- a/pyverbs/pd.pyx +++ b/pyverbs/pd.pyx @@ -4,6 +4,7 @@ import weakref from pyverbs.pyverbs_error import PyverbsUserError, PyverbsError from pyverbs.base import PyverbsRDMAErrno +from pyverbs.base cimport close_weakrefs from pyverbs.device cimport Context from libc.stdint cimport uintptr_t from pyverbs.cmid cimport CMID @@ -65,8 +66,8 @@ cdef class PD(PyverbsCM): :return: None """ self.logger.debug('Closing PD') - self.close_weakrefs([self.parent_domains, self.qps, self.ahs, self.mws, - self.mrs, self.srqs]) + close_weakrefs([self.parent_domains, self.qps, self.ahs, self.mws, + self.mrs, self.srqs]) if self.pd != NULL: rc = v.ibv_dealloc_pd(self.pd) if rc != 0: diff --git a/pyverbs/xrcd.pyx b/pyverbs/xrcd.pyx index 70b0c9adbbaa..91303daf3bc0 100755 --- a/pyverbs/xrcd.pyx +++ b/pyverbs/xrcd.pyx @@ -4,6 +4,7 @@ import weakref from pyverbs.pyverbs_error import PyverbsRDMAError, PyverbsError from pyverbs.base import PyverbsRDMAErrno +from pyverbs.base cimport close_weakrefs from pyverbs.device cimport Context from pyverbs.srq cimport SRQ from pyverbs.qp cimport QP @@ -68,7 +69,7 @@ cdef class XRCD(PyverbsCM): :return: None """ self.logger.debug('Closing XRCD') - self.close_weakrefs([self.qps, self.srqs]) + close_weakrefs([self.qps, self.srqs]) # XRCD may be deleted directly or indirectly by closing its context, # which leaves the Python XRCD object without the underlying C object, # so during destruction, need to check whether or not the C object From patchwork Wed Dec 18 13:02:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11300647 X-Patchwork-Delegate: leon@leon.nu Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CADC921 for ; Wed, 18 Dec 2019 13:02:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 195C3218AC for ; Wed, 18 Dec 2019 13:02:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="frErtVCy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726682AbfLRNCp (ORCPT ); Wed, 18 Dec 2019 08:02:45 -0500 Received: from mail-eopbgr30089.outbound.protection.outlook.com ([40.107.3.89]:39813 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726718AbfLRNCp (ORCPT ); Wed, 18 Dec 2019 08:02:45 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hGq32mRPd3TCDFyS9L5ioVFfnYdk7QgOUilAIJ2wS5s2HWY7DYKKN4qNu0sBzn7l0NHTobv+uPPSp9cZV5lHIKkpQv78AouV5TvWWjRPzs9xxKKfc3wl+ylPC3r/aVnwhzISAP1O5IF+erOacyhVR/1bSOHQKEqldfVhQKnyqUYc9b5Ng4+bT4LrIs/ixr56vdA23w1SGtGVwQ1SoH74cO1hbERxti0x1qH9K4Ir5binPYjye9YXGTpVGMPu2haUjKL8W1/VBAArhWAU7YhBaNTi6UC61Guc8OhDTSeoq5+ZwA3BzS5jS22K4nwYuOLaq8LUat9wDD9qSsxheeo62Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q2AqHPT2ohO1yS5/I9jQolFzli6iT1h5VnsuPXb+Bp0=; b=TAvYyne1hhMHPZJ+u3WM3iXzvPIs12+s+ZUM4nKW+PB/gQNYyXgSN8phACVKCnyWAN8BOcPTEPzlTjEKynrNbNICB55KCpewLR4pvLSLBGGMPuSa4zuOeNZgMzsw93nJ3w9XVa0NJwBRA5MzCLGX47Bn+Q6vq69z54DXyq9q7oE6DzfR6611Zm26pENxYGezjD06NfmlQjpBen00CwSqC2YcL/wdwjEFR7n+ayJMGwWFj3isYyn5jRFcLgM9Sh7PBmlgit1qmE1GxTgxmiZjLwD+GpNKT0+O8jzUdX6dkSghyR/+N7Hyp4C0FNyUsypie4rdqRptR38GpFR5bPpzJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q2AqHPT2ohO1yS5/I9jQolFzli6iT1h5VnsuPXb+Bp0=; b=frErtVCyjXZfivM1jGcBx5dO7VBuwVP49B2+zYIwyFcT2iAVPRfHgZFHhD565HYlweZnzjNL1Os072gMqpJ1SOdHoMEJ2L66io39uWh9+knRfPphhMhYQcg0Ml1XUTsQTwZ+0X3YUmeNKcg7ct6OLJx81hahAm8obwMosd1fyNE= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB5093.eurprd05.prod.outlook.com (20.177.35.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.20; Wed, 18 Dec 2019 13:02:30 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca%4]) with mapi id 15.20.2538.019; Wed, 18 Dec 2019 13:02:30 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich , Edward Srouji Subject: [PATCH rdma-core 2/3] pyverbs: Refactor objects creation process Thread-Topic: [PATCH rdma-core 2/3] pyverbs: Refactor objects creation process Thread-Index: AQHVtaNnhPbY37xMREm+zdBE3167bA== Date: Wed, 18 Dec 2019 13:02:30 +0000 Message-ID: <20191218130216.503-3-noaos@mellanox.com> References: <20191218130216.503-1-noaos@mellanox.com> In-Reply-To: <20191218130216.503-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: ZR0P278CA0009.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:16::19) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: cfab58bd-5c3f-4c97-f10d-08d783ba8a40 x-ms-traffictypediagnostic: AM6PR05MB5093:|AM6PR05MB5093: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1201; x-forefront-prvs: 0255DF69B9 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(396003)(346002)(376002)(39860400002)(366004)(136003)(199004)(189003)(30864003)(86362001)(1076003)(316002)(6486002)(478600001)(6506007)(5660300002)(66446008)(64756008)(66556008)(66946007)(186003)(6636002)(2906002)(26005)(81166006)(107886003)(8676002)(81156014)(52116002)(66476007)(71200400001)(6512007)(54906003)(36756003)(2616005)(110136005)(4326008)(8936002)(569006);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB5093;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: O+4NMrlWOfS9aV5/J1/ARYBA0BDuQdlY1FrycmeG/5eID6OGbaHN2gTNYY+pbhRTt+z5PKFAzRGEeF5u8efIAOCvbozi9oNEisWPbKY8QwvJ5nEHF6TQ3w+kr62ScpsCS7cwe45RKzZgp7dgUJnYHJpsKxl/eX/S/EWekh4FpPV7yLZo5BgOBeQSpOmqehTl1ngERPMdGypU8CD13snzzrB0e4K6kE5I7uUciOED3kGz9IbJStRqf5lQa/OZA7dJBxy7XZOKKKIXs9I3IR2JcTRZP/NU/fj9S++/5wgowGnyRyE84N75FQorUdGPG0+Hd7ui+OQDEtucydaI7segeGCou3Dp5QDCFMUFf8LbdPPgHz3XVeKDp58r1gz04wd6OMpRDpHLpO8d0PHIH4d/b76Xn6MRzuh+5CEHshDL48LB268y5WHsPn+SMwgKcQDuTLBYAGqjYtBCwVm/CjkuqdUqXvvEWcaXlBFxiv1ZtoE= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: cfab58bd-5c3f-4c97-f10d-08d783ba8a40 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2019 13:02:30.6779 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1aUOsqbsLUVESNRSeY1gP3HGah2tXPbhtQW4xVQmsznqbfaCTEySvoJTG2sJ/pIjRYQfCKLwmpIE7od+c0Znuw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB5093 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Pyverbs started out using Cython's __cinit__ method to initialize the underlying C objects. This is not recommended as the Python object may not be fully valid at this point. Move initialization to Python's __init__ instead. In addition, using __init__ allows us to control when and if the parent class's constructor is called. Signed-off-by: Noa Osherovich Reviewed-by: Edward Srouji --- pyverbs/addr.pyx | 23 ++++---- pyverbs/base.pyx | 4 +- pyverbs/cmid.pyx | 14 +++-- pyverbs/cq.pyx | 26 +++++---- pyverbs/device.pyx | 37 +++++++------ pyverbs/mr.pyx | 40 +++++++------- pyverbs/pd.pyx | 38 +++++++------- pyverbs/providers/mlx5/mlx5dv.pyx | 49 ++++++++++------- pyverbs/qp.pyx | 87 ++++++++++++++++--------------- pyverbs/srq.pyx | 20 ++++--- pyverbs/wr.pyx | 14 +++-- pyverbs/xrcd.pyx | 6 ++- 12 files changed, 200 insertions(+), 158 deletions(-) diff --git a/pyverbs/addr.pyx b/pyverbs/addr.pyx index 462a45bc7a4a..2ba611c26a89 100644 --- a/pyverbs/addr.pyx +++ b/pyverbs/addr.pyx @@ -18,7 +18,8 @@ cdef class GID(PyverbsObject): """ GID class represents ibv_gid. It enables user to query for GIDs values. """ - def __cinit__(self, val=None): + def __init__(self, val=None): + super().__init__() if val is not None: vals = gid_str_to_array(val) @@ -59,8 +60,8 @@ cdef class GRH(PyverbsObject): Represents ibv_grh struct. Used when creating or initializing an Address Handle from a Work Completion. """ - def __cinit__(self, GID sgid=None, GID dgid=None, version_tclass_flow=0, - paylen=0, next_hdr=0, hop_limit=1): + def __init__(self, GID sgid=None, GID dgid=None, version_tclass_flow=0, + paylen=0, next_hdr=0, hop_limit=1): """ Initializes a GRH object :param sgid: Source GID @@ -78,6 +79,7 @@ cdef class GRH(PyverbsObject): prior to being discarded :return: A GRH object """ + super().__init__() self.grh.dgid = dgid.gid self.grh.sgid = sgid.gid self.grh.version_tclass_flow = version_tclass_flow @@ -150,8 +152,8 @@ cdef class GlobalRoute(PyverbsObject): the values to be used in the GRH of the packets that will be sent using this Address Handle. """ - def __cinit__(self, GID dgid=None, flow_label=0, sgid_index=0, hop_limit=1, - traffic_class=0): + def __init__(self, GID dgid=None, flow_label=0, sgid_index=0, hop_limit=1, + traffic_class=0): """ Initializes a GlobalRoute object with given parameters. :param dgid: Destination GID @@ -167,6 +169,7 @@ cdef class GlobalRoute(PyverbsObject): delivery priority for routers :return: A GlobalRoute object """ + super().__init__() self.gr.dgid=dgid.gid self.gr.flow_label = flow_label self.gr.sgid_index = sgid_index @@ -222,8 +225,8 @@ cdef class GlobalRoute(PyverbsObject): cdef class AHAttr(PyverbsObject): """ Represents ibv_ah_attr struct """ - def __cinit__(self, dlid=0, sl=0, src_path_bits=0, static_rate=0, - is_global=0, port_num=1, GlobalRoute gr=None): + def __init__(self, dlid=0, sl=0, src_path_bits=0, static_rate=0, + is_global=0, port_num=1, GlobalRoute gr=None): """ Initializes an AHAttr object. :param dlid: Destination LID, a 16b unsigned integer @@ -242,6 +245,7 @@ cdef class AHAttr(PyverbsObject): is_global is non zero. :return: An AHAttr object """ + super().__init__() self.ah_attr.port_num = port_num self.ah_attr.sl = sl self.ah_attr.src_path_bits = src_path_bits @@ -363,7 +367,7 @@ cdef class AHAttr(PyverbsObject): cdef class AH(PyverbsCM): - def __cinit__(self, PD pd, **kwargs): + def __init__(self, PD pd, **kwargs): """ Initializes an AH object with the given values. Two creation methods are supported: @@ -371,7 +375,7 @@ cdef class AH(PyverbsCM): - Creation via a WC object (calls ibv_create_ah_from_wc) :param pd: PD object this AH belongs to :param kwargs: Arguments: - * *attr* (AHAttr) + * *attr* (AHAttr) An AHAttr object (represents ibv_ah_attr struct) * *wc* A WC object to use for AH initialization @@ -381,6 +385,7 @@ cdef class AH(PyverbsCM): Port number to be used for this AH (when using wc) :return: An AH object on success """ + super().__init__() if len(kwargs) == 1: # Create AH via ib_create_ah ah_attr = kwargs['attr'] diff --git a/pyverbs/base.pyx b/pyverbs/base.pyx index d69516285ece..c5b16795ddb6 100644 --- a/pyverbs/base.pyx +++ b/pyverbs/base.pyx @@ -5,9 +5,11 @@ import logging from pyverbs.pyverbs_error import PyverbsRDMAError from libc.errno cimport errno + cpdef PyverbsRDMAErrno(str msg): return PyverbsRDMAError(msg, errno) + LOG_LEVEL=logging.INFO LOG_FORMAT='[%(levelname)s] %(asctime)s %(filename)s:%(lineno)s: %(message)s' logging.basicConfig(format=LOG_FORMAT, level=LOG_LEVEL, datefmt='%d %b %Y %H:%M:%S') @@ -38,7 +40,7 @@ cdef close_weakrefs(iterables): cdef class PyverbsObject(object): - def __cinit__(self): + def __init__(self): self.logger = logging.getLogger(self.__class__.__name__) def set_log_level(self, val): diff --git a/pyverbs/cmid.pyx b/pyverbs/cmid.pyx index c752feda8781..5e4401436105 100755 --- a/pyverbs/cmid.pyx +++ b/pyverbs/cmid.pyx @@ -13,8 +13,8 @@ from pyverbs.cq cimport WC cdef class ConnParam(PyverbsObject): - def __cinit__(self, resources=1, depth=1, flow_control=0, retry=5, - rnr_retry=5, srq=0, qp_num=0): + def __init__(self, resources=1, depth=1, flow_control=0, retry=5, + rnr_retry=5, srq=0, qp_num=0): """ Initialize a ConnParam object over an underlying rdma_conn_param C object which contains connection parameters. There are a few types of @@ -38,6 +38,7 @@ cdef class ConnParam(PyverbsObject): CMID. :return: ConnParam object """ + super().__init__() memset(&self.conn_param, 0, sizeof(cm.rdma_conn_param)) self.conn_param.responder_resources = resources self.conn_param.initiator_depth = depth @@ -60,7 +61,7 @@ cdef class ConnParam(PyverbsObject): cdef class AddrInfo(PyverbsObject): - def __cinit__(self, node=None, service=None, port_space=0, flags=0): + def __init__(self, node=None, service=None, port_space=0, flags=0): """ Initialize an AddrInfo object over an underlying rdma_addrinfo C object. :param node: Name, dotted-decimal IPv4 or IPv6 hex address to resolve. @@ -75,6 +76,7 @@ cdef class AddrInfo(PyverbsObject): cdef cm.rdma_addrinfo hints cdef cm.rdma_addrinfo *hints_ptr = NULL + super().__init__() if node is not None: node = node.encode('utf-8') address = node @@ -102,8 +104,8 @@ cdef class AddrInfo(PyverbsObject): cdef class CMID(PyverbsCM): - def __cinit__(self, object creator=None, QPInitAttr qp_init_attr=None, - PD pd=None): + def __init__(self, object creator=None, QPInitAttr qp_init_attr=None, + PD pd=None): """ Initialize a CMID object over an underlying rdma_cm_id C object. This is the main RDMA CM object which provides most of the rdmacm API. @@ -118,6 +120,8 @@ cdef class CMID(PyverbsCM): """ cdef v.ibv_qp_init_attr *init cdef v.ibv_pd *in_pd = NULL + + super().__init__() self.pd = None self.ctx = None if creator is None: diff --git a/pyverbs/cq.pyx b/pyverbs/cq.pyx index 1ea443fc6966..dda47207507f 100755 --- a/pyverbs/cq.pyx +++ b/pyverbs/cq.pyx @@ -17,12 +17,13 @@ cdef class CompChannel(PyverbsCM): for a CQ, the event is delivered via the completion channel attached to the CQ. """ - def __cinit__(self, Context context not None): + def __init__(self, Context context not None): """ Initializes a completion channel object on the given device. :param context: The device's context to use :return: A CompChannel object on success """ + super().__init__() self.cc = v.ibv_create_comp_channel(context.context) if self.cc == NULL: raise PyverbsRDMAErrno('Failed to create a completion channel') @@ -68,8 +69,8 @@ cdef class CQ(PyverbsCM): A Completion Queue is the notification mechanism for work request completions. A CQ can have 0 or more associated QPs. """ - def __cinit__(self, Context context not None, cqe, cq_context=None, - CompChannel channel=None, comp_vector=0): + def __init__(self, Context context not None, cqe, cq_context=None, + CompChannel channel=None, comp_vector=0): """ Initializes a CQ object with the given parameters. :param context: The device's context on which to open the CQ @@ -81,6 +82,7 @@ cdef class CQ(PyverbsCM): context's num_comp_vectors :return: The newly created CQ """ + super().__init__() if channel is not None: self.cq = v.ibv_create_cq(context.context, cqe, cq_context, channel.cc, comp_vector) @@ -173,8 +175,8 @@ cdef class CQ(PyverbsCM): cdef class CqInitAttrEx(PyverbsObject): - def __cinit__(self, cqe = 100, CompChannel channel = None, comp_vector = 0, - wc_flags = 0, comp_mask = 0, flags = 0): + def __init__(self, cqe = 100, CompChannel channel = None, comp_vector = 0, + wc_flags = 0, comp_mask = 0, flags = 0): """ Initializes a CqInitAttrEx object with the given parameters. :param cqe: CQ's capacity @@ -189,6 +191,7 @@ cdef class CqInitAttrEx(PyverbsObject): ibv_create_cq_attr_flags enum :return: """ + super().__init__() self.attr.cqe = cqe self.attr.cq_context = NULL self.attr.channel = NULL if channel is None else channel.cc @@ -255,8 +258,7 @@ cdef class CqInitAttrEx(PyverbsObject): cdef class CQEX(PyverbsCM): - def __cinit__(self, Context context not None, CqInitAttrEx init_attr, - **kwargs): + def __init__(self, Context context not None, CqInitAttrEx init_attr): """ Initializes a CQEX object on the given device's context with the given attributes. @@ -264,9 +266,10 @@ cdef class CQEX(PyverbsCM): :param init_attr: Initial attributes that describe the CQ :return: The newly created CQEX on success """ + super().__init__() self.qps = weakref.WeakSet() self.srqs = weakref.WeakSet() - if len(kwargs) > 0: + if self.cq != NULL: # Leave CQ initialization to the provider return if init_attr is None: @@ -380,9 +383,10 @@ cdef class CQEX(PyverbsCM): cdef class WC(PyverbsObject): - def __cinit__(self, wr_id=0, status=0, opcode=0, vendor_err=0, byte_len=0, - qp_num=0, src_qp=0, imm_data=0, wc_flags=0, pkey_index=0, - slid=0, sl=0, dlid_path_bits=0): + def __init__(self, wr_id=0, status=0, opcode=0, vendor_err=0, byte_len=0, + qp_num=0, src_qp=0, imm_data=0, wc_flags=0, pkey_index=0, + slid=0, sl=0, dlid_path_bits=0): + super().__init__() self.wc.wr_id = wr_id self.wc.status = status self.wc.opcode = opcode diff --git a/pyverbs/device.pyx b/pyverbs/device.pyx index 56d7540ceb6c..529c4e4597c9 100755 --- a/pyverbs/device.pyx +++ b/pyverbs/device.pyx @@ -70,7 +70,7 @@ cdef class Context(PyverbsCM): """ Context class represents the C ibv_context. """ - def __cinit__(self, **kwargs): + def __init__(self, **kwargs): """ Initializes a Context object. The function searches the IB devices list for a device with the name provided by the user. If such a device is @@ -79,19 +79,22 @@ cdef class Context(PyverbsCM): initiated pointer, hence all we have to do is assign this pointer to Context's object pointer. :param kwargs: Arguments: - * *name* (str) - The RDMA device's name - * *attr* (object) - Device-specific attributes, meaning that the device is to be - opened by the provider - * *cmid* (CMID) - A CMID object (represents rdma_cm_id struct) + * *name* + The device's name + * *attr* + Provider-specific attributes. If not None, it means that the + device will be opened by the provider and __init__ will return + after locating the requested device. + * *cmid* + A CMID object. If not None, it means that the device was already + opened by a CMID class, and only a pointer assignment is missing. :return: None """ cdef int count cdef v.ibv_device **dev_list cdef CMID cmid + super().__init__() self.pds = weakref.WeakSet() self.dms = weakref.WeakSet() self.ccs = weakref.WeakSet() @@ -99,19 +102,16 @@ cdef class Context(PyverbsCM): self.qps = weakref.WeakSet() self.xrcds = weakref.WeakSet() - dev_name = kwargs.get('name') + self.name = kwargs.get('name') provider_attr = kwargs.get('attr') cmid = kwargs.get('cmid') - if cmid is not None: self.context = cmid.id.verbs cmid.ctx = self return - elif dev_name is not None: - self.name = dev_name - else: - raise PyverbsUserError('Device name must be provided') + if self.name is None: + raise PyverbsUserError('Device name must be provided') dev_list = v.ibv_get_device_list(&count) if dev_list == NULL: raise PyverbsRDMAError('Failed to get devices list') @@ -393,7 +393,8 @@ cdef class DeviceAttr(PyverbsObject): cdef class QueryDeviceExInput(PyverbsObject): - def __cinit__(self, comp_mask): + def __init__(self, comp_mask): + super().__init__() self.ex_input.comp_mask = comp_mask @@ -583,7 +584,7 @@ cdef class DeviceAttrEx(PyverbsObject): cdef class AllocDmAttr(PyverbsObject): - def __cinit__(self, length, log_align_req = 0, comp_mask = 0): + def __init__(self, length, log_align_req = 0, comp_mask = 0): """ Creates an AllocDmAttr object with the given parameters. This object can than be used to create a DM object. @@ -592,6 +593,7 @@ cdef class AllocDmAttr(PyverbsObject): :param comp_mask: compatibility mask :return: An AllocDmAttr object """ + super().__init__() self.alloc_dm_attr.length = length self.alloc_dm_attr.log_align_req = log_align_req self.alloc_dm_attr.comp_mask = comp_mask @@ -622,13 +624,14 @@ cdef class AllocDmAttr(PyverbsObject): cdef class DM(PyverbsCM): - def __cinit__(self, Context context, AllocDmAttr dm_attr not None): + def __init__(self, Context context, AllocDmAttr dm_attr not None): """ Allocate a device (direct) memory. :param context: The context of the device on which to allocate memory :param dm_attr: Attributes that define the DM :return: A DM object on success """ + super().__init__() self.dm_mrs = weakref.WeakSet() device_attr = context.query_device_ex() if device_attr.max_dm_size <= 0: diff --git a/pyverbs/mr.pyx b/pyverbs/mr.pyx index 8747d9cb81f9..6b28c8173ef8 100644 --- a/pyverbs/mr.pyx +++ b/pyverbs/mr.pyx @@ -1,15 +1,17 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2019, Mellanox Technologies. All rights reserved. See COPYING file +import resource +import logging + from pyverbs.pyverbs_error import PyverbsRDMAError, PyverbsError from pyverbs.base import PyverbsRDMAErrno -from pyverbs.device cimport DM -from .pd cimport PD -import resource from posix.stdlib cimport posix_memalign -from libc.stdlib cimport free from libc.string cimport memcpy, memset from libc.stdint cimport uintptr_t +from pyverbs.device cimport DM +from libc.stdlib cimport free +from .pd cimport PD cdef class MR(PyverbsCM): @@ -17,7 +19,7 @@ cdef class MR(PyverbsCM): MR class represents ibv_mr. Buffer allocation in done in the c'tor. Freeing it is done in close(). """ - def __cinit__(self, PD pd not None, length, access, **kwargs): + def __init__(self, PD pd not None, length, access): """ Allocate a user-level buffer of length and register a Memory Region of the given length and access flags. @@ -26,7 +28,8 @@ cdef class MR(PyverbsCM): :param access: Access flags, see ibv_access_flags enum :return: The newly created MR on success """ - if len(kwargs) != 0: + super().__init__() + if self.mr != NULL: return #We want to enable registering an MR of size 0 but this fails with a #buffer of size 0, so in this case lets increase the buffer @@ -107,13 +110,14 @@ cdef class MR(PyverbsCM): cdef class MW(PyverbsCM): - def __cinit__(self, PD pd not None, v.ibv_mw_type mw_type): + def __init__(self, PD pd not None, v.ibv_mw_type mw_type): """ Initializes a memory window object of the given type :param pd: A PD object :param mw_type: Type of of the memory window, see ibv_mw_type enum :return: """ + super().__init__() self.mw = NULL self.mw = v.ibv_alloc_mw(pd.pd, mw_type) if self.mw == NULL: @@ -144,29 +148,27 @@ cdef class MW(PyverbsCM): cdef class DMMR(MR): - def __cinit__(self, PD pd not None, length, access, **kwargs): + def __init__(self, PD pd not None, length, access, DM dm, offset): """ Initializes a DMMR (Device Memory Memory Region) of the given length and access flags using the given PD and DM objects. :param pd: A PD object :param length: Length in bytes :param access: Access flags, see ibv_access_flags enum - :param kwargs: see below + :param dm: A DM (device memory) object to be used for this DMMR + :param offset: Byte offset from the beginning of the allocated device + memory buffer :return: The newly create DMMR - - :keyword Arguments: - * *dm* (DM) - A DM (device memory) object to be used for this DMMR - * *offset* - Byte offset from the beginning of the allocated device memory - buffer """ - dm = kwargs['dm'] - offset = kwargs['offset'] - self.mr = v.ibv_reg_dm_mr(pd.pd, (dm).dm, offset, length, access) + # Initialize the logger here as the parent's __init__ is called after + # the DMMR is allocated. Allocation can fail, which will lead to + # exceptions thrown during object's teardown. + self.logger = logging.getLogger(self.__class__.__name__) + self.mr = v.ibv_reg_dm_mr(pd.pd, dm.dm, offset, length, access) if self.mr == NULL: raise PyverbsRDMAErrno('Failed to register a device MR. length: {len}, access flags: {flags}'. format(len=length, flags=access,)) + super().__init__(pd, length, access) self.pd = pd self.dm = dm pd.add_ref(self) diff --git a/pyverbs/pd.pyx b/pyverbs/pd.pyx index 8c17ffcba7e8..96d0078e3dbd 100755 --- a/pyverbs/pd.pyx +++ b/pyverbs/pd.pyx @@ -1,6 +1,7 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2019, Mellanox Technologies. All rights reserved. import weakref +import logging from pyverbs.pyverbs_error import PyverbsUserError, PyverbsError from pyverbs.base import PyverbsRDMAErrno @@ -15,20 +16,16 @@ from pyverbs.qp cimport QP cdef class PD(PyverbsCM): - def __cinit__(self, object creator not None, **kwargs): + def __init__(self, object creator not None): """ Initializes a PD object. A reference for the creating Context is kept so that Python's GC will destroy the objects in the right order. - :param context: The Context object creating the PD - :param kwargs: Arguments: - * *attr* (object) - If provided PD will not be allocated, leaving the allocation to - be made by an inheriting class + :param creator: The Context/CMID object creating the PD """ + super().__init__() if issubclass(type(creator), Context): - # If there's a Parent Domain attribute skip PD allocation - # since this is done by the Parent Domain class - if not kwargs.get('attr'): + # Check if the ibv_pd* was initialized by an inheriting class + if self.pd == NULL: self.pd = v.ibv_alloc_pd((creator).context) if self.pd == NULL: raise PyverbsRDMAErrno('Failed to allocate PD') @@ -130,7 +127,7 @@ cdef void pd_free(v.ibv_pd *pd, void *pd_context, void *ptr, cdef class ParentDomainContext(PyverbsObject): - def __cinit__(self, PD pd, alloc_func, free_func): + def __init__(self, PD pd, alloc_func, free_func): """ Initializes ParentDomainContext object which is used as a pd_context. It contains the relevant fields in order to allow the user to write @@ -140,19 +137,21 @@ cdef class ParentDomainContext(PyverbsObject): :param alloc_func: Python alloc function :param free_func: Python free function """ + super().__init__() self.pd = pd self.p_alloc = alloc_func self.p_free = free_func cdef class ParentDomainInitAttr(PyverbsObject): - def __cinit__(self, PD pd not None, ParentDomainContext pd_context=None): + def __init__(self, PD pd not None, ParentDomainContext pd_context=None): """ Represents ibv_parent_domain_init_attr C struct :param pd: PD to initialize the ParentDomain with :param pd_context: ParentDomainContext object including the alloc and free Python callbacks """ + super().__init__() self.pd = pd self.init_attr.pd = pd.pd if pd_context: @@ -171,25 +170,24 @@ cdef class ParentDomainInitAttr(PyverbsObject): cdef class ParentDomain(PD): - def __cinit__(self, Context context not None, **kwargs): + def __init__(self, Context context not None, ParentDomainInitAttr attr not None): """ Initializes ParentDomain object which represents a parent domain of ibv_pd C struct type :param context: Device context - :param kwargs: Arguments: - * *attr* (object) - Attribute of type ParentDomainInitAttr to initialize the - ParentDomain with + :param attr: Attribute of type ParentDomainInitAttr to initialize the + ParentDomain with """ - cdef ParentDomainInitAttr attr - attr = kwargs.get('attr') - if attr is None: - raise PyverbsUserError('ParentDomain must take attr') + # Initialize the logger here as the parent's __init__ is called after + # the PD is allocated. Allocation can fail, which will lead to exceptions + # thrown during object's teardown. + self.logger = logging.getLogger(self.__class__.__name__) (attr.pd).add_ref(self) self.protection_domain = attr.pd self.pd = v.ibv_alloc_parent_domain(context.context, &attr.init_attr) if self.pd == NULL: raise PyverbsRDMAErrno('Failed to allocate Parent Domain') + super().__init__(context) self.logger.debug('Allocated ParentDomain') def __dealloc__(self): diff --git a/pyverbs/providers/mlx5/mlx5dv.pyx b/pyverbs/providers/mlx5/mlx5dv.pyx index e4500567ba23..775c5504c2bb 100644 --- a/pyverbs/providers/mlx5/mlx5dv.pyx +++ b/pyverbs/providers/mlx5/mlx5dv.pyx @@ -1,6 +1,8 @@ # SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) # Copyright (c) 2019 Mellanox Technologies, Inc. All rights reserved. See COPYING file +import logging + from pyverbs.pyverbs_error import PyverbsUserError cimport pyverbs.providers.mlx5.mlx5dv_enums as dve cimport pyverbs.providers.mlx5.libmlx5 as dv @@ -16,7 +18,8 @@ cdef class Mlx5DVContextAttr(PyverbsObject): Represent mlx5dv_context_attr struct. This class is used to open an mlx5 device. """ - def __cinit__(self, flags=0, comp_mask=0): + def __init__(self, flags=0, comp_mask=0): + super().__init__() self.attr.flags = flags self.attr.comp_mask = comp_mask @@ -44,22 +47,16 @@ cdef class Mlx5Context(Context): """ Represent mlx5 context, which extends Context. """ - def __cinit__(self, **kwargs): + def __init__(self, Mlx5DVContextAttr attr not None, name=''): """ Open an mlx5 device using the given attributes - :param kwargs: Arguments: - * *name* (str) - The RDMA device's name (used by parent class) - * *attr* (Mlx5DVContextAttr) - mlx5-specific device attributes + :param name: The RDMA device's name (used by parent class) + :param attr: mlx5-specific device attributes :return: None """ - cdef Mlx5DVContextAttr attr - attr = kwargs.get('attr') - if not attr or not isinstance(attr, Mlx5DVContextAttr): - raise PyverbsUserError('Missing provider attributes') if not dv.mlx5dv_is_supported(self.device): raise PyverbsUserError('This is not an MLX5 device') + super().__init__(name=name, attr=attr) self.context = dv.mlx5dv_open_device(self.device, &attr.attr) def query_mlx5_device(self, comp_mask=-1): @@ -187,7 +184,7 @@ cdef class Mlx5DVDCInitAttr(PyverbsObject): Represents mlx5dv_dc_init_attr struct, which defines initial attributes for DC QP creation. """ - def __cinit__(self, dc_type=dve.MLX5DV_DCTYPE_DCI, dct_access_key=0): + def __init__(self, dc_type=dve.MLX5DV_DCTYPE_DCI, dct_access_key=0): """ Initializes an Mlx5DVDCInitAttr object with the given DC type and DCT access key. @@ -195,6 +192,7 @@ cdef class Mlx5DVDCInitAttr(PyverbsObject): :param dct_access_key: Access key to be used by the DCT :return: An initializes object """ + super().__init__() self.attr.dc_type = dc_type self.attr.dct_access_key = dct_access_key @@ -223,8 +221,8 @@ cdef class Mlx5DVQPInitAttr(PyverbsObject): Represents mlx5dv_qp_init_attr struct, initial attributes used for mlx5 QP creation. """ - def __cinit__(self, comp_mask=0, create_flags=0, - Mlx5DVDCInitAttr dc_init_attr=None, send_ops_flags=0): + def __init__(self, comp_mask=0, create_flags=0, + Mlx5DVDCInitAttr dc_init_attr=None, send_ops_flags=0): """ Initializes an Mlx5DVQPInitAttr object with the given user data. :param comp_mask: A bitmask specifying which fields are valid @@ -233,6 +231,7 @@ cdef class Mlx5DVQPInitAttr(PyverbsObject): :param send_ops_flags: A bitwise OR of mlx5dv_qp_create_send_ops_flags :return: An initialized Mlx5DVQPInitAttr object """ + super().__init__() self.attr.comp_mask = comp_mask self.attr.create_flags = create_flags self.attr.send_ops_flags = send_ops_flags @@ -291,8 +290,8 @@ cdef class Mlx5DVQPInitAttr(PyverbsObject): cdef class Mlx5QP(QP): - def __cinit__(self, Mlx5Context context, QPInitAttrEx init_attr, - Mlx5DVQPInitAttr dv_init_attr): + def __init__(self, Mlx5Context context, QPInitAttrEx init_attr, + Mlx5DVQPInitAttr dv_init_attr): """ Initializes an mlx5 QP according to the user-provided data. :param context: mlx5 Context object @@ -301,6 +300,11 @@ cdef class Mlx5QP(QP): :return: An initialized Mlx5QP """ cdef PD pd + + # Initialize the logger here as the parent's __init__ is called after + # the QP is allocated. Allocation can fail, which will lead to exceptions + # thrown during object's teardown. + self.logger = logging.getLogger(self.__class__.__name__) self.dc_type = dv_init_attr.dc_type if dv_init_attr else 0 if init_attr.pd is not None: pd = init_attr.pd @@ -314,6 +318,7 @@ cdef class Mlx5QP(QP): raise PyverbsRDMAErrno('Failed to create MLX5 QP.\nQPInitAttrEx ' 'attributes:\n{}\nMLX5DVQPInitAttr:\n{}'. format(init_attr, dv_init_attr)) + super().__init__(context, init_attr) def _get_comp_mask(self, dst): masks = {dve.MLX5DV_DCTYPE_DCT: {'INIT': e.IBV_QP_PKEY_INDEX | @@ -338,7 +343,7 @@ cdef class Mlx5DVCQInitAttr(PyverbsObject): Represents mlx5dv_cq_init_attr struct, initial attributes used for mlx5 CQ creation. """ - def __cinit__(self, comp_mask=0, cqe_comp_res_format=0, flags=0, cqe_size=0): + def __init__(self, comp_mask=0, cqe_comp_res_format=0, flags=0, cqe_size=0): """ Initializes an Mlx5CQInitAttr object with zeroes as default values. :param comp_mask: Marks which of the following fields should be @@ -353,6 +358,7 @@ cdef class Mlx5DVCQInitAttr(PyverbsObject): Valid when MLX5DV_CQ_INIT_ATTR_MASK_CQE_SIZE is set. :return: None """ + super().__init__() self.attr.comp_mask = comp_mask self.attr.cqe_comp_res_format = cqe_comp_res_format self.attr.flags = flags @@ -413,8 +419,12 @@ cdef class Mlx5DVCQInitAttr(PyverbsObject): cdef class Mlx5CQ(CQEX): - def __cinit__(self, Mlx5Context context, CqInitAttrEx init_attr, - Mlx5DVCQInitAttr dv_init_attr): + def __init__(self, Mlx5Context context, CqInitAttrEx init_attr, + Mlx5DVCQInitAttr dv_init_attr): + # Initialize the logger here as the parent's __init__ is called after + # the CQ is allocated. Allocation can fail, which will lead to exceptions + # thrown during object's teardown. + self.logger = logging.getLogger(self.__class__.__name__) self.cq = \ dv.mlx5dv_create_cq(context.context, &init_attr.attr, &dv_init_attr.attr if dv_init_attr is not None @@ -426,6 +436,7 @@ cdef class Mlx5CQ(CQEX): self.ibv_cq = v.ibv_cq_ex_to_cq(self.cq) self.context = context context.add_ref(self) + super().__init__(context, init_attr) def __str__(self): print_format = '{:<22}: {:<20}\n' diff --git a/pyverbs/qp.pyx b/pyverbs/qp.pyx index 36698d2119e8..9d368b62022d 100755 --- a/pyverbs/qp.pyx +++ b/pyverbs/qp.pyx @@ -18,8 +18,8 @@ from libc.string cimport memcpy cdef class QPCap(PyverbsObject): - def __cinit__(self, max_send_wr=1, max_recv_wr=10, max_send_sge=1, - max_recv_sge=1, max_inline_data=0): + def __init__(self, max_send_wr=1, max_recv_wr=10, max_send_sge=1, + max_recv_sge=1, max_inline_data=0): """ Initializes a QPCap object with user-provided or default values. :param max_send_wr: max number of outstanding WRs in the SQ @@ -32,6 +32,7 @@ cdef class QPCap(PyverbsObject): inline to the SQ, otherwise 0 :return: """ + super().__init__() self.cap.max_send_wr = max_send_wr self.cap.max_recv_wr = max_recv_wr self.cap.max_send_sge = max_send_sge @@ -83,9 +84,9 @@ cdef class QPCap(PyverbsObject): cdef class QPInitAttr(PyverbsObject): - def __cinit__(self, qp_type=e.IBV_QPT_UD, qp_context=None, - PyverbsObject scq=None, PyverbsObject rcq=None, - SRQ srq=None, QPCap cap=None, sq_sig_all=1): + def __init__(self, qp_type=e.IBV_QPT_UD, qp_context=None, + PyverbsObject scq=None, PyverbsObject rcq=None, + SRQ srq=None, QPCap cap=None, sq_sig_all=1): """ Initializes a QpInitAttr object representing ibv_qp_init_attr struct. Note that SRQ object is not yet supported in pyverbs so can't be passed @@ -100,6 +101,7 @@ cdef class QPInitAttr(PyverbsObject): entry :return: A QpInitAttr object """ + super().__init__() _copy_caps(cap, self) self.attr.qp_context = qp_context if scq is not None: @@ -233,12 +235,12 @@ cdef class QPInitAttr(PyverbsObject): cdef class QPInitAttrEx(PyverbsObject): - def __cinit__(self, qp_type=e.IBV_QPT_UD, qp_context=None, - PyverbsObject scq=None, PyverbsObject rcq=None, - SRQ srq=None, QPCap cap=None, sq_sig_all=0, comp_mask=0, - PD pd=None, XRCD xrcd=None, create_flags=0, - max_tso_header=0, source_qpn=0, object hash_conf=None, - object ind_table=None): + def __init__(self, qp_type=e.IBV_QPT_UD, qp_context=None, + PyverbsObject scq=None, PyverbsObject rcq=None, + SRQ srq=None, QPCap cap=None, sq_sig_all=0, comp_mask=0, + PD pd=None, XRCD xrcd=None, create_flags=0, + max_tso_header=0, source_qpn=0, object hash_conf=None, + object ind_table=None): """ Initialize a QPInitAttrEx object with user-defined or default values. :param qp_type: QP type to be created @@ -261,6 +263,7 @@ cdef class QPInitAttrEx(PyverbsObject): :param ind_table: Not yet supported :return: An initialized QPInitAttrEx object """ + super().__init__() _copy_caps(cap, self) if scq is not None: if type(scq) is CQ: @@ -481,8 +484,8 @@ cdef class QPInitAttrEx(PyverbsObject): cdef class QPAttr(PyverbsObject): - def __cinit__(self, qp_state=e.IBV_QPS_INIT, cur_qp_state=e.IBV_QPS_RESET, - port_num=1, path_mtu=e.IBV_MTU_1024): + def __init__(self, qp_state=e.IBV_QPS_INIT, cur_qp_state=e.IBV_QPS_RESET, + port_num=1, path_mtu=e.IBV_MTU_1024): """ Initializes a QPQttr object which represents ibv_qp_attr structs. It can be used to modify a QP. @@ -491,6 +494,7 @@ cdef class QPAttr(PyverbsObject): :param cur_qp_state: Current QP state :return: An initialized QpAttr object """ + super().__init__() self.attr.qp_state = qp_state self.attr.cur_qp_state = cur_qp_state self.attr.port_num = port_num @@ -854,8 +858,8 @@ cdef class QPAttr(PyverbsObject): cdef class QP(PyverbsCM): - def __cinit__(self, object creator not None, object init_attr not None, - QPAttr qp_attr=None, **kwargs): + def __init__(self, object creator not None, object init_attr not None, + QPAttr qp_attr=None): """ Initializes a QP object and performs state transitions according to user request. @@ -875,39 +879,38 @@ cdef class QP(PyverbsCM): using Context). :param qp_attr: Optional QPAttr object. Will be used for QP state transitions after creation. - :param kwargs: Provider-specific QP creation attributes, meaning that - the QP will be created by the provider. :return: An initialized QP object """ cdef PD pd cdef Context ctx + super().__init__() self.update_cqs(init_attr) - if len(kwargs) > 0: - # Leave QP initialization to the provider - return - # In order to use cdef'd methods, a proper casting must be done, let's - # infer the type. - if issubclass(type(creator), Context): - self._create_qp_ex(creator, init_attr) - ctx = creator - self.context = ctx - ctx.add_ref(self) - if init_attr.pd is not None: - pd = init_attr.pd - pd.add_ref(self) - self.pd = pd - if init_attr.xrcd is not None: - xrcd = init_attr.xrcd - xrcd.add_ref(self) - self.xrcd = xrcd - else: - self._create_qp(creator, init_attr) - pd = creator - self.pd = pd - pd.add_ref(self) - self.context = None + # QP initialization was not done by the provider, we should do it here if self.qp == NULL: - raise PyverbsRDMAErrno('Failed to create QP') + # In order to use cdef'd methods, a proper casting must be done, + # let's infer the type. + if issubclass(type(creator), Context): + self._create_qp_ex(creator, init_attr) + ctx = creator + self.context = ctx + ctx.add_ref(self) + if init_attr.pd is not None: + pd = init_attr.pd + pd.add_ref(self) + self.pd = pd + if init_attr.xrcd is not None: + xrcd = init_attr.xrcd + xrcd.add_ref(self) + self.xrcd = xrcd + else: + self._create_qp(creator, init_attr) + pd = creator + self.pd = pd + pd.add_ref(self) + self.context = None + if self.qp == NULL: + raise PyverbsRDMAErrno('Failed to create QP') + if qp_attr is not None: funcs = {e.IBV_QPT_RC: self.to_init, e.IBV_QPT_UC: self.to_init, e.IBV_QPT_UD: self.to_rts, diff --git a/pyverbs/srq.pyx b/pyverbs/srq.pyx index a60aa5dcb0e5..7dc24d3849e4 100755 --- a/pyverbs/srq.pyx +++ b/pyverbs/srq.pyx @@ -10,7 +10,8 @@ from libc.string cimport memcpy cdef class SrqAttr(PyverbsObject): - def __cinit__(self, max_wr=100, max_sge=1, srq_limit=0): + def __init__(self, max_wr=100, max_sge=1, srq_limit=0): + super().__init__() self.attr.max_wr = max_wr self.attr.max_sge = max_sge self.attr.srq_limit = srq_limit @@ -38,11 +39,12 @@ cdef class SrqAttr(PyverbsObject): cdef class SrqInitAttr(PyverbsObject): - def __cinit__(self, SrqAttr attr = None): - if attr is not None: - self.attr.attr.max_wr = attr.max_wr - self.attr.attr.max_sge = attr.max_sge - self.attr.attr.srq_limit = attr.srq_limit + def __init__(self, SrqAttr attr = None): + super().__init__() + if attr is not None: + self.attr.attr.max_wr = attr.max_wr + self.attr.attr.max_sge = attr.max_sge + self.attr.attr.srq_limit = attr.srq_limit @property def max_wr(self): @@ -58,7 +60,8 @@ cdef class SrqInitAttr(PyverbsObject): cdef class SrqInitAttrEx(PyverbsObject): - def __cinit__(self, max_wr=100, max_sge=1, srq_limit=0): + def __init__(self, max_wr=100, max_sge=1, srq_limit=0): + super().__init__() self.attr.attr.max_wr = max_wr self.attr.attr.max_sge = max_sge self.attr.attr.srq_limit = srq_limit @@ -122,7 +125,8 @@ cdef class SrqInitAttrEx(PyverbsObject): cdef class SRQ(PyverbsCM): - def __cinit__(self, object creator not None, object attr not None): + def __init__(self, object creator not None, object attr not None): + super().__init__() self.srq = NULL self.cq = None if isinstance(creator, PD): diff --git a/pyverbs/wr.pyx b/pyverbs/wr.pyx index 8505c1cfddff..06186e9b0fd2 100644 --- a/pyverbs/wr.pyx +++ b/pyverbs/wr.pyx @@ -17,7 +17,7 @@ cdef class SGE(PyverbsCM): write can't be done using memcpy that relies on CPU-specific optimizations. A SGE has no way to tell which memory it is using. """ - def __cinit__(self, addr, length, lkey): + def __init__(self, addr, length, lkey): """ Initializes a SGE object. :param addr: The address to be used for read/write @@ -25,6 +25,7 @@ cdef class SGE(PyverbsCM): :param lkey: Local key of the used MR/DMMR :return: A SGE object """ + super().__init__() self.sge = malloc(sizeof(v.ibv_sge)) if self.sge == NULL: raise PyverbsError('Failed to allocate an SGE') @@ -80,8 +81,8 @@ cdef class SGE(PyverbsCM): cdef class RecvWR(PyverbsCM): - def __cinit__(self, wr_id=0, num_sge=0, sg=None, - RecvWR next_wr=None): + def __init__(self, wr_id=0, num_sge=0, sg=None, + RecvWR next_wr=None): """ Initializes a RecvWR object. :param wr_id: A user-defined WR ID @@ -90,6 +91,7 @@ cdef class RecvWR(PyverbsCM): :param: next_wr: The next WR in the list :return: A RecvWR object """ + super().__init__() cdef v.ibv_sge *dst if num_sge < 1 or sg is None: raise PyverbsUserError('A WR needs at least one SGE') @@ -141,8 +143,8 @@ cdef class RecvWR(PyverbsCM): cdef class SendWR(PyverbsCM): - def __cinit__(self, wr_id=0, opcode=e.IBV_WR_SEND, num_sge=0, sg = None, - send_flags=e.IBV_SEND_SIGNALED, SendWR next_wr = None): + def __init__(self, wr_id=0, opcode=e.IBV_WR_SEND, num_sge=0, sg = None, + send_flags=e.IBV_SEND_SIGNALED, SendWR next_wr = None): """ Initialize a SendWR object with user-provided or default values. :param wr_id: A user-defined WR ID @@ -153,6 +155,8 @@ cdef class SendWR(PyverbsCM): :return: An initialized SendWR object """ cdef v.ibv_sge *dst + + super().__init__() if num_sge < 1 or sg is None: raise PyverbsUserError('A WR needs at least one SGE') self.send_wr.sg_list = malloc(num_sge * sizeof(v.ibv_sge)) diff --git a/pyverbs/xrcd.pyx b/pyverbs/xrcd.pyx index 91303daf3bc0..774f60e60dda 100755 --- a/pyverbs/xrcd.pyx +++ b/pyverbs/xrcd.pyx @@ -12,7 +12,8 @@ from libc.errno cimport errno cdef class XRCDInitAttr(PyverbsObject): - def __cinit__(self, comp_mask, oflags, fd): + def __init__(self, comp_mask, oflags, fd): + super().__init__() self.attr.fd = fd self.attr.comp_mask = comp_mask self.attr.oflags = oflags @@ -40,12 +41,13 @@ cdef class XRCDInitAttr(PyverbsObject): cdef class XRCD(PyverbsCM): - def __cinit__(self, Context context not None, XRCDInitAttr init_attr not None): + def __init__(self, Context context not None, XRCDInitAttr init_attr not None): """ Initializes a XRCD object. :param context: The Context object creating the XRCD :return: The newly created XRCD on success """ + super().__init__() self.xrcd = v.ibv_open_xrcd( context.context, &init_attr.attr) if self.xrcd == NULL: From patchwork Wed Dec 18 13:02:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noa Osherovich X-Patchwork-Id: 11300649 X-Patchwork-Delegate: leon@leon.nu Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1834B139A for ; Wed, 18 Dec 2019 13:02:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DFC2C218AC for ; Wed, 18 Dec 2019 13:02:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="VHYzClfS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726913AbfLRNCr (ORCPT ); Wed, 18 Dec 2019 08:02:47 -0500 Received: from mail-eopbgr30089.outbound.protection.outlook.com ([40.107.3.89]:39813 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726718AbfLRNCr (ORCPT ); Wed, 18 Dec 2019 08:02:47 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NoVWsDkESYNziDQC87jCvgwwNL++4eTWeTnY8211Yub2uzpxG9IRCpDob4niWdRxMdZFq/fuY3qR8V8giS+gOgJL+XFx9mBb9HiNDeX8G+/iGtwqKfR/pJVqMZ2yC4G2MYD3n1ybb4Y1Gl1e8FY/F0U4dpqpFsV77bZ0SD/6zctpNaCWa8WiLVO6zPyH++mbzUo4NUhuLCQpikOayHY4vYcpqGKWO528Ry9KEDRFPkg4huQT+5b8NTHbBQ1+hfWex2sshZlIabmO9qImysMTQKgf4sE6IAmMfORnhq3dzoEFL+Hhs+FxuKOu/VDjVt5ZH5451fdrEEZS2bTUqs8Q0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1UsZwTsQ4L9eOmauYmtxHgd05gIy6FFKuPQC1GSdxio=; b=BJZd8qj3hYIOgB46VdMy+XvWM+0GIv8b3ul2TFGth9s13vC/2WTRukKIG7RxwxMqWrKITF4OLB7MVqcpwLhMSYHJ/ppaGQur6ELIN1GRw6pQdELD4YwUP4H9uaBKByE4b1oAxSjoR98WiFXs9TAuTjiGPJio8a77ED2iocQanaHash7vL9BKyU4ERWRK8dhMP3C49sYSBq14rF5/Bh3xdJpri6nT0ZjZL5YR+VmY7sRceQAvYQk3UHd+A3bQqnhDtEYtDPQZe49CRbUNcwHWyVNyvyqf8uy9QTX9zw4iHx/rIh6hJvxJvWpxZIgBtEVRv0rE/q8BSfq6pcEU23rX8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1UsZwTsQ4L9eOmauYmtxHgd05gIy6FFKuPQC1GSdxio=; b=VHYzClfSaPoOTKCqAjsmCqJzPmVYV2qmxCz5o2sR07MHmgyiEeLGN0aDxUPq7NWqCy+prZ0Qh92LaCTiq00GLMApFND5pZJNWovyjv1Tnbwxb/4kXYNM9dJDjCUEFjwpvKs7m7Vl//5Y9XokowZ+QI21z8wLUXLKTw3aobvFa0c= Received: from AM6PR05MB4968.eurprd05.prod.outlook.com (20.177.33.17) by AM6PR05MB5093.eurprd05.prod.outlook.com (20.177.35.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.20; Wed, 18 Dec 2019 13:02:31 +0000 Received: from AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca]) by AM6PR05MB4968.eurprd05.prod.outlook.com ([fe80::bc8c:12ca:edd3:92ca%4]) with mapi id 15.20.2538.019; Wed, 18 Dec 2019 13:02:31 +0000 From: Noa Osherovich To: "dledford@redhat.com" , Jason Gunthorpe , Leon Romanovsky CC: "linux-rdma@vger.kernel.org" , Noa Osherovich , Edward Srouji Subject: [PATCH rdma-core 3/3] tests: Some cleanup Thread-Topic: [PATCH rdma-core 3/3] tests: Some cleanup Thread-Index: AQHVtaNoRHlseBtdv02G2wZUyHzIlQ== Date: Wed, 18 Dec 2019 13:02:31 +0000 Message-ID: <20191218130216.503-4-noaos@mellanox.com> References: <20191218130216.503-1-noaos@mellanox.com> In-Reply-To: <20191218130216.503-1-noaos@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-clientproxiedby: ZR0P278CA0009.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:16::19) To AM6PR05MB4968.eurprd05.prod.outlook.com (2603:10a6:20b:4::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=noaos@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [94.188.199.18] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d7fe6c08-01b8-493b-6e14-08d783ba8afd x-ms-traffictypediagnostic: AM6PR05MB5093:|AM6PR05MB5093: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:241; x-forefront-prvs: 0255DF69B9 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(396003)(346002)(376002)(39860400002)(366004)(136003)(199004)(189003)(86362001)(1076003)(316002)(6486002)(478600001)(6506007)(5660300002)(66446008)(64756008)(66556008)(66946007)(186003)(6636002)(2906002)(26005)(81166006)(107886003)(8676002)(81156014)(52116002)(66476007)(71200400001)(6512007)(54906003)(36756003)(2616005)(110136005)(4326008)(8936002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB5093;H:AM6PR05MB4968.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oSUO1kdq4M52THTrNknuQRwJyB+1iqoRY5eB9x/Nuars+EvosPaqVEOQX/l705EYS47gyKfjVZrTO5kFtrvmwz7sPHwR1eM1yJZzMFxUFSHeVzavnrJw7JH29exoRSGdL+F/+5zXO0xhi5brtMSyyQZDpbbxLvo9vhza5cnD95XhAUDn9Hm8q4FYL3GvFTAiE7Cly/K2cb5Nnbwle77NuhxoEMvTI2BozgkEssCqc/Y1lMB6AAF8EbAj8/jOUYe5D9vFB2FrUcuIMQQot3Xyd5a+heHR2Gas5Ucr+OTPl9fIxsTqt17rrwkb2wn+0xJXVEmeBzEoNzvG7Q/fZHXWHrfk958ox6+0dDRGlwaonKO+lmvsXJOL4AbV5Ptc1JtfBeqsuTciNK1ZjEApX1yxJGEJkINeJFEnSTaA1xE4+sJNNltValshmQ86F6P9lG1q MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: d7fe6c08-01b8-493b-6e14-08d783ba8afd X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2019 13:02:31.7623 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: wHFAWT94kv+KvhfwdLc8s2BP6UYzQ0/u0GBGOiBqq2i8m/Ha1LqhglwBnnpUV1bwHZLN2Xnl7gd3yKBzsLPy2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB5093 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Some classes' constructors were modified to include 'not None' parameters. Checking these scenarios gives no additional coverage as it checks Python/Cython and not rdma-core. Remove these test cases. Adding TSO header to a QP's init_attr_ex increases send WQE size. In some executions of the test, the randomized value of max_send_sge is too high combined with TSO. Decrease number of send SGEs in case of TSO to avoid these failures. Signed-off-by: Noa Osherovich Reviewed-by: Edward Srouji --- tests/test_mr.py | 14 -------------- tests/test_pd.py | 12 ------------ tests/test_qp.py | 2 -- tests/utils.py | 3 +++ 4 files changed, 3 insertions(+), 28 deletions(-) diff --git a/tests/test_mr.py b/tests/test_mr.py index 1b5482884be6..742c6032b1b3 100644 --- a/tests/test_mr.py +++ b/tests/test_mr.py @@ -45,20 +45,6 @@ class MRTest(PyverbsAPITestCase): with MR(pd, u.get_mr_length(), f) as mr: mr.close() - @staticmethod - def test_reg_mr_bad_flow(): - """ - Verify that trying to register a MR with None PD fails - """ - try: - # Use the simplest access flags necessary - MR(None, random.randint(0, 10000), e.IBV_ACCESS_LOCAL_WRITE) - except TypeError as te: - assert 'expected pyverbs.pd.PD' in te.args[0] - assert 'got NoneType' in te.args[0] - else: - raise PyverbsRDMAErrno('Created a MR with None PD') - def test_dereg_mr_twice(self): """ Verify that explicit call to MR's close() doesn't fail diff --git a/tests/test_pd.py b/tests/test_pd.py index a8d6eb2fb69f..d1b9c7e8a94f 100755 --- a/tests/test_pd.py +++ b/tests/test_pd.py @@ -40,18 +40,6 @@ class PDTest(PyverbsAPITestCase): with PD(ctx) as pd: pd.close() - @staticmethod - def test_create_pd_none_ctx(): - """ - Verify that PD can't be created with a None context - """ - try: - PD(None) - except TypeError as te: - assert 'must not be None' in te.args[0] - else: - raise PyverbsRDMAErrno('Created a PD with None context') - def test_destroy_pd_twice(self): """ Test bad flow cases in destruction of a PD object diff --git a/tests/test_qp.py b/tests/test_qp.py index 35a20adec5f1..e7fc447937fd 100644 --- a/tests/test_qp.py +++ b/tests/test_qp.py @@ -154,8 +154,6 @@ class QPTest(PyverbsAPITestCase): with PD(ctx) as pd: with CQ(ctx, 100, None, None, 0) as cq: for i in range(1, attr.phys_port_cnt + 1): - qpts = [e.IBV_QPT_UD, e.IBV_QPT_RAW_PACKET] \ - if is_eth(ctx, i) else [e.IBV_QPT_UD] qia = get_qp_init_attr_ex(cq, pd, attr, attr_ex, e.IBV_QPT_UD) with QP(ctx, qia, QPAttr()) as qp: diff --git a/tests/utils.py b/tests/utils.py index ab30f6692951..c45170dbd329 100755 --- a/tests/utils.py +++ b/tests/utils.py @@ -227,6 +227,9 @@ def random_qp_init_attr_ex(attr_ex, attr, qpt=None): random.randint(16, int(attr_ex.tso_caps.max_tso / 400)) qia = QPInitAttrEx(qp_type=qpt, cap=qp_cap, sq_sig_all=sig, comp_mask=mask, create_flags=cflags, max_tso_header=max_tso) + if mask & e.IBV_QP_INIT_ATTR_MAX_TSO_HEADER: + # TSO increases send WQE size, let's be on the safe side + qia.cap.max_send_sge = 2 return qia