From patchwork Thu Sep 17 13:57:37 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Barak X-Patchwork-Id: 7207821 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1E7219F380 for ; Thu, 17 Sep 2015 13:58:25 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 27F6120797 for ; Thu, 17 Sep 2015 13:58:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 16A002078F for ; Thu, 17 Sep 2015 13:58:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751112AbbIQN6T (ORCPT ); Thu, 17 Sep 2015 09:58:19 -0400 Received: from mail-am1on0067.outbound.protection.outlook.com ([157.56.112.67]:36689 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751087AbbIQN6R (ORCPT ); Thu, 17 Sep 2015 09:58:17 -0400 Received: from DB3PR05CA0061.eurprd05.prod.outlook.com (10.163.44.29) by AMSPR05MB359.eurprd05.prod.outlook.com (10.242.224.15) with Microsoft SMTP Server (TLS) id 15.1.268.17; Thu, 17 Sep 2015 13:58:13 +0000 Received: from AM1FFO11FD014.protection.gbl (2a01:111:f400:7e00::192) by DB3PR05CA0061.outlook.office365.com (2a01:111:e400:9448::29) with Microsoft SMTP Server (TLS) id 15.1.274.16 via Frontend Transport; Thu, 17 Sep 2015 13:58:12 +0000 Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; Avagotech.Com; dkim=none (message not signed) header.d=none; Avagotech.Com; dmarc=pass action=none header.from=mellanox.com; Received-SPF: Pass (protection.outlook.com: domain of mellanox.com designates 193.47.165.134 as permitted sender) receiver=protection.outlook.com; client-ip=193.47.165.134; helo=mtlcas13.mtl.com; Received: from mtlcas13.mtl.com (193.47.165.134) by AM1FFO11FD014.mail.protection.outlook.com (10.174.64.92) with Microsoft SMTP Server (TLS) id 15.1.274.4 via Frontend Transport; Thu, 17 Sep 2015 13:58:11 +0000 Received: from MTLCAS13.mtl.com (10.0.8.78) by mtlcas13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.775.38; Thu, 17 Sep 2015 16:58:25 +0300 Received: from MTLCAS01.mtl.com (10.0.8.71) by MTLCAS13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.775.38 via Frontend Transport; Thu, 17 Sep 2015 16:58:25 +0300 Received: from [10.223.0.82] (10.223.0.82) by MTLCAS01.mtl.com (10.0.8.71) with Microsoft SMTP Server (TLS) id 14.3.123.3; Thu, 17 Sep 2015 16:57:38 +0300 Subject: Re: [PATCH for-next 7/9] IB/cma: Add configfs for rdma_cm To: Doug Ledford References: <1439481794-27998-1-git-send-email-matanb@mellanox.com> <1439481794-27998-8-git-send-email-matanb@mellanox.com> CC: , Jason Gunthorpe , Or Gerlitz , "Haggai Eran" , Somnath Kotur From: Matan Barak Message-ID: <55FAC6D1.5080206@mellanox.com> Date: Thu, 17 Sep 2015 16:57:37 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <1439481794-27998-8-git-send-email-matanb@mellanox.com> X-Originating-IP: [10.223.0.82] X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; AM1FFO11FD014; 1:4Zx15ZfL2sY/iNsz0aQmZbnNxBESTjzAsusqG8EkgBOjS6ljKrTQbZwiIAB1IYj5aU69vAfnEhqYVI6aVlHvrr4RCCaJ2iqyosAmjbi051mGZwnHMZ3nLbDXrrhperAMB2wUMC6CLMst4wH5OXveYYLyQRJ+fAgKtJV68vUWdIYZI+fEudYNEUkN2Lv9kdfXAEXlfnmqBDSHo2FXKGyVVPDUqJ/KfDU/OrhdhWNEfViGXosz1axR60eMO+WLwpEjy1XqeiKqa5f1NZf8sYPp4N3aS8Wq2suFZQngkG9bq/eJIsonFYLGe00dZWL7eb5I96nRZ8fHbnpIp0h61MmYaHb0oP9PLRUc1j1GHmYFwrM/o/AFZIzJd/1nQ2NuYXDHUkZyT+4nSN2mAit8vCVxxQ== X-Forefront-Antispam-Report: CIP:193.47.165.134; CTRY:IL; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(377454003)(24454002)(189002)(479174004)(199003)(4001350100001)(87266999)(97736004)(4001540100001)(86362001)(5001830100001)(64126003)(77156002)(2950100001)(50466002)(92566002)(5890100001)(62966003)(50986999)(65956001)(65806001)(64706001)(76176999)(54356999)(68736005)(65816999)(47776003)(59896002)(77096005)(19580395003)(189998001)(80316001)(83506001)(6806004)(33656002)(106466001)(46102003)(110136002)(11100500001)(19580405001)(23746002)(5001860100001)(36756003)(5004730100002)(5007970100001)(87936001)(3940600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AMSPR05MB359; H:mtlcas13.mtl.com; FPR:; SPF:Pass; PTR:ErrorRetry; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; AMSPR05MB359; 2:9mWoKezUoLPn4m7pB55NUyhlZvrE9PYt5ooWFf+EifW1/l0J2Xq4JlW3GPmZRTsR7i9a4syY+SKmXNEjTCGGbelxg/zM1Xo8MKHt0wK1kMEbcn0F91O+7uZbAG46wmrPv48CtFd33uOxypCit/k4uiVGR9BmDqt07l5qBBXq1Dc=; 3:wzrvjiKTBv30Ba+e4LZS8KWYnMTQktqqnESoH0/d8veEeVVjXAQZM5L9/Fo7NsWKeAjTmqS8ST/bhw+547uthBdPEU/SbhxopMI3aQTsSlbZ10joZxbE9fWZUu7gofQOswqAZ2nfebpzrWXilCDv4y1k609tCLwM8canDqruwGiOMCKY7b1YWCOlNxnNsNPzmhGH6eCxGeIo+T+FnVqvPcy54flM1egs26+dOYhpqH8Ume7A+ymc0AowRwJQzJk+RRJ8GIqdeK0R+AIlhuvRMw==; 25:X1CqDRDfC1tVn9ZehebGSqLljza2BaXby2OVexR5/mv/yBz2L8iBAO6YS5mMusLBr9hb18MuRwvPCQRfeB+ySHo0XePZktDLf28qvv4ZMKDGpDzbjdhHXHibUQdH9eSeg2OhEY8ANppmuAxTkoSCdCJ9Fi8eqZ5jTPxwRSOz1z1QnPwXrGmOphldPSj6+iWmG2EfTK0OvhhXg3CCycEaflp3nsHCZ80q5RCcCHk0331XrdWlGnC/tMci4wfJ5jaPyiebZkf9A1Fsd3bib2tzcA== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501001); SRVR:AMSPR05MB359; X-Microsoft-Exchange-Diagnostics: 1; AMSPR05MB359; 20:AcmVcdSsru6PP+Dw0VaaIeBZVi2jQ5BpmHrJnl7TlfoMoyq2Y+04JkoO3qHsI1Dp5N7scaQrhTNFmy5E1ImmFhLSEpTUSJAXqSHjmBIIq5OD16QpKPoaJ7y1fcLbHdgq6EdRvCmJ5jyKFIjWK00jFw6vrN+me2Ib9+LjGnuV9wOqoyxUhE/3/QFQcp43WhLa8vAMwt2ZLpeSxFzWCdJfrEXxart4GqzzQSGVh3AFhagRWn322wBSYDr9GC5pQA+/76NTZHIZIStCoCObRdtlmoykFfAErMd8khK/B5WKi8cw67mjh8RuQWndOWcza480FRX3Jm8zkRv7Vg8AGQd06e1S9vOenPS/5TbGOjl+rKi6/UN22xTTH4s87J0r03KL+c1RpMlJfQF4HmnBRepxbgrDrvg7DGGs1SUHOiCVAiNmQ9Vin0im+3BaJGLZLDJqB2hVyg+cHnnccqFpEfclMTfFuM9SRQZ5eLuzbzMu5g47Uspo7wCWUJkioQw9oZQj; 4:BQJLyrb3tzwt8rZ8HDoqXCFCQHrtDjignT9IahhdchvIUblc8TA/CcCb6G9EpN94ZdZ5nkVaonbxDVFfPnNfmVyplVQOhre4hN91pdQ9bPjSrIFfLXN4TwnLo+CAkBukNGnwJazkgvNC27NPmpR82I7R1D3gjwuIoSnzflc3i5WwtMqyGlH/tXvTfDOPJoZRPJ5QIgD61iGtfZdfPLEN9c6zAeA77ttF9w4wKvLae7ol2GDQb3hfR5tTwKAPI/S1epizWRtiKYpiB2QZ5wuUM57wiWqEsWai2LKzmC0YyPthwxdJ0bbRIXLL7do1w1d3QLgpbQxBwJr3Giey8oRd8AQNsmjKBPxS9cwfqgNsgL8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(520075)(520078)(8121501046)(5005006)(3002001); SRVR:AMSPR05MB359; BCL:0; PCL:0; RULEID:; SRVR:AMSPR05MB359; X-Forefront-PRVS: 07025866F6 X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; AMSPR05MB359; 23:kBPzncJT/D6PTAOP6rcbjZ2YbG3V/jww2X0Afl?= =?Windows-1252?Q?EyRNeqvwjND7JBnxijSnv5a6inLRD8/93cKGXZfSwQ16J+bk3l1El7a7?= =?Windows-1252?Q?mI+t7Jg85uJX3FHscieNAf1MykEWczHrkPPyEGQx6cpoQn5kZPwk+QVr?= =?Windows-1252?Q?QG6I5vmv0XkaGaHd9Oy6dIhvtgZC8b/xwutN7upApGYV7Dg2N9GiLZgU?= =?Windows-1252?Q?d44RZ/cX3ku5qP8/qo1g+S5QNdw1vGL4m9GuoTFCXnvAQ7jxx/I7K5Jn?= =?Windows-1252?Q?TniUodHKTdOLGvWvEiaLfy6RBsyP2fVRFD3zmdy2k5i7jos68mTC6x3L?= =?Windows-1252?Q?w8XhFOqB+jntcwNML+EaqKIVHHiphCpqpwnTQsnGh87RwpEcqXxF8KAi?= =?Windows-1252?Q?pSuFkazG01kBRQi23g1XdOZuhnkaMliJfByem5AJLYGIGJPeWLX/bD/7?= =?Windows-1252?Q?QyRlpQv01TZ1ao8s5Ol1uwrQ62WWafAF+fK32NTC4xwW6pO68ttyRPt2?= =?Windows-1252?Q?t1nhE/6sXDuZdhY6RrYxlOYQ5wfWEGLLmB/4bHT6Xxi9aCUovMXuCQOw?= =?Windows-1252?Q?MpxSynyFVY2xzHLmKx2uCWEr4erGJtZHf+NGQA9ejmD8YJ8TQ1DHpi/P?= =?Windows-1252?Q?4NlTZ55nCCUB50ganS+GecUXtGfFPMETmSYC3c+kzGyfECQfCZbw4apF?= =?Windows-1252?Q?KdzNQAhxZcwtpSqMfKY8KBsbUZBoTw42JyPN/lmywuza2iu6WyMwUXIO?= =?Windows-1252?Q?OhRuJkbRDycuCMfyY/yKDItYkpb+8Oh9FkWVUe7gq214TCSiN3MI8E4x?= =?Windows-1252?Q?Zy9dnB0ZRssy16a2EobQTPpfdy86/KzZqsY3o4U0r/z8YRHlmEv58HDc?= =?Windows-1252?Q?cDTcEfRhpjjeeI1AMw2Laf1+J58q0p+TbjMbIjKoTC2N7OJc1J+CcrjO?= =?Windows-1252?Q?qDywDY+xDqWUIpJxLYBG73Ts64VauMyR6s0hDhqP2LGaGdXQVEocKOkX?= =?Windows-1252?Q?85pA075jgETRRjJ3jDsl9f4Dh5DkUSPmNeaugcA+2S/Eo4umxIu/ySlH?= =?Windows-1252?Q?w2dgiSw9Tu6B2sm0u+y8NOAvsVSBZOvcOPyQ6YRsoDzDhH6sQZ1lwjmc?= =?Windows-1252?Q?pig0Pv1erSEWv2YFtT7oohfBIiwnt4T13qeuw37QGUWHQ0CDJE6+Phlh?= =?Windows-1252?Q?m+fq0tUZVCpbCrohGT0UK9LZg16s13Hpmx4Eiyc1kchMpILjnc+uZwSQ?= =?Windows-1252?Q?XIJNC9PFrrVW2tSWDE8xCcgUsxhk6GsPM+8TC/jwHstByrr7R93IWoqz?= =?Windows-1252?Q?tgAkQKqzI6DgXkrMRaO9F9F+48gsPDzH37F7qZpLEvjRpjfurSdD/q5w?= =?Windows-1252?Q?Lm1tK3BNRyd+/DCiGBsfCvJyfOLOgOYn4SwnxE76aiTWjDzcMLy939h+?= =?Windows-1252?Q?EmKGp1z4q6CwNxWEiK?= X-Microsoft-Exchange-Diagnostics: 1; AMSPR05MB359; 5:Ob4Wvx828fLr9NOgf31RWoTtV5QLsltpvryuy0HcDYvglx1qeKxUsr6PEj6a5htX2oFdiDsmxc1Scjb3Y+MS73+j0VQwsoojhs24AGHspNPEc4W48eunjZOpdJzMLYwAZXIzHjUzihV4SFBn4d4yVQ==; 24:wPH8kC87M2f0bLeXoPS5bv38rldnuIxqacv2ABpNW+Ge8QBHEK6wkboGMPas0fp0IrKYK5vsmxkoePaQHJtgbFxGTQlpVSD+X0znd5xwhM4=; 20:SIAz5iv2PdXxEZ20CZfmR+pwBhYRPwGl5MW1sgCyZtZBMtj1LfecSEsJRpiXrdbdf7gnOFNjE6yn371l/+wdEQ== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Sep 2015 13:58:11.1650 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b; Ip=[193.47.165.134]; Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AMSPR05MB359 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 8/13/2015 7:03 PM, Matan Barak wrote: > Users would like to control the behaviour of rdma_cm. > For example, old applications which don't set the > required RoCE gid type could be executed on RoCE V2 > network types. In order to support this configuration, > we implement a configfs for rdma_cm. > > In order to use the configfs, one needs to mount it and > mkdir inside rdma_cm directory. > > The patch adds support for a single configuration file, > default_roce_mode. The mode can either be "IB/RoCE v1" or > "RoCE v2". > > Signed-off-by: Matan Barak > --- > drivers/infiniband/Kconfig | 9 + > drivers/infiniband/core/Makefile | 2 + > drivers/infiniband/core/cache.c | 24 +++ > drivers/infiniband/core/cma.c | 95 ++++++++- > drivers/infiniband/core/cma_configfs.c | 353 +++++++++++++++++++++++++++++++++ > drivers/infiniband/core/core_priv.h | 24 +++ > 6 files changed, 503 insertions(+), 4 deletions(-) > create mode 100644 drivers/infiniband/core/cma_configfs.c > > diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig > index da4c697..9ee82a2 100644 > --- a/drivers/infiniband/Kconfig > +++ b/drivers/infiniband/Kconfig > @@ -54,6 +54,15 @@ config INFINIBAND_ADDR_TRANS > depends on INFINIBAND > default y > > +config INFINIBAND_ADDR_TRANS_CONFIGFS > + bool > + depends on INFINIBAND_ADDR_TRANS && CONFIGFS_FS > + default y > + ---help--- > + ConfigFS support for RDMA communication manager (CM). > + This allows the user to config the default GID type that the CM > + uses for each device, when initiaing new connections. > + > source "drivers/infiniband/hw/mthca/Kconfig" > source "drivers/infiniband/hw/qib/Kconfig" > source "drivers/infiniband/hw/ehca/Kconfig" > diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile > index d43a899..7922fa7 100644 > --- a/drivers/infiniband/core/Makefile > +++ b/drivers/infiniband/core/Makefile > @@ -24,6 +24,8 @@ iw_cm-y := iwcm.o iwpm_util.o iwpm_msg.o > > rdma_cm-y := cma.o > > +rdma_cm-$(CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS) += cma_configfs.o > + > rdma_ucm-y := ucma.o > > ib_addr-y := addr.o > diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c > index ddd0406..66090ce 100644 > --- a/drivers/infiniband/core/cache.c > +++ b/drivers/infiniband/core/cache.c > @@ -127,6 +127,30 @@ const char *ib_cache_gid_type_str(enum ib_gid_type gid_type) > } > EXPORT_SYMBOL(ib_cache_gid_type_str); > > +int ib_cache_gid_parse_type_str(const char *buf) > +{ > + unsigned int i; > + size_t len; > + int err = -EINVAL; > + > + len = strlen(buf); > + if (len == 0) > + return -EINVAL; > + > + if (buf[len - 1] == '\n') > + len--; > + > + for (i = 0; i < ARRAY_SIZE(gid_type_str); ++i) > + if (gid_type_str[i] && !strncmp(buf, gid_type_str[i], len) && > + len == strlen(gid_type_str[i])) { > + err = i; > + break; > + } > + > + return err; > +} > +EXPORT_SYMBOL(ib_cache_gid_parse_type_str); > + > static int write_gid(struct ib_device *ib_dev, u8 port, > struct ib_gid_table *table, int ix, > const union ib_gid *gid, > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 22003dd..e4f4d23 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -121,6 +121,7 @@ struct cma_device { > struct completion comp; > atomic_t refcount; > struct list_head id_list; > + enum ib_gid_type *default_gid_type; > }; > > struct rdma_bind_list { > @@ -138,6 +139,62 @@ void cma_ref_dev(struct cma_device *cma_dev) > atomic_inc(&cma_dev->refcount); > } > > +struct cma_device *cma_enum_devices_by_ibdev(cma_device_filter filter, > + void *cookie) > +{ > + struct cma_device *cma_dev; > + struct cma_device *found_cma_dev = NULL; > + > + mutex_lock(&lock); > + > + list_for_each_entry(cma_dev, &dev_list, list) > + if (filter(cma_dev->device, cookie)) { > + found_cma_dev = cma_dev; > + break; > + } > + > + if (found_cma_dev) > + cma_ref_dev(found_cma_dev); > + mutex_unlock(&lock); > + return found_cma_dev; > +} > + > +int cma_get_default_gid_type(struct cma_device *cma_dev, > + unsigned int port) > +{ > + if (port < rdma_start_port(cma_dev->device) || > + port > rdma_end_port(cma_dev->device)) > + return -EINVAL; > + > + return cma_dev->default_gid_type[port - rdma_start_port(cma_dev->device)]; > +} > + > +int cma_set_default_gid_type(struct cma_device *cma_dev, > + unsigned int port, > + enum ib_gid_type default_gid_type) > +{ > + unsigned long supported_gids; > + > + if (port < rdma_start_port(cma_dev->device) || > + port > rdma_end_port(cma_dev->device)) > + return -EINVAL; > + > + supported_gids = roce_gid_type_mask_support(cma_dev->device, port); > + > + if (!(supported_gids & 1 << default_gid_type)) > + return -EINVAL; > + > + cma_dev->default_gid_type[port - rdma_start_port(cma_dev->device)] = > + default_gid_type; > + > + return 0; > +} > + > +struct ib_device *cma_get_ib_dev(struct cma_device *cma_dev) > +{ > + return cma_dev->device; > +} > + > /* > * Device removal can occur at anytime, so we need extra handling to > * serialize notifying the user of device removal with other callbacks. > @@ -285,6 +342,9 @@ static void cma_attach_to_dev(struct rdma_id_private *id_priv, > { > cma_ref_dev(cma_dev); > id_priv->cma_dev = cma_dev; > + id_priv->gid_type = > + cma_dev->default_gid_type[id_priv->id.port_num - > + rdma_start_port(cma_dev->device)]; > id_priv->id.device = cma_dev->device; > id_priv->id.route.addr.dev_addr.transport = > rdma_node_get_transport(cma_dev->device->node_type); > @@ -386,6 +446,7 @@ static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a > } > > static inline int cma_validate_port(struct ib_device *device, u8 port, > + enum ib_gid_type gid_type, > union ib_gid *gid, int dev_type, > int bound_if_index) > { > @@ -400,8 +461,10 @@ static inline int cma_validate_port(struct ib_device *device, u8 port, > > if (dev_type == ARPHRD_ETHER) > ndev = dev_get_by_index(&init_net, bound_if_index); > + else > + gid_type = IB_GID_TYPE_IB; > > - ret = ib_find_cached_gid_by_port(device, gid, IB_GID_TYPE_IB, port, > + ret = ib_find_cached_gid_by_port(device, gid, gid_type, port, > ndev, NULL); > > if (ndev) > @@ -436,7 +499,10 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, > gidp = rdma_protocol_roce(cma_dev->device, port) ? > &iboe_gid : &gid; > > - ret = cma_validate_port(cma_dev->device, port, gidp, > + ret = cma_validate_port(cma_dev->device, port, > + rdma_protocol_ib(cma_dev->device, port) ? > + IB_GID_TYPE_IB : > + listen_id_priv->gid_type, gidp, > dev_addr->dev_type, > dev_addr->bound_dev_if); > if (!ret) { > @@ -455,8 +521,11 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, > gidp = rdma_protocol_roce(cma_dev->device, port) ? > &iboe_gid : &gid; > > - ret = cma_validate_port(cma_dev->device, port, gidp, > - dev_addr->dev_type, > + ret = cma_validate_port(cma_dev->device, port, > + rdma_protocol_ib(cma_dev->device, port) ? > + IB_GID_TYPE_IB : > + cma_dev->default_gid_type[port - 1], > + gidp, dev_addr->dev_type, > dev_addr->bound_dev_if); > if (!ret) { > id_priv->id.port_num = port; > @@ -3501,12 +3570,27 @@ static void cma_add_one(struct ib_device *device) > { > struct cma_device *cma_dev; > struct rdma_id_private *id_priv; > + unsigned int i; > + unsigned long supported_gids = 0; > > cma_dev = kmalloc(sizeof *cma_dev, GFP_KERNEL); > if (!cma_dev) > return; > > cma_dev->device = device; > + cma_dev->default_gid_type = kcalloc(device->phys_port_cnt, > + sizeof(*cma_dev->default_gid_type), > + GFP_KERNEL); > + if (!cma_dev->default_gid_type) { > + kfree(cma_dev); > + return; > + } > + for (i = rdma_start_port(device); i <= rdma_end_port(device); i++) { > + supported_gids = roce_gid_type_mask_support(device, i); > + WARN_ON(!supported_gids); > + cma_dev->default_gid_type[i - rdma_start_port(device)] = > + find_first_bit(&supported_gids, BITS_PER_LONG); > + } > > init_completion(&cma_dev->comp); > atomic_set(&cma_dev->refcount, 1); > @@ -3587,6 +3671,7 @@ static void cma_remove_one(struct ib_device *device) > mutex_unlock(&lock); > > cma_process_remove(cma_dev); > + kfree(cma_dev->default_gid_type); > kfree(cma_dev); > } > > @@ -3687,6 +3772,7 @@ static int __init cma_init(void) > > if (ibnl_add_client(RDMA_NL_RDMA_CM, RDMA_NL_RDMA_CM_NUM_OPS, cma_cb_table)) > printk(KERN_WARNING "RDMA CMA: failed to add netlink callback\n"); > + cma_configfs_init(); > > return 0; > > @@ -3700,6 +3786,7 @@ err: > > static void __exit cma_cleanup(void) > { > + cma_configfs_exit(); > ibnl_remove_client(RDMA_NL_RDMA_CM); > ib_unregister_client(&cma_client); > unregister_netdevice_notifier(&cma_nb); > diff --git a/drivers/infiniband/core/cma_configfs.c b/drivers/infiniband/core/cma_configfs.c > new file mode 100644 > index 0000000..f59c454 > --- /dev/null > +++ b/drivers/infiniband/core/cma_configfs.c > @@ -0,0 +1,353 @@ > +/* > + * Copyright (c) 2015, Mellanox Technologies inc. All rights reserved. > + * > + * This software is available to you under a choice of one of two > + * licenses. You may choose to be licensed under the terms of the GNU > + * General Public License (GPL) Version 2, available from the file > + * COPYING in the main directory of this source tree, or the > + * OpenIB.org BSD license below: > + * > + * Redistribution and use in source and binary forms, with or > + * without modification, are permitted provided that the following > + * conditions are met: > + * > + * - Redistributions of source code must retain the above > + * copyright notice, this list of conditions and the following > + * disclaimer. > + * > + * - Redistributions in binary form must reproduce the above > + * copyright notice, this list of conditions and the following > + * disclaimer in the documentation and/or other materials > + * provided with the distribution. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND > + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS > + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN > + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE > + * SOFTWARE. > + */ > + > +#include > +#include > +#include "core_priv.h" > + > +struct cma_device; > + > +struct cma_dev_group; > + > +struct cma_dev_port_group { > + unsigned int port_num; > + struct cma_dev_group *cma_dev_group; > + struct config_group group; > +}; > + > +struct cma_dev_group { > + char name[IB_DEVICE_NAME_MAX]; > + struct config_group device_group; > + struct config_group ports_group; > + struct config_group *default_dev_group[2]; > + struct config_group **default_ports_group; > + struct cma_dev_port_group *ports; > +}; > + > +struct cma_configfs_attr { > + struct configfs_attribute attr; > + ssize_t (*show)(struct cma_device *cma_dev, > + struct cma_dev_port_group *group, > + char *buf); > + ssize_t (*store)(struct cma_device *cma_dev, > + struct cma_dev_port_group *group, > + const char *buf, size_t count); > +}; > + > +static struct cma_dev_port_group *to_dev_port_group(struct config_item *item) > +{ > + struct config_group *group; > + > + if (!item) > + return NULL; > + > + group = container_of(item, struct config_group, cg_item); > + return container_of(group, struct cma_dev_port_group, group); > +} > + > +static ssize_t show_default_roce_mode(struct cma_device *cma_dev, > + struct cma_dev_port_group *group, > + char *buf) > +{ > + unsigned int port = group->port_num; > + int gid_type = cma_get_default_gid_type(cma_dev, port); > + > + if (gid_type < 0) > + return gid_type; > + > + return sprintf(buf, "%s\n", ib_cache_gid_type_str(gid_type)); > +} > + > +static ssize_t store_default_roce_mode(struct cma_device *cma_dev, > + struct cma_dev_port_group *group, > + const char *buf, size_t count) > +{ > + int gid_type = ib_cache_gid_parse_type_str(buf); > + unsigned int port = group->port_num; > + int ret; > + > + if (gid_type < 0) > + return -EINVAL; > + > + ret = cma_set_default_gid_type(cma_dev, port, gid_type); > + if (ret) > + return ret; > + > + return strnlen(buf, count); > +} > + > +#define CMA_PARAM_PORT_ATTR_RW(_name) \ > +static struct cma_configfs_attr cma_configfs_attr_##_name = \ > + __CONFIGFS_ATTR(_name, S_IRUGO | S_IWUSR, show_##_name, store_##_name) > + > +CMA_PARAM_PORT_ATTR_RW(default_roce_mode); > + > +static bool filter_by_name(struct ib_device *ib_dev, void *cookie) > +{ > + return !strcmp(ib_dev->name, cookie); > +} > + > +static ssize_t cma_configfs_attr_show(struct config_item *item, > + struct configfs_attribute *attr, > + char *buf) > +{ > + ssize_t ret = -EINVAL; > + struct cma_dev_port_group *group = to_dev_port_group(item); > + struct cma_device *cma_dev; > + struct cma_configfs_attr *ca = > + container_of(attr, struct cma_configfs_attr, attr); > + > + if (!group) > + return -ENODEV; > + > + cma_dev = cma_enum_devices_by_ibdev(filter_by_name, > + group->cma_dev_group->name); > + if (!cma_dev) > + return -ENODEV; > + > + if (ca->show) > + ret = ca->show(cma_dev, group, buf); > + > + cma_deref_dev(cma_dev); > + return ret; > +} > + > +static ssize_t cma_configfs_attr_store(struct config_item *item, > + struct configfs_attribute *attr, > + const char *buf, size_t count) > +{ > + ssize_t ret = -EINVAL; > + struct cma_dev_port_group *group = to_dev_port_group(item); > + struct cma_device *cma_dev; > + struct cma_configfs_attr *ca = > + container_of(attr, struct cma_configfs_attr, attr); > + > + if (!group) > + return -ENODEV; > + > + cma_dev = cma_enum_devices_by_ibdev(filter_by_name, > + group->cma_dev_group->name); > + if (!cma_dev) > + return -ENODEV; > + > + if (ca->store) > + ret = ca->store(cma_dev, group, buf, count); > + > + cma_deref_dev(cma_dev); > + return ret; > +} > + > +static struct configfs_attribute *cma_configfs_attributes[] = { > + &cma_configfs_attr_default_roce_mode.attr, > + NULL, > +}; > + > +static struct configfs_item_operations cma_item_ops = { > + .show_attribute = cma_configfs_attr_show, > + .store_attribute = cma_configfs_attr_store, > +}; > + > +static struct config_item_type cma_port_group_type = { > + .ct_attrs = cma_configfs_attributes, > + .ct_item_ops = &cma_item_ops, > + .ct_owner = THIS_MODULE > +}; > + > +static int make_cma_ports(struct cma_dev_group *cma_dev_group, > + struct cma_device *cma_dev) > +{ > + struct ib_device *ibdev; > + unsigned int i; > + unsigned int ports_num; > + struct cma_dev_port_group *ports; > + int err; > + > + ibdev = cma_get_ib_dev(cma_dev); > + > + if (!ibdev) > + return -ENODEV; > + > + ports_num = ibdev->phys_port_cnt; > + ports = kcalloc(ports_num, sizeof(*cma_dev_group->ports), > + GFP_KERNEL); > + > + cma_dev_group->default_ports_group = kcalloc(ports_num + 1, > + sizeof(*cma_dev_group->ports), > + GFP_KERNEL); > + > + if (!ports || !cma_dev_group->default_ports_group) { > + err = -ENOMEM; > + goto free; > + } > + > + for (i = 0; i < ports_num; i++) { > + char port_str[10]; > + > + ports[i].port_num = i + 1; > + snprintf(port_str, sizeof(port_str), "%u", i + 1); > + ports[i].cma_dev_group = cma_dev_group; > + config_group_init_type_name(&ports[i].group, > + port_str, > + &cma_port_group_type); > + cma_dev_group->default_ports_group[i] = &ports[i].group; > + } > + cma_dev_group->default_ports_group[i] = NULL; > + cma_dev_group->ports = ports; > + > + return 0; > +free: > + kfree(ports); > + kfree(cma_dev_group->default_ports_group); > + cma_dev_group->ports = NULL; > + cma_dev_group->default_ports_group = NULL; > + return err; > +} > + > +static void release_cma_dev(struct config_item *item) > +{ > + struct config_group *group = container_of(item, struct config_group, > + cg_item); > + struct cma_dev_group *cma_dev_group = container_of(group, > + struct cma_dev_group, > + ports_group); > + > + kfree(cma_dev_group); > +}; > + > +static void release_cma_ports_group(struct config_item *item) > +{ > + struct config_group *group = container_of(item, struct config_group, > + cg_item); > + struct cma_dev_group *cma_dev_group = container_of(group, > + struct cma_dev_group, > + device_group); > + > + kfree(cma_dev_group->ports); > + kfree(cma_dev_group->default_ports_group); > + cma_dev_group->ports = NULL; > + cma_dev_group->default_ports_group = NULL; > +}; > + > +static struct configfs_item_operations cma_ports_item_ops = { > + .release = release_cma_ports_group > +}; > + > +static struct config_item_type cma_ports_group_type = { > + .ct_item_ops = &cma_ports_item_ops, > + .ct_owner = THIS_MODULE > +}; > + > +static struct configfs_item_operations cma_device_item_ops = { > + .release = release_cma_dev > +}; > + > +static struct config_item_type cma_device_group_type = { > + .ct_item_ops = &cma_device_item_ops, > + .ct_owner = THIS_MODULE > +}; > + > +static struct config_group *make_cma_dev(struct config_group *group, > + const char *name) > +{ > + int err = -ENODEV; > + struct cma_device *cma_dev = cma_enum_devices_by_ibdev(filter_by_name, > + (void *)name); > + struct cma_dev_group *cma_dev_group = NULL; > + > + if (!cma_dev) > + goto fail; > + > + cma_dev_group = kzalloc(sizeof(*cma_dev_group), GFP_KERNEL); > + > + if (!cma_dev_group) { > + err = -ENOMEM; > + goto fail; > + } > + > + strncpy(cma_dev_group->name, name, sizeof(cma_dev_group->name)); > + > + err = make_cma_ports(cma_dev_group, cma_dev); > + if (err) > + goto fail; > + > + cma_dev_group->ports_group.default_groups = > + cma_dev_group->default_ports_group; > + config_group_init_type_name(&cma_dev_group->ports_group, "ports", > + &cma_ports_group_type); > + > + cma_dev_group->device_group.default_groups > + = cma_dev_group->default_dev_group; > + cma_dev_group->default_dev_group[0] = &cma_dev_group->ports_group; > + cma_dev_group->default_dev_group[1] = NULL; > + > + config_group_init_type_name(&cma_dev_group->device_group, name, > + &cma_device_group_type); > + > + cma_deref_dev(cma_dev); > + return &cma_dev_group->device_group; > + > +fail: > + if (cma_dev) > + cma_deref_dev(cma_dev); > + kfree(cma_dev_group); > + return ERR_PTR(err); > +} > + > +static struct configfs_group_operations cma_subsys_group_ops = { > + .make_group = make_cma_dev, > +}; > + > +static struct config_item_type cma_subsys_type = { > + .ct_group_ops = &cma_subsys_group_ops, > + .ct_owner = THIS_MODULE, > +}; > + > +static struct configfs_subsystem cma_subsys = { > + .su_group = { > + .cg_item = { > + .ci_namebuf = "rdma_cm", > + .ci_type = &cma_subsys_type, > + }, > + }, > +}; > + > +int __init cma_configfs_init(void) > +{ > + config_group_init(&cma_subsys.su_group); > + mutex_init(&cma_subsys.su_mutex); > + return configfs_register_subsystem(&cma_subsys); > +} > + > +void __exit cma_configfs_exit(void) > +{ > + configfs_unregister_subsystem(&cma_subsys); > +} > diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h > index 7fbfea9..1cad092 100644 > --- a/drivers/infiniband/core/core_priv.h > +++ b/drivers/infiniband/core/core_priv.h > @@ -38,9 +38,31 @@ > > #include > > +#if IS_ENABLED(CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS) > +int cma_configfs_init(void); > +void cma_configfs_exit(void); > +#else > +static inline int cma_configfs_init(void) > +{ > + return 0; > +} > + > +static inline void cma_configfs_exit(void) > +{ > +} > +#endif > struct cma_device; > void cma_ref_dev(struct cma_device *cma_dev); > void cma_deref_dev(struct cma_device *cma_dev); > +typedef bool (*cma_device_filter)(struct ib_device *, void *); > +struct cma_device *cma_enum_devices_by_ibdev(cma_device_filter filter, > + void *cookie); > +int cma_get_default_gid_type(struct cma_device *cma_dev, > + unsigned int port); > +int cma_set_default_gid_type(struct cma_device *cma_dev, > + unsigned int port, > + enum ib_gid_type default_gid_type); > +struct ib_device *cma_get_ib_dev(struct cma_device *cma_dev); > > int ib_device_register_sysfs(struct ib_device *device, > int (*port_callback)(struct ib_device *, > @@ -74,6 +96,8 @@ enum ib_cache_gid_default_mode { > IB_CACHE_GID_DEFAULT_MODE_DELETE > }; > > +int ib_cache_gid_parse_type_str(const char *buf); > + > const char *ib_cache_gid_type_str(enum ib_gid_type gid_type); > > void ib_cache_gid_set_default_gid(struct ib_device *ib_dev, u8 port, > We've found a small bug in this commit. Here's the fix: kfree(cma_dev_group->default_ports_group); Would you like that we re-spin the whole RoCE v2 series? Regards, Matan --- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/core/cma_configfs.c b/drivers/infiniband/core/cma_configfs.c index f59c454..9911a08 100644 --- a/drivers/infiniband/core/cma_configfs.c +++ b/drivers/infiniband/core/cma_configfs.c @@ -238,7 +238,7 @@ static void release_cma_dev(struct config_item *item) cg_item); struct cma_dev_group *cma_dev_group = container_of(group, struct cma_dev_group, - ports_group); + device_group); kfree(cma_dev_group); }; @@ -249,7 +249,7 @@ static void release_cma_ports_group(struct config_item *item) cg_item); struct cma_dev_group *cma_dev_group = container_of(group, struct cma_dev_group, - device_group); + ports_group); kfree(cma_dev_group->ports);