From patchwork Mon Dec 3 20:58:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10710593 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E7641731 for ; Mon, 3 Dec 2018 20:58:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B965C29EC2 for ; Mon, 3 Dec 2018 20:58:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA3F929EE6; Mon, 3 Dec 2018 20:58:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77B6D29EC2 for ; Mon, 3 Dec 2018 20:58:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726144AbeLCU6y (ORCPT ); Mon, 3 Dec 2018 15:58:54 -0500 Received: from mail-eopbgr50057.outbound.protection.outlook.com ([40.107.5.57]:62112 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725908AbeLCU6x (ORCPT ); Mon, 3 Dec 2018 15:58:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9ZFh324VIGUqccZyvEhxHlvBTnlpebOv/fnk9I+sDOk=; b=M+rNiWQFlrU1IXOAKpA9ZC8uSVoDRyXQ6GoVXXbNfJdG1pHhCrIfQodfR0y+W/aQOChi7gIiBbjwu9ov2A0+x7P0CsfOT9CAarADo9ZIfMjrRdd8o8X/FSguQUNvjbJudW8pp/oA2/v1w8vemwEtkZKPG+4ban89RdT3vp1YGaU= Received: from AM6PR05MB5553.eurprd05.prod.outlook.com (20.177.119.202) by AM6PR05MB4981.eurprd05.prod.outlook.com (20.177.36.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1382.22; Mon, 3 Dec 2018 20:58:32 +0000 Received: from AM6PR05MB5553.eurprd05.prod.outlook.com ([fe80::6136:d1d6:67c2:3920]) by AM6PR05MB5553.eurprd05.prod.outlook.com ([fe80::6136:d1d6:67c2:3920%2]) with mapi id 15.20.1382.020; Mon, 3 Dec 2018 20:58:32 +0000 From: Jason Gunthorpe To: "linux-rdma@vger.kernel.org" Subject: [PATCH rdma-core] verbs: Allow all commands to be invoked by ioctl Thread-Topic: [PATCH rdma-core] verbs: Allow all commands to be invoked by ioctl Thread-Index: AQHUi0rzjSALsAu4Fk63h2YDvWBxOA== Date: Mon, 3 Dec 2018 20:58:32 +0000 Message-ID: <20181203205827.GA25410@ziepe.ca> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: MWHPR13CA0023.namprd13.prod.outlook.com (2603:10b6:300:16::33) To AM6PR05MB5553.eurprd05.prod.outlook.com (2603:10a6:20b:30::10) authentication-results: spf=none (sender IP is ) smtp.mailfrom=jgg@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [174.3.196.123] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;AM6PR05MB4981;6:j5HXuJEz8uFNb1wzX7ZLCmAt8Kbwat2w/MeST+0FTLHt3YHmgqOVSUFXEcru64Q5gQqSFKgi8OfpiDf77GNGgVu6vx5ouh88yBC7wbA9uCjgutPy5jOtlZycuLwsj9ZOODW7qukVXgcEme7M98WGank9o/8C6huoge5I6VL4SVnCF67XVA7J+j9JPaD8sSi6EyQz3a1+XxhfHVKtjpxebJ3WPeiwpsSF2CBVmEoMLzTzhIKr31ntlB8kHsiw8b7gcZYMQ5r8vxSg/EyrMjfY4Dkj+gsX4yFe/dPvvCbmH4IbFilXc9fHfMGehGXm7rvyGdTKmfr2QZPqShZrq1pxlmf2jfYcPhS2QhgpbhrYd8TTdiF3FTby7QP+kA9xlOHevfZiL0wbp1w4QGZ2kUj4KgygrbUZWScxGISn86UPC5SJbCl35iHo9PxqyJ70Gi2QPwZrYy1SLO7ekebZJ20ngA==;5:Zh0zwI445vMw5GzB4A0PgPt5GtuaFzXpbo7+EcVIzmFaFsHuI69qv9EEYrkZWA+KdDk/F4hSIJ9d0juAQ+GKDsHUagOmML1FNPu/YkSroWBRH1oGkQMOXLi7GzKurmYOebGwWEPw/W/Y7XvX13jXezpcB8CqU76xb/nVkBNvjw8=;7:oDpxDlOFJtdJHcblxtcyXLWMIsBC/iVBZISxTPAPUt92OkzCu0ydV8z74aAHfeRHwyOS1LgP/DlcGwzXrzlEI20nM8nc8J7mq4+SLgwwhMLUMk6cBgqXlGb367lCxRc/W3t3YqM+cFP7Plguzbzevw== x-ms-office365-filtering-correlation-id: 638d13a4-a9bd-433e-a701-08d65962157c x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390098)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:AM6PR05MB4981; x-ms-traffictypediagnostic: AM6PR05MB4981: x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231455)(999002)(944501493)(52105112)(93006095)(93001095)(6055026)(148016)(149066)(150057)(6041310)(20161123558120)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(201708071742011)(7699051)(76991095);SRVR:AM6PR05MB4981;BCL:0;PCL:0;RULEID:;SRVR:AM6PR05MB4981; x-forefront-prvs: 08756AC3C8 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(346002)(39860400002)(396003)(366004)(376002)(136003)(189003)(199004)(6436002)(6512007)(106356001)(486006)(2501003)(66066001)(102836004)(33896004)(478600001)(6506007)(386003)(6486002)(9686003)(5640700003)(966005)(86362001)(14454004)(316002)(71190400001)(52116002)(71200400001)(5660300001)(99286004)(105586002)(6306002)(8676002)(6116002)(81156014)(2351001)(14444005)(1076002)(256004)(36756003)(53936002)(3846002)(68736007)(81166006)(25786009)(97736004)(7736002)(186003)(33656002)(305945005)(26005)(2906002)(476003)(6916009)(8936002)(42262002);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR05MB4981;H:AM6PR05MB5553.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 16twjfflTQIJ3bM7075cX2S2L6WAGGAN1ujf/og2MuGJuvWUqYcT9//XKbEAC9prVP/2jO6ge8Cmi1QUjQBNREw3jaJkivjw8XfwI5F6yyvzCq0YgGKEINtyQmRSjEmY8Jq5YD90j69x6c5NCczBWTLX6ZW12vK0xVPk2yHBA9qsYx5yGDG+DFjKpIdMveHnAQdI23tvWd3VqejZPGMU5hgIKppJ3Q6C0Vis1fZiUX9UyX0EYwLSjJC1udwp3+b0OXDFFeRrMN/HV9oNHeqtsVTO2A65fR1LqmVoOk3ZrW43KutB/HHaqv2A0hCbLyLauBVO9CHJB7YLsHB1seofCWJb1afe9oTH7y3db/CoQOs= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 638d13a4-a9bd-433e-a701-08d65962157c X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Dec 2018 20:58:32.5004 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4981 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Using the new UVERBS_METHOD_INVOKE_WRITE interface all of the existing write() commands can be immediately converted to execute over ioctl. If -DIOCTL_MODE=ioctl is set then only the IOCTL interface is supported, and write will not be called. Otherwise the 'both' mode will auto detect the kernel support and use it. Signed-off-by: Jason Gunthorpe --- libibverbs/cmd_fallback.c | 44 ++++++++++++++++++++++++++++++++++++++- libibverbs/device.c | 27 ++++++++++++++++++++++++ libibverbs/ibverbs.h | 1 + 3 files changed, 71 insertions(+), 1 deletion(-) This needs the matching kernel patch: https://patchwork.kernel.org/patch/10706127/ a kernel headers update and the series at: https://github.com/linux-rdma/rdma-core/pull/434 To apply. diff --git a/libibverbs/cmd_fallback.c b/libibverbs/cmd_fallback.c index 0e50503e003484..38193a186a8ccb 100644 --- a/libibverbs/cmd_fallback.c +++ b/libibverbs/cmd_fallback.c @@ -206,11 +206,43 @@ void *_write_get_resp_ex(struct ibv_command_buffer *link, return resp_start; } +static int ioctl_write(struct ibv_context *ctx, unsigned int write_method, + const void *req, size_t core_req_size, size_t req_size, + void *resp, size_t core_resp_size, size_t resp_size) +{ + DECLARE_COMMAND_BUFFER(cmdb, UVERBS_OBJECT_DEVICE, + UVERBS_METHOD_INVOKE_WRITE, 5); + + fill_attr_const_in(cmdb, UVERBS_ATTR_WRITE_CMD, write_method); + + if (core_req_size) + fill_attr_in(cmdb, UVERBS_ATTR_CORE_IN, req, core_req_size); + if (core_resp_size) + fill_attr_out(cmdb, UVERBS_ATTR_CORE_OUT, resp, core_resp_size); + + if (req_size - core_req_size) + fill_attr_in(cmdb, UVERBS_ATTR_UHW_IN, req + core_req_size, + req_size - core_req_size); + if (resp_size - core_resp_size) + fill_attr_out(cmdb, UVERBS_ATTR_UHW_OUT, resp + core_resp_size, + resp_size - core_resp_size); + + return execute_ioctl(ctx, cmdb); +} + int _execute_cmd_write(struct ibv_context *ctx, unsigned int write_method, struct ib_uverbs_cmd_hdr *req, size_t core_req_size, size_t req_size, void *resp, size_t core_resp_size, size_t resp_size) { + struct verbs_ex_private *priv = get_priv(ctx); + + if (!VERBS_WRITE_ONLY && (VERBS_IOCTL_ONLY || priv->use_ioctl_write)) + return ioctl_write(ctx, write_method, req + 1, + core_req_size - sizeof(*req), + req_size - sizeof(*req), resp, + core_resp_size, resp_size); + req->command = write_method; req->in_words = __check_divide(req_size, 4); req->out_words = __check_divide(resp_size, 4); @@ -232,6 +264,15 @@ int _execute_cmd_write_ex(struct ibv_context *ctx, unsigned int write_method, size_t req_size, void *resp, size_t core_resp_size, size_t resp_size) { + struct verbs_ex_private *priv = get_priv(ctx); + + if (!VERBS_WRITE_ONLY && (VERBS_IOCTL_ONLY || priv->use_ioctl_write)) + return ioctl_write( + ctx, IB_USER_VERBS_CMD_FLAG_EXTENDED | write_method, + req + 1, core_req_size - sizeof(*req), + req_size - sizeof(*req), resp, core_resp_size, + resp_size); + req->hdr.command = IB_USER_VERBS_CMD_FLAG_EXTENDED | write_method; req->hdr.in_words = __check_divide(core_req_size - sizeof(struct ex_hdr), 8); @@ -245,7 +286,8 @@ int _execute_cmd_write_ex(struct ibv_context *ctx, unsigned int write_method, /* * Users assumes the stack buffer is zeroed before passing to the - * kernel for writing. + * kernel for writing. New kernels with the ioctl path do this + * automatically for us. */ if (resp) memset(resp, 0, resp_size); diff --git a/libibverbs/device.c b/libibverbs/device.c index e6e23e718d9fca..1fed29b17ec31d 100644 --- a/libibverbs/device.c +++ b/libibverbs/device.c @@ -43,6 +43,7 @@ #include #include +#include #include #include "ibverbs.h" @@ -193,6 +194,31 @@ __lib_ibv_create_cq_ex(struct ibv_context *context, return cq; } +static bool has_ioctl_write(struct ibv_context *ctx) +{ + int rc; + DECLARE_COMMAND_BUFFER(cmdb, UVERBS_OBJECT_DEVICE, + UVERBS_METHOD_INVOKE_WRITE, 1); + + if (VERBS_IOCTL_ONLY) + return true; + if (VERBS_WRITE_ONLY) + return false; + + /* + * This command should return ENOSPC since the request length is too + * small. + */ + fill_attr_const_in(cmdb, UVERBS_ATTR_WRITE_CMD, + IB_USER_VERBS_CMD_QUERY_DEVICE); + rc = execute_ioctl(ctx, cmdb); + if (rc == EPROTONOSUPPORT) + return false; + if (rc == ENOTTY) + return false; + return true; +} + /* * Ownership of cmd_fd is transferred into this function, and it will either * be released during the matching call to verbs_uninit_contxt or during the @@ -240,6 +266,7 @@ int verbs_init_context(struct verbs_context *context_ex, context_ex->priv->driver_id = driver_id; verbs_set_ops(context_ex, &verbs_dummy_ops); + context_ex->priv->use_ioctl_write = has_ioctl_write(context); return 0; } diff --git a/libibverbs/ibverbs.h b/libibverbs/ibverbs.h index 24843b49fdb3c7..36e29a6b14d87b 100644 --- a/libibverbs/ibverbs.h +++ b/libibverbs/ibverbs.h @@ -70,6 +70,7 @@ void load_drivers(void); struct verbs_ex_private { BITMAP_DECLARE(unsupported_ioctls, VERBS_OPS_NUM); uint32_t driver_id; + bool use_ioctl_write; struct verbs_context_ops ops; };