From patchwork Wed Aug 21 18:10:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771955 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2064.outbound.protection.outlook.com [40.107.102.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C11F1B3B21; Wed, 21 Aug 2024 18:11:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263881; cv=fail; b=nVnyTPjDEjgiw1mFlmApeBcCALoKYj0YDRU4GysbfUzjuiu2U6yE9w3BMcCsnOjTz/8MIM7DNrtowTAN3+cUo5g2w8t9tpLxT9HmlEGMYvqNYhJj6nbuHogxxLTBNoP1ZqaW/Mj7ozq4RhPM2NX71P79p9oLac1s/fKVHhVuEMM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263881; c=relaxed/simple; bh=nA/8nouje4cYhouLytNiDOUSXfCNAQDid1t77M3QiNw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=E/4rdTSy+2sSz7u+E1gaEQYVi6RgUE7G35UxnbGSGg1LMiq0Oj311g8h7U4kf63iQebHNZE7GfkfN2nOVG2Bzf82018eO8kw72QxBV+4xCWvNjlVOxj/DjN9gtLlizQseQMG883uyoRL6Quf02OupWv02jqCSSBd9G/OgZg9bNs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=cIEnmda6; arc=fail smtp.client-ip=40.107.102.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="cIEnmda6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Gp5cKtinkozgIO5ei3N9hMRn8pPnPTZ4RmJdUx9GU7pjrbJxKLAkfEqi9oqk+/C5YZSzhcF3oHfuiVWGuS/jxF40BRAHi+YcL9/dRsUqUUGDZ/ADALdCznRqK5BRaPERRVn2lOdxCO/pvXc11N0SNNc5FI4aHMVfHYiPLXxzX1A9BhMpohQt26N+dFMnx98PzMCGu8kCl+6XKSk6MAxc1zapmMj/QDs/wJL1/FLK7abbA+JMw0UyUzHUlJVe6AxAa+sqr8ezwmPOGrC+GJInHefrZPOyS+FFxaAs38+LnCWolnoaSQ7H3DNdnm7ggpvrxYsHTrTFlITiqzAaO0EYZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XdIeS77pvw9NrmsbvLmJ4XgKIfAqyn4uFZsi3oNCQhM=; b=yPdQEB/jm27/IANMzv11sAQDh3djc1mZZFWgjjckkzMeOpeV1CZ1UAcD20g3Nq/vmdGwftC18lCJcGCdr0E3gWmld3Lc+Vt5XGslRBbfzjgm/bRCbebNgGtCSyzFZFEQHqBsHAWLg3rrr9Y+5Lq/zVcYjHW/m2og7Lu+gNfUCESpXAOwkl3zCQTTlkAGBOb8yu+usdSdBSbwT3974daSh8V192KhNN4mvPn6nX4Fv3knmhDQIb6P9utvtiX4bSwkmtMRoS6x9HZ3+GvbhGmcdvjt2u3nUYGZTLQelU1mlqLJgXBo2atribs51TzREC/v9Fd4RSCo4mZ0kFYS9Y2YQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XdIeS77pvw9NrmsbvLmJ4XgKIfAqyn4uFZsi3oNCQhM=; b=cIEnmda6gGcMWLU9AU8ATX8sOyXgR21DQJZWHAWBGCrcn7FTCWIi9oSjO3m0+YrKsCRlY5lgJy6OrcgUgrIJtbDC1claDgh6vjK7QXBDIlq2XyjH+ah59VVo8C62wi3a3jodbV9KyvTdFUfvfLRG49maoQ5gfBqUyTFuBKDx55dE5gijGRlrqt9pkCFZxa1QwU7aZF+AwJb3waJ/ByVIPfrsQr21yhDgvN0wQRyym5VnJJvjMKAU+I2R2Ft3MIO4+dH4qwb2Mov6BkaQxA49YYVFGXtUrR7WmgAesc9mmbOBET3sFGbD652IpamWp01wyzCkmReGNh9jlMJkGun7Mw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:07 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:07 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 01/10] fwctl: Add basic structure for a class subsystem with a cdev Date: Wed, 21 Aug 2024 15:10:53 -0300 Message-ID: <1-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0049.namprd03.prod.outlook.com (2603:10b6:408:fb::24) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e8560c3-eb1e-4986-7015-08dcc20c9e94 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: cNcKkoSemlaGAJZ1/HpSPbkVjRGgDzNGgFQpMiYtVYasSyA7rVIi+ajSpQMk688XWnhOIef4V6xMDv1P+pY9Xm+gY+nJFf/r1JydnXAh9p1QmyUPAqXOSf8GlrrGyucuXv6pTPj8MGvn+ZxTMyY7lLQzm/MnIxtT4U/1Og2bRfpgRydvgPaD92g9DGjc8A+1J2OZDwlmVjr1Wf1kyB8PpWXNAb5sHpmHm1RP27ESe49qJYDHSUCOJ6ya0uKrSFiPKZQ9zACuqQZaVIZI8UGt0cce9SlGsXnObRQs82cqKkCwRWt5NDHp0a0OOCtxbD/WTlKa/W6R3ea4B5qcLFy1Y9fqxcREfyLBUAOp6GDcaB1CHzHBqAzLUqFpacrF+2RMcZtDFtu8oExXRPkpPpAiZUHJwNQBl8E2zECoEfmQb7PbAeP7ElUuVBUneCSdTe9PWtnfYtYibIv+tZnD/+x7yEOzKa3aA7i0nxIv9XNEl6RtdoIwOjNKTVgbUhpafg60cPRqPgnxWbMHa2/mU7X/R20zzi0RAZNrkx/qOb39Xjhk+wB1mP7mAZ9zjMrrdVNR2xu2qMyhMJiEbgzwAaxKk8GEm1bvdIf3esI8AjTgbEWJBDWgrjC6WfNAzdX1TLsv/WjxCrNbMLRH48DRfg+xMjj5Jwo2PExfem9CAfg78684VIL3Zv5NO7HOWqxR9zVzQnT5kkd0UFnN/L3D3Yl27EjE5r7ehv1ZE3lcoT/lei1qQJeSG9uQUz6vF8RXpMcPRiyYmx7kAaMmEHix8ibfApDPj7wrarE1tObqgFFM77WZU+U6RfskaKrl+aPECBPKF9omhhAeDxghNxfO+/RTnLmGhAtxMMIRDW3QLtZLWBxjSLgfEvxwfL0bWqH09QAp3vhLyRiDaJNRCFyBfRbokSUyRkxhQ5nta0usoMMBMkGGvsGD3t7lBRRGLs6qFB6Djt5EN5VI+1WrxP4Qte5lm1CQRifijRlKVskBXvDX0sC9LYxA+j3D4BL2q/v8xy0zftWepvau3RtlaHxKIOOD/OnyjQTWjMepJEdhl1SGPiBzjBEMpTf+bsqU/i2ZtYDBhv2Rw5mcL11vdIrm/2G/KifnnZSgaYlrKPK1SydiC5NnHekdb+HjVrN4Qp+QZWUvg+MBmvg7yXqWBu/8qalFvGH5XU650U5uWlqgTmPXgfVtQ189wngpbQ2AN3FHTMC5SiFGPtEmmfP64wV+VrxFxH0Z5TW69O24HDB1qFfxmoby0xkJ8bb1udaP7+Ae7XAU6PcZeY4Zx5LsgQf/3xDsBpX0vk1JDIELqPiAnTjly6xUAKZgW/hx5St1ssi6N+wytEqV7SMydiPeRPrlTMMGgA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jIHGQ6mf8WXfIUp8EERoMA/Sn8e5c1VQ5n8PaI7bj7LKUmABcsRog+g2bwC0Cfh6J6mzkpDl0LPQm0BXELNv+zWWJxIcLwB2BBvP3bq3COsWaKAOZT5rBpJ7+erXcEq7JpQEJodqIZLs5h/zFJfxSWwgvYerTTL9m45pIst+JTQHBNdLK9vHglIoRDhv5CpuEiPHCKfAC0YamRFn45hUiL1sp4vWJfRdTT3NOo6Mgdn+dVja4yQdww6ZfT2mjS87qz3osJjU1lkbGsOvLt39sdtve97aqLrLhxKnhFN/ogoyV5cqHfN+Dq9VUQD6KecqyQ3GZF7K0hxZS3fj4H3hauzdCPqKSaXr8rSUey9wIJR4giPftOOJYUiUVD+SR3cmum5NG871HzQ5HG03Wp4yxFC4BS27JzoHo5ndf5iPITexowlBqfH1pKqjpm0t8iJMha2LR+GYiw5r6N+MWFLzWGuon9i0O5SEiqDjcbgPyUAGFGqqKoxQXgtWWit70Xw5VSh6L4laUYdo5/W40l68A4AFH5/++ub2J0junEdw8UVf2//n5NmudT7E1BMyxWyJ42cVsNV4tNxDdtrU1FJk2hOi69hjAmukOcInu4139k3w8zeLUk0U46VjtmiGq8ZB9LoVPN37gopetei+OCLM4c3FXpwGVrvTR7ffJC7r/frleJhpZDs4SLy1BsAHTJxFhwwJHvnpGDsP9zEWpsFCi97hdq5I+jXKjfcWshZDlD5BEkfoRwXGvhYuXTEIwVFavT87tbGJJv0isRS5p24KTIXx/qJlPnPVr5+v+NHDPRpP8tEHSi9GSCBXse+k/9WVBcHDXsY4bSuXgA1ONQchKXXCZa37SNJfF2z5hm5UL3MhUUS3Gi3xJrr6iawtT7ghi3ZsERDaErtt56l/oIVN3+mrpk6SafLWcSB+zLUrlQuwHrR+w+De3trk+ZekCRZQS8rjUeJr8+JvTYamxnGkeGqlVIysYhmz8s8BIbLWeEH/J++A6IApr5PvTJhnm5CS8GzxM7kaI/6pAWUKzXlt2ChBABOIGNr3mWcgcSEuKFlDS4YtjNPF1hTzehvb6n61FU8C6LhVnhJ7gWOFatQvU9+AxesuPwC7QCZg6nsGyyUmfk0P3SEBV0HeLd22U/2yuGEYju4IICYSuR2e4aeRg2n/s42OcvoXCaZRsAa29oTDlFxv3h+m4/JJS+XxQUwNKelVxJExFQC8VYaWOqOLlGHHNaXqIBTdfywJpVkWmqOk4/SWfjPlSX/7uieq1lgj103VH3O9Zdeabg3wBBYU3RCnS63eSVpBSOFfFfqg8mDsa0Xzpr0a6hsvyq05msXBbbwZY8NK1NpKH/rNtbcdASYJ0ahQ6EOgTpdrQcj8x7SRlB1ZcXsgwvgSpxef+NaWIOFK2Ix6vHs63/pVJBG/5NNji/CRRjdkfqdGu0nw5Qw26Q1kGMN8IQehmVZ+hBuGHFsNgGRt9wsjYKMjfo6k0qFXm5J1CDFRpmxIB0p39eWhZEgFxQw9XKNsZDkrd33nvbh4tU05zydgEnh93kSZ3v2pRvPd5/En+PW+6RV/fdINzqRuHNH64fHbFmNmluFp X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e8560c3-eb1e-4986-7015-08dcc20c9e94 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.7350 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jMWb9ck3nBc5mZ19LZTopNij3Zks5o+fDl6qH57KFDQXCX9RmAkZPSzKXs50XUdd X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Create the class, character device and functions for a fwctl driver to un/register to the subsystem. A typical fwctl driver has a sysfs presence like: $ ls -l /dev/fwctl/fwctl0 crw------- 1 root root 250, 0 Apr 25 19:16 /dev/fwctl/fwctl0 $ ls /sys/class/fwctl/fwctl0 dev device power subsystem uevent $ ls /sys/class/fwctl/fwctl0/device/infiniband/ ibp0s10f0 $ ls /sys/class/infiniband/ibp0s10f0/device/fwctl/ fwctl0/ $ ls /sys/devices/pci0000:00/0000:00:0a.0/fwctl/fwctl0 dev device power subsystem uevent Which allows userspace to link all the multi-subsystem driver components together and learn the subsystem specific names for the device's components. Signed-off-by: Jason Gunthorpe Reviewed-by: Jonathan Cameron --- MAINTAINERS | 8 ++ drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/fwctl/Kconfig | 9 +++ drivers/fwctl/Makefile | 4 + drivers/fwctl/main.c | 168 +++++++++++++++++++++++++++++++++++++++++ include/linux/fwctl.h | 69 +++++++++++++++++ 7 files changed, 261 insertions(+) create mode 100644 drivers/fwctl/Kconfig create mode 100644 drivers/fwctl/Makefile create mode 100644 drivers/fwctl/main.c create mode 100644 include/linux/fwctl.h diff --git a/MAINTAINERS b/MAINTAINERS index 0ff21a07589b51..2efd8d14495431 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9241,6 +9241,14 @@ F: kernel/futex/* F: tools/perf/bench/futex* F: tools/testing/selftests/futex/ +FWCTL SUBSYSTEM +M: Jason Gunthorpe +M: Saeed Mahameed +S: Maintained +F: Documentation/userspace-api/fwctl.rst +F: drivers/fwctl/ +F: include/linux/fwctl.h + GALAXYCORE GC0308 CAMERA SENSOR DRIVER M: Sebastian Reichel L: linux-media@vger.kernel.org diff --git a/drivers/Kconfig b/drivers/Kconfig index 7bdad836fc6207..7c556c5ac4fddc 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -21,6 +21,8 @@ source "drivers/connector/Kconfig" source "drivers/firmware/Kconfig" +source "drivers/fwctl/Kconfig" + source "drivers/gnss/Kconfig" source "drivers/mtd/Kconfig" diff --git a/drivers/Makefile b/drivers/Makefile index fe9ceb0d2288ad..f6a241b747b29c 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -133,6 +133,7 @@ obj-$(CONFIG_MEMSTICK) += memstick/ obj-y += leds/ obj-$(CONFIG_INFINIBAND) += infiniband/ obj-y += firmware/ +obj-$(CONFIG_FWCTL) += fwctl/ obj-$(CONFIG_CRYPTO) += crypto/ obj-$(CONFIG_SUPERH) += sh/ obj-y += clocksource/ diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig new file mode 100644 index 00000000000000..37147a695add9a --- /dev/null +++ b/drivers/fwctl/Kconfig @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-only +menuconfig FWCTL + tristate "fwctl device firmware access framework" + help + fwctl provides a userspace API for restricted access to communicate + with on-device firmware. The communication channel is intended to + support a wide range of lockdown compatible device behaviors including + manipulating device FLASH, debugging, and other activities that don't + fit neatly into an existing subsystem. diff --git a/drivers/fwctl/Makefile b/drivers/fwctl/Makefile new file mode 100644 index 00000000000000..1cad210f6ba580 --- /dev/null +++ b/drivers/fwctl/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_FWCTL) += fwctl.o + +fwctl-y += main.o diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c new file mode 100644 index 00000000000000..7f3e7713d0e6e9 --- /dev/null +++ b/drivers/fwctl/main.c @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#define pr_fmt(fmt) "fwctl: " fmt +#include + +#include +#include +#include +#include + +enum { + FWCTL_MAX_DEVICES = 256, +}; +static dev_t fwctl_dev; +static DEFINE_IDA(fwctl_ida); + +static int fwctl_fops_open(struct inode *inode, struct file *filp) +{ + struct fwctl_device *fwctl = + container_of(inode->i_cdev, struct fwctl_device, cdev); + + get_device(&fwctl->dev); + filp->private_data = fwctl; + return 0; +} + +static int fwctl_fops_release(struct inode *inode, struct file *filp) +{ + struct fwctl_device *fwctl = filp->private_data; + + fwctl_put(fwctl); + return 0; +} + +static const struct file_operations fwctl_fops = { + .owner = THIS_MODULE, + .open = fwctl_fops_open, + .release = fwctl_fops_release, +}; + +static void fwctl_device_release(struct device *device) +{ + struct fwctl_device *fwctl = + container_of(device, struct fwctl_device, dev); + + ida_free(&fwctl_ida, fwctl->dev.devt - fwctl_dev); + kfree(fwctl); +} + +static char *fwctl_devnode(const struct device *dev, umode_t *mode) +{ + return kasprintf(GFP_KERNEL, "fwctl/%s", dev_name(dev)); +} + +static struct class fwctl_class = { + .name = "fwctl", + .dev_release = fwctl_device_release, + .devnode = fwctl_devnode, +}; + +static struct fwctl_device * +_alloc_device(struct device *parent, const struct fwctl_ops *ops, size_t size) +{ + struct fwctl_device *fwctl __free(kfree) = kzalloc(size, GFP_KERNEL); + int devnum; + + if (!fwctl) + return NULL; + + fwctl->dev.class = &fwctl_class; + fwctl->dev.parent = parent; + + devnum = ida_alloc_max(&fwctl_ida, FWCTL_MAX_DEVICES - 1, GFP_KERNEL); + if (devnum < 0) + return NULL; + fwctl->dev.devt = fwctl_dev + devnum; + + device_initialize(&fwctl->dev); + return_ptr(fwctl); +} + +/* Drivers use the fwctl_alloc_device() wrapper */ +struct fwctl_device *_fwctl_alloc_device(struct device *parent, + const struct fwctl_ops *ops, + size_t size) +{ + struct fwctl_device *fwctl __free(fwctl) = + _alloc_device(parent, ops, size); + + if (!fwctl) + return NULL; + + cdev_init(&fwctl->cdev, &fwctl_fops); + /* + * The driver module is protected by fwctl_register/unregister(), + * unregister won't complete until we are done with the driver's module. + */ + fwctl->cdev.owner = THIS_MODULE; + + if (dev_set_name(&fwctl->dev, "fwctl%d", fwctl->dev.devt - fwctl_dev)) + return NULL; + + fwctl->ops = ops; + return_ptr(fwctl); +} +EXPORT_SYMBOL_NS_GPL(_fwctl_alloc_device, FWCTL); + +/** + * fwctl_register - Register a new device to the subsystem + * @fwctl: Previously allocated fwctl_device + * + * On return the device is visible through sysfs and /dev, driver ops may be + * called. + */ +int fwctl_register(struct fwctl_device *fwctl) +{ + return cdev_device_add(&fwctl->cdev, &fwctl->dev); +} +EXPORT_SYMBOL_NS_GPL(fwctl_register, FWCTL); + +/** + * fwctl_unregister - Unregister a device from the subsystem + * @fwctl: Previously allocated and registered fwctl_device + * + * Undoes fwctl_register(). On return no driver ops will be called. The + * caller must still call fwctl_put() to free the fwctl. + * + * The design of fwctl allows this sort of disassociation of the driver from the + * subsystem primarily by keeping memory allocations owned by the core subsytem. + * The fwctl_device and fwctl_uctx can both be freed without requiring a driver + * callback. This allows the module to remain unlocked while FDs are open. + */ +void fwctl_unregister(struct fwctl_device *fwctl) +{ + cdev_device_del(&fwctl->cdev, &fwctl->dev); +} +EXPORT_SYMBOL_NS_GPL(fwctl_unregister, FWCTL); + +static int __init fwctl_init(void) +{ + int ret; + + ret = alloc_chrdev_region(&fwctl_dev, 0, FWCTL_MAX_DEVICES, "fwctl"); + if (ret) + return ret; + + ret = class_register(&fwctl_class); + if (ret) + goto err_chrdev; + return 0; + +err_chrdev: + unregister_chrdev_region(fwctl_dev, FWCTL_MAX_DEVICES); + return ret; +} + +static void __exit fwctl_exit(void) +{ + class_unregister(&fwctl_class); + unregister_chrdev_region(fwctl_dev, FWCTL_MAX_DEVICES); +} + +module_init(fwctl_init); +module_exit(fwctl_exit); +MODULE_DESCRIPTION("fwctl device firmware access framework"); +MODULE_LICENSE("GPL"); diff --git a/include/linux/fwctl.h b/include/linux/fwctl.h new file mode 100644 index 00000000000000..68ac2d5ab87481 --- /dev/null +++ b/include/linux/fwctl.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#ifndef __LINUX_FWCTL_H +#define __LINUX_FWCTL_H +#include +#include +#include + +struct fwctl_device; +struct fwctl_uctx; + +struct fwctl_ops { +}; + +/** + * struct fwctl_device - Per-driver registration struct + * @dev: The sysfs (class/fwctl/fwctlXX) device + * + * Each driver instance will have one of these structs with the driver private + * data following immediately after. This struct is refcounted, it is freed by + * calling fwctl_put(). + */ +struct fwctl_device { + struct device dev; + /* private: */ + struct cdev cdev; + const struct fwctl_ops *ops; +}; + +struct fwctl_device *_fwctl_alloc_device(struct device *parent, + const struct fwctl_ops *ops, + size_t size); +/** + * fwctl_alloc_device - Allocate a fwctl + * @parent: Physical device that provides the FW interface + * @ops: Driver ops to register + * @drv_struct: 'struct driver_fwctl' that holds the struct fwctl_device + * @member: Name of the struct fwctl_device in @drv_struct + * + * This allocates and initializes the fwctl_device embedded in the drv_struct. + * Upon success the pointer must be freed via fwctl_put(). Returns a 'drv_struct + * \*' on success, NULL on error. + */ +#define fwctl_alloc_device(parent, ops, drv_struct, member) \ + ({ \ + static_assert(__same_type(struct fwctl_device, \ + ((drv_struct *)NULL)->member)); \ + static_assert(offsetof(drv_struct, member) == 0); \ + (drv_struct *)_fwctl_alloc_device(parent, ops, \ + sizeof(drv_struct)); \ + }) + +static inline struct fwctl_device *fwctl_get(struct fwctl_device *fwctl) +{ + get_device(&fwctl->dev); + return fwctl; +} +static inline void fwctl_put(struct fwctl_device *fwctl) +{ + put_device(&fwctl->dev); +} +DEFINE_FREE(fwctl, struct fwctl_device *, if (_T) fwctl_put(_T)); + +int fwctl_register(struct fwctl_device *fwctl); +void fwctl_unregister(struct fwctl_device *fwctl); + +#endif From patchwork Wed Aug 21 18:10:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771950 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2064.outbound.protection.outlook.com [40.107.102.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F1601B2EEE; Wed, 21 Aug 2024 18:11:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263875; cv=fail; b=fxrTL22tgzTGmPcBQcKUIa/xZOHwi1fhh9R3ku36UFsPcO5GYU9U85kEmlQC2QH4G2P5nm6Ysmzp80KMnNEVj1At/khXx7dmEvlHMFveOlP0dyzq74yo9vBs1T7YjqKxCOd52puhtpDV/KvpkabOneqSU/XE63kvX6oCUTHpnms= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263875; c=relaxed/simple; bh=GYGp/T9EfUE2ityxYAQtDdxdxyXWfnoiIhiDWurDZ4s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=So936qqHaZkOT9PCe1QEPu9oCNe8/pLqUSu34eWZ4hHmOl0a9Jw2TgMlp9mmKfPaJFIA0mvo92RboVclOpprSJrqgrmB8H3mXvcn4yXab3uBsQA39I9+PdD0iwSmGP8dxpPIkQpke/WzfEx41URANsE+AbA6uTbiPqIcGJlZTkQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=oh1Gzt2X; arc=fail smtp.client-ip=40.107.102.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="oh1Gzt2X" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ki20pFgDyVA9UgUJQ388mF4142ytUiebg0rH94M60gmeWfue0mrWRYUTOTR44GtCDKMS06qoJSU1N5xuYjoXwmQibLAlbbl1dIFD8NiTEo4jezdG+3BVyrBQc2Gv8DsflFaFdU2rNIJX8Yr1pFTUpZ3l4LAG9yevTZn56Gk8UPChE9paeRpIceecGKD23MnspS15bAvLePCYzGIrHDahh5X2xhLzu45jJRP08Sud3AU/UTzZ+UTLFfdzzIKn9hH9SKP+nckVuxzH/CnUdmQHDyv2tjhQ0Me/wDnu+f1lxtuxY+uUuw4VsnawPAp3BzmnAS29b5nz0ecJ6GKbdrMaLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K2o3L4w/+KbG4jKblR2NFAvATq1v0LB+vLmU9D7lv2k=; b=SM2Ur1q1qW5tpziJzMABklZmg0FGBGaE+Tj4wfFChjec+aECFaNJDWuWZJF2cQqGXo3Tw/JzAQsZZ+LShPIa5xbLAp5DDSson/Mth9hHu0J54v1pJ2XB9vEo3kFt8mz9Ulmz/WF1zuBqmROtoznmdDbMDh0Km2xiWweas274cW77qrSDhwG7FmjcC/pPRE/otK1XpHmwyJzxX1fW2Yhw6wBuCVQUpC9Rd2VmF0kp+xMe+GRyvj6GPDr/CPccA0ALHyGBNLHPqKRuhZVxTzDKgEPAArzqpFyYbX3vqy9jH78rpcyohVj04fJipXg+4OHlVmXsrQCYT2ujfVxgiNjCpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K2o3L4w/+KbG4jKblR2NFAvATq1v0LB+vLmU9D7lv2k=; b=oh1Gzt2XYvESoSNt2Yk0t1oVjk/gLX2Q4cbgxB2k7bukO0pMFonxfXF9iarA7CU3DECPWVnm4a1JVNa2VUf9iSEIba+uiY8O2iv5osbbV54Y7e7f7YE5lwY21rp+kUZU8R2wvtgRDc2RZRXd9Id8ug5f+SZM/P1KZ+U17DGcE4c79r5gIHNoRqfRNNuXOebul4gHWUMr5AS9T+Xwx3DhZISA32h/P9owV++TSFV0rXHbh62CwwWgr/P3FWPnRpTPHoUcpMKzgc+rfRo55rf0B2Ew0hlyA63z2OPdrcHRKkstTD4rbGooXWP6jPkDKn/ZUdKGCMar1ptpn4i97Yl5BA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:04 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:04 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 02/10] fwctl: Basic ioctl dispatch for the character device Date: Wed, 21 Aug 2024 15:10:54 -0300 Message-ID: <2-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0927.namprd03.prod.outlook.com (2603:10b6:408:107::32) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 36ed9aa4-287f-4df1-c062-08dcc20c9e3c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: tIQgqHM3IzLl5RUmBR8kLlEL8+plMFhhTic0cEIKdH1oQ73i0s+70gjhT67Js2e2kYWQAh9KanpL2Hc0fgOKPnJYPK7I3HsoD1x3PKvFoAb0Aadyy6zZWEdxL5aD2hptSa0XGdQPVA+VLPXsNvrh4z512Zh8/AKboPDnP5JwiOKn6jS7NutVziDRtUtLzRESjtC8ZpcTBr2XpFbYrBG+fuXAhAWuRuwZqbU2CwMKsdtdYQNszoCo+gtj0FETSq9ZgaDfjJfigtP6Pq4kb0FDF066h7voi7XD7reQIsM5XStdrr9kML/u5IiQxtRUx5rCikeyk+ANRlVs+lNH6vdy6xLdErizYb36aigm4ilegoFBCLN7eOUq7umzyFz6XP2+C8AQ+EowrWLe3qYOcBS7sYCRDN4SuAr0a92xTsQjgJfanL+wwcZMvJRq3APGQT58uvT78NaeO35aODP9zKkxqnw5KMIy+DyHq+GK7s9wLy8B9f2M8uTD39OLEtahtwhREJ4xmXkMjV9y+4aXwwzqSdjMiHZlgirmh1txLT3lSDefvorR9caMA5lsdzb1Xxy4MZNsCd2vlKuK1K0+m+wzLclgYN6Ix63Kf55s1Cy0YVPrQDvv2EUIS91NTl2SZj4pMf/arjW3byQNw33grekrEvF8aqNoGndaT00ujqvXxzNxstcxAQNwPBnaqAzQI33Kry6b38bHF6RlriO+KRbTLYT2OE8CWkwqE5rqGLDETJrsv4KAovvm/d94Tx/oTQKxmQHmbfTHQ+uB83CQT8jGj+UhYFF0eOR/U40OiBfrzOFPapZW1myBv42jbdZLGw3LbFb6s151sMNGb0b0f5mAV5Ddl0UIAS3kl4SQxnWxcQBefMgS5a+KZLAcpGJq4tRG+lVpaZhOWdHuy3+55T6jSOakC7z8ZNghDDWjq4aRNYq0bXB6Fqy1NJ4H41gQ6tr2mUAVnHN9v06r3IVt/FsQAcgIHdT5oaCgCa8i2E9CSl3NGkLmXl8tJoB7KLa0sZTg0/BipoKkL4ovZbsM5chZMdd34gFyFk6sipcKyoJ/X4JkaC8/VaFHbDiAqeKsB/m6sXit9PaWdtE1SzlZYyn6VetTfJnRihtNtyHABJOuJXM8tIKRy6SfjOWLN5i4Z5Ei5uu5nI0InVC961kAFsBngXOhSh5lBT+NEiaNojjq2UgmgTc2Re21VZsXtROWla3vE1B4o1Vxx3DOuVXlvpTtphVqqOUbZFxuUS5PvPwjw9+5ey8zkBN9WybjnzbABR6RTUyCANBhxWuqiFdzLqbWjdEtqxpcDOi7tp3kfjLiKguQo2WrDPDGct4Ak/sY4kIFY91/ZZTY6Jnxu5IYoL7zUg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9QHGM6LCMHEbV4hCFW+5051I+Ty0Nvnz1f2qD4Go3EGLK6neKLQzXx/j6OZfso1DxWMmYMtiZmALy5rPHXaZe5dhJLLFgFHpcaZncI+ksZbTORYiUVJoUYqul0WGeWBNvyLKODJCz87B369TGx0FQabd9feltyv/tyqNQc3HzVE02NIJSrj/2xdPoFYnYRQcelF8EMY1Z/0bEo4CqlSE3NqKNTmX1/hWSP8lBWHwnySmbfbs2Pl9u0Znf7CvfDQnVlnRVmDmO3LkgakObmkeGC8UptbFJGe3aAF2atYrK4swPotXnhHgW8IUzUfNEqX5+cS6iStbuq+5j6f5/kgoKi+4hNCSc4ewwqBYFK+r1yxzJwXyjowfK0w/I200MiADKYUAZb9DUq2kklT0mRB2PzBkF50JjcjiSEz65BzymsJ8cHNeqXeAlEpyIKcEdyhmfT3BnZmVieGyKpAgbbH99N72sQQGt2C9lh39RRWV1XFDnSx6R6srmsc7FHaAKl9V0bwvjk/xHV+r/BiKDiFMchHrlRtJ4zKYNqYS1b/KowpuyHWi1d6fNJrfyj68tygc4tSl2lX3AjZRvp6aRtQnJk8Exmupb1M06zgJhNyR9135e7jkqVw4vzavQ7bJywsrxLlTV9DcfWxbPrS669GWYQlP9M5UKpW9xYS+DGZireHmJnGKdK1vw6DTcJ9R+0Yn0HLPF6SXaLrqT2g1M8C2/P/MzqTzlNav0WyXSm76GiPmeMoRmSmM84RI0qgn5VVDks6Foiyi1BjVvqh4lCgrUmqHZhbuYBJ3/7DYeci4zhu87sTsHgZ4dQrjtJ5g7TfYdkctXUa+0uQn+kjsymT813wAp2KUwoemHCajM9uhYx+vFAAM6ocSIu7MF4gRA+ML2mhGepXea/61BuXKIqDPT7XbWGSd396SNriI1nyVz5I4d0BXzBNuAM95RG2CXJq4oHVOqlQ4ngBhCQcmXl4nUZyYv1UoflcrSPVitdkLxgApW9GW8Zvau4rdUsAdYAHNa2tYKRjhYdcpSru9+psv1ahZv6qtzA51V8XFmm82dabCQdsOR8+3L2JYMbNdoQKD8f/ttHC4BzkiUnnDlELSw9qxGB99DiYtGBa4ypYKsmXp3BBEGTs5vLvAJB5Bn+WAKVClejJZHQ0rC+Tzm655eckxUzp+hj7meU6g5wos64eVKV53UQkmbMULLd5auuAgHF0Va+GbgMNnhBl6Tyq42FM2TRS4SEqxpPGvNi43qgmR19lbWvxdyebWcmSwsb2F6wk3digg2OTviuZjAK4c5Ga7j8KjDwB/WlichEcBB/u8E+tmNNLizAKNaUkifOqlIWqf52pvbnJ6vnrPQUbB7ACMP2TwQR48CPP3X1Ia/Lan9sd4lT+/wPCMUZI8tT02M/WhAT9qYSy3Zi0p9QYgqpqo43H0TI2aRXkZamftL6u406efRXKx+FDCxqAcx/ty8+Uka2c1EV/LuCz7IpA/sRZrrcJWrRggp/vL4Bw9hKm2mx2VptNAOwCpjgeXFrsQkIFpEMa5k6Rx1cUuh2ZNDpIg7wp+CPM22G95x6PavBAhDSQjckJKxAcTjDd6ZLoW X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 36ed9aa4-287f-4df1-c062-08dcc20c9e3c X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.1594 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 01MmbLfwzdYA278Riqvbswu5Z/hBWDaM/JREELsRNVWTkkbLkIgoHEBmVOsTzzGA X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Each file descriptor gets a chunk of per-FD driver specific context that allows the driver to attach a device specific struct to. The core code takes care of the memory lifetime for this structure. The ioctl dispatch and design is based on what was built for iommufd. The ioctls have a struct which has a combined in/out behavior with a typical 'zero pad' scheme for future extension and backwards compatibility. Like iommufd some shared logic does most of the ioctl marshalling and compatibility work and tables diatches to some function pointers for each unique iotcl. This approach has proven to work quite well in the iommufd and rdma subsystems. Allocate an ioctl number space for the subsystem. Signed-off-by: Jason Gunthorpe Reviewed-by: Jonathan Cameron --- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/fwctl/main.c | 139 +++++++++++++++++- include/linux/fwctl.h | 46 ++++++ include/uapi/fwctl/fwctl.h | 38 +++++ 5 files changed, 223 insertions(+), 2 deletions(-) create mode 100644 include/uapi/fwctl/fwctl.h diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index e91c0376ee5934..c581686451fb1e 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -327,6 +327,7 @@ Code Seq# Include File Comments 0x97 00-7F fs/ceph/ioctl.h Ceph file system 0x99 00-0F 537-Addinboard driver +0x9A 00-0F include/uapi/fwctl/fwctl.h 0xA0 all linux/sdp/sdp.h Industrial Device Project 0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver diff --git a/MAINTAINERS b/MAINTAINERS index 2efd8d14495431..97945ca04b1108 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9248,6 +9248,7 @@ S: Maintained F: Documentation/userspace-api/fwctl.rst F: drivers/fwctl/ F: include/linux/fwctl.h +F: include/uapi/fwctl/ GALAXYCORE GC0308 CAMERA SENSOR DRIVER M: Sebastian Reichel diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c index 7f3e7713d0e6e9..f2e30ffc1e0cb5 100644 --- a/drivers/fwctl/main.c +++ b/drivers/fwctl/main.c @@ -10,26 +10,136 @@ #include #include +#include + enum { FWCTL_MAX_DEVICES = 256, }; static dev_t fwctl_dev; static DEFINE_IDA(fwctl_ida); +struct fwctl_ucmd { + struct fwctl_uctx *uctx; + void __user *ubuffer; + void *cmd; + u32 user_size; +}; + +/* On stack memory for the ioctl structs */ +union ucmd_buffer { +}; + +struct fwctl_ioctl_op { + unsigned int size; + unsigned int min_size; + unsigned int ioctl_num; + int (*execute)(struct fwctl_ucmd *ucmd); +}; + +#define IOCTL_OP(_ioctl, _fn, _struct, _last) \ + [_IOC_NR(_ioctl) - FWCTL_CMD_BASE] = { \ + .size = sizeof(_struct) + \ + BUILD_BUG_ON_ZERO(sizeof(union ucmd_buffer) < \ + sizeof(_struct)), \ + .min_size = offsetofend(_struct, _last), \ + .ioctl_num = _ioctl, \ + .execute = _fn, \ + } +static const struct fwctl_ioctl_op fwctl_ioctl_ops[] = { +}; + +static long fwctl_fops_ioctl(struct file *filp, unsigned int cmd, + unsigned long arg) +{ + struct fwctl_uctx *uctx = filp->private_data; + const struct fwctl_ioctl_op *op; + struct fwctl_ucmd ucmd = {}; + union ucmd_buffer buf; + unsigned int nr; + int ret; + + nr = _IOC_NR(cmd); + if ((nr - FWCTL_CMD_BASE) >= ARRAY_SIZE(fwctl_ioctl_ops)) + return -ENOIOCTLCMD; + + op = &fwctl_ioctl_ops[nr - FWCTL_CMD_BASE]; + if (op->ioctl_num != cmd) + return -ENOIOCTLCMD; + + ucmd.uctx = uctx; + ucmd.cmd = &buf; + ucmd.ubuffer = (void __user *)arg; + ret = get_user(ucmd.user_size, (u32 __user *)ucmd.ubuffer); + if (ret) + return ret; + + if (ucmd.user_size < op->min_size) + return -EINVAL; + + ret = copy_struct_from_user(ucmd.cmd, op->size, ucmd.ubuffer, + ucmd.user_size); + if (ret) + return ret; + + guard(rwsem_read)(&uctx->fwctl->registration_lock); + if (!uctx->fwctl->ops) + return -ENODEV; + return op->execute(&ucmd); +} + static int fwctl_fops_open(struct inode *inode, struct file *filp) { struct fwctl_device *fwctl = container_of(inode->i_cdev, struct fwctl_device, cdev); + int ret; + + guard(rwsem_read)(&fwctl->registration_lock); + if (!fwctl->ops) + return -ENODEV; + + struct fwctl_uctx *uctx __free(kfree) = + kzalloc(fwctl->ops->uctx_size, GFP_KERNEL_ACCOUNT); + if (!uctx) + return -ENOMEM; + + uctx->fwctl = fwctl; + ret = fwctl->ops->open_uctx(uctx); + if (ret) + return ret; + + scoped_guard(mutex, &fwctl->uctx_list_lock) { + list_add_tail(&uctx->uctx_list_entry, &fwctl->uctx_list); + } get_device(&fwctl->dev); - filp->private_data = fwctl; + filp->private_data = no_free_ptr(uctx); return 0; } +static void fwctl_destroy_uctx(struct fwctl_uctx *uctx) +{ + lockdep_assert_held(&uctx->fwctl->uctx_list_lock); + list_del(&uctx->uctx_list_entry); + uctx->fwctl->ops->close_uctx(uctx); +} + static int fwctl_fops_release(struct inode *inode, struct file *filp) { - struct fwctl_device *fwctl = filp->private_data; + struct fwctl_uctx *uctx = filp->private_data; + struct fwctl_device *fwctl = uctx->fwctl; + scoped_guard(rwsem_read, &fwctl->registration_lock) { + /* + * fwctl_unregister() has already removed the driver and + * destroyed the uctx. + */ + if (fwctl->ops) { + guard(mutex)(&fwctl->uctx_list_lock); + fwctl_destroy_uctx(uctx); + } + } + + kfree(uctx); fwctl_put(fwctl); return 0; } @@ -38,6 +148,7 @@ static const struct file_operations fwctl_fops = { .owner = THIS_MODULE, .open = fwctl_fops_open, .release = fwctl_fops_release, + .unlocked_ioctl = fwctl_fops_ioctl, }; static void fwctl_device_release(struct device *device) @@ -46,6 +157,7 @@ static void fwctl_device_release(struct device *device) container_of(device, struct fwctl_device, dev); ida_free(&fwctl_ida, fwctl->dev.devt - fwctl_dev); + mutex_destroy(&fwctl->uctx_list_lock); kfree(fwctl); } @@ -71,6 +183,9 @@ _alloc_device(struct device *parent, const struct fwctl_ops *ops, size_t size) fwctl->dev.class = &fwctl_class; fwctl->dev.parent = parent; + init_rwsem(&fwctl->registration_lock); + mutex_init(&fwctl->uctx_list_lock); + INIT_LIST_HEAD(&fwctl->uctx_list); devnum = ida_alloc_max(&fwctl_ida, FWCTL_MAX_DEVICES - 1, GFP_KERNEL); if (devnum < 0) @@ -127,6 +242,10 @@ EXPORT_SYMBOL_NS_GPL(fwctl_register, FWCTL); * Undoes fwctl_register(). On return no driver ops will be called. The * caller must still call fwctl_put() to free the fwctl. * + * Unregister will return even if userspace still has file descriptors open. + * This will call ops->close_uctx() on any open FDs and after return no driver + * op will be called. The FDs remain open but all fops will return -ENODEV. + * * The design of fwctl allows this sort of disassociation of the driver from the * subsystem primarily by keeping memory allocations owned by the core subsytem. * The fwctl_device and fwctl_uctx can both be freed without requiring a driver @@ -134,7 +253,23 @@ EXPORT_SYMBOL_NS_GPL(fwctl_register, FWCTL); */ void fwctl_unregister(struct fwctl_device *fwctl) { + struct fwctl_uctx *uctx; + cdev_device_del(&fwctl->cdev, &fwctl->dev); + + /* Disable and free the driver's resources for any still open FDs. */ + guard(rwsem_write)(&fwctl->registration_lock); + guard(mutex)(&fwctl->uctx_list_lock); + while ((uctx = list_first_entry_or_null(&fwctl->uctx_list, + struct fwctl_uctx, + uctx_list_entry))) + fwctl_destroy_uctx(uctx); + + /* + * The driver module may unload after this returns, the op pointer will + * not be valid. + */ + fwctl->ops = NULL; } EXPORT_SYMBOL_NS_GPL(fwctl_unregister, FWCTL); diff --git a/include/linux/fwctl.h b/include/linux/fwctl.h index 68ac2d5ab87481..ca4245825e91bf 100644 --- a/include/linux/fwctl.h +++ b/include/linux/fwctl.h @@ -11,7 +11,30 @@ struct fwctl_device; struct fwctl_uctx; +/** + * struct fwctl_ops - Driver provided operations + * + * fwctl_unregister() will wait until all excuting ops are completed before it + * returns. Drivers should be mindful to not let their ops run for too long as + * it will block device hot unplug and module unloading. + */ struct fwctl_ops { + /** + * @uctx_size: The size of the fwctl_uctx struct to allocate. The first + * bytes of this memory will be a fwctl_uctx. The driver can use the + * remaining bytes as its private memory. + */ + size_t uctx_size; + /** + * @open_uctx: Called when a file descriptor is opened before the uctx + * is ever used. + */ + int (*open_uctx)(struct fwctl_uctx *uctx); + /** + * @close_uctx: Called when the uctx is destroyed, usually when the FD + * is closed. + */ + void (*close_uctx)(struct fwctl_uctx *uctx); }; /** @@ -26,6 +49,15 @@ struct fwctl_device { struct device dev; /* private: */ struct cdev cdev; + + /* + * Protect ops, held for write when ops becomes NULL during unregister, + * held for read whenver ops is loaded or an ops function is running. + */ + struct rw_semaphore registration_lock; + /* Protect uctx_list */ + struct mutex uctx_list_lock; + struct list_head uctx_list; const struct fwctl_ops *ops; }; @@ -66,4 +98,18 @@ DEFINE_FREE(fwctl, struct fwctl_device *, if (_T) fwctl_put(_T)); int fwctl_register(struct fwctl_device *fwctl); void fwctl_unregister(struct fwctl_device *fwctl); +/** + * struct fwctl_uctx - Per user FD context + * @fwctl: fwctl instance that owns the context + * + * Every FD opened by userspace will get a unique context allocation. Any driver + * private data will follow immediately after. + */ +struct fwctl_uctx { + struct fwctl_device *fwctl; + /* private: */ + /* Head at fwctl_device::uctx_list */ + struct list_head uctx_list_entry; +}; + #endif diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h new file mode 100644 index 00000000000000..22fa750d7e8184 --- /dev/null +++ b/include/uapi/fwctl/fwctl.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. + */ +#ifndef _UAPI_FWCTL_H +#define _UAPI_FWCTL_H + +#define FWCTL_TYPE 0x9A + +/** + * DOC: General ioctl format + * + * The ioctl interface follows a general format to allow for extensibility. Each + * ioctl is passed in a structure pointer as the argument providing the size of + * the structure in the first u32. The kernel checks that any structure space + * beyond what it understands is 0. This allows userspace to use the backward + * compatible portion while consistently using the newer, larger, structures. + * + * ioctls use a standard meaning for common errnos: + * + * - ENOTTY: The IOCTL number itself is not supported at all + * - E2BIG: The IOCTL number is supported, but the provided structure has + * non-zero in a part the kernel does not understand. + * - EOPNOTSUPP: The IOCTL number is supported, and the structure is + * understood, however a known field has a value the kernel does not + * understand or support. + * - EINVAL: Everything about the IOCTL was understood, but a field is not + * correct. + * - ENOMEM: Out of memory. + * - ENODEV: The underlying device has been hot-unplugged and the FD is + * orphaned. + * + * As well as additional errnos, within specific ioctls. + */ +enum { + FWCTL_CMD_BASE = 0, +}; + +#endif From patchwork Wed Aug 21 18:10:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771957 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2064.outbound.protection.outlook.com [40.107.102.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D6701B3B0C; Wed, 21 Aug 2024 18:11:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263882; cv=fail; b=CmWSHmyoixodxq/BwEh2Ev1IpORpVa/665Cr+RLMMeTGyqihglH+TGaGNxnd/jG4dVi3Rfi1tmP1bKagPnAl9/AQCWQzb9+LILJFFOlAsldpHXkk8VQCRpL4+Oi9CnRV0uycH1CZrsWsP8HkqPs8yfznnon3+4VM5/I6p1fBTlQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263882; c=relaxed/simple; bh=H8CRQHybIweGG8+VFkqysgEPGA81TafLEKlFY7L/gIo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=r1VV7/W94XTsqfQ9mkcH8fAJtjSB9plmlOPxq1rYZE26gX5QcAr0ziGLmfXLHAmS5Er9rkXf0vJ4sykcKcuiFadLcVsTxCrBNUGVExam2KyVlCCqZ7i4GyIKqeS1TVNyFQ7PFUNYHF+CWO75cBUlUWMo8Uq69UX44c3p+9R5p7Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=CXRsIkdO; arc=fail smtp.client-ip=40.107.102.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="CXRsIkdO" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SrjdZsRhmTA6T229AS9GSulKZS2wnyFcZbcSjBG1dz773Z93ae5RIzzNdjXCHsZbcvYV0VcF8T0AY3TC0adBge5db0LW3T8UFiAXgf7T6TN+h2KxpwcjjyUhkA8fGLYpzo+2Jc+sp8IkNyrs80jEede4KYcLKfBK1x7PjofgCvpKVdgQwVwwNA+o5rNU08uJ851IkKmKLAqdnzy5/YGUvo2S+Y9xSfEa/tAIgtPX3ZI3JYRAThYsvQ4LC/8SxDzmJDpwy8XgBDTlSqid5+J4mhAx5EXoEtck+V+z7AwMO0I52LjG99ZED8t0vqSeOvJfGm2dhMu342UWIeM1gkhL1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V8T3SfDAXcb5R8P/USBSV95lHUF35XIFqpVPS1ouj/I=; b=awpXsNJNeS9oEm0QxhaKoygMXX5ipmQjOPfd9dRhISpKiCNX4dSVmI3MshmCoxBjHQLBM629jA9LAwVpd4TLC9RIQhCUVdhMu1dNpL351FuNnywSap5KsuxQgM4g8OQ9PtTajy3XOjvt0AlR95P+3SaLSmtvCGNFUK5umFJLW+u4T6Ou5sit31q+TXntRJluGa2njHaBl/UbhanFEmWd8xjC3XSpz7fmqQrUYE0r1s95s/7CFQQUnzdLgRRs/dSw3o16Cr7Wd2n2wQthf0XpVOtXGw8eleqYZpNGj9JflPvUGnhd73T8x8bysZTbR+hv0IEBovpHqjZYQ1H9P/68PQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V8T3SfDAXcb5R8P/USBSV95lHUF35XIFqpVPS1ouj/I=; b=CXRsIkdOX6LdFylKEoFfWd5OHRW1AfpnfD1/tY4EFLiRLTXeI5M6neL13fMYZ3X8+YPFHo1x99NB5foFfgFF9jis/Y7TAG9kvhHhM9SKanZs9urLy96lPVl3iwrIWvCmC5ff6tIlQsEno45WLPdFqrUMHVWI5opu1wludNRyNgC3eSDT4tpZQuM+1oLXQDz8mTb7VP4V5Jt1rpG04Ocuxkcjg1GTZCGwWmIFIj8LryQ6kyiKIMTtvRuROMiGiaNlvnESPQYUsCQ26KdCDyYBsggT+PVB4Cd6U3z3/M9lJTS0/6+KpUri8AMvJ3H+D470ysp0gsByRmcImFfVpr2veg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:09 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:09 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 03/10] fwctl: FWCTL_INFO to return basic information about the device Date: Wed, 21 Aug 2024 15:10:55 -0300 Message-ID: <3-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN8PR04CA0037.namprd04.prod.outlook.com (2603:10b6:408:d4::11) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: fea86944-e00e-4609-2a57-08dcc20c9f19 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: 6o3KjWZ+mqwOdEgQMiU+jjke4jHUEkYgwRsfy5O1jH3mp54GoKUMOA4/gAubGEVsC9wJdqeBfLW78sS0fTfRt3ivVQ13Z1ezch6jeu22oOY7VNF/KHMj10QXEUPKuxVlBVAw86yC6HJ+HWlW89ZvcrhCixNqPtvARnoK/x4p3Q+SN82gd+HSdajjnd1QMGxhgDjY+a27A0m7PdoZv8Vyaw3OIGcfW5cqt8YtdumfF95Gvhy+K22cx7/y2MHSsnQKVRM34zJuiJuAZyya0HAwFjfoi/QekmN0YlI7Kec2q1D6s493OaWGG1wOlEkvJfCdfnuSlIOF+V5S9kP9urjCXte/UGqTuI5sDPdGoxNH+X0F1TIKn5uyzyLsMLxQ2oFWnobynOze/8LtIh+s8oozngeApsWaTfm1sIUVFv7jueWqHNhP0vSwg9GlauexQgQf2CSktZis81YEcbA5PQHI85qRgmzXTJE/Jb16+huoJWyt8ahN5j+qJ/xpE3Mzk/JG5qWmEIJSRc0Y3OXFcDcuO3kF6nkVVJ5fMvFeGBUuGixefimhhk5kDS8xeELOooyWQZ1x6wc2XujZorf19/vrmEMLe+8ABh97N/iMWWNs34bKWyqC6IXDM5cy8nnMEXHIaU7RYsf45av7EMVaNNjQxH6bsm9K0cTTpx1ZJP7QkYdSgBB5lSOjZqnJkwHVRTlAmctEVJKaS7JeXhnSEvlc6/w/z3b8hHprXYLy4cPNpOpnjEyAFQHFjRYaB1ZiGhSYaIQe7sbkM7R9S3oJ53H3cffpyc/1P9tLZWaaZTKz//K69Kyk7QAU/G1CGG9Ov/R+CQ6VTHPd+31rZIw+UfwghMWmyLH3jDRjgHCeSqNaOY8tBxqSaBrnmMBtSZXkY3aTkW4xPfnQRZxJFuUY/hFvlRYIPFVSdLk5K6KEFRVE5puZnMpWcl0EcdIJ+4Y8nwJVs0I8l9ra3RPxC+IebiEitil+ZVza7euqLoTh6tkO46GSl52sIToFAOzXwlM53HzwDAlm9oVBb3twRreG4sQAQkTZuGNitvUhlbdwsvN5HJSZ43xtsNMU/tSAQLoQUKXJc+jBrqkOs0dWc6ApXPw+O7ogzIKrrMUnGX8w6y9PFPsxQ6VTiG7VtqdGpTjUm+8fBhKKbdch2151XpYFKvbIvZXOPVd9rERNDhzKITnHHwlgb7HlnTHb6qq3Ohhp675PJn+uL+wbtwOU/uF+StXxEMnP72WEFgxwcqgYUCOz/mbqoWWMeWoOkz3ENDHb2zKnMrrJR2zQXVgVoTqMjcpcSolpy1hj5iAVYZrN9an2ccOBv22MO7RZBsJnCZSQrUFCzqf7CFw27DbS5BsBP91GRQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BinbwrOsz+S5FOsikxk1/LPs2FIgp6EVCqRLYOOp2ZJ7ireJi5RSSVi8hX958MhpDxSgCUhJBFxjdIUtr8JsUL0hYoBkG+qK27wgevuwS8Ip0Ra7S8NgW8obK0dcxEGNYzj7uJh/61srMSufLGadHl9JWrLhmXaUXO2LTmTpc7oGYOt9oprJpUeQDRtiS0TB5y9Be3sBiDhFngJWC2n2r3YzFNW7QO6fGfikO4XevH38lLjLoYi5gka4Yq/n82Kdb7bWllXdZH0F69ESlFkOTjbTIK0RaK5BCz1A6tgp3CFu53n3EJRa/kOUFyztz1V9BiILq46Osw61wm25kDXTLkWtBG1ZuYJWi6Aw7zQKv32g96NG1bwe0n+J8w7C//V7XHwzRdNu3b+x6kpn7sw1Fa8oQJKQo08l3O4wluWSoBmHBqMb/mOxaOK8JAr5yuDPvAUIjSSs0DhFV1dvkyXXTBMT90sCR+l+DaK7kFcdo6jeBmXnknusTxexrF+0Xds5GqXzpR8DtYu3feEHYZMj0P66ZVh8yi34NiUGWdNl82dH/Yh0I+bwM6hIa4EzpQGwPIvo65XAJNq/Lp8j2eA52mm3jqDYUp5eBw2yhBXWUcckcu3bQzS7arj751qNxPBV51Vibb5BLo7t+R3C5lT7iOdh1b+MPxPOYvEzczSoaI4xs3qm3ipMlI+Kr2zUGfEFpPy1vSwv4Ex80B/1yW9iiUdX4UyfDmwTuoJjqjwCYQjnyHzOu/3JdODjNXZwuu7D22mVwfWgOogAFIMAwIo8FckwGRfygoCPk84yvQqrOmkQrszftDy0UiZOYUukg8VMa4xM/lIW+CcupptglQ2Z5oUm1FcRU+1HIvxkQBBMN4Btr7/EreWiA1ldmeEDCHCAo2otYlLnlaAxHZIi+ZGxMawrTNyktyzysbnEWi/sS86iiDtsZEvG/PFgAuT+6oi5aRfERomNuLkbZvO4w/Z4a8xQ2IdKqNldumjCGG1UUKr3VJLn2PJIoN0AHU0w7o9tR+aoIjBb61GXOkTpAK7XS/fXP+9kP+08r33mZ+LcjFfiuEgywYbN9EXcMMrTqj0qCWpnUavf9hHL2vblJCKY1w2edjKgI9Y2HNEIJ0pQfBSrlw3TiXVjVxMzfkICWkNUdqG5Vg2IsSk+48xjyMQbhku5IMeGe5TBnUXJLMcrj1eZT4tDzZT/O2abEEsBXX8EYxd5gwApWzPL6l0v7XWYN1gwm7ErHDWL4X9VFcZIxXcz9hlKTcHOQvBryCmDFZvc+O+6tE9ZPkmBrR4jpSEKzQVfCnaT6qF4f77S8I2RLI8pGcwjyaKlJ2oWcNGEFJnzn6MSAp5wTIqcCqY4VBHrG8sQH/lG3KTBEpmSOKU4e1GE++F2d+QiKqP2CYicV6VR6W6ejDQg/cJrLHP5d8K+JkQpAqUu7ZjOFJ0hYaqwSvsonIPo2VsDATI1IaRs1/gNH2OR0DCIS76sJNMegF2XeQinwnzvLIHQxzVtR+OZx+pOpVMum2H0B0UDrUvh1D4ZhOgNWpTDowQcoe4MTlPKDNvF9oc4UwUfnDi9yX3/K04nftTJnvAsqmV1gtuDvw+z X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fea86944-e00e-4609-2a57-08dcc20c9f19 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:04.5669 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SgLaWeSiP4a0GPUpjrBsadQ0dzLXwYCcEzzkYAJ8uu3DmDmc1HnOt8iiuleNJWyA X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Userspace will need to know some details about the fwctl interface being used to locate the correct userspace code to communicate with the kernel. Provide a simple device_type enum indicating what the kernel driver is. Allow the device to provide a device specific info struct that contains any additional information that the driver may need to provide to userspace. Signed-off-by: Jason Gunthorpe Reviewed-by: Jonathan Cameron --- drivers/fwctl/main.c | 51 ++++++++++++++++++++++++++++++++++++++ include/linux/fwctl.h | 12 +++++++++ include/uapi/fwctl/fwctl.h | 32 ++++++++++++++++++++++++ 3 files changed, 95 insertions(+) diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c index f2e30ffc1e0cb5..b281ccc52b4e57 100644 --- a/drivers/fwctl/main.c +++ b/drivers/fwctl/main.c @@ -25,8 +25,58 @@ struct fwctl_ucmd { u32 user_size; }; +static int ucmd_respond(struct fwctl_ucmd *ucmd, size_t cmd_len) +{ + if (copy_to_user(ucmd->ubuffer, ucmd->cmd, + min_t(size_t, ucmd->user_size, cmd_len))) + return -EFAULT; + return 0; +} + +static int copy_to_user_zero_pad(void __user *to, const void *from, + size_t from_len, size_t user_len) +{ + size_t copy_len; + + copy_len = min(from_len, user_len); + if (copy_to_user(to, from, copy_len)) + return -EFAULT; + if (copy_len < user_len) { + if (clear_user(to + copy_len, user_len - copy_len)) + return -EFAULT; + } + return 0; +} + +static int fwctl_cmd_info(struct fwctl_ucmd *ucmd) +{ + struct fwctl_device *fwctl = ucmd->uctx->fwctl; + struct fwctl_info *cmd = ucmd->cmd; + size_t driver_info_len = 0; + + if (cmd->flags) + return -EOPNOTSUPP; + + if (cmd->device_data_len) { + void *driver_info __free(kfree) = + fwctl->ops->info(ucmd->uctx, &driver_info_len); + if (IS_ERR(driver_info)) + return PTR_ERR(driver_info); + + if (copy_to_user_zero_pad(u64_to_user_ptr(cmd->out_device_data), + driver_info, driver_info_len, + cmd->device_data_len)) + return -EFAULT; + } + + cmd->out_device_type = fwctl->ops->device_type; + cmd->device_data_len = driver_info_len; + return ucmd_respond(ucmd, sizeof(*cmd)); +} + /* On stack memory for the ioctl structs */ union ucmd_buffer { + struct fwctl_info info; }; struct fwctl_ioctl_op { @@ -46,6 +96,7 @@ struct fwctl_ioctl_op { .execute = _fn, \ } static const struct fwctl_ioctl_op fwctl_ioctl_ops[] = { + IOCTL_OP(FWCTL_INFO, fwctl_cmd_info, struct fwctl_info, out_device_data), }; static long fwctl_fops_ioctl(struct file *filp, unsigned int cmd, diff --git a/include/linux/fwctl.h b/include/linux/fwctl.h index ca4245825e91bf..6b596931a55169 100644 --- a/include/linux/fwctl.h +++ b/include/linux/fwctl.h @@ -7,6 +7,7 @@ #include #include #include +#include struct fwctl_device; struct fwctl_uctx; @@ -19,6 +20,10 @@ struct fwctl_uctx; * it will block device hot unplug and module unloading. */ struct fwctl_ops { + /** + * @device_type: The drivers assigned device_type number. This is uABI. + */ + enum fwctl_device_type device_type; /** * @uctx_size: The size of the fwctl_uctx struct to allocate. The first * bytes of this memory will be a fwctl_uctx. The driver can use the @@ -35,6 +40,13 @@ struct fwctl_ops { * is closed. */ void (*close_uctx)(struct fwctl_uctx *uctx); + /** + * @info: Implement FWCTL_INFO. Return a kmalloc() memory that is copied + * to out_device_data. On input length indicates the size of the user + * buffer on output it indicates the size of the memory. The driver can + * ignore length on input, the core code will handle everything. + */ + void *(*info)(struct fwctl_uctx *uctx, size_t *length); }; /** diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 22fa750d7e8184..39db9f09f8068e 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -4,6 +4,9 @@ #ifndef _UAPI_FWCTL_H #define _UAPI_FWCTL_H +#include +#include + #define FWCTL_TYPE 0x9A /** @@ -33,6 +36,35 @@ */ enum { FWCTL_CMD_BASE = 0, + FWCTL_CMD_INFO = 0, + FWCTL_CMD_RPC = 1, }; +enum fwctl_device_type { + FWCTL_DEVICE_TYPE_ERROR = 0, +}; + +/** + * struct fwctl_info - ioctl(FWCTL_INFO) + * @size: sizeof(struct fwctl_info) + * @flags: Must be 0 + * @out_device_type: Returns the type of the device from enum fwctl_device_type + * @device_data_len: On input the length of the out_device_data memory. On + * output the size of the kernel's device_data which may be larger or + * smaller than the input. Maybe 0 on input. + * @out_device_data: Pointer to a memory of device_data_len bytes. Kernel will + * fill the entire memory, zeroing as required. + * + * Returns basic information about this fwctl instance, particularly what driver + * is being used to define the device_data format. + */ +struct fwctl_info { + __u32 size; + __u32 flags; + __u32 out_device_type; + __u32 device_data_len; + __aligned_u64 out_device_data; +}; +#define FWCTL_INFO _IO(FWCTL_TYPE, FWCTL_CMD_INFO) + #endif From patchwork Wed Aug 21 18:10:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771948 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2064.outbound.protection.outlook.com [40.107.102.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E48931494D1; Wed, 21 Aug 2024 18:11:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263870; cv=fail; b=IfYZFbyXnjB+efOOfcvVyLVbp3vY2SPopXFj/x39xtXeYFcOGZ0cqLhZ8xOrdLY9a6fnUygUs+f4WjafM7wJLfFvp8MyNtE8EWP98ayyQvcimNxPQb1FfWrpkjW8zOvMuaps63f7WIG5XZ8MDm6P3sC9jrheEPWDAYhkKYLp8HA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263870; c=relaxed/simple; bh=e7gaNTLa62kj7XljTeVlxiPtsm7WHe3Q6ZKjl8xPqWQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cu2sAMW/zHdanEhoMmyNgar2wy144cAntKW89j84AjA2VU+hhpCMlvqdj0M7AvtymzuQHpGW+RXxRDp+VdOiukuKhP5X2viI8BQ6olFSLJfuggjMx7vsxpMRVbZc9SFUvVBBwcHL89e9GJ/oRQjDPXIsOLE1t+RA1Wzea7GDYlQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=FtR8rIPQ; arc=fail smtp.client-ip=40.107.102.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="FtR8rIPQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KbHtYDYtwAve3QpB7uoCbHxQkZs3okmi1BURSLDfw0LUa9GXJvGmC3jSrvDN1fM7J722d8Wo7jCPtnPovlqANSj5LUvfALn2dEYen6VZelyL0ge0lxfUrmVmN6Fq017b6TBGN16cnVxuB8g9HO1XhAJHHwpKyQEtco+J/sji3y628jK1IviJnwoo5VA/Ag1Kfei572J97wBbNtthz87WePEyOc64y4xbxxQCsVnJaf0miixVQBePaM1PNHJP0ki2IBbY2O5dQgbelGrwUKKvMdlmM4foOSLj6a8NKr3OW45+xcX3G/TokMg/5EtvOptNuzoEAtKoDBVmYKlLaxfuiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DkDEv1ZGGlKymQP0+EBPxx3d2ZZyHnwTVxd7ws1J78I=; b=KqxjhjmvPGmfKVDJIeLPKTTKY1SE9vsRepdTDm3duj/MxjHrlfk67OPNzlRIq/nr8bTiE2uNF9MBO0AB9Vidvt1uOzoI+mV0a7Wc1CU/s0bmJqgxA1zX58TxokmchWLng1EfXFtfPsxV62EACoJlQSJ5DmZ0ILWRdDRsEL8uVVDJCOxKMhP89jWYq6aHQudNBk/uE9tW96Kl3mOzp2LmxW0XTzhj9hrDAqSo7gRgcMJj31/gJroe95U+2/9yEq+UhXoV2ySK2GaUWMLKCgu0xTA5/qvA9DVxmyS1+fYKA/ooPv9KXD602mtZ8L1DPGkIM6oI8ttoLzuS2EQRxNxsKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DkDEv1ZGGlKymQP0+EBPxx3d2ZZyHnwTVxd7ws1J78I=; b=FtR8rIPQ3vBzDoPc3YUssnt6cEup976GO6cHnvLb2mt6sfh4fnzbCb0GKUEf58mgbSxJdzZzeIS0QPAFhMWys7zOIFd8l/Mqe7fn/4yb19zW0F+a7Ng6vba1SRjCpwalFskAyPJ++QG++Z5klXClnpXjwec3Ddq65uPlGxTAQOaBVEj+gjdQGLN9cb+oV2FbayGTnwIoxfZ4o2z65ZvRnWve/tUt8blZ1d7A/NgKIrs/fr+79xXUNuAfM7mJYiShLuXeU8w9iuubrFdVfxY63b9oao9+gVceLbyIY62FVd+xbZU7riarKhICJPDo///OxMYgkLG9oimpCnUE76Qvng== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:03 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:03 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 04/10] taint: Add TAINT_FWCTL Date: Wed, 21 Aug 2024 15:10:56 -0300 Message-ID: <4-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN8PR04CA0049.namprd04.prod.outlook.com (2603:10b6:408:d4::23) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: aa405f51-4152-4040-523c-08dcc20c9e3b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: Mk6u7f1WbONNNfWrRTyttB6Oboe76jiNOxCR5wy49M7LOVgDT5MiGUkko2NilTNuLlHY3KVMHcgoyKpC0W59B5klRhVke4juOXXRKExNxgJyx5PRT0iajuZNtxGW4KLgToDfKbWXK+fxGzwjKFPwHeqyBpTDKqpqP3RE/ebe9rDEvLwlmu78mSfHXgwSzSonrs/Hv44f1D9D5aJLev79P9LOTwS7IDCtUpCwrf6c8JC7krq5MquCD4zxOOqmw1fsd6N5HaGT4KwqRk2R+jjzfEJb0ecF7XBgsq39UmngMftKe52YcK9ZqsV+/mEYMfRObd7KCWa7iI65lpTixQjmwK3k1PmXaQUK/uk4R2RzdGdwnJ6z3kwzNHyZoYSvqGqHgXZtKSbfsUYnpAugZaVzanZPyHghfa+p3m6wAsZtd3R8Mlx7a3FkjlYpdsbd/t5HhiUt2ccBCqO+PAaA/syLmXn+bDuvwCHWB+PVke9dQhaKXJq7M44q1f5pp67fRNzYwNovwvJlyX/b1kyC27o5KqySx8x7AYfjBrIT9BqHS1o3h6Yhy7mlbxItBKQMsOO71qa5MQqvq4oNTLApCKWuf6eBsDeIUW8VSjNPBBElc+B36N9vP6V6LUj89CCN+0BlaTWTmg85BoEqSGMcgRqC6bJ9F++INpCq4sRcIWcq1ZNo1KKdJH86x5svsloJBDoT+qtJul9P6Z37De8+9aIgXQlE2MmCivjOH56n4Ju4bt30DFQbsKqpq6qCZ8xVoPBtgnJJNuislifQLdkLTTGIE/NjVuW+yTYs4yF8auI8QM6e7xB588Gkjy4wF3aDz31sJXnFsoM6h3dHBPRV888AXHAErl7Yi0SCaRNw/JckqolgyRtjHYK/WjBT5VZs3blPWj39cNF1UU0yOlCfXs3Bm+xe9DoGCyVmZ13r+mfaiJ/eA9yTJnVY8FSYrD0rg+4kOpexU7cKPfkhpCXvmvmSlYyiottFMB+BR5R+YIvfjCtupFau/4BlM4JyeF140wzGncYT9lox3ufAZma2jqEKVIhlP1LYOD4r2S8BB72AU2IRoqE/hEiCk4eaZVqYkyvzHm3xvcF8ULsQEs+ykbXQ2URFYm6c+P3HbZ1wfyA7J3gPLIN7ifMahUU1eeaMgJ1+b7b4RrQ6MIfH85p3aiGyD54QU3u4jyvkrB0AKW4KryePnablewThWSpbiRaBSkgFpOaqg9cQiz0bz/mesixFC5PHD92ojAxyhudwWZEK/ZuPIe2eR6j4v5xc34QzH0TCrGRhfZOFJHsOgyWWme8zNFqiUf713n0HAqTehXVZfGxICw8WXQvXMXwvGlrmOwpmc3LM75NUlbrnesNXpQb+HQ0xefDCxC0l3BoFFOzvCzc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: y1GtjNDmPlEixHs8Pi/Xm0Kou3XZskWH7JWWLDf8/81oEeLy6WvMii41V8W8tWD0XPNaYlVq55J3q95OCrGKsMoQEEbNilFY0s+tXO/Cr7S4F95/gU6I45NjPXvM5cw7cc9+jT771ljAMZkkItbM1k8AViAeeiWaWUO22IrQVjlc+paiC7IeW0y7qrAAfLQsMiCXwhNAV9kHR+JOPTRLqxnTy6JGWmc7NuHAAk57xk8HB+7wFSbMoxCf5Gz/YZfv486NKxOOCTseb5Pd4fY5dtybfuOLtJDRF12RSlchAlA0cXErdD+XKH3l5YPsQQ+tG7VpUZNTnshGkNS+kjOcCkGasXsgslflNxL+jPm8HFcH2i4lge5om24EuYD9hRwOXreNiGPAx6pH3zca8+LE07Y9DOFzN0YA98FT2D3DokjaZsc6Quc0/QT48IUAcRMSMk1sNiP6Kkl3IGJOxzIkKy7Bykf3SkN7hn7xQJZDK3hpbIu5VVJbATvaskrapAd9K6LhS/nIU6BdtBepdu5HXCaF+MTQ78+uPhHmFuDAgWcD6d95doUYk8xnqmO1MJEAnth0v6g4/b4fck2SPdHoIXKqGM4ght+PFjigVlTtXPmYxaq3SBdJ98U+zLWH7W8V8Mosv0YYM3jXl3OcWYoUHrB+y/LQnnuOL7biBFtzbx56OrQmLQOy3gYNfEnCeY3MvnzcfPaF7QnQQLzDoxdn5/40/n3iu2kqlJdJiMPXaHv99jiUZra1lqi7ap89F6xwz/9IvIfAIiNdnthilqA/UtzoBxQGXoAK0Tosgq3DetldqBbbF6jXnRmMD6iYPSi9fjiRHkKIcfBGCDgrypJkV6ehp61gQVFn2B/Xt+nQgwZvAewxNOQPGj6rqp9YMR8f1JFLYYmNE9cvrVynhuUj1jCi6WjDTZaardqIBIu/LGtW19E28p4I/6BUzdN8UszgOJUmgd3HFA71zFi1GP4gRnytDiC3EGFpcZc88tUdioXOGb0Oj+L70k91K5O6GYUd+tBZ7YEjhDBY6GaUA8bIVkYjrSA4N9JT4v7duaW3sMrufcUJjnqdbphu89x5JpEVweS0KiFn9VKl3CFp3EQOjJtKlSllh4uc5/lAp+JwOMw5395u+b0sa8AprQd10OAYGGa7IzYGp6uwZDOFoaO/VNTH1iYQV4BACiob6VXQxaBgYKfwaKaCkpuyiMmgOBViCmmjVSTZniNsVlo79M361AncDKrnpbWVpaQCfCKsPzIXuxbLwGs9iCJyKAAYMURHz4nL6KfTmgmaTi8EFc2eLp/AG0inMnBRwmbTRchO+MFNZJO9ofg/M9mh4JRVYhCuA2i3HhjX5lPB6OVw4FrjDER58crkpPPNHmf8oiIE4Nmc4Ekwuaa9hx8csYHrZjqrN3vz9booKM3nH51njMx3mDSGA8X9wXh8ik8aBS6mrRdq3VkxtOhLQOIF/3XCkPA2XjEaHxmk5so2bYKGZFOspz36Zci/EbmGHZRGxUqBGB4F8qkK2WgMe37cLJd/sbnRmdGoXmkUWq3Sb2xptm05mf6z/lRLIbrVcpCWCJRRDuOyvbUWf3JviDhi1vGJrAgb X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa405f51-4152-4040-523c-08dcc20c9e3b X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.1086 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: n5cj59U7Ygm/Sy6aqXCHT/lDYvTRoGAriuTJgeGITQgHlf+/7izEQZntb9nscReG X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Requesting a fwctl scope of access that includes mutating device debug data will cause the kernel to be tainted. Changing the device operation through things in the debug scope may cause the device to malfunction in undefined ways. This should be reflected in the TAINT flags to help any debuggers understand that something has been done. Signed-off-by: Jason Gunthorpe --- Documentation/admin-guide/tainted-kernels.rst | 5 +++++ include/linux/panic.h | 3 ++- kernel/panic.c | 1 + tools/debugging/kernel-chktaint | 8 ++++++++ 4 files changed, 16 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/tainted-kernels.rst b/Documentation/admin-guide/tainted-kernels.rst index f92551539e8a66..f91a54966a9719 100644 --- a/Documentation/admin-guide/tainted-kernels.rst +++ b/Documentation/admin-guide/tainted-kernels.rst @@ -101,6 +101,7 @@ Bit Log Number Reason that got the kernel tainted 16 _/X 65536 auxiliary taint, defined for and used by distros 17 _/T 131072 kernel was built with the struct randomization plugin 18 _/N 262144 an in-kernel test has been run + 19 _/J 524288 userspace used a mutating debug operation in fwctl === === ====== ======================================================== Note: The character ``_`` is representing a blank in this table to make reading @@ -182,3 +183,7 @@ More detailed explanation for tainting produce extremely unusual kernel structure layouts (even performance pathological ones), which is important to know when debugging. Set at build time. + + 18) ``J`` if userpace opened /dev/fwctl/* and performed a FWTCL_RPC_DEBUG_WRITE + to use the devices debugging features. Device debugging features could + cause the device to malfunction in undefined ways. diff --git a/include/linux/panic.h b/include/linux/panic.h index 3130e0b5116b03..bf4f276be661b7 100644 --- a/include/linux/panic.h +++ b/include/linux/panic.h @@ -73,7 +73,8 @@ static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout) #define TAINT_AUX 16 #define TAINT_RANDSTRUCT 17 #define TAINT_TEST 18 -#define TAINT_FLAGS_COUNT 19 +#define TAINT_FWCTL 19 +#define TAINT_FLAGS_COUNT 20 #define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1) struct taint_flag { diff --git a/kernel/panic.c b/kernel/panic.c index f861bedc1925e7..997b18c7455cd4 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -502,6 +502,7 @@ const struct taint_flag taint_flags[TAINT_FLAGS_COUNT] = { TAINT_FLAG(AUX, 'X', ' ', true), TAINT_FLAG(RANDSTRUCT, 'T', ' ', true), TAINT_FLAG(TEST, 'N', ' ', true), + TAINT_FLAG(FWCTL, 'J', ' ', true), }; #undef TAINT_FLAG diff --git a/tools/debugging/kernel-chktaint b/tools/debugging/kernel-chktaint index 279be06332be99..e7da0909d09707 100755 --- a/tools/debugging/kernel-chktaint +++ b/tools/debugging/kernel-chktaint @@ -204,6 +204,14 @@ else echo " * an in-kernel test (such as a KUnit test) has been run (#18)" fi +T=`expr $T / 2` +if [ `expr $T % 2` -eq 0 ]; then + addout " " +else + addout "J" + echo " * fwctl's mutating debug interface was used (#19)" +fi + echo "For a more detailed explanation of the various taint flags see" echo " Documentation/admin-guide/tainted-kernels.rst in the Linux kernel sources" echo " or https://kernel.org/doc/html/latest/admin-guide/tainted-kernels.html" From patchwork Wed Aug 21 18:10:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771958 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 299241B530C; Wed, 21 Aug 2024 18:11:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263883; cv=fail; b=ppq0RTyuR+8D2l06623joGcuNDZXY180bnMRZxf/8AnSq5vV6VOGSlIqlkuakgWEVs+/N4XbuDfM0/2higfLoodNtXm3GfOkJGRnvd6WOwwyvJnOQVVRPGemQv7wBZE03lB0z1VhDBAQ1HFrEpSZmr3y6RmpDndD7swsFtfsJfo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263883; c=relaxed/simple; bh=QyiUVwjJAnL6Q7xr01o1e0IPrQXuUv/I0ubyuoQUkFQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Wjh5s8Rm+xDsXkAfOaT8qICzBeeJDxeFk/DWx+SpjoiRo7PGRcMZhD/lyijHdYFuWXmJZuQrYWW5Nxf4Uizxef7lew3MNgY9G4HFlN0tN7GONo6CyaTkTAYqHcKEFnVnJSH3NPx7l2QIkV9iN1HjlOOWrn8xS5nN9B0LMuz9+rM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Yev/ma9Z; arc=fail smtp.client-ip=40.107.101.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Yev/ma9Z" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gOAz2bQ7jwysK12loXn7bC5Bu+a8KIfVodPfn6kJqYS71NqTD/ulYzV4R+0w4vgMHnwNOVB704iWgD7A5DX67NQwBva/SwTH4kwvTu6WysTkhAsGzEpZoBu0vO8fZ7osNlhpFk84zDc4nC556kM7HI5cBloS1OV6VxrH9OLihsPO/IObtup/OhDonjzTdSweyte3wzdCtpsAyeDR2zWSAXQ6b9GKV5l4Dw5TXM6uBVn2VHuezgjrhW/MCVAZidBoYjn5Jxerx1Uola6aMJfTKiSbGzoZknzmKwsll63Ft2Wb7Hm0a7NqgvClDNfRx93Qlmo9uWiC4KYMzxuwDAHa3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ccu1BePe4n7FR5e0XtDFHuAqyUiEnRDk1VnpHCSlUlY=; b=QPVYm8H1vcj4nxwl7gTGohHqXqWa1VzBDSl6YJAi6BrgNmfslp0pJRZEkR1f0I+QKQRb26IInZAbFYpzxFMPUO44LM1IZdwgCllE/qF28AL0g3Ajq1f9kqQyZ+yIJXyDSjPV+2+uht2PTfPPh6vhm+ZhoUN+zEyWY4panQ7Kc4296+alYniLWRP3qw4h51EV7vNCyT4imzSKYFAil3Axicz9OK9Tc2tWr9+X8UM9ewLCQiyNHrfAGkC64QZRu3h5k8uZizpBVcTLnKb3pAeczEF5uhdcD+RNpYZI1HpgyB14KN/gd5aGgh2HN23uw0RLQZIlCKtsxJzaH5SOHrpH3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ccu1BePe4n7FR5e0XtDFHuAqyUiEnRDk1VnpHCSlUlY=; b=Yev/ma9ZnKHjO10vJ/LbwFfoiegoMuTE4Qh/IDSh0FmLPp6MsAVFcZ3prwf70D7d9+uQWX34qBUyNbWYLB6BhvA9Ah6SZD85iDFjFAVclu+aLfMRnnXbYr4iLqIbNHPD3Ni88FlrA3B8STdpNyFJayTNuSmIEBrFwtgRgkhFrB/Ct350Ktm/7yYnHYT+/uQCAQURMK5Pl5qxuKqTsoqiLo2j7AbeObXYgmgRyAJEcuvlwedOUZQAM1Qvv06Y3WnQ5Kyc+LrhlNBgNGNzncqGI6r9aE7KGw+R4zGhBauVwzBnHq9AWGUoaa4lWGojZdDuEgVkTfgONMAQpz/MXxdEsw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:09 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:09 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 05/10] fwctl: FWCTL_RPC to execute a Remote Procedure Call to device firmware Date: Wed, 21 Aug 2024 15:10:57 -0300 Message-ID: <5-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0039.namprd03.prod.outlook.com (2603:10b6:408:fb::14) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: b831aaf7-b8b9-4ab9-04be-08dcc20c9f2d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: WVLYkACETtAmLGokE9a5PMZ9dsfAsgQRqtKDBbJy2vDpJ9lqDjzTT/MTMUZmvcEalFLvwRGnJOko+8q8zrvWAlUqJaNgqe9Uoa3xhwQRF3gKAG98fIG5TkNFRJez7V9JXymMY4rils3A3860LKnvOdpltj1PL0GOUI4ItTypOKaNMtSa0MlE9ongEqU8u18bYtzmvoKUIEw0PGAVXy4MEF2A1YhEFUEjPmISz2hrgPth6XWkvDwa5+lrT8wnjip1Mh3bfnGTsi8VnsnjD0C8SLrNr/Ijjiageq71OhzFCmxsG0Uj+l+6TuneplWzKMIqAuuUqWOUGWo5/B7BsPsp4WZGiY54m9fukOncgVrNAaaS8mpS95zuljHUXfQPsYJcCqrfKefmKkg1qQu133dByJAxmI6h763aeYUSVj4MFKtuCubDKXbFhGDtwXXlJ3suVXL9Rkf2TuEF5hHPSrmb5xobRCNuFNbOeL8oPN4cIhd8PnyIAc3nuC9sYs6HyREBYmkefo0lrgao0VzDpM3JfLsgbLAWJN+FHlvaYFjCdFLnPuzkRmDf/P8QPaRG+213NgUiZ/hZklKAP7o6dz2mA5+kpXi8mck11GFQ4xyhePZDegrW/tCWQHSIXFWmlw+cAErDjFYLlGwuJN+p6ON2RwnmtxWrj4m1YxY1XdeQQwMOqxyS1VWpn8FueY4w27dP5LhHXdZNg6L0xBknio4Et2GNqUDrCtrfHhBfwbOAYCzS+bPCfmgacbBRFdsC+5LLYpN0m9i+2bGmvCWW/kIM+NncaqajFMMribMDyAYgwD6gLIGmMjqZKOXcUE/aiBmQZYbgm8YXl/rtvy8E9cSG1mIIVVjaZyER+4vk9MAP5QbW9g+p6ArKib9/sktve4xK0j5bt4/XH2cseXPfc3ChyTTpfjjUeI/1q1LkXnisB+MKNkkY30R//pyikxNE8PNkwl2vZESVLI2FxjaJRd3vlPi8+h9NUfpPe2RlXp/n6LAo9oOkaXe2i/LyvmpIIc2FM5IykVBOr2pOlI0MZFbSL3h+ZR2AQWfxGyF/oo8uzjrJ08FWiyw6iBsZ1y4cyZugfzZdNuZ+K7VYZySHZHFwqCgr1sy/QTt5Lg2F0hVL3PPw9GA3NZdAm9fyk99+otaompD93QDEeWgxEd/nVeqk3v1d9s16rnE1tZQWK4jPmWgNMCXwKG5fx228CPAGRj7ycft7K1Q21RYeSiaT/IRLIFG7PKxRRzR0APpP12N5s+LQnRdyZXMcKokkIFgbp8z4Vj/O0Ou34NdI1u6sQQaj0ic3YHymHCAXXKjUMudGxLD9i7//0mkREwW4B9/ao5zSiP8ldTarI5i2T5oYgluysQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AIA6sIS4/vXbLqDM889fbpa4B4wx5fX7BpJ3xW5JCtGjqbQwxn7SRVlaTimTig54it6ctCA/d94bvj2Egp9ltC1rBoZyX7Si+g7LVrvFlWe77dCbei7ePRrgw8ekTiQa9TmQqeN37flGfdtwUiE5eQDf50WOOrnGU04F6nSAoSruBzPyI6fmb4uoAnM9DUov+Rf1j7GwkXK7vm5nh8O7L2s5JYh1f+Z4r27tBCjyoayXuyY0jH7d5S4SC+Ru78XQWhWhTr+dwlhwQp5aXNTYGv1GQmdGSSq1r5zkEv+pxNs3AHP1R1+cd/sM5A8xfdBcxA93LuA/MlOJ3icBTXgNFtgD9KYS9/xwBwBu4le4RRKlHkV4p9gUdA6Xjb5P4cnyGCJn6t74Tsx4joBQQf2mWFoitzdf9fTnj2pc6eM9dWGI868R2PZ5bj5Kh3TKe6Maaou6ZxysrEcWRYaTSWqZ5k718dKcGOWlI0Cxv/LLR1MCrkXn0W1q3Ug43T5NZZB8cy998JUL4gzLghsLbvPv2ro736e2sny46kK2HugG96zCrGAxlO6In1y0Rpip1dINFTP9NFQUVGOYN+nWCXSJWae4AbB3jGjfSSEupMRvk1sTLC5C6NdgmD3AaCkrf7ZsryL2jPZpNfNdngPJLNhLyeo+bOYrmJKEM1RuK4qZjv9sAZABirGpOCUzJ1fHOngaTKd0W8FBIZ4JIqze7jfg2LS+kRi7D5xJ4a2LBVIQqS4yDRnSi+OgT5YNithqtzIDjxn1Vc282ETxVIZ7BEfUdjl7QIPMzJOGx/qHkHRC7QJsYgcuPaRtOw3heFI5gm1tLmZnFyxGVdvBSCMMhk0llp/12fTPETYZj/zTN5kSGRdFB++4+u4VmJOtTxthgO2zJjGcrCu9Eo4FPhq5u4Y0bZk6KHaCPoQTSaWNToGsDIEW29ftBIvjIL+yU75JAV0OaSzIbmsRvkUItY7tFXnWCDtLEQzOIZR5KS4axRIxfr2AjtcZosaOX7uSR4j3W7iws93A6T9nq6uCgLnR0iMPBXX5wldr1H50DvPCIdyDsKkibh7XtRumeCRqILPwFfMJUmZwofQGwZl1c2a/7hDF6+TOhu+yT96qPRIi6d7vKH/TkruKAPnAzTB6h9V4Dyrq7uDQ0PCAMM87A6S8ittA19DD9riFfG3wxAxxgdsHvG05qKLUG/Lk8xNmq3UkfOZZtEb8hP1DJIXR1sn8YkLcbeEjeeQ23XB4as50huxUF0dG6kE92D2OiDp8zvb4YLmlYyag3CpEI5yyP3TG8/nYdEZm1V7oplqYtx2rY4i2uu32QRLZqE+pp+w774vSKAQ0luGxpZHmMM26WAIoZlyE5UOa7B/yoIQz05WFpfNjzkglwu7zo3TD85r+006Qpu5c2B4nsdpqTSX6TKIa5AtEXrCBAn7sddL/yYsrvBeze67EQ+0AV7pimoDo19H2s+2B2GQT6izjFGIMYWhnqonzEHo6WeXDDoHVWWxByYWa8z3PdIizPOzd5k2NBXKjTWkqubJRMHEBK18ZIaEDBwMEARpPyUv2EJMiw79Z67VP0WSzV5CiZSuRRuwpKI+TkNpC X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b831aaf7-b8b9-4ab9-04be-08dcc20c9f2d X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:04.6746 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YAHQiuuSHZxQMDYB832QHgYG+F9/fxLS4mZO+e38i/Kez2h7yj4XCUOpnTyTonR2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Add the FWCTL_RPC ioctl which allows a request/response RPC call to device firmware. Drivers implementing this call must follow the security guidelines under Documentation/userspace-api/fwctl.rst The core code provides some memory management helpers to get the messages copied from and back to userspace. The driver is responsible for allocating the output message memory and delivering the message to the device. Signed-off-by: Jason Gunthorpe Nacked-by: Jakub Kicinski # RFC 3514 Reviewed-by: Jonathan Cameron --- drivers/fwctl/main.c | 60 +++++++++++++++++++++++++++++++++ include/linux/fwctl.h | 8 +++++ include/uapi/fwctl/fwctl.h | 68 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 136 insertions(+) diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c index b281ccc52b4e57..54a7356e586cda 100644 --- a/drivers/fwctl/main.c +++ b/drivers/fwctl/main.c @@ -8,15 +8,18 @@ #include #include #include +#include #include #include enum { FWCTL_MAX_DEVICES = 256, + MAX_RPC_LEN = SZ_2M, }; static dev_t fwctl_dev; static DEFINE_IDA(fwctl_ida); +static unsigned long fwctl_tainted; struct fwctl_ucmd { struct fwctl_uctx *uctx; @@ -74,9 +77,65 @@ static int fwctl_cmd_info(struct fwctl_ucmd *ucmd) return ucmd_respond(ucmd, sizeof(*cmd)); } +static int fwctl_cmd_rpc(struct fwctl_ucmd *ucmd) +{ + struct fwctl_device *fwctl = ucmd->uctx->fwctl; + struct fwctl_rpc *cmd = ucmd->cmd; + size_t out_len; + + if (cmd->in_len > MAX_RPC_LEN || cmd->out_len > MAX_RPC_LEN) + return -EMSGSIZE; + + switch (cmd->scope) { + case FWCTL_RPC_CONFIGURATION: + case FWCTL_RPC_DEBUG_READ_ONLY: + break; + + case FWCTL_RPC_DEBUG_WRITE_FULL: + if (!capable(CAP_SYS_RAWIO)) + return -EPERM; + fallthrough; + case FWCTL_RPC_DEBUG_WRITE: + if (!test_and_set_bit(0, &fwctl_tainted)) { + dev_warn( + &fwctl->dev, + "%s(%d): has requested full access to the physical device device", + current->comm, task_pid_nr(current)); + add_taint(TAINT_FWCTL, LOCKDEP_STILL_OK); + } + break; + default: + return -EOPNOTSUPP; + } + + void *inbuf __free(kvfree) = kvzalloc(cmd->in_len, GFP_KERNEL_ACCOUNT); + if (!inbuf) + return -ENOMEM; + if (copy_from_user(inbuf, u64_to_user_ptr(cmd->in), cmd->in_len)) + return -EFAULT; + + out_len = cmd->out_len; + void *outbuf __free(kvfree) = fwctl->ops->fw_rpc( + ucmd->uctx, cmd->scope, inbuf, cmd->in_len, &out_len); + if (IS_ERR(outbuf)) + return PTR_ERR(outbuf); + if (outbuf == inbuf) { + /* The driver can re-use inbuf as outbuf */ + inbuf = NULL; + } + + if (copy_to_user(u64_to_user_ptr(cmd->out), outbuf, + min(cmd->out_len, out_len))) + return -EFAULT; + + cmd->out_len = out_len; + return ucmd_respond(ucmd, sizeof(*cmd)); +} + /* On stack memory for the ioctl structs */ union ucmd_buffer { struct fwctl_info info; + struct fwctl_rpc rpc; }; struct fwctl_ioctl_op { @@ -97,6 +156,7 @@ struct fwctl_ioctl_op { } static const struct fwctl_ioctl_op fwctl_ioctl_ops[] = { IOCTL_OP(FWCTL_INFO, fwctl_cmd_info, struct fwctl_info, out_device_data), + IOCTL_OP(FWCTL_RPC, fwctl_cmd_rpc, struct fwctl_rpc, out), }; static long fwctl_fops_ioctl(struct file *filp, unsigned int cmd, diff --git a/include/linux/fwctl.h b/include/linux/fwctl.h index 6b596931a55169..6eac9497ff1afc 100644 --- a/include/linux/fwctl.h +++ b/include/linux/fwctl.h @@ -47,6 +47,14 @@ struct fwctl_ops { * ignore length on input, the core code will handle everything. */ void *(*info)(struct fwctl_uctx *uctx, size_t *length); + /** + * @fw_rpc: Implement FWCTL_RPC. Deliver rpc_in/in_len to the FW and + * return the response and set out_len. rpc_in can be returned as the + * response pointer. Otherwise the returned pointer is freed with + * kvfree(). + */ + void *(*fw_rpc)(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *rpc_in, size_t in_len, size_t *out_len); }; /** diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 39db9f09f8068e..3af9f9eb9b1878 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -67,4 +67,72 @@ struct fwctl_info { }; #define FWCTL_INFO _IO(FWCTL_TYPE, FWCTL_CMD_INFO) +/** + * enum fwctl_rpc_scope - Scope of access for the RPC + * + * Refer to fwctl.rst for a more detailed discussion of these scopes. + */ +enum fwctl_rpc_scope { + /** + * @FWCTL_RPC_CONFIGURATION: Device configuration access scope + * + * Read/write access to device configuration. When configuration + * is written to the device it remains in a fully supported state. + */ + FWCTL_RPC_CONFIGURATION = 0, + /** + * @FWCTL_RPC_DEBUG_READ_ONLY: Read only access to debug information + * + * Readable debug information. Debug information is compatible with + * kernel lockdown, and does not disclose any sensitive information. For + * instance exposing any encryption secrets from this information is + * forbidden. + */ + FWCTL_RPC_DEBUG_READ_ONLY = 1, + /** + * @FWCTL_RPC_DEBUG_WRITE: Writable access to lockdown compatible debug information + * + * Allows write access to data in the device which may leave a fully + * supported state. This is intended to permit intensive and possibly + * invasive debugging. This scope will taint the kernel. + */ + FWCTL_RPC_DEBUG_WRITE = 2, + /** + * @FWCTL_RPC_DEBUG_WRITE_FULL: Write access to all debug information + * + * Allows read/write access to everything. Requires CAP_SYS_RAW_IO, so + * it is not required to follow lockdown principals. If in doubt + * debugging should be placed in this scope. This scope will taint the + * kernel. + */ + FWCTL_RPC_DEBUG_WRITE_FULL = 3, +}; + +/** + * struct fwctl_rpc - ioctl(FWCTL_RPC) + * @size: sizeof(struct fwctl_rpc) + * @scope: One of enum fwctl_rpc_scope, required scope for the RPC + * @in_len: Length of the in memory + * @out_len: Length of the out memory + * @in: Request message in device specific format + * @out: Response message in device specific format + * + * Deliver a Remote Procedure Call to the device FW and return the response. The + * call's parameters and return are marshaled into linear buffers of memory. Any + * errno indicates that delivery of the RPC to the device failed. Return status + * originating in the device during a successful delivery must be encoded into + * out. + * + * The format of the buffers matches the out_device_type from FWCTL_INFO. + */ +struct fwctl_rpc { + __u32 size; + __u32 scope; + __u32 in_len; + __u32 out_len; + __aligned_u64 in; + __aligned_u64 out; +}; +#define FWCTL_RPC _IO(FWCTL_TYPE, FWCTL_CMD_RPC) + #endif From patchwork Wed Aug 21 18:10:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771949 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2064.outbound.protection.outlook.com [40.107.102.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D16221B3B11; Wed, 21 Aug 2024 18:11:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263872; cv=fail; b=fi+oskbSVKJW6zOBufAqSVhXxxJKMbfrHco04SX2RqzsYBrHEP17nG6g4hX3q4XsW0ZB4b05fQOCNAMcv7s5wdmi4kVS14I/C3QdIzqigJDKYMX8CyoiAA+A7C1xceRZS7CY8+i/YX7gLMupMdlz6WJMZcExEioQQzhLgO3Qz4g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263872; c=relaxed/simple; bh=lOfGvwJieDYzOfBNqDRpCtbY8K3x1tWuP5a/8EmMkSw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=uaaV7RQC2jFLRwh60jMWfKMHOSn0zyQ8m1MQwK87Gyaoro0hWG49iODgShJ3ZA0xEmdOCM9ydRIMfglj56S9tJFl+mUA548JuEVz7aePAlm+5KH6Mv6vAzJ/+F09pkrE1BZIXuww/IKeSp7ha082mW+jxTGfPxHnlGPtN1DdeDw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XbRyQMn9; arc=fail smtp.client-ip=40.107.102.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XbRyQMn9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OdN9X9bUgQhd68q/0al1eZ2N+wlsoipV0VSd8aElawYth4cLPQ6+Yt+zyqYvZ9pfS6udWUTevxrG08vzJZUUNZbFvRHEll97L/0R9LKx93jWC8unAeg/eb/Y/Iu/k8ReefTASEyn4IjWRBNiHakSF/kvuz/D+104ucJVk8WXcZk/2aCPtV+r9dFwzzJcxHL3qpZBBsRZcyXgnYr4fAuW5fGpfNQGkQP6PKPEZkRW33g7QpHAD7JOysJLKSusU6AU3uC16o+Xp6EDsvc/qwsvhDTmWJ8IFSsLQSDh4rgGGltydzXJekY2gRm1gYVRBe1k0FDPYHCf5i2ODi/y6lKS2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rP1MMt0T5gmXxQryyzSTQCqvQLhx2j7YI+S6ZoiQpKM=; b=x69y4bbZJkVdH3BXmlTTY5cJSYJpMJOqdi+oi5ugGdmTmkcosuWhqpaYqZGCDZGuTSsacnsZJRRAsDem28i/lkj2xpfTDygxodGvn+2aYm8UZDBsw4c7/zL6txF7+6GnEWBHBLijreAGL2eL4rOvKY+hcBKpnwYyQA5v8NFobVsaX15XE2bbj9yhItMedSYjle9smGYUMXvrUowl2vzz+TAmd+Pr8F0GIWs1aS1Z1s7U4iNAX/l8J3qPZkvskCkk/gHkoKTictviUjzZQdi481Yw3D8wV0d63iSLEso/JCTO1DMdlaLxofUNI5TObFrmsBwgKRNd7yJ+aEHtuiLi3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rP1MMt0T5gmXxQryyzSTQCqvQLhx2j7YI+S6ZoiQpKM=; b=XbRyQMn9W4oucXzz3HTXPBV/9Jmf1AdYGwXr0rwKbdpqRGx76x/xL/M4CNvIEMvpKPR6GSJzDw6ggCaS1aqHhn32IvUUMxm9nYf4jiJoZ4qJkSPj/GqVKog2gn/xDLN+w7jIXvl3nnDXEWgpSJOQGSJw9hG9hlandsWIgwsYCjLheXyCQDqkCACCElE3QrLvuOdo8qBii6utSQt1A10F5jFaqVDiu8A4EUgjeMn7fk+ucWHh1y5MgBn4Esn2sCt1saMu56z/jfKBoQHN2ZDKZNJm418+vuvtDgcCYz4C8s9VhEECLWoOamZbcx6ExOPnofkdnG/dYyV6sRzftaNHfw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:04 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:03 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 06/10] fwctl: Add documentation Date: Wed, 21 Aug 2024 15:10:58 -0300 Message-ID: <6-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0684.namprd03.prod.outlook.com (2603:10b6:408:10e::29) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: fd39f5b6-b0ce-4d04-ae76-08dcc20c9e31 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?rlBcXrWDjZroxBWrcFoUnHpWotYg/6c?= =?utf-8?q?jrMg0pQ1nJ2HHOEvuPa+HfjZYFwgfQUPaiLWxp/gIP9wQLXerqa1e853dnnqhEMDX?= =?utf-8?q?t47TB1xgWODIWneIBgpnaVx9YIXcmWCvJkzqwL6lHwWKue9viuAy2RqyZg8o5XH9G?= =?utf-8?q?7RX8EO7PdppJWurp/ndpqmL0rX9n8mgHWC6Y1O9sOLM9lqKlprzdVCV+TkkNU4Syu?= =?utf-8?q?9ML8Y8AgsEOCjrAOZeuao+S1uAaApjr7btn2psj3zNKSU7QMUe1KrJIIf4YyXrOpz?= =?utf-8?q?B/HDO+n0t4zBIUmcQ+T7TzXCq5yTBZiByUUN4E7ZQtR5343qEE8sllL+7fnY+yO2O?= =?utf-8?q?orvjJMmPoEIOcumF+8VMkFqmP3BxwVx7NaJAU/wrn8IrNjnZaxI0jMcEThFQgS1+x?= =?utf-8?q?RzjbG24quZ05RUqAd4ihe0qubiHDNoe5H6pef3njyODPiBnFjzlXRyRv0Rm0mFg1U?= =?utf-8?q?8DCBdct82HB+f3v7U9gZMPGnBSy/y8z1k5rcrXDSIn9KFnlGQv4ecxLhl2uz/2IAo?= =?utf-8?q?Tu9trw6yvsViISTBWKRLPfCM1oJxBbf6QrIvjrWvXO4dlGO/Mq32Qdtyqaq9nDFKl?= =?utf-8?q?vsWISDEHIMynGJasaSzqTuQAMVAmPqjAAs+TIpuHSA4Ci2jM1FaKOGwV0pYpM7koQ?= =?utf-8?q?JRMoYiFWxKGKjli+S5F73XktdI8bBfZTJblOBpgKeB9rsEHaYQ18gByaaxEqzgCIp?= =?utf-8?q?hxMCq5z8KIwPOKTZauo6mQ4uGkxD5ibbktRyWo5W2G72e7rcshZz6wg8fF4iucxzT?= =?utf-8?q?5qEbdNU2yIMXyHVmGdTcyDmz1IgxvchVEenV8qpNzb+9YgMgZGabPok3HEs5Ax2DH?= =?utf-8?q?fyJHY70d8hc/eWqW3UYyfns1qwnhIA5wwSrjiKxNwte2S1HOkJ6wRsCzv8oI1raVU?= =?utf-8?q?Eeh+sA/d54zcZjyqM69TfFyG6YVmGq1S5XQ77x+8VelIKcOXCo7KIMqbInyoi4FgH?= =?utf-8?q?WLFeLJTJJfru2Mye9f6X9IbrO3Fzm8z4Vd6WL94XEI7XRagUhRPwlsWX2TqCEVMbv?= =?utf-8?q?cf6wZ/v/bsFnJnTiobTQh7IHkgVBNx7igd5X5wj0cMZi9wj5WsynIMAq9PCP/0mhB?= =?utf-8?q?R8NU9BbNkX4hLhdSpqSHa5JcodTBuk55VF5z754VTh6CzS0iQBLHxaT1YpCHsUBRE?= =?utf-8?q?tPjdYA2un2bbbZnNs/kceEP49agZxaYW0al9yks50naG9bXJF96LbOQWLHxmmw0SD?= =?utf-8?q?GYWoNuol5EvnVfvhcZxQQouiXonU45kZMWw36SErKiI5Cul41sXfJy4AETPe+xCsK?= =?utf-8?q?HBYO4Klmcf2KvYuOMp9LoHv1B42sx5d9iGoEEp5LrWw+QoAlXTrA5wk4=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?1qAHvmSet4YMRpfdxdIiO6GMPISC?= =?utf-8?q?ewsKS8v+gPwlehLLXCY72tP2dBjbIs6Jp6M9Yb9iFZhAjHdtE+rlqoVLvqoLU5tnH?= =?utf-8?q?lVwt412sOjLvXDXUjmK78vWjd37fhhqVj/r4HOZRiuZe3FDmxHJ3hs4eXGmIgOpl5?= =?utf-8?q?ivs6Q6mbA3WJ/ELPDtF3MRwS6Mbn9ij5VS3aNi7qLj5nHW6mKvWuT7mLmvYIFoC4A?= =?utf-8?q?3z4X8WOYJzdqt3xymf4w3X8LRazgJvzWgWRhopCFXSKi8HS4ryNDOyGuRiMmh0Z+S?= =?utf-8?q?TwGPM5i7207FBQ1GOF8cb7GF12aCYCTfcY5Cb0o7Ss/kO+DMRhdZixOkyLW+Awz94?= =?utf-8?q?X47j+ei0cW12F0wC987EnWGQciXgrqPKO1sZv2F8Vnd1s+TTSfdGnH+IH1ILjkb5L?= =?utf-8?q?xu3xAelGplfPUSaLiAVHWWMJso84SsTbWJBrSaPQlrPwgoV859dE4NxR5uM2WUur5?= =?utf-8?q?k5bcU1esdi+l8Ru76pemlDWyQwHQlJATJfqDvQGsE7deyCcM5daBmeI4yNboa1hSa?= =?utf-8?q?UfFgNIsqYDvOtyWBdtm3T9u49mX0Ws3DE5eI/4KnWR1U0DWxfgRux9/loL1DHk06w?= =?utf-8?q?RnsGyBGc2iZXOEyazC/CAv8w5/Y2XOAGPC5bmqnYeb2q/DMYYNRHGryJoi1pcnQxN?= =?utf-8?q?acwop30XmArjQqmRoK4K9W3EuQOEku7eGWh1xGhSkEDlKu7yf9i9M4+Z/mIHRvfgh?= =?utf-8?q?P8gpLISfl2QrgbpRDJ95sQO4YD9VhGQI+PCEKxg3hp0piBIowCetX2153p/2Bx5QK?= =?utf-8?q?z9ikDZ7UOYVqkaY0ENVYAfDJqqy3CUvUyKNHS5pSlyhrdk9bW/1gfzl3h+oafCB6g?= =?utf-8?q?n/txWa8L94D1lt69UDuColtDte9QWH0uO+Vy8UFuxeiNaWiLZunnGIs4T7+/6m3wU?= =?utf-8?q?Hs85ohrZcfgPAHjniet4PTQ5u+yHvmf8WAm93j2lx8hLy7IAGSHQe41aWVQeMEwNh?= =?utf-8?q?WGlrdeEIYGJ+Z8HNwOrmQgpOEeDJfjKkEuuVJtGbAIKsNnyE7kFJXesKT+PCBMmt2?= =?utf-8?q?jSyswAl4VP2ymHb+za37O9bGSOU5KbMjyPghOTYngRqyTTC8w7+e+qM3yrWmLRHtW?= =?utf-8?q?OvANCwOxOKaksIeeOKAhyr5RuSEhklQ7oK1VYL703ClIhg/B1Y6M9wP83A9N4mlMh?= =?utf-8?q?03JkIXn0s6GEGo6xBS+s9+aqOqpScXRy7JHzqnRROrnFum/bA07eFg02m8HLiasME?= =?utf-8?q?sP6/rytRJxjwfqGuHkE4PXctcO6nXg5WU4a/gMiqXCHYDfgmLmtXzRW3ntVTTqcFg?= =?utf-8?q?Bl9rXlreO7ffwIAn2yMKe1HwzvWXtYgNnjK7jp24+JSspSY25k905j/8qCSsVzKKc?= =?utf-8?q?MzFrCondE3pUAUQMBte3gXEOa4Rad8FDZ0YWHfJo4qKLzQh7TLofVVLiM9+ZG5FG3?= =?utf-8?q?UShj2cZ/N+uc1R03OIAv83Q/1yNSUi18k2yQxEdNAC9rrVYtRtt0io5lzuOOIYnWL?= =?utf-8?q?G+Kl3uKMJXSjDPLgFSNIeQixtNy9E35CJXiCemH0SKXe4+ik/Y6F+iycgMeNEv6La?= =?utf-8?q?evJPsO7CrYvP?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd39f5b6-b0ce-4d04-ae76-08dcc20c9e31 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.1222 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: naKeTi1rLtPpbRWABlWxshgchyuYX+CI1+kLFco5jqhEDJ/32s48IjoRl/tEmGRT X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 Document the purpose and rules for the fwctl subsystem. Link in kdocs to the doc tree. Nacked-by: Jakub Kicinski Link: https://lore.kernel.org/r/20240603114250.5325279c@kernel.org Acked-by: Daniel Vetter https://lore.kernel.org/r/ZrHY2Bds7oF7KRGz@phenom.ffwll.local Signed-off-by: Jason Gunthorpe Reviewed-by: Jonathan Cameron --- Documentation/userspace-api/fwctl.rst | 285 ++++++++++++++++++++++++++ Documentation/userspace-api/index.rst | 1 + 2 files changed, 286 insertions(+) create mode 100644 Documentation/userspace-api/fwctl.rst diff --git a/Documentation/userspace-api/fwctl.rst b/Documentation/userspace-api/fwctl.rst new file mode 100644 index 00000000000000..8f3da30ee7c91b --- /dev/null +++ b/Documentation/userspace-api/fwctl.rst @@ -0,0 +1,285 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +fwctl subsystem +=============== + +:Author: Jason Gunthorpe + +Overview +======== + +Modern devices contain extensive amounts of FW, and in many cases, are largely +software-defined pieces of hardware. The evolution of this approach is largely a +reaction to Moore's Law where a chip tape out is now highly expensive, and the +chip design is extremely large. Replacing fixed HW logic with a flexible and +tightly coupled FW/HW combination is an effective risk mitigation against chip +respin. Problems in the HW design can be counteracted in device FW. This is +especially true for devices which present a stable and backwards compatible +interface to the operating system driver (such as NVMe). + +The FW layer in devices has grown to incredible sizes and devices frequently +integrate clusters of fast processors to run it. For example, mlx5 devices have +over 30MB of FW code, and big configurations operate with over 1GB of FW managed +runtime state. + +The availability of such a flexible layer has created quite a variety in the +industry where single pieces of silicon are now configurable software-defined +devices and can operate in substantially different ways depending on the need. +Further, we often see cases where specific sites wish to operate devices in ways +that are highly specialized and require applications that have been tailored to +their unique configuration. + +Further, devices have become multi-functional and integrated to the point they +no longer fit neatly into the kernel's division of subsystems. Modern +multi-functional devices have drivers, such as bnxt/ice/mlx5/pds, that span many +subsystems while sharing the underlying hardware using the auxiliary device +system. + +All together this creates a challenge for the operating system, where devices +have an expansive FW environment that needs robust device-specific debugging +support, and FW-driven functionality that is not well suited to “generic” +interfaces. fwctl seeks to allow access to the full device functionality from +user space in the areas of debuggability, management, and first-boot/nth-boot +provisioning. + +fwctl is aimed at the common device design pattern where the OS and FW +communicate via an RPC message layer constructed with a queue or mailbox scheme. +In this case the driver will typically have some layer to deliver RPC messages +and collect RPC responses from device FW. The in-kernel subsystem drivers that +operate the device for its primary purposes will use these RPCs to build their +drivers, but devices also usually have a set of ancillary RPCs that don't really +fit into any specific subsystem. For example, a HW RAID controller is primarily +operated by the block layer but also comes with a set of RPCs to administer the +construction of drives within the HW RAID. + +In the past when devices were more single function, individual subsystems would +grow different approaches to solving some of these common problems. For instance +monitoring device health, manipulating its FLASH, debugging the FW, +provisioning, all have various unique interfaces across the kernel. + +fwctl's purpose is to define a common set of limited rules, described below, +that allow user space to securely construct and execute RPCs inside device FW. +The rules serve as an agreement between the operating system and FW on how to +correctly design the RPC interface. As a uAPI the subsystem provides a thin +layer of discovery and a generic uAPI to deliver the RPCs and collect the +response. It supports a system of user space libraries and tools which will +use this interface to control the device using the device native protocols. + +Scope of Action +--------------- + +fwctl drivers are strictly restricted to being a way to operate the device FW. +It is not an avenue to access random kernel internals, or other operating system +SW states. + +fwctl instances must operate on a well-defined device function, and the device +should have a well-defined security model for what scope within the physical +device the function is permitted to access. For instance, the most complex PCIe +device today may broadly have several function-level scopes: + + 1. A privileged function with full access to the on-device global state and + configuration + + 2. Multiple hypervisor functions with control over itself and child functions + used with VMs + + 3. Multiple VM functions tightly scoped within the VM + +The device may create a logical parent/child relationship between these scopes. +For instance a child VM's FW may be within the scope of the hypervisor FW. It is +quite common in the VFIO world that the hypervisor environment has a complex +provisioning/profiling/configuration responsibility for the function VFIO +assigns to the VM. + +Further, within the function, devices often have RPC commands that fall within +some general scopes of action (see enum fwctl_rpc_scope): + + 1. Access to function & child configuration, FLASH, etc. that becomes live at a + function reset. Access to function & child runtime configuration that is + transparent or non-disruptive to any driver or VM. + + 2. Read-only access to function debug information that may report on FW objects + in the function & child, including FW objects owned by other kernel + subsystems. + + 3. Write access to function & child debug information strictly compatible with + the principles of kernel lockdown and kernel integrity protection. Triggers + a kernel Taint. + + 4. Full debug device access. Triggers a kernel Taint, requires CAP_SYS_RAWIO. + +User space will provide a scope label on each RPC and the kernel must enforce the +above CAPs and taints based on that scope. A combination of kernel and FW can +enforce that RPCs are placed in the correct scope by user space. + +Denied behavior +--------------- + +There are many things this interface must not allow user space to do (without a +Taint or CAP), broadly derived from the principles of kernel lockdown. Some +examples: + + 1. DMA to/from arbitrary memory, hang the system, compromise FW integrity with + untrusted code, or otherwise compromise device or system security and + integrity. + + 2. Provide an abnormal “back door” to kernel drivers. No manipulation of kernel + objects owned by kernel drivers. + + 3. Directly configure or otherwise control kernel drivers. A subsystem kernel + driver can react to the device configuration at function reset/driver load + time, but otherwise must not be coupled to fwctl. + + 4. Operate the HW in a way that overlaps with the core purpose of another + primary kernel subsystem, such as read/write to LBAs, send/receive of + network packets, or operate an accelerator's data plane. + +fwctl is not a replacement for device direct access subsystems like uacce or +VFIO. + +Operations exposed through fwctl's non-taining interfaces should be fully +sharable with other users of the device. For instance exposing a RPC through +fwctl should never prevent a kernel subsystem from also concurrently using that +same RPC or hardware unit down the road. In such cases fwctl will be less +important than proper kernel subsystems that eventually emerge. Mistakes in this +area resulting in clashes will be resolved in favour of a kernel implementation. + +fwctl User API +============== + +.. kernel-doc:: include/uapi/fwctl/fwctl.h +.. kernel-doc:: include/uapi/fwctl/mlx5.h + +sysfs Class +----------- + +fwctl has a sysfs class (/sys/class/fwctl/fwctlNN/) and character devices +(/dev/fwctl/fwctlNN) with a simple numbered scheme. The character device +operates the iotcl uAPI described above. + +fwctl devices can be related to driver components in other subsystems through +sysfs:: + + $ ls /sys/class/fwctl/fwctl0/device/infiniband/ + ibp0s10f0 + + $ ls /sys/class/infiniband/ibp0s10f0/device/fwctl/ + fwctl0/ + + $ ls /sys/devices/pci0000:00/0000:00:0a.0/fwctl/fwctl0 + dev device power subsystem uevent + +User space Community +-------------------- + +Drawing inspiration from nvme-cli, participating in the kernel side must come +with a user space in a common TBD git tree, at a minimum to usefully operate the +kernel driver. Providing such an implementation is a pre-condition to merging a +kernel driver. + +The goal is to build user space community around some of the shared problems +we all have, and ideally develop some common user space programs with some +starting themes of: + + - Device in-field debugging + + - HW provisioning + + - VFIO child device profiling before VM boot + + - Confidential Compute topics (attestation, secure provisioning) + +that stretch across all subsystems in the kernel. fwupd is a great example of +how an excellent user space experience can emerge out of kernel-side diversity. + +fwctl Kernel API +================ + +.. kernel-doc:: drivers/fwctl/main.c + :export: +.. kernel-doc:: include/linux/fwctl.h + +fwctl Driver design +------------------- + +In many cases a fwctl driver is going to be part of a larger cross-subsystem +device possibly using the auxiliary_device mechanism. In that case several +subsystems are going to be sharing the same device and FW interface layer so the +device design must already provide for isolation and cooperation between kernel +subsystems. fwctl should fit into that same model. + +Part of the driver should include a description of how its scope restrictions +and security model work. The driver and FW together must ensure that RPCs +provided by user space are mapped to the appropriate scope. If the validation is +done in the driver then the validation can read a 'command effects' report from +the device, or hardwire the enforcement. If the validation is done in the FW, +then the driver should pass the fwctl_rpc_scope to the FW along with the command. + +The driver and FW must cooperate to ensure that either fwctl cannot allocate +any FW resources, or any resources it does allocate are freed on FD closure. A +driver primarily constructed around FW RPCs may find that its core PCI function +and RPC layer belongs under fwctl with auxiliary devices connecting to other +subsystems. + +Each device type must be mindful of Linux's philosophy for stable ABI. The FW +RPC interface does not have to meet a strictly stable ABI, but it does need to +meet an expectation that userspace tools that are deployed and in significant +use don't needlessly break. FW upgrade and kernel upgrade should keep widely +deployed tooling working. + +Development and debugging focused RPCs under more permissive scopes can have +less stablitiy if the tools using them are only run under exceptional +circumstances and not for every day use of the device. Debugging tools may even +require exact version matching as they may require something similar to DWARF +debug information from the FW binary. + +Security Response +================= + +The kernel remains the gatekeeper for this interface. If violations of the +scopes, security or isolation principles are found, we have options to let +devices fix them with a FW update, push a kernel patch to parse and block RPC +commands or push a kernel patch to block entire firmware versions/devices. + +While the kernel can always directly parse and restrict RPCs, it is expected +that the existing kernel pattern of allowing drivers to delegate validation to +FW to be a useful design. + +Existing Similar Examples +========================= + +The approach described in this document is not a new idea. Direct, or near +direct device access has been offered by the kernel in different areas for +decades. With more devices wanting to follow this design pattern it is becoming +clear that it is not entirely well understood and, more importantly, the +security considerations are not well defined or agreed upon. + +Some examples: + + - HW RAID controllers. This includes RPCs to do things like compose drives into + a RAID volume, configure RAID parameters, monitor the HW and more. + + - Baseboard managers. RPCs for configuring settings in the device and more + + - NVMe vendor command capsules. nvme-cli provides access to some monitoring + functions that different products have defined, but more exist. + + - CXL also has a NVMe-like vendor command system. + + - DRM allows user space drivers to send commands to the device via kernel + mediation + + - RDMA allows user space drivers to directly push commands to the device + without kernel involvement + + - Various “raw” APIs, raw HID (SDL2), raw USB, NVMe Generic Interface, etc. + +The first 4 are examples of areas that fwctl intends to cover. The latter three +are examples of denied behavior as they fully overlap with the primary purpose +of a kernel subsystem. + +Some key lessons learned from these past efforts are the importance of having a +common user space project to use as a pre-condition for obtaining a kernel +driver. Developing good community around useful software in user space is key to +getting companies to fund participation to enable their products. diff --git a/Documentation/userspace-api/index.rst b/Documentation/userspace-api/index.rst index 274cc7546efc2a..2bc43a65807486 100644 --- a/Documentation/userspace-api/index.rst +++ b/Documentation/userspace-api/index.rst @@ -44,6 +44,7 @@ Devices and I/O accelerators/ocxl dma-buf-alloc-exchange + fwctl gpio/index iommufd media/index From patchwork Wed Aug 21 18:10:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771953 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 321C21B3B0C; Wed, 21 Aug 2024 18:11:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263878; cv=fail; b=UegJ9wlkAPvmy7dlGth1717ixH/NF8Minijq+IVw/d2SB89+htXOGZ2t2ksnhzdTnYN8Qlmy6RQWClr+3uxjh/2BLLeKVv2P1RwzYIeLIVeA0PUUgsT6PdWwewOf92UAdyMdvg8AIvFWKqC+teF4DmjQodsRr4KCLv32HuOCV+0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263878; c=relaxed/simple; bh=kLUzLkH95R8u6jiAYTNwyFM/hZ+OrydQJGwGoWGbzIk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=S5YWdTuBfBtxFGlsRsjjmM7fmYvxWUfJIyC6q/ohyk6fyJInhq4su0XJKcNPHVg014CuQzFE7WVHCSIJxfwOUvuLNiytp5NxGR2JzfBM42xGB+dJ21i3Audb++ooKsFN4QOl/1pOwP1/MpBdpK8M3Cxs51VCLd2kS0G5X/lEj94= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=QfNNBl4q; arc=fail smtp.client-ip=40.107.101.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="QfNNBl4q" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=k7pVkTdDnaAE0L+am8ZfhsDotJfzu7Wx7GwNKk+SP2b5VAHBZTR2c4KeFQ9Xu0LMSLlFT5GrBDus+jPYRzgiTtY8TjcuoEsL1m7VF8XP2V9NLrADYPIiQS8hf9whzyXvsybEqlX+FmeGAwjSeSSCryenxx2Aw4hnBQDXFnu0o6zdFmcp7Z8MzorMqqdvDBx10OK3fZxmwUYj2J92cM+OgPd9VmBvUCpAzQJ835Ga/H/XxU8jCdmbFvrhAeAqZhDpJtBPCinsVSqwSfIm1Qpfcm/VDfFGgfA5Hf+SQKZMP70Q+jd1J9yKqbViRrcaBjEMULm6Kea0nfCexJVSowfAAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BlwvV7JPcZ5PzT+sCSlNqDOsGIiFuX4/kKdMrdQQYGY=; b=R8LQhMr+sYIV7NCOPRZcg+O40hcWSb3CE73UrgSSyjaUXLT2z4fSGv5RKkedcXNXbGKcZocJRmxhW5znDjXOPyCfUHdsuCMjK8nI2/c/j9igLygL3ElATq8FRUQAQwIhyoUU9wxUrj2G+c4VorPQO3GP3edR7fES8of9/VpE9v2cJ5GL1WP4jeIbKOtYFXfZr0Wg+7XXfYglTErQhLcYqKNaMIu7PNKELGcaSwwbK+zQE3LSvYt5iveFaWr8il03vh8girbiZmWfqNgqk/211yXJPThytGrOyn/w5pqi66UMFs0kmir2Y53lJfqZGds4UrSMIYmTtCLcEP1PwcIGBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BlwvV7JPcZ5PzT+sCSlNqDOsGIiFuX4/kKdMrdQQYGY=; b=QfNNBl4qJze96BF4G2phVkG69ZaqQjEYpUdiqZZw1+Ra+7Tk+W1N3i/ZaqGqYzGJHc597QrJfv76zauTXXKV+5jI1pJmEz6VnNE+eNEEWjsf5zsIzbKjhYxx4pnVETqx9lxsTxijmLUuTbLDBP00KiSOwZvuGk0VfabmLx64B6oNzgVWTH2GDoVseUeiIWz9Q2yK/ND5ZzsekpIvVtf2e0Bax9ApD+ZzGP/foJJp5e607LbrPhH1vLLZqHKWuAKw6oDx/ShwC8h2kN+3X+sRuQbkjR7BXtn7AsSaUw59GmQR+Qzsk3qeiITYIyVyEpYZ8ci+g436x/p3OqrrV1xrJQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:05 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:05 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 07/10] fwctl/mlx5: Support for communicating with mlx5 fw Date: Wed, 21 Aug 2024 15:10:59 -0300 Message-ID: <7-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN8PR04CA0058.namprd04.prod.outlook.com (2603:10b6:408:d4::32) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 3f921c07-f1fb-4cc4-c03d-08dcc20c9e54 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: KKiuPR215Ymf/PGSlzqCco1wyIV3peHrDHTMxK0ARR3obVMpcxXVn6hIGGUjj/fiZZzJO85nESIg8H1rkCBfjWMHltNy2TJIaXFEtiD1DjBsd1/s2Rx15BxKjIxlOtR/bLYG9WRsCyaT2l1DUlJjRinItiHaczJQnfVxI0TUuhNGoLjdCDI1QoPYMCv16pTWcekzHOjZyyw3Qt6h/lNr9sxYi2VisyIYWnaEPl87pfOPiObsFrXSYIDq5ny0x17zwTy/n2Wrd4BsQoBUnDcpyEkwa7XJ2kg3KyI7UKVfzhjAOd0mxkbxr+pNDU2w7rVTaauDc6+FFFu+d4pod81Ww3U8g0KewS73sKpUyx7HWYvUVUMU6D0moP1CvEjgxbWzfdBPbIWJfyIN1uvGxBJRT8s0m+3OFY0Ow/b+yPBEDMl7gciA/SW31WQrN31xZIgAjC6HJ4PaieVsDv8G8E5x3c+gD164p9ncYOotLZw/P4I7VTzRDx/SqcEiZH1+h6UQpQcxuirwF+YSb4pfw111aUDXYf0+6LrdSftLwP3ZEU5TlmUg5LVtTYWR1qgUfB2g8kpOd2q80FCmSn99IkMstQEZuF4hl0r5WThW6DYCNsPD97rKNN00Q+ISDy8BEYJkDe+x97fqtcAlBLMa0AuS1HeVtQ3HeZU3W4VeDAbxm9VbX8HSDlUa1e6dHAEE0sXDTcF9JAdcWCHfLpqOdZraoxIvNC/4zHX0mGxM529o6DpxNVvYS/hlAsI+2ilBNVAk6B9o1KgAIQHJr+PBmyyA4RM6Fh6zAzBjr+ocKxl8oKLTpfuE76bdLfZ7fotg0t7YmjUaj+zCnhY5VMMqkIMr/rTGD4FNdvUA1sPV+8vujB037biNQGajRR8JtMLeCleZOtYuuuC+4eO8fYD1giG9rQSex0b0Z2cxXFnJNh7VScF5apiDxxEGI4BemC19uJ8QeaV72jMKqB3aCBqdOP4YiuSXcA8Mfzw9+eKgJJ51KpaEjvDyibUDcoL1tuBAndmSMTNMWHNVlk0CQdfAwloL5dUrglpTBn4g8i/2qauC9Q7eUKZAzhwYaISWgDUVsiTyo1n2iAS4YE6PN7CQhHf6jAatR421Mngw2F8taqZSaikzON8WML6tMH2IU6Rm53v6nXjveFxCvoyXA2BM2IQjOaSnmMBEh5KRJ5udvtU9qfvxUx/TeaKyQFQwOCWLhbuJBZNCFSfWS7cXcLnn8sUdWRGnVoW9rW2e4Ds7HWrL0LrkhjCUvkVxZl/0xSZichG0qfD0FL1YbIKNip2Abc9GLKbU+3jjiqvopIAHnGWXWgR7eZK85H5OGeIzf9vgiy66nwn+WSEN0xcLt3AwR8fOPdHimUXl9JRS4RwfxurGkD8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vTbdb6wvyWOy0yBrnaMuXmYJaNmiwA2hur8mHI6iUGzU+rX+/9X9bmjXcPIR25tBcc1c2R1+9BkzdxT5DSRcKqVnVvPnYXo7JTgmd+4mRj1JwkxcXvqgOcFoFrVSgJw+ejk5DuPd+UUpw8i+shTIVzAxJ+0VjynCguZ/k8l4t0lV86AsC2961jqF/kUMGNFNv1g3dJ7IsrnJMudW5jFQhEtD29K3Rtp4V2gMYdE7lqh8bBbET/usJy3g2PINwKnle0tlx9p7+Mi0BPnGCnVkQS1UFNO2XDe4ycP/cgy3pmyv5FcY/SwmiI4nJtQD5AYEtHfX9WZLEF1K+TPB1edb72JAvZRvtKflOb8DdD0frCJJ9l5159qqck6RqORFXTgsimHlf+zdt9eppVBFNWL5bN7WrQCLu2v1CVwOhVvCB0KkrMb4VbY0e7tmI5jR+8nhr3dVDPAo1wtV5rSLtHXRzHc80fI6qkKKP5/6M3N3EjbDPuoGQTV8FQtZC7L7HJbtQ5fMO4GFiP6qopY6f209QGaaAHmm+ZELx//77wGSl4sETh3kgwnJZbtLFeoZFjSbWrAxtseQfRrb1JEkuJIFgppMpof4QuThMguG+wAkhMEugMMLrh6nEQsXjnocNRAyqXcks/RFRP4O0gKBg8KE1gmRtJGBhT7MpES4rsQ0EOYdZ+fcqHUsKIWv3ifhy1T36srOEDKNZZfgp/O6vydTvBCZ+jPS9oTkxzB/NUR6YpRb3qC2PZHmC3SWJlY/Qa90CTNjiCwR+0ffTJHzUXf3/cLkWE7pD2PE7V7ZkfGEO0RrqQl+qWC4UDybBTbJVNj92BaC/uA2I5WHju2z4DAPwhF5Rhquv+it4XO9G4uzI8sDJMF6xwmMGuM4RW9NQ7aDEcPr6UDLoMR50/71Lu4Vvxn/tk/CKCXoJZzoV9hZ5q40RopWeobzbweIzF4BaFxUyXzxVaMQPWzHRRlDkJyk12CLTMh4GUcHf5Nxm54ozp5J5vtImhy89OsB8y8SwikvkMVr/XEjhqnIrLcFWsPYJe8Gd7MWp6aZc7dZVTpPIiwWwgJf+XkQ3spoI6eWvnKftI9fBSsXAgkI7YJk8P9n1jfd2n8tdtmoK2OMK0ux3EcpWYxQppYJEOOTqPaqqaG/L3gf+Ekuv5uTHJ2HNYzNw/dsvCjnp30esovbjU4LH6m4hpwyUNRsrfdSv9up/yKJkN0V9x8AUI+kN3Me4mzzkZ0qQn/KlBuSWrgCpxs9eXRK3E8NyQEf5+zRO/BH1SN+aUybJI3wfXPVozml8nS5I2G/L2pp/g7LRpn5Fb/yOlpli3LJ5BRcEbmfefLucfzBXEywhmn7YgzirAKYSFQnGrzGtYJKBZn9v6bpyhoRxaxZPkp/1ZCmrXpx/wVByxPYUEKT4ZFR1kLDQqnzB9TMiEfa6xIyhXqzisO+8f7gVYQ8tb+IFss373XpqVOYv/hIRyZlWVhrJS9JRZ63C67w6/TcmvgP+J1DNMUS1cEO75PQQwz1HPx+EmGtufEegMYTD1Xo/Jg3FRcARDbQf4HvERDjp5H0HZVDEqrVDVtcLlhYOGBHmfjtuhOxVFL/iUGV X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3f921c07-f1fb-4cc4-c03d-08dcc20c9e54 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.3405 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xQLq6joXTTQt61ANNtGzWAiSsfqBwCcI7WuC7cnLdmrjCrcZGLHCUxAWlt5Pt9Qm X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 From: Saeed Mahameed mlx5's fw has long provided a User Context concept. This has a long history in RDMA as part of the devx extended verbs programming interface. A User Context is a security envelope that contains objects and controls access. It contains the Protection Domain object from the InfiniBand Architecture and both togther provide the OS with the necessary tools to bind a security context like a process to the device. The security context is restricted to not be able to touch the kernel or other processes. In the RDMA verbs case it is also restricted to not touch global device resources. The fwctl_mlx5 takes this approach and builds a User Context per fwctl file descriptor and uses a FW security capability on the User Context to enable access to global device resources. This makes the context useful for provisioning and debugging the global device state. mlx5 already has a robust infrastructure for delivering RPC messages to fw. Trivially connect fwctl's RPC mechanism to mlx5_cmd_do(). Enforce the User Context ID in every RPC header so the FW knows the security context of the issuing ID. Signed-off-by: Saeed Mahameed Signed-off-by: Jason Gunthorpe Reviewed-by: Jonathan Cameron --- MAINTAINERS | 7 + drivers/fwctl/Kconfig | 14 ++ drivers/fwctl/Makefile | 1 + drivers/fwctl/mlx5/Makefile | 4 + drivers/fwctl/mlx5/main.c | 337 ++++++++++++++++++++++++++++++++++++ include/uapi/fwctl/fwctl.h | 1 + include/uapi/fwctl/mlx5.h | 36 ++++ 7 files changed, 400 insertions(+) create mode 100644 drivers/fwctl/mlx5/Makefile create mode 100644 drivers/fwctl/mlx5/main.c create mode 100644 include/uapi/fwctl/mlx5.h diff --git a/MAINTAINERS b/MAINTAINERS index 97945ca04b1108..d7d12adc521be1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9250,6 +9250,13 @@ F: drivers/fwctl/ F: include/linux/fwctl.h F: include/uapi/fwctl/ +FWCTL MLX5 DRIVER +M: Saeed Mahameed +R: Itay Avraham +L: linux-kernel@vger.kernel.org +S: Maintained +F: drivers/fwctl/mlx5/ + GALAXYCORE GC0308 CAMERA SENSOR DRIVER M: Sebastian Reichel L: linux-media@vger.kernel.org diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig index 37147a695add9a..e5ee2d46d43126 100644 --- a/drivers/fwctl/Kconfig +++ b/drivers/fwctl/Kconfig @@ -7,3 +7,17 @@ menuconfig FWCTL support a wide range of lockdown compatible device behaviors including manipulating device FLASH, debugging, and other activities that don't fit neatly into an existing subsystem. + +if FWCTL +config FWCTL_MLX5 + tristate "mlx5 ConnectX control fwctl driver" + depends on MLX5_CORE + help + MLX5CTL provides interface for the user process to access the debug and + configuration registers of the ConnectX hardware family + (NICs, PCI switches and SmartNIC SoCs). + This will allow configuration and debug tools to work out of the box on + mainstream kernel. + + If you don't know what to do here, say N. +endif diff --git a/drivers/fwctl/Makefile b/drivers/fwctl/Makefile index 1cad210f6ba580..1c535f694d7fe4 100644 --- a/drivers/fwctl/Makefile +++ b/drivers/fwctl/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_FWCTL) += fwctl.o +obj-$(CONFIG_FWCTL_MLX5) += mlx5/ fwctl-y += main.o diff --git a/drivers/fwctl/mlx5/Makefile b/drivers/fwctl/mlx5/Makefile new file mode 100644 index 00000000000000..139a23e3c7c517 --- /dev/null +++ b/drivers/fwctl/mlx5/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_FWCTL_MLX5) += mlx5_fwctl.o + +mlx5_fwctl-y += main.o diff --git a/drivers/fwctl/mlx5/main.c b/drivers/fwctl/mlx5/main.c new file mode 100644 index 00000000000000..8839770fbe7ba5 --- /dev/null +++ b/drivers/fwctl/mlx5/main.c @@ -0,0 +1,337 @@ +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0 +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#include +#include +#include +#include +#include + +#define mlx5ctl_err(mcdev, format, ...) \ + dev_err(&mcdev->fwctl.dev, format, ##__VA_ARGS__) + +#define mlx5ctl_dbg(mcdev, format, ...) \ + dev_dbg(&mcdev->fwctl.dev, "PID %u: " format, current->pid, \ + ##__VA_ARGS__) + +struct mlx5ctl_uctx { + struct fwctl_uctx uctx; + u32 uctx_caps; + u32 uctx_uid; +}; + +struct mlx5ctl_dev { + struct fwctl_device fwctl; + struct mlx5_core_dev *mdev; +}; +DEFINE_FREE(mlx5ctl, struct mlx5ctl_dev *, if (_T) fwctl_put(&_T->fwctl)); + +struct mlx5_ifc_mbox_in_hdr_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_mbox_out_hdr_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x40]; +}; + +enum { + MLX5_UCTX_OBJECT_CAP_TOOLS_RESOURCES = 0x4, +}; + +enum { + MLX5_CMD_OP_QUERY_DIAGNOSTIC_PARAMS = 0x819, + MLX5_CMD_OP_SET_DIAGNOSTIC_PARAMS = 0x820, + MLX5_CMD_OP_QUERY_DIAGNOSTIC_COUNTERS = 0x821, + MLX5_CMD_OP_POSTPONE_CONNECTED_QP_TIMEOUT = 0xb2e, +}; + +static int mlx5ctl_alloc_uid(struct mlx5ctl_dev *mcdev, u32 cap) +{ + u32 out[MLX5_ST_SZ_DW(create_uctx_out)] = {}; + u32 in[MLX5_ST_SZ_DW(create_uctx_in)] = {}; + void *uctx; + int ret; + u16 uid; + + uctx = MLX5_ADDR_OF(create_uctx_in, in, uctx); + + mlx5ctl_dbg(mcdev, "%s: caps 0x%x\n", __func__, cap); + MLX5_SET(create_uctx_in, in, opcode, MLX5_CMD_OP_CREATE_UCTX); + MLX5_SET(uctx, uctx, cap, cap); + + ret = mlx5_cmd_exec(mcdev->mdev, in, sizeof(in), out, sizeof(out)); + if (ret) + return ret; + + uid = MLX5_GET(create_uctx_out, out, uid); + mlx5ctl_dbg(mcdev, "allocated uid %u with caps 0x%x\n", uid, cap); + return uid; +} + +static void mlx5ctl_release_uid(struct mlx5ctl_dev *mcdev, u16 uid) +{ + u32 in[MLX5_ST_SZ_DW(destroy_uctx_in)] = {}; + struct mlx5_core_dev *mdev = mcdev->mdev; + int ret; + + MLX5_SET(destroy_uctx_in, in, opcode, MLX5_CMD_OP_DESTROY_UCTX); + MLX5_SET(destroy_uctx_in, in, uid, uid); + + ret = mlx5_cmd_exec_in(mdev, destroy_uctx, in); + mlx5ctl_dbg(mcdev, "released uid %u %pe\n", uid, ERR_PTR(ret)); +} + +static int mlx5ctl_open_uctx(struct fwctl_uctx *uctx) +{ + struct mlx5ctl_uctx *mfd = + container_of(uctx, struct mlx5ctl_uctx, uctx); + struct mlx5ctl_dev *mcdev = + container_of(uctx->fwctl, struct mlx5ctl_dev, fwctl); + int uid; + + /* + * New FW supports the TOOLS_RESOURCES uid security label + * which allows commands to manipulate the global device state. + * Otherwise only basic existing RDMA devx privilege are allowed. + */ + if (MLX5_CAP_GEN(mcdev->mdev, uctx_cap) & + MLX5_UCTX_OBJECT_CAP_TOOLS_RESOURCES) + mfd->uctx_caps |= MLX5_UCTX_OBJECT_CAP_TOOLS_RESOURCES; + + uid = mlx5ctl_alloc_uid(mcdev, mfd->uctx_caps); + if (uid < 0) + return uid; + + mfd->uctx_uid = uid; + return 0; +} + +static void mlx5ctl_close_uctx(struct fwctl_uctx *uctx) +{ + struct mlx5ctl_dev *mcdev = + container_of(uctx->fwctl, struct mlx5ctl_dev, fwctl); + struct mlx5ctl_uctx *mfd = + container_of(uctx, struct mlx5ctl_uctx, uctx); + + mlx5ctl_release_uid(mcdev, mfd->uctx_uid); +} + +static void *mlx5ctl_info(struct fwctl_uctx *uctx, size_t *length) +{ + struct mlx5ctl_uctx *mfd = + container_of(uctx, struct mlx5ctl_uctx, uctx); + struct fwctl_info_mlx5 *info; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return ERR_PTR(-ENOMEM); + + info->uid = mfd->uctx_uid; + info->uctx_caps = mfd->uctx_caps; + *length = sizeof(*info); + return info; +} + +static bool mlx5ctl_validate_rpc(const void *in, enum fwctl_rpc_scope scope) +{ + u16 opcode = MLX5_GET(mbox_in_hdr, in, opcode); + + /* + * Currently the driver can't keep track of commands that allocate + * objects in the FW, these commands are safe from a security + * perspective but nothing will free the memory when the FD is closed. + * For now permit only query commands. Also the caps for the scope have + * not been defined yet, filter commands manually for now. + */ + switch (opcode) { + case MLX5_CMD_OP_POSTPONE_CONNECTED_QP_TIMEOUT: + case MLX5_CMD_OP_QUERY_ADAPTER: + case MLX5_CMD_OP_QUERY_ESW_FUNCTIONS: + case MLX5_CMD_OP_QUERY_HCA_CAP: + case MLX5_CMD_OP_QUERY_HCA_VPORT_CONTEXT: + case MLX5_CMD_OP_QUERY_ROCE_ADDRESS: + return scope <= FWCTL_RPC_CONFIGURATION; + + case MLX5_CMD_OP_QUERY_CONG_PARAMS: + case MLX5_CMD_OP_QUERY_CONG_STATISTICS: + case MLX5_CMD_OP_QUERY_CONG_STATUS: + case MLX5_CMD_OP_QUERY_CQ: + case MLX5_CMD_OP_QUERY_DCT: + case MLX5_CMD_OP_QUERY_DIAGNOSTIC_COUNTERS: + case MLX5_CMD_OP_QUERY_DIAGNOSTIC_PARAMS: + case MLX5_CMD_OP_QUERY_EQ: + case MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT: + case MLX5_CMD_OP_QUERY_FLOW_COUNTER: + case MLX5_CMD_OP_QUERY_FLOW_GROUP: + case MLX5_CMD_OP_QUERY_FLOW_TABLE_ENTRY: + case MLX5_CMD_OP_QUERY_FLOW_TABLE: + case MLX5_CMD_OP_QUERY_GENERAL_OBJECT: + case MLX5_CMD_OP_QUERY_ISSI: + case MLX5_CMD_OP_QUERY_L2_TABLE_ENTRY: + case MLX5_CMD_OP_QUERY_LAG: + case MLX5_CMD_OP_QUERY_MAD_DEMUX: + case MLX5_CMD_OP_QUERY_MKEY: + case MLX5_CMD_OP_QUERY_MODIFY_HEADER_CONTEXT: + case MLX5_CMD_OP_QUERY_PACKET_REFORMAT_CONTEXT: + case MLX5_CMD_OP_QUERY_PAGES: + case MLX5_CMD_OP_QUERY_Q_COUNTER: + case MLX5_CMD_OP_QUERY_QP: + case MLX5_CMD_OP_QUERY_RMP: + case MLX5_CMD_OP_QUERY_RQ: + case MLX5_CMD_OP_QUERY_RQT: + case MLX5_CMD_OP_QUERY_SCHEDULING_ELEMENT: + case MLX5_CMD_OP_QUERY_SPECIAL_CONTEXTS: + case MLX5_CMD_OP_QUERY_SQ: + case MLX5_CMD_OP_QUERY_SRQ: + case MLX5_CMD_OP_QUERY_TIR: + case MLX5_CMD_OP_QUERY_TIS: + case MLX5_CMD_OP_QUERY_VHCA_MIGRATION_STATE: + case MLX5_CMD_OP_QUERY_VNIC_ENV: + case MLX5_CMD_OP_QUERY_VPORT_COUNTER: + case MLX5_CMD_OP_QUERY_VPORT_STATE: + case MLX5_CMD_OP_QUERY_WOL_ROL: + case MLX5_CMD_OP_QUERY_XRC_SRQ: + case MLX5_CMD_OP_QUERY_XRQ_DC_PARAMS_ENTRY: + case MLX5_CMD_OP_QUERY_XRQ_ERROR_PARAMS: + case MLX5_CMD_OP_QUERY_XRQ: + return scope <= FWCTL_RPC_DEBUG_READ_ONLY; + + case MLX5_CMD_OP_SET_DIAGNOSTIC_PARAMS: + return scope <= FWCTL_RPC_DEBUG_WRITE; + + case MLX5_CMD_OP_ACCESS_REG: + return scope <= FWCTL_RPC_DEBUG_WRITE_FULL; + default: + return false; + } +} + +static void *mlx5ctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *rpc_in, size_t in_len, size_t *out_len) +{ + struct mlx5ctl_dev *mcdev = + container_of(uctx->fwctl, struct mlx5ctl_dev, fwctl); + struct mlx5ctl_uctx *mfd = + container_of(uctx, struct mlx5ctl_uctx, uctx); + void *rpc_alloc __free(kvfree) = NULL; + void *rpc_out; + int ret; + + if (in_len < MLX5_ST_SZ_BYTES(mbox_in_hdr) || + *out_len < MLX5_ST_SZ_BYTES(mbox_out_hdr)) + return ERR_PTR(-EMSGSIZE); + + /* FIXME: Requires device support for more scopes */ + if (scope != FWCTL_RPC_CONFIGURATION && + scope != FWCTL_RPC_DEBUG_READ_ONLY) + return ERR_PTR(-EOPNOTSUPP); + + mlx5ctl_dbg(mcdev, "[UID %d] cmdif: opcode 0x%x inlen %zu outlen %zu\n", + mfd->uctx_uid, MLX5_GET(mbox_in_hdr, rpc_in, opcode), + in_len, *out_len); + + if (!mlx5ctl_validate_rpc(rpc_in, scope)) + return ERR_PTR(-EBADMSG); + + /* + * mlx5_cmd_do() copies the input message to its own buffer before + * executing it, so we can reuse the allocation for the output. + */ + if (*out_len <= in_len) { + rpc_out = rpc_in; + } else { + rpc_out = rpc_alloc = kvzalloc(*out_len, GFP_KERNEL); + if (!rpc_alloc) + return ERR_PTR(-ENOMEM); + } + + /* Enforce the user context for the command */ + MLX5_SET(mbox_in_hdr, rpc_in, uid, mfd->uctx_uid); + ret = mlx5_cmd_do(mcdev->mdev, rpc_in, in_len, rpc_out, *out_len); + + mlx5ctl_dbg(mcdev, + "[UID %d] cmdif: opcode 0x%x status 0x%x retval %pe\n", + mfd->uctx_uid, MLX5_GET(mbox_in_hdr, rpc_in, opcode), + MLX5_GET(mbox_out_hdr, rpc_out, status), ERR_PTR(ret)); + + /* + * -EREMOTEIO means execution succeeded and the out is valid, + * but an error code was returned inside out. Everything else + * means the RPC did not make it to the device. + */ + if (ret && ret != -EREMOTEIO) + return ERR_PTR(ret); + if (rpc_out == rpc_in) + return rpc_in; + return_ptr(rpc_alloc); +} + +static const struct fwctl_ops mlx5ctl_ops = { + .device_type = FWCTL_DEVICE_TYPE_MLX5, + .uctx_size = sizeof(struct mlx5ctl_uctx), + .open_uctx = mlx5ctl_open_uctx, + .close_uctx = mlx5ctl_close_uctx, + .info = mlx5ctl_info, + .fw_rpc = mlx5ctl_fw_rpc, +}; + +static int mlx5ctl_probe(struct auxiliary_device *adev, + const struct auxiliary_device_id *id) + +{ + struct mlx5_adev *madev = container_of(adev, struct mlx5_adev, adev); + struct mlx5_core_dev *mdev = madev->mdev; + struct mlx5ctl_dev *mcdev __free(mlx5ctl) = fwctl_alloc_device( + &mdev->pdev->dev, &mlx5ctl_ops, struct mlx5ctl_dev, fwctl); + int ret; + + if (!mcdev) + return -ENOMEM; + + mcdev->mdev = mdev; + + ret = fwctl_register(&mcdev->fwctl); + if (ret) + return ret; + auxiliary_set_drvdata(adev, no_free_ptr(mcdev)); + return 0; +} + +static void mlx5ctl_remove(struct auxiliary_device *adev) +{ + struct mlx5ctl_dev *mcdev __free(mlx5ctl) = auxiliary_get_drvdata(adev); + + fwctl_unregister(&mcdev->fwctl); +} + +static const struct auxiliary_device_id mlx5ctl_id_table[] = { + {.name = MLX5_ADEV_NAME ".fwctl",}, + {} +}; +MODULE_DEVICE_TABLE(auxiliary, mlx5ctl_id_table); + +static struct auxiliary_driver mlx5ctl_driver = { + .name = "mlx5_fwctl", + .probe = mlx5ctl_probe, + .remove = mlx5ctl_remove, + .id_table = mlx5ctl_id_table, +}; + +module_auxiliary_driver(mlx5ctl_driver); + +MODULE_IMPORT_NS(FWCTL); +MODULE_DESCRIPTION("mlx5 ConnectX fwctl driver"); +MODULE_AUTHOR("Saeed Mahameed "); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 3af9f9eb9b1878..f9b27fb5c1618c 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -42,6 +42,7 @@ enum { enum fwctl_device_type { FWCTL_DEVICE_TYPE_ERROR = 0, + FWCTL_DEVICE_TYPE_MLX5 = 1, }; /** diff --git a/include/uapi/fwctl/mlx5.h b/include/uapi/fwctl/mlx5.h new file mode 100644 index 00000000000000..bcb4602ffdeee4 --- /dev/null +++ b/include/uapi/fwctl/mlx5.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + * + * These are definitions for the command interface for mlx5 HW. mlx5 FW has a + * User Context mechanism which allows the FW to understand a security scope. + * FWCTL binds each FD to a FW user context and then places the User Context ID + * (UID) in each command header. The created User Context has a capability set + * that is appropriate for FWCTL's security model. + * + * Command formation should use a copy of the structs in mlx5_ifc.h following + * the Programmers Reference Manual. A open release is available here: + * + * https://network.nvidia.com/files/doc-2020/ethernet-adapters-programming-manual.pdf + * + * The device_type for this file is FWCTL_DEVICE_TYPE_MLX5. + */ +#ifndef _UAPI_FWCTL_MLX5_H +#define _UAPI_FWCTL_MLX5_H + +#include + +/** + * struct fwctl_info_mlx5 - ioctl(FWCTL_INFO) out_device_data + * @uid: The FW UID this FD is bound to. Each command header will force + * this value. + * @uctx_caps: The FW capabilities that are enabled for the uid. + * + * Return basic information about the FW interface available. + */ +struct fwctl_info_mlx5 { + __u32 uid; + __u32 uctx_caps; +}; + +#endif From patchwork Wed Aug 21 18:11:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771951 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 881511B3B11; Wed, 21 Aug 2024 18:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263876; cv=fail; b=LFNz5BaBwFhnYMRdfLDCAFrr7LB/MEIW4vIfudJ3lJ/yXQ4ZB29dhivyFstemxpxM7V6NcohtiUqTbgALios0MN6FTBn25cqqNcJZpM69dm77LwoEeiFb1TTZIfM68lMhsO8V14YPeCb25FVlDp+3bLMZ+A3llyrrAvr6fG0Dp0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263876; c=relaxed/simple; bh=suCgcmNktUvgJaB5zPvYDmKarrXp48jh0gfh8cQalx0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Ji1XQKKS/Ur55QlA+yYkV8whIkotPQC2yimOPnhBwF/hbalBkg9tA0Wzf/izSUp9hPHGGTNu5oAW12xb65ApfYSPqS+AxqEZfoZ1qlh972c5FNH5hke2UfA3mkgcAT3rrluaqX6HAVlObb4rGr9uW6Z0Si9ZggWEvJSp5ZKZitU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=U468kwG8; arc=fail smtp.client-ip=40.107.101.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="U468kwG8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fUctp0wBe7IfY+b6D71ksBxrjuxJA5b5Hi/i8e9/tYAzDnrI7hGaljc+5HWZzDkbMA2NHu7skLMBSAlGj16N0WYqgiLcgLhfaVtCdkR2HvpAbSYe9zQ2f0YsMhHsPCi3anCAGiZaJDtETlxWQOWfNs0qYWqpY2xygg/Fw363CcPJ0+Zc79yV6DBM7w8tddrNgjJGbWN+Pn2BKR5TNNzNnIBoJSGfvmDCRYM6WsxTn2T/n786p4EsYjUJWhPzWc1C4iu23hNcI2hYVmvi9CE0ImN4LAFFAqz2pxWsSglcO8qvfhAczRMnYjJeUua1jYX36ZglRFgL42EMGNUQIa1AYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AoEUo796KhnBL7/w2VPe8nzAdv2FMhK/MN5+Z4HN6DY=; b=k+gPXatbRRy0G/rA4mLlIcOEscFZr+ldYyrr31N+B8Jsq3PVUUKwNQP4wHue92xLPRavNtXrnuUTfRKtHERwsGYuimZcPylL0lTnPDkw90WwYWMUGsBWx7TUzrbG52LHhJdopvUktYjC7/lHixagQr3KaBcVX8OHbSMP5u4sjJIQRH7kqrvkGm8H+DU/xdWgRlPqnq8+XoZ0A6Sripe1qvt8Ulaf233T6SwbHXkxSHghsvUg8iHEcx8Fgl0gPl62bdEDYk5Xf7EOQqzAU3iFx06ERg+bxA+WjQY0plH+oj9zKe0iJ6zw6ZlxymSsgiT5SbRbtDY2Z75pEL/I9xpf3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AoEUo796KhnBL7/w2VPe8nzAdv2FMhK/MN5+Z4HN6DY=; b=U468kwG8hsRKZ28Jr6d+GUIsF9XtWEw9XiEguMJMV8rdbUk4RQSV/UtSJ/onDWU+irzmMGo+2BHuviUxCuPrSLyCGMROIvEIMVZgmLOZDy+uvRNEmZ2hile11HzB3/VxQ+oygKPlMjV4tlc6mGhW8Sp80ilVJZUcS+mPx2HOr6248q+HCWe1Fh7cGX1dmfO8FOH6EDBB3ZXolam/ov//LWbaG3Pk1I+1C6Imr/SJflNlhoZGbEHv/Ot1GHd/ovIGR9YjmbWuOShFDiBU+Z1mLG7wUjuLjHOB0bJ3pMWj7oIJpu58a5QNKSw6TYtA3OPbRRnh1WCiRKyjwAo8SDBymQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:04 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:04 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 08/10] mlx5: Create an auxiliary device for fwctl_mlx5 Date: Wed, 21 Aug 2024 15:11:00 -0300 Message-ID: <8-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN8PR04CA0050.namprd04.prod.outlook.com (2603:10b6:408:d4::24) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 684a2ad1-1f08-4053-23a3-08dcc20c9e5b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: Xv6LOO+8cqxZs62bItCytTJQ/14m2DQvhUz6phJ1f7JUFZYb8it53wy+WG04O1GgibGOEmytcIKGPfJddKk9Z7AnkKORKKgk9KbL8Nim2PdNKYb/Nh8l/Ez0bddsXV6ZgION4rV8to95mQWF6f2DhE9aeMA2Lp3EZiNBwrcaLWb0GQ2gd8ixLncIyaFHv0clgvCCjJjAWhGPXEzmmrlJ/AeY+JfgLfxBEoW3WhZNzXS8GDK+VChuFjxtxFlvDs9b/eaa34VMRGDwBM2v954paddbHnLgz0dYbR2+bLufbiBtbROjA+/HQ2jHKufhOCLTSCmG3DQuCBzBZshGTs9B/EynSFRY2KTOiYjzz7LaubltlswkBUCHb6Xc6b695nLuya9ggIUh6uY7kECyJIxKvNTazGgPh2nongLhMbAafx3kEr2xDA3JaXXnY6NF+oaXhttlVOZZVat9nIGW9v90Apn7va2K0ZiGjw1MaHQTUaBnjP3vJ1ikb+jyA5ST9eWc8BqUjxzTPqy1rwzGgZkicCl6SBLe6fsB+N1yBh9gxsbf/jih64afeT3iJA/F7BstVfjxYXJOaGzDcuob6QnH65bzFiRTZW4JnPe7oLsmenqAT65SvBix5w4Ae3jwI6/wDIJaztrf3cHFxalhl9wJLv6C0X8dzXxrjFTkyZ38aLgqoL+uOlpmKmh2V9YWfW6bk+TqQlj8LOjP0N0pSXYngiJKQEtgq7Ae22Y1lylqrAGM6HnmkM3eno7EiPciJq9k2vSlSo2eJufuY/kYURwGoFRvBIGDoh31YJ3oTXV0IeoymK112ZIfAuEon0P/BVjwWWOeu1bGTwoXY2M6Ya+NR1c7QWDUSkje3eHw2W96m2mRInAA3PSaJN2hbSSxj+3znDDfjRVY020VoS95qq3gnx14Hm5LX+4f54wIGSsPr+iqrjj+qdhRxT4QBzO/uo2mqX19Utrl4oO5Uq6eA/n1bMm7hpXTktV7X9sFPs5Dbq0xAfMX6OhisRCg094NoZtn4uN/+v72lH1g28xJ/X08l2lmmTL5cRi6IZ/Jy4qw/myrMvij4Q1CZ3qcDKEacNZ/7SFLMmptKXtNbMPKB1rkYqEZomAhmrXjpKBv7NEMgqvFF2LCzoAQKIvjaC9hlrqKL9Kw1+zolAvM3JDiGG7be+ktEODSvur3hmdl4iOUNlNEO/ORRTYBGjeas9SQx4rO7YjCJSbaMZ7+eBy1aLz1u49NRTXuXUt+VCAu8DwzookEmVyzUXoQCFZrzso9+Ze6nLaaaZ+j3L9WBzexwfDTSHp5pNxe8nhqtdF+7kWPuvRtBj8JEOo4QJU5s7JtHsD/WBZUNQbrcUOOWVL8z6+K1w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oxkCEx+OSd+sznIaTtOBcpAlxMWOEwjVC2fguXI+GUKNdFXtt2LrP2+8LKrz2+ei1yNpmulwH233uZCGF+A6VhU34GZMGBZxR+IQkZYXv7UgO0zGLHcHvpOaDoN0eFrwQznxu0vw1kPeyF30fHEqPuTWpmlypGYwBlqrlG0INACIB0iDG4lF3sf0IBZe2hksl1hGtaPcxBILN5IaTjr/b5D54R4q8fdNYgJ5H4v3A/eeIOkn1w8QHgozQLKQW2HDLTqsnBPnNu5oVJse4xwej+JBOO7HBPhtEOqZP2ifCF8xh6E8tKLbn0ckuB/x29rePCFatXzNDLVbcAoGXKIbWur9RqjwYhdfY6ftUEs+Fgh9BS7ADxn6SQX4pwSaYRobDse4NgVWq0PN3bjkrqX7irmASY682Iy8MCvr6MnJxYJnpCro8f+YT6TDviPSnO7ZgsQm7IymwKzBFlZwRpIc5GQVKM7XEv/6QwgrLycXBTQQMCGzcC7xiAlo5ftKlu/03IKovnvtvksjmJN30bTNr0mH8LeuU3vaNDiNRrvBS9NQTRi6rQraRNgu4zN9MSkbqy08+KlTsyUiVpubZbHi34sKtJ+kZzJuq6MPmbli64gIme2+praxGPK4U0v7hP4HY+XtR9FoRyIYHSn1HlergCAK9/zGoJKlstPzG6lzfC3L188JD69mKvGN6RJqvGDv6ryz+tXhfw0JqE8ahRHdG0h2kw59FuT846P2o2vpIRAHqf93yRbi4Q4lDSXzwJJ5sNiFOQgNsSQPkJ1tE6zGIUfuNYvw7KFXPHafbugHboJM8+Lu15MPniAiRdlHBdeQPBcGNFJe+q9V04BrnbnFBkXZGU0m70fXExJcUfVSwKKk7SF1atMr7bwIuOQHO4saHY+XdJPftGGY8o3jKblklwYw4LR/l2OLgvNACcr+HZ5Lv9TS6qdeLdsGshMZLpFp8r+xshF5MlZixlFT26lj5aufCVKAzOtfNajAHyazx8FaQMq+VEa0P2hw9dcOfk9taXFRhwD6zHXds+XpMCDpa+Ge16evRAPhdQHaeFKFfIXfmQ6e8z1bYI2uvmj3zTePW1E48uhhN4gXcAPWcFzEvLuYNcTyO/1v5Rtg5YexooWu3AVVFm60UYtVawmqFaXfAKk9vr/05TwWaDmYHGxtO0zEb6DS1fZ8Emk4eYWWX7G/XFbu3Gqjd7sNDXQHWJnTw3yB+sBXtfQDHTAOlnwWiTq9MLn6OwCRQC2KREj/AwKNCDUI9babL115HEd1wm89LDItchuCadC21yPwEr0MqqhMiMmiAKzaM7unuxWFR5Ojc0s42NECeoSKkP1Q6yzurt+mO1R9A39J8zGqDpdmLLZP6WDenJvIg9Z5wr1R/HLlPJrbIyFmQm/y0SK647nk9tgqLCJTF2FAXpqIKUgi2XzMOznG9pTWWgcw/vy8Nfp7w/JCTdD6gFXvI3d1rabEWzVwRLqE0PKGeCQdktd7qQyl3nSXLZv3N02UyrvLMQIQBemunOAF7EzoPloiJ7uq8cmop9ozUwyFOCfNF9jLikxacKFcGflwONOZcIIL8yI3BIYDuxW3EBi09aw2SiFe X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 684a2ad1-1f08-4053-23a3-08dcc20c9e5b X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:03.3009 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G6ozlKcRXosnAaAVxNekEYJs+emaRh5GnzGzetsE5wYuIDykJ/QpuEMhgSKsaX1v X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 From: Saeed Mahameed If the device supports User Context then it can support fwctl. Create an auxiliary device to allow fwctl to bind to it. Create a sysfs like: $ ls /sys/devices/pci0000:00/0000:00:0a.0/mlx5_core.fwctl.0/driver -l lrwxrwxrwx 1 root root 0 Apr 25 19:46 /sys/devices/pci0000:00/0000:00:0a.0/mlx5_core.fwctl.0/driver -> ../../../../bus/auxiliary/drivers/mlx5_fwctl.mlx5_fwctl Signed-off-by: Saeed Mahameed Signed-off-by: Jason Gunthorpe --- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index 9a79674d27f15a..0786b011a8bc29 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -228,8 +228,14 @@ enum { MLX5_INTERFACE_PROTOCOL_VNET, MLX5_INTERFACE_PROTOCOL_DPLL, + MLX5_INTERFACE_PROTOCOL_FWCTL, }; +static bool is_fwctl_supported(struct mlx5_core_dev *dev) +{ + return MLX5_CAP_GEN(dev, uctx_cap); +} + static const struct mlx5_adev_device { const char *suffix; bool (*is_supported)(struct mlx5_core_dev *dev); @@ -252,6 +258,8 @@ static const struct mlx5_adev_device { .is_supported = &is_mp_supported }, [MLX5_INTERFACE_PROTOCOL_DPLL] = { .suffix = "dpll", .is_supported = &is_dpll_supported }, + [MLX5_INTERFACE_PROTOCOL_FWCTL] = { .suffix = "fwctl", + .is_supported = &is_fwctl_supported }, }; int mlx5_adev_idx_alloc(void) From patchwork Wed Aug 21 18:11:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771956 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE9901B3B1E; Wed, 21 Aug 2024 18:11:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263882; cv=fail; b=odW/0LZoJVNqbTGb/+YrfOMXrbpmQows4i0xxwi69+EaQCuqHUE44ysJo/Qoh6isSsGngF0YeGClD6OQC6pAuYk3wawUe3YmPsLw/2frr7YOHPcC22KB82TU6jy0AAoZ8T3PdlgFY0655FjlkRnJEv9aoemK+BRRnYInPRkYyAU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263882; c=relaxed/simple; bh=L1G3aY3SFzQaMWD4/RhS/G3jsSpf47QkdgNgdzKTBHs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=nL46lq47joaS+bQ+And41l2b+eumOP761XJst0Gi2PcfDngzyIs+PfY4AUqxOsc+Q932lsa4h8F8IiNIwx7BbPUCtvllxazcA9oDB2E8WLt67/CrpLElkZG0KtGMGxLpqm09I56K6Jqif+UrW9x3GW/PSUlhnZHlCkc/A4kq4OQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=A121P8Vh; arc=fail smtp.client-ip=40.107.101.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="A121P8Vh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wdotGlM/28nTaRgvGxd0lyWHftZkgeAoAtrvBG2wGVgYfNhnsZBYNlEQnteJZ1+WpA72TePD8KNGE87SC2tr0CewFsleFM/XkF8++t6xWh1rsRjiPrtPUgIAZh1CqOnQkeHVWOXpF2lLgvrDcZd5Fz7YoqTfcp6XNPm7fA+WqS65KqSoKIP6NX9gGC3kapwZUuPE3jEcdFqTjCYuCi7t1UGZWHpU3Xkwdydda7SiBdqD+YuqVQnEFKVj6+AXHHrapdCM7cse92T4O8GGa8wrFNzPzfRiOdNZxYoqfkiPQn+FYNgm7s/I0tQcUlmsfIqtrpn9oOkanIJvgVXGGEvnsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QdLpZJk2bwMP0v6fNqM3lYHDyXfOw0sr/41ozrvhJeU=; b=fnfLyGJuN9F0nkzCi8XHOx3CFX3LUoakQn72rlKj1+WTL2uL02q7NCbD/SpQRTaZ6Z/dC6e4uTalBx9Fdzcu9Ud0uahUKzp71qIs3qDojVj8pAsuTYbNRoMCaQmB3OxPsyog0xyrG6nQQNUWVvA/4lPHWfo7HhdUwaJviehimpeGl0pqH2II4YG0oIv77+R9NP9syIR5fOHaex65bX2kWcQer6Weo0nonLikgdM+h1gFqHSqSKyDC/GNET5MKr5fC/KcqCQrULMuoV8pVnN55p61p1+KSwSTPnZjdF/9ECub6RnXDa4Sno3T9rNRx7JQQpqHi6kbuKKuqwHvSEBSIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QdLpZJk2bwMP0v6fNqM3lYHDyXfOw0sr/41ozrvhJeU=; b=A121P8VhQZ7BjZPy/zaE8qBLfsBJ898raUxb2a9eubqPEniEgQYzo7IQNe1a1oKoyJpgyA/8EvOShVHsuqJP/RU8wzD/Vxo7uuNECD5WVfWwap097tBixvCt4vW56HfIJhf6FIYkLbZbxvK1xA8OLtNmTYLgnF32QMT+JuTO1xE8VfoXAvBu2L6IdHzJdMuG+swgCJwGpRMt5/nKi7ouzI77VTfu23hMRjjy6DJFChVo9bi/1SsE7F2PaXTutsv08wqPWYlyO3pNWPNGvZgC89XO+cquoQysH2t/pM7Gl3sSPfZyou76dna1kNa+cYBJeF1+wfPfu/qpEp1VnX6Mrg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:09 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:08 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 09/10] fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands (RFC) Date: Wed, 21 Aug 2024 15:11:01 -0300 Message-ID: <9-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0035.namprd03.prod.outlook.com (2603:10b6:408:fb::10) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 15990304-603b-420a-6199-08dcc20c9efd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: f0Ve92lnhxgtTL97m/uB3DicjAl7EJYDINNGRksh01EE2yrfnO19zUMBpSWjAbNKV9crXlNIqwVrpQTSR6JMmoA0AyRgS3nI6lduZ0NlP5qQEFW5nX4RYffMSicdtzB0K3BNIv4eGrreWRcyH3XGSNOu7hjzYQZxsxV3iIZBIC2v0l4xsMXi9TAPl54NpgSTQPYnQqOFU4NjMf0X46CKC1n/hOM2PzcGiNTIW8wJdlkYSMJY0aat8aYjxZ3OOSGjGTgezwEdpe7QOCZP1uNAQBialzj+Kh4ZVKo+2efKG6uMl3sZqzVf2v7CAfOEaOux+zw1X7w/btthIIFYbb1uo+Ewscrapm858jlPR048oQU5PZzvDiZQD3m8c0LZjrTefucyysKZ6NnkBmk/gNh2a3fs7LIVNeyYogPzeA3Me+XClhMyBRJmDeZkVYF0TVDiORqVWh8icDm8+D0dk6Nxal8xWT66t9Iw6P/vGlHEtDfzSRWxj7zYhLCgNDEIm2AqtD4IUoewNX8c/7Jt1w+3bAlZfHkpECzzcG5RPxDHhIUGgk1Q8wVLo4f5ktm9jnu3DZPckyi2lWEQUEunN0wfK+RQ8VOy9bm/CeJGTkxLN4J7oOtuq0fAj1P0koujdox3KlCbPnuJz+lpzevBmXlWbmJtPUlgc9hUQPcL7LLN11HtkO0BXQ0YXG3vCiGM2lCJRlQC0tMS6CvQ46a56bTerfPqOPsVOgV0GBrWDIynYLY8kA94uORggMJQMe8aU3vkdnvsZ6h5/QxauPyR3sqn2lU8RuNI6lNuAqXdEHfQt75LCTOpKLiTYFjTCAJxboIDIOsBTZcOdsWJ6ErMZg/mh6U1Sh2lCHd/feEVQWtvOVyWkc7KSLlOsFpofqQWAYGDz5qTQJB8dkvGiK3JLtvq8kgPNF/CPY+eoM0J7KOIdB8vi4ZROqRnRXbd3lf7232vQ/LYrQ/VdH5tnjCyEpHCGcPKMuGvGUY2cmbtiSKcrG1E4t4FqFinN7ubyzA+/wH/aL6Gezee7Qn0vdxvljjfjoVDf9OciPBQrrTC8FsWaegjT2/0TLV47IlYIuwCd1BcXExNKD0nHxfdmpeT1dTp8AsZDYTF9e3f4+rwHpWUBD/cbVXyDTUiwK3bG0qDglA1MfHZDs+EJTKMgfChlQ0rybCQcsvPijPjLfGiUHZKsio31hBn8mPGIHGvuNPVi2SWuGK3m9K9fVQgapbdm1eHasLGm6g8Ess/6MiEN4pQ/GKxRYxBXr3BrHbUTNdJeae+qOGC9OzHJ5H6CzLdfIfPCf5iG4pcs5FtvzCPZ16qNdkVbLx7viByXDLZxDuW0rJhTBXybD5UVsbRsMpCUaw9Gg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: VLWsmnclC+ybcx4mUGPPbUrdSN20IW3DdpOhCjpTkAx4a98whUmkq4EsZiPWb38SrJT1YYEJtZ7gPJx4c9XEHGG21k+iR2tsottwf7GhKE4BgAeyGq9QvFhHaUC9Tg+jJYUwlp0rsOCViYxt3Nn4dJBUjm7vl95JJqvpj05oI1nuHImaEbBj5LRDZaTK3+DzSvAq0L2TQOjG+3P0LOaxhl3INJ5LVaqkf+oVpdmDTY4LN/+VLARkodYhWHasqNEsrkYuZnOhGpdKE6p+RYkmT0Rm5rurZv1nqVd32UkEwUkt9FbTdD3hxGhU27CzkAUWZElbeAxMUOUpVX+oob+vzlfvLuXEIt91QXMMTYz1t5sx99DKsz1jNLdhntJyA8eByqt7l+jzdylzLGWXoGya8ezRk/s8vH1nJiYHaQUl0uWfkThXwCEtV6HclmnuU7ZRwGLzilG7EwBQPqWN0z2E+SioKLoAaDDiXRi7mCmCYb4RR+noFdj3wQvXawPwvQXTq1tYz1pjkz0YOI34O1vmib0tm00pV9DmK0twg3MRY2a7ZAuDgL+RZmEwa5iXhcW/sOoXux8pOKEYVY6/dK5Xlf7VUwoanmIa3kv8V8Xkbsr77tSRL5vvW2T2pFxh35+jXmRZL37igmeOn4bZsVSFblPESjAUD+GIgznYsWqJ/IEeMox0PmL2z6fk+DepWxtHqf6Y98sKYGHi3ZA7PEhDClsQVnbunDrao7z+lGwb3QTeba6G8gYs1dTlwGCo+Cn6DIfvFcvw0EZVWgaUglUKlxlIywdx5f36hMHewiYOLNKY1Oz9G2jl4IvIbke7E6qJcvxGsvOdbBBQUHWgqojPnw0zCWcKQcK7Jh10NJpwEe4p4fOvR85qL2sGyttvMkh7gD31jZFtcBiZDbpaut9E0osTyKVSRMMaW4fc9UDwvXWpOrpWTlQIZDO5JxQ32ai1DrA5Qye/dVnQjzPVZ+eDGWpm8FhIgETq3+5L8CkoedBQzTaUZfAQbKHIoyr1DJWTtjjX0v/he+cMjzx34s+D9pMM4InZrVMDeXiqOS1TaDY2v+RXa8MJKg7gAiac02nVJohU1S+3QBCmGBpBt/fS6RrY1kwopBDjACQQ61J9r45coJYAKy68N5AkR7Fet5LZfLqJ9QRdogMUkqbod8deXj1waO4E+sRfgrI4pC90PlCVvbySJeiGs3aP9a/uteoOgAjHB3VirJAHULuJLbd08660GiJW4ud52A4LoDTyWNL+bLGOu/sXeQiMqr/4JEqeBotkMcj+tOEupx3eU5m2moi77V8jVuj2hyATpipRM8OMG7jnm4/9CB6Gn5GFE+/0r8sh36rEQa0/Yq7F2Vq2bOOWuUHORU+hN+Cu+Tlo7MQF7Fhs8CWS8ozzFyMHW44iaLaxKk2fUR5aVT24WTzO8GRVtniZxAQ45w1xf3gXL1ZzoOeSgKnR7PJN7ciig3dHFX4ZKs0b+OaZYyGU9HO4OZ3wzpDmJBSahFP/ROp7r7RM12xvZlPaU6OcXR8JUeWadh8LB4E81OrA/zIEf6Bn1i9uRPm+8sSNvONleIpkrRsYRp2x3FUrI0CdjUWvw4A4 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 15990304-603b-420a-6199-08dcc20c9efd X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:04.4761 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KxbTS1GkfO98pSnqxtttc/wO0fyx8np2OUZ3tGkKepSuL5PiAbQkSPzCtHWjEOS4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 From: Dave Jiang Add an fwctl (auxiliary bus) driver to allow sending of CXL feature commands from userspace through as ioctls. Create a driver skeleton for initial setup. FWCTL_INFO will return the commands supported by the fwctl char device as a bitmap of enable commands. fwctl provides a fwctl_ops->fw_rpc() callback in order to issue ioctls to a device. FWCTL_RPC will start by supporting the CXL feature commands: Get Supported Features, Get Feature, and Set Feature. FWCTL_RPC provides 'enum fwctl_rpc_scope' parameter where it indicates the security scope of the call. The Get Supported Features and Get Feature calls can be executed with the scope of FWCTL_RPC_DEBUG_READ_ONLY. The Set Feature call is gated by the effects of the feature reported by Get Supported Features call for the specific feature. Add a software command through FWCTL_RPC in order for the user to retrieve information about the commands that are supported. In this instance only 3 commands are supported: Get Supported Features, Get Feature, and Set Feature. The expected flow of operation is to send the call first with 0 set to the n_commands parameter to indicate query of total commands available. And then a second call provides the number of commands to retrieve with the appropriate amount of memory allocated to store information about the commands. Link: https://patch.msgid.link/r/20240718213446.1750135-9-dave.jiang@intel.com Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- MAINTAINERS | 7 + drivers/fwctl/Kconfig | 9 ++ drivers/fwctl/Makefile | 1 + drivers/fwctl/cxl/Makefile | 4 + drivers/fwctl/cxl/cxl.c | 274 ++++++++++++++++++++++++++++++++++++ include/linux/cxl/mailbox.h | 104 ++++++++++++++ include/uapi/fwctl/cxl.h | 94 +++++++++++++ include/uapi/fwctl/fwctl.h | 1 + 8 files changed, 494 insertions(+) create mode 100644 drivers/fwctl/cxl/Makefile create mode 100644 drivers/fwctl/cxl/cxl.c create mode 100644 include/uapi/fwctl/cxl.h diff --git a/MAINTAINERS b/MAINTAINERS index d7d12adc521be1..9933c67303f0ab 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9257,6 +9257,13 @@ L: linux-kernel@vger.kernel.org S: Maintained F: drivers/fwctl/mlx5/ +FWCTL CXL DRIVER +M: Dave Jiang +R: Dan Williams +L: linux-cxl@vger.kernel.org +S: Maintained +F: drivers/fwctl/cxl/ + GALAXYCORE GC0308 CAMERA SENSOR DRIVER M: Sebastian Reichel L: linux-media@vger.kernel.org diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig index e5ee2d46d43126..e49903a9d0d34f 100644 --- a/drivers/fwctl/Kconfig +++ b/drivers/fwctl/Kconfig @@ -19,5 +19,14 @@ config FWCTL_MLX5 This will allow configuration and debug tools to work out of the box on mainstream kernel. + If you don't know what to do here, say N. + +config FWCTL_CXL + tristate "CXL fwctl driver" + depends on CXL_BUS + help + CXLCTL provides interface for the user process to access user allowed + mailbox commands for CXL device. + If you don't know what to do here, say N. endif diff --git a/drivers/fwctl/Makefile b/drivers/fwctl/Makefile index 1c535f694d7fe4..bd356e6f2e5af1 100644 --- a/drivers/fwctl/Makefile +++ b/drivers/fwctl/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_FWCTL) += fwctl.o obj-$(CONFIG_FWCTL_MLX5) += mlx5/ +obj-$(CONFIG_FWCTL_CXL) += cxl/ fwctl-y += main.o diff --git a/drivers/fwctl/cxl/Makefile b/drivers/fwctl/cxl/Makefile new file mode 100644 index 00000000000000..62319452157272 --- /dev/null +++ b/drivers/fwctl/cxl/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_FWCTL_CXL) += cxl_fwctl.o + +cxl_fwctl-y += cxl.o diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c new file mode 100644 index 00000000000000..8836a806763f54 --- /dev/null +++ b/drivers/fwctl/cxl/cxl.c @@ -0,0 +1,274 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, Intel Corporation + */ +#include +#include +#include +#include +#include + +struct cxlctl_uctx { + struct fwctl_uctx uctx; + u32 uctx_caps; + u32 uctx_uid; +}; + +struct cxlctl_dev { + struct fwctl_device fwctl; + struct cxl_mailbox *mbox; +}; + +DEFINE_FREE(cxlctl, struct cxlctl_dev *, if (_T) fwctl_put(&_T->fwctl)) + +static int cxlctl_open_uctx(struct fwctl_uctx *uctx) +{ + struct cxlctl_uctx *cxlctl_uctx = + container_of(uctx, struct cxlctl_uctx, uctx); + + cxlctl_uctx->uctx_caps = BIT(FWCTL_CXL_QUERY_COMMANDS) | + BIT(FWCTL_CXL_SEND_COMMAND); + + return 0; +} + +static void cxlctl_close_uctx(struct fwctl_uctx *uctx) +{ +} + +static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) +{ + struct cxlctl_uctx *cxlctl_uctx = + container_of(uctx, struct cxlctl_uctx, uctx); + struct fwctl_info_cxl *info; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return ERR_PTR(-ENOMEM); + + info->uctx_caps = cxlctl_uctx->uctx_caps; + + return info; +} + +static bool cxlctl_validate_set_features(struct cxl_mailbox *cxl_mbox, + const struct fwctl_cxl_command *send_cmd, + enum fwctl_rpc_scope scope) +{ + struct cxl_feat_entry *feat; + bool found = false; + uuid_t uuid; + u16 mask; + + if (send_cmd->in.size < sizeof(struct set_feature_input)) + return false; + + if (copy_from_user(&uuid, u64_to_user_ptr(send_cmd->in.payload), + sizeof(uuid))) + return false; + + for (int i = 0; i < cxl_mbox->num_features; i++) { + feat = &cxl_mbox->entries[i]; + if (uuid_equal(&uuid, &feat->uuid)) { + found = true; + break; + } + } + + if (!found) + return false; + + /* Currently no user background command support */ + if (feat->effects & CXL_CMD_BACKGROUND) + return false; + + mask = CXL_CMD_CONFIG_CHANGE_IMMEDIATE | + CXL_CMD_DATA_CHANGE_IMMEDIATE | + CXL_CMD_POLICY_CHANGE_IMMEDIATE | + CXL_CMD_LOG_CHANGE_IMMEDIATE; + if (feat->effects & mask && scope >= FWCTL_RPC_DEBUG_WRITE) + return true; + + /* These effects supported for all scope */ + if ((feat->effects & CXL_CMD_CONFIG_CHANGE_COLD_RESET || + feat->effects & CXL_CMD_CONFIG_CHANGE_CONV_RESET) && + scope >= FWCTL_RPC_DEBUG_READ_ONLY) + return true; + + return false; +} + +static bool cxlctl_validate_hw_cmds(struct cxl_mailbox *cxl_mbox, + const struct fwctl_cxl_command *send_cmd, + enum fwctl_rpc_scope scope) +{ + struct cxl_mem_command *cmd; + + /* + * Only supporting feature commands. + */ + if (!cxl_mbox->num_features) + return false; + + cmd = cxl_get_mem_command(send_cmd->id); + if (!cmd) + return false; + + if (test_bit(cmd->info.id, cxl_mbox->enabled_cmds)) + return false; + + if (test_bit(cmd->info.id, cxl_mbox->exclusive_cmds)) + return false; + + switch (cmd->opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + case CXL_MBOX_OP_GET_FEATURE: + if (scope >= FWCTL_RPC_DEBUG_READ_ONLY) + return true; + break; + case CXL_MBOX_OP_SET_FEATURE: + return cxlctl_validate_set_features(cxl_mbox, send_cmd, scope); + default: + return false; + }; + + return false; +} + +static bool cxlctl_validate_query_commands(struct fwctl_rpc_cxl *rpc_in) +{ + int cmds; + + if (rpc_in->payload_size < sizeof(rpc_in->query)) + return false; + + cmds = rpc_in->query.n_commands; + if (cmds) { + int cmds_size = rpc_in->payload_size - sizeof(rpc_in->query); + + if (cmds != cmds_size / sizeof(struct cxl_command_info)) + return false; + } + + return true; +} + +static bool cxlctl_validate_rpc(struct fwctl_uctx *uctx, + struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + struct cxlctl_dev *cxlctl = + container_of(uctx->fwctl, struct cxlctl_dev, fwctl); + + switch (rpc_in->rpc_cmd) { + case FWCTL_CXL_QUERY_COMMANDS: + return cxlctl_validate_query_commands(rpc_in); + + case FWCTL_CXL_SEND_COMMAND: + return cxlctl_validate_hw_cmds(cxlctl->mbox, + &rpc_in->send_cmd, scope); + + default: + return false; + } +} + +static void *send_cxl_command(struct cxl_mailbox *cxl_mbox, + struct fwctl_cxl_command *send_cmd, + size_t *out_len) +{ + struct cxl_mbox_cmd mbox_cmd; + int rc; + + rc = cxl_fwctl_send_cmd(cxl_mbox, send_cmd, &mbox_cmd, out_len); + if (rc) + return ERR_PTR(rc); + + *out_len = mbox_cmd.size_out; + + return mbox_cmd.payload_out; +} + +static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *in, size_t in_len, size_t *out_len) +{ + struct cxlctl_dev *cxlctl = + container_of(uctx->fwctl, struct cxlctl_dev, fwctl); + struct cxl_mailbox *cxl_mbox = cxlctl->mbox; + struct fwctl_rpc_cxl *rpc_in = in; + + if (!cxlctl_validate_rpc(uctx, rpc_in, scope)) + return ERR_PTR(-EPERM); + + switch (rpc_in->rpc_cmd) { + case FWCTL_CXL_QUERY_COMMANDS: + return cxl_query_cmd_from_fwctl(cxl_mbox, &rpc_in->query, + out_len); + + case FWCTL_CXL_SEND_COMMAND: + return send_cxl_command(cxl_mbox, &rpc_in->send_cmd, out_len); + + default: + return ERR_PTR(-EOPNOTSUPP); + } +} + +static const struct fwctl_ops cxlctl_ops = { + .device_type = FWCTL_DEVICE_TYPE_CXL, + .uctx_size = sizeof(struct cxlctl_uctx), + .open_uctx = cxlctl_open_uctx, + .close_uctx = cxlctl_close_uctx, + .info = cxlctl_info, + .fw_rpc = cxlctl_fw_rpc, +}; + +static int cxlctl_probe(struct auxiliary_device *adev, + const struct auxiliary_device_id *id) +{ + struct cxl_mailbox *mbox = container_of(adev, struct cxl_mailbox, adev); + struct cxlctl_dev *cxlctl __free(cxlctl) = + fwctl_alloc_device(mbox->host, &cxlctl_ops, + struct cxlctl_dev, fwctl); + int rc; + + if (!cxlctl) + return -ENOMEM; + + cxlctl->mbox = mbox; + + rc = fwctl_register(&cxlctl->fwctl); + if (rc) + return rc; + + auxiliary_set_drvdata(adev, no_free_ptr(cxlctl)); + + return 0; +} + +static void cxlctl_remove(struct auxiliary_device *adev) +{ + struct cxlctl_dev *ctldev __free(cxlctl) = auxiliary_get_drvdata(adev); + + fwctl_unregister(&ctldev->fwctl); +} + +static const struct auxiliary_device_id cxlctl_id_table[] = { + { .name = "CXL.fwctl", }, + {}, +}; +MODULE_DEVICE_TABLE(auxiliary, cxlctl_id_table); + +static struct auxiliary_driver cxlctl_driver = { + .name = "cxl_fwctl", + .probe = cxlctl_probe, + .remove = cxlctl_remove, + .id_table = cxlctl_id_table, +}; + +module_auxiliary_driver(cxlctl_driver); + +MODULE_IMPORT_NS(CXL); +MODULE_IMPORT_NS(FWCTL); +MODULE_DESCRIPTION("CXL fwctl driver"); +MODULE_AUTHOR("Intel Corporation"); +MODULE_LICENSE("GPL"); diff --git a/include/linux/cxl/mailbox.h b/include/linux/cxl/mailbox.h index 570864239b8f14..13b5bb6e5bc310 100644 --- a/include/linux/cxl/mailbox.h +++ b/include/linux/cxl/mailbox.h @@ -4,6 +4,7 @@ #define __CXL_MBOX_H__ #include +#include #include /** @@ -68,4 +69,107 @@ struct cxl_mailbox { struct cxl_feat_entry *entries; }; +enum cxl_opcode { + CXL_MBOX_OP_INVALID = 0x0000, + CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, + CXL_MBOX_OP_GET_EVENT_RECORD = 0x0100, + CXL_MBOX_OP_CLEAR_EVENT_RECORD = 0x0101, + CXL_MBOX_OP_GET_EVT_INT_POLICY = 0x0102, + CXL_MBOX_OP_SET_EVT_INT_POLICY = 0x0103, + CXL_MBOX_OP_GET_FW_INFO = 0x0200, + CXL_MBOX_OP_TRANSFER_FW = 0x0201, + CXL_MBOX_OP_ACTIVATE_FW = 0x0202, + CXL_MBOX_OP_GET_TIMESTAMP = 0x0300, + CXL_MBOX_OP_SET_TIMESTAMP = 0x0301, + CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400, + CXL_MBOX_OP_GET_LOG = 0x0401, + CXL_MBOX_OP_GET_LOG_CAPS = 0x0402, + CXL_MBOX_OP_CLEAR_LOG = 0x0403, + CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, + CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, + CXL_MBOX_OP_GET_FEATURE = 0x0501, + CXL_MBOX_OP_SET_FEATURE = 0x0502, + CXL_MBOX_OP_IDENTIFY = 0x4000, + CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, + CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, + CXL_MBOX_OP_GET_LSA = 0x4102, + CXL_MBOX_OP_SET_LSA = 0x4103, + CXL_MBOX_OP_GET_HEALTH_INFO = 0x4200, + CXL_MBOX_OP_GET_ALERT_CONFIG = 0x4201, + CXL_MBOX_OP_SET_ALERT_CONFIG = 0x4202, + CXL_MBOX_OP_GET_SHUTDOWN_STATE = 0x4203, + CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204, + CXL_MBOX_OP_GET_POISON = 0x4300, + CXL_MBOX_OP_INJECT_POISON = 0x4301, + CXL_MBOX_OP_CLEAR_POISON = 0x4302, + CXL_MBOX_OP_GET_SCAN_MEDIA_CAPS = 0x4303, + CXL_MBOX_OP_SCAN_MEDIA = 0x4304, + CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305, + CXL_MBOX_OP_SANITIZE = 0x4400, + CXL_MBOX_OP_SECURE_ERASE = 0x4401, + CXL_MBOX_OP_GET_SECURITY_STATE = 0x4500, + CXL_MBOX_OP_SET_PASSPHRASE = 0x4501, + CXL_MBOX_OP_DISABLE_PASSPHRASE = 0x4502, + CXL_MBOX_OP_UNLOCK = 0x4503, + CXL_MBOX_OP_FREEZE_SECURITY = 0x4504, + CXL_MBOX_OP_PASSPHRASE_SECURE_ERASE = 0x4505, + CXL_MBOX_OP_MAX = 0x10000 +}; + +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET (BIT(9) | BIT(10)) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET (BIT(9) | BIT(11)) + +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + u8 get_feat_ver; + u8 set_feat_ver; + __le16 effects; + u8 reserved[18]; +} __packed; + +/** + * struct cxl_mem_command - Driver representation of a memory device command + * @info: Command information as it exists for the UAPI + * @opcode: The actual bits used for the mailbox protocol + * @flags: Set of flags effecting driver behavior. + * + * * %CXL_CMD_FLAG_FORCE_ENABLE: In cases of error, commands with this flag + * will be enabled by the driver regardless of what hardware may have + * advertised. + * + * The cxl_mem_command is the driver's internal representation of commands that + * are supported by the driver. Some of these commands may not be supported by + * the hardware. The driver will use @info to validate the fields passed in by + * the user then submit the @opcode to the hardware. + * + * See struct cxl_command_info. + */ +struct cxl_mem_command { + struct cxl_command_info info; + enum cxl_opcode opcode; + u32 flags; +#define CXL_CMD_FLAG_FORCE_ENABLE BIT(0) +}; + +struct cxl_mem_command *cxl_get_mem_command(u32 id); +int cxl_fwctl_send_cmd(struct cxl_mailbox *cxl_mbox, + struct fwctl_cxl_command *fwctl_cmd, + struct cxl_mbox_cmd *mbox_cmd, + size_t *out_len); +void *cxl_query_cmd_from_fwctl(struct cxl_mailbox *cxl_mbox, + struct cxl_mem_query_commands *q, + size_t *out_len); + #endif diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h new file mode 100644 index 00000000000000..a3a96195f6da4c --- /dev/null +++ b/include/uapi/fwctl/cxl.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024, Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_FWCTL_CXL_H_ +#define _UAPI_FWCTL_CXL_H_ + +#include + +enum fwctl_cxl_commands { + FWCTL_CXL_QUERY_COMMANDS = 0, + FWCTL_CXL_SEND_COMMAND, +}; + +/** + * struct fwctl_info_cxl - ioctl(FWCTL_INFO) out_device_data + * @uctx_caps: The command capabilities driver accepts. + * + * Return basic information about the FW interface available. + */ +struct fwctl_info_cxl { + __u32 uctx_caps; +}; + +/* + * CXL spec r3.1 Table 8-101 Set Feature Input Payload + */ +struct set_feature_input { + __u8 uuid[16]; + __u32 flags; + __u16 offset; + __u8 version; + __u8 reserved[9]; + __u8 data[]; +} __packed; + +/** + * struct cxl_send_command - Send a command to a memory device. + * @id: The command to send to the memory device. This must be one of the + * commands returned by the query command. + * @flags: Flags for the command (input). + * @raw: Special fields for raw commands + * @raw.opcode: Opcode passed to hardware when using the RAW command. + * @raw.rsvd: Must be zero. + * @rsvd: Must be zero. + * @retval: Return value from the memory device (output). + * @in: Parameters associated with input payload. + * @in.size: Size of the payload to provide to the device (input). + * @in.rsvd: Must be zero. + * @in.payload: Pointer to memory for payload input, payload is little endian. + * + * Output payload is defined with 'struct fwctl_rpc' and is the hardware output + */ +struct fwctl_cxl_command { + __u32 id; + __u32 flags; + union { + struct { + __u16 opcode; + __u16 rsvd; + } raw; + __u32 rsvd; + }; + + struct { + __u32 size; + __u32 rsvd; + __u64 payload; + } in; +}; + +/** + * struct fwctl_rpc_cxl - ioctl(FWCTL_RPC) input + */ +struct fwctl_rpc_cxl { + __u32 rpc_cmd; + __u32 payload_size; + __u32 version; + __u32 rsvd; + union { + struct cxl_mem_query_commands query; + struct fwctl_cxl_command send_cmd; + }; +}; + +struct fwctl_rpc_cxl_out { + __u32 retval; + __u32 rsvd; + __u8 payload[]; +}; + +#endif diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index f9b27fb5c1618c..4e4d30104667c7 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -43,6 +43,7 @@ enum { enum fwctl_device_type { FWCTL_DEVICE_TYPE_ERROR = 0, FWCTL_DEVICE_TYPE_MLX5 = 1, + FWCTL_DEVICE_TYPE_CXL = 2, }; /** From patchwork Wed Aug 21 18:11:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13771954 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2068.outbound.protection.outlook.com [40.107.101.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 614721B3B38; Wed, 21 Aug 2024 18:11:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263879; cv=fail; b=H7x/KTflT/DRzU5OJdNPU35Mz3etgTyT4Nyqb1rgWirUvGgUUhIhN49IDCekBcwTmoHE3ICCb0AljsJkkix26r/aHE+llkpJeW+d3G9e88xfNRRL63SftdTswBifd1ax+fY4ZZhpfldifOtGdZFg/OQ698LCbtmhJwPro6+ExKQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724263879; c=relaxed/simple; bh=USlx7YBAbBV3ZbwmRWo20Jf9Zxz6LnS7OFYdml5zYfM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=EbONeJB0Se0VpkdqMUVVM8yYf4BgngUqOvN1rgkDo8GM3zeZY30KUpg0VcLZTg+vYGGsFtwqPcL/mySnseTsE/KWT10EAZj45f3QGeH7qkJD7zITH5XNRd0mrNdf48WPkhutKaR989VoVDZM0XRtnV5bOL66B+A2ODc8lTuetQE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=sO+M4j/B; arc=fail smtp.client-ip=40.107.101.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="sO+M4j/B" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EgfTdFm/v0VxmEzZZAfOf44XaQW+I7QGPSPJi7op8C0bdkOmUOBxcSy7JOdLqwaYQECMAHYd434FnytYI4zcjuqqxqkBHHxncu7IaweaiyJ987dc7g87mhFEakHK/PWVgEjhWVqs87X5N7WN26HsbJaNGQVgBkzq5jbh6ZVHGbxnemnSiOdePdSVPu1BLOrkvnV1ZmsNYtdGNsIenLQfQ8zvGHEndSw4GHUOfnft6rCdXJ855gFXpMd7lKW3VcZOP0OBNlCr64ZkFPosFIC1T3e3ItOlHR29X4e4gFmKzq6S0IDarnPKQuJEJLppBxuiTwWsxNbEfGdpLxw+h+YYZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GIt3ge5f+/l3Iay/sGuncebBG61co+3+9bM0Kcnrt0A=; b=PdCNf9fpa1fm6/SFf0T1p/77Bud14iiEBRWK5QQr5QVRd3b1Q1Ak/iOPZzVC5TukEOd30w7EBwu9xDD7yxhhdZ8LZQtHJzeJFyIcL+uX1jErBsMFkQAlo+PXGV8/ytqbWdThCrlU0TlxiwFXD1bvZXKGFSkC0PfYPXgy/YnZ6RFAa6SPxXm3DTy654n0lXpql/DYjc09PT48i1WKxMQi/Yyv3ChuPwv5iVy5ztNX/rX2/K+E2XSg4mIYMkaPv/IvoXd1/l68fPP15vvwMKiJ9qGBTyh5eNiVCsIu0clQlW21en7RZ73wlWNZBbZaGIJVBobfa1YBgYEEt2m+/y8SUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GIt3ge5f+/l3Iay/sGuncebBG61co+3+9bM0Kcnrt0A=; b=sO+M4j/BoEOLIz630vNSzhyNo1+EUA3RSP/k3EAXEVXFpcW2ZZEOfgvTW31YjOsMkTpOgPknV9nS7BRFwwiTvuUc37t/8MyhgfKtSmARpY23UVTcGDTcIBnZVzytGOfbJfdHAxzNHiHgIhgKKuXePLidLEoZxxoAxyH2H8USmWkHXZNm3NJdjf0o0Pal3AmVt4UFfcxv3yiYZZI0QYeG3DzOyHJq8Q6MuimL++V2PHDmsug6Xw6sYD8Y2F4w5aNugUijiTZ6Op8Qytrl3xuNiEp03gf0+Xo4W/69rvIuErtqds/GuS5U0z97FZrF17EuFTWeHEhBg0clu6sgIScsog== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SA1PR12MB7199.namprd12.prod.outlook.com (2603:10b6:806:2bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.17; Wed, 21 Aug 2024 18:11:08 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.023; Wed, 21 Aug 2024 18:11:08 +0000 From: Jason Gunthorpe To: Cc: Andy Gospodarek , Aron Silverton , Dan Williams , Daniel Vetter , Dave Jiang , David Ahern , Greg Kroah-Hartman , Christoph Hellwig , Itay Avraham , Jiri Pirko , Jakub Kicinski , Leonid Bloch , Leon Romanovsky , linux-cxl@vger.kernel.org, linux-rdma@vger.kernel.org, Saeed Mahameed Subject: [PATCH v3 10/10] cxl: Create an auxiliary device for fwctl_cxl Date: Wed, 21 Aug 2024 15:11:02 -0300 Message-ID: <10-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> In-Reply-To: <0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0035.namprd03.prod.outlook.com (2603:10b6:408:fb::10) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SA1PR12MB7199:EE_ X-MS-Office365-Filtering-Correlation-Id: 0859e93c-ee97-4f6e-2ae9-08dcc20c9ef9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: gibgDFBjITM/pMtZUvWDAsaAH96fCDKD3o5Yp7FeFLcjjaHbobblI8t89O9pGYXveab3H4bC7kMa6iyVEpv8tZzCzvO02Ic7hOtjX7K6PSgt2ygEyLnafsxZ7JEFQuIllFGTUQ/SzaGnH7cMn11/40X3TvpLIeFfHlTXJua6ze8CQXQqxsuTQvwE1y6eeGSIryRh/5fcitRgPUkledfX05Z7ud6az9dj8+4Ft5g9dSp6OcDNqdWMIbqWTMmEfqvcIEmdoxzCfEmBsNAfzmsZIDSHtXBvXw2dJxWPxR/xGa4NWl8B+CuwtlNMOZdEv6S37zwLK90hO3LZOPEKARAaeDYl4DnD8j7F4QPQoZfiHzgqZd295V2295OUuOLrDKNknEWPMgBW5ZPfEcBPC/Ja9AqSW8ekGLlh11mwaoGNaaz+3cgyaUFpHkf3Zp8QtFVYdzRZ1FTh2JcX63zDQh3zqbOTK1UtZknlpdcTgbzwMbVheeBys8SavkSqZ4EVzCcRMnniIu2yn5PEkEt9gKQPTqtSRmjbUKfI4vMVCzkfizP8NLHhKZmOBdPjRIizHVVVfWbbsFx/WMJ2D4rQA+QqRPRRfKmLONhmMBV1Q7vQEGC+UETyuC4YDCk2pLDOKWHXANQ5dFeFKshO3MfnVxWUKu0Z/VsBCGkYG12CNHBdIpWKqQz9q2QUZq/k3+mTBM+GtmIQq3Te3X6F6PltMN1DZxY8gH7k+bDVxl3nTxsunNoFSoiLy+AvAlPjK9lHaJ4FTGCIbPt3XSWWQ4jWZ90SBF5eUI26bLExhQGkXTQSr7EZZrqNT+g+eYltyiH1rmM76hBuzicwz8OMk0RbrOkx3HAelaFgJbd7xt9ViG3nzNT+JuSRGlpvRgv5GH0DfOK4v0uvLpGL+l315BaVcNN118T4jobwFw26T35rCXibzZgMFM051HK0LYcHCLTENHN2csp6UjNi9lxdBFFcEHsnGOCA4GAOZh8XpiQf+9H8ViNpJDVrnPrFPJPOGSkuHHQpJrtIcGEEl7NpCmf3flhoGoVhyuZHRnC46rrfQ7wd9XAN1kO0yJGF95wRwlU+Lz8NVmR8pBOnTsjCuworUxjOa1WBEYKwcE4LlwoqVylPgi4WmCGUGjmzclViP9zI2nu3kxOwc3eimQrPUTjZD2+MKkNaliRmqh/qH/laZhITAiXpIg9WZLo8qzP+LHDYLuVhnH0UlXecr5ER4NvA8pXDxFELhbV8m+o6Ukm8FCXJkdykUsmaalGgIVLoFbVHGYe81pD3R6l8BRLt9U9rqaRfFyusyr5KLo84GrDkGeOAQoKim5hj9ezAXcNjAkY7dDLHxBygcKVOuxAXlm+HAg9WQg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: l0gSzONR8mc+X5Ev/L0ZxhAYXmty3JHgMCcp3T1FYfh8n51mfrxnjxaZXiHDogoYBpD/Z4dJB0TyijtLHpCH51G7zeZ4JpUk01QSx+Ak34Xj2eiqb5GGsX+yui6aUwJByPD7IMWBGy4SJEq5+AMsTM/mXdhwlKp92CZI1uBh3+hib91sD61B1WP6Hxd53iiq10RdtX6wJUCKHS/+ud23o6ygPJsGEUx2HinmVQht8fuZ24wqLyFvEdSPjgRComDbXzH0uh1bz76kSGiuo9FIFo3365LW/hLRtLP3AcsTyF+IsMccn60JN2UgyL7zSw3GlBrMMN2YL2JXSpiiYam41Sefrjw+p+jghgrHAzPWMce31cFg0FMe1NbxMnmR/+RpT1QZk/rRcrZkx5TrQpCnPZ5TPIQC8N6EeAgQSCqnN7kMjXnwJ9Vpm+iEI0ycpRGDXpEXY9KjinvFKed3eeS2F7mYtaHPcRZRLMSDblNDG2dr7A93Ao2zjfmRopEigkOX5YmreEAubvt7OSnha4vt60kRfna1ITjYE0gem18fxiZ+Z5PI2v4d9JNW5oEAFyi189ySvuTGoyMl+hUT91M5pK/SCb61i2S9/FQaK6n1kjz6S9WBc424sGGbfKw8iWct39k4L4ayZh/3wCFhtepTFGzBtX7PdVMmthOnDbtEeAEfh0ivexLaD4rofKTc7d8Smq7hQJeNtxSAmvH33gP9KleEoLMfVn16Rvlr+0MOZJpPYYg/ZZyZDYHEBQCrThEbYFKFaYTdIIuw4s63s73Tklh92IDJVs1EQ1E8VRcUNHvj8SITELS21lVsLD+iPgY0DVAvnX9dQ+GWGSuIiUt8Ia3fbXlBISJUQKaZX5Wj7yxsQOKKm60nn67M7ROyNroxk3LY9E8LWalOIz5q5QiVLJoWGYKLtARsPe5LPqO9iTeHIqHhUyB+SV6s1HM0umeHUd1Ls6YFvShlp/fazubJdz03I/LVI/ZCa+3/LcXGgXPuEVSWu8Jn2PjvJz1hQZeYIVc+LEZvWmi8VeUqjB46iFH99fJdsqEgmskhXqwenJwMjIgW/2baHw/wvnSbnFE9MyQPvk1deai5bx63HqzYdYVugPOWCD2StTDXkQhTr/jQBg3EluKfQD3dllhhcL3kZ/BG0WsfZA7dkZdv7yAZUB38UeFFXJgtFGyNGZcs3ukDwN8XRsDvBklS9ZKhMpvV1Uo+jD/lOz730FWkaFz/PxjYonZobUaPSo1NlFnxZRF/09dWyn7mK7fEJqFuAzNuAboYg7n18u5TNXk3IgWK5vl/zERNOUV5uo4hdPHKUeiIgyyihaVgDRmRKZAiTGFj6U8+aYOfdr/3nq3OwaVGSZdVGOLCzkOXciG+qln/I/r/e02DOY9QOZBtTFeoE10dhtrvblu9ov1I1FQmX3S42T4WXI0eWihJflD2Qm9RF744RWHF2Sk+8SEYGPNRJY1nklxOLVYwGhxTpy6tXEKR6jPO3LZAVQawb3zE28HZHlK9lVEcR/aO1KXDySf/592JAASM3AsGLqhYDX3OcxR9pKYILOJLAFRUyFTadkyZlBWjtk3GZUyW0Ves8M3biJJ3 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0859e93c-ee97-4f6e-2ae9-08dcc20c9ef9 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 18:11:04.3465 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GBPfCgoAKpiB1MAXkmqNzflanCKQtUFqlOzFDuHBbYayNf6gsb/ley2015Gd6zDX X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7199 This will link the fwctl subsystem to CXL devices. Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- drivers/cxl/core/memdev.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 15c36971fe43ea..f6f33f0f733741 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -13,6 +13,8 @@ DECLARE_RWSEM(cxl_memdev_rwsem); +#define CXL_ADEV_NAME "fwctl-cxl" + /* * An entire PCI topology full of devices should be enough for any * config @@ -1030,6 +1032,7 @@ static const struct file_operations cxl_memdev_fops = { struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds) { + struct auxiliary_device *adev; struct cxl_memdev *cxlmd; struct device *dev; struct cdev *cdev; @@ -1056,11 +1059,27 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device *host, if (rc) goto err; + adev = &cxlds->cxl_mbox.adev; + adev->id = cxlmd->id; + adev->name = CXL_ADEV_NAME; + adev->dev.parent = dev; + + rc = auxiliary_device_init(adev); + if (rc) + goto err; + + rc = auxiliary_device_add(adev); + if (rc) + goto aux_err; + rc = devm_add_action_or_reset(host, cxl_memdev_unregister, cxlmd); if (rc) return ERR_PTR(rc); return cxlmd; +aux_err: + auxiliary_device_uninit(adev); + err: /* * The cdev was briefly live, shutdown any ioctl operations that