From patchwork Fri Mar 7 20:55:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007134 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 597CE257446 for ; Fri, 7 Mar 2025 20:56:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381013; cv=none; b=fTdwBzDb/sW14lCv5AFiUeZacoorQzvmLdeHMk5fBiCGmWlPGg6eRdfQvsqeBfSPpCu1qsyilcrcKd/v6mwqRT3hwdf4GLqdVdjvqeHuTWreOdX/3WkNNuR4G5navOqvP2sGq0jUiyeUSY70EQc3aM6priZ7QDMme9Uwa3LAw4g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381013; c=relaxed/simple; bh=CfnW+XYc9PP+9vWm/U+TrbGGKsBi42OW2JZdtuo5ndQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TGPcajKuQ03NWs/eFg80YS9xeUuZdpWSHV/lmvhS9FlZ+vt7A+VVqgG/7ahmoa3zytDpjILRy776kc1fbjwfXi5ifo1OzQwnrBP1ts+JxO8sW/3HPT7OJh4ockgfj4u/9vQqdTTjUR6aB6mzAg6xk9JrU9dJRV4ALZ3Qw9aPi84= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB018C4CEE2; Fri, 7 Mar 2025 20:56:52 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Li Ming , Jonathan Cameron Subject: [PATCH v8 1/9] cxl: Add FWCTL support to CXL Date: Fri, 7 Mar 2025 13:55:31 -0700 Message-ID: <20250307205648.1021626-2-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add fwctl support code to allow sending of CXL feature commands from userspace through as ioctls via FWCTL. Provide initial setup bits. The CXL PCI probe function will call devm_cxl_setup_fwctl() after the cxl_memdev has been enumerated in order to setup FWCTL char device under the cxl_memdev like the existing memdev char device for issuing CXL raw mailbox commands from userspace via ioctls. Link: https://patch.msgid.link/r/20250220194438.2281088-8-dave.jiang@intel.com Signed-off-by: Dave Jiang Reviewed-by: Li Ming Reviewed-by: Jonathan Cameron Signed-off-by: Jason Gunthorpe --- drivers/cxl/Kconfig | 1 + drivers/cxl/core/features.c | 74 ++++++++++++++++++++++++++++++++++++ drivers/cxl/pci.c | 4 ++ include/cxl/features.h | 10 +++++ include/uapi/fwctl/fwctl.h | 1 + tools/testing/cxl/test/mem.c | 4 ++ 6 files changed, 94 insertions(+) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index ad2e796e4ac6..2b6e662c89bf 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -105,6 +105,7 @@ config CXL_MEM config CXL_FEATURES bool "CXL: Features" depends on CXL_PCI + select FWCTL help Enable support for CXL Features. A CXL device that includes a mailbox supports commands that allows listing, getting, and setting of diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 048ba4fc3538..846de3294a5e 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2024-2025 Intel Corporation. All rights reserved. */ +#include #include #include #include @@ -331,3 +332,76 @@ int cxl_set_feature(struct cxl_mailbox *cxl_mbox, } } while (true); } + +/* FWCTL support */ + +static inline struct cxl_memdev *fwctl_to_memdev(struct fwctl_device *fwctl_dev) +{ + return to_cxl_memdev(fwctl_dev->dev.parent); +} + +static int cxlctl_open_uctx(struct fwctl_uctx *uctx) +{ + return 0; +} + +static void cxlctl_close_uctx(struct fwctl_uctx *uctx) +{ +} + +static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *in, size_t in_len, size_t *out_len) +{ + /* Place holder */ + return ERR_PTR(-EOPNOTSUPP); +} + +static const struct fwctl_ops cxlctl_ops = { + .device_type = FWCTL_DEVICE_TYPE_CXL, + .uctx_size = sizeof(struct fwctl_uctx), + .open_uctx = cxlctl_open_uctx, + .close_uctx = cxlctl_close_uctx, + .fw_rpc = cxlctl_fw_rpc, +}; + +DEFINE_FREE(free_fwctl_dev, struct fwctl_device *, if (_T) fwctl_put(_T)) + +static void free_memdev_fwctl(void *_fwctl_dev) +{ + struct fwctl_device *fwctl_dev = _fwctl_dev; + + fwctl_unregister(fwctl_dev); + fwctl_put(fwctl_dev); +} + +int devm_cxl_setup_fwctl(struct cxl_memdev *cxlmd) +{ + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_features_state *cxlfs; + int rc; + + cxlfs = to_cxlfs(cxlds); + if (!cxlfs) + return -ENODEV; + + /* No need to setup FWCTL if there are no user allowed features found */ + if (!cxlfs->entries->num_user_features) + return -ENODEV; + + struct fwctl_device *fwctl_dev __free(free_fwctl_dev) = + _fwctl_alloc_device(&cxlmd->dev, &cxlctl_ops, sizeof(*fwctl_dev)); + if (!fwctl_dev) + return -ENOMEM; + + rc = fwctl_register(fwctl_dev); + if (rc) + return rc; + + cxlfs->fwctl_dev = fwctl_dev; + + return devm_add_action_or_reset(&cxlmd->dev, free_memdev_fwctl, + no_free_ptr(fwctl_dev)); +} +EXPORT_SYMBOL_NS_GPL(devm_cxl_setup_fwctl, "CXL"); + +MODULE_IMPORT_NS("FWCTL"); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 3e666ec51580..993fa60fe453 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1013,6 +1013,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + rc = devm_cxl_setup_fwctl(cxlmd); + if (rc) + dev_dbg(&pdev->dev, "No CXL FWCTL setup\n"); + pmu_count = cxl_count_regblock(pdev, CXL_REGLOC_RBI_PMU); if (pmu_count < 0) return pmu_count; diff --git a/include/cxl/features.h b/include/cxl/features.h index d2cde46b0fec..fce3c2dbde5a 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -4,6 +4,7 @@ #define __CXL_FEATURES_H__ #include +#include /* Feature UUIDs used by the kernel */ #define CXL_FEAT_PATROL_SCRUB_UUID \ @@ -162,6 +163,7 @@ enum cxl_set_feat_flag_data_transfer { * @entries: CXl feature entry context * @num_features: total Features supported by the device * @ent: Flex array of Feature detail entries from the device + * @fwctl_dev: Firmware Control device */ struct cxl_features_state { struct cxl_dev_state *cxlds; @@ -170,12 +172,15 @@ struct cxl_features_state { int num_user_features; struct cxl_feat_entry ent[] __counted_by(num_features); } *entries; + struct fwctl_device *fwctl_dev; }; struct cxl_mailbox; +struct cxl_memdev; #ifdef CONFIG_CXL_FEATURES inline struct cxl_features_state *to_cxlfs(struct cxl_dev_state *cxlds); int devm_cxl_setup_features(struct cxl_dev_state *cxlds); +int devm_cxl_setup_fwctl(struct cxl_memdev *cxlmd); #else static inline struct cxl_features_state *to_cxlfs(struct cxl_dev_state *cxlds) { @@ -186,6 +191,11 @@ static inline int devm_cxl_setup_features(struct cxl_dev_state *cxlds) { return -EOPNOTSUPP; } + +static inline int devm_cxl_setup_fwctl(struct cxl_memdev *cxlmd) +{ + return -EOPNOTSUPP; +} #endif #endif diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 584a5ea8ecee..c2d5abc5a726 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -43,6 +43,7 @@ enum { enum fwctl_device_type { FWCTL_DEVICE_TYPE_ERROR = 0, FWCTL_DEVICE_TYPE_MLX5 = 1, + FWCTL_DEVICE_TYPE_CXL = 2, }; /** diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 4809a90ff9b6..848db399102c 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1646,6 +1646,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) return rc; + rc = devm_cxl_setup_fwctl(cxlmd); + if (rc) + dev_dbg(dev, "No CXL FWCTL setup\n"); + cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); return 0; From patchwork Fri Mar 7 20:55:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007135 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77F722561BD for ; Fri, 7 Mar 2025 20:56:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381014; cv=none; b=rGwogU2S1wEteUFkp6357jTlBcMwiyNotaAivDstIOg4h800JRR5cUOJXnmRK+jiBh6WzbEaCsf0BOeQQoi+bKWm6yNoFX42FoLB9e/i0aodEjowI5BYOPhIjcaDIQPKdtWQe3qZxihhrRxKO8fB2NHOf4gDHGk+sRs1AJqCG0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381014; c=relaxed/simple; bh=8ESHOmURfQBtDTegIT64+hTGjFeQRd1oySYeCKnKtbA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jleOFNlKvKjwNzvCWh3zr5LH9aW7sTT1XmJpqFuC5pljxvM5M0KpmpjsKrg9ep9vc3DU4EyhsKe2txfLGMaZ0ns6J2aeDcLIHJCNWOxbynmq2M08srEpO8QyYVrd0SZQkQ7NXFEbzvhgXqW/Vgo0kRSg4eGJK5DLqgYEvheWp8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36614C4CED1; Fri, 7 Mar 2025 20:56:54 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Dan Williams , Li Ming Subject: [PATCH v8 2/9] cxl: Move cxl feature command structs to user header Date: Fri, 7 Mar 2025 13:55:32 -0700 Message-ID: <20250307205648.1021626-3-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation for cxl fwctl enabling, move data structures related to cxl feature commands to a user header file. Reviewed-by; Jonathan Cameron Link: https://patch.msgid.link/r/20250220194438.2281088-9-dave.jiang@intel.com Reviewed-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- - Add kdoc updates --- include/cxl/features.h | 112 +----------------------- include/uapi/cxl/features.h | 169 ++++++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+), 111 deletions(-) create mode 100644 include/uapi/cxl/features.h diff --git a/include/cxl/features.h b/include/cxl/features.h index fce3c2dbde5a..ead63573b0b4 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -5,6 +5,7 @@ #include #include +#include /* Feature UUIDs used by the kernel */ #define CXL_FEAT_PATROL_SCRUB_UUID \ @@ -46,117 +47,6 @@ enum cxl_features_capability { CXL_FEATURES_RW, }; -/* Get Supported Features (0x500h) CXL r3.2 8.2.9.6.1 */ -struct cxl_mbox_get_sup_feats_in { - __le32 count; - __le16 start_idx; - u8 reserved[2]; -} __packed; - -/* CXL spec r3.2 Table 8-87 command effects */ -#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) -#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) -#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) -#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) -#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) -#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) -#define CXL_CMD_BACKGROUND BIT(6) -#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) -#define CXL_CMD_EFFECTS_VALID BIT(9) -#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) -#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) - -/* - * CXL spec r3.2 Table 8-109 - * Get Supported Features Supported Feature Entry - */ -struct cxl_feat_entry { - uuid_t uuid; - __le16 id; - __le16 get_feat_size; - __le16 set_feat_size; - __le32 flags; - u8 get_feat_ver; - u8 set_feat_ver; - __le16 effects; - u8 reserved[18]; -} __packed; - -/* @flags field for 'struct cxl_feat_entry' */ -#define CXL_FEATURE_F_CHANGEABLE BIT(0) -#define CXL_FEATURE_F_PERSIST_FW_UPDATE BIT(4) -#define CXL_FEATURE_F_DEFAULT_SEL BIT(5) -#define CXL_FEATURE_F_SAVED_SEL BIT(6) - -/* - * CXL spec r3.2 Table 8-108 - * Get supported Features Output Payload - */ -struct cxl_mbox_get_sup_feats_out { - __struct_group(cxl_mbox_get_sup_feats_out_hdr, hdr, /* no attrs */, - __le16 num_entries; - __le16 supported_feats; - __u8 reserved[4]; - ); - struct cxl_feat_entry ents[] __counted_by_le(num_entries); -} __packed; - -/* - * Get Feature CXL spec r3.2 Spec 8.2.9.6.2 - */ - -/* - * Get Feature input payload - * CXL spec r3.2 section 8.2.9.6.2 Table 8-99 - */ -struct cxl_mbox_get_feat_in { - uuid_t uuid; - __le16 offset; - __le16 count; - u8 selection; -} __packed; - -/* Selection field for 'struct cxl_mbox_get_feat_in' */ -enum cxl_get_feat_selection { - CXL_GET_FEAT_SEL_CURRENT_VALUE, - CXL_GET_FEAT_SEL_DEFAULT_VALUE, - CXL_GET_FEAT_SEL_SAVED_VALUE, - CXL_GET_FEAT_SEL_MAX -}; - -/* - * Set Feature CXL spec r3.2 8.2.9.6.3 - */ - -/* - * Set Feature input payload - * CXL spec r3.2 section 8.2.9.6.3 Table 8-101 - */ -struct cxl_mbox_set_feat_in { - __struct_group(cxl_mbox_set_feat_hdr, hdr, /* no attrs */, - uuid_t uuid; - __le32 flags; - __le16 offset; - u8 version; - u8 rsvd[9]; - ); - __u8 feat_data[]; -} __packed; - -/* Set Feature flags field */ -enum cxl_set_feat_flag_data_transfer { - CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER = 0, - CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX -}; - -#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) - -#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) - /** * struct cxl_features_state - The Features state for the device * @cxlds: Pointer to CXL device state diff --git a/include/uapi/cxl/features.h b/include/uapi/cxl/features.h new file mode 100644 index 000000000000..7f7ab4aefec2 --- /dev/null +++ b/include/uapi/cxl/features.h @@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024,2025, Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_CXL_FEATURES_H_ +#define _UAPI_CXL_FEATURES_H_ + +#include +#ifndef __KERNEL__ +#include +#else +#include +#endif + +/* + * struct cxl_mbox_get_sup_feats_in - Get Supported Features input + * + * @count: bytes of Feature data to return in output + * @start_idx: index of first requested Supported Feature Entry, 0 based. + * @reserved: reserved field, must be 0s. + * + * Get Supported Features (0x500h) CXL r3.2 8.2.9.6.1 command. + * Input block for Get support Feature + */ +struct cxl_mbox_get_sup_feats_in { + __le32 count; + __le16 start_idx; + __u8 reserved[2]; +} __attribute__ ((__packed__)); + +/* CXL spec r3.2 Table 8-87 command effects */ +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_EFFECTS_VALID BIT(9) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) + +/* + * struct cxl_feat_entry - Supported Feature Entry + * @uuid: UUID of the Feature + * @id: id to identify the feature. 0 based + * @get_feat_size: max bytes required for Get Feature command for this Feature + * @set_feat_size: max bytes required for Set Feature command for this Feature + * @flags: attribute flags + * @get_feat_ver: Get Feature version + * @set_feat_ver: Set Feature version + * @effects: Set Feature command effects + * @reserved: reserved, must be 0 + * + * CXL spec r3.2 Table 8-109 + * Get Supported Features Supported Feature Entry + */ +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + __u8 get_feat_ver; + __u8 set_feat_ver; + __le16 effects; + __u8 reserved[18]; +} __attribute__ ((__packed__)); + +/* @flags field for 'struct cxl_feat_entry' */ +#define CXL_FEATURE_F_CHANGEABLE BIT(0) +#define CXL_FEATURE_F_PERSIST_FW_UPDATE BIT(4) +#define CXL_FEATURE_F_DEFAULT_SEL BIT(5) +#define CXL_FEATURE_F_SAVED_SEL BIT(6) + +/* + * struct cxl_mbox_get_sup_feats_out - Get Supported Features output + * @num_entries: number of Supported Feature Entries returned + * @supported_feats: number of supported Features + * @reserved: reserved, must be 0s. + * @ents: Supported Feature Entries array + * + * CXL spec r3.2 Table 8-108 + * Get supported Features Output Payload + */ +struct cxl_mbox_get_sup_feats_out { + __struct_group(cxl_mbox_get_sup_feats_out_hdr, hdr, /* no attrs */, + __le16 num_entries; + __le16 supported_feats; + __u8 reserved[4]; + ); + struct cxl_feat_entry ents[] __counted_by_le(num_entries); +} __attribute__ ((__packed__)); + +/* + * Get Feature CXL spec r3.2 Spec 8.2.9.6.2 + */ + +/* + * struct cxl_mbox_get_feat_in - Get Feature input + * @uuid: UUID for Feature + * @offset: offset of the first byte in Feature data for output payload + * @count: count in bytes of Feature data returned + * @selection: 0 current value, 1 default value, 2 saved value + * + * CXL spec r3.2 section 8.2.9.6.2 Table 8-99 + */ +struct cxl_mbox_get_feat_in { + uuid_t uuid; + __le16 offset; + __le16 count; + __u8 selection; +} __attribute__ ((__packed__)); + +/* + * enum cxl_get_feat_selection - selection field of Get Feature input + */ +enum cxl_get_feat_selection { + CXL_GET_FEAT_SEL_CURRENT_VALUE, + CXL_GET_FEAT_SEL_DEFAULT_VALUE, + CXL_GET_FEAT_SEL_SAVED_VALUE, + CXL_GET_FEAT_SEL_MAX +}; + +/* + * Set Feature CXL spec r3.2 8.2.9.6.3 + */ + +/* + * struct cxl_mbox_set_feat_in - Set Features input + * @uuid: UUID for Feature + * @flags: set feature flags + * @offset: byte offset of Feature data to update + * @version: Feature version of the data in Feature Data + * @rsvd: reserved, must be 0s. + * @feat_data: raw byte stream of Features data to update + * + * CXL spec r3.2 section 8.2.9.6.3 Table 8-101 + */ +struct cxl_mbox_set_feat_in { + __struct_group(cxl_mbox_set_feat_hdr, hdr, /* no attrs */, + uuid_t uuid; + __le32 flags; + __le16 offset; + __u8 version; + __u8 rsvd[9]; + ); + __u8 feat_data[]; +} __packed; + +/* + * enum cxl_set_feat_flag_data_transfer - Set Feature flags field + */ +enum cxl_set_feat_flag_data_transfer { + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER = 0, + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX +}; + +#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) +#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) + +#endif From patchwork Fri Mar 7 20:55:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007136 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 678F725745C for ; Fri, 7 Mar 2025 20:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381016; cv=none; b=dmE7LWgYKvoRNHwHZm9sp+3+GGLzUlO7QsTK2MYiaY1+QMmKSzLmuSr5MGppLIh2UvokD7JxJiOXpTqAQWjFNRUxu6lyOL7edCsYDLrbUEx23KDudDJiAFjEnl1VH5n9AUzG0wJ+i8Gz0WDGv9sqJ7mUhduwld56x4lniP7ubBg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381016; c=relaxed/simple; bh=+T95i0YHvEg01b2XXlfdSVmKoreJyaQrweXAx1pscqI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VgFvR/CU7Qvlm/nVCPXW1q/mnEhFOLPYIHjb45huPI/rEP89ZZNeVPk5zM3apYXUOz6srPodYhoK9Wybe009MYRghIjEkqXrdcL0NKxvA5os2AatI3NsZT9P1isvcQmKFm2fmGh9pqhOn38lFwNIn2f/bre7Rgfevmk3qdosEGM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id A552AC4CED1; Fri, 7 Mar 2025 20:56:55 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Dan Williams , Jonathan Cameron , Li Ming Subject: [PATCH v8 3/9] cxl: Add support for fwctl RPC command to enable CXL feature commands Date: Fri, 7 Mar 2025 13:55:33 -0700 Message-ID: <20250307205648.1021626-4-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 fwctl provides a fwctl_ops->fw_rpc() callback in order to issue ioctls to a device. The cxl fwctl driver will start by supporting the CXL Feature commands: Get Supported Features, Get Feature, and Set Feature. The fw_rpc() callback provides 'enum fwctl_rpc_scope' parameter where it indicates the security scope of the call. The Get Supported Features and Get Feature calls can be executed with the scope of FWCTL_RPC_CONFIGRATION. The Set Feature call is gated by the effects of the Feature reported by Get Supported Features call for the specific Feature. Only "Get Supported Features" is supported in this patch. Additional commands will be added in follow on patches. "Get Supported Features" will filter the Features that are exclusive to the kernel. The flag field of the Feature details will be cleared of the "Changeable" field and the "set feat size" will be set to 0 to indicate that the feature is not changeable. Link: https://patch.msgid.link/r/20250220194438.2281088-10-dave.jiang@intel.com Reviewed-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- drivers/cxl/core/features.c | 121 +++++++++++++++++++++++++++++++++++- include/uapi/cxl/features.h | 1 + include/uapi/fwctl/cxl.h | 46 ++++++++++++++ 3 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 include/uapi/fwctl/cxl.h diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 846de3294a5e..6cec89977fa0 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -4,6 +4,7 @@ #include #include #include +#include #include "cxl.h" #include "core.h" #include "cxlmem.h" @@ -349,11 +350,127 @@ static void cxlctl_close_uctx(struct fwctl_uctx *uctx) { } +static void *cxlctl_get_supported_features(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + const struct cxl_mbox_get_sup_feats_in *feat_in; + struct cxl_mbox_get_sup_feats_out *feat_out; + struct cxl_feat_entry *pos; + size_t out_size; + int requested; + u32 count; + u16 start; + int i; + + if (rpc_in->op_size != sizeof(*feat_in)) + return ERR_PTR(-EINVAL); + + feat_in = &rpc_in->get_sup_feats_in; + count = le32_to_cpu(feat_in->count); + start = le16_to_cpu(feat_in->start_idx); + requested = count / sizeof(*pos); + + /* + * Make sure that the total requested number of entries is not greater + * than the total number of supported features allowed for userspace. + */ + if (start >= cxlfs->entries->num_features) + return ERR_PTR(-EINVAL); + + requested = min_t(int, requested, cxlfs->entries->num_features - start); + + out_size = sizeof(struct fwctl_rpc_cxl_out) + + struct_size(feat_out, ents, requested); + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + rpc_out->size = struct_size(feat_out, ents, requested); + feat_out = &rpc_out->get_sup_feats_out; + if (requested == 0) { + feat_out->num_entries = cpu_to_le16(requested); + feat_out->supported_feats = + cpu_to_le16(cxlfs->entries->num_features); + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = out_size; + return no_free_ptr(rpc_out); + } + + for (i = start, pos = &feat_out->ents[0]; + i < cxlfs->entries->num_features; i++, pos++) { + if (i - start == requested) + break; + + memcpy(pos, &cxlfs->entries->ent[i], sizeof(*pos)); + /* + * If the feature is exclusive, set the set_feat_size to 0 to + * indicate that the feature is not changeable. + */ + if (is_cxl_feature_exclusive(pos)) { + u32 flags; + + pos->set_feat_size = 0; + flags = le32_to_cpu(pos->flags); + flags &= ~CXL_FEATURE_F_CHANGEABLE; + pos->flags = cpu_to_le32(flags); + } + } + + feat_out->num_entries = cpu_to_le16(requested); + feat_out->supported_feats = cpu_to_le16(cxlfs->entries->num_features); + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = out_size; + + return no_free_ptr(rpc_out); +} + +static bool cxlctl_validate_hw_command(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope, + u16 opcode) +{ + struct cxl_mailbox *cxl_mbox = &cxlfs->cxlds->cxl_mbox; + + switch (opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + if (cxl_mbox->feat_cap < CXL_FEATURES_RO) + return false; + if (scope >= FWCTL_RPC_CONFIGURATION) + return true; + return false; + default: + return false; + } +} + +static void *cxlctl_handle_commands(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len, u16 opcode) +{ + switch (opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + return cxlctl_get_supported_features(cxlfs, rpc_in, out_len); + default: + return ERR_PTR(-EOPNOTSUPP); + } +} + static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, void *in, size_t in_len, size_t *out_len) { - /* Place holder */ - return ERR_PTR(-EOPNOTSUPP); + struct fwctl_device *fwctl_dev = uctx->fwctl; + struct cxl_memdev *cxlmd = fwctl_to_memdev(fwctl_dev); + struct cxl_features_state *cxlfs = to_cxlfs(cxlmd->cxlds); + const struct fwctl_rpc_cxl *rpc_in = in; + u16 opcode = rpc_in->opcode; + + if (!cxlctl_validate_hw_command(cxlfs, rpc_in, scope, opcode)) + return ERR_PTR(-EINVAL); + + return cxlctl_handle_commands(cxlfs, rpc_in, out_len, opcode); } static const struct fwctl_ops cxlctl_ops = { diff --git a/include/uapi/cxl/features.h b/include/uapi/cxl/features.h index 7f7ab4aefec2..d6db8984889f 100644 --- a/include/uapi/cxl/features.h +++ b/include/uapi/cxl/features.h @@ -42,6 +42,7 @@ struct cxl_mbox_get_sup_feats_in { #define CXL_CMD_EFFECTS_VALID BIT(9) #define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) #define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) +#define CXL_CMD_EFFECTS_RESERVED GENMASK(15, 12) /* * struct cxl_feat_entry - Supported Feature Entry diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h new file mode 100644 index 000000000000..4bbfd7727526 --- /dev/null +++ b/include/uapi/fwctl/cxl.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024-2025 Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_FWCTL_CXL_H_ +#define _UAPI_FWCTL_CXL_H_ + +#include +#include +#include + +/** + * struct fwctl_rpc_cxl - ioctl(FWCTL_RPC) input for CXL + * @opcode: CXL mailbox command opcode + * @flags: Flags for the command (input). + * @op_size: Size of input payload. + * @reserved1: Reserved. Must be 0s. + * @get_sup_feats_in: Get Supported Features input + */ +struct fwctl_rpc_cxl { + __struct_group(fwctl_rpc_cxl_hdr, hdr, /* no attrs */, + __u32 opcode; + __u32 flags; + __u32 op_size; + __u32 reserved1; + ); + struct cxl_mbox_get_sup_feats_in get_sup_feats_in; +}; + +/** + * struct fwctl_rpc_cxl_out - ioctl(FWCTL_RPC) output for CXL + * @size: Size of the output payload + * @retval: Return value from device + * @get_sup_feats_out: Get Supported Features output + */ +struct fwctl_rpc_cxl_out { + __struct_group(fwctl_rpc_cxl_out_hdr, hdr, /* no attrs */, + __u32 size; + __u32 retval; + ); + struct cxl_mbox_get_sup_feats_out get_sup_feats_out; +}; + +#endif From patchwork Fri Mar 7 20:55:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007137 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFF0F257AF5 for ; Fri, 7 Mar 2025 20:56:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381017; cv=none; b=GejRfipHp84/kTMgL8ZbNSCthNqnHWquVJlW06xAL4/LoCgBto9ZpgYEbvraXG6eRSHAUPj/VYjueaMx4wnFHkdBmc13SHP1/JAV2bxwQ7dFXq/sUOPTc1RPqyMxjVEMFsSg4Y7fGy6vEYrJGyZ/EpF07U5SwswLI/hgDVo4cik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381017; c=relaxed/simple; bh=h+q8HUscKi1UuGIvjKe1PbdreAZjY3OiLZjTxMr/4hY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LZ36SmqIdTvgXYc16eGr466uTkprDDnxMqwahDkMqMMFDl+SZwPxwUmigCZFL0YorpU2UdfhF2FB9u/SBCZ83P36LwGuwi1FQttT5Q/XGe+ttUI8Ke0lPGXCe6ober+xolqfSxCCV9FkC6ATlG5h/EaE/12ZuUxBDgZrtZHfoBA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27D65C4CED1; Fri, 7 Mar 2025 20:56:57 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Jonathan Cameron , Dan Williams , Li Ming Subject: [PATCH v8 4/9] cxl: Add support to handle user feature commands for get feature Date: Fri, 7 Mar 2025 13:55:34 -0700 Message-ID: <20250307205648.1021626-5-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helper function to parse the user data from fwctl RPC ioctl and send the parsed input parameters to cxl_get_feature() call. Link: https://patch.msgid.link/r/20250220194438.2281088-11-dave.jiang@intel.com Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- Add kdoc fix for missing payload member --- drivers/cxl/core/features.c | 44 +++++++++++++++++++++++++++++++++++++ include/uapi/fwctl/cxl.h | 12 ++++++++-- 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 6cec89977fa0..2a35581d2677 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -427,6 +427,47 @@ static void *cxlctl_get_supported_features(struct cxl_features_state *cxlfs, return no_free_ptr(rpc_out); } +static void *cxlctl_get_feature(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + struct cxl_mailbox *cxl_mbox = &cxlfs->cxlds->cxl_mbox; + const struct cxl_mbox_get_feat_in *feat_in; + u16 offset, count, return_code; + size_t out_size = *out_len; + + if (rpc_in->op_size != sizeof(*feat_in)) + return ERR_PTR(-EINVAL); + + feat_in = &rpc_in->get_feat_in; + offset = le16_to_cpu(feat_in->offset); + count = le16_to_cpu(feat_in->count); + + if (!count) + return ERR_PTR(-EINVAL); + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + out_size = cxl_get_feature(cxl_mbox, &feat_in->uuid, + feat_in->selection, rpc_out->payload, + count, offset, &return_code); + *out_len = sizeof(struct fwctl_rpc_cxl_out); + if (!out_size) { + rpc_out->size = 0; + rpc_out->retval = return_code; + return no_free_ptr(rpc_out); + } + + rpc_out->size = out_size; + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len += out_size; + + return no_free_ptr(rpc_out); +} + static bool cxlctl_validate_hw_command(struct cxl_features_state *cxlfs, const struct fwctl_rpc_cxl *rpc_in, enum fwctl_rpc_scope scope, @@ -436,6 +477,7 @@ static bool cxlctl_validate_hw_command(struct cxl_features_state *cxlfs, switch (opcode) { case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + case CXL_MBOX_OP_GET_FEATURE: if (cxl_mbox->feat_cap < CXL_FEATURES_RO) return false; if (scope >= FWCTL_RPC_CONFIGURATION) @@ -453,6 +495,8 @@ static void *cxlctl_handle_commands(struct cxl_features_state *cxlfs, switch (opcode) { case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: return cxlctl_get_supported_features(cxlfs, rpc_in, out_len); + case CXL_MBOX_OP_GET_FEATURE: + return cxlctl_get_feature(cxlfs, rpc_in, out_len); default: return ERR_PTR(-EOPNOTSUPP); } diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h index 4bbfd7727526..f589bfb77426 100644 --- a/include/uapi/fwctl/cxl.h +++ b/include/uapi/fwctl/cxl.h @@ -18,6 +18,7 @@ * @op_size: Size of input payload. * @reserved1: Reserved. Must be 0s. * @get_sup_feats_in: Get Supported Features input + * @get_feat_in: Get Feature input */ struct fwctl_rpc_cxl { __struct_group(fwctl_rpc_cxl_hdr, hdr, /* no attrs */, @@ -26,7 +27,10 @@ struct fwctl_rpc_cxl { __u32 op_size; __u32 reserved1; ); - struct cxl_mbox_get_sup_feats_in get_sup_feats_in; + union { + struct cxl_mbox_get_sup_feats_in get_sup_feats_in; + struct cxl_mbox_get_feat_in get_feat_in; + }; }; /** @@ -34,13 +38,17 @@ struct fwctl_rpc_cxl { * @size: Size of the output payload * @retval: Return value from device * @get_sup_feats_out: Get Supported Features output + * @payload: raw byte stream of payload */ struct fwctl_rpc_cxl_out { __struct_group(fwctl_rpc_cxl_out_hdr, hdr, /* no attrs */, __u32 size; __u32 retval; ); - struct cxl_mbox_get_sup_feats_out get_sup_feats_out; + union { + struct cxl_mbox_get_sup_feats_out get_sup_feats_out; + __DECLARE_FLEX_ARRAY(__u8, payload); + }; }; #endif From patchwork Fri Mar 7 20:55:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007138 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4B39257AFA for ; Fri, 7 Mar 2025 20:56:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381018; cv=none; b=hZ+m9ziXedA4c07eUNOYPVLFFT9HshTPPRG0qEoNcbK9j2hLHSclQCsnkkBEqwtmj6guKJDH9owO97UNA91jrrv2cpuqxD0Ul0ubVanTg1W5RxD0aVjM+R3N8PgTFPB0DTxSZIt09qORMIoqSf1EunLfs6eAkRSPMpbeUJvasTA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381018; c=relaxed/simple; bh=Cm7RuuUpS3kiCdIG/SQkxF3lX3qFmerHu6yKg9+eyKI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GoBnRvYiS089oJon4MdubnbFTk1G0Uen6fYDvIoz73P2tWvHu45ITc+FDPJ3AXJz0MkyfUy6OXhIZ1iuKLw2kBevHVyamtraAWiSxxwkQ2UWJW7YGAPAIa10/cw8nlvKdbm1i/oE8RWVYvC3TPC8l3I+/kI/8ttlJkSApT4VSnw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E4A4C4CEEC; Fri, 7 Mar 2025 20:56:58 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Jonathan Cameron , Dan Williams , Li Ming Subject: [PATCH v8 5/9] cxl: Add support to handle user feature commands for set feature Date: Fri, 7 Mar 2025 13:55:35 -0700 Message-ID: <20250307205648.1021626-6-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helper function to parse the user data from fwctl RPC ioctl and send the parsed input parameters to cxl_set_feature() call. Link: https://patch.msgid.link/r/20250220194438.2281088-12-dave.jiang@intel.com Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- drivers/cxl/core/features.c | 146 +++++++++++++++++++++++++++++++++++- include/uapi/fwctl/cxl.h | 2 + 2 files changed, 146 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 2a35581d2677..caf92e9cea21 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -21,16 +21,21 @@ static const uuid_t cxl_exclusive_feats[] = { CXL_FEAT_RANK_SPARING_UUID, }; -static bool is_cxl_feature_exclusive(struct cxl_feat_entry *entry) +static bool is_cxl_feature_exclusive_by_uuid(const uuid_t *uuid) { for (int i = 0; i < ARRAY_SIZE(cxl_exclusive_feats); i++) { - if (uuid_equal(&entry->uuid, &cxl_exclusive_feats[i])) + if (uuid_equal(uuid, &cxl_exclusive_feats[i])) return true; } return false; } +static bool is_cxl_feature_exclusive(struct cxl_feat_entry *entry) +{ + return is_cxl_feature_exclusive_by_uuid(&entry->uuid); +} + inline struct cxl_features_state *to_cxlfs(struct cxl_dev_state *cxlds) { return cxlds->cxlfs; @@ -350,6 +355,27 @@ static void cxlctl_close_uctx(struct fwctl_uctx *uctx) { } +static struct cxl_feat_entry * +get_support_feature_info(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in) +{ + struct cxl_feat_entry *feat; + const uuid_t *uuid; + + if (rpc_in->op_size < sizeof(uuid)) + return ERR_PTR(-EINVAL); + + uuid = &rpc_in->set_feat_in.uuid; + + for (int i = 0; i < cxlfs->entries->num_features; i++) { + feat = &cxlfs->entries->ent[i]; + if (uuid_equal(uuid, &feat->uuid)) + return feat; + } + + return ERR_PTR(-EINVAL); +} + static void *cxlctl_get_supported_features(struct cxl_features_state *cxlfs, const struct fwctl_rpc_cxl *rpc_in, size_t *out_len) @@ -468,6 +494,116 @@ static void *cxlctl_get_feature(struct cxl_features_state *cxlfs, return no_free_ptr(rpc_out); } +static void *cxlctl_set_feature(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + struct cxl_mailbox *cxl_mbox = &cxlfs->cxlds->cxl_mbox; + const struct cxl_mbox_set_feat_in *feat_in; + size_t out_size, data_size; + u16 offset, return_code; + u32 flags; + int rc; + + if (rpc_in->op_size <= sizeof(feat_in->hdr)) + return ERR_PTR(-EINVAL); + + feat_in = &rpc_in->set_feat_in; + + if (is_cxl_feature_exclusive_by_uuid(&feat_in->uuid)) + return ERR_PTR(-EPERM); + + offset = le16_to_cpu(feat_in->offset); + flags = le32_to_cpu(feat_in->flags); + out_size = *out_len; + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + rpc_out->size = 0; + + data_size = rpc_in->op_size - sizeof(feat_in->hdr); + rc = cxl_set_feature(cxl_mbox, &feat_in->uuid, + feat_in->version, feat_in->feat_data, + data_size, flags, offset, &return_code); + if (rc) { + rpc_out->retval = return_code; + return no_free_ptr(rpc_out); + } + + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = sizeof(*rpc_out); + + return no_free_ptr(rpc_out); +} + +static bool cxlctl_validate_set_features(struct cxl_features_state *cxlfs, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + u16 effects, imm_mask, reset_mask; + struct cxl_feat_entry *feat; + u32 flags; + + feat = get_support_feature_info(cxlfs, rpc_in); + if (IS_ERR(feat)) + return false; + + /* Ensure that the attribute is changeable */ + flags = le32_to_cpu(feat->flags); + if (!(flags & CXL_FEATURE_F_CHANGEABLE)) + return false; + + effects = le16_to_cpu(feat->effects); + + /* + * Reserved bits are set, rejecting since the effects is not + * comprehended by the driver. + */ + if (effects & CXL_CMD_EFFECTS_RESERVED) { + dev_warn_once(cxlfs->cxlds->dev, + "Reserved bits set in the Feature effects field!\n"); + return false; + } + + /* Currently no user background command support */ + if (effects & CXL_CMD_BACKGROUND) + return false; + + /* Effects cause immediate change, highest security scope is needed */ + imm_mask = CXL_CMD_CONFIG_CHANGE_IMMEDIATE | + CXL_CMD_DATA_CHANGE_IMMEDIATE | + CXL_CMD_POLICY_CHANGE_IMMEDIATE | + CXL_CMD_LOG_CHANGE_IMMEDIATE; + + reset_mask = CXL_CMD_CONFIG_CHANGE_COLD_RESET | + CXL_CMD_CONFIG_CHANGE_CONV_RESET | + CXL_CMD_CONFIG_CHANGE_CXL_RESET; + + /* If no immediate or reset effect set, The hardware has a bug */ + if (!(effects & imm_mask) && !(effects & reset_mask)) + return false; + + /* + * If the Feature setting causes immediate configuration change + * then we need the full write permission policy. + */ + if (effects & imm_mask && scope >= FWCTL_RPC_DEBUG_WRITE_FULL) + return true; + + /* + * If the Feature setting only causes configuration change + * after a reset, then the lesser level of write permission + * policy is ok. + */ + if (!(effects & imm_mask) && scope >= FWCTL_RPC_DEBUG_WRITE) + return true; + + return false; +} + static bool cxlctl_validate_hw_command(struct cxl_features_state *cxlfs, const struct fwctl_rpc_cxl *rpc_in, enum fwctl_rpc_scope scope, @@ -483,6 +619,10 @@ static bool cxlctl_validate_hw_command(struct cxl_features_state *cxlfs, if (scope >= FWCTL_RPC_CONFIGURATION) return true; return false; + case CXL_MBOX_OP_SET_FEATURE: + if (cxl_mbox->feat_cap < CXL_FEATURES_RW) + return false; + return cxlctl_validate_set_features(cxlfs, rpc_in, scope); default: return false; } @@ -497,6 +637,8 @@ static void *cxlctl_handle_commands(struct cxl_features_state *cxlfs, return cxlctl_get_supported_features(cxlfs, rpc_in, out_len); case CXL_MBOX_OP_GET_FEATURE: return cxlctl_get_feature(cxlfs, rpc_in, out_len); + case CXL_MBOX_OP_SET_FEATURE: + return cxlctl_set_feature(cxlfs, rpc_in, out_len); default: return ERR_PTR(-EOPNOTSUPP); } diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h index f589bfb77426..43f522f0cdcd 100644 --- a/include/uapi/fwctl/cxl.h +++ b/include/uapi/fwctl/cxl.h @@ -19,6 +19,7 @@ * @reserved1: Reserved. Must be 0s. * @get_sup_feats_in: Get Supported Features input * @get_feat_in: Get Feature input + * @set_feat_in: Set Feature input */ struct fwctl_rpc_cxl { __struct_group(fwctl_rpc_cxl_hdr, hdr, /* no attrs */, @@ -30,6 +31,7 @@ struct fwctl_rpc_cxl { union { struct cxl_mbox_get_sup_feats_in get_sup_feats_in; struct cxl_mbox_get_feat_in get_feat_in; + struct cxl_mbox_set_feat_in set_feat_in; }; }; From patchwork Fri Mar 7 20:55:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007139 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E055257449 for ; Fri, 7 Mar 2025 20:57:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381020; cv=none; b=mVQvyp1JDorY0MQVBKkVLbGdGshMi1YafvP1Zx4tuX0T1i37Yrh8rsH4vsR6V6iK0E7g8V4GBVeZVwlm/Xrc1PI5f5OTg94zNqQ8Q7azUmeQtxxkmuUh9Gumq3EnJHirXHl33AlRaEzfpdMd6zFU/X948wA2+mp2flVJoWZa8kc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381020; c=relaxed/simple; bh=4VWrDW5zca75TVE5+3OPaGR1EKI5gVYQqbQDyApEeaI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IEqXib0zzP1Pmu/hDr6XyNK5Hu8ULBjkT+JOOvHcQNf7fcnLrTOsK+2CvTpaxYchxHFL42tu/w0jUc3qJtTGQXPkoseyIhrc3M06SRuNWiscBJ67iBRARqOQkDCQztxszIOW3fQGh9eSldosorHuYA1/fL5jWl1pcZcMdWvjkDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F454C4CED1; Fri, 7 Mar 2025 20:57:00 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Jonathan Cameron , Dan Williams , Li Ming Subject: [PATCH v8 6/9] cxl/test: Add Get Feature support to cxl_test Date: Fri, 7 Mar 2025 13:55:36 -0700 Message-ID: <20250307205648.1021626-7-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation of Get Feature command to cxl_test. The feature for device patrol scrub is returned by the emulation code. This is the only feature currently supported by cxl_test. It returns the information for the device patrol scrub feature. Link: https://patch.msgid.link/r/20250220194438.2281088-13-dave.jiang@intel.com Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- tools/testing/cxl/test/mem.c | 56 ++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 848db399102c..af70932c9bb0 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -48,6 +48,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_FEATURES), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), + .effect = CXL_CMD_EFFECT_NONE, + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -149,6 +153,10 @@ struct mock_event_store { u32 ev_status; }; +struct vendor_test_feat { + __le32 data; +} __packed; + struct cxl_mockmem_data { void *lsa; void *fw; @@ -165,6 +173,7 @@ struct cxl_mockmem_data { u8 event_buf[SZ_4K]; u64 timestamp; unsigned long sanitize_timeout; + struct vendor_test_feat test_feat; }; static struct mock_event_log *event_find_log(struct device *dev, int log_type) @@ -1379,6 +1388,44 @@ static void fill_feature_vendor_test(struct cxl_feat_entry *feat) #define MAX_CXL_TEST_FEATS 1 +static int mock_get_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct vendor_test_feat *output = cmd->payload_out; + struct cxl_mbox_get_feat_in *input = cmd->payload_in; + u16 offset = le16_to_cpu(input->offset); + u16 count = le16_to_cpu(input->count); + u8 *ptr; + + if (offset > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + if (offset + count > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + ptr = (u8 *)&mdata->test_feat + offset; + memcpy((u8 *)output + offset, ptr, count); + + return 0; +} + +static int mock_get_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_feat_in *input = cmd->payload_in; + + if (uuid_equal(&input->uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_get_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1509,6 +1556,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: rc = mock_get_supported_features(mdata, cmd); break; + case CXL_MBOX_OP_GET_FEATURE: + rc = mock_get_feature(mdata, cmd); + break; default: break; } @@ -1556,6 +1606,11 @@ static int cxl_mock_mailbox_create(struct cxl_dev_state *cxlds) return 0; } +static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) +{ + mdata->test_feat.data = cpu_to_le32(0xdeadbeef); +} + static int cxl_mock_mem_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1651,6 +1706,7 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) dev_dbg(dev, "No CXL FWCTL setup\n"); cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); + cxl_mock_test_feat_init(mdata); return 0; } From patchwork Fri Mar 7 20:55:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007140 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F9972561BD for ; Fri, 7 Mar 2025 20:57:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381022; cv=none; b=BNilOHgmAAk23jGNIZ4R+9QpnTBz5ItKWNjtDU0ThxYJTf92dFT1yXUxuWULszRe8a/EU94TDdu9FcTMikVt4982hnyK//vK2hxd/rGf7b6U1qqyea1tlGlKbiNxrhQeixxudur1JLsoO3pb1nh+bo4SvnePv8ZW8ThvmC7Q6TQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381022; c=relaxed/simple; bh=sPoqgg7g61imSdLmjfvAxEL6R7aQGP9YRI7F69dORDs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OHmhP27VzUP/s5kSTo83/P4apr5j2AQEByyVZdKUlVmyrreSVwHUBsPwE1BeemlphguHpLjcV1Gii9tkhaCYFBjEI9maNO3pbGyl/v5dV0JJvceJwk1U+tMH+ceoS/ONMMqDELcfwH4TC2Zas1jW61Lq6EimBL+AoN+JoVsJwPQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6EC9C4CED1; Fri, 7 Mar 2025 20:57:01 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Jonathan Cameron , Dan Williams , Li Ming Subject: [PATCH v8 7/9] cxl/test: Add Set Feature support to cxl_test Date: Fri, 7 Mar 2025 13:55:37 -0700 Message-ID: <20250307205648.1021626-8-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation to support Set Feature mailbox command to cxl_test. The only feature supported is the device patrol scrub feature. The set feature allows activation of patrol scrub for the cxl_test emulated device. The command does not support partial data transfer even though the spec allows it. This restriction is to reduce complexity of the emulation given the patrol scrub feature is very minimal. Link: https://patch.msgid.link/r/20250220194438.2281088-14-dave.jiang@intel.com Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Reviewed-by: Li Ming Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- tools/testing/cxl/test/mem.c | 51 ++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index af70932c9bb0..9495dbcc03a7 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -52,6 +52,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_SET_FEATURE), + .effect = cpu_to_le16(EFFECT(CONF_CHANGE_IMMEDIATE)), + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -1426,6 +1430,50 @@ static int mock_get_feature(struct cxl_mockmem_data *mdata, return -EOPNOTSUPP; } +static int mock_set_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_set_feat_in *input = cmd->payload_in; + struct vendor_test_feat *test = + (struct vendor_test_feat *)input->feat_data; + u32 action; + + action = FIELD_GET(CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK, + le32_to_cpu(input->hdr.flags)); + /* + * While it is spec compliant to support other set actions, it is not + * necessary to add the complication in the emulation currently. Reject + * anything besides full xfer. + */ + if (action != CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + /* Offset should be reserved when doing full transfer */ + if (input->hdr.offset) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + memcpy(&mdata->test_feat.data, &test->data, sizeof(u32)); + + return 0; +} + +static int mock_set_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_set_feat_in *input = cmd->payload_in; + + if (uuid_equal(&input->hdr.uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_set_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1559,6 +1607,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_FEATURE: rc = mock_get_feature(mdata, cmd); break; + case CXL_MBOX_OP_SET_FEATURE: + rc = mock_set_feature(mdata, cmd); + break; default: break; } From patchwork Fri Mar 7 20:55:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007141 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C633257459 for ; Fri, 7 Mar 2025 20:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381023; cv=none; b=iaTz8jizSk/jExPJ/fdM06uOGtrXZifxWqWc4DkcwsAS7oL6xeFlc82UKxXH2ujBNouEoGQTIgHKL0U9fwZ2N1I9E/hpkVJ8ykwSkkScH3xZCFH2xKOnL4O8/yM6Qz+jIav+EVN4MlQVbfu5jv9iOZT+QdCpaWoAzQUqB0j+uu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381023; c=relaxed/simple; bh=E/evfK/32RQ1EQNVdjhd5mXlmMWC3G8tbgkkFOCZrog=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k0KozbblY9UDoiTGdEsqQMmXymgEliaZyh/Eu1Bnm6X2gLqP5+b3agciwmCTZTY7bB1VNKTtw3aUaN/t7ICXbSRN9NsTQCs0+zVS9W9KdjXLUo+i8VJTcYl+wLtHbyHEMDjzEzMJeEG0iPcWuwM4uEAAlS7NnABXl06rpL26z0c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23788C4CED1; Fri, 7 Mar 2025 20:57:03 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com, Jonathan Cameron , Dan Williams Subject: [PATCH v8 8/9] fwctl/cxl: Add documentation to FWCTL CXL Date: Fri, 7 Mar 2025 13:55:38 -0700 Message-ID: <20250307205648.1021626-9-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add policy and operational documentation for FWCTL CXL. Link: https://patch.msgid.link/r/20250220194438.2281088-15-dave.jiang@intel.com Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Signed-off-by: Dave Jiang Signed-off-by: Jason Gunthorpe --- - Fix referenced source files in documentation --- .../userspace-api/fwctl/fwctl-cxl.rst | 143 ++++++++++++++++++ Documentation/userspace-api/fwctl/index.rst | 1 + MAINTAINERS | 1 + 3 files changed, 145 insertions(+) create mode 100644 Documentation/userspace-api/fwctl/fwctl-cxl.rst diff --git a/Documentation/userspace-api/fwctl/fwctl-cxl.rst b/Documentation/userspace-api/fwctl/fwctl-cxl.rst new file mode 100644 index 000000000000..14c0159c5856 --- /dev/null +++ b/Documentation/userspace-api/fwctl/fwctl-cxl.rst @@ -0,0 +1,143 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================ +fwctl cxl driver +================ + +:Author: Dave Jiang + +Overview +======== + +The CXL spec defines a set of commands that can be issued to the mailbox of a +CXL device or switch. It also left room for vendor specific commands to be +issued to the mailbox as well. fwctl provides a path to issue a set of allowed +mailbox commands from user space to the device moderated by the kernel driver. + +The following 3 commands will be used to support CXL Features: +CXL spec r3.1 8.2.9.6.1 Get Supported Features (Opcode 0500h) +CXL spec r3.1 8.2.9.6.2 Get Feature (Opcode 0501h) +CXL spec r3.1 8.2.9.6.3 Set Feature (Opcode 0502h) + +The "Get Supported Features" return data may be filtered by the kernel driver to +drop any features that are forbidden by the kernel or being exclusively used by +the kernel. The driver will set the "Set Feature Size" of the "Get Supported +Features Supported Feature Entry" to 0 to indicate that the Feature cannot be +modified. The "Get Supported Features" command and the "Get Features" falls +under the fwctl policy of FWCTL_RPC_CONFIGURATION. + +For "Set Feature" command, the access policy currently is broken down into two +categories depending on the Set Feature effects reported by the device. If the +Set Feature will cause immediate change to the device, the fwctl access policy +must be FWCTL_RPC_DEBUG_WRITE_FULL. The effects for this level are +"immediate config change", "immediate data change", "immediate policy change", +or "immediate log change" for the set effects mask. If the effects are "config +change with cold reset" or "config change with conventional reset", then the +fwctl access policy must be FWCTL_RPC_DEBUG_WRITE or higher. + +fwctl cxl User API +================== + +.. kernel-doc:: include/uapi/fwctl/cxl.h +.. kernel-doc:: include/uapi/cxl/features.h + +1. Driver info query +-------------------- + +First step for the app is to issue the ioctl(FWCTL_CMD_INFO). Successful +invocation of the ioctl implies the Features capability is operational and +returns an all zeros 32bit payload. A ``struct fwctl_info`` needs to be filled +out with the ``fwctl_info.out_device_type`` set to ``FWCTL_DEVICE_TYPE_CXL``. +The return data should be ``struct fwctl_info_cxl`` that contains a reserved +32bit field that should be all zeros. + +2. Send hardware commands +------------------------- + +Next step is to send the 'Get Supported Features' command to the driver from +user space via ioctl(FWCTL_RPC). A ``struct fwctl_rpc_cxl`` is pointed to +by ``fwctl_rpc.in``. ``struct fwctl_rpc_cxl.in_payload`` points to +the hardware input structure that is defined by the CXL spec. ``fwctl_rpc.out`` +points to the buffer that contains a ``struct fwctl_rpc_cxl_out`` that includes +the hardware output data inlined as ``fwctl_rpc_cxl_out.payload``. This command +is called twice. First time to retrieve the number of features supported. +A second time to retrieve the specific feature details as the output data. + +After getting the specific feature details, a Get/Set Feature command can be +appropriately programmed and sent. For a "Set Feature" command, the retrieved +feature info contains an effects field that details the resulting +"Set Feature" command will trigger. That will inform the user whether +the system is configured to allowed the "Set Feature" command or not. + +Code example of a Get Feature +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + static int cxl_fwctl_rpc_get_test_feature(int fd, struct test_feature *feat_ctx, + const uint32_t expected_data) + { + struct cxl_mbox_get_feat_in *feat_in; + struct fwctl_rpc_cxl_out *out; + struct fwctl_rpc rpc = {0}; + struct fwctl_rpc_cxl *in; + size_t out_size, in_size; + uint32_t val; + void *data; + int rc; + + in_size = sizeof(*in) + sizeof(*feat_in); + rc = posix_memalign((void **)&in, 16, in_size); + if (rc) + return -ENOMEM; + memset(in, 0, in_size); + feat_in = &in->get_feat_in; + + uuid_copy(feat_in->uuid, feat_ctx->uuid); + feat_in->count = feat_ctx->get_size; + + out_size = sizeof(*out) + feat_ctx->get_size; + rc = posix_memalign((void **)&out, 16, out_size); + if (rc) + goto free_in; + memset(out, 0, out_size); + + in->opcode = CXL_MBOX_OPCODE_GET_FEATURE; + in->op_size = sizeof(*feat_in); + + rpc.size = sizeof(rpc); + rpc.scope = FWCTL_RPC_CONFIGURATION; + rpc.in_len = in_size; + rpc.out_len = out_size; + rpc.in = (uint64_t)(uint64_t *)in; + rpc.out = (uint64_t)(uint64_t *)out; + + rc = send_command(fd, &rpc, out); + if (rc) + goto free_all; + + data = out->payload; + val = le32toh(*(__le32 *)data); + if (memcmp(&val, &expected_data, sizeof(val)) != 0) { + rc = -ENXIO; + goto free_all; + } + + free_all: + free(out); + free_in: + free(in); + return rc; + } + +Take a look at CXL CLI test directory + for a detailed user code +for examples on how to exercise this path. + + +fwctl cxl Kernel API +==================== + +.. kernel-doc:: drivers/cxl/core/features.c + :export: +.. kernel-doc:: include/cxl/features.h diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst index 06959fbf1547..d9d40a468a31 100644 --- a/Documentation/userspace-api/fwctl/index.rst +++ b/Documentation/userspace-api/fwctl/index.rst @@ -10,3 +10,4 @@ to securely construct and execute RPCs inside device firmware. :maxdepth: 1 fwctl + fwctl-cxl diff --git a/MAINTAINERS b/MAINTAINERS index 93e64a54c56f..3381e41dcf37 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5837,6 +5837,7 @@ M: Dan Williams L: linux-cxl@vger.kernel.org S: Maintained F: Documentation/driver-api/cxl +F: Documentation/userspace-api/fwctl/fwctl-cxl.rst F: drivers/cxl/ F: include/cxl/ F: include/uapi/linux/cxl_mem.h From patchwork Fri Mar 7 20:55:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 14007142 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40EC0257AC1 for ; Fri, 7 Mar 2025 20:57:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381025; cv=none; b=ABni0JWrC82Z7hGNOhf3XXKXyHk2ZJn04fYVL8TF/fjAZFfs+jgHdBrKmzo33zdhFEPkFABZb6Rvw+I65Qoto+vzmMorCsl/fGAT4fZzA8yBfOq71povZSTiCprMz9PLJP1oEE4dNg4Ss0rFfxkQNkYN4fbx3HlrS8oduUNdTMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741381025; c=relaxed/simple; bh=9ZPQoaY7GflGkXKoabSGUB0KhvGP2niv7+kkN3b1LJA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=D3ExjTk1k7qGjn6x6sZFBylxBnffDML8aRYpPlsBxKmh2JtJ6ZuIqqjSqFhMS2rEYFsNqSYcxRbiEN2Yy6tQ3FP0x/J6rPWxRe7HbCTmXkDdzkpPFOIZtg6EgGC8JcrcubO1fxb1UKqt29d+Rpc8nvFUHOkeZducKN8/wj9TYhA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99D14C4CED1; Fri, 7 Mar 2025 20:57:04 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: jgg@nvidia.com, saeedm@nvidia.com, jonathan.cameron@huawei.com Subject: [PATCH v8 9/9] cxl: Fixup kdoc issues for include/cxl/features.h Date: Fri, 7 Mar 2025 13:55:39 -0700 Message-ID: <20250307205648.1021626-10-dave.jiang@intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250307205648.1021626-1-dave.jiang@intel.com> References: <20250307205648.1021626-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Fixup kdoc issues for include/cxl/features.h from linux-next build. Signed-off-by: Dave Jiang Reviewed-by: Li Ming --- include/cxl/features.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/cxl/features.h b/include/cxl/features.h index ead63573b0b4..cdd4c7131cc3 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -51,8 +51,6 @@ enum cxl_features_capability { * struct cxl_features_state - The Features state for the device * @cxlds: Pointer to CXL device state * @entries: CXl feature entry context - * @num_features: total Features supported by the device - * @ent: Flex array of Feature detail entries from the device * @fwctl_dev: Firmware Control device */ struct cxl_features_state {