From patchwork Fri Nov 15 21:25:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877284 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7965B1E283E for ; Fri, 15 Nov 2024 21:27:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706070; cv=none; b=DksoJSo2q/EqUvlgXa+kkClnE4FoBlrYuLYPgwosHu2QmDvUvSZRpoMN1cEI4vADREeMidXUMzg5KFcTWGlvFZoDv6KfoZ10OOFDptYyk4Wf/PTE9fu1D1NK3Ag7yFI0Urcwpg0/8tBuXVDz7LHnU7FOyTRabJNMQoZqf5j4WsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706070; c=relaxed/simple; bh=PzejU5VAS9uL1RqaXrvQCEcJ8czXSnEtapAGk6Gsh8M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Be4wAiHgSqpBILTKZMTxr+C4Vp9JSd6lw18iZ904jF9npnNFq9OwksL7WmchMWyLqsgROuh7WHamtkunUFL315XvtRgm5fabORDAY1k+b5xuwYRPPfQ7S1zJFDBLZsBbvulhNWcCafoBzzU/XuSbNPGXhzeTvESyHCrQjQeUWwo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA606C4CECF; Fri, 15 Nov 2024 21:27:49 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 01/20] cxl: Refactor user ioctl command path from mds to mailbox Date: Fri, 15 Nov 2024 14:25:34 -0700 Message-ID: <20241115212745.869552-2-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With 'struct cxl_mailbox' context introduced, the helper functions cxl_query_cmd() and cxl_send_cmd() can take a cxl_mailbox directly rather than a cxl_memdev parameter. Refactor to use cxl_mailbox directly. Signed-off-by: Dave Jiang --- drivers/cxl/core/core.h | 6 ++- drivers/cxl/core/mbox.c | 89 +++++++++++++++++++-------------------- drivers/cxl/core/memdev.c | 22 +++++++--- drivers/cxl/cxlmem.h | 40 ------------------ include/cxl/mailbox.h | 41 +++++++++++++++++- 5 files changed, 102 insertions(+), 96 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 0c62b4069ba0..ff30d1c99ca7 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -4,6 +4,8 @@ #ifndef __CXL_CORE_H__ #define __CXL_CORE_H__ +#include + extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; extern const struct device_type cxl_pmu_type; @@ -65,9 +67,9 @@ static inline void cxl_region_exit(void) struct cxl_send_command; struct cxl_mem_query_commands; -int cxl_query_cmd(struct cxl_memdev *cxlmd, +int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q); -int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s); +int cxl_send_cmd(struct cxl_mailbox *cxl_mailbox, struct cxl_send_command __user *s); void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, resource_size_t length); diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 5175138c4fb7..880ac1dba3cc 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -349,40 +349,40 @@ static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) return true; } -static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox, - struct cxl_memdev_state *mds, u16 opcode, +static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd, + struct cxl_mailbox *cxl_mbox, u16 opcode, size_t in_size, size_t out_size, u64 in_payload) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - *mbox = (struct cxl_mbox_cmd) { + *mbox_cmd = (struct cxl_mbox_cmd) { .opcode = opcode, .size_in = in_size, }; if (in_size) { - mbox->payload_in = vmemdup_user(u64_to_user_ptr(in_payload), - in_size); - if (IS_ERR(mbox->payload_in)) - return PTR_ERR(mbox->payload_in); + mbox_cmd->payload_in = vmemdup_user(u64_to_user_ptr(in_payload), + in_size); + if (IS_ERR(mbox_cmd->payload_in)) + return PTR_ERR(mbox_cmd->payload_in); - if (!cxl_payload_from_user_allowed(opcode, mbox->payload_in)) { - dev_dbg(mds->cxlds.dev, "%s: input payload not allowed\n", + if (!cxl_payload_from_user_allowed(opcode, + mbox_cmd->payload_in)) { + dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n", cxl_mem_opcode_to_name(opcode)); - kvfree(mbox->payload_in); + kvfree(mbox_cmd->payload_in); return -EBUSY; } } /* Prepare to handle a full payload for variable sized output */ if (out_size == CXL_VARIABLE_PAYLOAD) - mbox->size_out = cxl_mbox->payload_size; + mbox_cmd->size_out = cxl_mbox->payload_size; else - mbox->size_out = out_size; + mbox_cmd->size_out = out_size; - if (mbox->size_out) { - mbox->payload_out = kvzalloc(mbox->size_out, GFP_KERNEL); - if (!mbox->payload_out) { - kvfree(mbox->payload_in); + if (mbox_cmd->size_out) { + mbox_cmd->payload_out = kvzalloc(mbox_cmd->size_out, GFP_KERNEL); + if (!mbox_cmd->payload_out) { + kvfree(mbox_cmd->payload_in); return -ENOMEM; } } @@ -397,10 +397,8 @@ static void cxl_mbox_cmd_dtor(struct cxl_mbox_cmd *mbox) static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, const struct cxl_send_command *send_cmd, - struct cxl_memdev_state *mds) + struct cxl_mailbox *cxl_mbox) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - if (send_cmd->raw.rsvd) return -EINVAL; @@ -415,7 +413,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, if (!cxl_mem_raw_command_allowed(send_cmd->raw.opcode)) return -EPERM; - dev_WARN_ONCE(mds->cxlds.dev, true, "raw command path used\n"); + dev_WARN_ONCE(cxl_mbox->host, true, "raw command path used\n"); *mem_cmd = (struct cxl_mem_command) { .info = { @@ -431,7 +429,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, const struct cxl_send_command *send_cmd, - struct cxl_memdev_state *mds) + struct cxl_mailbox *cxl_mbox) { struct cxl_mem_command *c = &cxl_mem_commands[send_cmd->id]; const struct cxl_command_info *info = &c->info; @@ -446,11 +444,11 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, return -EINVAL; /* Check that the command is enabled for hardware */ - if (!test_bit(info->id, mds->enabled_cmds)) + if (!test_bit(info->id, cxl_mbox->enabled_cmds)) return -ENOTTY; /* Check that the command is not claimed for exclusive kernel use */ - if (test_bit(info->id, mds->exclusive_cmds)) + if (test_bit(info->id, cxl_mbox->exclusive_cmds)) return -EBUSY; /* Check the input buffer is the expected size */ @@ -479,7 +477,7 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, /** * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND. * @mbox_cmd: Sanitized and populated &struct cxl_mbox_cmd. - * @mds: The driver data for the operation + * @cxl_mbox: CXL mailbox context * @send_cmd: &struct cxl_send_command copied in from userspace. * * Return: @@ -494,10 +492,9 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, * safe to send to the hardware. */ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, - struct cxl_memdev_state *mds, + struct cxl_mailbox *cxl_mbox, const struct cxl_send_command *send_cmd) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_mem_command mem_cmd; int rc; @@ -514,24 +511,23 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, /* Sanitize and construct a cxl_mem_command */ if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW) - rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, mds); + rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, cxl_mbox); else - rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, mds); + rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, cxl_mbox); if (rc) return rc; /* Sanitize and construct a cxl_mbox_cmd */ - return cxl_mbox_cmd_ctor(mbox_cmd, mds, mem_cmd.opcode, + return cxl_mbox_cmd_ctor(mbox_cmd, cxl_mbox, mem_cmd.opcode, mem_cmd.info.size_in, mem_cmd.info.size_out, send_cmd->in.payload); } -int cxl_query_cmd(struct cxl_memdev *cxlmd, +int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q) { - struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); - struct device *dev = &cxlmd->dev; + struct device *dev = cxl_mbox->host; struct cxl_mem_command *cmd; u32 n_commands; int j = 0; @@ -552,9 +548,9 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, cxl_for_each_cmd(cmd) { struct cxl_command_info info = cmd->info; - if (test_bit(info.id, mds->enabled_cmds)) + if (test_bit(info.id, cxl_mbox->enabled_cmds)) info.flags |= CXL_MEM_COMMAND_FLAG_ENABLED; - if (test_bit(info.id, mds->exclusive_cmds)) + if (test_bit(info.id, cxl_mbox->exclusive_cmds)) info.flags |= CXL_MEM_COMMAND_FLAG_EXCLUSIVE; if (copy_to_user(&q->commands[j++], &info, sizeof(info))) @@ -569,7 +565,7 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, /** * handle_mailbox_cmd_from_user() - Dispatch a mailbox command for userspace. - * @mds: The driver data for the operation + * @cxl_mbox: The mailbox context for the operation. * @mbox_cmd: The validated mailbox command. * @out_payload: Pointer to userspace's output payload. * @size_out: (Input) Max payload size to copy out. @@ -590,13 +586,12 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, * * See cxl_send_cmd(). */ -static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds, +static int handle_mailbox_cmd_from_user(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *mbox_cmd, u64 out_payload, s32 *size_out, u32 *retval) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - struct device *dev = mds->cxlds.dev; + struct device *dev = cxl_mbox->host; int rc; dev_dbg(dev, @@ -633,10 +628,9 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds, return rc; } -int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s) +int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s) { - struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); - struct device *dev = &cxlmd->dev; + struct device *dev = cxl_mbox->host; struct cxl_send_command send; struct cxl_mbox_cmd mbox_cmd; int rc; @@ -646,11 +640,11 @@ int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s) if (copy_from_user(&send, s, sizeof(send))) return -EFAULT; - rc = cxl_validate_cmd_from_user(&mbox_cmd, mds, &send); + rc = cxl_validate_cmd_from_user(&mbox_cmd, cxl_mbox, &send); if (rc) return rc; - rc = handle_mailbox_cmd_from_user(mds, &mbox_cmd, send.out.payload, + rc = handle_mailbox_cmd_from_user(cxl_mbox, &mbox_cmd, send.out.payload, &send.out.size, &send.retval); if (rc) return rc; @@ -724,6 +718,7 @@ static int cxl_xfer_log(struct cxl_memdev_state *mds, uuid_t *uuid, */ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_cel_entry *cel_entry; const int cel_entries = size / sizeof(*cel_entry); struct device *dev = mds->cxlds.dev; @@ -737,7 +732,7 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) int enabled = 0; if (cmd) { - set_bit(cmd->info.id, mds->enabled_cmds); + set_bit(cmd->info.id, cxl_mbox->enabled_cmds); enabled++; } @@ -807,6 +802,7 @@ static const uuid_t log_uuid[] = { */ int cxl_enumerate_cmds(struct cxl_memdev_state *mds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_mbox_get_supported_logs *gsl; struct device *dev = mds->cxlds.dev; struct cxl_mem_command *cmd; @@ -845,7 +841,7 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds) /* In case CEL was bogus, enable some default commands. */ cxl_for_each_cmd(cmd) if (cmd->flags & CXL_CMD_FLAG_FORCE_ENABLE) - set_bit(cmd->info.id, mds->enabled_cmds); + set_bit(cmd->info.id, cxl_mbox->enabled_cmds); /* Found the required CEL */ rc = 0; @@ -1448,6 +1444,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mutex_init(&mds->event.log_lock); mds->cxlds.dev = dev; mds->cxlds.reg_map.host = dev; + mds->cxlds.cxl_mbox.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 84fefb76dafa..4d544a55ac3e 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -564,9 +564,11 @@ EXPORT_SYMBOL_NS_GPL(is_cxl_memdev, CXL); void set_exclusive_cxl_commands(struct cxl_memdev_state *mds, unsigned long *cmds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + down_write(&cxl_memdev_rwsem); - bitmap_or(mds->exclusive_cmds, mds->exclusive_cmds, cmds, - CXL_MEM_COMMAND_ID_MAX); + bitmap_or(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds, + cmds, CXL_MEM_COMMAND_ID_MAX); up_write(&cxl_memdev_rwsem); } EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, CXL); @@ -579,9 +581,11 @@ EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, CXL); void clear_exclusive_cxl_commands(struct cxl_memdev_state *mds, unsigned long *cmds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + down_write(&cxl_memdev_rwsem); - bitmap_andnot(mds->exclusive_cmds, mds->exclusive_cmds, cmds, - CXL_MEM_COMMAND_ID_MAX); + bitmap_andnot(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds, + cmds, CXL_MEM_COMMAND_ID_MAX); up_write(&cxl_memdev_rwsem); } EXPORT_SYMBOL_NS_GPL(clear_exclusive_cxl_commands, CXL); @@ -656,11 +660,14 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, unsigned long arg) { + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + switch (cmd) { case CXL_MEM_QUERY_COMMANDS: - return cxl_query_cmd(cxlmd, (void __user *)arg); + return cxl_query_cmd(cxl_mbox, (void __user *)arg); case CXL_MEM_SEND_COMMAND: - return cxl_send_cmd(cxlmd, (void __user *)arg); + return cxl_send_cmd(cxl_mbox, (void __user *)arg); default: return -ENOTTY; } @@ -994,10 +1001,11 @@ static void cxl_remove_fw_upload(void *fwl) int devm_cxl_setup_fw_upload(struct device *host, struct cxl_memdev_state *mds) { struct cxl_dev_state *cxlds = &mds->cxlds; + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; struct device *dev = &cxlds->cxlmd->dev; struct fw_upload *fwl; - if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, mds->enabled_cmds)) + if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, cxl_mbox->enabled_cmds)) return 0; fwl = firmware_upload_register(THIS_MODULE, dev, dev_name(dev), diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 2a25d1957ddb..a0a49809cd76 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -106,42 +106,6 @@ static inline struct cxl_ep *cxl_ep_load(struct cxl_port *port, return xa_load(&port->endpoints, (unsigned long)&cxlmd->dev); } -/** - * struct cxl_mbox_cmd - A command to be submitted to hardware. - * @opcode: (input) The command set and command submitted to hardware. - * @payload_in: (input) Pointer to the input payload. - * @payload_out: (output) Pointer to the output payload. Must be allocated by - * the caller. - * @size_in: (input) Number of bytes to load from @payload_in. - * @size_out: (input) Max number of bytes loaded into @payload_out. - * (output) Number of bytes generated by the device. For fixed size - * outputs commands this is always expected to be deterministic. For - * variable sized output commands, it tells the exact number of bytes - * written. - * @min_out: (input) internal command output payload size validation - * @poll_count: (input) Number of timeouts to attempt. - * @poll_interval_ms: (input) Time between mailbox background command polling - * interval timeouts. - * @return_code: (output) Error code returned from hardware. - * - * This is the primary mechanism used to send commands to the hardware. - * All the fields except @payload_* correspond exactly to the fields described in - * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and - * @payload_out are written to, and read from the Command Payload Registers - * defined in CXL 2.0 8.2.8.4.8. - */ -struct cxl_mbox_cmd { - u16 opcode; - void *payload_in; - void *payload_out; - size_t size_in; - size_t size_out; - size_t min_out; - int poll_count; - int poll_interval_ms; - u16 return_code; -}; - /* * Per CXL 3.0 Section 8.2.8.4.5.1 */ @@ -461,8 +425,6 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox) * @lsa_size: Size of Label Storage Area * (CXL 2.0 8.2.9.5.1.1 Identify Memory Device) * @firmware_version: Firmware version for the memory device. - * @enabled_cmds: Hardware commands found enabled in CEL. - * @exclusive_cmds: Commands that are kernel-internal only * @total_bytes: sum of all possible capacities * @volatile_only_bytes: hard volatile capacity * @persistent_only_bytes: hard persistent capacity @@ -485,8 +447,6 @@ struct cxl_memdev_state { struct cxl_dev_state cxlds; size_t lsa_size; char firmware_version[0x10]; - DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); - DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); u64 total_bytes; u64 volatile_only_bytes; u64 persistent_only_bytes; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index bacd111e75f1..cc894f07a435 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -3,12 +3,49 @@ #ifndef __CXL_MBOX_H__ #define __CXL_MBOX_H__ #include +#include -struct cxl_mbox_cmd; +/** + * struct cxl_mbox_cmd - A command to be submitted to hardware. + * @opcode: (input) The command set and command submitted to hardware. + * @payload_in: (input) Pointer to the input payload. + * @payload_out: (output) Pointer to the output payload. Must be allocated by + * the caller. + * @size_in: (input) Number of bytes to load from @payload_in. + * @size_out: (input) Max number of bytes loaded into @payload_out. + * (output) Number of bytes generated by the device. For fixed size + * outputs commands this is always expected to be deterministic. For + * variable sized output commands, it tells the exact number of bytes + * written. + * @min_out: (input) internal command output payload size validation + * @poll_count: (input) Number of timeouts to attempt. + * @poll_interval_ms: (input) Time between mailbox background command polling + * interval timeouts. + * @return_code: (output) Error code returned from hardware. + * + * This is the primary mechanism used to send commands to the hardware. + * All the fields except @payload_* correspond exactly to the fields described in + * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and + * @payload_out are written to, and read from the Command Payload Registers + * defined in CXL 2.0 8.2.8.4.8. + */ +struct cxl_mbox_cmd { + u16 opcode; + void *payload_in; + void *payload_out; + size_t size_in; + size_t size_out; + size_t min_out; + int poll_count; + int poll_interval_ms; + u16 return_code; +}; /** * struct cxl_mailbox - context for CXL mailbox operations * @host: device that hosts the mailbox + * @enabled_cmds: mailbox commands that are enabled by the driver + * @exclusive_cmds: mailbox commands that are exclusive to the kernel * @payload_size: Size of space for payload * (CXL 3.1 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: mutex protects device mailbox and firmware @@ -17,6 +54,8 @@ struct cxl_mbox_cmd; */ struct cxl_mailbox { struct device *host; + DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); + DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); size_t payload_size; struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; From patchwork Fri Nov 15 21:25:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877285 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE1341E283E for ; Fri, 15 Nov 2024 21:27:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706071; cv=none; b=Yd7uITWGCBAEwmtMjgPkjqS+V2MOzW5BspTgB0CoCS+eUt2JsHK5NRRfJMpc6l5xlNBznQj+Eq4gHqw9oTYZlU+mFqSGQNNfjwgi6avCkAw4ax0i/wYCBxSbIe221EGHJKx8pM8KuymOMDvLkabggZkb+ZWLq8aahUC0bR+KZq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706071; c=relaxed/simple; bh=aXs2FCLNzmX8sLkqu9yB6CPfRgIx+6f/6EOCt/B9d8o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Hb4G6xSu1QR/DDpjWVgCwNnzjAVsxhuKVBJ6QtghwYMtaNqEN7McKFZt6Tvb+mZcFk9RLL7a+BDURHnXFeKi9LLTm3QpxMugsHgGqsamQAeKWkEABKbsHo6RFeIj5iSLmD4Di9J3oEgPb2aEBiY47NB+7xBBVSaSjZkLWw2E7IA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24028C4CECF; Fri, 15 Nov 2024 21:27:51 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 02/20] cxl: Add Get Supported Features command for kernel usage Date: Fri, 15 Nov 2024 14:25:35 -0700 Message-ID: <20241115212745.869552-3-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 CXL spec r3.1 8.2.9.6.1 Get Supported Features (Opcode 0500h) The command retrieve the list of supported device-specific features (identified by UUID) and general information about each Feature. The driver will retrieve the feature entries in order to make checks and provide information for the Get Feature and Set Feature command. One of the main piece of information retrieved are the effects a Set Feature command would have for a particular feature. Co-developed-by: Shiju Jose Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang --- v2: - fix feature entry pointer math. - free mbox_out in cxl_get_supported_features(). (Shiju, Jonathan) - replace sizeof(struct) with feat_size. (Shiju) - replace reserved in feature structs with u8 type. (Shiju) - change cxl_feat_entry->effects to set_effects. (Shiju) - rearrange assigment of cxl_mbox->entries. (Jonathan) - Separate no features from actual error. (Jonathan) - Fix missing kdoc for entries. (Jonathan) --- drivers/cxl/core/mbox.c | 178 +++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 31 ++++++ drivers/cxl/pci.c | 4 + include/cxl/mailbox.h | 4 + include/uapi/linux/cxl_mem.h | 1 + 5 files changed, 218 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 880ac1dba3cc..4ba56f3d5a65 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -67,6 +67,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { CXL_CMD(SET_SHUTDOWN_STATE, 0x1, 0, 0), CXL_CMD(GET_SCAN_MEDIA_CAPS, 0x10, 0x4, 0), CXL_CMD(GET_TIMESTAMP, 0, 0x8, 0), + CXL_CMD(GET_SUPPORTED_FEATURES, 0x8, CXL_VARIABLE_PAYLOAD, 0), }; /* @@ -790,6 +791,183 @@ static const uuid_t log_uuid[] = { [VENDOR_DEBUG_UUID] = DEFINE_CXL_VENDOR_DEBUG_UUID, }; +static void cxl_free_features(void *features) +{ + kvfree(features); +} + +static int cxl_get_supported_features_count(struct cxl_dev_state *cxlds) +{ + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + struct cxl_mbox_get_sup_feats_out mbox_out; + struct cxl_mbox_get_sup_feats_in mbox_in; + struct cxl_mbox_cmd mbox_cmd; + int rc; + + memset(&mbox_in, 0, sizeof(mbox_in)); + mbox_in.count = cpu_to_le32(sizeof(mbox_out)); + memset(&mbox_out, 0, sizeof(mbox_out)); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_SUPPORTED_FEATURES, + .size_in = sizeof(mbox_in), + .payload_in = &mbox_in, + .size_out = sizeof(mbox_out), + .payload_out = &mbox_out, + .min_out = sizeof(mbox_out), + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + cxl_mbox->num_features = le16_to_cpu(mbox_out.supported_feats); + + return 0; +} + +int cxl_get_supported_features(struct cxl_dev_state *cxlds) +{ + struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) = NULL; + int remain_feats, max_size, max_feats, start, rc; + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + int feat_size = sizeof(struct cxl_feat_entry); + struct cxl_mbox_get_sup_feats_in mbox_in; + int hdr_size = sizeof(*mbox_out); + struct cxl_mbox_cmd mbox_cmd; + struct cxl_mem_command *cmd; + struct cxl_feat_entry *entry; + + /* Get supported features is optional, need to check */ + cmd = cxl_mem_find_command(CXL_MBOX_OP_GET_SUPPORTED_FEATURES); + if (!cmd) + return -EOPNOTSUPP; + if (!test_bit(cmd->info.id, cxl_mbox->enabled_cmds)) + return -EOPNOTSUPP; + + rc = cxl_get_supported_features_count(cxlds); + if (rc) + return rc; + + if (!cxl_mbox->num_features) { + dev_dbg(cxl_mbox->host, "No CXL features enumerated.\n"); + return 0; + } + + struct cxl_feat_entry *entries __free(kvfree) = + kvmalloc(cxl_mbox->num_features * feat_size, GFP_KERNEL); + + if (!entries) + return -ENOMEM; + + max_size = cxl_mbox->payload_size - hdr_size; + /* max feat entries that can fit in mailbox max payload size */ + max_feats = max_size / feat_size; + entry = &entries[0]; + + mbox_out = kvmalloc(cxl_mbox->payload_size, GFP_KERNEL); + if (!mbox_out) + return -ENOMEM; + + start = 0; + remain_feats = cxl_mbox->num_features; + do { + int retrieved, alloc_size, copy_feats; + int num_entries; + + if (remain_feats > max_feats) { + alloc_size = sizeof(*mbox_out) + max_feats * feat_size; + remain_feats = remain_feats - max_feats; + copy_feats = max_feats; + } else { + alloc_size = sizeof(*mbox_out) + remain_feats * feat_size; + copy_feats = remain_feats; + remain_feats = 0; + } + + memset(&mbox_in, 0, sizeof(mbox_in)); + mbox_in.count = cpu_to_le32(alloc_size); + mbox_in.start_idx = cpu_to_le16(start); + memset(mbox_out, 0, alloc_size); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_SUPPORTED_FEATURES, + .size_in = sizeof(mbox_in), + .payload_in = &mbox_in, + .size_out = alloc_size, + .payload_out = mbox_out, + .min_out = hdr_size, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + if (mbox_cmd.size_out <= hdr_size) { + rc = -ENXIO; + goto err; + } + + /* + * Make sure retrieved out buffer is multiple of feature + * entries. + */ + retrieved = mbox_cmd.size_out - hdr_size; + if (retrieved % feat_size) { + rc = -ENXIO; + goto err; + } + + num_entries = le16_to_cpu(mbox_out->num_entries); + /* + * If the reported output entries * defined entry size != + * retrieved output bytes, then the output package is incorrect. + */ + if (num_entries * feat_size != retrieved) { + rc = -ENXIO; + goto err; + } + + memcpy(entry, mbox_out->ents, retrieved); + entry++; + /* + * If the number of output entries is less than expected, add the + * remaining entries to the next batch. + */ + remain_feats += copy_feats - num_entries; + start += num_entries; + } while (remain_feats); + + cxl_mbox->entries = no_free_ptr(entries); + rc = devm_add_action_or_reset(cxl_mbox->host, cxl_free_features, + cxl_mbox->entries); + if (rc) + return rc; + + return 0; + +err: + cxl_mbox->num_features = 0; + return rc; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_supported_features, CXL); + +int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *feat_uuid, + struct cxl_feat_entry *feat_entry_out) +{ + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + struct cxl_feat_entry *feat_entry; + int count; + + /* Check CXL dev supports the feature */ + feat_entry = &cxl_mbox->entries[0]; + for (count = 0; count < cxl_mbox->num_features; count++, feat_entry++) { + if (uuid_equal(&feat_entry->uuid, feat_uuid)) { + memcpy(feat_entry_out, feat_entry, sizeof(*feat_entry_out)); + return 0; + } + } + + return -EOPNOTSUPP; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_supported_feature_entry, CXL); + /** * cxl_enumerate_cmds() - Enumerate commands for a device. * @mds: The driver data for the operation diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index a0a49809cd76..6685dd76985a 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -490,6 +490,7 @@ enum cxl_opcode { CXL_MBOX_OP_GET_LOG_CAPS = 0x0402, CXL_MBOX_OP_CLEAR_LOG = 0x0403, CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, + CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, @@ -773,6 +774,32 @@ enum { CXL_PMEM_SEC_PASS_USER, }; +/* Get Supported Features (0x500h) CXL r3.1 8.2.9.6.1 */ +struct cxl_mbox_get_sup_feats_in { + __le32 count; + __le16 start_idx; + u8 reserved[2]; +} __packed; + +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + u8 get_feat_ver; + u8 set_feat_ver; + __le16 set_effects; + u8 reserved[18]; +} __packed; + +struct cxl_mbox_get_sup_feats_out { + __le16 num_entries; + __le16 supported_feats; + u8 reserved[4]; + struct cxl_feat_entry ents[] __counted_by_le(supported_feats); +} __packed; + int cxl_internal_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); int cxl_dev_state_identify(struct cxl_memdev_state *mds); @@ -832,4 +859,8 @@ struct cxl_hdm { struct seq_file; struct dentry *cxl_debugfs_create_dir(const char *dir); void cxl_dpa_debug(struct seq_file *file, struct cxl_dev_state *cxlds); + +int cxl_get_supported_features(struct cxl_dev_state *cxlds); +int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *feat_uuid, + struct cxl_feat_entry *feat_entry_out); #endif /* __CXL_MEM_H__ */ diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 188412d45e0d..cbb86ecf0e2f 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -887,6 +887,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + rc = cxl_get_supported_features(cxlds); + if (rc) + dev_dbg(&pdev->dev, "No features enumerated.\n"); + rc = cxl_set_timestamp(mds); if (rc) return rc; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index cc894f07a435..03c4b8ad84c1 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -50,6 +50,8 @@ struct cxl_mbox_cmd { * (CXL 3.1 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: mutex protects device mailbox and firmware * @mbox_wait: rcuwait for mailbox + * @num_features: number of supported features entries + * @features: list of supported feature entries * @mbox_send: @dev specific transport for transmitting mailbox commands */ struct cxl_mailbox { @@ -59,6 +61,8 @@ struct cxl_mailbox { size_t payload_size; struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; + int num_features; + struct cxl_feat_entry *entries; int (*mbox_send)(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); }; diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index c6c0fe27495d..bd2535962f70 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -50,6 +50,7 @@ ___C(GET_LOG_CAPS, "Get Log Capabilities"), \ ___C(CLEAR_LOG, "Clear Log"), \ ___C(GET_SUP_LOG_SUBLIST, "Get Supported Logs Sub-List"), \ + ___C(GET_SUPPORTED_FEATURES, "Get Supported Features"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a From patchwork Fri Nov 15 21:25:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877286 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A9381E283E for ; Fri, 15 Nov 2024 21:27:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706073; cv=none; b=qrrCCEBCD7h1LbIcYXf1yNiHlN6bKZUt8KxziWDD6nkvWpxXmyb7AdH50jpyIqg9+FnOO09K8vVR9SytH48rcGDGkRS91tnwvi/q6banNtkuFWStDGp/643p+O6ch1QnOn75c8pUQL5fx+jrKiqrC3hKNGBdybT64yJCs9luQtY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706073; c=relaxed/simple; bh=Fo89upy1OIReQsjpJJ9T/oV8JRWr2iuRqT5TfxGxDUo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=My8AC5Z3UH+U3uvYq67Ei8EKZSBmfBXmNbwPBXg7wvjemZ7IFEpg+nM6jC/PA8AKGUSsRAa5vuLpheBnBrTAVmkPYchByftaumS0ltSMmq8eVhJWmNK52DuPWC8CdbQ2BP1nCfobRjxu0zbbYoLSwtJDPvxOATQTPuFqudYMV9k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F6B1C4CECF; Fri, 15 Nov 2024 21:27:52 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 03/20] cxl/test: Add Get Supported Features mailbox command support Date: Fri, 15 Nov 2024 14:25:36 -0700 Message-ID: <20241115212745.869552-4-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add cxl-test emulation of Get Supported Features mailbox command. Currently only adding a test feature with feature identifier of all f's for testing. Signed-off-by: Dave Jiang --- tools/testing/cxl/test/mem.c | 69 ++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 71916e0e1546..e0c7e49795ed 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -44,6 +44,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_LOGS), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_FEATURES), + .effect = CXL_CMD_EFFECT_NONE, + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -1337,6 +1341,64 @@ static int mock_activate_fw(struct cxl_mockmem_data *mdata, return -EINVAL; } +#define CXL_VENDOR_FEATURE_TEST \ + UUID_INIT(0xffffffff, 0xffff, 0xffff, 0xff, 0xff, 0xff, 0xff, 0xff, \ + 0xff, 0xff, 0xff) + +static void fill_feature_vendor_test(struct cxl_feat_entry *feat) +{ + feat->uuid = CXL_VENDOR_FEATURE_TEST; + feat->id = 0; + feat->get_feat_size = cpu_to_le16(0x4); + feat->set_feat_size = cpu_to_le16(0x4); + feat->flags = cpu_to_le32(BIT(5)); + feat->get_feat_ver = 1; + feat->set_feat_ver = 1; + feat->set_effects = cpu_to_le16(BIT(0) | BIT(9)); +} + +static int mock_get_supported_features(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_sup_feats_in *in = cmd->payload_in; + struct cxl_mbox_get_sup_feats_out *out = cmd->payload_out; + struct cxl_feat_entry *feat; + u16 start_idx, count; + + if (cmd->size_out < sizeof(*out)) { + cmd->return_code = CXL_MBOX_CMD_RC_PAYLOADLEN; + return -EINVAL; + } + + /* + * Current emulation only supports 1 feature + */ + start_idx = le16_to_cpu(in->start_idx); + if (start_idx != 0) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + count = le16_to_cpu(in->count); + if (count < sizeof(*out)) { + cmd->return_code = CXL_MBOX_CMD_RC_PAYLOADLEN; + return -EINVAL; + } + + out->supported_feats = cpu_to_le16(1); + cmd->return_code = 0; + if (count < sizeof(*out) + sizeof(*feat)) { + out->num_entries = 0; + return 0; + } + + out->num_entries = 1; + feat = out->ents; + fill_feature_vendor_test(feat); + + return 0; +} + static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd) { @@ -1422,6 +1484,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_ACTIVATE_FW: rc = mock_activate_fw(mdata, cmd); break; + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + rc = mock_get_supported_features(mdata, cmd); + break; default: break; } @@ -1524,6 +1589,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) return rc; + rc = cxl_get_supported_features(cxlds); + if (rc) + dev_dbg(dev, "No features enumerated\n"); + rc = cxl_poison_state_init(mds); if (rc) return rc; From patchwork Fri Nov 15 21:25:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877287 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0F5C1E283E for ; Fri, 15 Nov 2024 21:27:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706074; cv=none; b=ChIam1XUSQbIOh1HJz3r0eBJ5QQh/lccAuIOAyiHzgQY0giDLS+UuAkd5tC2yfWpj35Uxm47D/uZ4PolzpvIDgdM3p97i6mnhytmI/LW8lxEjq1zByAUukhrW7DZPxqwMoWJbiPvDp2qN5w75tDc2rwsc/CpvH6DFbGbvDC4jCY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706074; c=relaxed/simple; bh=r+z2m9Neq5I//V0BhWjlKEGisd9DSrWGdV77VIDBw48=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hMDhhjM5TieGoGm5hN9zaAMu83Fm/E6Yb/F0bUD13JEuhF1AXw1oTQVSmxn2rMAmSHlMKhy7l0y3Mq8vczMkdlgoUiEHnmwShJFEfJdy5KbjD21sXgzvUo+z+ACPvt64tLarEpm8gsIcLxmps0jjljBtKRGOMf9KSkWID05FVRY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6AB6C4CECF; Fri, 15 Nov 2024 21:27:53 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 04/20] cxl/mbox: Add GET_FEATURE mailbox command Date: Fri, 15 Nov 2024 14:25:37 -0700 Message-ID: <20241115212745.869552-5-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Shiju Jose Add support for GET_FEATURE mailbox command. CXL spec 3.1 section 8.2.9.6 describes optional device specific features. The settings of a feature can be retrieved using Get Feature command. CXL spec 3.1 section 8.2.9.6.2 describes Get Feature command. Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang --- This patch is not needed by this series except the opcode enumeration. --- drivers/cxl/core/mbox.c | 41 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 26 ++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 4ba56f3d5a65..f70bd7d28ff8 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -968,6 +968,47 @@ int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *f } EXPORT_SYMBOL_NS_GPL(cxl_get_supported_feature_entry, CXL); +size_t cxl_get_feature(struct cxl_dev_state *cxlds, const uuid_t feat_uuid, + enum cxl_get_feat_selection selection, + void *feat_out, size_t feat_out_size) +{ + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + size_t data_to_rd_size, size_out; + struct cxl_mbox_get_feat_in pi; + struct cxl_mbox_cmd mbox_cmd; + size_t data_rcvd_size = 0; + int rc; + + if (!feat_out || !feat_out_size) + return 0; + + size_out = min(feat_out_size, cxl_mbox->payload_size); + pi.uuid = feat_uuid; + pi.selection = selection; + do { + data_to_rd_size = min(feat_out_size - data_rcvd_size, + cxl_mbox->payload_size); + pi.offset = cpu_to_le16(data_rcvd_size); + pi.count = cpu_to_le16(data_to_rd_size); + + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_FEATURE, + .size_in = sizeof(pi), + .payload_in = &pi, + .size_out = size_out, + .payload_out = feat_out + data_rcvd_size, + .min_out = data_to_rd_size, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0 || !mbox_cmd.size_out) + return 0; + data_rcvd_size += mbox_cmd.size_out; + } while (data_rcvd_size < feat_out_size); + + return data_rcvd_size; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_feature, CXL); + /** * cxl_enumerate_cmds() - Enumerate commands for a device. * @mds: The driver data for the operation diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 6685dd76985a..8fa3a817a0dd 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -491,6 +491,7 @@ enum cxl_opcode { CXL_MBOX_OP_CLEAR_LOG = 0x0403, CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, + CXL_MBOX_OP_GET_FEATURE = 0x0501, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, @@ -800,6 +801,28 @@ struct cxl_mbox_get_sup_feats_out { struct cxl_feat_entry ents[] __counted_by_le(supported_feats); } __packed; +/* + * Get Feature CXL 3.1 Spec 8.2.9.6.2 + */ + +/* + * Get Feature input payload + * CXL rev 3.1 section 8.2.9.6.2 Table 8-99 + */ +enum cxl_get_feat_selection { + CXL_GET_FEAT_SEL_CURRENT_VALUE, + CXL_GET_FEAT_SEL_DEFAULT_VALUE, + CXL_GET_FEAT_SEL_SAVED_VALUE, + CXL_GET_FEAT_SEL_MAX +}; + +struct cxl_mbox_get_feat_in { + uuid_t uuid; + __le16 offset; + __le16 count; + u8 selection; +} __packed; + int cxl_internal_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); int cxl_dev_state_identify(struct cxl_memdev_state *mds); @@ -863,4 +886,7 @@ void cxl_dpa_debug(struct seq_file *file, struct cxl_dev_state *cxlds); int cxl_get_supported_features(struct cxl_dev_state *cxlds); int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *feat_uuid, struct cxl_feat_entry *feat_entry_out); +size_t cxl_get_feature(struct cxl_dev_state *cxlds, const uuid_t feat_uuid, + enum cxl_get_feat_selection selection, + void *feat_out, size_t feat_out_size); #endif /* __CXL_MEM_H__ */ From patchwork Fri Nov 15 21:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877288 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E97191FB3D4 for ; Fri, 15 Nov 2024 21:27:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706075; cv=none; b=TGjqGXTi9ptxg6dUNYttt299ALeCT8+CQJLvaYqv88T1KsADjwHwMbsqqv3WVdkzFBFqL/oeczYcgzOoSrHt6qVZsggSMMwomYS1X3yNLeBZGJ9FMaTTpcSvv7h4uNS4QW+dB6S6xKIumr5w6psYdgnt+sIA4fl0DZjazXKzzLg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706075; c=relaxed/simple; bh=2bEFhd0YTqeO8rlInJt7aPO+FmyOXWrwPqtcu3YMliQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q9VcwaCswmF1eHfxl8nQMbBSKLJ9Wv57PrpnC8a5LJZPjSJMbWmOpH6MESJHu76lsPfziw/vWE/CvESRR2M07LNdpaOBh1mL2b/a8Rwm4sCJBnyHGRS8/LRYKDR58WZ/N4fi2MsIHLP3kl8yym47EkEdayUE5tyN19BhQytLHHc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54DCCC4CECF; Fri, 15 Nov 2024 21:27:55 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 05/20] cxl: Add Get Feature command support for user submission Date: Fri, 15 Nov 2024 14:25:38 -0700 Message-ID: <20241115212745.869552-6-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add enumeration of Get Feature mailbox command for the kernel to recognize the command being passed in from user space. CXL spec r3.1 8.2.9.6.2 Get Feature (Opcode 0501h) The feature requested is identified by specific UUID. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 1 + include/uapi/linux/cxl_mem.h | 1 + 2 files changed, 2 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index f70bd7d28ff8..f49665d3b4d6 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -68,6 +68,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { CXL_CMD(GET_SCAN_MEDIA_CAPS, 0x10, 0x4, 0), CXL_CMD(GET_TIMESTAMP, 0, 0x8, 0), CXL_CMD(GET_SUPPORTED_FEATURES, 0x8, CXL_VARIABLE_PAYLOAD, 0), + CXL_CMD(GET_FEATURE, 0x15, CXL_VARIABLE_PAYLOAD, 0), }; /* diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index bd2535962f70..90dcb9723997 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -51,6 +51,7 @@ ___C(CLEAR_LOG, "Clear Log"), \ ___C(GET_SUP_LOG_SUBLIST, "Get Supported Logs Sub-List"), \ ___C(GET_SUPPORTED_FEATURES, "Get Supported Features"), \ + ___C(GET_FEATURE, "Get Feature"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a From patchwork Fri Nov 15 21:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877289 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F0A21FB3CF for ; Fri, 15 Nov 2024 21:27:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706077; cv=none; b=U+eeBhoc9xHRoiIwQRp8ePyTtTL/I+yO2KHB/TCGv+TQr4yDVHC2QyLIMU2lWXp2liI8nrdChplfj9JvK3809S+CSlXjJCGp++fiyAyYHzm3Ks/ZLdAFrcebSPfQFcLiybJWp5QnKw/ID2Lfo59/az4/0E0NDYRk47RXNvsEpFE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706077; c=relaxed/simple; bh=uEFtvFETEGtpihUgPT8NuYLndrNzM6AHbh29QNVbcPo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k99d5RsTropxkHZpYZORHvlb/azhkBcJg/hChp4Ablcl2XB07i0kXRhLY5rOdRoFXoafkwAB/prFjWuQ7FISEn2wo2dMEM2Bue296HLUrjNgDcYA9BhTY12QaNrK1Jkz5zdKz4qovySLFz5wzyioGMJv1x3gUZm5tVajrC/zADI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5F5BC4CECF; Fri, 15 Nov 2024 21:27:56 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 06/20] cxl/mbox: Add SET_FEATURE mailbox command Date: Fri, 15 Nov 2024 14:25:39 -0700 Message-ID: <20241115212745.869552-7-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Shiju Jose Add support for SET_FEATURE mailbox command. CXL spec 3.1 section 8.2.9.6 describes optional device specific features. CXL devices supports features with changeable attributes. The settings of a feature can be optionally modified using Set Feature command. CXL spec 3.1 section 8.2.9.6.3 describes Set Feature command. Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang --- This patch is not needed except for the opcode enumeration. --- drivers/cxl/core/mbox.c | 73 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 34 +++++++++++++++++++ 2 files changed, 107 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index f49665d3b4d6..4b9abf9a5b2b 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1010,6 +1010,79 @@ size_t cxl_get_feature(struct cxl_dev_state *cxlds, const uuid_t feat_uuid, } EXPORT_SYMBOL_NS_GPL(cxl_get_feature, CXL); +/* + * FEAT_DATA_MIN_PAYLOAD_SIZE - min extra number of bytes should be + * available in the mailbox for storing the actual feature data so that + * the feature data transfer would work as expected. + */ +#define FEAT_DATA_MIN_PAYLOAD_SIZE 10 +int cxl_set_feature(struct cxl_dev_state *cxlds, + const uuid_t feat_uuid, u8 feat_version, + void *feat_data, size_t feat_data_size, + u8 feat_flag) +{ + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + struct cxl_memdev_set_feat_pi { + struct cxl_mbox_set_feat_hdr hdr; + u8 feat_data[]; + } __packed; + size_t data_in_size, data_sent_size = 0; + struct cxl_mbox_cmd mbox_cmd; + size_t hdr_size; + int rc = 0; + + struct cxl_memdev_set_feat_pi *pi __free(kfree) = + kmalloc(cxl_mbox->payload_size, GFP_KERNEL); + pi->hdr.uuid = feat_uuid; + pi->hdr.version = feat_version; + feat_flag &= ~CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK; + feat_flag |= CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET; + hdr_size = sizeof(pi->hdr); + /* + * Check minimum mbox payload size is available for + * the feature data transfer. + */ + if (hdr_size + FEAT_DATA_MIN_PAYLOAD_SIZE > cxl_mbox->payload_size) + return -ENOMEM; + + if ((hdr_size + feat_data_size) <= cxl_mbox->payload_size) { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER); + data_in_size = feat_data_size; + } else { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER); + data_in_size = cxl_mbox->payload_size - hdr_size; + } + + do { + pi->hdr.offset = cpu_to_le16(data_sent_size); + memcpy(pi->feat_data, feat_data + data_sent_size, data_in_size); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_SET_FEATURE, + .size_in = hdr_size + data_in_size, + .payload_in = pi, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + data_sent_size += data_in_size; + if (data_sent_size >= feat_data_size) + return 0; + + if ((feat_data_size - data_sent_size) <= (cxl_mbox->payload_size - hdr_size)) { + data_in_size = feat_data_size - data_sent_size; + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER); + } else { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER); + } + } while (true); +} +EXPORT_SYMBOL_NS_GPL(cxl_set_feature, CXL); + /** * cxl_enumerate_cmds() - Enumerate commands for a device. * @mds: The driver data for the operation diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 8fa3a817a0dd..2c15817c80a5 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -492,6 +492,7 @@ enum cxl_opcode { CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, CXL_MBOX_OP_GET_FEATURE = 0x0501, + CXL_MBOX_OP_SET_FEATURE = 0x0502, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, @@ -823,6 +824,35 @@ struct cxl_mbox_get_feat_in { u8 selection; } __packed; +/* + * Set Feature CXL 3.1 Spec 8.2.9.6.3 + */ + +/* + * Set Feature input payload + * CXL rev 3.1 section 8.2.9.6.3 Table 8-101 + */ +/* Set Feature : Payload in flags */ +#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) +enum cxl_set_feat_flag_data_transfer { + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX +}; + +#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) + +struct cxl_mbox_set_feat_hdr { + uuid_t uuid; + __le32 flags; + __le16 offset; + u8 version; + u8 rsvd[9]; +} __packed; + int cxl_internal_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); int cxl_dev_state_identify(struct cxl_memdev_state *mds); @@ -889,4 +919,8 @@ int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *f size_t cxl_get_feature(struct cxl_dev_state *cxlds, const uuid_t feat_uuid, enum cxl_get_feat_selection selection, void *feat_out, size_t feat_out_size); +int cxl_set_feature(struct cxl_dev_state *cxlds, + const uuid_t feat_uuid, u8 feat_version, + void *feat_data, size_t feat_data_size, + u8 feat_flag); #endif /* __CXL_MEM_H__ */ From patchwork Fri Nov 15 21:25:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877290 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB851FB3CF for ; Fri, 15 Nov 2024 21:27:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706078; cv=none; b=X/JoLu9FD8jworhvb3UGxocDPTNyvG/Q8dGWyxwxkI1cfi8oFhJNKXA+LGlBcJlHCwN23QT3RqjEbAc7xR8bjXjxJtKgfEQC0sBRJfIJUZNlmQmMIF7Te4P127kObsvfDNvVcC43PvSSoT0Ld/OIzzIthSBnnp6xHVCTdU/bEDs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706078; c=relaxed/simple; bh=Oti3uXlgx4lCxIan2nV54r2yX1qIvLLH/AiPbJOSZKc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ToFuvi239jaLPvAQe55FeYZaxOg8B41J3q62cUcGIynlrRdUs8U6NTQg8tJXmlaobAdgArtqYvgbNLMm+szjNamektWOeJ/OhkqVKzv2bmFMxyWIjrH9VoX9OlmrUUmFIvdDU+ejFXxBLIBdH0q3GKJobqrKCup+ijBwr+cyQvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2C650C4CECF; Fri, 15 Nov 2024 21:27:58 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 07/20] cxl: Add Set Feature command support for user submission Date: Fri, 15 Nov 2024 14:25:40 -0700 Message-ID: <20241115212745.869552-8-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add enumeration of Set Feature mailbox command for the kernel to recognize the command being passed in from user space. CXL spec r3.1 8.2.9.6.3 Set Feature (Opcode 0502h) The feature requested is identified by specific UUID. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 1 + include/uapi/linux/cxl_mem.h | 1 + 2 files changed, 2 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 4b9abf9a5b2b..739444d34130 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -69,6 +69,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { CXL_CMD(GET_TIMESTAMP, 0, 0x8, 0), CXL_CMD(GET_SUPPORTED_FEATURES, 0x8, CXL_VARIABLE_PAYLOAD, 0), CXL_CMD(GET_FEATURE, 0x15, CXL_VARIABLE_PAYLOAD, 0), + CXL_CMD(SET_FEATURE, CXL_VARIABLE_PAYLOAD, 0, 0), }; /* diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index 90dcb9723997..469dd481795f 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -52,6 +52,7 @@ ___C(GET_SUP_LOG_SUBLIST, "Get Supported Logs Sub-List"), \ ___C(GET_SUPPORTED_FEATURES, "Get Supported Features"), \ ___C(GET_FEATURE, "Get Feature"), \ + ___C(SET_FEATURE, "Set Feature"), \ ___C(MAX, "invalid / last command") #define ___C(a, b) CXL_MEM_COMMAND_ID_##a From patchwork Fri Nov 15 21:25:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877291 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E6AF1FB3E3 for ; Fri, 15 Nov 2024 21:27:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706080; cv=none; b=ey7uV5qn6J228owGtYTTLjbd8iYhbcOU8IEazmacavK0c6wJoJ4sYOTk/p6CgvZhfvsHLTZPofHSXcDBR0gSdXWCCrCs+JLvhggxgudmyYrdr8g1antWll9jM+VWNeTQmROWs3b5YPcaKVN0Bjs6o/1+FajqYSKWJTgCvPGTZ2g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706080; c=relaxed/simple; bh=WkeqDWUBilVzjwKPBgCZ1O1lmHPzg0T5Z+jxtS84B0o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bNpevzbMGF3Sjr0WwSZldi4qhxvfsned9cbGA5J+MFNzCDLIiyA56NunEWwwqBv5IxtxnA9gzYmHu9vNXSB6Tl42hSCYIw+DpM/983HT4FCFKsJ+jJgkLEmcr86kklGBGvxNaDuLTSaGT62eCq57vbmaF9ogxCOpMy2ArFQKl0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84EC5C4CECF; Fri, 15 Nov 2024 21:27:59 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 08/20] cxl: Move cxl_driver related bits to be usable by external drivers Date: Fri, 15 Nov 2024 14:25:41 -0700 Message-ID: <20241115212745.869552-9-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Move the code related to cxl_driver to allow external drivers such as cxl_fwctl to utilize the cxl_bus. Signed-off-by: Dave Jiang --- drivers/cxl/cxl.h | 32 +------------------------------- include/cxl/cxl.h | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 31 deletions(-) create mode 100644 include/cxl/cxl.h diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 5406e3ab3d4a..18e78b2c3612 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -11,6 +11,7 @@ #include #include #include +#include extern const struct nvdimm_security_ops *cxl_security_ops; @@ -819,37 +820,6 @@ bool is_cxl_region(struct device *dev); extern struct bus_type cxl_bus_type; -struct cxl_driver { - const char *name; - int (*probe)(struct device *dev); - void (*remove)(struct device *dev); - struct device_driver drv; - int id; -}; - -#define to_cxl_drv(__drv) container_of_const(__drv, struct cxl_driver, drv) - -int __cxl_driver_register(struct cxl_driver *cxl_drv, struct module *owner, - const char *modname); -#define cxl_driver_register(x) __cxl_driver_register(x, THIS_MODULE, KBUILD_MODNAME) -void cxl_driver_unregister(struct cxl_driver *cxl_drv); - -#define module_cxl_driver(__cxl_driver) \ - module_driver(__cxl_driver, cxl_driver_register, cxl_driver_unregister) - -#define CXL_DEVICE_NVDIMM_BRIDGE 1 -#define CXL_DEVICE_NVDIMM 2 -#define CXL_DEVICE_PORT 3 -#define CXL_DEVICE_ROOT 4 -#define CXL_DEVICE_MEMORY_EXPANDER 5 -#define CXL_DEVICE_REGION 6 -#define CXL_DEVICE_PMEM_REGION 7 -#define CXL_DEVICE_DAX_REGION 8 -#define CXL_DEVICE_PMU 9 - -#define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") -#define CXL_MODALIAS_FMT "cxl:t%d" - struct cxl_nvdimm_bridge *to_cxl_nvdimm_bridge(struct device *dev); struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, struct cxl_port *port); diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h new file mode 100644 index 000000000000..07a2983275a0 --- /dev/null +++ b/include/cxl/cxl.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2024 Intel Corporation. */ +#ifndef __CXL_GLOBAL_H__ +#define __CXL_GLOBAL_H__ + +struct cxl_driver { + const char *name; + int (*probe)(struct device *dev); + void (*remove)(struct device *dev); + struct device_driver drv; + int id; +}; + +#define to_cxl_drv(__drv) container_of_const(__drv, struct cxl_driver, drv) + +int __cxl_driver_register(struct cxl_driver *cxl_drv, struct module *owner, + const char *modname); +#define cxl_driver_register(x) __cxl_driver_register(x, THIS_MODULE, KBUILD_MODNAME) +void cxl_driver_unregister(struct cxl_driver *cxl_drv); + +#define module_cxl_driver(__cxl_driver) \ + module_driver(__cxl_driver, cxl_driver_register, cxl_driver_unregister) + +#define CXL_DEVICE_NVDIMM_BRIDGE 1 +#define CXL_DEVICE_NVDIMM 2 +#define CXL_DEVICE_PORT 3 +#define CXL_DEVICE_ROOT 4 +#define CXL_DEVICE_MEMORY_EXPANDER 5 +#define CXL_DEVICE_REGION 6 +#define CXL_DEVICE_PMEM_REGION 7 +#define CXL_DEVICE_DAX_REGION 8 +#define CXL_DEVICE_PMU 9 + +#define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") +#define CXL_MODALIAS_FMT "cxl:t%d" + +#endif From patchwork Fri Nov 15 21:25:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877292 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8808A1FB3F1 for ; Fri, 15 Nov 2024 21:28:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706081; cv=none; b=h1ZHks6MFTI7PUxlJZIdvPgefOLzwYWy1lIzqG6NAsbjZmuR2FMXP70pZJ2lIeFBV5I4FpW3pcQJSNxG2JXRP4/X6zqNtjkKzIASFyHCvB7AZIX1udaGqDGtHQuR0V3/QIRNmI1TIdBkrHSuXIiJtkcCqU6tT/VaLqgeKDig4Xg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706081; c=relaxed/simple; bh=ky7QqMBksymCTVGtHG3BmsMmLlJN9Wy39+RORoFfwZk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HezarrhKV43a7EMAruKhP40JUF3RN0elAIaJGtxpn2hUEk9xm5OzpcOTChyiJ/A+9B3mX14Rc5Sjbj1189d4axrZZz3iXSiRxeUCtPW/Bssi1GA53w8mosN+g9dwydnWH7hGLS+Kzlyz7lC4sD8EjRFZERbidtX+KIlSBXv+mgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4228C4CECF; Fri, 15 Nov 2024 21:28:00 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 09/20] fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands Date: Fri, 15 Nov 2024 14:25:42 -0700 Message-ID: <20241115212745.869552-10-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a cxl fwctl driver to allow sending of CXL feature commands from userspace through as ioctls. Create a driver skeleton for initial setup. Signed-off-by: Dave Jiang --- v2: - Move to cxl_bus from auxiliary_bus (Jonathan) --- MAINTAINERS | 8 ++++ drivers/cxl/core/mbox.c | 41 +++++++++++++++- drivers/fwctl/Kconfig | 9 ++++ drivers/fwctl/Makefile | 1 + drivers/fwctl/cxl/Makefile | 4 ++ drivers/fwctl/cxl/cxl.c | 97 ++++++++++++++++++++++++++++++++++++++ include/cxl/cxl.h | 1 + include/cxl/mailbox.h | 17 +++++++ include/uapi/fwctl/fwctl.h | 1 + 9 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 drivers/fwctl/cxl/Makefile create mode 100644 drivers/fwctl/cxl/cxl.c diff --git a/MAINTAINERS b/MAINTAINERS index 93f74bc58d99..6b52c4168b94 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9389,6 +9389,14 @@ L: linux-kernel@vger.kernel.org S: Maintained F: drivers/fwctl/mlx5/ +FWCTL CXL DRIVER +M: Dave Jiang +R: Dan Williams +R: Jonathan Cameron +L: linux-cxl@vger.kernel.org +S: Maintained +F: drivers/fwctl/cxl/ + GALAXYCORE GC0308 CAMERA SENSOR DRIVER M: Sebastian Reichel L: linux-media@vger.kernel.org diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 739444d34130..12758e763650 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1712,8 +1712,29 @@ int cxl_poison_state_init(struct cxl_memdev_state *mds) } EXPORT_SYMBOL_NS_GPL(cxl_poison_state_init, CXL); +static void cxl_fwctl_release(struct device *dev) +{ + struct cxl_fwctl *fwctl = to_cxl_fwctl(dev); + + kfree(fwctl); +} + +static void remove_fwctl_dev(void *dev) +{ + device_unregister(dev); +} + +static const struct device_type cxl_fwctl_type = { + .name = "cxl_fwctl", + .release = cxl_fwctl_release, +}; + int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host) { + struct cxl_fwctl *fwctl __free(kfree) = kzalloc(sizeof(*fwctl), GFP_KERNEL); + struct device *dev; + int rc; + if (!cxl_mbox || !host) return -EINVAL; @@ -1721,7 +1742,25 @@ int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host) mutex_init(&cxl_mbox->mbox_mutex); rcuwait_init(&cxl_mbox->mbox_wait); - return 0; + fwctl->cxl_mbox = cxl_mbox; + dev = &fwctl->dev; + device_initialize(dev); + device_set_pm_not_required(dev); + dev->parent = host; + dev->bus = &cxl_bus_type; + dev->type = &cxl_fwctl_type; + + rc = device_add(dev); + if (rc) + goto err; + + cxl_mbox->fwctl = no_free_ptr(fwctl); + + return devm_add_action_or_reset(host, remove_fwctl_dev, dev); + +err: + put_device(&fwctl->dev); + return rc; } EXPORT_SYMBOL_NS_GPL(cxl_mailbox_init, CXL); diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig index e5ee2d46d431..e49903a9d0d3 100644 --- a/drivers/fwctl/Kconfig +++ b/drivers/fwctl/Kconfig @@ -19,5 +19,14 @@ config FWCTL_MLX5 This will allow configuration and debug tools to work out of the box on mainstream kernel. + If you don't know what to do here, say N. + +config FWCTL_CXL + tristate "CXL fwctl driver" + depends on CXL_BUS + help + CXLCTL provides interface for the user process to access user allowed + mailbox commands for CXL device. + If you don't know what to do here, say N. endif diff --git a/drivers/fwctl/Makefile b/drivers/fwctl/Makefile index 1c535f694d7f..bd356e6f2e5a 100644 --- a/drivers/fwctl/Makefile +++ b/drivers/fwctl/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_FWCTL) += fwctl.o obj-$(CONFIG_FWCTL_MLX5) += mlx5/ +obj-$(CONFIG_FWCTL_CXL) += cxl/ fwctl-y += main.o diff --git a/drivers/fwctl/cxl/Makefile b/drivers/fwctl/cxl/Makefile new file mode 100644 index 000000000000..623194521572 --- /dev/null +++ b/drivers/fwctl/cxl/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_FWCTL_CXL) += cxl_fwctl.o + +cxl_fwctl-y += cxl.o diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c new file mode 100644 index 000000000000..c6a11cbbd937 --- /dev/null +++ b/drivers/fwctl/cxl/cxl.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, Intel Corporation + */ +#include +#include +#include +#include + +struct cxlctl_uctx { + struct fwctl_uctx uctx; + u32 nr_commands; +}; + +struct cxlctl_dev { + struct fwctl_device fwctl; + struct cxl_mailbox *mbox; +}; + +DEFINE_FREE(cxlctl, struct cxlctl_dev *, if (_T) fwctl_put(&_T->fwctl)) + +static int cxlctl_open_uctx(struct fwctl_uctx *uctx) +{ + return 0; +} + +static void cxlctl_close_uctx(struct fwctl_uctx *uctx) +{ +} + +static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) +{ + /* Place holder */ + return ERR_PTR(-EOPNOTSUPP); +} + +static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *rpc_in, size_t in_len, size_t *out_len) +{ + /* Place holder */ + return ERR_PTR(-EOPNOTSUPP); +} + +static const struct fwctl_ops cxlctl_ops = { + .device_type = FWCTL_DEVICE_TYPE_CXL, + .uctx_size = sizeof(struct cxlctl_uctx), + .open_uctx = cxlctl_open_uctx, + .close_uctx = cxlctl_close_uctx, + .info = cxlctl_info, + .fw_rpc = cxlctl_fw_rpc, +}; + +static int cxlctl_probe(struct device *dev) +{ + struct cxl_fwctl *cxl_fwctl = to_cxl_fwctl(dev); + struct cxl_mailbox *cxl_mbox = cxl_fwctl->cxl_mbox; + struct cxlctl_dev *cxlctl __free(cxlctl) = + fwctl_alloc_device(cxl_mbox->host, &cxlctl_ops, + struct cxlctl_dev, fwctl); + int rc; + + if (!cxlctl) + return -ENOMEM; + + cxlctl->mbox = cxl_mbox; + + rc = fwctl_register(&cxlctl->fwctl); + if (rc) + return rc; + + dev_set_drvdata(dev, no_free_ptr(cxlctl)); + + return 0; +} + +static void cxlctl_remove(struct device *dev) +{ + struct cxlctl_dev *ctldev = dev_get_drvdata(dev); + + fwctl_unregister(&ctldev->fwctl); +} + +static struct cxl_driver cxl_fwctl_driver = { + .name = "cxl_fwctl", + .probe = cxlctl_probe, + .remove = cxlctl_remove, + .id = CXL_DEVICE_FWCTL, +}; + +module_cxl_driver(cxl_fwctl_driver); + +MODULE_IMPORT_NS(CXL); +MODULE_IMPORT_NS(FWCTL); +MODULE_DESCRIPTION("CXL fwctl driver"); +MODULE_AUTHOR("Intel Corporation"); +MODULE_LICENSE("GPL"); +MODULE_ALIAS_CXL(CXL_DEVICE_FWCTL); diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 07a2983275a0..6bfd7942a3f7 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -30,6 +30,7 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); #define CXL_DEVICE_PMEM_REGION 7 #define CXL_DEVICE_DAX_REGION 8 #define CXL_DEVICE_PMU 9 +#define CXL_DEVICE_FWCTL 10 #define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") #define CXL_MODALIAS_FMT "cxl:t%d" diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 03c4b8ad84c1..16d21f63464c 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -3,6 +3,7 @@ #ifndef __CXL_MBOX_H__ #define __CXL_MBOX_H__ #include +#include #include /** @@ -41,8 +42,23 @@ struct cxl_mbox_cmd { u16 return_code; }; +struct cxl_mailbox; + +/** + * struct cxl_fwctl - context for FWCTL + * @dev: device for fwctl + * @cxl_mbox: pointer to cxl mailbox context + */ +struct cxl_fwctl { + struct device dev; + struct cxl_mailbox *cxl_mbox; +}; + +#define to_cxl_fwctl(dev) container_of(dev, struct cxl_fwctl, dev) + /** * struct cxl_mailbox - context for CXL mailbox operations + * @fwctl: points to fwctl context * @host: device that hosts the mailbox * @enabled_cmds: mailbox commands that are enabled by the driver * @exclusive_cmds: mailbox commands that are exclusive to the kernel @@ -55,6 +71,7 @@ struct cxl_mbox_cmd { * @mbox_send: @dev specific transport for transmitting mailbox commands */ struct cxl_mailbox { + struct cxl_fwctl *fwctl; struct device *host; DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index f9b27fb5c161..4e4d30104667 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -43,6 +43,7 @@ enum { enum fwctl_device_type { FWCTL_DEVICE_TYPE_ERROR = 0, FWCTL_DEVICE_TYPE_MLX5 = 1, + FWCTL_DEVICE_TYPE_CXL = 2, }; /** From patchwork Fri Nov 15 21:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877293 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B9AB1FB3CF for ; Fri, 15 Nov 2024 21:28:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706083; cv=none; b=qecLJtgyHDTOJr6a/2n6PPE0Nds0wAYpMhnrJi44SS+L49nrTToj6KDaUL/2/mchzJE4inVnRObwzbnHOYSZCM2H5ETiMoQ+t1lB9onzwMOoZOITt+qnVKberFcRzUgcPw66FgsnuhmhkUAPmILpUa6TDbTm9e6tOqmrV2jzx8w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706083; c=relaxed/simple; bh=5f34DlUEiXdQEyWZe3WO9xzFVT135Bgjz52N124eNjE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=laIWqlNHBCqKG6cie6vHKJTDcjeMhr6Gor7ehz517yuITAWDMVu4BaPTCg17c8erpkKMI9U9gQcc7E9Co3QbgPabJd6BfPCaZyZ4m8QOQonoVXxODZpG6KarEBuZN86SebrbePtXFxYTEckEs3iFJG9w0MZaGY/4l48prOqJb0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B4ABC4CECF; Fri, 15 Nov 2024 21:28:02 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 10/20] fwctl/cxl: Add support for get driver information Date: Fri, 15 Nov 2024 14:25:43 -0700 Message-ID: <20241115212745.869552-11-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add definition for fwctl_ops->info() to return driver information. The function will return the number of device mailbox commands supported by the fwctl char device. Signed-off-by: Dave Jiang --- v2: - Change driver info to report number of commands supported by the driver and device. --- drivers/cxl/core/mbox.c | 46 ++++++++++++++++++++++++++++++++++++++++ drivers/fwctl/cxl/cxl.c | 23 ++++++++++++++++++-- include/cxl/cxl.h | 2 ++ include/cxl/mailbox.h | 1 + include/uapi/fwctl/cxl.h | 22 +++++++++++++++++++ 5 files changed, 92 insertions(+), 2 deletions(-) create mode 100644 include/uapi/fwctl/cxl.h diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 12758e763650..f464eb42f08a 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -118,6 +118,14 @@ static u8 security_command_sets[] = { 0x46, /* Security Passthrough */ }; +#define FWCTL_CXL_MAX_COMMANDS 3 +/* Command set that is allowed with FWCTL-CXL */ +static u16 fwctl_command_sets[] = { + CXL_MBOX_OP_GET_SUPPORTED_FEATURES, + CXL_MBOX_OP_GET_FEATURE, + CXL_MBOX_OP_SET_FEATURE, +}; + static bool cxl_is_security_command(u16 opcode) { int i; @@ -527,6 +535,44 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, send_cmd->in.payload); } +static struct cxl_mem_command * +fwctl_cxl_find_command(struct cxl_mailbox *cxl_mbox, u16 opcode) +{ + struct cxl_command_info *info; + struct cxl_mem_command *cmd; + + cmd = cxl_mem_find_command(opcode); + if (!cmd) + return NULL; + + info = &cmd->info; + if (test_bit(info->id, cxl_mbox->enabled_cmds) && + !test_bit(info->id, cxl_mbox->exclusive_cmds)) + return cmd; + + return NULL; +} + +/** + * cxl_mailbox_user_commands_supported() - Return number of user mailbox + * commands supported. + * @cxl_mbox: cxl mailbox context + * + * Return: number of commands supported + */ +int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox) +{ + int nr_cmds = 0; + + for (int i = 0; i < FWCTL_CXL_MAX_COMMANDS; i++) { + nr_cmds += !!fwctl_cxl_find_command(cxl_mbox, + fwctl_command_sets[i]); + } + + return nr_cmds; +} +EXPORT_SYMBOL_NS_GPL(cxl_mailbox_user_commands_supported, CXL); + int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q) { diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c index c6a11cbbd937..5eb5eabf2bff 100644 --- a/drivers/fwctl/cxl/cxl.c +++ b/drivers/fwctl/cxl/cxl.c @@ -6,6 +6,7 @@ #include #include #include +#include struct cxlctl_uctx { struct fwctl_uctx uctx; @@ -21,6 +22,15 @@ DEFINE_FREE(cxlctl, struct cxlctl_dev *, if (_T) fwctl_put(&_T->fwctl)) static int cxlctl_open_uctx(struct fwctl_uctx *uctx) { + struct cxlctl_uctx *cxlctl_uctx = + container_of(uctx, struct cxlctl_uctx, uctx); + struct fwctl_device *fwctl = uctx->fwctl; + struct cxlctl_dev *cxlctl = + container_of(fwctl, struct cxlctl_dev, fwctl); + + cxlctl_uctx->nr_commands = + cxl_mailbox_user_commands_supported(cxlctl->mbox); + return 0; } @@ -30,8 +40,17 @@ static void cxlctl_close_uctx(struct fwctl_uctx *uctx) static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) { - /* Place holder */ - return ERR_PTR(-EOPNOTSUPP); + struct cxlctl_uctx *cxlctl_uctx = + container_of(uctx, struct cxlctl_uctx, uctx); + struct fwctl_info_cxl *info; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return ERR_PTR(-ENOMEM); + + info->nr_commands = cxlctl_uctx->nr_commands; + + return info; } static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h index 6bfd7942a3f7..d0939e3bcbc0 100644 --- a/include/cxl/cxl.h +++ b/include/cxl/cxl.h @@ -3,6 +3,8 @@ #ifndef __CXL_GLOBAL_H__ #define __CXL_GLOBAL_H__ +#include + struct cxl_driver { const char *name; int (*probe)(struct device *dev); diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 16d21f63464c..af01fb78ae10 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -84,5 +84,6 @@ struct cxl_mailbox { }; int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host); +int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox); #endif diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h new file mode 100644 index 000000000000..a32c4c752db6 --- /dev/null +++ b/include/uapi/fwctl/cxl.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024, Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_FWCTL_CXL_H_ +#define _UAPI_FWCTL_CXL_H_ + +#include + +/** + * struct fwctl_info_cxl - ioctl(FWCTL_INFO) out_device_data + * @uctx_caps: The number of commands the driver and device supports + * + * Return basic information about the FW interface available. + */ +struct fwctl_info_cxl { + __u32 nr_commands; +}; + +#endif From patchwork Fri Nov 15 21:25:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877294 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6065E1FB3F5 for ; Fri, 15 Nov 2024 21:28:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706084; cv=none; b=gP9rgDL7+exPqWaOhDTxdJ+6wD0v9oB4Am52sp5mI7HeiKqfGLhynb8nZAPBwrbKQ6hI2tJZ/A8oFZLs1WMp0OaJ3r0xpYeaJoSkURGmfG6O/bUaBuatk0wx3vO5gNygvnwcA/Pga5K1fV7dUeMpmnBj9kstB3ht5y72GLUDB7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706084; c=relaxed/simple; bh=dVEThEJbJl4cF7dZgmCIVrswqNbztqZ3ouzQeLytH7s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HTkEkETpDaSecmduGBPiC/R6xVFkyphfRKYwLqcrNdcRxIN82mfSrkalyx4dpP+Oku5Mq1xfjDXO2erAiRD3VVLQDO4AWiKqYmy7RS3qs0Tu3nWAHOv7Exla44sJs53UEByavdCecPzz+stuqq0IEJZqP6SF9W/+cZskgTwb4mQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7AD6C4CECF; Fri, 15 Nov 2024 21:28:03 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 11/20] fwctl: FWCTL_HW_INFO to return hardware information Date: Fri, 15 Nov 2024 14:25:44 -0700 Message-ID: <20241115212745.869552-12-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add an optional ioctl FWCTL_HW_INFO to pass command specific information to user space. An array of 'struct fwctl_command_info' will be returned from the ioctl. These commands are send to the driver via FWCTL_RPC call. The command info struct contains the command id, the related hardware opcode, input and output size for the command, and the effects the command has if it's a write command. Signed-off-by: Dave Jiang --- drivers/fwctl/main.c | 33 ++++++++++++++++++++++ include/linux/fwctl.h | 6 ++++ include/uapi/fwctl/fwctl.h | 58 +++++++++++++++++++++++++++++++++++++- 3 files changed, 96 insertions(+), 1 deletion(-) diff --git a/drivers/fwctl/main.c b/drivers/fwctl/main.c index 54a7356e586c..073e3d20558b 100644 --- a/drivers/fwctl/main.c +++ b/drivers/fwctl/main.c @@ -77,6 +77,38 @@ static int fwctl_cmd_info(struct fwctl_ucmd *ucmd) return ucmd_respond(ucmd, sizeof(*cmd)); } +static int fwctl_hw_cmd_info(struct fwctl_ucmd *ucmd) +{ + struct fwctl_device *fwctl = ucmd->uctx->fwctl; + struct fwctl_hw_info *cmd = ucmd->cmd; + size_t out_len; + int nr_cmds; + + if (cmd->flags) + return -EOPNOTSUPP; + + nr_cmds = cmd->nr_cmds; + if (!nr_cmds) + return -EINVAL; + + out_len = sizeof(struct fwctl_hw_info_out); + out_len += sizeof(struct fwctl_command_info) * nr_cmds; + + if (cmd->device_data_len < out_len) + return -EINVAL; + + void *outbuf __free(kvfree) = fwctl->ops->hw_info(ucmd->uctx, nr_cmds, + &out_len); + if (IS_ERR(outbuf)) + return PTR_ERR(outbuf); + + if (copy_to_user(u64_to_user_ptr(cmd->out_device_data), outbuf, out_len)) + return -EFAULT; + + cmd->device_data_len = out_len; + return ucmd_respond(ucmd, sizeof(*cmd)); +} + static int fwctl_cmd_rpc(struct fwctl_ucmd *ucmd) { struct fwctl_device *fwctl = ucmd->uctx->fwctl; @@ -156,6 +188,7 @@ struct fwctl_ioctl_op { } static const struct fwctl_ioctl_op fwctl_ioctl_ops[] = { IOCTL_OP(FWCTL_INFO, fwctl_cmd_info, struct fwctl_info, out_device_data), + IOCTL_OP(FWCTL_HW_INFO, fwctl_hw_cmd_info, struct fwctl_info, out_device_data), IOCTL_OP(FWCTL_RPC, fwctl_cmd_rpc, struct fwctl_rpc, out), }; diff --git a/include/linux/fwctl.h b/include/linux/fwctl.h index 6eac9497ff1a..ccef64da710c 100644 --- a/include/linux/fwctl.h +++ b/include/linux/fwctl.h @@ -47,6 +47,12 @@ struct fwctl_ops { * ignore length on input, the core code will handle everything. */ void *(*info)(struct fwctl_uctx *uctx, size_t *length); + /** + * @hw_info: Implement FWCTL_HW_INFO. Return a kmalloc() memory that is + * copied to out_device_dataa. On input commands indicates the number + * of command info structs requested. + */ + void *(*hw_info)(struct fwctl_uctx *uctx, int commands, size_t *out_len); /** * @fw_rpc: Implement FWCTL_RPC. Deliver rpc_in/in_len to the FW and * return the response and set out_len. rpc_in can be returned as the diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 4e4d30104667..7334907e27c1 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -37,7 +37,8 @@ enum { FWCTL_CMD_BASE = 0, FWCTL_CMD_INFO = 0, - FWCTL_CMD_RPC = 1, + FWCTL_CMD_HW_INFO = 1, + FWCTL_CMD_RPC = 2, }; enum fwctl_device_type { @@ -69,6 +70,61 @@ struct fwctl_info { }; #define FWCTL_INFO _IO(FWCTL_TYPE, FWCTL_CMD_INFO) +/** + * struct fwctl_command_info - Hardware command information returned from a query. + * @id: Driver ID number for the command + * @opcode: Hardware command opcode + * @set_effects: Effects to the OS and hardware when command is executed. + * Provided by the device. + * @size_in: Expected input size, or ~0U if variable length. + * @size_out: Expected output size, or ~0U if variable length. + * + * Represents a single command that is supported by both the driver and the + * hardware. This is returned as part of an array from the FWCTL_HW_INFO ioctl. + */ +struct fwctl_command_info { + __u32 id; + __u16 opcode; + __u16 effects; + __u32 size_in; + __u32 size_out; +}; + +/** + * struct fwctl_hw_info_out - output struct for FWCTL_HW_INFO + * @nr_cmds: Number of commands for output + * @reserved: Reserved u32 for alignment. + * @commands: Array of 'struct fwctl_command_info' + */ +struct fwctl_hw_info_out { + __u32 nr_cmds; + __u32 reserved; + + struct fwctl_command_info commands[] __counted_by(nr_cmds); +}; + +/** + * struct fwctl_hw_info - ioctl(FWCTL_HW_INFO) + * @size: sizeof(struct fwctl_hw_info) + * @flags: Must be 0 + * @device_data_len: On input the length of the out_device_data memory. On + * output the size of the kernel's device_data which may be larger or + * smaller than the input. Maybe 0 on input. + * @nr_cmds: Number of commands requested. 0 returns only nr_cmds in output. + * @out_device_data: Pointer to a memory of device_data_len bytes. Kernel will + * fill the entire memory, zeroing as required. + * + * Returns hardware commands information about this fwctl instance. + */ +struct fwctl_hw_info { + __u32 size; + __u32 flags; + __u32 device_data_len; + __u32 nr_cmds; + __aligned_u64 out_device_data; +}; +#define FWCTL_HW_INFO _IO(FWCTL_TYPE, FWCTL_CMD_HW_INFO) + /** * enum fwctl_rpc_scope - Scope of access for the RPC * From patchwork Fri Nov 15 21:25:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877296 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2C391FB3F8 for ; Fri, 15 Nov 2024 21:28:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706086; cv=none; b=oGL22o2s6h124D8v8FlIMI3Y0LQe+fHoC7BI2H8eKcrPI7JWgBp8n1q/fFTkfBHsNDjWBFhuuHJQPoKq/4d/cYCItmN8ko3K6+9Tg7tPz7B/R/vBtDEzijjSao6ydPbOxROH4AdQzS8gTRLKMRCIfBemuJkL3OzZIGajbAF2c0Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706086; c=relaxed/simple; bh=I+D+Hn2hmqk4QgdVsTxVBuZBTO9ampZTBon06QPd39U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dNT5V784+cZ4BCZooCyJxcS4WzoxOhxdoKBvctLdBtaLesgcz5JqfMU1lJaf+0KMaoxdg6hgnhoP3V84xv32JW1owIkDG2rCxScvPCez2P0FgVT1xQTx0aTo3z1oF+e2sd1wuYXcOQvWj+f8WLDKm80UngMztbTTauHX3CNOZp0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 451B1C4CECF; Fri, 15 Nov 2024 21:28:05 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 12/20] cxl: Save Command Effects Log (CEL) effects for enabled commands Date: Fri, 15 Nov 2024 14:25:45 -0700 Message-ID: <20241115212745.869552-13-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Store the command effects via what device provided via the CEL in the 'struct cxl_command_info' structure. Steal the upper 16 bits to store the effects value in order to keep user API compatibility. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 2 ++ include/uapi/fwctl/fwctl.h | 4 ++-- include/uapi/linux/cxl_mem.h | 5 +++-- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index f464eb42f08a..fba6bdd30a82 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -795,6 +795,8 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) enabled++; } + cmd->info.effects = le16_to_cpu(cel_entry[i].effect); + dev_dbg(dev, "Opcode 0x%04x %s\n", opcode, enabled ? "enabled" : "unsupported by driver"); } diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index 7334907e27c1..04da549cd4ae 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -74,8 +74,8 @@ struct fwctl_info { * struct fwctl_command_info - Hardware command information returned from a query. * @id: Driver ID number for the command * @opcode: Hardware command opcode - * @set_effects: Effects to the OS and hardware when command is executed. - * Provided by the device. + * @effects: Effects to the OS and hardware when command is executed. + * Provided by the device. * @size_in: Expected input size, or ~0U if variable length. * @size_out: Expected output size, or ~0U if variable length. * diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index 469dd481795f..9dd37849c450 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -118,6 +118,7 @@ static const __u8 cxl_deprecated_commands[] * the label-storage-area can not be written while the kernel is * actively managing that space. * + * @effects: Command effects reported by the device. * @size_in: Expected input size, or ~0 if variable length. * @size_out: Expected output size, or ~0 if variable length. * @@ -136,11 +137,11 @@ static const __u8 cxl_deprecated_commands[] struct cxl_command_info { __u32 id; - __u32 flags; + __u16 flags; #define CXL_MEM_COMMAND_FLAG_MASK GENMASK(1, 0) #define CXL_MEM_COMMAND_FLAG_ENABLED BIT(0) #define CXL_MEM_COMMAND_FLAG_EXCLUSIVE BIT(1) - + __u16 effects; __u32 size_in; __u32 size_out; }; From patchwork Fri Nov 15 21:25:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877295 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D871A1FB3F5 for ; Fri, 15 Nov 2024 21:28:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706086; cv=none; b=UHQcUJ7Y39Uu3wh/a4dNyB5sy4UnH3W73hifTzTQj1eS4KUIj1PzDvLuve3SwVgOQm4xmOW+I1lSVKR1JX0bOJROnPR2pv0vkXoShIfxji3C1LOWNrISHe9KowReq8jNjmpOvo+1QIoDDrDW19d9fHnT6sjC8z6rRr4IL5AhpJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706086; c=relaxed/simple; bh=8DrqE4E084lX6hxn4sbU1M2i5AiIWgGsLS8TQJS2ewY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MRTRtgaXfJ9OIbKTQDTRhyhOydPE0zaj1RQJnKHX0v1WQMQkcqtslI80OvRJl7oxrs53goHMMq4IdLVDww6qIjdD6v5SqJMlj3TM59BtOiqqe88V75GUtu1CAx6IONN0TZkgLzVUWC1N1NNkcS7FHLbVcn0OqQGTdfPJ2C6JtFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3F5BC4CED4; Fri, 15 Nov 2024 21:28:06 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 13/20] fwctl/cxl: Add hw_info callback Date: Fri, 15 Nov 2024 14:25:46 -0700 Message-ID: <20241115212745.869552-14-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 42 +++++++++++++++++++++++++++++++++++++++++ drivers/fwctl/cxl/cxl.c | 25 ++++++++++++++++++++++++ include/cxl/mailbox.h | 2 ++ 3 files changed, 69 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index fba6bdd30a82..65fceabb9fe7 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -9,6 +9,8 @@ #include #include +#include + #include "core.h" #include "trace.h" @@ -573,6 +575,46 @@ int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox) } EXPORT_SYMBOL_NS_GPL(cxl_mailbox_user_commands_supported, CXL); +/** + * cxl_mailbox_user_commands_info_get() - Retrieve array of command info + * @cxl_mbox: cxl mailbox context + * @nr_cmds: number of commands to retrieve + * @outbuf: Output buffer to store array 'struct fwctl_command_info' + * @out_len: size of final output buffer + * + * Return: 0 for success, or -errno for failure. + */ +int cxl_mailbox_user_commands_info_get(struct cxl_mailbox *cxl_mbox, int nr_cmds, + void *outbuf, size_t *out_len) +{ + struct fwctl_command_info *entry; + struct cxl_command_info *info; + struct cxl_mem_command *cmd; + + if (nr_cmds > FWCTL_CXL_MAX_COMMANDS) + return -EINVAL; + + entry = outbuf; + for (int i = 0; i < nr_cmds; i++) { + cmd = fwctl_cxl_find_command(cxl_mbox, fwctl_command_sets[i]); + if (!cmd) + continue; + info = &cmd->info; + memset(entry, 0, sizeof(*entry)); + entry->id = info->id; + entry->opcode = fwctl_command_sets[i]; + entry->effects = info->effects; + entry->size_in = info->size_in; + entry->size_out = info->size_out; + entry++; + } + + *out_len = sizeof(*entry) * nr_cmds; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_mailbox_user_commands_info_get, CXL); + int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q) { diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c index 5eb5eabf2bff..ce8960a9beaa 100644 --- a/drivers/fwctl/cxl/cxl.c +++ b/drivers/fwctl/cxl/cxl.c @@ -53,6 +53,30 @@ static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) return info; } +static void *cxlctl_hw_info(struct fwctl_uctx *uctx, int commands, size_t *out_len) +{ + struct cxlctl_uctx *cxlctl_uctx = + container_of(uctx, struct cxlctl_uctx, uctx); + struct fwctl_device *fwctl = uctx->fwctl; + struct cxlctl_dev *cxlctl = + container_of(fwctl, struct cxlctl_dev, fwctl); + int rc; + + if (commands > cxlctl_uctx->nr_commands) + return ERR_PTR(-EINVAL); + + void *out __free(kvfree) = kvzalloc(*out_len, GFP_KERNEL); + if (!out) + return ERR_PTR(-ENOMEM); + + rc = cxl_mailbox_user_commands_info_get(cxlctl->mbox, + commands, out, out_len); + if (rc) + return ERR_PTR(rc); + + return_ptr(out); +} + static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, void *rpc_in, size_t in_len, size_t *out_len) { @@ -66,6 +90,7 @@ static const struct fwctl_ops cxlctl_ops = { .open_uctx = cxlctl_open_uctx, .close_uctx = cxlctl_close_uctx, .info = cxlctl_info, + .hw_info = cxlctl_hw_info, .fw_rpc = cxlctl_fw_rpc, }; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index af01fb78ae10..f48eebb855f7 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -85,5 +85,7 @@ struct cxl_mailbox { int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host); int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox); +int cxl_mailbox_user_commands_info_get(struct cxl_mailbox *cxl_mbox, int nr_cmds, + void *outbuf, size_t *out_len); #endif From patchwork Fri Nov 15 21:25:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877297 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 458551FB3FE for ; Fri, 15 Nov 2024 21:28:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706088; cv=none; b=FMfVIRQ+/gLynHDmMF0FCPkUfTQC0I6g/rPF83K5VybIg1qnpxQg+VLC1q7Mn1m7xdrKniwZGiNsPjhcfZ5kWwN9pH4sqxHhvy36hMc88hOHW/oUl5O+cRsykbGJ0BrPZNqt0654msMyLnJrhASBSMVv/w9pSdMk1JfAK4Wqa0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706088; c=relaxed/simple; bh=vvlGqONTOfplpuACSyO+2rCQkLOT2wlGxjI1c5lebns=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dblj340Zvawqd1t4zNVqTP4wH24Nr3A7zJRPwQoYHEBa8hunD67OA5vgE5YSWxpWvyImGnq95qd8cTPPDct/Gr3dN6KaPvS6A+ctatRou8NYfUchA4hHGGiFQfzftf4+V0Eq5y0WFMT01UXQCT38Eo5ZmUy1FrCMoVVYPst+JNU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0183AC4CECF; Fri, 15 Nov 2024 21:28:07 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 14/20] cxl: Move defines and error codes from cxlmem.h to cxl/mailbox.h Date: Fri, 15 Nov 2024 14:25:47 -0700 Message-ID: <20241115212745.869552-15-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Moving some internal definitions to cxl/mailbox.h in order to be accessed by FWCTL CXL driver. Signed-off-by: Dave Jiang --- drivers/cxl/cxlmem.h | 83 --------------------------- include/cxl/mailbox.h | 106 +++++++++++++++++++++++++++++++++++ tools/testing/cxl/test/mem.c | 2 +- 3 files changed, 107 insertions(+), 84 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 2c15817c80a5..890ed3199235 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -473,53 +473,6 @@ to_cxl_memdev_state(struct cxl_dev_state *cxlds) return container_of(cxlds, struct cxl_memdev_state, cxlds); } -enum cxl_opcode { - CXL_MBOX_OP_INVALID = 0x0000, - CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, - CXL_MBOX_OP_GET_EVENT_RECORD = 0x0100, - CXL_MBOX_OP_CLEAR_EVENT_RECORD = 0x0101, - CXL_MBOX_OP_GET_EVT_INT_POLICY = 0x0102, - CXL_MBOX_OP_SET_EVT_INT_POLICY = 0x0103, - CXL_MBOX_OP_GET_FW_INFO = 0x0200, - CXL_MBOX_OP_TRANSFER_FW = 0x0201, - CXL_MBOX_OP_ACTIVATE_FW = 0x0202, - CXL_MBOX_OP_GET_TIMESTAMP = 0x0300, - CXL_MBOX_OP_SET_TIMESTAMP = 0x0301, - CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400, - CXL_MBOX_OP_GET_LOG = 0x0401, - CXL_MBOX_OP_GET_LOG_CAPS = 0x0402, - CXL_MBOX_OP_CLEAR_LOG = 0x0403, - CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, - CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, - CXL_MBOX_OP_GET_FEATURE = 0x0501, - CXL_MBOX_OP_SET_FEATURE = 0x0502, - CXL_MBOX_OP_IDENTIFY = 0x4000, - CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, - CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, - CXL_MBOX_OP_GET_LSA = 0x4102, - CXL_MBOX_OP_SET_LSA = 0x4103, - CXL_MBOX_OP_GET_HEALTH_INFO = 0x4200, - CXL_MBOX_OP_GET_ALERT_CONFIG = 0x4201, - CXL_MBOX_OP_SET_ALERT_CONFIG = 0x4202, - CXL_MBOX_OP_GET_SHUTDOWN_STATE = 0x4203, - CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204, - CXL_MBOX_OP_GET_POISON = 0x4300, - CXL_MBOX_OP_INJECT_POISON = 0x4301, - CXL_MBOX_OP_CLEAR_POISON = 0x4302, - CXL_MBOX_OP_GET_SCAN_MEDIA_CAPS = 0x4303, - CXL_MBOX_OP_SCAN_MEDIA = 0x4304, - CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305, - CXL_MBOX_OP_SANITIZE = 0x4400, - CXL_MBOX_OP_SECURE_ERASE = 0x4401, - CXL_MBOX_OP_GET_SECURITY_STATE = 0x4500, - CXL_MBOX_OP_SET_PASSPHRASE = 0x4501, - CXL_MBOX_OP_DISABLE_PASSPHRASE = 0x4502, - CXL_MBOX_OP_UNLOCK = 0x4503, - CXL_MBOX_OP_FREEZE_SECURITY = 0x4504, - CXL_MBOX_OP_PASSPHRASE_SECURE_ERASE = 0x4505, - CXL_MBOX_OP_MAX = 0x10000 -}; - #define DEFINE_CXL_CEL_UUID \ UUID_INIT(0xda9c0b5, 0xbf41, 0x4b78, 0x8f, 0x79, 0x96, 0xb1, 0x62, \ 0x3b, 0x3f, 0x17) @@ -717,30 +670,6 @@ struct cxl_mbox_clear_poison { u8 write_data[CXL_POISON_LEN_MULT]; } __packed; -/** - * struct cxl_mem_command - Driver representation of a memory device command - * @info: Command information as it exists for the UAPI - * @opcode: The actual bits used for the mailbox protocol - * @flags: Set of flags effecting driver behavior. - * - * * %CXL_CMD_FLAG_FORCE_ENABLE: In cases of error, commands with this flag - * will be enabled by the driver regardless of what hardware may have - * advertised. - * - * The cxl_mem_command is the driver's internal representation of commands that - * are supported by the driver. Some of these commands may not be supported by - * the hardware. The driver will use @info to validate the fields passed in by - * the user then submit the @opcode to the hardware. - * - * See struct cxl_command_info. - */ -struct cxl_mem_command { - struct cxl_command_info info; - enum cxl_opcode opcode; - u32 flags; -#define CXL_CMD_FLAG_FORCE_ENABLE BIT(0) -}; - #define CXL_PMEM_SEC_STATE_USER_PASS_SET 0x01 #define CXL_PMEM_SEC_STATE_MASTER_PASS_SET 0x02 #define CXL_PMEM_SEC_STATE_LOCKED 0x04 @@ -783,18 +712,6 @@ struct cxl_mbox_get_sup_feats_in { u8 reserved[2]; } __packed; -struct cxl_feat_entry { - uuid_t uuid; - __le16 id; - __le16 get_feat_size; - __le16 set_feat_size; - __le32 flags; - u8 get_feat_ver; - u8 set_feat_ver; - __le16 set_effects; - u8 reserved[18]; -} __packed; - struct cxl_mbox_get_sup_feats_out { __le16 num_entries; __le16 supported_feats; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index f48eebb855f7..e753d5d1d708 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -4,6 +4,7 @@ #define __CXL_MBOX_H__ #include #include +#include #include /** @@ -83,6 +84,111 @@ struct cxl_mailbox { int (*mbox_send)(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); }; +enum cxl_opcode { + CXL_MBOX_OP_INVALID = 0x0000, + CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID, + CXL_MBOX_OP_GET_EVENT_RECORD = 0x0100, + CXL_MBOX_OP_CLEAR_EVENT_RECORD = 0x0101, + CXL_MBOX_OP_GET_EVT_INT_POLICY = 0x0102, + CXL_MBOX_OP_SET_EVT_INT_POLICY = 0x0103, + CXL_MBOX_OP_GET_FW_INFO = 0x0200, + CXL_MBOX_OP_TRANSFER_FW = 0x0201, + CXL_MBOX_OP_ACTIVATE_FW = 0x0202, + CXL_MBOX_OP_GET_TIMESTAMP = 0x0300, + CXL_MBOX_OP_SET_TIMESTAMP = 0x0301, + CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400, + CXL_MBOX_OP_GET_LOG = 0x0401, + CXL_MBOX_OP_GET_LOG_CAPS = 0x0402, + CXL_MBOX_OP_CLEAR_LOG = 0x0403, + CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, + CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, + CXL_MBOX_OP_GET_FEATURE = 0x0501, + CXL_MBOX_OP_SET_FEATURE = 0x0502, + CXL_MBOX_OP_IDENTIFY = 0x4000, + CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, + CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, + CXL_MBOX_OP_GET_LSA = 0x4102, + CXL_MBOX_OP_SET_LSA = 0x4103, + CXL_MBOX_OP_GET_HEALTH_INFO = 0x4200, + CXL_MBOX_OP_GET_ALERT_CONFIG = 0x4201, + CXL_MBOX_OP_SET_ALERT_CONFIG = 0x4202, + CXL_MBOX_OP_GET_SHUTDOWN_STATE = 0x4203, + CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204, + CXL_MBOX_OP_GET_POISON = 0x4300, + CXL_MBOX_OP_INJECT_POISON = 0x4301, + CXL_MBOX_OP_CLEAR_POISON = 0x4302, + CXL_MBOX_OP_GET_SCAN_MEDIA_CAPS = 0x4303, + CXL_MBOX_OP_SCAN_MEDIA = 0x4304, + CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305, + CXL_MBOX_OP_SANITIZE = 0x4400, + CXL_MBOX_OP_SECURE_ERASE = 0x4401, + CXL_MBOX_OP_GET_SECURITY_STATE = 0x4500, + CXL_MBOX_OP_SET_PASSPHRASE = 0x4501, + CXL_MBOX_OP_DISABLE_PASSPHRASE = 0x4502, + CXL_MBOX_OP_UNLOCK = 0x4503, + CXL_MBOX_OP_FREEZE_SECURITY = 0x4504, + CXL_MBOX_OP_PASSPHRASE_SECURE_ERASE = 0x4505, + CXL_MBOX_OP_MAX = 0x10000 +}; + +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET (BIT(9) | BIT(10)) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET (BIT(9) | BIT(11)) + +/** + * struct cxl_feat_entry - CXL Spec r3.1 Table 8-97 + * @uuid: Feature identifier + * @id: Feature Index + * @get_feat_size: Get Feature Size + * @set_feat_size: Set Feature Size + * @flags: Attribute Flags + * @get_feat_ver: Get Feature Version + * @set_feat_ver: Set Feature Version + * @reserved: reserved, must be 0 + */ +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + u8 get_feat_ver; + u8 set_feat_ver; + __le16 effects; + u8 reserved[18]; +} __packed; + +/** + * struct cxl_mem_command - Driver representation of a memory device command + * @info: Command information as it exists for the UAPI + * @opcode: The actual bits used for the mailbox protocol + * @flags: Set of flags effecting driver behavior. + * + * * %CXL_CMD_FLAG_FORCE_ENABLE: In cases of error, commands with this flag + * will be enabled by the driver regardless of what hardware may have + * advertised. + * + * The cxl_mem_command is the driver's internal representation of commands that + * are supported by the driver. Some of these commands may not be supported by + * the hardware. The driver will use @info to validate the fields passed in by + * the user then submit the @opcode to the hardware. + * + * See struct cxl_command_info. + */ +struct cxl_mem_command { + struct cxl_command_info info; + enum cxl_opcode opcode; + u32 flags; +#define CXL_CMD_FLAG_FORCE_ENABLE BIT(0) +}; + int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host); int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox); int cxl_mailbox_user_commands_info_get(struct cxl_mailbox *cxl_mbox, int nr_cmds, diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index e0c7e49795ed..9103bac054b9 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1354,7 +1354,7 @@ static void fill_feature_vendor_test(struct cxl_feat_entry *feat) feat->flags = cpu_to_le32(BIT(5)); feat->get_feat_ver = 1; feat->set_feat_ver = 1; - feat->set_effects = cpu_to_le16(BIT(0) | BIT(9)); + feat->effects = cpu_to_le16(BIT(0) | BIT(9)); } static int mock_get_supported_features(struct cxl_mockmem_data *mdata, From patchwork Fri Nov 15 21:25:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877298 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BF8C1F470B for ; Fri, 15 Nov 2024 21:28:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706089; cv=none; b=fuk5//grakZpJxeVTODK4kcrFmSoODZxiwZlqxAr25z5yq0muGzMeEPfDSVAYKLHLz9rzgrC0sTK4b0m3M+gVQ4bJLwhhc6AfFmuuXpd9t150WujVKnqa0MzMQAycU1BILEWDVP1heBw/vkwjp4ojB8Og/0Y8qdsN+nvh+OjKeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706089; c=relaxed/simple; bh=kuN+DknImRDX6fkJpCDM3zJdDMjxfUrMnNn1RqbQiss=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dj59gn6hy/6N/h1u40bgouJxs16nh61Zurrof8jyAvYSvVvu7KuA8LvTuLvaFd4dCPvUWN0SBQYFPWe4lpykHovfz5394k9pYeq2drujPIRuG+Sz/bFofvb7phKa4mw11VHV0JW98LBDomj8q9gcAndyYRwBQmU1jU+Aq+MlUzM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5C8D0C4CECF; Fri, 15 Nov 2024 21:28:09 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 15/20] fwctl/cxl: Add support for fwctl RPC command to enable CXL feature commands Date: Fri, 15 Nov 2024 14:25:48 -0700 Message-ID: <20241115212745.869552-16-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 fwctl provides a fwctl_ops->fw_rpc() callback in order to issue ioctls to a device. The cxl fwctl driver will start by supporting the CXL feature commands: Get Supported Features, Get Feature, and Set Feature. The fw_rpc() callback provides 'enum fwctl_rpc_scope' parameter where it indicates the security scope of the call. The Get Supported Features and Get Feature calls can be executed with the scope of FWCTL_RPC_DEBUG_READ_ONLY. The Set Feature call is gated by the effects of the feature reported by Get Supported Features call for the specific feature. Signed-off-by: Dave Jiang --- drivers/cxl/core/core.h | 5 +- drivers/cxl/core/mbox.c | 260 +++++++++++++++++++++++++++++++---- drivers/cxl/core/memdev.c | 4 +- drivers/fwctl/cxl/cxl.c | 101 +++++++++++++- include/cxl/mailbox.h | 6 + include/uapi/fwctl/cxl.h | 29 ++++ include/uapi/linux/cxl_mem.h | 12 ++ 7 files changed, 387 insertions(+), 30 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index ff30d1c99ca7..2b28fd8845d6 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -6,6 +6,8 @@ #include +extern struct rw_semaphore cxl_memdev_rwsem; + extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; extern const struct device_type cxl_pmu_type; @@ -69,7 +71,8 @@ struct cxl_send_command; struct cxl_mem_query_commands; int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q); -int cxl_send_cmd(struct cxl_mailbox *cxl_mailbox, struct cxl_send_command __user *s); +int cxl_send_cmd_from_user(struct cxl_mailbox *cxl_mbox, + struct cxl_send_command __user *s); void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, resource_size_t length); diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 65fceabb9fe7..e7c5c709ac79 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -225,6 +226,38 @@ static struct cxl_mem_command *cxl_mem_find_command(u16 opcode) return NULL; } +static struct cxl_mem_command *cxl_mem_find_command_by_id(int id) +{ + struct cxl_mem_command *c; + + cxl_for_each_cmd(c) + if (c->info.id == id) + return c; + + return NULL; + +} + +struct cxl_mem_command * +cxl_get_mem_command_for_fwctl(struct cxl_mailbox *cxl_mbox, u32 id) +{ + struct cxl_mem_command *cmd; + + if (id > CXL_MEM_COMMAND_ID_MAX) + return NULL; + + cmd = &cxl_mem_commands[id]; + + if (!test_bit(cmd->info.id, cxl_mbox->enabled_cmds)) + return NULL; + + if (test_bit(cmd->info.id, cxl_mbox->exclusive_cmds)) + return NULL; + + return cmd; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_mem_command_for_fwctl, CXL); + static const char *cxl_mem_opcode_to_name(u16 opcode) { struct cxl_mem_command *c; @@ -387,10 +420,13 @@ static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd, } /* Prepare to handle a full payload for variable sized output */ - if (out_size == CXL_VARIABLE_PAYLOAD) - mbox_cmd->size_out = cxl_mbox->payload_size; - else + if (out_size == CXL_VARIABLE_PAYLOAD) { + /* Adding extra 8 bytes for FWCTL, should not impact operation */ + mbox_cmd->size_out = cxl_mbox->payload_size + + sizeof(struct fwctl_rpc_cxl_out); + } else { mbox_cmd->size_out = out_size; + } if (mbox_cmd->size_out) { mbox_cmd->payload_out = kvzalloc(mbox_cmd->size_out, GFP_KERNEL); @@ -487,6 +523,73 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, return 0; } +static int cxl_fwctl_to_mem_cmd(struct cxl_mem_command *mem_cmd, + const struct cxl_send_command *send_cmd, + struct cxl_mailbox *cxl_mbox) +{ + struct cxl_mem_command *c = &cxl_mem_commands[send_cmd->id]; + const struct cxl_command_info *info = &c->info; + + if (send_cmd->flags & ~CXL_MEM_COMMAND_FLAG_MASK) + return -EINVAL; + + if (send_cmd->rsvd) + return -EINVAL; + + if (send_cmd->in.rsvd || send_cmd->out.rsvd) + return -EINVAL; + + /* Check the input buffer is the expected size */ + if (info->size_in != CXL_VARIABLE_PAYLOAD && + info->size_in != send_cmd->in.size) + return -ENOMEM; + + /* Check the output buffer is at least large enough */ + if (info->size_out != CXL_VARIABLE_PAYLOAD && + send_cmd->out.size < info->size_out) + return -ENOMEM; + + *mem_cmd = (struct cxl_mem_command) { + .info = { + .id = info->id, + .flags = info->flags, + .size_in = send_cmd->in.size, + .size_out = send_cmd->out.size, + }, + .opcode = c->opcode + }; + + return 0; +} + +static int verify_send_command(const struct cxl_send_command *send_cmd, + struct cxl_mailbox *cxl_mbox) +{ + if (send_cmd->id == 0 || send_cmd->id >= CXL_MEM_COMMAND_ID_MAX) + return -ENOTTY; + + /* + * The user can never specify an input payload larger than what hardware + * supports, but output can be arbitrarily large (simply write out as + * much data as the hardware provides). + */ + if (send_cmd->in.size > cxl_mbox->payload_size) + return -EINVAL; + + return 0; +} + +/* Sanitize and construct a cxl_mbox_cmd */ +static int construct_mbox_cmd(struct cxl_mbox_cmd *mbox_cmd, + struct cxl_mem_command *mem_cmd, + struct cxl_mailbox *cxl_mbox, + const struct cxl_send_command *send_cmd) +{ + return cxl_mbox_cmd_ctor(mbox_cmd, cxl_mbox, mem_cmd->opcode, + mem_cmd->info.size_in, mem_cmd->info.size_out, + send_cmd->in.payload); +} + /** * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND. * @mbox_cmd: Sanitized and populated &struct cxl_mbox_cmd. @@ -511,16 +614,9 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, struct cxl_mem_command mem_cmd; int rc; - if (send_cmd->id == 0 || send_cmd->id >= CXL_MEM_COMMAND_ID_MAX) - return -ENOTTY; - - /* - * The user can never specify an input payload larger than what hardware - * supports, but output can be arbitrarily large (simply write out as - * much data as the hardware provides). - */ - if (send_cmd->in.size > cxl_mbox->payload_size) - return -EINVAL; + rc = verify_send_command(send_cmd, cxl_mbox); + if (rc) + return rc; /* Sanitize and construct a cxl_mem_command */ if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW) @@ -531,10 +627,26 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, if (rc) return rc; - /* Sanitize and construct a cxl_mbox_cmd */ - return cxl_mbox_cmd_ctor(mbox_cmd, cxl_mbox, mem_cmd.opcode, - mem_cmd.info.size_in, mem_cmd.info.size_out, - send_cmd->in.payload); + return construct_mbox_cmd(mbox_cmd, &mem_cmd, cxl_mbox, send_cmd); +} + +static int cxl_validate_cmd_from_fwctl(struct cxl_mbox_cmd *mbox_cmd, + struct cxl_mailbox *cxl_mbox, + const struct cxl_send_command *send_cmd) +{ + struct cxl_mem_command mem_cmd; + int rc; + + rc = verify_send_command(send_cmd, cxl_mbox); + if (rc) + return rc; + + /* Sanitize and construct a cxl_mem_command */ + rc = cxl_fwctl_to_mem_cmd(&mem_cmd, send_cmd, cxl_mbox); + if (rc) + return rc; + + return construct_mbox_cmd(mbox_cmd, &mem_cmd, cxl_mbox, send_cmd); } static struct cxl_mem_command * @@ -719,7 +831,112 @@ static int handle_mailbox_cmd_from_user(struct cxl_mailbox *cxl_mbox, return rc; } -int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s) +static int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command *send, + struct cxl_mbox_cmd *mbox_cmd) +{ + int rc; + + rc = cxl_validate_cmd_from_user(mbox_cmd, cxl_mbox, send); + if (rc) + return rc; + + rc = handle_mailbox_cmd_from_user(cxl_mbox, mbox_cmd, send->out.payload, + &send->out.size, &send->retval); + if (rc) + return rc; + + return 0; +} + +/** + * handle_mailbox_cmd_from_fwctl() - Dispatch a mailbox command for userspace. + * @cxl_mbox: The mailbox context for the operation. + * @mbox_cmd: The validated mailbox command. + * + * Return: + * * %0 - Mailbox transaction succeeded. This implies the mailbox + * protocol completed successfully not that the operation itself + * was successful. + * * %-ENOMEM - Couldn't allocate a bounce buffer. + * * %-EINTR - Mailbox acquisition interrupted. + * * %-EXXX - Transaction level failures. + * + * Dispatches a mailbox command on behalf of a userspace request. + * The output payload is copied to userspace by fwctl. + * + * See cxl_send_cmd(). + */ +static int handle_mailbox_cmd_from_fwctl(struct cxl_mailbox *cxl_mbox, + struct cxl_mbox_cmd *mbox_cmd) +{ + struct device *dev = cxl_mbox->host; + struct fwctl_rpc_cxl_out *orig_out; + int rc; + + /* + * Save the payload_out pointer and move it to where hardware output + * can be copied to. + */ + orig_out = mbox_cmd->payload_out; + mbox_cmd->payload_out = (void *)orig_out + sizeof(*orig_out); + + dev_dbg(dev, + "Submitting %s command for user\n" + "\topcode: %x\n" + "\tsize: %zx\n", + cxl_mem_opcode_to_name(mbox_cmd->opcode), + mbox_cmd->opcode, mbox_cmd->size_in); + + rc = cxl_mbox->mbox_send(cxl_mbox, mbox_cmd); + if (rc) + return rc; + + orig_out->retval = mbox_cmd->return_code; + mbox_cmd->payload_out = (void *)orig_out; + + return 0; +} + +int cxl_mbox_send_cmd(struct cxl_mailbox *cxl_mbox, + struct fwctl_rpc_cxl *rpc_in, + struct cxl_mbox_cmd *mbox_cmd, size_t *out_len) +{ + struct cxl_send_command send_cmd = { + .id = rpc_in->id, + .flags = rpc_in->flags, + .in.size = rpc_in->op_size, + .in.payload = rpc_in->in_payload, + .out.size = *out_len, + }; + struct cxl_mem_command *cmd; + int rc; + + cmd = cxl_mem_find_command_by_id(rpc_in->id); + if (!cmd) + return -EINVAL; + send_cmd.raw.opcode = cmd->opcode; + + rc = cxl_validate_cmd_from_fwctl(mbox_cmd, cxl_mbox, &send_cmd); + if (rc) + return rc; + + rc = handle_mailbox_cmd_from_fwctl(cxl_mbox, mbox_cmd); + if (rc) + return rc; + + guard(rwsem_read)(&cxl_memdev_rwsem); + rc = cxl_mbox->mbox_send(cxl_mbox, mbox_cmd); + if (rc) + return rc; + + *out_len = mbox_cmd->size_out; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_mbox_send_cmd, CXL); + +int cxl_send_cmd_from_user(struct cxl_mailbox *cxl_mbox, + struct cxl_send_command __user *s) { struct device *dev = cxl_mbox->host; struct cxl_send_command send; @@ -731,12 +948,7 @@ int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s if (copy_from_user(&send, s, sizeof(send))) return -EFAULT; - rc = cxl_validate_cmd_from_user(&mbox_cmd, cxl_mbox, &send); - if (rc) - return rc; - - rc = handle_mailbox_cmd_from_user(cxl_mbox, &mbox_cmd, send.out.payload, - &send.out.size, &send.retval); + rc = cxl_send_cmd(cxl_mbox, &send, &mbox_cmd); if (rc) return rc; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 4d544a55ac3e..7fa16930cd85 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -11,7 +11,7 @@ #include "trace.h" #include "core.h" -static DECLARE_RWSEM(cxl_memdev_rwsem); +DECLARE_RWSEM(cxl_memdev_rwsem); /* * An entire PCI topology full of devices should be enough for any @@ -667,7 +667,7 @@ static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, case CXL_MEM_QUERY_COMMANDS: return cxl_query_cmd(cxl_mbox, (void __user *)arg); case CXL_MEM_SEND_COMMAND: - return cxl_send_cmd(cxl_mbox, (void __user *)arg); + return cxl_send_cmd_from_user(cxl_mbox, (void __user *)arg); default: return -ENOTTY; } diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c index ce8960a9beaa..164f6774a2c1 100644 --- a/drivers/fwctl/cxl/cxl.c +++ b/drivers/fwctl/cxl/cxl.c @@ -77,11 +77,106 @@ static void *cxlctl_hw_info(struct fwctl_uctx *uctx, int commands, size_t *out_l return_ptr(out); } +static bool cxlctl_validate_set_features(struct cxl_mailbox *cxl_mbox, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + struct cxl_feat_entry *feat; + bool found = false; + uuid_t uuid; + u16 effects, mask; + + if (rpc_in->op_size < sizeof(struct cxl_set_feature_input)) + return false; + + if (copy_from_user(&uuid, u64_to_user_ptr(rpc_in->in_payload), + sizeof(uuid))) + return false; + + for (int i = 0; i < cxl_mbox->num_features; i++) { + feat = &cxl_mbox->entries[i]; + if (uuid_equal(&uuid, &feat->uuid)) { + found = true; + break; + } + } + + if (!found) + return false; + + effects = le16_to_cpu(feat->effects); + /* Currently no user background command support */ + if (effects & CXL_CMD_BACKGROUND) + return false; + + mask = CXL_CMD_CONFIG_CHANGE_IMMEDIATE | + CXL_CMD_DATA_CHANGE_IMMEDIATE | + CXL_CMD_POLICY_CHANGE_IMMEDIATE | + CXL_CMD_LOG_CHANGE_IMMEDIATE; + if (effects & mask && scope >= FWCTL_RPC_DEBUG_WRITE_FULL) + return true; + + /* These effects supported for all scope */ + if ((effects & CXL_CMD_CONFIG_CHANGE_COLD_RESET || + effects & CXL_CMD_CONFIG_CHANGE_CONV_RESET) && + scope >= FWCTL_RPC_DEBUG_WRITE) + return true; + + return false; +} + +static bool cxlctl_validate_hw_cmds(struct cxl_mailbox *cxl_mbox, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + struct cxl_mem_command *cmd; + + /* + * Only supporting feature commands for now. + */ + if (!cxl_mbox->num_features) + return false; + + cmd = cxl_get_mem_command_for_fwctl(cxl_mbox, rpc_in->id); + if (!cmd) + return false; + + switch (cmd->opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + if (scope >= FWCTL_RPC_CONFIGURATION) + return true; + return false; + case CXL_MBOX_OP_GET_FEATURE: + if (scope >= FWCTL_RPC_DEBUG_READ_ONLY) + return true; + return false; + case CXL_MBOX_OP_SET_FEATURE: + return cxlctl_validate_set_features(cxl_mbox, rpc_in, scope); + default: + return false; + } +} + static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, - void *rpc_in, size_t in_len, size_t *out_len) + void *in, size_t in_len, size_t *out_len) { - /* Place holder */ - return ERR_PTR(-EOPNOTSUPP); + struct cxlctl_dev *cxlctl = + container_of(uctx->fwctl, struct cxlctl_dev, fwctl); + struct cxl_mailbox *cxl_mbox = cxlctl->mbox; + struct fwctl_rpc_cxl *rpc_in = in; + struct cxl_mbox_cmd mbox_cmd; + int rc; + + if (!cxlctl_validate_hw_cmds(cxlctl->mbox, rpc_in, scope)) + return ERR_PTR(-EPERM); + + rc = cxl_mbox_send_cmd(cxl_mbox, rpc_in, &mbox_cmd, out_len); + if (rc) + return ERR_PTR(rc); + + *out_len = mbox_cmd.size_out; + + return mbox_cmd.payload_out; } static const struct fwctl_ops cxlctl_ops = { diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index e753d5d1d708..3e5e9c9362f5 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -193,5 +193,11 @@ int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host); int cxl_mailbox_user_commands_supported(struct cxl_mailbox *cxl_mbox); int cxl_mailbox_user_commands_info_get(struct cxl_mailbox *cxl_mbox, int nr_cmds, void *outbuf, size_t *out_len); +struct cxl_mem_command *cxl_get_mem_command(u32 id); +struct cxl_mem_command * +cxl_get_mem_command_for_fwctl(struct cxl_mailbox *cxl_mbox, u32 id); +int cxl_mbox_send_cmd(struct cxl_mailbox *cxl_mbox, + struct fwctl_rpc_cxl *rpc_in, + struct cxl_mbox_cmd *mbox_cmd, size_t *out_len); #endif diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h index a32c4c752db6..99804fc72f28 100644 --- a/include/uapi/fwctl/cxl.h +++ b/include/uapi/fwctl/cxl.h @@ -19,4 +19,33 @@ struct fwctl_info_cxl { __u32 nr_commands; }; +/** + * struct fwctl_rpc_cxl - ioctl(FWCTL_RPC) input for CXL + * @id: The command id to send to the memory device. This must be one of the + * commands returned by the query command. + * @flags: Flags for the command (input). + * @op_size: Size of hw operation + * @reserved: Reserved. Must be 0s. + * @in_payload: User address of the hardware op input + */ +struct fwctl_rpc_cxl { + __u32 id; + __u32 flags; + __u32 op_size; + __u32 reserved; + __aligned_u64 in_payload; +}; + +/** + * struct fwctl_rpc_cxl_out - ioctl9FWCTL_RPC) output for CXL + * @size: Size of the output payload + * @retval: Return value from device + * @payload: Return data from device + */ +struct fwctl_rpc_cxl_out { + __u32 size; + __u32 retval; + __u8 payload[]; +}; + #endif diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h index 9dd37849c450..4e7c8c03cfe8 100644 --- a/include/uapi/linux/cxl_mem.h +++ b/include/uapi/linux/cxl_mem.h @@ -234,4 +234,16 @@ struct cxl_send_command { } out; }; +/* + * CXL spec r3.1 Table 8-101 Set Feature Input Payload + */ +struct cxl_set_feature_input { + __u8 uuid[16]; + __u32 flags; + __u16 offset; + __u8 version; + __u8 reserved[9]; + __u8 data[]; +} __packed; + #endif From patchwork Fri Nov 15 21:25:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877299 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CEE81FB733 for ; Fri, 15 Nov 2024 21:28:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706091; cv=none; b=tR4nLKYAgQU8EWl264Gk+Na0k6ytTDF4dmeORQXEVq/M0tFyYzgDwQVOifsGknOOGT3l+0OUX1QXLK4cCnNsdtsCGaon9ZX9cXZ9TW+xtO9bAvr0rum++maRJhJCjsMb2kV5HD6P/m1mSHbmWYdtt72dOKIddIi18zB/oRctE10= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706091; c=relaxed/simple; bh=0pUxAN0rBFPf5K0jx1ZADdSpz4O3Cax0rtRrjZ05JV4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hKszRttAmDUUWeIm0Us188GeA/69LSo40uHbI73rlIGjVH4G0hLWKpElVxLwqTo2J5qCWJs0SdzwpRGFiwk/nbrwiFyr0itIXyVXHGEs9byVoOUvc1HjzZkWZ9EwnLqv6znSonf2Sgmm1Ams6cXydgRqeRlra5Rv7TGvtN8iacA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id BD550C4CECF; Fri, 15 Nov 2024 21:28:10 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 16/20] fwctl/cxl: Add support to filter exclusive features Date: Fri, 15 Nov 2024 14:25:49 -0700 Message-ID: <20241115212745.869552-17-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add support to allow filtering of CXL features that are exclusive to the kernel and make them not visible to user space. The "get supported features" mailbox command returned to the userspace is emulated and will skip the features that is marked exclusive for the kernel. The exclusion allows certain the feature setting of certain commands such as claimed by RAS to be exclusive to the kernel. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 115 ++++++++++++++++++++++++++++++++++++---- drivers/fwctl/cxl/cxl.c | 7 +-- include/cxl/features.h | 52 ++++++++++++++++++ include/cxl/mailbox.h | 8 ++- 4 files changed, 168 insertions(+), 14 deletions(-) create mode 100644 include/cxl/features.h diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index e7c5c709ac79..12ace2951f7c 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -897,6 +898,78 @@ static int handle_mailbox_cmd_from_fwctl(struct cxl_mailbox *cxl_mbox, return 0; } +/* + * Emulate the 'get supported features' mailbox command and only copy out the + * features that are not marked as exclusive for kernel. + */ +static void cxl_mbox_get_supported_features_filtered(struct cxl_mailbox *cxl_mbox, + struct cxl_mbox_cmd *mbox_cmd) +{ + struct cxl_mbox_get_sup_feats_in *feat_in = mbox_cmd->payload_in; + struct cxl_mbox_get_sup_feats_out *feat_out = mbox_cmd->payload_out; + const int feat_out_size = sizeof(*feat_out); + struct cxl_feat_entry *pos; + const int feat_ent_size = sizeof(*pos); + int out_count, ents, u; + u32 count; + u16 start; + + count = le32_to_cpu(feat_in->count); + start = le16_to_cpu(feat_in->start_idx); + ents = count / sizeof(struct cxl_feat_entry); + ents -= start; + if (ents < 0) { + mbox_cmd->size_out = 0; + mbox_cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return; + } + + if (mbox_cmd->size_out < feat_out_size) { + mbox_cmd->size_out = 0; + mbox_cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return; + } + + feat_out->supported_feats = + cpu_to_le16(cxl_mbox->num_user_feats); + if (ents == 0) { + feat_out->num_entries = cpu_to_le16(0); + mbox_cmd->size_out = feat_out_size; + mbox_cmd->return_code = CXL_MBOX_CMD_RC_SUCCESS; + return; + } + + pos = &feat_out->ents[0]; + out_count = feat_out_size; + for (int i = 0, u = 0; i < cxl_mbox->num_features; i++) { + struct cxl_feature *feat = &cxl_mbox->entries[i]; + + if (feat->exclusive) + continue; + + if (u < start) { + u++; + continue; + } + + memcpy(pos, &feat->entry, feat_ent_size); + out_count += feat_ent_size; + pos++; + u++; + + if (u == count) + break; + + /* Make sure it does not go over total output buffer size */ + if (out_count + feat_ent_size >= mbox_cmd->size_out) + break; + } + + feat_out->num_entries = cpu_to_le16(u); + mbox_cmd->size_out = out_count; + mbox_cmd->return_code = CXL_MBOX_CMD_RC_SUCCESS; +} + int cxl_mbox_send_cmd(struct cxl_mailbox *cxl_mbox, struct fwctl_rpc_cxl *rpc_in, struct cxl_mbox_cmd *mbox_cmd, size_t *out_len) @@ -925,9 +998,13 @@ int cxl_mbox_send_cmd(struct cxl_mailbox *cxl_mbox, return rc; guard(rwsem_read)(&cxl_memdev_rwsem); - rc = cxl_mbox->mbox_send(cxl_mbox, mbox_cmd); - if (rc) - return rc; + if (send_cmd.raw.opcode != CXL_MBOX_OP_GET_SUPPORTED_FEATURES) { + rc = cxl_mbox->mbox_send(cxl_mbox, mbox_cmd); + if (rc) + return rc; + } else { + cxl_mbox_get_supported_features_filtered(cxl_mbox, mbox_cmd); + } *out_len = mbox_cmd->size_out; @@ -1128,6 +1205,18 @@ static int cxl_get_supported_features_count(struct cxl_dev_state *cxlds) return 0; } +static bool is_feature_exclusive(struct cxl_feature *feat) +{ + int i; + + for (i = 0; i < CXL_FEAT_UUID_MAX; i++) { + if (uuid_equal(&feat->entry.uuid, &cxl_exclusive_feats[i])) + return true; + } + + return false; +} + int cxl_get_supported_features(struct cxl_dev_state *cxlds) { struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) = NULL; @@ -1138,7 +1227,7 @@ int cxl_get_supported_features(struct cxl_dev_state *cxlds) int hdr_size = sizeof(*mbox_out); struct cxl_mbox_cmd mbox_cmd; struct cxl_mem_command *cmd; - struct cxl_feat_entry *entry; + struct cxl_feature *feat; /* Get supported features is optional, need to check */ cmd = cxl_mem_find_command(CXL_MBOX_OP_GET_SUPPORTED_FEATURES); @@ -1156,7 +1245,7 @@ int cxl_get_supported_features(struct cxl_dev_state *cxlds) return 0; } - struct cxl_feat_entry *entries __free(kvfree) = + struct cxl_feature *entries __free(kvfree) = kvmalloc(cxl_mbox->num_features * feat_size, GFP_KERNEL); if (!entries) @@ -1165,7 +1254,7 @@ int cxl_get_supported_features(struct cxl_dev_state *cxlds) max_size = cxl_mbox->payload_size - hdr_size; /* max feat entries that can fit in mailbox max payload size */ max_feats = max_size / feat_size; - entry = &entries[0]; + feat = &entries[0]; mbox_out = kvmalloc(cxl_mbox->payload_size, GFP_KERNEL); if (!mbox_out) @@ -1228,8 +1317,13 @@ int cxl_get_supported_features(struct cxl_dev_state *cxlds) goto err; } - memcpy(entry, mbox_out->ents, retrieved); - entry++; + memcpy(&feat->entry, mbox_out->ents, retrieved); + if (is_feature_exclusive(feat)) + feat->exclusive = true; + else + cxl_mbox->num_user_feats++; + feat++; + /* * If the number of output entries is less than expected, add the * remaining entries to the next batch. @@ -1256,13 +1350,14 @@ int cxl_get_supported_feature_entry(struct cxl_dev_state *cxlds, const uuid_t *f struct cxl_feat_entry *feat_entry_out) { struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; - struct cxl_feat_entry *feat_entry; + struct cxl_feature *feat_entry; int count; /* Check CXL dev supports the feature */ feat_entry = &cxl_mbox->entries[0]; for (count = 0; count < cxl_mbox->num_features; count++, feat_entry++) { - if (uuid_equal(&feat_entry->uuid, feat_uuid)) { + if (uuid_equal(&feat_entry->entry.uuid, feat_uuid) && + !is_feature_exclusive(feat_entry)) { memcpy(feat_entry_out, feat_entry, sizeof(*feat_entry_out)); return 0; } diff --git a/drivers/fwctl/cxl/cxl.c b/drivers/fwctl/cxl/cxl.c index 164f6774a2c1..3915a59c7408 100644 --- a/drivers/fwctl/cxl/cxl.c +++ b/drivers/fwctl/cxl/cxl.c @@ -81,7 +81,7 @@ static bool cxlctl_validate_set_features(struct cxl_mailbox *cxl_mbox, const struct fwctl_rpc_cxl *rpc_in, enum fwctl_rpc_scope scope) { - struct cxl_feat_entry *feat; + struct cxl_feature *feat; bool found = false; uuid_t uuid; u16 effects, mask; @@ -95,7 +95,8 @@ static bool cxlctl_validate_set_features(struct cxl_mailbox *cxl_mbox, for (int i = 0; i < cxl_mbox->num_features; i++) { feat = &cxl_mbox->entries[i]; - if (uuid_equal(&uuid, &feat->uuid)) { + if (uuid_equal(&uuid, &feat->entry.uuid) && + !feat->exclusive) { found = true; break; } @@ -104,7 +105,7 @@ static bool cxlctl_validate_set_features(struct cxl_mailbox *cxl_mbox, if (!found) return false; - effects = le16_to_cpu(feat->effects); + effects = le16_to_cpu(feat->entry.effects); /* Currently no user background command support */ if (effects & CXL_CMD_BACKGROUND) return false; diff --git a/include/cxl/features.h b/include/cxl/features.h new file mode 100644 index 000000000000..6074a3f79d70 --- /dev/null +++ b/include/cxl/features.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2024 Intel Corporation. */ +#ifndef __CXL_FEATS_H_ +#define __CXL_FEATS_H_ + +#include + +#define CXL_FEAT_PATROL_SCRUB_UUID \ + UUID_INIT(0x96dad7d6, 0xfde8, 0x482b, 0xa7, 0x33, 0x75, 0x77, 0x4e, \ + 0x06, 0xdb, 0x8a) + +#define CXL_FEAT_ECS_UUID \ + UUID_INIT(0xe5b13f22, 0x2328, 0x4a14, 0xb8, 0xba, 0xb9, 0x69, 0x1e, \ + 0x89, 0x33, 0x86) + +#define CXL_FEAT_SPPR_UUID \ + UUID_INIT(0x892ba475, 0xfad8, 0x474e, 0x9d, 0x3e, 0x69, 0x2c, 0x91, \ + 0x75, 0x68, 0xbb) + +#define CXL_FEAT_HPPR_UUID \ + UUID_INIT(0x80ea4521, 0x786f, 0x4127, 0xaf, 0xb1, 0xec, 0x74, 0x59, \ + 0xfb, 0x0e, 0x24) + +#define CXL_FEAT_CACHELINE_SPARING_UUID \ + UUID_INIT(0x96C33386, 0x91dd, 0x44c7, 0x9e, 0xcb, 0xfd, 0xaf, 0x65, \ + 0x03, 0xba, 0xc4) + +#define CXL_FEAT_ROW_SPARING_UUID \ + UUID_INIT(0x450ebf67, 0xb135, 0x4f97, 0xa4, 0x98, 0xc2, 0xd5, 0x7f, \ + 0x27, 0x9b, 0xed) + +#define CXL_FEAT_BANK_SPARING_UUID \ + UUID_INIT(0x78b79636, 0x90ac, 0x4b64, 0xa4, 0xef, 0xfa, 0xac, 0x5d, \ + 0x18, 0xa8, 0x63) + +#define CXL_FEAT_RANK_SPARING_UUID \ + UUID_INIT(0x34dbaff5, 0x0552, 0x4281, 0x8f, 0x76, 0xda, 0x0b, 0x5e, \ + 0x7a, 0x76, 0xa7) + +#define CXL_FEAT_UUID_MAX 8 +static uuid_t cxl_exclusive_feats[CXL_FEAT_UUID_MAX] = { + CXL_FEAT_PATROL_SCRUB_UUID, + CXL_FEAT_ECS_UUID, + CXL_FEAT_SPPR_UUID, + CXL_FEAT_HPPR_UUID, + CXL_FEAT_CACHELINE_SPARING_UUID, + CXL_FEAT_ROW_SPARING_UUID, + CXL_FEAT_BANK_SPARING_UUID, + CXL_FEAT_RANK_SPARING_UUID, +}; + +#endif diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 3e5e9c9362f5..f44ac93ebdf4 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -80,7 +80,8 @@ struct cxl_mailbox { struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; int num_features; - struct cxl_feat_entry *entries; + int num_user_feats; + struct cxl_feature *entries; int (*mbox_send)(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); }; @@ -165,6 +166,11 @@ struct cxl_feat_entry { u8 reserved[18]; } __packed; +struct cxl_feature { + bool exclusive; + struct cxl_feat_entry entry; +}; + /** * struct cxl_mem_command - Driver representation of a memory device command * @info: Command information as it exists for the UAPI From patchwork Fri Nov 15 21:25:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877300 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D691FB733 for ; Fri, 15 Nov 2024 21:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706092; cv=none; b=pM2qIwI8ac4x3UYedaDIkIm6BWLMjktD+PkH7Rqq0NOMCHQIEf9W7lXRaeMmmpkYAe0DT81ZP3gI55c9aPxQSExgLEsm96iBRz0F222BRB0R7/zrO3LWHruPIA8um1hxQY0/pX+xLAvKKeir9dPp6RCV41KpF7EpsDowMy2LlGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706092; c=relaxed/simple; bh=Pg3M7VwAvmfMzigp9SmXmMFd+cztJ/wWo94uproU6Ig=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dzeXPuKk2/K+1hak9IcVq8NeYqbfAet2OU6o9JC+laefgImlrRXJTwvypSWW2SXRZyuSz/JuBOV6IIJTlcplvt7UICRs2iusEXC6acc9Wn/6nwvzu6BuOEvf+EVAYB8QGkr8ghndzOyr/2CYX63iBcKWsUsRoW51sbPHu01Z5sA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15C7DC4CECF; Fri, 15 Nov 2024 21:28:12 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 17/20] cxl/test: Add Get Feature support to cxl_test Date: Fri, 15 Nov 2024 14:25:50 -0700 Message-ID: <20241115212745.869552-18-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation of Get Feature command to cxl_test. The feature for device patrol scrub is returned by the emulation code. This is the only feature currently supported by cxl_test. It returns the information for the device patrol scrub feature. Signed-off-by: Dave Jiang --- tools/testing/cxl/test/mem.c | 63 ++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 9103bac054b9..547d7289d8df 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -48,6 +48,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_FEATURES), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), + .effect = CXL_CMD_EFFECT_NONE, + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -149,6 +153,10 @@ struct mock_event_store { u32 ev_status; }; +struct vendor_test_feat { + __le32 data; +} __packed; + struct cxl_mockmem_data { void *lsa; void *fw; @@ -165,6 +173,7 @@ struct cxl_mockmem_data { u8 event_buf[SZ_4K]; u64 timestamp; unsigned long sanitize_timeout; + struct vendor_test_feat test_feat; }; static struct mock_event_log *event_find_log(struct device *dev, int log_type) @@ -1357,6 +1366,51 @@ static void fill_feature_vendor_test(struct cxl_feat_entry *feat) feat->effects = cpu_to_le16(BIT(0) | BIT(9)); } +struct get_feat_input { + uuid_t uuid; + __le16 offset; + __le16 count; + u8 selection; +} __packed; + +static int mock_get_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct vendor_test_feat *output = cmd->payload_out; + struct get_feat_input *input = cmd->payload_in; + u16 offset = le16_to_cpu(input->offset); + u16 count = le16_to_cpu(input->count); + u8 *ptr; + + if (offset > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + if (offset + count > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + ptr = (u8 *)&mdata->test_feat + offset; + memcpy((u8 *)output + offset, ptr, count); + + return 0; +} + +static int mock_get_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct get_feat_input *input = cmd->payload_in; + + if (uuid_equal(&input->uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_get_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1487,6 +1541,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: rc = mock_get_supported_features(mdata, cmd); break; + case CXL_MBOX_OP_GET_FEATURE: + rc = mock_get_feature(mdata, cmd); + break; default: break; } @@ -1534,6 +1591,11 @@ static int cxl_mock_mailbox_create(struct cxl_dev_state *cxlds) return 0; } +static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) +{ + mdata->test_feat.data = cpu_to_le32(0xdeadbeef); +} + static int cxl_mock_mem_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1625,6 +1687,7 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) return rc; cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); + cxl_mock_test_feat_init(mdata); return 0; } From patchwork Fri Nov 15 21:25:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877301 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EDD21FB733 for ; Fri, 15 Nov 2024 21:28:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706094; cv=none; b=EBYrHUpr1huUFhCrnY8nYuWpKEgHiHvm8r29A3/Q65Tf0anc71GX0XHu6jYL9aT/O7v/JuKrkajyGPZ0x442TvrE+S6q+g1eYTqjErnTE8j3yNLKAivhvrxCYj+mUgRy1RUV6HVhoxTMdOSrEqwDxX0IA9Ktmyp11nCQVTj2TNk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706094; c=relaxed/simple; bh=eYr2Y57/DWFFYj9Nk2Jxd8yT40zqiBcaB4xQB6CYOlY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tdu9QCytOz7BU7tCqODRWjBBsvy2tj7Y1men1Eu063EJ58/5bYRuQEie3gv1LCwad6AqnqUVtBp3Ejjk/jRNSNigzz4deunR4ljnb2+1VZnddF9VsvhGVlT4iE2CtRepQTvjy7Mu1hpWUWZJNj65+PiKlPb7BJSHFVvfAzW9+Vk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6E796C4CECF; Fri, 15 Nov 2024 21:28:13 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 18/20] cxl/test: Add Set Feature support to cxl_test Date: Fri, 15 Nov 2024 14:25:51 -0700 Message-ID: <20241115212745.869552-19-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation to support Set Feature mailbox command to cxl_test. The only feature supported is the device patrol scrub feature. The set feature allows activation of patrol scrub for the cxl_test emulated device. The command does not support partial data transfer even though the spec allows it. This restriction is to reduce complexity of the emulation given the patrol scrub feature is very minimal. Signed-off-by: Dave Jiang --- tools/testing/cxl/test/mem.c | 69 ++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 547d7289d8df..25f07843cb84 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -52,6 +52,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_SET_FEATURE), + .effect = cpu_to_le16(EFFECT(CONF_CHANGE_IMMEDIATE)), + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -1411,6 +1415,68 @@ static int mock_get_feature(struct cxl_mockmem_data *mdata, return -EOPNOTSUPP; } +struct set_feat_input { + uuid_t uuid; + __le32 flags; + __le16 offset; + u8 version; + u8 rsvd[9]; + u8 data[]; +} __packed; + +enum set_feature_flags { + SET_FEATURE_F_FULL_XFER = 0, + SET_FEATURE_F_INIT_XFER, + SET_FEATURE_F_CONT_XFER, + SET_FEATURE_F_FIN_XFER, + SET_FEATURE_F_ABORT_XFER, +}; + +#define SET_FEATURE_F_ACTION_MASK GENMASK(2, 0) + +static int mock_set_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct set_feat_input *input = cmd->payload_in; + struct vendor_test_feat *test = (struct vendor_test_feat *)input->data; + u32 action; + + action = FIELD_GET(SET_FEATURE_F_ACTION_MASK, + le32_to_cpu(input->flags)); + /* + * While it is spec compliant to support other set actions, it is not + * necessary to add the complication in the emulation currently. Reject + * anything besides full xfer. + */ + if (action != SET_FEATURE_F_FULL_XFER) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + /* Offset should be reserved when doing full transfer */ + if (input->offset) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + memcpy(&mdata->test_feat.data, &test->data, sizeof(u32)); + + return 0; +} + +static int mock_set_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct set_feat_input *input = cmd->payload_in; + + if (uuid_equal(&input->uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_set_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1544,6 +1610,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_FEATURE: rc = mock_get_feature(mdata, cmd); break; + case CXL_MBOX_OP_SET_FEATURE: + rc = mock_set_feature(mdata, cmd); + break; default: break; } From patchwork Fri Nov 15 21:25:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877302 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43D651FB73C for ; Fri, 15 Nov 2024 21:28:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706095; cv=none; b=Aa0zNcLi6KDaCalP/S9YwqtCBwCdL01wAcWcOJQykbtGXYbi8kDZRHfzyecGaR6PX0/ws+Debjwd3q/T7LqkKrcCUAhvv7qj8cncJW/E+kW1p/PPgq6UeMMbiW/w0nus/XKHNzivZ84aeruKRZr64K7oIAev+SEVhEYGBMp/F2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706095; c=relaxed/simple; bh=7rrcwKrmyJTcwl3Z2QKDfPYNuyaoAU6cIiB5EiGEgFs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cCkQMfP4ZpsbwKsTqPyTTbz3J/wtkyMOyoKpDzIHvSAvUMPFcX+vgZ3cOGgH73vsEFl33JQ5vbVhqn5QBIQ+R6ssJ73GQeZFTl1HKWrwcLBD6GhGJiZ5w5MnPlG8ecUQFV6JgYTvDoraGh5si6DA1ph9rxjvzkd8/OzFTZyPAPw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7F89C4CECF; Fri, 15 Nov 2024 21:28:14 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 19/20] fwctl: Move fwctl documentation to its own directory Date: Fri, 15 Nov 2024 14:25:52 -0700 Message-ID: <20241115212745.869552-20-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Move fwctl.rst to its own directory in prepraration for adding more documentation. Signed-off-by: Dave Jiang --- Documentation/userspace-api/{ => fwctl}/fwctl.rst | 0 Documentation/userspace-api/fwctl/index.rst | 12 ++++++++++++ Documentation/userspace-api/index.rst | 2 +- MAINTAINERS | 2 +- 4 files changed, 14 insertions(+), 2 deletions(-) rename Documentation/userspace-api/{ => fwctl}/fwctl.rst (100%) create mode 100644 Documentation/userspace-api/fwctl/index.rst diff --git a/Documentation/userspace-api/fwctl.rst b/Documentation/userspace-api/fwctl/fwctl.rst similarity index 100% rename from Documentation/userspace-api/fwctl.rst rename to Documentation/userspace-api/fwctl/fwctl.rst diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst new file mode 100644 index 000000000000..06959fbf1547 --- /dev/null +++ b/Documentation/userspace-api/fwctl/index.rst @@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Firmware Control (FWCTL) Userspace API +====================================== + +A framework that define a common set of limited rules that allows user space +to securely construct and execute RPCs inside device firmware. + +.. toctree:: + :maxdepth: 1 + + fwctl diff --git a/Documentation/userspace-api/index.rst b/Documentation/userspace-api/index.rst index 2bc43a658074..c2f067b3de01 100644 --- a/Documentation/userspace-api/index.rst +++ b/Documentation/userspace-api/index.rst @@ -44,7 +44,7 @@ Devices and I/O accelerators/ocxl dma-buf-alloc-exchange - fwctl + fwctl/index gpio/index iommufd media/index diff --git a/MAINTAINERS b/MAINTAINERS index 6b52c4168b94..1699316f86ae 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9377,7 +9377,7 @@ FWCTL SUBSYSTEM M: Jason Gunthorpe M: Saeed Mahameed S: Maintained -F: Documentation/userspace-api/fwctl.rst +F: Documentation/userspace-api/fwctl/ F: drivers/fwctl/ F: include/linux/fwctl.h F: include/uapi/fwctl/ From patchwork Fri Nov 15 21:25:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13877303 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B650F1FB3D9 for ; Fri, 15 Nov 2024 21:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706097; cv=none; b=lYUPl87YIteDnrb0Pm7WzJyHOSMPnn8ne6hi2CaOmipkW8tOxBKbT+yUCkn5TXYJrHuUEEi6VnTvl2h+dJ1ZtXQ4XGQhw5XxQFmf84FFfAuAeXRJ5/TTuqElIeGk3rX5PpqVltn3qFdf2CbCeF7IufgWBmK+3eOdXsjp4qtK+O4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731706097; c=relaxed/simple; bh=+hwQ0imvuOGkz6mgYY+eQ7cNsyXYRKrIuOaqwQLcN6I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=a6ieCuSox37FMB/UfuUBWGte3SaL253HXewi8TjL2jrURfgrWGbeNf7xgyVPt0QcwbcQU9b5tBb7FidIf6X4GQnVI77onqW3lFcM2j3uL5T/d0iSi3MWf5K3eEd1ldBI97jV7RxPQspqx144XzmzemTv42BinH8TGiYDJOqFcII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27D5DC4CECF; Fri, 15 Nov 2024 21:28:16 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [RFC PATCH v2 20/20] fwctl/cxl: Add documentation to FWCTL CXL Date: Fri, 15 Nov 2024 14:25:53 -0700 Message-ID: <20241115212745.869552-21-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241115212745.869552-1-dave.jiang@intel.com> References: <20241115212745.869552-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add policy and operational documentation for FWCTL CXL. Signed-off-by: Dave Jiang --- .../userspace-api/fwctl/fwctl-cxl.rst | 91 +++++++++++++++++++ Documentation/userspace-api/fwctl/index.rst | 1 + 2 files changed, 92 insertions(+) create mode 100644 Documentation/userspace-api/fwctl/fwctl-cxl.rst diff --git a/Documentation/userspace-api/fwctl/fwctl-cxl.rst b/Documentation/userspace-api/fwctl/fwctl-cxl.rst new file mode 100644 index 000000000000..158e9aa45609 --- /dev/null +++ b/Documentation/userspace-api/fwctl/fwctl-cxl.rst @@ -0,0 +1,91 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================ +fwctl cxl driver +================ + +:Author: Dave Jiang + +Overview +======== + +The CXL spec defines a set of commands that can be issued to the mailbox of a +CXL device or switch. It also left room for vendor specific commands to be +issued to the mailbox as well. fwctl provides a path to issue a set of allowed +mailbox commands from user space to the device moderated by the kernel driver. + +While there are a large set of mailbox commands, only the feature related +commands will be allowed to be issued through fwctl initially. No background +commands will be supported at this time. + +CXL spec r3.1 8.2.9.6.1 Get Supported Features (Opcode 0500h) +CXL spec r3.1 8.2.9.6.2 Get Feature (Opcode 0501h) +CXL spec r3.1 8.2.9.6.3 Set Feature (Opcode 0502h) + +The "Get Supported Features" return data may be filtered by the kernel driver to +drop any features that are forbidden by the kernel or being exclusively used by +the kernel. The "Get Supported Featuers" command falls under the fwctl policy +of FWCTL_RPC_CONFIGURATION. + +The "Get Features" commands falls under the policy FWCTL_RPC_DEBUG_READ_ONLY. + +For "Set Feature" command, the access policy currently is broken down into two +categories depending on the set feature effects reported by the device. If the +set feature will cause immediate change to the device, the fwctl access policy +must be FWCTL_RPC_DEBUG_WRITE_FULL. The effects for this level are +"immediate config change", "immediate data change", "immediate policy change", +or "immediate log change" for the set effects mask. If the effects are "config +change with cold reset" or "config change with conventional reset", then the +fwctl access policy must be FWCTL_RPC_DEBUG_WRITE or higher. + +fwctl cxl User API +================== + +.. kernel-doc:: include/uapi/fwctl/cxl.h + +1. Driver info query +-------------------- + +First step for the app is to issue the ioctl(FWCTL_CMD_INFO) in order to retrieve +the information from the driver. A ``struct fwctl_info`` needs to be filled out +with the ``fwctl_info.out_device_type`` set to ``FWCTL_DEVICE_TYPE_CXL``. The +return data should be ``struct fwctl_info_cxl`` that contains the number of +commands supported by the driver. + +2. Hardware command info query +------------------------------ + +Second step is to retrieve the command information from the driver. A ``struct +fwctl_hw_info`` is sent with the ioctl(FWCTL_CMD_HW_INFO). +``fwctl_hw_info.nr_cmds`` should be set to the number retrieved from the driver +info query. A user buffer of ``struct fwctl_hw_info_out`` including an inlined +array of ``struct fwctl_hw_info_out`` numbered of nr_cmds should be allocated and +set as ``fwctl_hw_info.out_device_data``. For each command supported, its +relevant information is returned such as the driver command id, the opcode it +supports, the effect reported by the device if it's a write command, and its +expected input and output sizes. + +3. Send hardware command +------------------------ + +Last step is to send hardware command to the driver from user space via +ioctl(FWCTL_RPC). A ``struct fwctl_rpc_cxl`` should be pointed to by +``fwctl_rpc.in``. ``struct fwctl_rpc_cxl.in_payload`` should point to +the hardware input structure that is defined by the CXL spec. ``fwctl_rpc.out`` +points to the buffer that contains a ``struct fwctl_rpc_cxl_out`` that includes +the hardware output data inlined as ``fwctl_rpc_cxl_out.payload``. + +For a get/set feature command, it is recommended to run the "get supported +features" command first to retrieve the desired feature information first +before sending a get or set feature command. For a "set feature" command, +the retrieved info contains an effects field that details the resulting +"set feature" command will trigger. That should inform the user whether +the system is configured to allowed the "set feature" command or not. + + +fwctl cxl Kernel API +==================== + +.. kernel-doc:: drivers/fwctl/cxl/cxl.c + :export: +.. kernel-doc:: include/cxl/mailbox.h diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst index 06959fbf1547..d9d40a468a31 100644 --- a/Documentation/userspace-api/fwctl/index.rst +++ b/Documentation/userspace-api/fwctl/index.rst @@ -10,3 +10,4 @@ to securely construct and execute RPCs inside device firmware. :maxdepth: 1 fwctl + fwctl-cxl