| Message ID | 20250113064345.6835-1-huaisheng.ye@intel.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v3] cxl/core/regs: Refactor out functions to count regblocks of given type | expand |
On Mon, 13 Jan 2025 14:43:45 +0800 Huaisheng Ye <huaisheng.ye@intel.com> wrote: > In commit d717d7f3df18494baafd9595fb4bcb9c380d7389, cxl_count_regblock was > added for counting regblocks of a given RBI (Register Block Identifier). > It is workable, but the implementation can be improved. > > 1. In order to get the count of instances, cxl_count_regblock has to tentatively > repeat the call of cxl_find_regblock_instance by increasing index from 0. It > will not stop until an error value is returned. Actually, It needs to search for > Register Blocks in dvsec again every time by taking a start from the head. The > operations can be optimized. > > For example, to determine if PMU1 exists, cxl_find_regblock_instance must check > all Register Blocks by the type and index from RB1 to RB4, starting from scratch, > even if PMU0 just has been searched. If there are more RBs of the same type in the > future, the situation will be even worse. > > 16 81 00 23 PCIe extended Capability Header > 02 c0 1e 98 Header 1 > 00 00 00 08 Header 2 > -------------------------------------- > 00 00 01 00 RB 1 - Offset Low Component > 00 00 00 00 RB 1 - Offset High > -------------------------------------- > 00 00 03 02 RB 2 - Offset Low Device Register > 00 00 00 00 RB 2 - Offset High > -------------------------------------- > 00 01 04 02 RB 3 - Offset Low PMU0 > 00 00 00 00 RB 3 - Offset High > -------------------------------------- > 00 02 04 02 RB 4 - Offset Low PMU1 > 00 00 00 00 RB 4 - Offset High > > RB: Register Block > > 2. cxl_count_regblock blocks the opportunity to get error codes from > cxl_find_regblock_instance. cxl_pci_probe has error code checking for almost > all function calls. This is a good behavior, but existing cxl_count_regblock > couldn't return error codes. > > With this patch, only need to have minor modifications in > __cxl_find_regblock_instance, which can return the count of regblocks by given > RBI in just one call. It is more effective than before. Besides, the error code > could be obtained by the called function, here is cxl_pci_probe. > > Based on the above reasons, refactor out cxl_count_regblock and create > __cxl_find_regblock_instance for counting instances more efficiently. > I'd suggest moving the testing setup to a cover letter. It is more detail that we necessarily need in the git log. Maybe if this change is otherwise acceptable, Dave might tidy this up whilst applying. > This patch is tested by ndctl cxl_test and physical CXL expander card > with v6.13-rc6. > > 1. Ndctl CXL test suite v80 could pass with this patch applied. > $ meson test -C build --suite cxl > ninja: Entering directory `/home/work/source/ndctl/build' > [1/48] Generating version.h with a custom command > 1/11 ndctl:cxl / cxl-topology.sh OK 3.48s > 2/11 ndctl:cxl / cxl-region-sysfs.sh OK 2.74s > 3/11 ndctl:cxl / cxl-labels.sh OK 1.75s > 4/11 ndctl:cxl / cxl-create-region.sh OK 3.51s > 5/11 ndctl:cxl / cxl-xor-region.sh OK 1.89s > 6/11 ndctl:cxl / cxl-events.sh OK 1.63s > 7/11 ndctl:cxl / cxl-sanitize.sh OK 4.48s > 8/11 ndctl:cxl / cxl-destroy-region.sh OK 1.90s > 9/11 ndctl:cxl / cxl-qos-class.sh OK 2.65s > 10/11 ndctl:cxl / cxl-poison.sh OK 2.86s > 11/11 ndctl:cxl / cxl-security.sh OK 0.91s > > 2. Test patch with Qemu x4 switch topology: > > ACPI0017:00 [root0] > | > HB_0 [port1] > / \ > RP_0 RP_1 > | | > USP [port2] > / / \ \ > DSP DSP DSP DSP > | | | | > mem1 mem0 mem2 mem3 > > Every card has 2 PMU RBs, here are the pmu_mem devices. > > $ pwd > /sys/bus/cxl/devices > $ tree > [snip] > ├── pmu_mem0.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:01.0/0000:10:00.0/pmu_mem0.0 > ├── pmu_mem0.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:01.0/0000:10:00.0/pmu_mem0.1 > ├── pmu_mem1.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:00.0/0000:0f:00.0/pmu_mem1.0 > ├── pmu_mem1.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:00.0/0000:0f:00.0/pmu_mem1.1 > ├── pmu_mem2.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:02.0/0000:11:00.0/pmu_mem2.0 > ├── pmu_mem2.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:02.0/0000:11:00.0/pmu_mem2.1 > ├── pmu_mem3.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:03.0/0000:12:00.0/pmu_mem3.0 > ├── pmu_mem3.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:03.0/0000:12:00.0/pmu_mem3.1 > Change log belongs below the --- > Changes > ======= > v2 -> v3: > > 1. Create static function __cxl_find_regblock_instance() for implementation of > locating a register block or counting instances by type / index > underneath. (Jonathan Cameron) > 2. cxl_count_regblock() and cxl_find_regblock_instance() respectively call > __cxl_find_regblock_instance for the purpose of counting instances and > locating RB. (Jonathan Cameron) > 3. Change parameter index's type to 'unsigned int' in cxl_find_regblock_instance (Jonathan Cameron) > 4. Rebase patch to v6.13-rc6 > > ======= > v1 -> v2: > > 1. Reserved cxl_count_regblock() for original function interface (Ming Li) > 2. Reset 'map->resource' to 'CXL_RESOURCE_NONE' before returning the count of instances in > cxl_find_regblock_instance() (Ming Li) > 3. Append results of ndctl test suite and Qemu testing PMU devices to commit log (Ming Li) > 4. Rebase patch to v6.13-rc5 > > [v2] https://lore.kernel.org/all/20241230122239.3445117-1-huaisheng.ye@intel.com/ > > Signed-off-by: Huaisheng Ye <huaisheng.ye@intel.com> > --- Put change logs here. The code looks good to me. I did wonder if it made sense to keep so much documentation for the static helper function, but on balance I think that is fine. So with the patch description tidied up Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 59cb35b40c7e..d9c1818acdd5 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -290,19 +290,22 @@ static bool cxl_decode_regblock(struct pci_dev *pdev, u32 reg_lo, u32 reg_hi, } /** - * cxl_find_regblock_instance() - Locate a register block by type / index + * __cxl_find_regblock_instance() - Locate a register block or count instances by type / index * @pdev: The CXL PCI device to enumerate. * @type: Register Block Indicator id * @map: Enumeration output, clobbered on error * @index: Index into which particular instance of a regblock wanted in the * order found in register locator DVSEC. * - * Return: 0 if register block enumerated, negative error code otherwise + * Return: 0 if register block enumerated, non-negative if instances counted, + * negative error code otherwise * * A CXL DVSEC may point to one or more register blocks, search for them * by @type and @index. + * + * Use CXL_INSTANCES_COUNT for @index if counting instances by @type and @index. */ -int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, +static int __cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map, int index) { u32 regloc_size, regblocks; @@ -342,8 +345,30 @@ int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, } map->resource = CXL_RESOURCE_NONE; + if (index == CXL_INSTANCES_COUNT) + return instance; + return -ENODEV; } + +/** + * cxl_find_regblock_instance() - Locate a register block by type / index + * @pdev: The CXL PCI device to enumerate. + * @type: Register Block Indicator id + * @map: Enumeration output, clobbered on error + * @index: Index into which particular instance of a regblock wanted in the + * order found in register locator DVSEC. + * + * Return: 0 if register block enumerated, negative error code otherwise + * + * A CXL DVSEC may point to one or more register blocks, search for them + * by @type and @index. + */ +int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, + struct cxl_register_map *map, unsigned int index) +{ + return __cxl_find_regblock_instance(pdev, type, map, index); +} EXPORT_SYMBOL_NS_GPL(cxl_find_regblock_instance, "CXL"); /** @@ -360,7 +385,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_find_regblock_instance, "CXL"); int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map) { - return cxl_find_regblock_instance(pdev, type, map, 0); + return __cxl_find_regblock_instance(pdev, type, map, 0); } EXPORT_SYMBOL_NS_GPL(cxl_find_regblock, "CXL"); @@ -371,19 +396,13 @@ EXPORT_SYMBOL_NS_GPL(cxl_find_regblock, "CXL"); * * Some regblocks may be repeated. Count how many instances. * - * Return: count of matching regblocks. + * Return: non-negative count of matching regblocks, negative error code otherwise. */ int cxl_count_regblock(struct pci_dev *pdev, enum cxl_regloc_type type) { struct cxl_register_map map; - int rc, count = 0; - while (1) { - rc = cxl_find_regblock_instance(pdev, type, &map, count); - if (rc) - return count; - count++; - } + return __cxl_find_regblock_instance(pdev, type, &map, CXL_INSTANCES_COUNT); } EXPORT_SYMBOL_NS_GPL(cxl_count_regblock, "CXL"); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f6015f24ad38..7fc456d5b917 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -302,10 +302,11 @@ int cxl_map_device_regs(const struct cxl_register_map *map, struct cxl_device_regs *regs); int cxl_map_pmu_regs(struct cxl_register_map *map, struct cxl_pmu_regs *regs); +#define CXL_INSTANCES_COUNT -1 enum cxl_regloc_type; int cxl_count_regblock(struct pci_dev *pdev, enum cxl_regloc_type type); int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, - struct cxl_register_map *map, int index); + struct cxl_register_map *map, unsigned int index); int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map); int cxl_setup_regs(struct cxl_register_map *map); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 6d94ff4a4f1a..a96e54c6259e 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -907,7 +907,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) struct cxl_dev_state *cxlds; struct cxl_register_map map; struct cxl_memdev *cxlmd; - int i, rc, pmu_count; + int rc, pmu_count; + unsigned int i; bool irq_avail; /* @@ -1009,6 +1010,9 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) return rc; pmu_count = cxl_count_regblock(pdev, CXL_REGLOC_RBI_PMU); + if (pmu_count < 0) + return pmu_count; + for (i = 0; i < pmu_count; i++) { struct cxl_pmu_regs pmu_regs;
In commit d717d7f3df18494baafd9595fb4bcb9c380d7389, cxl_count_regblock was added for counting regblocks of a given RBI (Register Block Identifier). It is workable, but the implementation can be improved. 1. In order to get the count of instances, cxl_count_regblock has to tentatively repeat the call of cxl_find_regblock_instance by increasing index from 0. It will not stop until an error value is returned. Actually, It needs to search for Register Blocks in dvsec again every time by taking a start from the head. The operations can be optimized. For example, to determine if PMU1 exists, cxl_find_regblock_instance must check all Register Blocks by the type and index from RB1 to RB4, starting from scratch, even if PMU0 just has been searched. If there are more RBs of the same type in the future, the situation will be even worse. 16 81 00 23 PCIe extended Capability Header 02 c0 1e 98 Header 1 00 00 00 08 Header 2 -------------------------------------- 00 00 01 00 RB 1 - Offset Low Component 00 00 00 00 RB 1 - Offset High -------------------------------------- 00 00 03 02 RB 2 - Offset Low Device Register 00 00 00 00 RB 2 - Offset High -------------------------------------- 00 01 04 02 RB 3 - Offset Low PMU0 00 00 00 00 RB 3 - Offset High -------------------------------------- 00 02 04 02 RB 4 - Offset Low PMU1 00 00 00 00 RB 4 - Offset High RB: Register Block 2. cxl_count_regblock blocks the opportunity to get error codes from cxl_find_regblock_instance. cxl_pci_probe has error code checking for almost all function calls. This is a good behavior, but existing cxl_count_regblock couldn't return error codes. With this patch, only need to have minor modifications in __cxl_find_regblock_instance, which can return the count of regblocks by given RBI in just one call. It is more effective than before. Besides, the error code could be obtained by the called function, here is cxl_pci_probe. Based on the above reasons, refactor out cxl_count_regblock and create __cxl_find_regblock_instance for counting instances more efficiently. This patch is tested by ndctl cxl_test and physical CXL expander card with v6.13-rc6. 1. Ndctl CXL test suite v80 could pass with this patch applied. $ meson test -C build --suite cxl ninja: Entering directory `/home/work/source/ndctl/build' [1/48] Generating version.h with a custom command 1/11 ndctl:cxl / cxl-topology.sh OK 3.48s 2/11 ndctl:cxl / cxl-region-sysfs.sh OK 2.74s 3/11 ndctl:cxl / cxl-labels.sh OK 1.75s 4/11 ndctl:cxl / cxl-create-region.sh OK 3.51s 5/11 ndctl:cxl / cxl-xor-region.sh OK 1.89s 6/11 ndctl:cxl / cxl-events.sh OK 1.63s 7/11 ndctl:cxl / cxl-sanitize.sh OK 4.48s 8/11 ndctl:cxl / cxl-destroy-region.sh OK 1.90s 9/11 ndctl:cxl / cxl-qos-class.sh OK 2.65s 10/11 ndctl:cxl / cxl-poison.sh OK 2.86s 11/11 ndctl:cxl / cxl-security.sh OK 0.91s 2. Test patch with Qemu x4 switch topology: ACPI0017:00 [root0] | HB_0 [port1] / \ RP_0 RP_1 | | USP [port2] / / \ \ DSP DSP DSP DSP | | | | mem1 mem0 mem2 mem3 Every card has 2 PMU RBs, here are the pmu_mem devices. $ pwd /sys/bus/cxl/devices $ tree [snip] ├── pmu_mem0.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:01.0/0000:10:00.0/pmu_mem0.0 ├── pmu_mem0.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:01.0/0000:10:00.0/pmu_mem0.1 ├── pmu_mem1.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:00.0/0000:0f:00.0/pmu_mem1.0 ├── pmu_mem1.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:00.0/0000:0f:00.0/pmu_mem1.1 ├── pmu_mem2.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:02.0/0000:11:00.0/pmu_mem2.0 ├── pmu_mem2.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:02.0/0000:11:00.0/pmu_mem2.1 ├── pmu_mem3.0 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:03.0/0000:12:00.0/pmu_mem3.0 ├── pmu_mem3.1 -> ../../../devices/pci0000:0c/0000:0c:00.0/0000:0d:00.0/0000:0e:03.0/0000:12:00.0/pmu_mem3.1 Changes ======= v2 -> v3: 1. Create static function __cxl_find_regblock_instance() for implementation of locating a register block or counting instances by type / index underneath. (Jonathan Cameron) 2. cxl_count_regblock() and cxl_find_regblock_instance() respectively call __cxl_find_regblock_instance for the purpose of counting instances and locating RB. (Jonathan Cameron) 3. Change parameter index's type to 'unsigned int' in cxl_find_regblock_instance (Jonathan Cameron) 4. Rebase patch to v6.13-rc6 ======= v1 -> v2: 1. Reserved cxl_count_regblock() for original function interface (Ming Li) 2. Reset 'map->resource' to 'CXL_RESOURCE_NONE' before returning the count of instances in cxl_find_regblock_instance() (Ming Li) 3. Append results of ndctl test suite and Qemu testing PMU devices to commit log (Ming Li) 4. Rebase patch to v6.13-rc5 [v2] https://lore.kernel.org/all/20241230122239.3445117-1-huaisheng.ye@intel.com/ Signed-off-by: Huaisheng Ye <huaisheng.ye@intel.com> --- drivers/cxl/core/regs.c | 43 +++++++++++++++++++++++++++++------------ drivers/cxl/cxl.h | 3 ++- drivers/cxl/pci.c | 6 +++++- 3 files changed, 38 insertions(+), 14 deletions(-)