From patchwork Fri Jan 28 00:26:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727581 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 030F22CA5; Fri, 28 Jan 2022 00:27:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329644; x=1674865644; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k4EyZJwC3PbUFi46FkaEvm2S5Ygc92wg6uxvDMN65bM=; b=JDZ//XDl78U/8e5tRhn/mu02rTaTdl+1m7OUS+6sXIGIO9C5thg3Npdk DxYhVZkoJxhASK8K98tZz8Q2yFnEh04qw59+9GHARKmkR9gMKTF/gmAc9 DqpX5/qCGYUKsWJH8RGWyXCQevfb+DVpaRLsxpHyXwLRN6EcaddR/+vfj vK6AcdXswNnngNFv/SHyL/BlOaamE95g6D6H5JxGGPzKmWaIkgWfB0gTM TKELD4es5kctX0cO1fbevK/iy0pNLx3fJjDpEnOuYY5kwWLKiI9asAelI waNYRPjR+ETfehexWm1XCpnV4cMMb3785Dmm3HmTelCR2bcdjTDAVnHvy g==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982062" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982062" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:22 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909600" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:22 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 01/14] cxl/region: Add region creation ABI Date: Thu, 27 Jan 2022 16:26:54 -0800 Message-Id: <20220128002707.391076-2-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Regions are created as a child of the decoder that encompasses an address space with constraints. Regions have a number of attributes that must be configured before the region can be activated. The ABI is not meant to be secure, but is meant to avoid accidental races. As a result, a buggy process may create a region by name that was allocated by a different process. However, multiple processes which are trying not to race with each other shouldn't need special synchronization to do so. // Allocate a new region name region=$(cat /sys/bus/cxl/devices/decoder0.0/create_region) // Create a new region by name echo $region > /sys/bus/cxl/devices/decoder0.0/create_region // Region now exists in sysfs stat -t /sys/bus/cxl/devices/decoder0.0/$region // Delete the region, and name echo $region > /sys/bus/cxl/devices/decoder0.0/delete_region Signed-off-by: Ben Widawsky --- Changes since v2: - Rename 'region' variables to 'cxlr' - Update ABI docs for possible actual upstream version --- Documentation/ABI/testing/sysfs-bus-cxl | 24 ++ .../driver-api/cxl/memory-devices.rst | 11 + drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 3 + drivers/cxl/core/port.c | 16 ++ drivers/cxl/core/region.c | 208 ++++++++++++++++++ drivers/cxl/cxl.h | 9 + drivers/cxl/region.h | 38 ++++ tools/testing/cxl/Kbuild | 1 + 9 files changed, 311 insertions(+) create mode 100644 drivers/cxl/core/region.c create mode 100644 drivers/cxl/region.h diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 7c2b846521f3..dcc728458936 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -163,3 +163,27 @@ Description: memory (type-3). The 'target_type' attribute indicates the current setting which may dynamically change based on what memory regions are activated in this decode hierarchy. + +What: /sys/bus/cxl/devices/decoderX.Y/create_region +Date: August, 2021 +KernelVersion: v5.18 +Contact: linux-cxl@vger.kernel.org +Description: + Creates a new CXL region. Writing a value of the form + "regionX.Y:Z" will create a new uninitialized region that will + be mapped by the CXL decoderX.Y. Reading from this node will + return a newly allocated region name. In order to create a + region (writing) you must use a value returned from reading the + node. Regions must be created for root decoders, and must + subsequently configured and bound to a region driver before they + can be used. + +What: /sys/bus/cxl/devices/decoderX.Y/delete_region +Date: August, 2021 +KernelVersion: v5.18 +Contact: linux-cxl@vger.kernel.org +Description: + Deletes the named region. A region must be unbound from the + region driver before being deleted. The attributes expects a + region in the form "regionX.Y:Z". The region's name, allocated + by reading create_region, will also be released. diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst index db476bb170b6..66ddc58a21b1 100644 --- a/Documentation/driver-api/cxl/memory-devices.rst +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -362,6 +362,17 @@ CXL Core .. kernel-doc:: drivers/cxl/core/mbox.c :doc: cxl mbox +CXL Regions +----------- +.. kernel-doc:: drivers/cxl/region.h + :identifiers: + +.. kernel-doc:: drivers/cxl/core/region.c + :doc: cxl core region + +.. kernel-doc:: drivers/cxl/core/region.c + :identifiers: + External Interfaces =================== diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 6d37cd78b151..39ce8f2f2373 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -4,6 +4,7 @@ obj-$(CONFIG_CXL_BUS) += cxl_core.o ccflags-y += -I$(srctree)/drivers/cxl cxl_core-y := port.o cxl_core-y += pmem.o +cxl_core-y += region.o cxl_core-y += regs.o cxl_core-y += memdev.o cxl_core-y += mbox.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index efbaa851929d..35fd08d560e2 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -10,6 +10,9 @@ extern const struct device_type cxl_memdev_type; extern struct attribute_group cxl_base_attribute_group; +extern struct device_attribute dev_attr_create_region; +extern struct device_attribute dev_attr_delete_region; + struct cxl_send_command; struct cxl_mem_query_commands; int cxl_query_cmd(struct cxl_memdev *cxlmd, diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 631dec0fa79e..0826208b2bdf 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -215,6 +215,8 @@ static struct attribute_group cxl_decoder_base_attribute_group = { }; static struct attribute *cxl_decoder_root_attrs[] = { + &dev_attr_create_region.attr, + &dev_attr_delete_region.attr, &dev_attr_cap_pmem.attr, &dev_attr_cap_ram.attr, &dev_attr_cap_type2.attr, @@ -267,11 +269,23 @@ static const struct attribute_group *cxl_decoder_endpoint_attribute_groups[] = { NULL, }; +static int delete_region(struct device *dev, void *arg) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev->parent); + + return cxl_delete_region(cxld, dev_name(dev)); +} + static void cxl_decoder_release(struct device *dev) { struct cxl_decoder *cxld = to_cxl_decoder(dev); struct cxl_port *port = to_cxl_port(dev->parent); + device_for_each_child(&cxld->dev, cxld, delete_region); + + dev_WARN_ONCE(dev, !ida_is_empty(&cxld->region_ida), + "Lost track of a region"); + ida_free(&port->decoder_ida, cxld->id); kfree(cxld); } @@ -1194,6 +1208,8 @@ static struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, cxld->target_type = CXL_DECODER_EXPANDER; cxld->platform_res = (struct resource)DEFINE_RES_MEM(0, 0); + ida_init(&cxld->region_ida); + return cxld; err: kfree(cxld); diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c new file mode 100644 index 000000000000..1a448543db0d --- /dev/null +++ b/drivers/cxl/core/region.c @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2021 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include +#include "core.h" + +/** + * DOC: cxl core region + * + * Regions are managed through the Linux device model. Each region instance is a + * unique struct device. CXL core provides functionality to create, destroy, and + * configure regions. This is all implemented here. Binding a region + * (programming the hardware) is handled by a separate region driver. + */ + +static void cxl_region_release(struct device *dev); + +static const struct device_type cxl_region_type = { + .name = "cxl_region", + .release = cxl_region_release, +}; + +static ssize_t create_region_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_port *port = to_cxl_port(dev->parent); + struct cxl_decoder *cxld = to_cxl_decoder(dev); + int rc; + + if (dev_WARN_ONCE(dev, !is_root_decoder(dev), + "Invalid decoder selected for region.")) { + return -ENODEV; + } + + rc = ida_alloc(&cxld->region_ida, GFP_KERNEL); + if (rc < 0) { + dev_err(&cxld->dev, "Couldn't get a new id\n"); + return rc; + } + + return sysfs_emit(buf, "region%d.%d:%d\n", port->id, cxld->id, rc); +} + +static ssize_t create_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_port *port = to_cxl_port(dev->parent); + struct cxl_decoder *cxld = to_cxl_decoder(dev); + int decoder_id, port_id, region_id; + struct cxl_region *cxlr; + ssize_t rc; + + if (sscanf(buf, "region%d.%d:%d", &port_id, &decoder_id, ®ion_id) != 3) + return -EINVAL; + + if (decoder_id != cxld->id) + return -EINVAL; + + if (port_id != port->id) + return -EINVAL; + + cxlr = cxl_alloc_region(cxld, region_id); + if (IS_ERR(cxlr)) + return PTR_ERR(cxlr); + + rc = cxl_add_region(cxld, cxlr); + if (rc) { + kfree(cxlr); + return rc; + } + + return len; +} +DEVICE_ATTR_RW(create_region); + +static ssize_t delete_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev); + int rc; + + rc = cxl_delete_region(cxld, buf); + if (rc) + return rc; + + return len; +} +DEVICE_ATTR_WO(delete_region); + +struct cxl_region *to_cxl_region(struct device *dev) +{ + if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type, + "not a cxl_region device\n")) + return NULL; + + return container_of(dev, struct cxl_region, dev); +} +EXPORT_SYMBOL_GPL(to_cxl_region); + +static void cxl_region_release(struct device *dev) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev->parent); + struct cxl_region *cxlr = to_cxl_region(dev); + + ida_free(&cxld->region_ida, cxlr->id); + kfree(cxlr); +} + +struct cxl_region *cxl_alloc_region(struct cxl_decoder *cxld, int id) +{ + struct cxl_region *cxlr; + + cxlr = kzalloc(sizeof(*cxlr), GFP_KERNEL); + if (!cxlr) + return ERR_PTR(-ENOMEM); + + cxlr->id = id; + + return cxlr; +} + +/** + * cxl_add_region - Adds a region to a decoder + * @cxld: Parent decoder. + * @cxlr: Region to be added to the decoder. + * + * This is the second step of region initialization. Regions exist within an + * address space which is mapped by a @cxld. That @cxld must be a root decoder, + * and it enforces constraints upon the region as it is configured. + * + * Return: 0 if the region was added to the @cxld, else returns negative error + * code. The region will be named "regionX.Y.Z" where X is the port, Y is the + * decoder id, and Z is the region number. + */ +int cxl_add_region(struct cxl_decoder *cxld, struct cxl_region *cxlr) +{ + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + struct device *dev = &cxlr->dev; + int rc; + + device_initialize(dev); + dev->parent = &cxld->dev; + device_set_pm_not_required(dev); + dev->bus = &cxl_bus_type; + dev->type = &cxl_region_type; + rc = dev_set_name(dev, "region%d.%d:%d", port->id, cxld->id, cxlr->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + dev_dbg(dev, "Added to %s\n", dev_name(&cxld->dev)); + + return 0; + +err: + put_device(dev); + return rc; +} + +static struct cxl_region *cxl_find_region_by_name(struct cxl_decoder *cxld, + const char *name) +{ + struct device *region_dev; + + region_dev = device_find_child_by_name(&cxld->dev, name); + if (!region_dev) + return ERR_PTR(-ENOENT); + + return to_cxl_region(region_dev); +} + +/** + * cxl_delete_region - Deletes a region + * @cxld: Parent decoder + * @region_name: Named region, ie. regionX.Y:Z + */ +int cxl_delete_region(struct cxl_decoder *cxld, const char *region_name) +{ + struct cxl_region *cxlr; + + device_lock(&cxld->dev); + + cxlr = cxl_find_region_by_name(cxld, region_name); + if (IS_ERR(cxlr)) { + device_unlock(&cxld->dev); + return PTR_ERR(cxlr); + } + + dev_dbg(&cxld->dev, "Requested removal of %s from %s\n", + dev_name(&cxlr->dev), dev_name(&cxld->dev)); + + device_unregister(&cxlr->dev); + device_unlock(&cxld->dev); + + put_device(&cxlr->dev); + + return 0; +} diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 13fb06849199..b9f0099c1f39 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -221,6 +221,7 @@ enum cxl_decoder_type { * @target_type: accelerator vs expander (type2 vs type3) selector * @flags: memory type capabilities and locking * @target_lock: coordinate coherent reads of the target list + * @region_ida: allocator for region ids. * @nr_targets: number of elements in @target * @target: active ordered target list in current decoder configuration */ @@ -236,6 +237,7 @@ struct cxl_decoder { enum cxl_decoder_type target_type; unsigned long flags; seqlock_t target_lock; + struct ida region_ida; int nr_targets; struct cxl_dport *target[]; }; @@ -323,6 +325,13 @@ struct cxl_ep { struct list_head list; }; +bool is_cxl_region(struct device *dev); +struct cxl_region *to_cxl_region(struct device *dev); +struct cxl_region *cxl_alloc_region(struct cxl_decoder *cxld, + int interleave_ways); +int cxl_add_region(struct cxl_decoder *cxld, struct cxl_region *cxlr); +int cxl_delete_region(struct cxl_decoder *cxld, const char *region); + static inline bool is_cxl_root(struct cxl_port *port) { return port->uport == port->dev.parent; diff --git a/drivers/cxl/region.h b/drivers/cxl/region.h new file mode 100644 index 000000000000..eb1249e3c1d4 --- /dev/null +++ b/drivers/cxl/region.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2021 Intel Corporation. */ +#ifndef __CXL_REGION_H__ +#define __CXL_REGION_H__ + +#include + +#include "cxl.h" + +/** + * struct cxl_region - CXL region + * @dev: This region's device. + * @id: This regions id. Id is globally unique across all regions. + * @list: Node in decoder's region list. + * @res: Resource this region carves out of the platform decode range. + * @config: HDM decoder program config + * @config.size: Size of the region determined from LSA or userspace. + * @config.uuid: The UUID for this region. + * @config.interleave_ways: Number of interleave ways this region is configured for. + * @config.interleave_granularity: Interleave granularity of region + * @config.targets: The memory devices comprising the region. + */ +struct cxl_region { + struct device dev; + int id; + struct list_head list; + struct resource *res; + + struct { + u64 size; + uuid_t uuid; + int interleave_ways; + int interleave_granularity; + struct cxl_memdev *targets[CXL_DECODER_MAX_INTERLEAVE]; + } config; +}; + +#endif diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index 82e49ab0937d..3fe6d34e6d59 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -46,6 +46,7 @@ cxl_core-y += $(CXL_CORE_SRC)/memdev.o cxl_core-y += $(CXL_CORE_SRC)/mbox.o cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o +cxl_core-y += $(CXL_CORE_SRC)/region.o cxl_core-y += config_check.o obj-m += test/ From patchwork Fri Jan 28 00:26:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727582 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36BC52CA8; Fri, 28 Jan 2022 00:27:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329645; x=1674865645; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6lj81oX7B06c7PykM7fTdzFA6ZxKENp6mQB/oDrHmeE=; b=M8NV3BRDKShrNQEkKbCauHUnzlGFb/D8nIaao7AqiDy1p3PbzCj2VrHF yui/2t14T+oGn5Vb3vKDqFdInaNgtxJNVux4LezMsNHKGNiLeTWNpSrSf p06ePdItzXvLH8DtNDiygCVA03cmRkpkZMAp1k0UO0lgByDuuRIURSoOy GJ9lkSSK401gZLItpW1UTBsxR0Mrf7YSoDTNQV0hLeP92WTi7ADj7uXEM m60MvGrWRAvh+7wFgAP0WaIiNHb0FbvCgrwTbUTIduBBh6nCoKKJVUo+E ES320MbquMMqmwOanZHeI4dps5UY+qDvBvHzHIdYkU2woPE02wC+TIc1w Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982064" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982064" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:23 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909604" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:23 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , kernel test robot , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 02/14] cxl/region: Introduce concept of region configuration Date: Thu, 27 Jan 2022 16:26:55 -0800 Message-Id: <20220128002707.391076-3-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The region creation APIs create a vacant region. Configuring the region works in the same way as similar subsystems such as devdax. Sysfs attrs will be provided to allow userspace to configure the region. Finally once all configuration is complete, userspace may activate the region. Introduced here are the most basic attributes needed to configure a region. Details of these attribute are described in the ABI Documentation. Sanity checking of configuration parameters are done at region binding time. This consolidates all such logic in one place, rather than being strewn across multiple places. A example is provided below: /sys/bus/cxl/devices/region0.0:0 ├── interleave_granularity ├── interleave_ways ├── offset ├── size ├── subsystem -> ../../../../../../bus/cxl ├── target0 ├── uevent └── uuid Reported-by: kernel test robot (v2) Signed-off-by: Ben Widawsky --- Documentation/ABI/testing/sysfs-bus-cxl | 40 ++++ drivers/cxl/core/region.c | 300 ++++++++++++++++++++++++ 2 files changed, 340 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index dcc728458936..50ba5018014d 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -187,3 +187,43 @@ Description: region driver before being deleted. The attributes expects a region in the form "regionX.Y:Z". The region's name, allocated by reading create_region, will also be released. + +What: /sys/bus/cxl/devices/decoderX.Y/regionX.Y:Z/offset +Date: August, 2021 +KernelVersion: v5.18 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) A region resides within an address space that is claimed by + a decoder. Region space allocation is handled by the driver, but + the offset may be read by userspace tooling in order to + determine fragmentation, and available size for new regions. + +What: +/sys/bus/cxl/devices/decoderX.Y/regionX.Y:Z/{interleave,size,uuid,target[0-15]} +Date: August, 2021 +KernelVersion: v5.18 +Contact: linux-cxl@vger.kernel.org +Description: + (RW) Configuring regions requires a minimal set of parameters in + order for the subsequent bind operation to succeed. The + following parameters are defined: + + == ======================================================== + interleave_granularity Mandatory. Number of consecutive bytes + each device in the interleave set will claim. The + possible interleave granularity values are determined by + the CXL spec and the participating devices. + interleave_ways Mandatory. Number of devices participating in the + region. Each device will provide 1/interleave of storage + for the region. + size Manadatory. Phsyical address space the region will + consume. + target Mandatory. Memory devices are the backing storage for a + region. There will be N targets based on the number of + interleave ways that the top level decoder is configured + for. Each target must be set with a memdev device ie. + 'mem1'. This attribute only becomes available after + setting the 'interleave' attribute. + uuid Optional. A unique identifier for the region. If none is + selected, the kernel will create one. + == ======================================================== diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 1a448543db0d..3b48e0469fc7 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3,9 +3,12 @@ #include #include #include +#include #include +#include #include #include +#include #include #include "core.h" @@ -18,11 +21,305 @@ * (programming the hardware) is handled by a separate region driver. */ +struct cxl_region *to_cxl_region(struct device *dev); +static const struct attribute_group region_interleave_group; + +static bool is_region_active(struct cxl_region *cxlr) +{ + /* TODO: Regions can't be activated yet. */ + return false; +} + +static void remove_target(struct cxl_region *cxlr, int target) +{ + struct cxl_memdev *cxlmd; + + cxlmd = cxlr->config.targets[target]; + if (cxlmd) + put_device(&cxlmd->dev); + cxlr->config.targets[target] = NULL; +} + +static ssize_t interleave_ways_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%d\n", cxlr->config.interleave_ways); +} + +static ssize_t interleave_ways_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + int ret, prev_iw; + int val; + + prev_iw = cxlr->config.interleave_ways; + ret = kstrtoint(buf, 0, &val); + if (ret) + return ret; + if (ret < 0 || ret > CXL_DECODER_MAX_INTERLEAVE) + return -EINVAL; + + cxlr->config.interleave_ways = val; + + ret = sysfs_update_group(&dev->kobj, ®ion_interleave_group); + if (ret < 0) + goto err; + + sysfs_notify(&dev->kobj, NULL, "target_interleave"); + + while (prev_iw > cxlr->config.interleave_ways) + remove_target(cxlr, --prev_iw); + + return len; + +err: + cxlr->config.interleave_ways = prev_iw; + return ret; +} +static DEVICE_ATTR_RW(interleave_ways); + +static ssize_t interleave_granularity_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%d\n", cxlr->config.interleave_granularity); +} + +static ssize_t interleave_granularity_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + int val, ret; + + ret = kstrtoint(buf, 0, &val); + if (ret) + return ret; + cxlr->config.interleave_granularity = val; + + return len; +} +static DEVICE_ATTR_RW(interleave_granularity); + +static ssize_t offset_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev->parent); + struct cxl_region *cxlr = to_cxl_region(dev); + resource_size_t offset; + + if (!cxlr->res) + return sysfs_emit(buf, "\n"); + + offset = cxld->platform_res.start - cxlr->res->start; + + return sysfs_emit(buf, "%pa\n", &offset); +} +static DEVICE_ATTR_RO(offset); + +static ssize_t size_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%llu\n", cxlr->config.size); +} + +static ssize_t size_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + unsigned long long val; + ssize_t rc; + + rc = kstrtoull(buf, 0, &val); + if (rc) + return rc; + + device_lock(&cxlr->dev); + if (is_region_active(cxlr)) + rc = -EBUSY; + else + cxlr->config.size = val; + device_unlock(&cxlr->dev); + + return rc ? rc : len; +} +static DEVICE_ATTR_RW(size); + +static ssize_t uuid_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%pUb\n", &cxlr->config.uuid); +} + +static ssize_t uuid_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + ssize_t rc; + + if (len != UUID_STRING_LEN + 1) + return -EINVAL; + + device_lock(&cxlr->dev); + if (is_region_active(cxlr)) + rc = -EBUSY; + else + rc = uuid_parse(buf, &cxlr->config.uuid); + device_unlock(&cxlr->dev); + + return rc ? rc : len; +} +static DEVICE_ATTR_RW(uuid); + +static struct attribute *region_attrs[] = { + &dev_attr_interleave_ways.attr, + &dev_attr_interleave_granularity.attr, + &dev_attr_offset.attr, + &dev_attr_size.attr, + &dev_attr_uuid.attr, + NULL, +}; + +static const struct attribute_group region_group = { + .attrs = region_attrs, +}; + +static size_t show_targetN(struct cxl_region *cxlr, char *buf, int n) +{ + int ret; + + device_lock(&cxlr->dev); + if (!cxlr->config.targets[n]) + ret = sysfs_emit(buf, "\n"); + else + ret = sysfs_emit(buf, "%s\n", + dev_name(&cxlr->config.targets[n]->dev)); + device_unlock(&cxlr->dev); + + return ret; +} + +static size_t set_targetN(struct cxl_region *cxlr, const char *buf, int n, + size_t len) +{ + struct device *memdev_dev; + struct cxl_memdev *cxlmd; + + device_lock(&cxlr->dev); + + if (len == 1 || cxlr->config.targets[n]) + remove_target(cxlr, n); + + /* Remove target special case */ + if (len == 1) { + device_unlock(&cxlr->dev); + return len; + } + + memdev_dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf); + if (!memdev_dev) { + device_unlock(&cxlr->dev); + return -ENOENT; + } + + /* reference to memdev held until target is unset or region goes away */ + + cxlmd = to_cxl_memdev(memdev_dev); + cxlr->config.targets[n] = cxlmd; + + device_unlock(&cxlr->dev); + + return len; +} + +#define TARGET_ATTR_RW(n) \ + static ssize_t target##n##_show( \ + struct device *dev, struct device_attribute *attr, char *buf) \ + { \ + return show_targetN(to_cxl_region(dev), buf, (n)); \ + } \ + static ssize_t target##n##_store(struct device *dev, \ + struct device_attribute *attr, \ + const char *buf, size_t len) \ + { \ + return set_targetN(to_cxl_region(dev), buf, (n), len); \ + } \ + static DEVICE_ATTR_RW(target##n) + +TARGET_ATTR_RW(0); +TARGET_ATTR_RW(1); +TARGET_ATTR_RW(2); +TARGET_ATTR_RW(3); +TARGET_ATTR_RW(4); +TARGET_ATTR_RW(5); +TARGET_ATTR_RW(6); +TARGET_ATTR_RW(7); +TARGET_ATTR_RW(8); +TARGET_ATTR_RW(9); +TARGET_ATTR_RW(10); +TARGET_ATTR_RW(11); +TARGET_ATTR_RW(12); +TARGET_ATTR_RW(13); +TARGET_ATTR_RW(14); +TARGET_ATTR_RW(15); + +static struct attribute *interleave_attrs[] = { + &dev_attr_target0.attr, + &dev_attr_target1.attr, + &dev_attr_target2.attr, + &dev_attr_target3.attr, + &dev_attr_target4.attr, + &dev_attr_target5.attr, + &dev_attr_target6.attr, + &dev_attr_target7.attr, + &dev_attr_target8.attr, + &dev_attr_target9.attr, + &dev_attr_target10.attr, + &dev_attr_target11.attr, + &dev_attr_target12.attr, + &dev_attr_target13.attr, + &dev_attr_target14.attr, + &dev_attr_target15.attr, + NULL, +}; + +static umode_t visible_targets(struct kobject *kobj, struct attribute *a, int n) +{ + struct device *dev = container_of(kobj, struct device, kobj); + struct cxl_region *cxlr = to_cxl_region(dev); + + if (n < cxlr->config.interleave_ways) + return a->mode; + return 0; +} + +static const struct attribute_group region_interleave_group = { + .attrs = interleave_attrs, + .is_visible = visible_targets, +}; + +static const struct attribute_group *region_groups[] = { + ®ion_group, + ®ion_interleave_group, + NULL, +}; + static void cxl_region_release(struct device *dev); static const struct device_type cxl_region_type = { .name = "cxl_region", .release = cxl_region_release, + .groups = region_groups }; static ssize_t create_region_show(struct device *dev, @@ -108,8 +405,11 @@ static void cxl_region_release(struct device *dev) { struct cxl_decoder *cxld = to_cxl_decoder(dev->parent); struct cxl_region *cxlr = to_cxl_region(dev); + int i; ida_free(&cxld->region_ida, cxlr->id); + for (i = 0; i < cxlr->config.interleave_ways; i++) + remove_target(cxlr, i); kfree(cxlr); } From patchwork Fri Jan 28 00:26:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727583 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5ABB62C9E; Fri, 28 Jan 2022 00:27:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329645; x=1674865645; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T9XKBUWEiW5Gc/EQfMiMf7KQVNCUAgi9KBZ44Hu22xc=; b=UkVwQcvXVpcBSTL+v+MlZ5CiO7w6bWINzE+iulnx4oqNtXcaS+5vLXBV Jz9Zm6ljXbCyCdxxfcI4YikkYU+mYkrw/VBU0UtwwUS3wvDz81MPaeGlE /ZgM/us4Kflt02vwfSix+S6ix4bg3RPEL8km5opi3yrTquI3eSGuaKgyc fV7dbNox8gt0t0edDT4lH9jFeNGiP7fUgZq8ic3p1lSHvakU6hu4O3/+T mp4WMWmPA30hstss28z2cse9PBLPC68JXelcHnxsMx0jrocxKIaLsE2lI uPJdYdtj0CdwKx6KVB+z3WRKIhDKqMf3m2igJutGESSyPpWXVJ3b2rNTZ w==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982066" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982066" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:24 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909611" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:23 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 03/14] cxl/mem: Cache port created by the mem dev Date: Thu, 27 Jan 2022 16:26:56 -0800 Message-Id: <20220128002707.391076-4-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since region programming sees all components in the topology as a port, it's required that endpoints are treated equally. The easiest way to go from endpoint to port is to simply cache it at creation time. Signed-off-by: Ben Widawsky --- Changes since v2: - Rebased on Dan's latest port/mem changes - Keep a reference to the port until the memdev goes away - add action to release device reference for the port --- drivers/cxl/cxlmem.h | 2 ++ drivers/cxl/mem.c | 35 ++++++++++++++++++++++++++++------- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 7ba0edb4a1ab..2b8c66616d4e 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -37,6 +37,7 @@ * @id: id number of this memdev instance. * @detach_work: active memdev lost a port in its ancestry * @component_reg_phys: register base of component registers + * @port: The port created by this device */ struct cxl_memdev { struct device dev; @@ -44,6 +45,7 @@ struct cxl_memdev { struct cxl_dev_state *cxlds; struct work_struct detach_work; int id; + struct cxl_port *port; }; static inline struct cxl_memdev *to_cxl_memdev(struct device *dev) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 27f9dd0d55b6..c36219193886 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -45,26 +45,31 @@ static int wait_for_media(struct cxl_memdev *cxlmd) return 0; } -static int create_endpoint(struct cxl_memdev *cxlmd, - struct cxl_port *parent_port) +static struct cxl_port *create_endpoint(struct cxl_memdev *cxlmd, + struct cxl_port *parent_port) { struct cxl_dev_state *cxlds = cxlmd->cxlds; struct cxl_port *endpoint; + int rc; endpoint = devm_cxl_add_port(&parent_port->dev, &cxlmd->dev, cxlds->component_reg_phys, parent_port); if (IS_ERR(endpoint)) - return PTR_ERR(endpoint); + return endpoint; dev_dbg(&cxlmd->dev, "add: %s\n", dev_name(&endpoint->dev)); if (!endpoint->dev.driver) { dev_err(&cxlmd->dev, "%s failed probe\n", dev_name(&endpoint->dev)); - return -ENXIO; + return ERR_PTR(-ENXIO); } - return cxl_endpoint_autoremove(cxlmd, endpoint); + rc = cxl_endpoint_autoremove(cxlmd, endpoint); + if (rc) + return ERR_PTR(rc); + + return endpoint; } /** @@ -127,11 +132,18 @@ __mock bool cxl_dvsec_decode_init(struct cxl_dev_state *cxlds) return do_hdm_init; } +static void delete_memdev(void *dev) +{ + struct cxl_memdev *cxlmd = dev; + + put_device(&cxlmd->port->dev); +} + static int cxl_mem_probe(struct device *dev) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; - struct cxl_port *parent_port; + struct cxl_port *parent_port, *ep_port; int rc; /* @@ -201,7 +213,16 @@ static int cxl_mem_probe(struct device *dev) goto out; } - rc = create_endpoint(cxlmd, parent_port); + ep_port = create_endpoint(cxlmd, parent_port); + if (IS_ERR(ep_port)) { + rc = PTR_ERR(ep_port); + goto out; + } + + get_device(&ep_port->dev); + cxlmd->port = ep_port; + + rc = devm_add_action_or_reset(dev, delete_memdev, cxlmd); out: cxl_device_unlock(&parent_port->dev); put_device(&parent_port->dev); From patchwork Fri Jan 28 00:26:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727585 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0E1E2CAF; Fri, 28 Jan 2022 00:27:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329645; x=1674865645; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UqDJZYM/MW5ZrhJFplO1Gfi5Eh6cWZzYP26iw4Mzfx4=; b=lWQOH2YKo9CHxsjQ6hYnAcjCSKt3YlFiylw+vJ7UVwHBL57wGTUZOmfY kiBq6Lhqx9paCe27uZR9qMU04ZFS5Lm7987Jzb4baE1t5hiGj0TtqzBeO qcDDHFhU8I51gPC4fn0x3iNxSptGYyk8ictNHazXBTI5g/Z9kAVtMlwyv w/0E+v77vPoBfjnCwaEUuzOS4l1It9NEfkd2wqDHEkF+HclvJfdgSRUjY GlIXqWB7H8hF2nCXG3bP0IPGACiCxCMfLNcv50wZeK5xsBO41cYyeBb3f OhyoGR2YgSwww/Ld25eyIfVdOPpxU514plSXIw+LtBI4Ta0BLIpR6nMWt Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982069" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982069" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:24 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909616" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:24 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , kernel test robot , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 04/14] cxl/region: Introduce a cxl_region driver Date: Thu, 27 Jan 2022 16:26:57 -0800 Message-Id: <20220128002707.391076-5-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The cxl_region driver is responsible for managing the HDM decoder programming in the CXL topology. Once a region is created it must be configured and bound to the driver in order to activate it. The following is a sample of how such controls might work: region=$(cat /sys/bus/cxl/devices/decoder0.0/create_region) echo $region > /sys/bus/cxl/devices/decoder0.0/create_region echo 2 > /sys/bus/cxl/devices/decoder0.0/region0.0:0/interleave echo $((256<<20)) > /sys/bus/cxl/devices/decoder0.0/region0.0:0/size echo mem0 > /sys/bus/cxl/devices/decoder0.0/region0.0:0/target0 echo mem1 > /sys/bus/cxl/devices/decoder0.0/region0.0:0/target1 echo region0.0:0 > /sys/bus/cxl/drivers/cxl_region/bind In order to handle the eventual rise in failure modes of binding a region, a new trace event is created to help track these failures for debug and reconfiguration paths in userspace. Reported-by: kernel test robot (v2) Signed-off-by: Ben Widawsky --- Changes since v2: - Add CONFIG_CXL_REGION - Check ways/granularity in sanitize --- .../driver-api/cxl/memory-devices.rst | 3 + drivers/cxl/Kconfig | 4 + drivers/cxl/Makefile | 2 + drivers/cxl/core/core.h | 1 + drivers/cxl/core/port.c | 17 +- drivers/cxl/core/region.c | 25 +- drivers/cxl/cxl.h | 31 ++ drivers/cxl/region.c | 349 ++++++++++++++++++ drivers/cxl/region.h | 4 + 9 files changed, 431 insertions(+), 5 deletions(-) create mode 100644 drivers/cxl/region.c diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst index 66ddc58a21b1..8cb4dece5b17 100644 --- a/Documentation/driver-api/cxl/memory-devices.rst +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -364,6 +364,9 @@ CXL Core CXL Regions ----------- +.. kernel-doc:: drivers/cxl/region.c + :doc: cxl region + .. kernel-doc:: drivers/cxl/region.h :identifiers: diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index b88ab956bb7c..742847503c16 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -98,4 +98,8 @@ config CXL_PORT default CXL_BUS tristate +config CXL_REGION + default CXL_PORT + tristate + endif diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile index ce267ef11d93..02a4776e7ab9 100644 --- a/drivers/cxl/Makefile +++ b/drivers/cxl/Makefile @@ -5,9 +5,11 @@ obj-$(CONFIG_CXL_MEM) += cxl_mem.o obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o obj-$(CONFIG_CXL_PMEM) += cxl_pmem.o obj-$(CONFIG_CXL_PORT) += cxl_port.o +obj-$(CONFIG_CXL_REGION) += cxl_region.o cxl_mem-y := mem.o cxl_pci-y := pci.o cxl_acpi-y := acpi.o cxl_pmem-y := pmem.o cxl_port-y := port.o +cxl_region-y := region.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 35fd08d560e2..b8a154da34df 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -7,6 +7,7 @@ extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; extern const struct device_type cxl_memdev_type; +extern const struct device_type cxl_region_type; extern struct attribute_group cxl_base_attribute_group; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 0826208b2bdf..0847e6ce19ef 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include "core.h" @@ -49,6 +50,8 @@ static int cxl_device_id(struct device *dev) } if (dev->type == &cxl_memdev_type) return CXL_DEVICE_MEMORY_EXPANDER; + if (dev->type == &cxl_region_type) + return CXL_DEVICE_REGION; return 0; } @@ -1425,13 +1428,23 @@ static int cxl_bus_match(struct device *dev, struct device_driver *drv) static int cxl_bus_probe(struct device *dev) { - int rc; + int id = cxl_device_id(dev); + int rc = -ENODEV; cxl_nested_lock(dev); - rc = to_cxl_drv(dev->driver)->probe(dev); + if (id == CXL_DEVICE_REGION) { + /* Regions cannot bind until parameters are set */ + struct cxl_region *cxlr = to_cxl_region(dev); + + if (is_cxl_region_configured(cxlr)) + rc = to_cxl_drv(dev->driver)->probe(dev); + } else { + rc = to_cxl_drv(dev->driver)->probe(dev); + } cxl_nested_unlock(dev); dev_dbg(dev, "probe: %d\n", rc); + return rc; } diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 3b48e0469fc7..784e4ba25128 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -12,6 +12,8 @@ #include #include "core.h" +#include "core.h" + /** * DOC: cxl core region * @@ -26,10 +28,27 @@ static const struct attribute_group region_interleave_group; static bool is_region_active(struct cxl_region *cxlr) { - /* TODO: Regions can't be activated yet. */ - return false; + return cxlr->active; } +/* + * Most sanity checking is left up to region binding. This does the most basic + * check to determine whether or not the core should try probing the driver. + */ +bool is_cxl_region_configured(const struct cxl_region *cxlr) +{ + /* zero sized regions aren't a thing. */ + if (cxlr->config.size <= 0) + return false; + + /* all regions have at least 1 target */ + if (!cxlr->config.targets[0]) + return false; + + return true; +} +EXPORT_SYMBOL_GPL(is_cxl_region_configured); + static void remove_target(struct cxl_region *cxlr, int target) { struct cxl_memdev *cxlmd; @@ -316,7 +335,7 @@ static const struct attribute_group *region_groups[] = { static void cxl_region_release(struct device *dev); -static const struct device_type cxl_region_type = { +const struct device_type cxl_region_type = { .name = "cxl_region", .release = cxl_region_release, .groups = region_groups diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index b9f0099c1f39..d1a8ca19c9ea 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -81,6 +81,31 @@ static inline int cxl_to_interleave_ways(u8 eniw) } } +static inline bool cxl_is_interleave_ways_valid(int iw) +{ + switch (iw) { + case 0 ... 4: + case 6: + case 8: + case 12: + case 16: + return true; + default: + return false; + } + + unreachable(); +} + +static inline bool cxl_is_interleave_granularity_valid(int ig) +{ + if (!is_power_of_2(ig)) + return false; + + /* 16K is the max */ + return ((ig >> 15) == 0); +} + /* CXL 2.0 8.2.8.1 Device Capabilities Array Register */ #define CXLDEV_CAP_ARRAY_OFFSET 0x0 #define CXLDEV_CAP_ARRAY_CAP_ID 0 @@ -199,6 +224,10 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, #define CXL_DECODER_F_ENABLE BIT(5) #define CXL_DECODER_F_MASK GENMASK(5, 0) +#define cxl_is_pmem_t3(flags) \ + (((flags) & (CXL_DECODER_F_TYPE3 | CXL_DECODER_F_PMEM)) == \ + (CXL_DECODER_F_TYPE3 | CXL_DECODER_F_PMEM)) + enum cxl_decoder_type { CXL_DECODER_ACCELERATOR = 2, CXL_DECODER_EXPANDER = 3, @@ -357,6 +386,7 @@ struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, resource_size_t component_reg_phys); struct cxl_dport *cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dev); +struct cxl_port *ep_find_cxl_port(struct cxl_memdev *cxlmd, unsigned int depth); struct cxl_decoder *to_cxl_decoder(struct device *dev); bool is_root_decoder(struct device *dev); @@ -404,6 +434,7 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); #define CXL_DEVICE_PORT 3 #define CXL_DEVICE_ROOT 4 #define CXL_DEVICE_MEMORY_EXPANDER 5 +#define CXL_DEVICE_REGION 6 #define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") #define CXL_MODALIAS_FMT "cxl:t%d" diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c new file mode 100644 index 000000000000..cc41939a2f0a --- /dev/null +++ b/drivers/cxl/region.c @@ -0,0 +1,349 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2021 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include "cxlmem.h" +#include "region.h" +#include "cxl.h" + +/** + * DOC: cxl region + * + * This module implements a region driver that is capable of programming CXL + * hardware to setup regions. + * + * A CXL region encompasses a chunk of host physical address space that may be + * consumed by a single device (x1 interleave aka linear) or across multiple + * devices (xN interleaved). The region driver has the following + * responsibilities: + * + * * Walk topology to obtain decoder resources for region configuration. + * * Program decoder resources based on region configuration. + * * Bridge CXL regions to LIBNVDIMM + * * Initiates reading and configuring LSA regions + * * Enumerates regions created by BIOS (typically volatile) + */ + +#define region_ways(region) ((region)->config.interleave_ways) +#define region_granularity(region) ((region)->config.interleave_granularity) + +static struct cxl_decoder *rootd_from_region(struct cxl_region *cxlr) +{ + struct device *d = cxlr->dev.parent; + + if (WARN_ONCE(!is_root_decoder(d), + "Corrupt topology for root region\n")) + return NULL; + + return to_cxl_decoder(d); +} + +static struct cxl_port *get_hostbridge(const struct cxl_memdev *ep) +{ + struct cxl_port *port = ep->port; + + while (!is_cxl_root(port)) { + port = to_cxl_port(port->dev.parent); + if (port->depth == 1) + return port; + } + + BUG(); + return NULL; +} + +static struct cxl_port *get_root_decoder(const struct cxl_memdev *endpoint) +{ + struct cxl_port *hostbridge = get_hostbridge(endpoint); + + if (hostbridge) + return to_cxl_port(hostbridge->dev.parent); + + return NULL; +} + +/** + * sanitize_region() - Check is region is reasonably configured + * @cxlr: The region to check + * + * Determination as to whether or not a region can possibly be configured is + * described in CXL Memory Device SW Guide. In order to implement the algorithms + * described there, certain more basic configuration parameters must first need + * to be validated. That is accomplished by this function. + * + * Returns 0 if the region is reasonably configured, else returns a negative + * error code. + */ +static int sanitize_region(const struct cxl_region *cxlr) +{ + const int ig = region_granularity(cxlr); + const int iw = region_ways(cxlr); + int i; + + if (dev_WARN_ONCE(&cxlr->dev, !is_cxl_region_configured(cxlr), + "unconfigured regions can't be probed (race?)\n")) { + return -ENXIO; + } + + /* + * Interleave attributes should be caught by later math, but it's + * easiest to find those issues here, now. + */ + if (!cxl_is_interleave_ways_valid(iw)) { + dev_dbg(&cxlr->dev, "Invalid number of ways\n"); + return -ENXIO; + } + + if (!cxl_is_interleave_granularity_valid(ig)) { + dev_dbg(&cxlr->dev, "Invalid interleave granularity\n"); + return -ENXIO; + } + + if (cxlr->config.size % (SZ_256M * iw)) { + dev_dbg(&cxlr->dev, "Invalid size. Must be multiple of %uM\n", + 256 * iw); + return -ENXIO; + } + + for (i = 0; i < iw; i++) { + if (!cxlr->config.targets[i]) { + dev_dbg(&cxlr->dev, "Missing memory device target%u", + i); + return -ENXIO; + } + if (!cxlr->config.targets[i]->dev.driver) { + dev_dbg(&cxlr->dev, "%s isn't CXL.mem capable\n", + dev_name(&cxlr->config.targets[i]->dev)); + return -ENODEV; + } + } + + return 0; +} + +/** + * allocate_address_space() - Gets address space for the region. + * @cxlr: The region that will consume the address space + */ +static int allocate_address_space(struct cxl_region *cxlr) +{ + /* TODO */ + return 0; +} + +/** + * find_cdat_dsmas() - Find a valid DSMAS for the region + * @cxlr: The region + */ +static bool find_cdat_dsmas(const struct cxl_region *cxlr) +{ + return true; +} + +/** + * qtg_match() - Does this root decoder have desirable QTG for the endpoint + * @rootd: The root decoder for the region + * @endpoint: Endpoint whose QTG is being compared + * + * Prior to calling this function, the caller should verify that all endpoints + * in the region have the same QTG ID. + * + * Returns true if the QTG ID of the root decoder matches the endpoint + */ +static bool qtg_match(const struct cxl_decoder *rootd, + const struct cxl_memdev *endpoint) +{ + /* TODO: */ + return true; +} + +/** + * region_xhb_config_valid() - determine cross host bridge validity + * @cxlr: The region being programmed + * @rootd: The root decoder to check against + * + * The algorithm is outlined in 2.13.14 "Verify XHB configuration sequence" of + * the CXL Memory Device SW Guide (Rev1p0). + * + * Returns true if the configuration is valid. + */ +static bool region_xhb_config_valid(const struct cxl_region *cxlr, + const struct cxl_decoder *rootd) +{ + /* TODO: */ + return true; +} + +/** + * region_hb_rp_config_valid() - determine root port ordering is correct + * @cxlr: Region to validate + * @rootd: root decoder for this @cxlr + * + * The algorithm is outlined in 2.13.15 "Verify HB root port configuration + * sequence" of the CXL Memory Device SW Guide (Rev1p0). + * + * Returns true if the configuration is valid. + */ +static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, + const struct cxl_decoder *rootd) +{ + /* TODO: */ + return true; +} + +/** + * rootd_contains() - determine if this region can exist in the root decoder + * @rootd: root decoder that potentially decodes to this region + * @cxlr: region to be routed by the @rootd + */ +static bool rootd_contains(const struct cxl_region *cxlr, + const struct cxl_decoder *rootd) +{ + /* TODO: */ + return true; +} + +static bool rootd_valid(const struct cxl_region *cxlr, + const struct cxl_decoder *rootd) +{ + const struct cxl_memdev *endpoint = cxlr->config.targets[0]; + + if (!qtg_match(rootd, endpoint)) + return false; + + if (!cxl_is_pmem_t3(rootd->flags)) + return false; + + if (!region_xhb_config_valid(cxlr, rootd)) + return false; + + if (!region_hb_rp_config_valid(cxlr, rootd)) + return false; + + if (!rootd_contains(cxlr, rootd)) + return false; + + return true; +} + +struct rootd_context { + const struct cxl_region *cxlr; + struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; + int count; +}; + +static int rootd_match(struct device *dev, void *data) +{ + struct rootd_context *ctx = (struct rootd_context *)data; + const struct cxl_region *cxlr = ctx->cxlr; + + if (!is_root_decoder(dev)) + return 0; + + return !!rootd_valid(cxlr, to_cxl_decoder(dev)); +} + +/* + * This is a roughly equivalent implementation to "Figure 45 - High-level + * sequence: Finding CFMWS for region" from the CXL Memory Device SW Guide + * Rev1p0. + */ +static struct cxl_decoder *find_rootd(const struct cxl_region *cxlr, + const struct cxl_port *root) +{ + struct rootd_context ctx; + struct device *ret; + + ctx.cxlr = cxlr; + + ret = device_find_child((struct device *)&root->dev, &ctx, rootd_match); + if (ret) + return to_cxl_decoder(ret); + + return NULL; +} + +static int collect_ep_decoders(const struct cxl_region *cxlr) +{ + /* TODO: */ + return 0; +} + +static int bind_region(const struct cxl_region *cxlr) +{ + /* TODO: */ + return 0; +} + +static int cxl_region_probe(struct device *dev) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + struct cxl_port *root_port; + struct cxl_decoder *rootd, *ours; + int ret; + + device_lock_assert(&cxlr->dev); + + if (cxlr->active) + return 0; + + if (uuid_is_null(&cxlr->config.uuid)) + uuid_gen(&cxlr->config.uuid); + + /* TODO: What about volatile, and LSA generated regions? */ + + ret = sanitize_region(cxlr); + if (ret) + return ret; + + ret = allocate_address_space(cxlr); + if (ret) + return ret; + + if (!find_cdat_dsmas(cxlr)) + return -ENXIO; + + rootd = rootd_from_region(cxlr); + if (!rootd) { + dev_err(dev, "Couldn't find root decoder\n"); + return -ENXIO; + } + + if (!rootd_valid(cxlr, rootd)) { + dev_err(dev, "Picked invalid rootd\n"); + return -ENXIO; + } + + root_port = get_root_decoder(cxlr->config.targets[0]); + ours = find_rootd(cxlr, root_port); + if (ours != rootd) + dev_dbg(dev, "Picked different rootd %s %s\n", + dev_name(&rootd->dev), dev_name(&ours->dev)); + if (ours) + put_device(&ours->dev); + + ret = collect_ep_decoders(cxlr); + if (ret) + return ret; + + ret = bind_region(cxlr); + if (!ret) { + cxlr->active = true; + dev_info(dev, "Bound"); + } + + return ret; +} + +static struct cxl_driver cxl_region_driver = { + .name = "cxl_region", + .probe = cxl_region_probe, + .id = CXL_DEVICE_REGION, +}; +module_cxl_driver(cxl_region_driver); + +MODULE_LICENSE("GPL v2"); +MODULE_IMPORT_NS(CXL); +MODULE_ALIAS_CXL(CXL_DEVICE_REGION); diff --git a/drivers/cxl/region.h b/drivers/cxl/region.h index eb1249e3c1d4..00a6dc729c26 100644 --- a/drivers/cxl/region.h +++ b/drivers/cxl/region.h @@ -13,6 +13,7 @@ * @id: This regions id. Id is globally unique across all regions. * @list: Node in decoder's region list. * @res: Resource this region carves out of the platform decode range. + * @active: If the region has been activated. * @config: HDM decoder program config * @config.size: Size of the region determined from LSA or userspace. * @config.uuid: The UUID for this region. @@ -25,6 +26,7 @@ struct cxl_region { int id; struct list_head list; struct resource *res; + bool active; struct { u64 size; @@ -35,4 +37,6 @@ struct cxl_region { } config; }; +bool is_cxl_region_configured(const struct cxl_region *cxlr); + #endif From patchwork Fri Jan 28 00:26:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727584 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB2052CB4; Fri, 28 Jan 2022 00:27:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329646; x=1674865646; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eZfTd1SDHxpeSkQFf4Hi1IY90tP7t53lECHImTN5KkA=; b=ieqjGwP3rvbDSwAX01sd9JS9Y4RUgVPz9rprFxB43yn1J69DLyj58bxA KP87lTCFqwNiaQ7+9pwUyETysnvDOA0xpyDcGyUQ3PCPLrRixXK7L5NWt gtyI9HA8Qe1ufr6c4n36COcSFqwJlxXP9U58RSHqoL8ckhir/FDE9tpdI O7A4Bef9hi5IUOe9qL4oQOXS6jkKkVNT0KfiGPhB8F530LV3QYyPHcbWX Tlh8cMHDueQ1Lwm/CFzK7dpfRvhmXHJsJ1MJMABKrqtS55r6uawPkg9PM D18NePsb3KehOUJd1mAzcbFvd1k5xo2bdJpL78oU2xI/xBduFVfYojJ43 A==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982071" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982071" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:25 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909623" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:24 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 05/14] cxl/acpi: Handle address space allocation Date: Thu, 27 Jan 2022 16:26:58 -0800 Message-Id: <20220128002707.391076-6-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Regions are carved out of an addresses space which is claimed by top level decoders, and subsequently their children decoders. Regions are created with a size and therefore must fit, with proper alignment, in that address space. The support for doing this fitting is handled by the driver automatically. As an example, a platform might configure a top level decoder to claim 1TB of address space @ 0x800000000 -> 0x10800000000; it would be possible to create M regions with appropriate alignment to occupy that address space. Each of those regions would have a host physical address somewhere in the range between 32G and 1.3TB, and the location will be determined by the logic added here. The request_region() usage is not strictly mandatory at this point as the actual handling of the address space is done with genpools. It is highly likely however that the resource/region APIs will become useful in the not too distant future. All decoders manage a host physical address space while active. Only the root decoder has constraints on location and size. As a result, it makes most sense for the root decoder to be responsible for managing the entire address space, and mid-level decoders and endpoints can ask the root decoder for suballocations. Signed-off-by: Ben Widawsky --- drivers/cxl/acpi.c | 30 ++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 2 ++ 2 files changed, 32 insertions(+) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index d6dcb2b6af48..74681bfbf53c 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2021 Intel Corporation. All rights reserved. */ #include +#include #include #include #include @@ -73,6 +74,27 @@ static int cxl_acpi_cfmws_verify(struct device *dev, return 0; } +/* + * Every decoder while active has an address space that it is decoding. However, + * only the root level decoders have fixed host physical address space ranges. + */ +static int cxl_create_cfmws_address_space(struct cxl_decoder *cxld, + struct acpi_cedt_cfmws *cfmws) +{ + const int order = ilog2(SZ_256M * cxld->interleave_ways); + struct device *dev = &cxld->dev; + struct gen_pool *pool; + + pool = devm_gen_pool_create(dev, order, NUMA_NO_NODE, dev_name(dev)); + if (IS_ERR(pool)) + return PTR_ERR(pool); + + cxld->address_space = pool; + + return gen_pool_add(cxld->address_space, cfmws->base_hpa, + cfmws->window_size, NUMA_NO_NODE); +} + struct cxl_cfmws_context { struct device *dev; struct cxl_port *root_port; @@ -113,6 +135,14 @@ static int cxl_parse_cfmws(union acpi_subtable_headers *header, void *arg, cxld->interleave_ways = CFMWS_INTERLEAVE_WAYS(cfmws); cxld->interleave_granularity = CFMWS_INTERLEAVE_GRANULARITY(cfmws); + rc = cxl_create_cfmws_address_space(cxld, cfmws); + if (rc) { + dev_err(dev, + "Failed to create CFMWS address space for decoder\n"); + put_device(&cxld->dev); + return 0; + } + rc = cxl_decoder_add(cxld, target_map); if (rc) put_device(&cxld->dev); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index d1a8ca19c9ea..b300673072f5 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -251,6 +251,7 @@ enum cxl_decoder_type { * @flags: memory type capabilities and locking * @target_lock: coordinate coherent reads of the target list * @region_ida: allocator for region ids. + * @address_space: Used/free address space for regions. * @nr_targets: number of elements in @target * @target: active ordered target list in current decoder configuration */ @@ -267,6 +268,7 @@ struct cxl_decoder { unsigned long flags; seqlock_t target_lock; struct ida region_ida; + struct gen_pool *address_space; int nr_targets; struct cxl_dport *target[]; }; From patchwork Fri Jan 28 00:26:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727586 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5E963FD5; Fri, 28 Jan 2022 00:27:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329646; x=1674865646; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JRLB7CUs7371NAYrkD3KPcwzK+rjDWnGSvurqfC0vT4=; b=VEaI+scpWLeso+qQzl1gxDNncptXDctbUD2bfAKHrnJzTJza6eMjT5Jq pRR4rrskNNbQL6pANtg82Egcsxy0401LCCZnG5usps+gpHjgwlE1PIjdU LkfqNSbSrEiNftFnDBbzNrazY0uboGGI8s2YQTD1ZF5qeaaWKsMoVUulK m7boVO3c9bclTC6pMDF4N5UFFwBKaSfiYRT6fYCrEx5AEN2QUEhsG49z7 M6qpkHw3wED33vUnJQqtW2nNqPYjHXsA2KxzSkzRIw2S7JjlCCjmLPdVm hDHlxCt4HvddQ95n7biNbA8my7z+hXAggBcn2Irt4MbQDiGDGhtnWLMF6 Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982073" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982073" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:26 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909627" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:25 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 06/14] cxl/region: Address space allocation Date: Thu, 27 Jan 2022 16:26:59 -0800 Message-Id: <20220128002707.391076-7-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When a region is not assigned a host physical address, one is picked by the driver. As the address will determine which CFMWS contains the region, it's usually a better idea to let the driver make this determination. Signed-off-by: Ben Widawsky --- drivers/cxl/region.c | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index cc41939a2f0a..5588873dd250 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2021 Intel Corporation. All rights reserved. */ #include +#include #include #include #include @@ -64,6 +65,20 @@ static struct cxl_port *get_root_decoder(const struct cxl_memdev *endpoint) return NULL; } +static void release_cxl_region(void *r) +{ + struct cxl_region *cxlr = (struct cxl_region *)r; + struct cxl_decoder *rootd = rootd_from_region(cxlr); + struct resource *res = &rootd->platform_res; + resource_size_t start, size; + + start = cxlr->res->start; + size = resource_size(cxlr->res); + + __release_region(res, start, size); + gen_pool_free(rootd->address_space, start, size); +} + /** * sanitize_region() - Check is region is reasonably configured * @cxlr: The region to check @@ -129,8 +144,29 @@ static int sanitize_region(const struct cxl_region *cxlr) */ static int allocate_address_space(struct cxl_region *cxlr) { - /* TODO */ - return 0; + struct cxl_decoder *rootd = rootd_from_region(cxlr); + unsigned long start; + + start = gen_pool_alloc(rootd->address_space, cxlr->config.size); + if (!start) { + dev_dbg(&cxlr->dev, "Couldn't allocate %lluM of address space", + cxlr->config.size >> 20); + return -ENOMEM; + } + + cxlr->res = + __request_region(&rootd->platform_res, start, cxlr->config.size, + dev_name(&cxlr->dev), IORESOURCE_MEM); + if (!cxlr->res) { + dev_dbg(&cxlr->dev, "Couldn't obtain region from %s (%pR)\n", + dev_name(&rootd->dev), &rootd->platform_res); + gen_pool_free(rootd->address_space, start, cxlr->config.size); + return -ENOMEM; + } + + dev_dbg(&cxlr->dev, "resource %pR", cxlr->res); + + return devm_add_action_or_reset(&cxlr->dev, release_cxl_region, cxlr); } /** From patchwork Fri Jan 28 00:27:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727587 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED8003FE0; Fri, 28 Jan 2022 00:27:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329647; x=1674865647; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/qLuADfUOCfFVCPn9Z+LbWyL/p2QgPjE/xoSdfj8Kj0=; b=i0e74yBsqKZNdZVjOsSemT1czPwpm3zCP+QYSL8dbvvhY7RN1E5tZnjE YvVvKp9/TIAU4IePOk81eZtEQIpLCKri8puYZ++86xt6RABXJmwDuEH6o zYiXXWycP0cE7S8YFbJVgHHUTosTTRR9B6VHUNaVfJNIXSdsg7nOb+n6J zaPoc2kS62+iq5Hd1eycC0Fi+AaO31FzYYEV3831Ce8OVK22L3D8/Dq0t cxH63rwF+VJJKvUhO6J2Z3cyUK5+u7bcE0DAesQicV211mcyPccWOO8zN 4DXuGznJ1YYZFTu7v5RUU7G6VIJvJKQ82msrRLZnnAyQr0LWp81Yis3aN g==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982075" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982075" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:26 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909631" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:26 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 07/14] cxl/region: Implement XHB verification Date: Thu, 27 Jan 2022 16:27:00 -0800 Message-Id: <20220128002707.391076-8-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Cross host bridge verification primarily determines if the requested interleave ordering can be achieved by the root decoder, which isn't as programmable as other decoders. The algorithm implemented here is based on the CXL Type 3 Memory Device Software Guide, chapter 2.13.14 Signed-off-by: Ben Widawsky --- Changes since v2: - Fail earlier on lack of host bridges. This should only be capable as of now with cxl_test memdevs. --- .clang-format | 2 + drivers/cxl/cxl.h | 13 +++++++ drivers/cxl/region.c | 89 +++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 103 insertions(+), 1 deletion(-) diff --git a/.clang-format b/.clang-format index fa959436bcfd..1221d53be90b 100644 --- a/.clang-format +++ b/.clang-format @@ -169,6 +169,8 @@ ForEachMacros: - 'for_each_cpu_and' - 'for_each_cpu_not' - 'for_each_cpu_wrap' + - 'for_each_cxl_decoder_target' + - 'for_each_cxl_endpoint' - 'for_each_dapm_widgets' - 'for_each_dev_addr' - 'for_each_dev_scope' diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index b300673072f5..a291999431c7 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -81,6 +81,19 @@ static inline int cxl_to_interleave_ways(u8 eniw) } } +static inline u8 cxl_to_eniw(u8 ways) +{ + if (is_power_of_2(ways)) + return ilog2(ways); + + return ways / 3 + 8; +} + +static inline u8 cxl_to_ig(u16 g) +{ + return ilog2(g) - 8; +} + static inline bool cxl_is_interleave_ways_valid(int iw) { switch (iw) { diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index 5588873dd250..562c8720da56 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -29,6 +29,17 @@ #define region_ways(region) ((region)->config.interleave_ways) #define region_granularity(region) ((region)->config.interleave_granularity) +#define region_eniw(region) (cxl_to_eniw(region_ways(region))) +#define region_ig(region) (cxl_to_ig(region_granularity(region))) + +#define for_each_cxl_endpoint(ep, region, idx) \ + for (idx = 0, ep = (region)->config.targets[idx]; \ + idx < region_ways(region); ep = (region)->config.targets[++idx]) + +#define for_each_cxl_decoder_target(dport, decoder, idx) \ + for (idx = 0, dport = (decoder)->target[idx]; \ + idx < (decoder)->nr_targets - 1; \ + dport = (decoder)->target[++idx]) static struct cxl_decoder *rootd_from_region(struct cxl_region *cxlr) { @@ -195,6 +206,30 @@ static bool qtg_match(const struct cxl_decoder *rootd, return true; } +static int get_unique_hostbridges(const struct cxl_region *cxlr, + struct cxl_port **hbs) +{ + struct cxl_memdev *ep; + int i, hb_count = 0; + + for_each_cxl_endpoint(ep, cxlr, i) { + struct cxl_port *hb = get_hostbridge(ep); + bool found = false; + int j; + + BUG_ON(!hb); + + for (j = 0; j < hb_count; j++) { + if (hbs[j] == hb) + found = true; + } + if (!found) + hbs[hb_count++] = hb; + } + + return hb_count; +} + /** * region_xhb_config_valid() - determine cross host bridge validity * @cxlr: The region being programmed @@ -208,7 +243,59 @@ static bool qtg_match(const struct cxl_decoder *rootd, static bool region_xhb_config_valid(const struct cxl_region *cxlr, const struct cxl_decoder *rootd) { - /* TODO: */ + const int rootd_eniw = cxl_to_eniw(rootd->interleave_ways); + const int rootd_ig = cxl_to_ig(rootd->interleave_granularity); + const int cxlr_ig = region_ig(cxlr); + const int cxlr_iw = region_ways(cxlr); + struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; + struct cxl_dport *target; + int i; + + i = get_unique_hostbridges(cxlr, hbs); + if (dev_WARN_ONCE(&cxlr->dev, i == 0, "Cannot find a valid host bridge\n")) + return false; + + /* Are all devices in this region on the same CXL host bridge */ + if (i == 1) + return true; + + /* CFMWS.HBIG >= Device.Label.IG */ + if (rootd_ig < cxlr_ig) { + dev_dbg(&cxlr->dev, + "%s HBIG must be greater than region IG (%d < %d)\n", + dev_name(&rootd->dev), rootd_ig, cxlr_ig); + return false; + } + + /* + * ((2^(CFMWS.HBIG - Device.RLabel.IG) * (2^CFMWS.ENIW)) > Device.RLabel.NLabel) + * + * XXX: 2^CFMWS.ENIW is trying to decode the NIW. Instead, use the look + * up function which supports non power of 2 interleave configurations. + */ + if (((1 << (rootd_ig - cxlr_ig)) * (1 << rootd_eniw)) > cxlr_iw) { + dev_dbg(&cxlr->dev, + "granularity ratio requires a larger number of devices (%d) than currently configured (%d)\n", + ((1 << (rootd_ig - cxlr_ig)) * (1 << rootd_eniw)), + cxlr_iw); + return false; + } + + /* + * CFMWS.InterleaveTargetList[n] must contain all devices, x where: + * (Device[x],RegionLabel.Position >> (CFMWS.HBIG - + * Device[x].RegionLabel.InterleaveGranularity)) & + * ((2^CFMWS.ENIW) - 1) = n + */ + for_each_cxl_decoder_target(target, rootd, i) { + if (((i >> (rootd_ig - cxlr_ig))) & + (((1 << rootd_eniw) - 1) != target->port_id)) { + dev_dbg(&cxlr->dev, + "One or more devices are not connected to the correct hostbridge.\n"); + return false; + } + } + return true; } From patchwork Fri Jan 28 00:27:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727588 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AB9E3FE5; Fri, 28 Jan 2022 00:27:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329648; x=1674865648; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3Kaes42dLpbACpMEACjZ5MpAsHsOTY/I4nfdu15eLrk=; b=OaMAW6V7U2r/CHd7039E4pErdnZ/jjFLWY9KoAeovgVtAlqpS8SZ8aP4 lWtF2lcDt1nJmLCVSxcMFBoKg5guSypgPbI2EgC3eT63nCf/V/lZ89LrK zrXbCdtKNkqdX3AZxUtBto4B/2tl4MuZEUoBUWrWyesTsYcGygNSMMPyt 6lBTdmDv6QV5W/9GknfVGawvOXICZvwQUBopJGTHZfKNhEXClzQJVcTEA c8NBDxfR93QACw7SKarHIP9xJB3c8mxs8+AyXNt2wePdBNBPWP1FgmP+b tnIOEh1lM73P9O5q52CBlByyb6xi0tVnGCowpFVM3P40zyMbYahfqQpTJ Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982078" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982078" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:27 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909636" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:26 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 08/14] cxl/region: HB port config verification Date: Thu, 27 Jan 2022 16:27:01 -0800 Message-Id: <20220128002707.391076-9-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Host bridge root port verification determines if the device ordering in an interleave set can be programmed through the host bridges and switches. The algorithm implemented here is based on the CXL Type 3 Memory Device Software Guide, chapter 2.13.15. The current version of the guide does not yet support x3 interleave configurations, and so that's not supported here either. Signed-off-by: Ben Widawsky --- .clang-format | 1 + drivers/cxl/core/port.c | 1 + drivers/cxl/cxl.h | 2 + drivers/cxl/region.c | 127 +++++++++++++++++++++++++++++++++++++++- 4 files changed, 130 insertions(+), 1 deletion(-) diff --git a/.clang-format b/.clang-format index 1221d53be90b..5e20206f905e 100644 --- a/.clang-format +++ b/.clang-format @@ -171,6 +171,7 @@ ForEachMacros: - 'for_each_cpu_wrap' - 'for_each_cxl_decoder_target' - 'for_each_cxl_endpoint' + - 'for_each_cxl_endpoint_hb' - 'for_each_dapm_widgets' - 'for_each_dev_addr' - 'for_each_dev_scope' diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 0847e6ce19ef..1d81c5f56a3e 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -706,6 +706,7 @@ struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, return ERR_PTR(-ENOMEM); INIT_LIST_HEAD(&dport->list); + INIT_LIST_HEAD(&dport->verify_link); dport->dport = dport_dev; dport->port_id = port_id; dport->component_reg_phys = component_reg_phys; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index a291999431c7..ed984465b59c 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -350,6 +350,7 @@ struct cxl_port { * @component_reg_phys: downstream port component registers * @port: reference to cxl_port that contains this downstream port * @list: node for a cxl_port's list of cxl_dport instances + * @verify_link: node used for hb root port verification */ struct cxl_dport { struct device *dport; @@ -357,6 +358,7 @@ struct cxl_dport { resource_size_t component_reg_phys; struct cxl_port *port; struct list_head list; + struct list_head verify_link; }; /** diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index 562c8720da56..d2f6c990c8a8 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include "cxlmem.h" #include "region.h" @@ -36,6 +37,12 @@ for (idx = 0, ep = (region)->config.targets[idx]; \ idx < region_ways(region); ep = (region)->config.targets[++idx]) +#define for_each_cxl_endpoint_hb(ep, region, hb, idx) \ + for (idx = 0, (ep) = (region)->config.targets[idx]; \ + idx < region_ways(region); \ + idx++, (ep) = (region)->config.targets[idx]) \ + if (get_hostbridge(ep) == (hb)) + #define for_each_cxl_decoder_target(dport, decoder, idx) \ for (idx = 0, dport = (decoder)->target[idx]; \ idx < (decoder)->nr_targets - 1; \ @@ -299,6 +306,59 @@ static bool region_xhb_config_valid(const struct cxl_region *cxlr, return true; } +static struct cxl_dport *get_rp(struct cxl_memdev *ep) +{ + struct cxl_port *port, *parent_port = port = ep->port; + struct cxl_dport *dport; + + while (!is_cxl_root(port)) { + parent_port = to_cxl_port(port->dev.parent); + if (parent_port->depth == 1) + list_for_each_entry(dport, &parent_port->dports, list) + if (dport->dport == port->uport->parent->parent) + return dport; + port = parent_port; + } + + BUG(); + return NULL; +} + +static int get_num_root_ports(const struct cxl_region *cxlr) +{ + struct cxl_memdev *endpoint; + struct cxl_dport *dport, *tmp; + int num_root_ports = 0; + LIST_HEAD(root_ports); + int idx; + + for_each_cxl_endpoint(endpoint, cxlr, idx) { + struct cxl_dport *root_port = get_rp(endpoint); + + if (list_empty(&root_port->verify_link)) { + list_add_tail(&root_port->verify_link, &root_ports); + num_root_ports++; + } + } + + list_for_each_entry_safe(dport, tmp, &root_ports, verify_link) + list_del_init(&dport->verify_link); + + return num_root_ports; +} + +static bool has_switch(const struct cxl_region *cxlr) +{ + struct cxl_memdev *ep; + int i; + + for_each_cxl_endpoint(ep, cxlr, i) + if (ep->port->depth > 2) + return true; + + return false; +} + /** * region_hb_rp_config_valid() - determine root port ordering is correct * @cxlr: Region to validate @@ -312,7 +372,72 @@ static bool region_xhb_config_valid(const struct cxl_region *cxlr, static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, const struct cxl_decoder *rootd) { - /* TODO: */ + const int num_root_ports = get_num_root_ports(cxlr); + struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; + int hb_count, i; + + hb_count = get_unique_hostbridges(cxlr, hbs); + + /* TODO: Switch support */ + if (has_switch(cxlr)) + return false; + + /* + * Are all devices in this region on the same CXL Host Bridge + * Root Port? + */ + if (num_root_ports == 1 && !has_switch(cxlr)) + return true; + + for (i = 0; i < hb_count; i++) { + int idx, position_mask; + struct cxl_dport *rp; + struct cxl_port *hb; + + /* Get next CXL Host Bridge this region spans */ + hb = hbs[i]; + + /* + * Calculate the position mask: NumRootPorts = 2^PositionMask + * for this region. + * + * XXX: pos_mask is actually (1 << PositionMask) - 1 + */ + position_mask = (1 << (ilog2(num_root_ports))) - 1; + + /* + * Calculate the PortGrouping for each device on this CXL Host + * Bridge Root Port: + * PortGrouping = RegionLabel.Position & PositionMask + * + * The following nest iterators effectively iterate over each + * root port in the region. + * for_each_unique_rootport(rp, cxlr) + */ + list_for_each_entry(rp, &hb->dports, list) { + struct cxl_memdev *ep; + int port_grouping = -1; + + for_each_cxl_endpoint_hb(ep, cxlr, hb, idx) { + if (get_rp(ep) != rp) + continue; + + if (port_grouping == -1) + port_grouping = idx & position_mask; + + /* + * Do all devices in the region connected to this CXL + * Host Bridge Root Port have the same PortGrouping? + */ + if ((idx & position_mask) != port_grouping) { + dev_dbg(&cxlr->dev, + "One or more devices are not connected to the correct Host Bridge Root Port\n"); + return false; + } + } + } + } + return true; } From patchwork Fri Jan 28 00:27:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727589 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B6823FEC; Fri, 28 Jan 2022 00:27:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329648; x=1674865648; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FLeYXW6NWFrswyPwmJIx1vkrYrqwZ/21bKcdDaAYGsU=; b=DqGeWWqwAHoXw+Lc/A8J3WxXUYW9nZmRRZVAL9H8zJ9WMG/wWPeitZZI VqBDgpokCMy45BuuulQ5akFHLpjky9TRfucfxbBYwTVuIl/ExEEXZL/IS 6m6fTw8EG21qD1iJTG1ZaEc8qEr/XzIPtJhNwDNFIMXx6vEMi1+Cj5gob rxK2VV37wgzeRay7UYvf6jFP++tHFSPSNBs0rhxgz7uJPie0nHldZ8roF zKwANmj+0u257ZpEHZqPKmJCrXCE2xPnRJQ09MSp5d6uTsULg3rmOmvWH EpMvZTC3BiAeR0YpGaQZsUlk4OyYIivL2SdPKxUohvE10fm4TUg8NMxUR A==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982079" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982079" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:27 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909645" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:27 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 09/14] cxl/region: Add infrastructure for decoder programming Date: Thu, 27 Jan 2022 16:27:02 -0800 Message-Id: <20220128002707.391076-10-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There are 3 steps in handling region programming once it has been configured by userspace. 1. Sanitize the parameters against the system. 2. Collect decoder resources from the topology 3. Program decoder resources The infrastructure added here addresses #2. Two new APIs are introduced to allow collecting and returning decoder resources. Additionally the infrastructure includes two lists managed by the region driver, a staged list, and a commit list. The staged list contains those collected in step #2, and the commit list are all the decoders programmed in step #3. Signed-off-by: Ben Widawsky --- drivers/cxl/core/port.c | 71 ++++++++++++++++++++++ drivers/cxl/core/region.c | 2 + drivers/cxl/cxl.h | 8 +++ drivers/cxl/cxlmem.h | 7 +++ drivers/cxl/port.c | 62 ++++++++++++++++++- drivers/cxl/region.c | 125 +++++++++++++++++++++++++++++++++----- drivers/cxl/region.h | 5 ++ 7 files changed, 263 insertions(+), 17 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1d81c5f56a3e..92aaaa65ec61 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1212,6 +1212,8 @@ static struct cxl_decoder *cxl_decoder_alloc(struct cxl_port *port, cxld->target_type = CXL_DECODER_EXPANDER; cxld->platform_res = (struct resource)DEFINE_RES_MEM(0, 0); + INIT_LIST_HEAD(&cxld->region_link); + ida_init(&cxld->region_ida); return cxld; @@ -1366,6 +1368,75 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) } EXPORT_SYMBOL_NS_GPL(cxl_decoder_add, CXL); +/** + * cxl_get_decoder() - Get an unused decoder from the port. + * @port: The port to obtain a decoder from. + * + * Region programming requires obtaining decoder resources from all ports that + * participate in the interleave set. This function shall be used to pull the + * decoder resource out of the list of available. + * + * Context: Process context. Takes and releases the device lock of the port. + * + * Return: A cxl_decoder that can be used for programming if successful, else a + * negative error code. + */ +struct cxl_decoder *cxl_get_decoder(struct cxl_port *port) +{ + struct cxl_hdm *cxlhdm; + int dec; + + cxlhdm = dev_get_drvdata(&port->dev); + if (dev_WARN_ONCE(&port->dev, !cxlhdm, "No port drvdata\n")) + return ERR_PTR(-ENXIO); + + device_lock(&port->dev); + dec = find_first_bit(cxlhdm->decoders.free_mask, + cxlhdm->decoders.count); + if (dec == cxlhdm->decoders.count) { + device_unlock(&port->dev); + return ERR_PTR(-ENODEV); + } + + clear_bit(dec, cxlhdm->decoders.free_mask); + device_unlock(&port->dev); + + return cxlhdm->decoders.cxld[dec]; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_decoder, CXL); + +/** + * cxl_put_decoder() - Return an inactive decoder to the port. + * @cxld: The decoder being returned. + */ +void cxl_put_decoder(struct cxl_decoder *cxld) +{ + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + struct cxl_hdm *cxlhdm; + int i; + + cxlhdm = dev_get_drvdata(&port->dev); + if (dev_WARN_ONCE(&port->dev, !cxlhdm, "No port drvdata\n")) + return; + + device_lock(&port->dev); + + for (i = 0; i < CXL_DECODER_MAX_INSTANCES; i++) { + struct cxl_decoder *d = cxlhdm->decoders.cxld[i]; + + if (!d) + continue; + + if (d == cxld) { + set_bit(i, cxlhdm->decoders.free_mask); + break; + } + } + + device_unlock(&port->dev); +} +EXPORT_SYMBOL_NS_GPL(cxl_put_decoder, CXL); + static void cxld_unregister(void *dev) { device_unregister(dev); diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 784e4ba25128..a62d48454a56 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -440,6 +440,8 @@ struct cxl_region *cxl_alloc_region(struct cxl_decoder *cxld, int id) if (!cxlr) return ERR_PTR(-ENOMEM); + INIT_LIST_HEAD(&cxlr->staged_list); + INIT_LIST_HEAD(&cxlr->commit_list); cxlr->id = id; return cxlr; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ed984465b59c..8ace6cca0776 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -35,6 +35,8 @@ #define CXL_CM_CAP_CAP_ID_HDM 0x5 #define CXL_CM_CAP_CAP_HDM_VERSION 1 +#define CXL_DECODER_MAX_INSTANCES 10 + /* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */ #define CXL_HDM_DECODER_CAP_OFFSET 0x0 #define CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0) @@ -265,6 +267,7 @@ enum cxl_decoder_type { * @target_lock: coordinate coherent reads of the target list * @region_ida: allocator for region ids. * @address_space: Used/free address space for regions. + * @region_link: This decoder's place on either the staged, or commit list. * @nr_targets: number of elements in @target * @target: active ordered target list in current decoder configuration */ @@ -282,6 +285,7 @@ struct cxl_decoder { seqlock_t target_lock; struct ida region_ida; struct gen_pool *address_space; + struct list_head region_link; int nr_targets; struct cxl_dport *target[]; }; @@ -326,6 +330,7 @@ struct cxl_nvdimm { * @id: id for port device-name * @dports: cxl_dport instances referenced by decoders * @endpoints: cxl_ep instances, endpoints that are a descendant of this port + * @region_link: this port's node on the region's list of ports * @decoder_ida: allocator for decoder ids * @component_reg_phys: component register capability base address (optional) * @dead: last ep has been removed, force port re-creation @@ -396,6 +401,8 @@ struct cxl_port *find_cxl_root(struct device *dev); int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd); int cxl_bus_rescan(void); struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd); +struct cxl_decoder *cxl_get_decoder(struct cxl_port *port); +void cxl_put_decoder(struct cxl_decoder *cxld); bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd); struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, @@ -406,6 +413,7 @@ struct cxl_dport *cxl_find_dport_by_dev(struct cxl_port *port, struct cxl_port *ep_find_cxl_port(struct cxl_memdev *cxlmd, unsigned int depth); struct cxl_decoder *to_cxl_decoder(struct device *dev); +bool is_cxl_decoder(struct device *dev); bool is_root_decoder(struct device *dev); bool is_cxl_decoder(struct device *dev); struct cxl_decoder *cxl_root_decoder_alloc(struct cxl_port *port, diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 2b8c66616d4e..6db66eaf51be 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -305,5 +305,12 @@ struct cxl_hdm { unsigned int target_count; unsigned int interleave_mask; struct cxl_port *port; + + struct port_decoders { + unsigned long *free_mask; + int count; + + struct cxl_decoder *cxld[CXL_DECODER_MAX_INSTANCES]; + } decoders; }; #endif /* __CXL_MEM_H__ */ diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d420da5fc39c..fdb62ed06433 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,11 +30,55 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } +static int count_decoders(struct device *dev, void *data) +{ + if (is_cxl_decoder(dev)) + (*(int *)data)++; + + return 0; +} + +struct dec_init_ctx { + struct cxl_hdm *cxlhdm; + int ndx; +}; + +static int set_decoders(struct device *dev, void *data) +{ + struct cxl_decoder *cxld; + struct dec_init_ctx *ctx; + struct cxl_hdm *cxlhdm; + int dec; + + if (!is_cxl_decoder(dev)) + return 0; + + cxld = to_cxl_decoder(dev); + + ctx = data; + + cxlhdm = ctx->cxlhdm; + dec = ctx->ndx++; + cxlhdm->decoders.cxld[dec] = cxld; + + if (cxld->flags & CXL_DECODER_F_ENABLE) { + dev_dbg(dev, "Not adding to free decoders\n"); + return 0; + } + + set_bit(dec, cxlhdm->decoders.free_mask); + + dev_dbg(dev, "Adding to free decoder list\n"); + + return 0; +} + static int cxl_port_probe(struct device *dev) { struct cxl_port *port = to_cxl_port(dev); + int rc, decoder_count = 0; + struct dec_init_ctx ctx; struct cxl_hdm *cxlhdm; - int rc; if (is_cxl_endpoint(port)) { struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); @@ -61,6 +105,22 @@ static int cxl_port_probe(struct device *dev) return rc; } + device_for_each_child(dev, &decoder_count, count_decoders); + + cxlhdm->decoders.free_mask = + devm_bitmap_zalloc(dev, decoder_count, GFP_KERNEL); + cxlhdm->decoders.count = decoder_count; + + ctx.cxlhdm = cxlhdm; + ctx.ndx = 0; + if (device_for_each_child(dev, &ctx, set_decoders)) + return -ENXIO; + + dev_set_drvdata(dev, cxlhdm); + + dev_dbg(dev, "Setup complete. Free decoders %*pb\n", + cxlhdm->decoders.count, &cxlhdm->decoders.free_mask); + return 0; } diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index d2f6c990c8a8..145d7bb02714 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -359,21 +359,59 @@ static bool has_switch(const struct cxl_region *cxlr) return false; } +static struct cxl_decoder *get_decoder(struct cxl_region *cxlr, + struct cxl_port *p) +{ + struct cxl_decoder *cxld; + + cxld = cxl_get_decoder(p); + if (IS_ERR(cxld)) { + dev_dbg(&cxlr->dev, "Couldn't get decoder for %s\n", + dev_name(&p->dev)); + return cxld; + } + + cxld->decoder_range = (struct range){ .start = cxlr->res->start, + .end = cxlr->res->end }; + + list_add_tail(&cxld->region_link, + (struct list_head *)&cxlr->staged_list); + + return cxld; +} + +static bool simple_config(struct cxl_region *cxlr, struct cxl_port *hb) +{ + struct cxl_decoder *cxld; + + cxld = get_decoder(cxlr, hb); + if (IS_ERR(cxld)) + return false; + + cxld->interleave_ways = 1; + cxld->interleave_granularity = region_granularity(cxlr); + cxld->target[0] = get_rp(cxlr->config.targets[0]); + return true; +} + /** * region_hb_rp_config_valid() - determine root port ordering is correct * @cxlr: Region to validate * @rootd: root decoder for this @cxlr + * @state_update: Whether or not to update port state * * The algorithm is outlined in 2.13.15 "Verify HB root port configuration * sequence" of the CXL Memory Device SW Guide (Rev1p0). * * Returns true if the configuration is valid. */ -static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, - const struct cxl_decoder *rootd) +static bool region_hb_rp_config_valid(struct cxl_region *cxlr, + const struct cxl_decoder *rootd, + bool state_update) { const int num_root_ports = get_num_root_ports(cxlr); struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; + struct cxl_decoder *cxld, *c; int hb_count, i; hb_count = get_unique_hostbridges(cxlr, hbs); @@ -386,8 +424,8 @@ static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, * Are all devices in this region on the same CXL Host Bridge * Root Port? */ - if (num_root_ports == 1 && !has_switch(cxlr)) - return true; + if (num_root_ports == 1 && !has_switch(cxlr) && state_update) + return simple_config(cxlr, hbs[0]); for (i = 0; i < hb_count; i++) { int idx, position_mask; @@ -397,6 +435,20 @@ static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, /* Get next CXL Host Bridge this region spans */ hb = hbs[i]; + if (state_update) { + cxld = get_decoder(cxlr, hb); + if (IS_ERR(cxld)) { + dev_dbg(&cxlr->dev, + "Couldn't get decoder for %s\n", + dev_name(&hb->dev)); + goto err; + } + cxld->interleave_ways = 0; + cxld->interleave_granularity = region_granularity(cxlr); + } else { + cxld = NULL; + } + /* * Calculate the position mask: NumRootPorts = 2^PositionMask * for this region. @@ -432,13 +484,20 @@ static bool region_hb_rp_config_valid(const struct cxl_region *cxlr, if ((idx & position_mask) != port_grouping) { dev_dbg(&cxlr->dev, "One or more devices are not connected to the correct Host Bridge Root Port\n"); - return false; + goto err; } } } } return true; + +err: + dev_dbg(&cxlr->dev, "Couldn't get decoder for region\n"); + list_for_each_entry_safe(cxld, c, &cxlr->staged_list, region_link) + cxl_put_decoder(cxld); + + return false; } /** @@ -454,7 +513,7 @@ static bool rootd_contains(const struct cxl_region *cxlr, } static bool rootd_valid(const struct cxl_region *cxlr, - const struct cxl_decoder *rootd) + const struct cxl_decoder *rootd, bool state_update) { const struct cxl_memdev *endpoint = cxlr->config.targets[0]; @@ -467,7 +526,8 @@ static bool rootd_valid(const struct cxl_region *cxlr, if (!region_xhb_config_valid(cxlr, rootd)) return false; - if (!region_hb_rp_config_valid(cxlr, rootd)) + if (!region_hb_rp_config_valid((struct cxl_region *)cxlr, rootd, + state_update)) return false; if (!rootd_contains(cxlr, rootd)) @@ -490,7 +550,7 @@ static int rootd_match(struct device *dev, void *data) if (!is_root_decoder(dev)) return 0; - return !!rootd_valid(cxlr, to_cxl_decoder(dev)); + return !!rootd_valid(cxlr, to_cxl_decoder(dev), false); } /* @@ -513,10 +573,39 @@ static struct cxl_decoder *find_rootd(const struct cxl_region *cxlr, return NULL; } -static int collect_ep_decoders(const struct cxl_region *cxlr) +static void cleanup_staged_decoders(struct cxl_region *cxlr) { - /* TODO: */ + struct cxl_decoder *cxld, *d; + + list_for_each_entry_safe(cxld, d, &cxlr->staged_list, region_link) { + cxl_put_decoder(cxld); + list_del_init(&cxld->region_link); + } +} + +static int collect_ep_decoders(struct cxl_region *cxlr) +{ + struct cxl_memdev *ep; + int i, rc = 0; + + for_each_cxl_endpoint(ep, cxlr, i) { + struct cxl_decoder *cxld; + + cxld = get_decoder(cxlr, ep->port); + if (IS_ERR(cxld)) { + rc = PTR_ERR(cxld); + goto err; + } + + cxld->interleave_granularity = region_granularity(cxlr); + cxld->interleave_ways = region_ways(cxlr); + } + return 0; + +err: + cleanup_staged_decoders(cxlr); + return rc; } static int bind_region(const struct cxl_region *cxlr) @@ -559,7 +648,7 @@ static int cxl_region_probe(struct device *dev) return -ENXIO; } - if (!rootd_valid(cxlr, rootd)) { + if (!rootd_valid(cxlr, rootd, true)) { dev_err(dev, "Picked invalid rootd\n"); return -ENXIO; } @@ -574,14 +663,18 @@ static int cxl_region_probe(struct device *dev) ret = collect_ep_decoders(cxlr); if (ret) - return ret; + goto err; ret = bind_region(cxlr); - if (!ret) { - cxlr->active = true; - dev_info(dev, "Bound"); - } + if (ret) + goto err; + cxlr->active = true; + dev_info(dev, "Bound"); + return 0; + +err: + cleanup_staged_decoders(cxlr); return ret; } diff --git a/drivers/cxl/region.h b/drivers/cxl/region.h index 00a6dc729c26..fc15abaeb638 100644 --- a/drivers/cxl/region.h +++ b/drivers/cxl/region.h @@ -14,6 +14,9 @@ * @list: Node in decoder's region list. * @res: Resource this region carves out of the platform decode range. * @active: If the region has been activated. + * @staged_list: All decoders staged for programming. + * @commit_list: All decoders programmed for this region's parameters. + * * @config: HDM decoder program config * @config.size: Size of the region determined from LSA or userspace. * @config.uuid: The UUID for this region. @@ -27,6 +30,8 @@ struct cxl_region { struct list_head list; struct resource *res; bool active; + struct list_head staged_list; + struct list_head commit_list; struct { u64 size; From patchwork Fri Jan 28 00:27:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727590 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 347073FF4; Fri, 28 Jan 2022 00:27:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329649; x=1674865649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CNc2RfHJQp8YCYBaAmfP7EjNqi6QM52sPg4WlCQ49+o=; b=EiArtpM5AUDYhvLHF2wDS1mpWncy6jcJCyIQX3woHuPkNMAy6rdZ/l0o 4Syqu2xC/BY16anmH2FDh3NP7qdv5SboRc5BOAoFWcWefO3dlEPkd05EW HEx7QuHcP+m1pVh6iIotyPE+FSfRAwtL84qbFnT+bd4vLYPiB+trZdGa3 LGHUFgg+aBG+iwDPNUZYxKRmjV+T1N5wWnY4douEHdHcqsf/WCeCDoIEW GNSvdm9+YjwX2LbVFzDPxlPW916FcvOEGakrpyP38sPxVUaB76UYcIijl XYLC4yYNA4x9YofIFpZ91AHoDGxNMYggFN9uf8jtTmzFiuYmWgfk7Kk56 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982080" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982080" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:28 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909649" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:27 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 10/14] cxl/region: Collect host bridge decoders Date: Thu, 27 Jan 2022 16:27:03 -0800 Message-Id: <20220128002707.391076-11-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Part of host bridge verification in the CXL Type 3 Memory Device Software Guide calculates the host bridge interleave target list (6th step in the flow chart), ie. verification and state update are done in the same step. Host bridge verification is already in place, so go ahead and store the decoders with their target lists. Switches are implemented in a separate patch. Signed-off-by: Ben Widawsky --- drivers/cxl/region.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index 145d7bb02714..b8982be13bfe 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -428,6 +428,7 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, return simple_config(cxlr, hbs[0]); for (i = 0; i < hb_count; i++) { + struct cxl_decoder *cxld; int idx, position_mask; struct cxl_dport *rp; struct cxl_port *hb; @@ -486,6 +487,18 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, "One or more devices are not connected to the correct Host Bridge Root Port\n"); goto err; } + + if (!state_update) + continue; + + if (dev_WARN_ONCE(&cxld->dev, + port_grouping >= cxld->nr_targets, + "Invalid port grouping %d/%d\n", + port_grouping, cxld->nr_targets)) + goto err; + + cxld->interleave_ways++; + cxld->target[port_grouping] = get_rp(ep); } } } @@ -538,7 +551,7 @@ static bool rootd_valid(const struct cxl_region *cxlr, struct rootd_context { const struct cxl_region *cxlr; - struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; + const struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; int count; }; @@ -564,7 +577,7 @@ static struct cxl_decoder *find_rootd(const struct cxl_region *cxlr, struct rootd_context ctx; struct device *ret; - ctx.cxlr = cxlr; + ctx.cxlr = (struct cxl_region *)cxlr; ret = device_find_child((struct device *)&root->dev, &ctx, rootd_match); if (ret) From patchwork Fri Jan 28 00:27:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727591 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E1063FF9; Fri, 28 Jan 2022 00:27:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329649; x=1674865649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8bZ/y5CvmAUZBLyNdbWRgpD9oM1QXHWBY0H6dpPm7No=; b=g1JccmLAAaOh7vfK9nwTIC03B9cqj+4qaCtSIKqUunUUmYqprdkGslxK 9yMiCmuoNpumfw2cxSSN9eQkiTIA8ilBd1LpXe+VC/5aK5ob8napfzqXa 7wPW9gZwbk3texEnk9/zvq7g1dr5ZOWeQA2ABSrNDtyIT4LEWjwk9c7pW bXK0IUevcxsjNcr60daZw8Q0CIyfGW7lsPyR568f72EaVdBwEKS9+01VW OEjmUqFHy/YJHiD4Vs2tRHr5wWFG7KayQVZZTEarHakkjXyc6J54k6Y5n 30rCrcsifEWFCociAwIdrkeq3+rj53/84EFZ3aaiHFWDKTAFnWvzzjnBg g==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982083" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982083" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:28 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909654" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:28 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 11/14] cxl/region: Add support for single switch level Date: Thu, 27 Jan 2022 16:27:04 -0800 Message-Id: <20220128002707.391076-12-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 CXL switches have HDM decoders just like host bridges and endpoints. Their programming works in a similar fashion. The spec does not prohibit multiple levels of switches, however, those are not implemented at this time. Signed-off-by: Ben Widawsky --- drivers/cxl/cxl.h | 5 ++++ drivers/cxl/region.c | 61 ++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 8ace6cca0776..d70d8c85d05f 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -96,6 +96,11 @@ static inline u8 cxl_to_ig(u16 g) return ilog2(g) - 8; } +static inline int cxl_to_ways(u8 ways) +{ + return 1 << ways; +} + static inline bool cxl_is_interleave_ways_valid(int iw) { switch (iw) { diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index b8982be13bfe..f748060733dd 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -359,6 +359,23 @@ static bool has_switch(const struct cxl_region *cxlr) return false; } +static bool has_multi_switch(const struct cxl_region *cxlr) +{ + struct cxl_memdev *ep; + int i; + + for_each_cxl_endpoint(ep, cxlr, i) + if (ep->port->depth > 3) + return true; + + return false; +} + +static struct cxl_port *get_switch(struct cxl_memdev *ep) +{ + return to_cxl_port(ep->port->dev.parent); +} + static struct cxl_decoder *get_decoder(struct cxl_region *cxlr, struct cxl_port *p) { @@ -409,6 +426,8 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, const struct cxl_decoder *rootd, bool state_update) { + const int region_ig = cxl_to_ig(cxlr->config.interleave_granularity); + const int region_eniw = cxl_to_eniw(cxlr->config.interleave_ways); const int num_root_ports = get_num_root_ports(cxlr); struct cxl_port *hbs[CXL_DECODER_MAX_INTERLEAVE]; struct cxl_decoder *cxld, *c; @@ -416,8 +435,12 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, hb_count = get_unique_hostbridges(cxlr, hbs); - /* TODO: Switch support */ - if (has_switch(cxlr)) + /* TODO: support multiple levels of switches */ + if (has_multi_switch(cxlr)) + return false; + + /* TODO: x3 interleave for switches is hard. */ + if (has_switch(cxlr) && !is_power_of_2(region_ways(cxlr))) return false; /* @@ -470,8 +493,14 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, list_for_each_entry(rp, &hb->dports, list) { struct cxl_memdev *ep; int port_grouping = -1; + int target_ndx; for_each_cxl_endpoint_hb(ep, cxlr, hb, idx) { + struct cxl_decoder *switch_cxld; + struct cxl_dport *target; + struct cxl_port *switch_port; + bool found = false; + if (get_rp(ep) != rp) continue; @@ -499,6 +528,34 @@ static bool region_hb_rp_config_valid(struct cxl_region *cxlr, cxld->interleave_ways++; cxld->target[port_grouping] = get_rp(ep); + + /* + * At least one switch is connected here if the endpoint + * has a depth > 2 + */ + if (ep->port->depth == 2) + continue; + + /* Check the staged list to see if this + * port has already been added + */ + switch_port = get_switch(ep); + list_for_each_entry(switch_cxld, &cxlr->staged_list, region_link) { + if (to_cxl_port(switch_cxld->dev.parent) == switch_port) + found = true; + } + + if (found) { + target = cxl_find_dport_by_dev(switch_port, ep->dev.parent->parent); + switch_cxld->target[target_ndx++] = target; + continue; + } + + target_ndx = 0; + + switch_cxld = get_decoder(cxlr, switch_port); + switch_cxld->interleave_ways++; + switch_cxld->interleave_granularity = cxl_to_ways(region_ig + region_eniw); } } } From patchwork Fri Jan 28 00:27:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727592 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEA6E3FFB; Fri, 28 Jan 2022 00:27:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329649; x=1674865649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CnVQ+raP+8W2aD0G0WlAiX+0TelcLCLH1moW72XwUD0=; b=KciBakJYysX/SJvWjbRyxxLoiT7ynxLNpJeNqIRnx5qIAEkpovNveQgs sRTLT1oTlM3GytzL4o2PESJxQx8oeW+v2t3KzZ9eEdH57fjGnFPL3XCE8 tJFoInKZd+c0hgJ477RwioUEZ/K2edLtTYN5FRFYE3tp3ZzGcKw5MnXb2 WQECZaXeFqBklv1SubNrSNCdHU2vMu6Lsr6oRiDdX9J77C7IXJtvzgFDO iCbxO7AuqRwqQkp/YoWr+xgsegqqam/2XBHmBlpX0URhvK2FCyopXgL/x rAiAETElhVlx/RBlj6YGfknfkSH37fFunRlP58dSkGiM+kQRCttJ1cwdx w==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982088" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982088" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:29 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909657" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:28 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 12/14] cxl: Program decoders for regions Date: Thu, 27 Jan 2022 16:27:05 -0800 Message-Id: <20220128002707.391076-13-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Configure and commit the HDM decoders for the region. Since the region driver already was able to walk the topology and build the list of needed decoders, all that was needed to finish region setup was to actually write the HDM decoder MMIO. CXL regions appear as linear addresses in the system's physical address space. CXL memory devices comprise the storage for the region. In order for traffic to be properly routed to the memory devices in the region, a set of Host-manged Device Memory decoders must be present. The decoders are a piece of hardware defined in the CXL specification. Signed-off-by: Ben Widawsky --- Changes since v2: - Fix unwind issue in bind_region introduced in v2 --- drivers/cxl/core/hdm.c | 209 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 + drivers/cxl/region.c | 72 +++++++++++--- 3 files changed, 272 insertions(+), 12 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index a28369f264da..66c08d69f7a6 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -268,3 +268,212 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) return 0; } EXPORT_SYMBOL_NS_GPL(devm_cxl_enumerate_decoders, CXL); + +#define COMMIT_TIMEOUT_MS 10 +static int wait_for_commit(struct cxl_decoder *cxld) +{ + const unsigned long end = jiffies + msecs_to_jiffies(COMMIT_TIMEOUT_MS); + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + void __iomem *hdm_decoder; + struct cxl_hdm *cxlhdm; + u32 ctrl; + + cxlhdm = dev_get_drvdata(&port->dev); + hdm_decoder = cxlhdm->regs.hdm_decoder; + + while (1) { + ctrl = readl(hdm_decoder + + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_COMMITTED, ctrl)) + break; + + if (time_after(jiffies, end)) { + dev_err(&cxld->dev, "HDM decoder commit timeout %x\n", + ctrl); + return -ETIMEDOUT; + } + if ((ctrl & CXL_HDM_DECODER0_CTRL_COMMIT_ERROR) != 0) { + dev_err(&cxld->dev, "HDM decoder commit error %x\n", + ctrl); + return -ENXIO; + } + } + + return 0; +} + +/** + * cxl_commit_decoder() - Program a configured cxl_decoder + * @cxld: The preconfigured cxl decoder. + * + * A cxl decoder that is to be committed should have been earmarked as enabled. + * This mechanism acts as a soft reservation on the decoder. + * + * Returns 0 if commit was successful, negative error code otherwise. + */ +int cxl_commit_decoder(struct cxl_decoder *cxld) +{ + u32 ctrl, tl_lo, tl_hi, base_lo, base_hi, size_lo, size_hi; + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + void __iomem *hdm_decoder; + struct cxl_hdm *cxlhdm; + int rc; + + /* + * Decoder flags are entirely software controlled and therefore this + * case is purely a driver bug. + */ + if (dev_WARN_ONCE(&port->dev, (cxld->flags & CXL_DECODER_F_ENABLE) != 0, + "Invalid %s enable state\n", dev_name(&cxld->dev))) + return -ENXIO; + + cxlhdm = dev_get_drvdata(&port->dev); + hdm_decoder = cxlhdm->regs.hdm_decoder; + ctrl = readl(hdm_decoder + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); + + /* + * A decoder that's currently active cannot be changed without the + * system being quiesced. While the driver should prevent against this, + * for a variety of reasons the hardware might not be in sync with the + * hardware and so, do not splat on error. + */ + size_hi = readl(hdm_decoder + + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(cxld->id)); + size_lo = + readl(hdm_decoder + CXL_HDM_DECODER0_SIZE_LOW_OFFSET(cxld->id)); + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_COMMITTED, ctrl) && + (size_lo + size_hi)) { + dev_err(&port->dev, "Tried to change an active decoder (%s)\n", + dev_name(&cxld->dev)); + return -EBUSY; + } + + u32p_replace_bits(&ctrl, cxl_to_ig(cxld->interleave_granularity), + CXL_HDM_DECODER0_CTRL_IG_MASK); + u32p_replace_bits(&ctrl, cxl_to_eniw(cxld->interleave_ways), + CXL_HDM_DECODER0_CTRL_IW_MASK); + u32p_replace_bits(&ctrl, 1, CXL_HDM_DECODER0_CTRL_COMMIT); + + /* TODO: set based on type */ + u32p_replace_bits(&ctrl, 1, CXL_HDM_DECODER0_CTRL_TYPE); + + base_lo = GENMASK(31, 28) & lower_32_bits(cxld->decoder_range.start); + base_hi = upper_32_bits(cxld->decoder_range.start); + + size_lo = GENMASK(31, 28) & (u32)(range_len(&cxld->decoder_range)); + size_hi = upper_32_bits(range_len(&cxld->decoder_range) >> 32); + + if (cxld->nr_targets > 0) { + tl_hi = 0; + + tl_lo = FIELD_PREP(GENMASK(7, 0), cxld->target[0]->port_id); + + if (cxld->interleave_ways > 1) + tl_lo |= FIELD_PREP(GENMASK(15, 8), + cxld->target[1]->port_id); + if (cxld->interleave_ways > 2) + tl_lo |= FIELD_PREP(GENMASK(23, 16), + cxld->target[2]->port_id); + if (cxld->interleave_ways > 3) + tl_lo |= FIELD_PREP(GENMASK(31, 24), + cxld->target[3]->port_id); + if (cxld->interleave_ways > 4) + tl_hi |= FIELD_PREP(GENMASK(7, 0), + cxld->target[4]->port_id); + if (cxld->interleave_ways > 5) + tl_hi |= FIELD_PREP(GENMASK(15, 8), + cxld->target[5]->port_id); + if (cxld->interleave_ways > 6) + tl_hi |= FIELD_PREP(GENMASK(23, 16), + cxld->target[6]->port_id); + if (cxld->interleave_ways > 7) + tl_hi |= FIELD_PREP(GENMASK(31, 24), + cxld->target[7]->port_id); + + writel(tl_hi, hdm_decoder + CXL_HDM_DECODER0_TL_HIGH(cxld->id)); + writel(tl_lo, hdm_decoder + CXL_HDM_DECODER0_TL_LOW(cxld->id)); + } else { + /* Zero out skip list for devices */ + writel(0, hdm_decoder + CXL_HDM_DECODER0_TL_HIGH(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_TL_LOW(cxld->id)); + } + + writel(size_hi, + hdm_decoder + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(cxld->id)); + writel(size_lo, + hdm_decoder + CXL_HDM_DECODER0_SIZE_LOW_OFFSET(cxld->id)); + writel(base_hi, + hdm_decoder + CXL_HDM_DECODER0_BASE_HIGH_OFFSET(cxld->id)); + writel(base_lo, + hdm_decoder + CXL_HDM_DECODER0_BASE_LOW_OFFSET(cxld->id)); + writel(ctrl, hdm_decoder + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); + + rc = wait_for_commit(cxld); + if (rc) + return rc; + + cxld->flags |= CXL_DECODER_F_ENABLE; + +#define DPORT_TL_STR "%d %d %d %d %d %d %d %d" +#define DPORT(i) \ + (cxld->nr_targets && cxld->interleave_ways > (i)) ? \ + cxld->target[(i)]->port_id : \ + -1 +#define DPORT_TL \ + DPORT(0), DPORT(1), DPORT(2), DPORT(3), DPORT(4), DPORT(5), DPORT(6), \ + DPORT(7) + + dev_dbg(&cxld->dev, + "%s (depth %d)\n\tBase %pa\n\tSize %llu\n\tIG %u (%ub)\n\tENIW %u (x%u)\n\tTargetList: \n" DPORT_TL_STR, + dev_name(&port->dev), port->depth, &cxld->decoder_range.start, + range_len(&cxld->decoder_range), + cxl_to_ig(cxld->interleave_granularity), + cxld->interleave_granularity, + cxl_to_eniw(cxld->interleave_ways), cxld->interleave_ways, + DPORT_TL); +#undef DPORT_TL +#undef DPORT +#undef DPORT_TL_STR + return 0; +} +EXPORT_SYMBOL_GPL(cxl_commit_decoder); + +/** + * cxl_disable_decoder() - Disables a decoder + * @cxld: The active cxl decoder. + * + * CXL decoders (as of 2.0 spec) have no way to deactivate them other than to + * set the size of the HDM to 0. This function will clear all registers, and if + * the decoder is active, commit the 0'd out registers. + */ +void cxl_disable_decoder(struct cxl_decoder *cxld) +{ + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + void __iomem *hdm_decoder; + struct cxl_hdm *cxlhdm; + u32 ctrl; + + cxlhdm = dev_get_drvdata(&port->dev); + hdm_decoder = cxlhdm->regs.hdm_decoder; + ctrl = readl(hdm_decoder + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); + + if (dev_WARN_ONCE(&port->dev, (cxld->flags & CXL_DECODER_F_ENABLE) == 0, + "Invalid decoder enable state\n")) + return; + + cxld->flags &= ~CXL_DECODER_F_ENABLE; + + /* There's no way to "uncommit" a committed decoder, only 0 size it */ + writel(0, hdm_decoder + CXL_HDM_DECODER0_TL_HIGH(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_TL_LOW(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_SIZE_LOW_OFFSET(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_BASE_HIGH_OFFSET(cxld->id)); + writel(0, hdm_decoder + CXL_HDM_DECODER0_BASE_LOW_OFFSET(cxld->id)); + + /* If the device isn't actually active, just zero out all the fields */ + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_COMMITTED, ctrl)) + writel(CXL_HDM_DECODER0_CTRL_COMMIT, + hdm_decoder + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); +} +EXPORT_SYMBOL_GPL(cxl_disable_decoder); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index d70d8c85d05f..f9dab312ed26 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -55,6 +55,7 @@ #define CXL_HDM_DECODER0_CTRL_LOCK BIT(8) #define CXL_HDM_DECODER0_CTRL_COMMIT BIT(9) #define CXL_HDM_DECODER0_CTRL_COMMITTED BIT(10) +#define CXL_HDM_DECODER0_CTRL_COMMIT_ERROR BIT(11) #define CXL_HDM_DECODER0_CTRL_TYPE BIT(12) #define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24) #define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28) @@ -416,6 +417,8 @@ struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, struct cxl_dport *cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dev); struct cxl_port *ep_find_cxl_port(struct cxl_memdev *cxlmd, unsigned int depth); +int cxl_commit_decoder(struct cxl_decoder *cxld); +void cxl_disable_decoder(struct cxl_decoder *cxld); struct cxl_decoder *to_cxl_decoder(struct device *dev); bool is_cxl_decoder(struct device *dev); diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index f748060733dd..ac290677534d 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -678,10 +678,52 @@ static int collect_ep_decoders(struct cxl_region *cxlr) return rc; } -static int bind_region(const struct cxl_region *cxlr) +static int bind_region(struct cxl_region *cxlr) { - /* TODO: */ - return 0; + struct cxl_decoder *cxld, *d; + int rc; + + list_for_each_entry_safe(cxld, d, &cxlr->staged_list, region_link) { + rc = cxl_commit_decoder(cxld); + if (!rc) { + list_move_tail(&cxld->region_link, &cxlr->commit_list); + } else { + dev_dbg(&cxlr->dev, "Failed to commit %s\n", + dev_name(&cxld->dev)); + break; + } + } + + list_for_each_entry_safe(cxld, d, &cxlr->commit_list, region_link) { + if (rc) { + cxl_disable_decoder(cxld); + list_del(&cxld->region_link); + } + } + + if (rc) + cleanup_staged_decoders(cxlr); + + BUG_ON(!list_empty(&cxlr->staged_list)); + return rc; +} + +static void region_unregister(void *dev) +{ + struct cxl_region *region = to_cxl_region(dev); + struct cxl_decoder *cxld, *d; + + if (dev_WARN_ONCE(dev, !list_empty(®ion->staged_list), + "Decoders still staged")) + cleanup_staged_decoders(region); + + /* TODO: teardown the nd_region */ + + list_for_each_entry_safe(cxld, d, ®ion->commit_list, region_link) { + cxl_disable_decoder(cxld); + list_del(&cxld->region_link); + cxl_put_decoder(cxld); + } } static int cxl_region_probe(struct device *dev) @@ -732,20 +774,26 @@ static int cxl_region_probe(struct device *dev) put_device(&ours->dev); ret = collect_ep_decoders(cxlr); - if (ret) - goto err; + if (ret) { + cleanup_staged_decoders(cxlr); + return ret; + } ret = bind_region(cxlr); - if (ret) - goto err; + if (ret) { + /* bind_region should cleanup after itself */ + if (dev_WARN_ONCE(dev, !list_empty(&cxlr->staged_list), + "Region bind failed to cleanup staged decoders\n")) + cleanup_staged_decoders(cxlr); + if (dev_WARN_ONCE(dev, !list_empty(&cxlr->commit_list), + "Region bind failed to cleanup committed decoders\n")) + region_unregister(&cxlr->dev); + return ret; + } cxlr->active = true; dev_info(dev, "Bound"); - return 0; - -err: - cleanup_staged_decoders(cxlr); - return ret; + return devm_add_action_or_reset(dev, region_unregister, dev); } static struct cxl_driver cxl_region_driver = { From patchwork Fri Jan 28 00:27:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727593 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BAF96D19; Fri, 28 Jan 2022 00:27:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329650; x=1674865650; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=I2mDMGeUO3hX56XamEiDwM3Cha4aTFhKrdNK7gj9sTo=; b=lH0XYpuJRcuH1cvrZ8GxLD38OLG2BpKqg3BBAJNhilyzAAxiGjFnBiJl JkCVZ+dCFww3igRdFJvgKIXn5uYkP95DCzXs4X/K6wJP/+fMT7wIvFGab LyAFJpHozHg+9NPbDQJ2XqfwBXBuFGB60zKvNgXoprwbm+My56qAslmBT YHT0gV3YQJYFgvPQyc6sZVtED69Jcb+48CgoKIX9bRbW8zAd6VN7kujbQ rAF+1IldumtS3uI/ELV/WYjib0oiwUxwcomfQ5BqJodrtpy52FYyLg877 bC2tsYOwoldYX2WTWyZ1I3PMwMXi8zIQ0UoYP504+HjDdP0i7NmvLspMp A==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982092" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982092" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:30 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909660" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:29 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 13/14] cxl/pmem: Convert nvdimm bridge API to use dev Date: Thu, 27 Jan 2022 16:27:06 -0800 Message-Id: <20220128002707.391076-14-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The cxl_pmem driver specific cxl_nvdimm structure isn't a suitable parameter for an exported API that can be used by other drivers. Instead, use a dev structure, which should be woven into any caller using this API. This will allow for either the nvdimm's dev, or the memdev's dev to be used. Signed-off-by: Ben Widawsky --- Changes since v2: - Added kdoc to cxl_find_nvdimm_bridge() --- drivers/cxl/core/pmem.c | 12 +++++++++--- drivers/cxl/cxl.h | 2 +- drivers/cxl/pmem.c | 2 +- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/pmem.c b/drivers/cxl/core/pmem.c index 8de240c4d96b..7e431667ade1 100644 --- a/drivers/cxl/core/pmem.c +++ b/drivers/cxl/core/pmem.c @@ -62,10 +62,16 @@ static int match_nvdimm_bridge(struct device *dev, void *data) return is_cxl_nvdimm_bridge(dev); } -struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_nvdimm *cxl_nvd) +/** + * cxl_find_nvdimm_bridge() - Find an nvdimm bridge for a given device + * @dev: The device to find a bridge for. This device must be in the part of the + * CXL topology which is being bridged. + * + * Return: bridge device that hosts cxl_nvdimm objects if found, else NULL. + */ +struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev) { - struct cxl_port *port = find_cxl_root(&cxl_nvd->dev); - struct device *dev; + struct cxl_port *port = find_cxl_root(dev); if (!port) return NULL; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f9dab312ed26..062654204eca 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -479,7 +479,7 @@ struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev); bool is_cxl_nvdimm(struct device *dev); bool is_cxl_nvdimm_bridge(struct device *dev); int devm_cxl_add_nvdimm(struct device *host, struct cxl_memdev *cxlmd); -struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_nvdimm *cxl_nvd); +struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev); /* * Unit test builds overrides this to __weak, find the 'strong' version diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c index 15ad666ab03e..fabdb0c6dbf2 100644 --- a/drivers/cxl/pmem.c +++ b/drivers/cxl/pmem.c @@ -39,7 +39,7 @@ static int cxl_nvdimm_probe(struct device *dev) struct nvdimm *nvdimm; int rc; - cxl_nvb = cxl_find_nvdimm_bridge(cxl_nvd); + cxl_nvb = cxl_find_nvdimm_bridge(&cxl_nvd->dev); if (!cxl_nvb) return -ENXIO; From patchwork Fri Jan 28 00:27:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12727594 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 226302C9E; Fri, 28 Jan 2022 00:27:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643329651; x=1674865651; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qnDCWzwqZN3I15w/s8Qb2KClLbLckceUFz3RVBc72fw=; b=hA3O46E7AOzj4Nvho6GaB0Ib+oD1+jidJBRA+gN7O1SCSV7G5cSMjgxq lh7hbfLheMI0q0juI+gWPN9QJ/t8FoKeOYadB1R3pqfClJfMwc/hodYkF ch+pIqrW2XIdOrdEofCNJd6+hq6SKTSZwwEt5L4hFxAARrDPQ/yvI8wos VGczjXnPJpQMkdYiOf3TF4zzCamUfIzIspxP60zDpGgc7Ohm8g1HkqJxv x+bUaGVwzxY2r4d/t9rUcfLHaMPyKMd9mGv9dpouQDqyRvY4dHwYy4od5 15nMgeZXfWnHbzRABwopsxagf120NGRwdD/tUoAcAQAB1uBNbPnWNcoRa Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10239"; a="226982096" X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="226982096" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:30 -0800 X-IronPort-AV: E=Sophos;i="5.88,322,1635231600"; d="scan'208";a="674909665" Received: from vrao2-mobl1.gar.corp.intel.com (HELO localhost.localdomain) ([10.252.129.6]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2022 16:27:30 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: patches@lists.linux.dev, Ben Widawsky , kernel test robot , Alison Schofield , Dan Williams , Ira Weiny , Jonathan Cameron , Vishal Verma , Bjorn Helgaas , nvdimm@lists.linux.dev, linux-pci@vger.kernel.org Subject: [PATCH v3 14/14] cxl/region: Create an nd_region Date: Thu, 27 Jan 2022 16:27:07 -0800 Message-Id: <20220128002707.391076-15-ben.widawsky@intel.com> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20220128002707.391076-1-ben.widawsky@intel.com> References: <20220128002707.391076-1-ben.widawsky@intel.com> Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 LIBNVDIMM supports the creation of regions for both persistent and volatile memory ranges. The cxl_region driver is capable of handling the CXL side of region creation but will reuse LIBVDIMM for interfacing with the rest of the kernel. TODO: CXL regions can go away. As a result the nd_region must also be torn down. TODO2: Handle mappings. LIBNVDIMM is capable of being informed about which parts of devices contribute to a region and validating whether or not the region is configured properly. To do this properly requires tracking allocations per device. Reported-by: kernel test robot (v2) Signed-off-by: Ben Widawsky --- Changes since v2: - Check nvb is non-null - Give a dev_dbg for non-existent nvdimm_bus --- drivers/cxl/Kconfig | 3 ++- drivers/cxl/core/pmem.c | 16 ++++++++++++ drivers/cxl/cxl.h | 1 + drivers/cxl/region.c | 58 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 77 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 742847503c16..054dc78d6f7d 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -99,7 +99,8 @@ config CXL_PORT tristate config CXL_REGION - default CXL_PORT + depends on CXL_PMEM + default CXL_BUS tristate endif diff --git a/drivers/cxl/core/pmem.c b/drivers/cxl/core/pmem.c index 7e431667ade1..58dc6fba3130 100644 --- a/drivers/cxl/core/pmem.c +++ b/drivers/cxl/core/pmem.c @@ -220,6 +220,22 @@ struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev) } EXPORT_SYMBOL_NS_GPL(to_cxl_nvdimm, CXL); +static int match_cxl_nvdimm(struct device *dev, void *data) +{ + return is_cxl_nvdimm(dev); +} + +struct cxl_nvdimm *cxl_find_nvdimm(struct cxl_memdev *cxlmd) +{ + struct device *dev; + + dev = device_find_child(&cxlmd->dev, NULL, match_cxl_nvdimm); + if (!dev) + return NULL; + return to_cxl_nvdimm(dev); +} +EXPORT_SYMBOL_NS_GPL(cxl_find_nvdimm, CXL); + static struct cxl_nvdimm *cxl_nvdimm_alloc(struct cxl_memdev *cxlmd) { struct cxl_nvdimm *cxl_nvd; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 062654204eca..7eb8f36af30b 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -480,6 +480,7 @@ bool is_cxl_nvdimm(struct device *dev); bool is_cxl_nvdimm_bridge(struct device *dev); int devm_cxl_add_nvdimm(struct device *host, struct cxl_memdev *cxlmd); struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev); +struct cxl_nvdimm *cxl_find_nvdimm(struct cxl_memdev *cxlmd); /* * Unit test builds overrides this to __weak, find the 'strong' version diff --git a/drivers/cxl/region.c b/drivers/cxl/region.c index ac290677534d..be472560fc6a 100644 --- a/drivers/cxl/region.c +++ b/drivers/cxl/region.c @@ -708,6 +708,58 @@ static int bind_region(struct cxl_region *cxlr) return rc; } +static int connect_to_libnvdimm(struct cxl_region *region) +{ + struct nd_region_desc ndr_desc; + struct cxl_nvdimm_bridge *nvb; + struct nd_region *ndr; + int rc = 0; + + nvb = cxl_find_nvdimm_bridge(®ion->config.targets[0]->dev); + if (!nvb) { + dev_dbg(®ion->dev, "Couldn't find nvdimm bridge\n"); + return -ENODEV; + } + + device_lock(&nvb->dev); + if (!nvb->nvdimm_bus) { + dev_dbg(&nvb->dev, "Couldn't find nvdimm bridge's bus\n"); + rc = -ENXIO; + goto out; + } + + memset(&ndr_desc, 0, sizeof(ndr_desc)); + + ndr_desc.res = region->res; + + ndr_desc.numa_node = memory_add_physaddr_to_nid(region->res->start); + ndr_desc.target_node = phys_to_target_node(region->res->start); + if (ndr_desc.numa_node == NUMA_NO_NODE) { + ndr_desc.numa_node = + memory_add_physaddr_to_nid(region->res->start); + dev_info(®ion->dev, + "changing numa node from %d to %d for CXL region %pR", + NUMA_NO_NODE, ndr_desc.numa_node, region->res); + } + if (ndr_desc.target_node == NUMA_NO_NODE) { + ndr_desc.target_node = ndr_desc.numa_node; + dev_info(®ion->dev, + "changing target node from %d to %d for CXL region %pR", + NUMA_NO_NODE, ndr_desc.target_node, region->res); + } + + ndr = nvdimm_pmem_region_create(nvb->nvdimm_bus, &ndr_desc); + if (IS_ERR(ndr)) + rc = PTR_ERR(ndr); + else + dev_set_drvdata(®ion->dev, ndr); + +out: + device_unlock(&nvb->dev); + put_device(&nvb->dev); + return rc; +} + static void region_unregister(void *dev) { struct cxl_region *region = to_cxl_region(dev); @@ -791,6 +843,12 @@ static int cxl_region_probe(struct device *dev) return ret; } + ret = connect_to_libnvdimm(cxlr); + if (ret) { + region_unregister(dev); + return ret; + } + cxlr->active = true; dev_info(dev, "Bound"); return devm_add_action_or_reset(dev, region_unregister, dev);