From patchwork Thu Oct 12 18:53:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419625 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C0AF2AB39 for ; Thu, 12 Oct 2023 18:53:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="meAs8Qi/" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 797B383 for ; Thu, 12 Oct 2023 11:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136832; x=1728672832; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vn/8WJWlGwbF8QwLOPLf3/SNazRSw7bVFzAGSBnmx10=; b=meAs8Qi/yCwravWGO82AWeBF/zA4lfq0vgnRYRUZwxa9WDb2rFGwFWCv eZCM2N6JyeeAemg1V5rpWTld9BLc/xocWIHBaBDRTR60hQhS/XeidWyCf GC4xjI+TkUi0aCbZy+Q3eSM9oN1e7xtUFwxkG0RX0b5u/1BI1Tg4Z8hfR yWZq4QHijByZ7af+Y71RSrvQFPfuxasfjtRwd1LDwRd7BGAkzyNpz201n qsBOx+9GI6dw2e2sNubqudf18ASe44lxRhSa74Cpkt/gutxNFAp/F8U3j ODrYRjFJK1YA9hUepw0nlAwmNyTrZOgG2BGqGe49dqQIHNDP0R+703nb9 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387864889" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387864889" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:53:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="898245583" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="898245583" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:51:48 -0700 Subject: [PATCH v11 01/22] cxl: Export QTG ids from CFMWS to sysfs as qos_class attribute From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Davidlohr Bueso , Ira Weiny , Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:53:37 -0700 Message-ID: <169713681699.2205276.14475306324720093079.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Export the QoS Throttling Group ID from the CXL Fixed Memory Window Structure (CFMWS) under the root decoder sysfs attributes as qos_class. CXL rev3.0 9.17.1.3 CXL Fixed Memory Window Structure (CFMWS) cxl cli will use this id to match with the _DSM retrieved id for a hot-plugged CXL memory device DPA memory range to make sure that the DPA range is under the right CFMWS window. Reviewed-by: Davidlohr Bueso Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- Documentation/ABI/testing/sysfs-bus-cxl | 15 +++++++++++++++ drivers/cxl/acpi.c | 3 +++ drivers/cxl/core/port.c | 11 +++++++++++ drivers/cxl/cxl.h | 3 +++ 4 files changed, 32 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 087f762ebfd5..44ffbbb36654 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -369,6 +369,21 @@ Description: provided it is currently idle / not bound to a driver. +What: /sys/bus/cxl/devices/decoderX.Y/qos_class +Date: May, 2023 +KernelVersion: v6.5 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" this + root-decoder-only attribute conveys a platform specific cookie + that identifies a QoS performance class for the CXL Window. + This class-id can be compared against a similar "qos_class" + published for each memory-type that an endpoint supports. While + it is not required that endpoints map their local memory-class + to a matching platform class, mismatches are not recommended and + there are platform specific side-effects that may result. + + What: /sys/bus/cxl/devices/regionZ/uuid Date: May, 2022 KernelVersion: v6.0 diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 40d055560e52..2034eb4ce83f 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -289,6 +289,9 @@ static int cxl_parse_cfmws(union acpi_subtable_headers *header, void *arg, } } } + + cxlrd->qos_class = cfmws->qtg_id; + rc = cxl_decoder_add(cxld, target_map); err_xormap: if (rc) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 7ca01a834e18..fc31ff8954c0 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -278,6 +278,15 @@ static ssize_t interleave_ways_show(struct device *dev, static DEVICE_ATTR_RO(interleave_ways); +static ssize_t qos_class_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + + return sysfs_emit(buf, "%d\n", cxlrd->qos_class); +} +static DEVICE_ATTR_RO(qos_class); + static struct attribute *cxl_decoder_base_attrs[] = { &dev_attr_start.attr, &dev_attr_size.attr, @@ -297,6 +306,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type2.attr, &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, + &dev_attr_qos_class.attr, SET_CXL_REGION_ATTR(create_pmem_region) SET_CXL_REGION_ATTR(create_ram_region) SET_CXL_REGION_ATTR(delete_region) @@ -1691,6 +1701,7 @@ struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, } atomic_set(&cxlrd->region_id, rc); + cxlrd->qos_class = CXL_QOS_CLASS_INVALID; return cxlrd; } EXPORT_SYMBOL_NS_GPL(cxl_root_decoder_alloc, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 76d92561af29..eb7924648cb0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -321,6 +321,7 @@ enum cxl_decoder_type { */ #define CXL_DECODER_MAX_INTERLEAVE 16 +#define CXL_QOS_CLASS_INVALID -1 /** * struct cxl_decoder - Common CXL HDM Decoder Attributes @@ -432,6 +433,7 @@ typedef struct cxl_dport *(*cxl_calc_hb_fn)(struct cxl_root_decoder *cxlrd, * @calc_hb: which host bridge covers the n'th position by granularity * @platform_data: platform specific configuration data * @range_lock: sync region autodiscovery by address range + * @qos_class: QoS performance class cookie * @cxlsd: base cxl switch decoder */ struct cxl_root_decoder { @@ -440,6 +442,7 @@ struct cxl_root_decoder { cxl_calc_hb_fn calc_hb; void *platform_data; struct mutex range_lock; + int qos_class; struct cxl_switch_decoder cxlsd; }; From patchwork Thu Oct 12 18:53:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419627 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDB5E2AB39 for ; Thu, 12 Oct 2023 18:53:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="igfY0+Hm" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08892C0 for ; Thu, 12 Oct 2023 11:53:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136835; x=1728672835; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JpI2N622JoMRugXjc75fETIHfXpPArKUc34Mst8KloM=; b=igfY0+Hm/2uWs3KEsXw8kF+A9aT40RkWe52wyo1UAq4/4IcT0TZbKg8g vJB/piMIjXVeo1E0yj7pSTLHx4xlkT7dHQ6t9aAv+dQ46DpVq2A7NhRiz OlZxMHObVhkLUV94oQi+fKGhc6hGWyPHpl1Nv/lDIcq235kJjoI/obPhk PJyEYEL3ZXSK7x5XYIti/6ZD76XhE1xJkqYRCIiztgVmsbNYJYNnFVrR1 +wjCJKj/DUyfTMnHjgm4uXhyRVTqVPB8o5IZWbpdo8ZfgIorjToUd2ebT hW3BKwlAY+JMgUEeimI2K4999NdgfKJts/ecXG60dUnUUq3h56l+ND+Ci w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387864910" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387864910" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:53:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="898245605" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="898245605" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:51:54 -0700 Subject: [PATCH v11 02/22] cxl: Add checksum verification to CDAT from CXL From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Ira Weiny , Jonathan Cameron , Davidlohr Bueso , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:53:42 -0700 Message-ID: <169713682277.2205276.2687265961314933628.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net A CDAT table is available from a CXL device. The table is read by the driver and cached in software. With the CXL subsystem needing to parse the CDAT table, the checksum should be verified. Add checksum verification after the CDAT table is read from device. Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Reviewed-by: Davidlohr Bueso Signed-off-by: Dave Jiang --- drivers/cxl/core/pci.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index c7a7887ebdcf..9321a3865c30 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -595,6 +595,16 @@ static int cxl_cdat_read_table(struct device *dev, return 0; } +static unsigned char cdat_checksum(void *buf, size_t size) +{ + unsigned char sum, *data = buf; + size_t i; + + for (sum = 0, i = 0; i < size; i++) + sum += data[i]; + return sum; +} + /** * read_cdat_data - Read the CDAT data on this port * @port: Port to read data from @@ -634,15 +644,21 @@ void read_cdat_data(struct cxl_port *port) return; rc = cxl_cdat_read_table(dev, cdat_doe, cdat_table, &cdat_length); - if (rc) { - /* Don't leave table data allocated on error */ - devm_kfree(dev, cdat_table); - dev_err(dev, "CDAT data read error\n"); - return; - } + if (rc) + goto err; - port->cdat.table = cdat_table + sizeof(__le32); + cdat_table = cdat_table + sizeof(__le32); + if (cdat_checksum(cdat_table, cdat_length)) + goto err; + + port->cdat.table = cdat_table; port->cdat.length = cdat_length; + return; + +err: + /* Don't leave table data allocated on error */ + devm_kfree(dev, cdat_table); + dev_err(dev, "Failed to read/validate CDAT.\n"); } EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); From patchwork Thu Oct 12 18:53:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419626 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 118D3358B5 for ; Thu, 12 Oct 2023 18:53:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="II/JPzAa" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E10C2CC for ; Thu, 12 Oct 2023 11:53:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136836; x=1728672836; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B4QwlekaDCmh4nU+rJqHNHFk/yzfADdGnns0sqNANpc=; b=II/JPzAaSMNRKHVafKCnuamLq36J5C/MZ5geD6V0s4UxO6YzVjinALrJ k9tTj1pbgbVNKr06a0NfP8Rig00x9MrG/3Cp0n7W2rHauFPK2wrs6uqbP Su7q4JcsXAPoh7tveYVzNc8j2awlDsDyc3p0uwQ6RE8ebrCTznmp17dIN L61BnFJxrBDb7JfJsf0HxAJ/mBVJt+Zakhc6Hvhm9UycnC/IlkJ+Iziot NRDYkmFzAkJny9YrMoSei083UzTTnwcdtJgM7CYQM0/XB+Ork9e1+mFI0 9jhAjRmpCfwqTSgI5xaOZ/SD30VSPvZVokQuAI/20OuvqHfeCTDKVoO2n Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387864931" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387864931" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:53:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="898245676" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="898245676" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:52:00 -0700 Subject: [PATCH v11 03/22] cxl: Add support for reading CXL switch CDAT table From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Davidlohr Bueso , Ira Weiny , Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:53:48 -0700 Message-ID: <169713682855.2205276.6418370379144967443.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add read_cdat_data() call in cxl_switch_port_probe() to allow reading of CDAT data for CXL switches. read_cdat_data() needs to be adjusted for the retrieving of the PCIe device depending on if the passed in port is endpoint or switch. Reviewed-by: Davidlohr Bueso Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/core/pci.c | 22 +++++++++++++++++----- drivers/cxl/port.c | 3 +++ 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 9321a3865c30..c86bcc181a5d 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -613,18 +613,30 @@ static unsigned char cdat_checksum(void *buf, size_t size) */ void read_cdat_data(struct cxl_port *port) { - struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport_dev); - struct device *host = cxlmd->dev.parent; + struct device *uport = port->uport_dev; struct device *dev = &port->dev; struct pci_doe_mb *cdat_doe; + struct pci_dev *pdev = NULL; + struct cxl_memdev *cxlmd; size_t cdat_length; void *cdat_table; int rc; - if (!dev_is_pci(host)) + if (is_cxl_memdev(uport)) { + struct device *host; + + cxlmd = to_cxl_memdev(uport); + host = cxlmd->dev.parent; + if (dev_is_pci(host)) + pdev = to_pci_dev(host); + } else if (dev_is_pci(uport)) { + pdev = to_pci_dev(uport); + } + + if (!pdev) return; - cdat_doe = pci_find_doe_mailbox(to_pci_dev(host), - PCI_DVSEC_VENDOR_ID_CXL, + + cdat_doe = pci_find_doe_mailbox(pdev, PCI_DVSEC_VENDOR_ID_CXL, CXL_DOE_PROTOCOL_TABLE_ACCESS); if (!cdat_doe) { dev_dbg(dev, "No CDAT mailbox\n"); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 6240e05b9542..47bc8e0b8590 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -62,6 +62,9 @@ static int cxl_switch_port_probe(struct cxl_port *port) struct cxl_hdm *cxlhdm; int rc; + /* Cache the data early to ensure is_visible() works */ + read_cdat_data(port); + rc = devm_cxl_port_enumerate_dports(port); if (rc < 0) return rc; From patchwork Thu Oct 12 18:53:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419628 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3B212AB39 for ; Thu, 12 Oct 2023 18:54:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TtPoUdVw" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9673C83 for ; Thu, 12 Oct 2023 11:53:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136838; x=1728672838; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zSGSBQ0N6OJAwNjL9h2bSlxcazYWsAvbzEo8Gbb7Ct4=; b=TtPoUdVw8r0d6Kc6OfTOyhEKXb3vi5rm6U7QyVa6Aymh8pzzO2iqm1ug jaH0sKP7qors1A9Ekgk9rji7gw6HFlCgwckbLTMWYEyLi8aEZkzCH7waf ffoCoNYFpVwo55w+4Gto/AR/GDCFOyUg8JJqF9ubaa7zth9mFvb82EC7o X8ymDH97dN/aRV9a1S1XLClUyJGNEcCgI1xJDxy/DRxWcS3wBo/iWcXly rlYCciyzsDpQJpYRskQOfsw99PGpEybIzn465flLMxmgvSqugtMvCZk8u l3OYbABnHbd2WEcgwdEt8330/5FxffGDRsw96AWUvCkjXD49y3qcXKdrP w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387864952" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387864952" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:53:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="898245721" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="898245721" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:52:06 -0700 Subject: [PATCH v11 04/22] acpi: Move common tables helper functions to common lib From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: "Rafael J. Wysocki" , Hanjun Guo , Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:53:54 -0700 Message-ID: <169713683430.2205276.17899451119920103445.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Some of the routines in ACPI driver/acpi/tables.c can be shared with parsing CDAT. CDAT is a device-provided data structure that is formatted similar to a platform provided ACPI table. CDAT is used by CXL and can exist on platforms that do not use ACPI. Split out the common routine from ACPI to accommodate platforms that do not support ACPI and move that to /lib. The common routines can be built outside of ACPI if FIRMWARE_TABLES is selected. Link: https://lore.kernel.org/linux-cxl/CAJZ5v0jipbtTNnsA0-o5ozOk8ZgWnOg34m34a9pPenTyRLj=6A@mail.gmail.com/ Suggested-by: Rafael J. Wysocki Reviewed-by: Hanjun Guo Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- MAINTAINERS | 2 drivers/acpi/Kconfig | 1 drivers/acpi/tables.c | 173 ------------------------------------------ include/linux/acpi.h | 42 +++------- include/linux/fw_table.h | 43 ++++++++++ lib/Kconfig | 3 + lib/Makefile | 2 lib/fw_table.c | 189 ++++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 251 insertions(+), 204 deletions(-) create mode 100644 include/linux/fw_table.h create mode 100644 lib/fw_table.c diff --git a/MAINTAINERS b/MAINTAINERS index 35977b269d5e..c6b08ec50109 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -294,6 +294,8 @@ F: drivers/pnp/pnpacpi/ F: include/acpi/ F: include/linux/acpi.h F: include/linux/fwnode.h +F: include/linux/fw_table.h +F: lib/fw_table.c F: tools/power/acpi/ ACPI APEI diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index cee82b473dc5..c0a2d43f8451 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -12,6 +12,7 @@ menuconfig ACPI select PNP select NLS select CRC32 + select FIRMWARE_TABLE default y if X86 help Advanced Configuration and Power Interface (ACPI) support for diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 8ab0a82b4da4..c1516337f668 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -37,18 +37,6 @@ static struct acpi_table_desc initial_tables[ACPI_MAX_TABLES] __initdata; static int acpi_apic_instance __initdata_or_acpilib; -enum acpi_subtable_type { - ACPI_SUBTABLE_COMMON, - ACPI_SUBTABLE_HMAT, - ACPI_SUBTABLE_PRMT, - ACPI_SUBTABLE_CEDT, -}; - -struct acpi_subtable_entry { - union acpi_subtable_headers *hdr; - enum acpi_subtable_type type; -}; - /* * Disable table checksum verification for the early stage due to the size * limitation of the current x86 early mapping implementation. @@ -237,167 +225,6 @@ void acpi_table_print_madt_entry(struct acpi_subtable_header *header) } } -static unsigned long __init_or_acpilib -acpi_get_entry_type(struct acpi_subtable_entry *entry) -{ - switch (entry->type) { - case ACPI_SUBTABLE_COMMON: - return entry->hdr->common.type; - case ACPI_SUBTABLE_HMAT: - return entry->hdr->hmat.type; - case ACPI_SUBTABLE_PRMT: - return 0; - case ACPI_SUBTABLE_CEDT: - return entry->hdr->cedt.type; - } - return 0; -} - -static unsigned long __init_or_acpilib -acpi_get_entry_length(struct acpi_subtable_entry *entry) -{ - switch (entry->type) { - case ACPI_SUBTABLE_COMMON: - return entry->hdr->common.length; - case ACPI_SUBTABLE_HMAT: - return entry->hdr->hmat.length; - case ACPI_SUBTABLE_PRMT: - return entry->hdr->prmt.length; - case ACPI_SUBTABLE_CEDT: - return entry->hdr->cedt.length; - } - return 0; -} - -static unsigned long __init_or_acpilib -acpi_get_subtable_header_length(struct acpi_subtable_entry *entry) -{ - switch (entry->type) { - case ACPI_SUBTABLE_COMMON: - return sizeof(entry->hdr->common); - case ACPI_SUBTABLE_HMAT: - return sizeof(entry->hdr->hmat); - case ACPI_SUBTABLE_PRMT: - return sizeof(entry->hdr->prmt); - case ACPI_SUBTABLE_CEDT: - return sizeof(entry->hdr->cedt); - } - return 0; -} - -static enum acpi_subtable_type __init_or_acpilib -acpi_get_subtable_type(char *id) -{ - if (strncmp(id, ACPI_SIG_HMAT, 4) == 0) - return ACPI_SUBTABLE_HMAT; - if (strncmp(id, ACPI_SIG_PRMT, 4) == 0) - return ACPI_SUBTABLE_PRMT; - if (strncmp(id, ACPI_SIG_CEDT, 4) == 0) - return ACPI_SUBTABLE_CEDT; - return ACPI_SUBTABLE_COMMON; -} - -static __init_or_acpilib bool has_handler(struct acpi_subtable_proc *proc) -{ - return proc->handler || proc->handler_arg; -} - -static __init_or_acpilib int call_handler(struct acpi_subtable_proc *proc, - union acpi_subtable_headers *hdr, - unsigned long end) -{ - if (proc->handler) - return proc->handler(hdr, end); - if (proc->handler_arg) - return proc->handler_arg(hdr, proc->arg, end); - return -EINVAL; -} - -/** - * acpi_parse_entries_array - for each proc_num find a suitable subtable - * - * @id: table id (for debugging purposes) - * @table_size: size of the root table - * @table_header: where does the table start? - * @proc: array of acpi_subtable_proc struct containing entry id - * and associated handler with it - * @proc_num: how big proc is? - * @max_entries: how many entries can we process? - * - * For each proc_num find a subtable with proc->id and run proc->handler - * on it. Assumption is that there's only single handler for particular - * entry id. - * - * The table_size is not the size of the complete ACPI table (the length - * field in the header struct), but only the size of the root table; i.e., - * the offset from the very first byte of the complete ACPI table, to the - * first byte of the very first subtable. - * - * On success returns sum of all matching entries for all proc handlers. - * Otherwise, -ENODEV or -EINVAL is returned. - */ -static int __init_or_acpilib acpi_parse_entries_array( - char *id, unsigned long table_size, - struct acpi_table_header *table_header, struct acpi_subtable_proc *proc, - int proc_num, unsigned int max_entries) -{ - struct acpi_subtable_entry entry; - unsigned long table_end, subtable_len, entry_len; - int count = 0; - int errs = 0; - int i; - - table_end = (unsigned long)table_header + table_header->length; - - /* Parse all entries looking for a match. */ - - entry.type = acpi_get_subtable_type(id); - entry.hdr = (union acpi_subtable_headers *) - ((unsigned long)table_header + table_size); - subtable_len = acpi_get_subtable_header_length(&entry); - - while (((unsigned long)entry.hdr) + subtable_len < table_end) { - if (max_entries && count >= max_entries) - break; - - for (i = 0; i < proc_num; i++) { - if (acpi_get_entry_type(&entry) != proc[i].id) - continue; - if (!has_handler(&proc[i]) || - (!errs && - call_handler(&proc[i], entry.hdr, table_end))) { - errs++; - continue; - } - - proc[i].count++; - break; - } - if (i != proc_num) - count++; - - /* - * If entry->length is 0, break from this loop to avoid - * infinite loop. - */ - entry_len = acpi_get_entry_length(&entry); - if (entry_len == 0) { - pr_err("[%4.4s:0x%02x] Invalid zero length\n", id, proc->id); - return -EINVAL; - } - - entry.hdr = (union acpi_subtable_headers *) - ((unsigned long)entry.hdr + entry_len); - } - - if (max_entries && count > max_entries) { - pr_warn("[%4.4s:0x%02x] found the maximum %i entries\n", - id, proc->id, count); - } - - return errs ? -EINVAL : count; -} - int __init_or_acpilib acpi_table_parse_entries_array( char *id, unsigned long table_size, struct acpi_subtable_proc *proc, int proc_num, unsigned int max_entries) diff --git a/include/linux/acpi.h b/include/linux/acpi.h index a73246c3c35e..0d334975c0c9 100644 --- a/include/linux/acpi.h +++ b/include/linux/acpi.h @@ -15,6 +15,7 @@ #include #include #include +#include struct irq_domain; struct irq_domain_ops; @@ -24,6 +25,16 @@ struct irq_domain_ops; #endif #include +#ifdef CONFIG_ACPI_TABLE_LIB +#define EXPORT_SYMBOL_ACPI_LIB(x) EXPORT_SYMBOL_NS_GPL(x, ACPI) +#define __init_or_acpilib +#define __initdata_or_acpilib +#else +#define EXPORT_SYMBOL_ACPI_LIB(x) +#define __init_or_acpilib __init +#define __initdata_or_acpilib __initdata +#endif + #ifdef CONFIG_ACPI #include @@ -119,21 +130,8 @@ enum acpi_address_range_id { /* Table Handlers */ -union acpi_subtable_headers { - struct acpi_subtable_header common; - struct acpi_hmat_structure hmat; - struct acpi_prmt_module_header prmt; - struct acpi_cedt_header cedt; -}; - typedef int (*acpi_tbl_table_handler)(struct acpi_table_header *table); -typedef int (*acpi_tbl_entry_handler)(union acpi_subtable_headers *header, - const unsigned long end); - -typedef int (*acpi_tbl_entry_handler_arg)(union acpi_subtable_headers *header, - void *arg, const unsigned long end); - /* Debugger support */ struct acpi_debugger_ops { @@ -207,14 +205,6 @@ static inline int acpi_debugger_notify_command_complete(void) (!entry) || (unsigned long)entry + sizeof(*entry) > end || \ ((struct acpi_subtable_header *)entry)->length < sizeof(*entry)) -struct acpi_subtable_proc { - int id; - acpi_tbl_entry_handler handler; - acpi_tbl_entry_handler_arg handler_arg; - void *arg; - int count; -}; - void __iomem *__acpi_map_table(unsigned long phys, unsigned long size); void __acpi_unmap_table(void __iomem *map, unsigned long size); int early_acpi_boot_init(void); @@ -229,16 +219,6 @@ void acpi_reserve_initial_tables (void); void acpi_table_init_complete (void); int acpi_table_init (void); -#ifdef CONFIG_ACPI_TABLE_LIB -#define EXPORT_SYMBOL_ACPI_LIB(x) EXPORT_SYMBOL_NS_GPL(x, ACPI) -#define __init_or_acpilib -#define __initdata_or_acpilib -#else -#define EXPORT_SYMBOL_ACPI_LIB(x) -#define __init_or_acpilib __init -#define __initdata_or_acpilib __initdata -#endif - int acpi_table_parse(char *id, acpi_tbl_table_handler handler); int __init_or_acpilib acpi_table_parse_entries(char *id, unsigned long table_size, int entry_id, diff --git a/include/linux/fw_table.h b/include/linux/fw_table.h new file mode 100644 index 000000000000..ff8fa58d5818 --- /dev/null +++ b/include/linux/fw_table.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * fw_tables.h - Parsing support for ACPI and ACPI-like tables provided by + * platform or device firmware + * + * Copyright (C) 2001 Paul Diefenbaugh + * Copyright (C) 2023 Intel Corp. + */ +#ifndef _FW_TABLE_H_ +#define _FW_TABLE_H_ + +union acpi_subtable_headers; + +typedef int (*acpi_tbl_entry_handler)(union acpi_subtable_headers *header, + const unsigned long end); + +typedef int (*acpi_tbl_entry_handler_arg)(union acpi_subtable_headers *header, + void *arg, const unsigned long end); + +struct acpi_subtable_proc { + int id; + acpi_tbl_entry_handler handler; + acpi_tbl_entry_handler_arg handler_arg; + void *arg; + int count; +}; + +#include +#include + +union acpi_subtable_headers { + struct acpi_subtable_header common; + struct acpi_hmat_structure hmat; + struct acpi_prmt_module_header prmt; + struct acpi_cedt_header cedt; +}; + +int acpi_parse_entries_array(char *id, unsigned long table_size, + struct acpi_table_header *table_header, + struct acpi_subtable_proc *proc, + int proc_num, unsigned int max_entries); + +#endif diff --git a/lib/Kconfig b/lib/Kconfig index c686f4adc124..d0474e251da5 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -764,3 +764,6 @@ config ASN1_ENCODER config POLYNOMIAL tristate + +config FIRMWARE_TABLE + bool diff --git a/lib/Makefile b/lib/Makefile index 740109b6e2c8..f113b44bc4e4 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -405,6 +405,8 @@ obj-$(CONFIG_SIPHASH_KUNIT_TEST) += siphash_kunit.o obj-$(CONFIG_GENERIC_LIB_DEVMEM_IS_ALLOWED) += devmem_is_allowed.o +obj-$(CONFIG_FIRMWARE_TABLE) += fw_table.o + # FORTIFY_SOURCE compile-time behavior tests TEST_FORTIFY_SRCS = $(wildcard $(srctree)/$(src)/test_fortify/*-*.c) TEST_FORTIFY_LOGS = $(patsubst $(srctree)/$(src)/%.c, %.log, $(TEST_FORTIFY_SRCS)) diff --git a/lib/fw_table.c b/lib/fw_table.c new file mode 100644 index 000000000000..e84bd0866e10 --- /dev/null +++ b/lib/fw_table.c @@ -0,0 +1,189 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * fw_tables.c - Parsing support for ACPI and ACPI-like tables provided by + * platform or device firmware + * + * Copyright (C) 2001 Paul Diefenbaugh + * Copyright (C) 2023 Intel Corp. + */ +#include +#include +#include +#include +#include +#include + +enum acpi_subtable_type { + ACPI_SUBTABLE_COMMON, + ACPI_SUBTABLE_HMAT, + ACPI_SUBTABLE_PRMT, + ACPI_SUBTABLE_CEDT, +}; + +struct acpi_subtable_entry { + union acpi_subtable_headers *hdr; + enum acpi_subtable_type type; +}; + +static unsigned long __init_or_acpilib +acpi_get_entry_type(struct acpi_subtable_entry *entry) +{ + switch (entry->type) { + case ACPI_SUBTABLE_COMMON: + return entry->hdr->common.type; + case ACPI_SUBTABLE_HMAT: + return entry->hdr->hmat.type; + case ACPI_SUBTABLE_PRMT: + return 0; + case ACPI_SUBTABLE_CEDT: + return entry->hdr->cedt.type; + } + return 0; +} + +static unsigned long __init_or_acpilib +acpi_get_entry_length(struct acpi_subtable_entry *entry) +{ + switch (entry->type) { + case ACPI_SUBTABLE_COMMON: + return entry->hdr->common.length; + case ACPI_SUBTABLE_HMAT: + return entry->hdr->hmat.length; + case ACPI_SUBTABLE_PRMT: + return entry->hdr->prmt.length; + case ACPI_SUBTABLE_CEDT: + return entry->hdr->cedt.length; + } + return 0; +} + +static unsigned long __init_or_acpilib +acpi_get_subtable_header_length(struct acpi_subtable_entry *entry) +{ + switch (entry->type) { + case ACPI_SUBTABLE_COMMON: + return sizeof(entry->hdr->common); + case ACPI_SUBTABLE_HMAT: + return sizeof(entry->hdr->hmat); + case ACPI_SUBTABLE_PRMT: + return sizeof(entry->hdr->prmt); + case ACPI_SUBTABLE_CEDT: + return sizeof(entry->hdr->cedt); + } + return 0; +} + +static enum acpi_subtable_type __init_or_acpilib +acpi_get_subtable_type(char *id) +{ + if (strncmp(id, ACPI_SIG_HMAT, 4) == 0) + return ACPI_SUBTABLE_HMAT; + if (strncmp(id, ACPI_SIG_PRMT, 4) == 0) + return ACPI_SUBTABLE_PRMT; + if (strncmp(id, ACPI_SIG_CEDT, 4) == 0) + return ACPI_SUBTABLE_CEDT; + return ACPI_SUBTABLE_COMMON; +} + +static __init_or_acpilib bool has_handler(struct acpi_subtable_proc *proc) +{ + return proc->handler || proc->handler_arg; +} + +static __init_or_acpilib int call_handler(struct acpi_subtable_proc *proc, + union acpi_subtable_headers *hdr, + unsigned long end) +{ + if (proc->handler) + return proc->handler(hdr, end); + if (proc->handler_arg) + return proc->handler_arg(hdr, proc->arg, end); + return -EINVAL; +} + +/** + * acpi_parse_entries_array - for each proc_num find a suitable subtable + * + * @id: table id (for debugging purposes) + * @table_size: size of the root table + * @table_header: where does the table start? + * @proc: array of acpi_subtable_proc struct containing entry id + * and associated handler with it + * @proc_num: how big proc is? + * @max_entries: how many entries can we process? + * + * For each proc_num find a subtable with proc->id and run proc->handler + * on it. Assumption is that there's only single handler for particular + * entry id. + * + * The table_size is not the size of the complete ACPI table (the length + * field in the header struct), but only the size of the root table; i.e., + * the offset from the very first byte of the complete ACPI table, to the + * first byte of the very first subtable. + * + * On success returns sum of all matching entries for all proc handlers. + * Otherwise, -ENODEV or -EINVAL is returned. + */ +int __init_or_acpilib +acpi_parse_entries_array(char *id, unsigned long table_size, + struct acpi_table_header *table_header, + struct acpi_subtable_proc *proc, + int proc_num, unsigned int max_entries) +{ + unsigned long table_end, subtable_len, entry_len; + struct acpi_subtable_entry entry; + int count = 0; + int errs = 0; + int i; + + table_end = (unsigned long)table_header + table_header->length; + + /* Parse all entries looking for a match. */ + + entry.type = acpi_get_subtable_type(id); + entry.hdr = (union acpi_subtable_headers *) + ((unsigned long)table_header + table_size); + subtable_len = acpi_get_subtable_header_length(&entry); + + while (((unsigned long)entry.hdr) + subtable_len < table_end) { + if (max_entries && count >= max_entries) + break; + + for (i = 0; i < proc_num; i++) { + if (acpi_get_entry_type(&entry) != proc[i].id) + continue; + if (!has_handler(&proc[i]) || + (!errs && + call_handler(&proc[i], entry.hdr, table_end))) { + errs++; + continue; + } + + proc[i].count++; + break; + } + if (i != proc_num) + count++; + + /* + * If entry->length is 0, break from this loop to avoid + * infinite loop. + */ + entry_len = acpi_get_entry_length(&entry); + if (entry_len == 0) { + pr_err("[%4.4s:0x%02x] Invalid zero length\n", id, proc->id); + return -EINVAL; + } + + entry.hdr = (union acpi_subtable_headers *) + ((unsigned long)entry.hdr + entry_len); + } + + if (max_entries && count > max_entries) { + pr_warn("[%4.4s:0x%02x] found the maximum %i entries\n", + id, proc->id, count); + } + + return errs ? -EINVAL : count; +} +EXPORT_SYMBOL_GPL(acpi_parse_entries_array); From patchwork Thu Oct 12 18:54:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419629 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15FBF33986 for ; Thu, 12 Oct 2023 18:54:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ddK8b3Y3" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3DD5BE for ; Thu, 12 Oct 2023 11:54:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136841; x=1728672841; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bi0CevuUkg4bEOhHj2TAK8AMG2eLo29iczB+qNDwgRQ=; b=ddK8b3Y3bOerdBLGweUyBdEaMUT+0cjISuPynlIl0KXYc+o9TqeOLQbk 4NIq1ALPWuwEt6w86S7Q0CrNlEAAtl83k2kInqQ8NjS+vgx3Ihmeiukmj u66DRPrcnrvmjcbKENfnfQZ3uXrAj6fON1EbEWC1dQS4BWfMPtqjHrSYf Hhaifc2hCqlyt3SqReDmfo0FRqnBJ0N53J3U0QKQxaUiBMxULCSMGnhV1 /l1/tPCzln0N+p8sibfFgEE2GYD1SBge3SUIwLQmfqzLUyUSoX8PfJ5Je yeXA34NGLQcl6GLW0yjyGmsIHLx5VUY62yRSHi6kBhLCS4HF+voHzuetu Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387864978" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387864978" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="731040604" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="731040604" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:00 -0700 Subject: [PATCH v11 05/22] lib/firmware_table: tables: Add CDAT table parsing support From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: "Rafael J. Wysocki" , Len Brown , Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:00 -0700 Message-ID: <169713684023.2205276.10694121707579364992.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The CDAT table is very similar to ACPI tables when it comes to sub-table and entry structures. The helper functions can be also used to parse the CDAT table. Add support to the helper functions to deal with an external CDAT table, and also handle the endieness since CDAT can be processed by a BE host. Export a function cdat_table_parse() for CXL driver to parse a CDAT table. In order to minimize ACPICA code changes, __force is being utilized to deal with the case of a big endian (BE) host parsing a CDAT. All CDAT data structure variables are being force casted to __leX as appropriate. Cc: Rafael J. Wysocki Cc: Len Brown Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/tables.c | 5 +++- include/linux/fw_table.h | 11 +++++++++ lib/fw_table.c | 54 +++++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 64 insertions(+), 6 deletions(-) diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index c1516337f668..b07f7d091d13 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -251,8 +251,9 @@ int __init_or_acpilib acpi_table_parse_entries_array( return -ENODEV; } - count = acpi_parse_entries_array(id, table_size, table_header, - proc, proc_num, max_entries); + count = acpi_parse_entries_array(id, table_size, + (union fw_table_header *)table_header, + proc, proc_num, max_entries); acpi_put_table(table_header); return count; diff --git a/include/linux/fw_table.h b/include/linux/fw_table.h index ff8fa58d5818..a97d368df520 100644 --- a/include/linux/fw_table.h +++ b/include/linux/fw_table.h @@ -28,16 +28,25 @@ struct acpi_subtable_proc { #include #include +union fw_table_header { + struct acpi_table_header acpi; + struct acpi_table_cdat cdat; +}; + union acpi_subtable_headers { struct acpi_subtable_header common; struct acpi_hmat_structure hmat; struct acpi_prmt_module_header prmt; struct acpi_cedt_header cedt; + struct acpi_cdat_header cdat; }; int acpi_parse_entries_array(char *id, unsigned long table_size, - struct acpi_table_header *table_header, + union fw_table_header *table_header, struct acpi_subtable_proc *proc, int proc_num, unsigned int max_entries); +int cdat_table_parse(enum acpi_cdat_type type, + acpi_tbl_entry_handler_arg handler_arg, void *arg, + struct acpi_table_cdat *table_header); #endif diff --git a/lib/fw_table.c b/lib/fw_table.c index e84bd0866e10..108f1db3f1a6 100644 --- a/lib/fw_table.c +++ b/lib/fw_table.c @@ -18,6 +18,7 @@ enum acpi_subtable_type { ACPI_SUBTABLE_HMAT, ACPI_SUBTABLE_PRMT, ACPI_SUBTABLE_CEDT, + CDAT_SUBTABLE, }; struct acpi_subtable_entry { @@ -37,6 +38,8 @@ acpi_get_entry_type(struct acpi_subtable_entry *entry) return 0; case ACPI_SUBTABLE_CEDT: return entry->hdr->cedt.type; + case CDAT_SUBTABLE: + return entry->hdr->cdat.type; } return 0; } @@ -53,6 +56,11 @@ acpi_get_entry_length(struct acpi_subtable_entry *entry) return entry->hdr->prmt.length; case ACPI_SUBTABLE_CEDT: return entry->hdr->cedt.length; + case CDAT_SUBTABLE: { + __le16 length = (__force __le16)entry->hdr->cdat.length; + + return le16_to_cpu(length); + } } return 0; } @@ -69,6 +77,8 @@ acpi_get_subtable_header_length(struct acpi_subtable_entry *entry) return sizeof(entry->hdr->prmt); case ACPI_SUBTABLE_CEDT: return sizeof(entry->hdr->cedt); + case CDAT_SUBTABLE: + return sizeof(entry->hdr->cdat); } return 0; } @@ -82,9 +92,24 @@ acpi_get_subtable_type(char *id) return ACPI_SUBTABLE_PRMT; if (strncmp(id, ACPI_SIG_CEDT, 4) == 0) return ACPI_SUBTABLE_CEDT; + if (strncmp(id, ACPI_SIG_CDAT, 4) == 0) + return CDAT_SUBTABLE; return ACPI_SUBTABLE_COMMON; } +static unsigned long __init_or_acpilib +acpi_table_get_length(enum acpi_subtable_type type, + union fw_table_header *header) +{ + if (type == CDAT_SUBTABLE) { + __le32 length = (__force __le32)header->cdat.length; + + return le32_to_cpu(length); + } + + return header->acpi.length; +} + static __init_or_acpilib bool has_handler(struct acpi_subtable_proc *proc) { return proc->handler || proc->handler_arg; @@ -126,21 +151,24 @@ static __init_or_acpilib int call_handler(struct acpi_subtable_proc *proc, */ int __init_or_acpilib acpi_parse_entries_array(char *id, unsigned long table_size, - struct acpi_table_header *table_header, + union fw_table_header *table_header, struct acpi_subtable_proc *proc, int proc_num, unsigned int max_entries) { unsigned long table_end, subtable_len, entry_len; struct acpi_subtable_entry entry; + enum acpi_subtable_type type; int count = 0; int errs = 0; int i; - table_end = (unsigned long)table_header + table_header->length; + type = acpi_get_subtable_type(id); + table_end = (unsigned long)table_header + + acpi_table_get_length(type, table_header); /* Parse all entries looking for a match. */ - entry.type = acpi_get_subtable_type(id); + entry.type = type; entry.hdr = (union acpi_subtable_headers *) ((unsigned long)table_header + table_size); subtable_len = acpi_get_subtable_header_length(&entry); @@ -187,3 +215,23 @@ acpi_parse_entries_array(char *id, unsigned long table_size, return errs ? -EINVAL : count; } EXPORT_SYMBOL_GPL(acpi_parse_entries_array); + +int cdat_table_parse(enum acpi_cdat_type type, + acpi_tbl_entry_handler_arg handler_arg, void *arg, + struct acpi_table_cdat *table_header) +{ + struct acpi_subtable_proc proc = { + .id = type, + .handler_arg = handler_arg, + .arg = arg, + }; + + if (!table_header) + return -EINVAL; + + return acpi_parse_entries_array(ACPI_SIG_CDAT, + sizeof(struct acpi_table_cdat), + (union fw_table_header *)table_header, + &proc, 1, 0); +} +EXPORT_SYMBOL_NS_GPL(cdat_table_parse, CXL); From patchwork Thu Oct 12 18:54:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419630 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22B922AB39 for ; Thu, 12 Oct 2023 18:54:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dA9aulh5" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 850BB83 for ; Thu, 12 Oct 2023 11:54:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136850; x=1728672850; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JOOJTR7d5Fh+sk+cptuKX/FJAlrQZKVu8EZorwo1mZg=; b=dA9aulh5a06XqPyqaMA+LG6KhAQwLcpnsXp9aGsf3E+LIYaohISIW0CJ W/xhHytxgv5k9ywru2EoosCH15vUKx3gHVHAFcqPYmcdsItZk2UXldh9e AQI2IkQAvPou5Zc59wdWAL84j/V+zJ4ag9R3btmMpvFmJoLLzVoloNnQh Ez3NAe1yMHIgxsmbS2/2dwB/0RtzX3jNH/i7ZmQrR6zJdL3Egc+wbiXiW y37N53f5SGwJWBzw/tQcA1sOoewcsF9sOfEoKesacVAwG0l2amGSt0UDz nbwFogSqWd/eU5athnxdW3M/JDGBvYnN4itm4wibt2HH6EM1WZ06jIHvL Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387865033" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387865033" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="731040647" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="731040647" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:07 -0700 Subject: [PATCH v11 06/22] base/node / acpi: Change 'node_hmem_attrs' to 'access_coordinates' From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Dan Williams , Dan Williams , Jonathan Cameron , Greg Kroah-Hartman , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:06 -0700 Message-ID: <169713684660.2205276.14834262516220839119.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Dan Williams suggested changing the struct 'node_hmem_attrs' to 'access_coordinates' [1]. The struct is a container of r/w-latency and r/w-bandwidth numbers. Moving forward, this container will also be used by CXL to store the performance characteristics of each link hop in the PCIE/CXL topology. So, where node_hmem_attrs is just the access parameters of a memory-node, access_coordinates applies more broadly to hardware topology characteristics. The observation is that seemed like an excercise in having the application identify "where" it falls on a spectrum of bandwidth and latency needs. For the tuple of read/write-latency and read/write-bandwidth, "coordinates" is not a perfect fit. Sometimes it is just conveying values in isolation and not a "location" relative to other performance points, but in the end this data is used to identify the performance operation point of a given memory-node. [2] Link: http://lore.kernel.org/r/64471313421f7_1b66294d5@dwillia2-xfh.jf.intel.com.notmuch/ Link: https://lore.kernel.org/linux-cxl/645e6215ee0de_1e6f2945e@dwillia2-xfh.jf.intel.com.notmuch/ Suggested-by: Dan Williams Reviewed-by: Dan Williams Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Greg Kroah-Hartman --- drivers/acpi/numa/hmat.c | 20 ++++++++++---------- drivers/base/node.c | 12 ++++++------ include/linux/node.h | 8 ++++---- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index bba268ecd802..f9ff992038fa 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -62,7 +62,7 @@ struct memory_target { unsigned int memory_pxm; unsigned int processor_pxm; struct resource memregions; - struct node_hmem_attrs hmem_attrs[2]; + struct access_coordinate coord[2]; struct list_head caches; struct node_cache_attrs cache_attrs; bool registered; @@ -227,24 +227,24 @@ static void hmat_update_target_access(struct memory_target *target, { switch (type) { case ACPI_HMAT_ACCESS_LATENCY: - target->hmem_attrs[access].read_latency = value; - target->hmem_attrs[access].write_latency = value; + target->coord[access].read_latency = value; + target->coord[access].write_latency = value; break; case ACPI_HMAT_READ_LATENCY: - target->hmem_attrs[access].read_latency = value; + target->coord[access].read_latency = value; break; case ACPI_HMAT_WRITE_LATENCY: - target->hmem_attrs[access].write_latency = value; + target->coord[access].write_latency = value; break; case ACPI_HMAT_ACCESS_BANDWIDTH: - target->hmem_attrs[access].read_bandwidth = value; - target->hmem_attrs[access].write_bandwidth = value; + target->coord[access].read_bandwidth = value; + target->coord[access].write_bandwidth = value; break; case ACPI_HMAT_READ_BANDWIDTH: - target->hmem_attrs[access].read_bandwidth = value; + target->coord[access].read_bandwidth = value; break; case ACPI_HMAT_WRITE_BANDWIDTH: - target->hmem_attrs[access].write_bandwidth = value; + target->coord[access].write_bandwidth = value; break; default: break; @@ -701,7 +701,7 @@ static void hmat_register_target_cache(struct memory_target *target) static void hmat_register_target_perf(struct memory_target *target, int access) { unsigned mem_nid = pxm_to_node(target->memory_pxm); - node_set_perf_attrs(mem_nid, &target->hmem_attrs[access], access); + node_set_perf_attrs(mem_nid, &target->coord[access], access); } static void hmat_register_target_devices(struct memory_target *target) diff --git a/drivers/base/node.c b/drivers/base/node.c index 493d533f8375..cb2b6cc7f6e6 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -74,14 +74,14 @@ static BIN_ATTR_RO(cpulist, CPULIST_FILE_MAX_BYTES); * @dev: Device for this memory access class * @list_node: List element in the node's access list * @access: The access class rank - * @hmem_attrs: Heterogeneous memory performance attributes + * @coord: Heterogeneous memory performance coordinates */ struct node_access_nodes { struct device dev; struct list_head list_node; unsigned int access; #ifdef CONFIG_HMEM_REPORTING - struct node_hmem_attrs hmem_attrs; + struct access_coordinate coord; #endif }; #define to_access_nodes(dev) container_of(dev, struct node_access_nodes, dev) @@ -167,7 +167,7 @@ static ssize_t property##_show(struct device *dev, \ char *buf) \ { \ return sysfs_emit(buf, "%u\n", \ - to_access_nodes(dev)->hmem_attrs.property); \ + to_access_nodes(dev)->coord.property); \ } \ static DEVICE_ATTR_RO(property) @@ -187,10 +187,10 @@ static struct attribute *access_attrs[] = { /** * node_set_perf_attrs - Set the performance values for given access class * @nid: Node identifier to be set - * @hmem_attrs: Heterogeneous memory performance attributes + * @coord: Heterogeneous memory performance coordinates * @access: The access class the for the given attributes */ -void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, +void node_set_perf_attrs(unsigned int nid, struct access_coordinate *coord, unsigned int access) { struct node_access_nodes *c; @@ -205,7 +205,7 @@ void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, if (!c) return; - c->hmem_attrs = *hmem_attrs; + c->coord = *coord; for (i = 0; access_attrs[i] != NULL; i++) { if (sysfs_add_file_to_group(&c->dev.kobj, access_attrs[i], "initiators")) { diff --git a/include/linux/node.h b/include/linux/node.h index 427a5975cf40..25b66d705ee2 100644 --- a/include/linux/node.h +++ b/include/linux/node.h @@ -20,14 +20,14 @@ #include /** - * struct node_hmem_attrs - heterogeneous memory performance attributes + * struct access_coordinate - generic performance coordinates container * * @read_bandwidth: Read bandwidth in MB/s * @write_bandwidth: Write bandwidth in MB/s * @read_latency: Read latency in nanoseconds * @write_latency: Write latency in nanoseconds */ -struct node_hmem_attrs { +struct access_coordinate { unsigned int read_bandwidth; unsigned int write_bandwidth; unsigned int read_latency; @@ -65,7 +65,7 @@ struct node_cache_attrs { #ifdef CONFIG_HMEM_REPORTING void node_add_cache(unsigned int nid, struct node_cache_attrs *cache_attrs); -void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, +void node_set_perf_attrs(unsigned int nid, struct access_coordinate *coord, unsigned access); #else static inline void node_add_cache(unsigned int nid, @@ -74,7 +74,7 @@ static inline void node_add_cache(unsigned int nid, } static inline void node_set_perf_attrs(unsigned int nid, - struct node_hmem_attrs *hmem_attrs, + struct access_coordinate *coord, unsigned access) { } From patchwork Thu Oct 12 18:54:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419631 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 053C72AB39 for ; Thu, 12 Oct 2023 18:54:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eUx4pMPx" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 744CF83 for ; Thu, 12 Oct 2023 11:54:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136854; x=1728672854; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aKn0SPEHGlZv/ogsxW74cETbi4HUtSWJPu7/P3VITpo=; b=eUx4pMPxNlUnk4ca39QCckErSmp/YgVB7lCHUT29MlOo77ToAsyISxVs BYLuotAQ7vxCi10a9yrDXuswPMwgT3D31apmmCVsPdBo/h4lrcEOBtlAd 3OXSIdW2iK4T5+LIkG5ZrlbdjBmmCaNXWD9A95zeteOFNEaf4YaBCX5JV t0+cyI6vJusp6zpHokoAnsBBXxtkfI7pGUMMRYGkwnCwFCDHVkHiyXD8b 2y0D9chhnrsKKEG+5Aun0SwLXupNnPsepZ0VngRtRYYW56k3dgkuCyqqG 45wt/3AeWOqHfiApboSbcqRbFlU3YYXnNqcZqfnqw0wYaD2ABtT/VvVNQ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387865065" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387865065" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="731040708" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="731040708" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:13 -0700 Subject: [PATCH v11 07/22] acpi: numa: Create enum for memory_target access coordinates indexing From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:12 -0700 Message-ID: <169713685275.2205276.2297863607737092588.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Create enums to provide named indexing for the access coordinate array. This is in preparation for adding generic port support which will add a third index in the array to keep the generic port attributes separate from the memory attributes. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/numa/hmat.c | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index f9ff992038fa..abed728bf09d 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -57,12 +57,18 @@ struct target_cache { struct node_cache_attrs cache_attrs; }; +enum { + NODE_ACCESS_CLASS_0 = 0, + NODE_ACCESS_CLASS_1, + NODE_ACCESS_CLASS_MAX, +}; + struct memory_target { struct list_head node; unsigned int memory_pxm; unsigned int processor_pxm; struct resource memregions; - struct access_coordinate coord[2]; + struct access_coordinate coord[NODE_ACCESS_CLASS_MAX]; struct list_head caches; struct node_cache_attrs cache_attrs; bool registered; @@ -338,10 +344,12 @@ static __init int hmat_parse_locality(union acpi_subtable_headers *header, if (mem_hier == ACPI_HMAT_MEMORY) { target = find_mem_target(targs[targ]); if (target && target->processor_pxm == inits[init]) { - hmat_update_target_access(target, type, value, 0); + hmat_update_target_access(target, type, value, + NODE_ACCESS_CLASS_0); /* If the node has a CPU, update access 1 */ if (node_state(pxm_to_node(inits[init]), N_CPU)) - hmat_update_target_access(target, type, value, 1); + hmat_update_target_access(target, type, value, + NODE_ACCESS_CLASS_1); } } } @@ -600,10 +608,12 @@ static void hmat_register_target_initiators(struct memory_target *target) */ if (target->processor_pxm != PXM_INVAL) { cpu_nid = pxm_to_node(target->processor_pxm); - register_memory_node_under_compute_node(mem_nid, cpu_nid, 0); + register_memory_node_under_compute_node(mem_nid, cpu_nid, + NODE_ACCESS_CLASS_0); access0done = true; if (node_state(cpu_nid, N_CPU)) { - register_memory_node_under_compute_node(mem_nid, cpu_nid, 1); + register_memory_node_under_compute_node(mem_nid, cpu_nid, + NODE_ACCESS_CLASS_1); return; } } @@ -644,12 +654,13 @@ static void hmat_register_target_initiators(struct memory_target *target) } if (best) hmat_update_target_access(target, loc->hmat_loc->data_type, - best, 0); + best, NODE_ACCESS_CLASS_0); } for_each_set_bit(i, p_nodes, MAX_NUMNODES) { cpu_nid = pxm_to_node(i); - register_memory_node_under_compute_node(mem_nid, cpu_nid, 0); + register_memory_node_under_compute_node(mem_nid, cpu_nid, + NODE_ACCESS_CLASS_0); } } @@ -681,11 +692,13 @@ static void hmat_register_target_initiators(struct memory_target *target) clear_bit(initiator->processor_pxm, p_nodes); } if (best) - hmat_update_target_access(target, loc->hmat_loc->data_type, best, 1); + hmat_update_target_access(target, loc->hmat_loc->data_type, best, + NODE_ACCESS_CLASS_1); } for_each_set_bit(i, p_nodes, MAX_NUMNODES) { cpu_nid = pxm_to_node(i); - register_memory_node_under_compute_node(mem_nid, cpu_nid, 1); + register_memory_node_under_compute_node(mem_nid, cpu_nid, + NODE_ACCESS_CLASS_1); } } @@ -746,8 +759,8 @@ static void hmat_register_target(struct memory_target *target) if (!target->registered) { hmat_register_target_initiators(target); hmat_register_target_cache(target); - hmat_register_target_perf(target, 0); - hmat_register_target_perf(target, 1); + hmat_register_target_perf(target, NODE_ACCESS_CLASS_0); + hmat_register_target_perf(target, NODE_ACCESS_CLASS_1); target->registered = true; } mutex_unlock(&target_lock); From patchwork Thu Oct 12 18:54:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419632 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 024272AB39 for ; Thu, 12 Oct 2023 18:54:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SaWcJHJ5" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4A5683 for ; Thu, 12 Oct 2023 11:54:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136864; x=1728672864; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Pz40fMWdMjoAs2AbjWz6UwcHMDscPFrw+hiFpp2L15Y=; b=SaWcJHJ5z4Q/n/CfIKI67GsbI9qQpWq7Gn6tVCTXDNTrGvvdSvVQ+6BA XcUSg3AWE2HSrEy8Gk1KLFLGLADuWkoznsbVFC411KXhO0xgCVvxNKPV/ 7ecIwYOhPbH+f6f+Z0be+zjWERgyrnDagtiIMi32l1qvbiJS96N+rqvmm pMAH2kkwSPa19906VRRjCZ74IaMK6jxZkm5zT2Za+xuhWw7BEbecWoG07 ELZBl2E2nkntaKBAthUeOeOpSem/9lrzmSQNAWQVIYZtxV3KhECUwWmCZ T1nxc5TLLd6cMsTpFyWxr/NsuWWWIfWaVrFc98IonDZx3QNVYJxXkI6hp Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387865098" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387865098" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="731040781" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="731040781" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:19 -0700 Subject: [PATCH v11 08/22] acpi: numa: Add genport target allocation to the HMAT parsing From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:19 -0700 Message-ID: <169713685902.2205276.11660088369310269923.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add SRAT parsing for the HMAT init in order to collect the device handle from the Generic Port Affinity Structure. The device handle will serve as the key to search for target data. Consoliate the common code with alloc_memory_target() in a helper function alloc_target(). Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/numa/hmat.c | 55 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index abed728bf09d..8380460cfee2 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -71,6 +71,7 @@ struct memory_target { struct access_coordinate coord[NODE_ACCESS_CLASS_MAX]; struct list_head caches; struct node_cache_attrs cache_attrs; + u8 gen_port_device_handle[ACPI_SRAT_DEVICE_HANDLE_SIZE]; bool registered; }; @@ -125,8 +126,7 @@ static __init void alloc_memory_initiator(unsigned int cpu_pxm) list_add_tail(&initiator->node, &initiators); } -static __init void alloc_memory_target(unsigned int mem_pxm, - resource_size_t start, resource_size_t len) +static __init struct memory_target *alloc_target(unsigned int mem_pxm) { struct memory_target *target; @@ -134,7 +134,7 @@ static __init void alloc_memory_target(unsigned int mem_pxm, if (!target) { target = kzalloc(sizeof(*target), GFP_KERNEL); if (!target) - return; + return NULL; target->memory_pxm = mem_pxm; target->processor_pxm = PXM_INVAL; target->memregions = (struct resource) { @@ -147,6 +147,19 @@ static __init void alloc_memory_target(unsigned int mem_pxm, INIT_LIST_HEAD(&target->caches); } + return target; +} + +static __init void alloc_memory_target(unsigned int mem_pxm, + resource_size_t start, + resource_size_t len) +{ + struct memory_target *target; + + target = alloc_target(mem_pxm); + if (!target) + return; + /* * There are potentially multiple ranges per PXM, so record each * in the per-target memregions resource tree. @@ -157,6 +170,18 @@ static __init void alloc_memory_target(unsigned int mem_pxm, start, start + len, mem_pxm); } +static __init void alloc_genport_target(unsigned int mem_pxm, u8 *handle) +{ + struct memory_target *target; + + target = alloc_target(mem_pxm); + if (!target) + return; + + memcpy(target->gen_port_device_handle, handle, + ACPI_SRAT_DEVICE_HANDLE_SIZE); +} + static __init const char *hmat_data_type(u8 type) { switch (type) { @@ -498,6 +523,23 @@ static __init int srat_parse_mem_affinity(union acpi_subtable_headers *header, return 0; } +static __init int srat_parse_genport_affinity(union acpi_subtable_headers *header, + const unsigned long end) +{ + struct acpi_srat_generic_affinity *ga = (void *)header; + + if (!ga) + return -EINVAL; + + if (!(ga->flags & ACPI_SRAT_GENERIC_AFFINITY_ENABLED)) + return 0; + + alloc_genport_target(ga->proximity_domain, + (u8 *)ga->device_handle); + + return 0; +} + static u32 hmat_initiator_perf(struct memory_target *target, struct memory_initiator *initiator, struct acpi_hmat_locality *hmat_loc) @@ -848,6 +890,13 @@ static __init int hmat_init(void) ACPI_SRAT_TYPE_MEMORY_AFFINITY, srat_parse_mem_affinity, 0) < 0) goto out_put; + + if (acpi_table_parse_entries(ACPI_SIG_SRAT, + sizeof(struct acpi_table_srat), + ACPI_SRAT_TYPE_GENERIC_PORT_AFFINITY, + srat_parse_genport_affinity, 0) < 0) + goto out_put; + acpi_put_table(tbl); status = acpi_get_table(ACPI_SIG_HMAT, 0, &tbl); From patchwork Thu Oct 12 18:54:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419633 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 130982AB39 for ; Thu, 12 Oct 2023 18:54:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="G8jfCNd8" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3170983 for ; Thu, 12 Oct 2023 11:54:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136868; x=1728672868; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zW8LpVbVXgtqAQBMzLcnbEadqdI9sMYSZiggNxif8C4=; b=G8jfCNd8j2YR/atXyrouObIBcTaroPtWRteL4ETnjECm3N/1qO/Ivd3d GmRsKPkTFNfCcUcknV8fm5SgGmVIB+dOh0Gmh7H5PnZ6vWrGJVk67t811 +fPiswbXsWWz+kJzDZuIYPWjzhMejOC5NnGvYRgvd14QdrFHrAxm3O5eN 3enEF9NP83Nqr8Fz38SV39/Y1k80i1ybSbmu7eeNwoHayJVHwVYAiqUgj u2jnwtTsd2M3QP9L2gHbf9ezTv7TPZ8zhFdsdzZ2p6J9oGt2jCu3X9QyY YvbvxQ24qCjXCi9Ka5g1e3+YME1UFnYFmiPtKYrU/GR7sEcoSbAXu6mNY w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="387865127" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="387865127" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="731040828" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="731040828" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:25 -0700 Subject: [PATCH v11 09/22] acpi: Break out nesting for hmat_parse_locality() From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:25 -0700 Message-ID: <169713686534.2205276.15733915252420300950.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Refactor hmat_parse_locality() to break up the deep nesting of the function. Suggested-by: Jonathan Cameron Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/numa/hmat.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 8380460cfee2..957a38137c7e 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -321,11 +321,28 @@ static __init void hmat_add_locality(struct acpi_hmat_locality *hmat_loc) } } +static __init void hmat_update_target(unsigned int tgt_pxm, unsigned int init_pxm, + u8 mem_hier, u8 type, u32 value) +{ + struct memory_target *target = find_mem_target(tgt_pxm); + + if (mem_hier != ACPI_HMAT_MEMORY) + return; + + if (target && target->processor_pxm == init_pxm) { + hmat_update_target_access(target, type, value, + NODE_ACCESS_CLASS_0); + /* If the node has a CPU, update access 1 */ + if (node_state(pxm_to_node(init_pxm), N_CPU)) + hmat_update_target_access(target, type, value, + NODE_ACCESS_CLASS_1); + } +} + static __init int hmat_parse_locality(union acpi_subtable_headers *header, const unsigned long end) { struct acpi_hmat_locality *hmat_loc = (void *)header; - struct memory_target *target; unsigned int init, targ, total_size, ipds, tpds; u32 *inits, *targs, value; u16 *entries; @@ -366,17 +383,8 @@ static __init int hmat_parse_locality(union acpi_subtable_headers *header, inits[init], targs[targ], value, hmat_data_type_suffix(type)); - if (mem_hier == ACPI_HMAT_MEMORY) { - target = find_mem_target(targs[targ]); - if (target && target->processor_pxm == inits[init]) { - hmat_update_target_access(target, type, value, - NODE_ACCESS_CLASS_0); - /* If the node has a CPU, update access 1 */ - if (node_state(pxm_to_node(inits[init]), N_CPU)) - hmat_update_target_access(target, type, value, - NODE_ACCESS_CLASS_1); - } - } + hmat_update_target(targs[targ], inits[init], + mem_hier, type, value); } } From patchwork Thu Oct 12 18:54:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419634 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF8532AB39 for ; Thu, 12 Oct 2023 18:54:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CmAWvJcZ" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0187E83 for ; Thu, 12 Oct 2023 11:54:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136888; x=1728672888; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8hxjDmVX1OydIcenRR9z10bTX4lDKT4LHWmEbjr1YBQ=; b=CmAWvJcZ0dxuOvhMR69XsBeW5pwDKEXgYByGqHtH1qUHmfCEq9ZUtay8 dxNjfPRwf7e3Zq5EHVqiuDbL3WcaJZJ6ZJxanXAbhFY7UjEOzCeVpNR6z aC4/yMmSZ6DTKDKAB6haZ3zXy9YQqApLB04yz17XgmCNFWYCL5EK4TcKD zuP8ENYAHhVmpy5jSnEsORfbENiS8yO0T+bGvDSdYCgprOvmon4rXcbkl cw4b1jjKbXMgKdgbwdC7ePWqm7oyl7ooytXFzjnseaKMKn/LjcgFNLefl 3Vr9H9WyyTUSb39OviP4aRSoUkSnPJ4MtoCVNXP8Y93w54UWNJ8V7cYcG g==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880216" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880216" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711192" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711192" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:32 -0700 Subject: [PATCH v11 10/22] acpi: numa: Add setting of generic port system locality attributes From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:31 -0700 Message-ID: <169713687154.2205276.17278968721451562796.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add generic port support for the parsing of HMAT system locality sub-table. The attributes will be added to the third array member of the access coordinates in order to not mix with the existing memory attributes. It only provides the system locality attributes from initator to the generic port targets and is missing the rest of the data to the actual memory device. The complete attributes will be updated when a memory device is attached and the system locality information is calculated end to end. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/numa/hmat.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 957a38137c7e..aabd89c79e26 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -60,6 +60,7 @@ struct target_cache { enum { NODE_ACCESS_CLASS_0 = 0, NODE_ACCESS_CLASS_1, + NODE_ACCESS_CLASS_GENPORT_SINK, NODE_ACCESS_CLASS_MAX, }; @@ -336,6 +337,10 @@ static __init void hmat_update_target(unsigned int tgt_pxm, unsigned int init_px if (node_state(pxm_to_node(init_pxm), N_CPU)) hmat_update_target_access(target, type, value, NODE_ACCESS_CLASS_1); + /* Update access from generic port target */ + if (*target->gen_port_device_handle) + hmat_update_target_access(target, type, value, + NODE_ACCESS_CLASS_GENPORT_SINK); } } From patchwork Thu Oct 12 18:54:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419635 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69A2E2AB39 for ; Thu, 12 Oct 2023 18:54:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="B73vk/AI" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 424A083 for ; Thu, 12 Oct 2023 11:54:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136891; x=1728672891; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jD3m7dUC7fN+KYxFKM0m9H0ocrjLWb3YoBEfn1rj+Vc=; b=B73vk/AI2ktGfym2Pm9vzYOe/cpRHu9bn7rEla6M8D5wkfiSis26fkhE 4O/44DWrE2LZqrrkj1vtQ6MiHl2yJsiQ6vqmhXpxIshdyYqSPPY5ljyND QTRx7EQoC9R345uq0bi3zQNDuDeKEhkUfurQATpuBoJErpgMJ46kBfRid lmB8xCfXr32UegFBbjQLD0baMGShlZBGvIjBHy6UxgAQn/WQO+Km9k2kW AzxbdS1qFs4i4aWFNsemJWi0/12tQGgLdOx0MUMEKsIM0d4ookn8G9Y2/ aDsq4oqF+vfuOZTiCveyuJZYjvWcMfHHrC5IR50mR0aqh4MBaUhGd5S8c w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880243" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880243" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711307" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711307" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:37 -0700 Subject: [PATCH v11 11/22] acpi: numa: Add helper function to retrieve the performance attributes From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , "Rafael J. Wysocki" , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:37 -0700 Message-ID: <169713687744.2205276.17806838881616683926.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add helper to retrieve the performance attributes based on the device handle. The helper function is exported so the CXL driver can use that to acquire the performance data between the CPU and the CXL host bridge. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Acked-by: Rafael J. Wysocki --- drivers/acpi/numa/hmat.c | 35 +++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 12 ++++++++++++ 2 files changed, 47 insertions(+) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index aabd89c79e26..e3a29ddacbca 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -107,6 +107,41 @@ static struct memory_target *find_mem_target(unsigned int mem_pxm) return NULL; } +static struct memory_target *acpi_find_genport_target(u8 *device_handle) +{ + struct memory_target *target; + + list_for_each_entry(target, &targets, node) { + if (!memcmp(target->gen_port_device_handle, device_handle, + ACPI_SRAT_DEVICE_HANDLE_SIZE)) + return target; + } + + return NULL; +} + +/** + * acpi_get_genport_coordinates - Retrieve the access coordinates for a generic port + * @device_handle: Device handle string (ACPI or PCI) to match up to the gen port + * @coord: The access coordinates written back out for the generic port + * + * Return: 0 on success. Errno on failure. + */ +int acpi_get_genport_coordinates(u8 *device_handle, + struct access_coordinate *coord) +{ + struct memory_target *target; + + target = acpi_find_genport_target(device_handle); + if (!target) + return -ENOENT; + + *coord = target->coord[NODE_ACCESS_CLASS_GENPORT_SINK]; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(acpi_get_genport_coordinates, CXL); + static __init void alloc_memory_initiator(unsigned int cpu_pxm) { struct memory_initiator *initiator; diff --git a/include/linux/acpi.h b/include/linux/acpi.h index 0d334975c0c9..7ffe054befa2 100644 --- a/include/linux/acpi.h +++ b/include/linux/acpi.h @@ -16,6 +16,7 @@ #include #include #include +#include struct irq_domain; struct irq_domain_ops; @@ -419,6 +420,17 @@ extern int acpi_blacklisted(void); extern void acpi_osi_setup(char *str); extern bool acpi_osi_is_win8(void); +#ifdef CONFIG_ACPI_HMAT +int acpi_get_genport_coordinates(u8 *device_handle, + struct access_coordinate *coord); +#else +static inline int acpi_get_genport_coordinates(u8 *device_handle, + struct access_coordinate *coord) +{ + return -EOPNOTSUPP; +} +#endif + #ifdef CONFIG_ACPI_NUMA int acpi_map_pxm_to_node(int pxm); int acpi_get_node(acpi_handle handle); From patchwork Thu Oct 12 18:54:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419636 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F7522AB39 for ; Thu, 12 Oct 2023 18:55:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aG1DGOYi" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EA5983 for ; Thu, 12 Oct 2023 11:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136899; x=1728672899; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vvw6Skq+8irwDNvPcYNrT1WoIuqx7oL4Z5LZvjbOtIw=; b=aG1DGOYicJIm6CckPj9U7rgk/A3m7iAXkVmV2y7Wyv4A1mKzZwsKcUo4 8c+9VAr8H2qj5Ezux+ZIipCyJEyNvW4/OInDuNR/waVP10X0hSiTs+oBo 9TvjksXCwNuu6rxP7IS28p6BUsijnTWqXNzKVDjXeX8p9VqwGzFDTfJJG zKUH4pC5UrDvtQfs16ZA4U2SHYAPlDc40BY27FTDuPJr6e6OnUcbHLfe0 8J43vZ45K6n3u78Yn2ioVMLOehHd6F0NXbcmPhqZvPUQ8nlDAgk8Ta8Dx BktJwilPQeS9zu85c+0mODEQaT9hX9SG1V95tus4+bjzd9/B4YKgejV5G A==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880305" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880305" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711400" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711400" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:43 -0700 Subject: [PATCH v11 12/22] cxl: Add callback to parse the DSMAS subtables from CDAT From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:43 -0700 Message-ID: <169713688328.2205276.14682106535163902532.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Provide a callback function to the CDAT parser in order to parse the Device Scoped Memory Affinity Structure (DSMAS). Each DSMAS structure contains the DPA range and its associated attributes in each entry. See the CDAT specification for details. The device handle and the DPA range is saved and to be associated with the DSLBIS locality data when the DSLBIS entries are parsed. The list is a local list. When the total path performance data is calculated and storred this list can be discarded. Coherent Device Attribute Table 1.03 2.1 Device Scoped memory Affinity Structure (DSMAS) Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/Kconfig | 1 + drivers/cxl/core/Makefile | 1 + drivers/cxl/core/cdat.c | 58 +++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 23 ++++++++++++++++++ drivers/cxl/port.c | 12 +++++++++ 5 files changed, 95 insertions(+) create mode 100644 drivers/cxl/core/cdat.c diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 8ea1d340e438..a1ae35bd1afc 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -5,6 +5,7 @@ menuconfig CXL_BUS select FW_LOADER select FW_UPLOAD select PCI_DOE + select FIRMWARE_TABLE help CXL is a bus that is electrically compatible with PCI Express, but layers three protocols on that signalling (CXL.io, CXL.cache, and diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 1f66b5d4d935..f4eabfdd50ce 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -13,5 +13,6 @@ cxl_core-y += mbox.o cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += pmu.o +cxl_core-$(CONFIG_FIRMWARE_TABLE) += cdat.o cxl_core-$(CONFIG_TRACING) += trace.o cxl_core-$(CONFIG_CXL_REGION) += region.o diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c new file mode 100644 index 000000000000..a858fd1b5744 --- /dev/null +++ b/drivers/cxl/core/cdat.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Intel Corporation. All rights reserved. */ +#include +#include +#include "cxlpci.h" +#include "cxl.h" + +static int cdat_dsmas_handler(union acpi_subtable_headers *header, void *arg, + const unsigned long end) +{ + struct acpi_cdat_header *hdr = &header->cdat; + struct acpi_cdat_dsmas *dsmas; + int size = sizeof(*hdr) + sizeof(*dsmas); + struct list_head *dsmas_list = arg; + struct dsmas_entry *dent; + u16 len; + + len = le16_to_cpu((__force __le16)hdr->length); + if (len != size || (unsigned long)hdr + len > end) { + pr_warn("Malformed DSMAS table length: (%u:%u)\n", size, len); + return -EINVAL; + } + + /* Skip common header */ + dsmas = (struct acpi_cdat_dsmas *)(hdr + 1); + + dent = kzalloc(sizeof(*dent), GFP_KERNEL); + if (!dent) + return -ENOMEM; + + dent->handle = dsmas->dsmad_handle; + dent->dpa_range.start = le64_to_cpu((__force __le64)dsmas->dpa_base_address); + dent->dpa_range.end = le64_to_cpu((__force __le64)dsmas->dpa_base_address) + + le64_to_cpu((__force __le64)dsmas->dpa_length) - 1; + list_add_tail(&dent->list, dsmas_list); + + return 0; +} + +int cxl_cdat_endpoint_process(struct cxl_port *port, struct list_head *list) +{ + return cdat_table_parse(ACPI_CDAT_TYPE_DSMAS, cdat_dsmas_handler, + list, port->cdat.table); +} +EXPORT_SYMBOL_NS_GPL(cxl_cdat_endpoint_process, CXL); + +void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list) +{ + struct dsmas_entry *dentry, *n; + + list_for_each_entry_safe(dentry, n, dsmas_list, list) { + list_del(&dentry->list); + kfree(dentry); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_cdat_dsmas_list_destroy, CXL); + +MODULE_IMPORT_NS(CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index eb7924648cb0..be2fd8dcb37b 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -824,6 +825,28 @@ static inline struct cxl_dax_region *to_cxl_dax_region(struct device *dev) } #endif +/* CDAT related bits */ +struct dsmas_entry { + struct list_head list; + struct range dpa_range; + u8 handle; +}; + +#ifdef CONFIG_FIRMWARE_TABLE +int cxl_cdat_endpoint_process(struct cxl_port *port, struct list_head *list); +void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list); +#else +static inline int cxl_cdat_endpoint_process(struct cxl_port *port, + struct list_head *list) +{ + return -EOPNOTSUPP; +} + +static inline void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list) +{ +} +#endif + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 47bc8e0b8590..cceee9dbae2c 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -136,6 +136,18 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) device_for_each_child(&port->dev, root, discover_region); put_device(&root->dev); + if (port->cdat.table) { + LIST_HEAD(dsmas_list); + + rc = cxl_cdat_endpoint_process(port, &dsmas_list); + if (rc < 0) + dev_dbg(&port->dev, "Failed to parse CDAT: %d\n", rc); + + /* Performance data processing */ + + cxl_cdat_dsmas_list_destroy(&dsmas_list); + } + return 0; } From patchwork Thu Oct 12 18:54:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419637 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B6DC2AB39 for ; Thu, 12 Oct 2023 18:55:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="A4DswgIN" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BD61B7 for ; Thu, 12 Oct 2023 11:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136901; x=1728672901; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PNrprlIB9YWPsV23/DDHjo938JNKr71o1O2ok5LuLzw=; b=A4DswgIN8MZyOe8DbO6A4L0SsIOOivln1gZSl+zWXFizBW3D8BGnCB0e JxstKwk2clA8wI1ukGWM8Dt8M9TIsling4DUEQdtjfHg0Nujy+8kObtQ1 howQMw2/TFkxuW5aHNrm6oprS9d9fJUiGDGRKRRBM+5uIOoQAcDbJv26v zr5GEcBnXKeH5s3FFiokgH2gj6VjDILJzG7bQ86YgdAYAYKthM1NwWvdS Ig8UwyYfgAECKMjiuyxM1vECYSVGfnhktwMvwPFP8KKtTTsyMy/sKlGtB E1V2ERSQZcc46A5O9Q6u5GxhuUO4j4bP6RpCBjY1MTM81mRFAvXFQ63b3 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880325" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880325" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711490" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711490" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:49 -0700 Subject: [PATCH v11 13/22] cxl: Add callback to parse the DSLBIS subtable from CDAT From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:49 -0700 Message-ID: <169713688916.2205276.18437749333119431904.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Provide a callback to parse the Device Scoped Latency and Bandwidth Information Structure (DSLBIS) in the CDAT structures. The DSLBIS contains the bandwidth and latency information that's tied to a DSMAS handle. The driver will retrieve the read and write latency and bandwidth associated with the DSMAS which is tied to a DPA range. Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and Bandwidth Information Structure (DSLBIS) Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/core/cdat.c | 102 ++++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 2 + 2 files changed, 102 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index a858fd1b5744..361983393666 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -2,6 +2,7 @@ /* Copyright(c) 2023 Intel Corporation. All rights reserved. */ #include #include +#include #include "cxlpci.h" #include "cxl.h" @@ -37,10 +38,107 @@ static int cdat_dsmas_handler(union acpi_subtable_headers *header, void *arg, return 0; } +static void cxl_access_coordinate_set(struct access_coordinate *coord, + int access, unsigned int val) +{ + switch (access) { + case ACPI_HMAT_ACCESS_LATENCY: + coord->read_latency = val; + coord->write_latency = val; + break; + case ACPI_HMAT_READ_LATENCY: + coord->read_latency = val; + break; + case ACPI_HMAT_WRITE_LATENCY: + coord->write_latency = val; + break; + case ACPI_HMAT_ACCESS_BANDWIDTH: + coord->read_bandwidth = val; + coord->write_bandwidth = val; + break; + case ACPI_HMAT_READ_BANDWIDTH: + coord->read_bandwidth = val; + break; + case ACPI_HMAT_WRITE_BANDWIDTH: + coord->write_bandwidth = val; + break; + } +} + +static int cdat_dslbis_handler(union acpi_subtable_headers *header, void *arg, + const unsigned long end) +{ + struct acpi_cdat_header *hdr = &header->cdat; + struct acpi_cdat_dslbis *dslbis; + int size = sizeof(*hdr) + sizeof(*dslbis); + struct list_head *dsmas_list = arg; + struct dsmas_entry *dent; + u16 len; + + len = le16_to_cpu((__force __le16)hdr->length); + if (len != size || (unsigned long)hdr + len > end) { + pr_warn("Malformed DSLBIS table length: (%u:%u)\n", size, len); + return -EINVAL; + } + + /* Skip common header */ + dslbis = (struct acpi_cdat_dslbis *)(hdr + 1); + + /* Skip unrecognized data type */ + if (dslbis->data_type > ACPI_HMAT_WRITE_BANDWIDTH) + return 0; + + list_for_each_entry(dent, dsmas_list, list) { + __le64 le_base; + __le16 le_val; + u64 val; + int rc; + + if (dslbis->handle != dent->handle) + continue; + + /* Not a memory type, skip */ + if ((dslbis->flags & ACPI_HMAT_MEMORY_HIERARCHY) != + ACPI_HMAT_MEMORY) + return 0; + + le_base = (__force __le64)dslbis->entry_base_unit; + le_val = (__force __le16)dslbis->entry[0]; + rc = check_mul_overflow(le64_to_cpu(le_base), + le16_to_cpu(le_val), &val); + if (rc) + pr_warn("DSLBIS value overflowed.\n"); + + cxl_access_coordinate_set(&dent->coord, dslbis->data_type, val); + break; + } + + return 0; +} + +static int cdat_table_parse_output(int rc) +{ + if (rc < 0) + return rc; + if (rc == 0) + return -ENOENT; + + return 0; +} + int cxl_cdat_endpoint_process(struct cxl_port *port, struct list_head *list) { - return cdat_table_parse(ACPI_CDAT_TYPE_DSMAS, cdat_dsmas_handler, - list, port->cdat.table); + int rc; + + rc = cdat_table_parse(ACPI_CDAT_TYPE_DSMAS, cdat_dsmas_handler, + list, port->cdat.table); + rc = cdat_table_parse_output(rc); + if (rc) + return rc; + + rc = cdat_table_parse(ACPI_CDAT_TYPE_DSLBIS, cdat_dslbis_handler, + list, port->cdat.table); + return cdat_table_parse_output(rc); } EXPORT_SYMBOL_NS_GPL(cxl_cdat_endpoint_process, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index be2fd8dcb37b..dc3b3b746c70 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -830,6 +831,7 @@ struct dsmas_entry { struct list_head list; struct range dpa_range; u8 handle; + struct access_coordinate coord; }; #ifdef CONFIG_FIRMWARE_TABLE From patchwork Thu Oct 12 18:54:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419638 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBF6F2AB39 for ; Thu, 12 Oct 2023 18:55:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CknMZ7Sn" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EF22C0 for ; Thu, 12 Oct 2023 11:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136902; x=1728672902; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0ME855GyU48qI0eE3OisiPHzr9ullnbGHFfAJjngQPE=; b=CknMZ7Sn6LhTMZNuflbg3+JaC5BzGZHHYp/nDRtTxd0QW/FzFeSmSJB3 tbpjhAvvY0BxLaxht+RUTwSR1AfoS0vTVs4VeCkn65b5vDrSZFayWg7lZ tqRbDuT0Vq5cAQWzT+iX46TPUpDAFfkquzzW+B6vmio3zxxQAxW/cf2A2 qaKfe69BbP/+Sp1sB+/+oBhCNmIP6H4p1D13xsMw1/QCcqdg9e7v0nCQ0 ptiYY9KDTvYEKu0v+3yYwwef7FDbSrkdExBt9fMfpHRf6tTJFJfvshrDd aZbnHhJ366om3KWX3N4d2D1TSRs8aG/YuErm07OBmOqVcHX/6YoeKgjeS g==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880345" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880345" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711520" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711520" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:54:55 -0700 Subject: [PATCH v11 14/22] cxl: Add callback to parse the SSLBIS subtable from CDAT From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:54:54 -0700 Message-ID: <169713689498.2205276.17152557836753299107.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Provide a callback to parse the Switched Scoped Latency and Bandwidth Information Structure (SSLBIS) in the CDAT structures. The SSLBIS contains the bandwidth and latency information that's tied to the CXL switch that the data table has been read from. The extracted values are stored to the cxl_dport correlated by the port_id depending on the SSLBIS entry. Coherent Device Attribute Table 1.03 2.1 Switched Scoped Latency and Bandwidth Information Structure (DSLBIS) Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/core/cdat.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 8 ++++ drivers/cxl/port.c | 6 +++ 3 files changed, 107 insertions(+) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index 361983393666..f233cebca37a 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -153,4 +153,97 @@ void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list) } EXPORT_SYMBOL_NS_GPL(cxl_cdat_dsmas_list_destroy, CXL); +static int cdat_sslbis_handler(union acpi_subtable_headers *header, void *arg, + const unsigned long end) +{ + struct acpi_cdat_sslbis *sslbis; + int size = sizeof(header->cdat) + sizeof(*sslbis); + struct cxl_port *port = arg; + struct device *dev = &port->dev; + struct acpi_cdat_sslbe *entry; + int remain, entries, i; + u16 len; + + len = le16_to_cpu((__force __le16)header->cdat.length); + remain = len - size; + if (!remain || remain % sizeof(*entry) || + (unsigned long)header + len > end) { + dev_warn(dev, "Malformed SSLBIS table length: (%u)\n", len); + return -EINVAL; + } + + /* Skip common header */ + sslbis = (struct acpi_cdat_sslbis *)((unsigned long)header + + sizeof(header->cdat)); + + /* Unrecognized data type, we can skip */ + if (sslbis->data_type > ACPI_HMAT_WRITE_BANDWIDTH) + return 0; + + entries = remain / sizeof(*entry); + entry = (struct acpi_cdat_sslbe *)((unsigned long)header + sizeof(*sslbis)); + + for (i = 0; i < entries; i++) { + u16 x = le16_to_cpu((__force __le16)entry->portx_id); + u16 y = le16_to_cpu((__force __le16)entry->porty_id); + __le64 le_base; + __le16 le_val; + struct cxl_dport *dport; + unsigned long index; + u16 dsp_id; + u64 val; + + switch (x) { + case ACPI_CDAT_SSLBIS_US_PORT: + dsp_id = y; + break; + case ACPI_CDAT_SSLBIS_ANY_PORT: + switch (y) { + case ACPI_CDAT_SSLBIS_US_PORT: + dsp_id = x; + break; + case ACPI_CDAT_SSLBIS_ANY_PORT: + dsp_id = ACPI_CDAT_SSLBIS_ANY_PORT; + break; + default: + dsp_id = y; + break; + } + break; + default: + dsp_id = x; + break; + } + + le_base = (__force __le64)sslbis->entry_base_unit; + le_val = (__force __le16)entry->latency_or_bandwidth; + + if (check_mul_overflow(le64_to_cpu(le_base), + le16_to_cpu(le_val), &val)) + dev_warn(dev, "SSLBIS value overflowed!\n"); + + xa_for_each(&port->dports, index, dport) { + if (dsp_id == ACPI_CDAT_SSLBIS_ANY_PORT || + dsp_id == dport->port_id) + cxl_access_coordinate_set(&dport->coord, + sslbis->data_type, + val); + } + + entry++; + } + + return 0; +} + +int cxl_cdat_switch_process(struct cxl_port *port) +{ + int rc; + + rc = cdat_table_parse(ACPI_CDAT_TYPE_SSLBIS, cdat_sslbis_handler, + port, port->cdat.table); + return cdat_table_parse_output(rc); +} +EXPORT_SYMBOL_NS_GPL(cxl_cdat_switch_process, CXL); + MODULE_IMPORT_NS(CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index dc3b3b746c70..2d1fd6d0cce5 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -630,6 +630,7 @@ struct cxl_rcrb_info { * @rcrb: Data about the Root Complex Register Block layout * @rch: Indicate whether this dport was enumerated in RCH or VH mode * @port: reference to cxl_port that contains this downstream port + * @coord: access coordinates (performance) for switch from CDAT */ struct cxl_dport { struct device *dport_dev; @@ -638,6 +639,7 @@ struct cxl_dport { struct cxl_rcrb_info rcrb; bool rch; struct cxl_port *port; + struct access_coordinate coord; }; /** @@ -837,6 +839,7 @@ struct dsmas_entry { #ifdef CONFIG_FIRMWARE_TABLE int cxl_cdat_endpoint_process(struct cxl_port *port, struct list_head *list); void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list); +int cxl_cdat_switch_process(struct cxl_port *port); #else static inline int cxl_cdat_endpoint_process(struct cxl_port *port, struct list_head *list) @@ -847,6 +850,11 @@ static inline int cxl_cdat_endpoint_process(struct cxl_port *port, static inline void cxl_cdat_dsmas_list_destroy(struct list_head *dsmas_list) { } + +static inline int cxl_cdat_switch_process(struct cxl_port *port) +{ + return -EOPNOTSUPP; +} #endif /* diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index cceee9dbae2c..f5e2be79f217 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -78,6 +78,12 @@ static int cxl_switch_port_probe(struct cxl_port *port) return PTR_ERR(cxlhdm); } + if (port->cdat.table) { + rc = cxl_cdat_switch_process(port); + if (rc < 0) + dev_warn(&port->dev, "Failed to parse SSLBIS: %d\n", rc); + } + if (rc == 1) { dev_dbg(&port->dev, "Fallback to passthrough decoder\n"); return devm_cxl_add_passthrough_decoder(port); From patchwork Thu Oct 12 18:55:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419639 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EC6133986 for ; Thu, 12 Oct 2023 18:55:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XQTZ5+lp" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC76EB7 for ; Thu, 12 Oct 2023 11:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136906; x=1728672906; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YGfD9zTYV41UpBvCw7Vyc/ljAam3z9Ayc4x8uvMxFMk=; b=XQTZ5+lpFaL1XME0pC4nyfNuCClYTYcM4rUcwvyMPziEir1ezqAv/vnW lTk9a5ApByeGafEVzoWd3c2wmRl9+lkVtQbtkRPUFLkM56ViR0v1Fvx3A u+kIvmPUXSQzCM3xQ9T8JuSC7pgjpUdZfIWRLOl/Fj8Mcf1E2eMJzglz9 AX8rrB9wgipjPd5JBcUXfvol3oWqcKYvq7ymI89MhvRucrOgiXUUl6Ya1 GbkUp/Bckly7mN+D+6ku99FXpFjRjA52UTZ6mrrO7aMzBJ5ej/+ao3V7f Axt/GR3s+CKNJTQqDhpdMUnmKqQrB9I07yLzhHEaORSr2Ri/b7em429JH A==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880367" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880367" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711571" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711571" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:01 -0700 Subject: [PATCH v11 15/22] cxl: Add support for _DSM Function for retrieving QTG ID From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:00 -0700 Message-ID: <169713690074.2205276.14288360792472635288.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net CXL spec v3.0 9.17.3 CXL Root Device Specific Methods (_DSM) Add support to retrieve QTG ID via ACPI _DSM call. The _DSM call requires an input of an ACPI package with 4 dwords (read latency, write latency, read bandwidth, write bandwidth). The call returns a package with 1 WORD that provides the max supported QTG ID and a package that may contain 0 or more WORDs as the recommended QTG IDs in the recommended order. Create a cxl_root container for the root cxl_port and provide a callback ->get_qos_class() in order to retrieve the QoS class. For the ACPI case, the _DSM helper is used to retrieve the QTG ID and returned. A devm_cxl_add_root() function is added for root port setup and registration of the cxl_root callback operation(s). Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v11: - Fix function comment header (Jonathan) - Remove rc init (Jonathan) - Return 0 when no QTG IDs provided. (Jonathan) v10: - Remove allocation in _DSM handler. Change input parameter to request number of ids to return. - Removed Jonathan's review tag due to significant changes v9: - Fix input to 4 ints instead of using buffers. (Dan) - Remove all endien conversion inside acpi handling code. (Dan) v8: - Change DSM return package parsing to use integers. v7: - Fix stray lines in commit log. (Jonathan) v5: - Make the helper a callback for the CXL root. (Dan) - Drop the addition of core/acpi.c. (Dan) - Add endiness handling. (Jonathan) - Refactor error exits. (Jonathan) - Update evaluate function description. (Jonathan) - Make uuid static. (Dan) v2: - Reorder var declaration and use C99 style. (Jonathan) - Allow >2 ACPI objects in package for future expansion. (Jonathan) - Check QTG IDs against MAX QTG ID provided by output package. (Jonathan) --- drivers/cxl/acpi.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/core/port.c | 41 +++++++++++++-- drivers/cxl/cxl.h | 25 +++++++++ 3 files changed, 190 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 2034eb4ce83f..1eb477e8274c 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include "cxlpci.h" #include "cxl.h" @@ -17,6 +18,10 @@ struct cxl_cxims_data { u64 xormaps[] __counted_by(nr_maps); }; +static const guid_t acpi_cxl_qtg_id_guid = + GUID_INIT(0xF365F9A6, 0xA7DE, 0x4071, + 0xA6, 0x6A, 0xB4, 0x0C, 0x0B, 0x4F, 0x8E, 0x52); + /* * Find a targets entry (n) in the host bridge interleave list. * CXL Specification 3.0 Table 9-22 @@ -194,6 +199,125 @@ struct cxl_cfmws_context { int id; }; +/** + * cxl_acpi_evaluate_qtg_dsm - Retrieve QTG ids via ACPI _DSM + * @handle: ACPI handle + * @coord: performance access coordinates + * @entries: number of QTG IDs to return + * @qos_class: int array provided by caller to return QTG IDs + * + * Return: number of QTG IDs returned, or -errno for errors + * + * Issue QTG _DSM with accompanied bandwidth and latency data in order to get + * the QTG IDs that are suitable for the performance point in order of most + * suitable to least suitable. Write back array of QTG IDs and return the + * actual number of QTG IDs written back. + */ +static int +cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, struct access_coordinate *coord, + int entries, int *qos_class) +{ + union acpi_object *out_obj, *out_buf, *pkg; + union acpi_object in_array[4] = { + [0].integer = { ACPI_TYPE_INTEGER, coord->read_latency }, + [1].integer = { ACPI_TYPE_INTEGER, coord->write_latency }, + [2].integer = { ACPI_TYPE_INTEGER, coord->read_bandwidth }, + [3].integer = { ACPI_TYPE_INTEGER, coord->write_bandwidth }, + }; + union acpi_object in_obj = { + .package = { + .type = ACPI_TYPE_PACKAGE, + .count = 4, + .elements = in_array, + }, + }; + int count, pkg_entries, i; + u16 max_qtg; + int rc; + + if (!entries) + return -EINVAL; + + out_obj = acpi_evaluate_dsm(handle, &acpi_cxl_qtg_id_guid, 1, 1, &in_obj); + if (!out_obj) + return -ENXIO; + + if (out_obj->type != ACPI_TYPE_PACKAGE) { + rc = -ENXIO; + goto out; + } + + /* Check Max QTG ID */ + pkg = &out_obj->package.elements[0]; + if (pkg->type != ACPI_TYPE_INTEGER) { + rc = -ENXIO; + goto out; + } + + max_qtg = pkg->integer.value; + + /* It's legal to have 0 QTG entries */ + pkg_entries = out_obj->package.count; + if (pkg_entries <= 1) { + rc = 0; + goto out; + } + + /* Retrieve QTG IDs package */ + pkg = &out_obj->package.elements[1]; + if (pkg->type != ACPI_TYPE_PACKAGE) { + rc = -ENXIO; + goto out; + } + + pkg_entries = pkg->package.count; + count = min(entries, pkg_entries); + for (i = 0; i < count; i++) { + u16 qtg_id; + + out_buf = &pkg->package.elements[i]; + if (out_buf->type != ACPI_TYPE_INTEGER) { + rc = -ENXIO; + goto out; + } + + qtg_id = out_buf->integer.value; + if (qtg_id > max_qtg) + pr_warn("QTG ID %u greater than MAX %u\n", + qtg_id, max_qtg); + + qos_class[i] = qtg_id; + } + rc = count; + +out: + ACPI_FREE(out_obj); + return rc; +} + +static int cxl_acpi_get_qos_class(struct cxl_port *root_port, + struct access_coordinate *coord, + int entries, int *qos_class) +{ + acpi_handle handle; + struct device *dev; + + dev = root_port->uport_dev; + + if (!dev_is_platform(dev)) + return -ENODEV; + + handle = ACPI_HANDLE(dev); + if (!handle) + return -ENODEV; + + return cxl_acpi_evaluate_qtg_dsm(handle, coord, entries, qos_class); +} + +static const struct cxl_root_ops acpi_root_ops = { + .get_qos_class = cxl_acpi_get_qos_class, +}; + static int cxl_parse_cfmws(union acpi_subtable_headers *header, void *arg, const unsigned long end) { @@ -656,6 +780,7 @@ static int cxl_acpi_probe(struct platform_device *pdev) { int rc; struct resource *cxl_res; + struct cxl_root *cxl_root; struct cxl_port *root_port; struct device *host = &pdev->dev; struct acpi_device *adev = ACPI_COMPANION(host); @@ -675,9 +800,10 @@ static int cxl_acpi_probe(struct platform_device *pdev) cxl_res->end = -1; cxl_res->flags = IORESOURCE_MEM; - root_port = devm_cxl_add_port(host, host, CXL_RESOURCE_NONE, NULL); - if (IS_ERR(root_port)) - return PTR_ERR(root_port); + cxl_root = devm_cxl_add_root(host, &acpi_root_ops); + if (IS_ERR(cxl_root)) + return PTR_ERR(cxl_root); + root_port = &cxl_root->port; rc = bus_for_each_dev(adev->dev.bus, NULL, root_port, add_host_bridge_dport); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index fc31ff8954c0..23403538505b 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -528,7 +528,10 @@ static void cxl_port_release(struct device *dev) xa_destroy(&port->dports); xa_destroy(&port->regions); ida_free(&cxl_port_ida, port->id); - kfree(port); + if (is_cxl_root(port)) + kfree(to_cxl_root(port)); + else + kfree(port); } static const struct attribute_group *cxl_port_attribute_groups[] = { @@ -632,13 +635,22 @@ static struct cxl_port *cxl_port_alloc(struct device *uport_dev, resource_size_t component_reg_phys, struct cxl_dport *parent_dport) { + struct cxl_root *cxl_root = NULL; struct cxl_port *port; struct device *dev; int rc; - port = kzalloc(sizeof(*port), GFP_KERNEL); - if (!port) - return ERR_PTR(-ENOMEM); + /* No parent_dport, root cxl_port */ + if (!parent_dport) { + cxl_root = kzalloc(sizeof(*cxl_root), GFP_KERNEL); + if (!cxl_root) + return ERR_PTR(-ENOMEM); + port = &cxl_root->port; + } else { + port = kzalloc(sizeof(*port), GFP_KERNEL); + if (!port) + return ERR_PTR(-ENOMEM); + } rc = ida_alloc(&cxl_port_ida, GFP_KERNEL); if (rc < 0) @@ -697,7 +709,10 @@ static struct cxl_port *cxl_port_alloc(struct device *uport_dev, return port; err: - kfree(port); + if (cxl_root) + kfree(cxl_root); + else + kfree(port); return ERR_PTR(rc); } @@ -821,6 +836,22 @@ struct cxl_port *devm_cxl_add_port(struct device *host, } EXPORT_SYMBOL_NS_GPL(devm_cxl_add_port, CXL); +struct cxl_root *devm_cxl_add_root(struct device *host, + const struct cxl_root_ops *ops) +{ + struct cxl_root *cxl_root; + struct cxl_port *port; + + port = devm_cxl_add_port(host, host, CXL_RESOURCE_NONE, NULL); + if (IS_ERR(port)) + return (struct cxl_root *)port; + + cxl_root = to_cxl_root(port); + cxl_root->ops = ops; + return cxl_root; +} +EXPORT_SYMBOL_NS_GPL(devm_cxl_add_root, CXL); + struct pci_bus *cxl_port_to_pci_bus(struct cxl_port *port) { /* There is no pci_bus associated with a CXL platform-root port */ diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 2d1fd6d0cce5..587677fc597c 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -611,6 +611,29 @@ struct cxl_port { bool cdat_available; }; +struct cxl_root_ops { + int (*get_qos_class)(struct cxl_port *root_port, + struct access_coordinate *coord, + int entries, int *qos_class); +}; + +/** + * struct cxl_root - logical collection of root cxl_port items + * + * @port: cxl_port member + * @ops: cxl root operations + */ +struct cxl_root { + struct cxl_port port; + const struct cxl_root_ops *ops; +}; + +static inline struct cxl_root * +to_cxl_root(const struct cxl_port *port) +{ + return container_of(port, struct cxl_root, port); +} + static inline struct cxl_dport * cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dport_dev) { @@ -696,6 +719,8 @@ struct cxl_port *devm_cxl_add_port(struct device *host, struct device *uport_dev, resource_size_t component_reg_phys, struct cxl_dport *parent_dport); +struct cxl_root *devm_cxl_add_root(struct device *host, + const struct cxl_root_ops *ops); struct cxl_port *find_cxl_root(struct cxl_port *port); int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd); void cxl_bus_rescan(void); From patchwork Thu Oct 12 18:55:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419640 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A9A92AB39 for ; Thu, 12 Oct 2023 18:55:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hJJXQDt3" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB679CF for ; Thu, 12 Oct 2023 11:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136908; x=1728672908; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Qex/w5IVkLrZZZUQS0rAurWWSFqtdndBaChU8ALe7iM=; b=hJJXQDt3sxOuMakKObYyxlWmkCpA5xUNaMQ+5GwXj8TMD4aVFL7tKQzH DMMlTsYM5XoIKda/FqTyZM0g0FZvcVt7qr2ulgx42rd0IWZ5QhwIABEvV RkVY2Sj+B4a8OmnGJMI80+FkgTS2CsWh5OJBjV9fKT+7ymdBphQsTSL/n wlobHZMPIAAblXxdLxliwxHE5OMTEn/11FJ3CyfBNmQB5PA8tI6/QNPbk nY/yb/Im6pANxd3CIsvvr5GivFnBqTMyiT1dT3G1oSBC/cCVHyPreL7Ik sU8sAFcsnnW8wqfrMGUEos9H5mJmzWmXm6LW1yhNsbP2vy3sfzujIGR4V Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604182" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604182" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724509" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724509" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:06 -0700 Subject: [PATCH v11 16/22] cxl: Calculate and store PCI link latency for the downstream ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:06 -0700 Message-ID: <169713690650.2205276.6608428147494650680.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The latency is calculated by dividing the flit size over the bandwidth. Add support to retrieve the flit size for the CXL switch device and calculate the latency of the PCIe link. Cache the latency number with cxl_dport. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v11: - Fix 0-day report on cxl_pci_get_latency comment header. '-' to ':' for @pdev --- drivers/cxl/core/pci.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/port.c | 6 ++++ drivers/cxl/cxl.h | 4 +++ drivers/cxl/cxlpci.h | 15 ++++++++++ 4 files changed, 98 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index c86bcc181a5d..2ba6655dcfd1 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2021 Intel Corporation. All rights reserved. */ +#include #include #include #include @@ -783,3 +784,75 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, return PCI_ERS_RESULT_NEED_RESET; } EXPORT_SYMBOL_NS_GPL(cxl_error_detected, CXL); + +extern const unsigned char pcie_link_speed[]; + +static enum pci_bus_speed get_link_speed(struct pci_dev *pdev) +{ + u16 linkstat; + int err; + + err = pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &linkstat); + if (err) + return -EINVAL; + + return pcie_link_speed[linkstat & PCI_EXP_LNKSTA_CLS]; +} + +static int pci_bus_speed_to_mbps(enum pci_bus_speed speed) +{ + switch (speed) { + case PCIE_SPEED_2_5GT: + return 2500; + case PCIE_SPEED_5_0GT: + return 5000; + case PCIE_SPEED_8_0GT: + return 8000; + case PCIE_SPEED_16_0GT: + return 16000; + case PCIE_SPEED_32_0GT: + return 32000; + case PCIE_SPEED_64_0GT: + return 64000; + default: + break; + } + + return -EINVAL; +} + +static int cxl_flit_size(struct pci_dev *pdev) +{ + if (cxl_pci_flit_256(pdev)) + return 256; + + return 68; +} + +/** + * cxl_pci_get_latency - calculate the link latency for the PCIe link + * @pdev: PCI device + * + * return: calculated latency or 0 for no latency + * + * CXL Memory Device SW Guide v1.0 2.11.4 Link latency calculation + * Link latency = LinkPropagationLatency + FlitLatency + RetimerLatency + * LinkProgationLatency is negligible, so 0 will be used + * RetimerLatency is assumed to be negligible and 0 will be used + * FlitLatency = FlitSize / LinkBandwidth + * FlitSize is defined by spec. CXL rev3.0 4.2.1. + * 68B flit is used up to 32GT/s. >32GT/s, 256B flit size is used. + * The FlitLatency is converted to picoseconds. + */ +long cxl_pci_get_latency(struct pci_dev *pdev) +{ + long bw; + + bw = pci_bus_speed_to_mbps(get_link_speed(pdev)); + if (bw < 0) + return 0; + bw /= BITS_PER_BYTE; + + return cxl_flit_size(pdev) * MEGA / bw; +} +EXPORT_SYMBOL_NS_GPL(cxl_pci_get_latency, CXL); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 23403538505b..3b45da2b425e 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -793,6 +793,9 @@ static struct cxl_port *__devm_cxl_add_port(struct device *host, if (rc) return ERR_PTR(rc); + if (parent_dport && dev_is_pci(uport_dev)) + port->pci_latency = cxl_pci_get_latency(to_pci_dev(uport_dev)); + return port; err: @@ -1067,6 +1070,9 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, if (rc) return ERR_PTR(rc); + if (dev_is_pci(dport_dev)) + dport->link_latency = cxl_pci_get_latency(to_pci_dev(dport_dev)); + return dport; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 587677fc597c..f1c43f02d65e 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -586,6 +586,7 @@ struct cxl_dax_region { * @depth: How deep this port is relative to the root. depth 0 is the root. * @cdat: Cached CDAT data * @cdat_available: Should a CDAT attribute be available in sysfs + * @pci_latency: Upstream latency in picoseconds */ struct cxl_port { struct device dev; @@ -609,6 +610,7 @@ struct cxl_port { size_t length; } cdat; bool cdat_available; + long pci_latency; }; struct cxl_root_ops { @@ -654,6 +656,7 @@ struct cxl_rcrb_info { * @rch: Indicate whether this dport was enumerated in RCH or VH mode * @port: reference to cxl_port that contains this downstream port * @coord: access coordinates (performance) for switch from CDAT + * @link_latency: calculated PCIe downstream latency */ struct cxl_dport { struct device *dport_dev; @@ -663,6 +666,7 @@ struct cxl_dport { bool rch; struct cxl_port *port; struct access_coordinate coord; + long link_latency; }; /** diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 0fa4799ea316..466563536a92 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -85,6 +85,19 @@ struct cdat_entry_header { __le16 length; } __packed; +/* + * CXL v3.0 6.2.3 Table 6-4 + * The table indicates that if PCIe Flit Mode is set, then CXL is in 256B flits + * mode, otherwise it's 68B flits mode. + */ +static inline bool cxl_pci_flit_256(struct pci_dev *pdev) +{ + u16 lnksta2; + + pcie_capability_read_word(pdev, PCI_EXP_LNKSTA2, &lnksta2); + return lnksta2 & PCI_EXP_LNKSTA2_FLIT; +} + int devm_cxl_port_enumerate_dports(struct cxl_port *port); struct cxl_dev_state; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, @@ -93,4 +106,6 @@ void read_cdat_data(struct cxl_port *port); void cxl_cor_error_detected(struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state); +long cxl_pci_get_latency(struct pci_dev *pdev); + #endif /* __CXL_PCI_H__ */ From patchwork Thu Oct 12 18:55:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419641 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C85F02AB39 for ; Thu, 12 Oct 2023 18:55:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EbtkrpcO" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9E0483 for ; Thu, 12 Oct 2023 11:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136927; x=1728672927; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dGvthp3qvPH1lghkUXhbfXG/Iv6vqSOgP0V0xvaNpOI=; b=EbtkrpcO1RNP1i4GmAVdsPpM4yAgzIBqzVVlNIEHaOIXNA2Go+j0J9h4 pMNd49yK2+AmkPlM1aC1YO5/hU0HCQ+d18Us0nzD9y4c9fXvEtQqY7S9S 4bIA7G4Ng76i/aFNKwghv+9rUa6VxsjeRDO8yWNnhmSrfEqn3AuYoDs3n oiDJDOcw3HglgdIf/bJAgeeWhLgVBn6lqeyNifrupQMZwZifwsCQS0HjT Fj7rlRpf+xp0mcdflOhIt8Y4zZhtbcInWtUYoOSMsH3e/fa/BWz7/fpV+ LoMkLqib0GmPlKVfDVVwr8BnrXe8YodsGqcLZV/IGnK3S56Gt3eMmcy2P w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604209" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604209" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724538" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724538" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:13 -0700 Subject: [PATCH v11 17/22] cxl: Store the access coordinates for the generic ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:12 -0700 Message-ID: <169713691235.2205276.8146804174143909257.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Each CXL host bridge is represented by an ACPI0016 device. A generic port device handle that is an ACPI device is represented by a string of ACPI0016 device HID and UID. Create a device handle from the ACPI device and retrieve the access coordinates from the stored memory targets. The access coordinates are stored under the cxl_dport that is associated with the CXL host bridge. The access coordinates struct is dynamically allocated under cxl_dport in order for code later on to detect whether the data exists or not. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/acpi.c | 17 +++++++++++++++++ drivers/cxl/cxl.h | 2 ++ 2 files changed, 19 insertions(+) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 1eb477e8274c..cfce4cc7a719 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -513,8 +513,21 @@ static int cxl_get_chbs(struct device *dev, struct acpi_device *hb, return 0; } +static int get_genport_coordinates(struct device *dev, struct cxl_dport *dport) +{ + struct acpi_device *hb = to_cxl_host_bridge(NULL, dev); + u8 handle[ACPI_SRAT_DEVICE_HANDLE_SIZE] = { 0 }; + + /* ACPI spec 6.5 table 5.65 */ + strncpy(handle, acpi_device_hid(hb), 8); + strncpy(&handle[8], acpi_device_uid(hb), 4); + + return acpi_get_genport_coordinates(handle, &dport->hb_access); +} + static int add_host_bridge_dport(struct device *match, void *arg) { + int ret; acpi_status rc; struct device *bridge; struct cxl_dport *dport; @@ -564,6 +577,10 @@ static int add_host_bridge_dport(struct device *match, void *arg) if (IS_ERR(dport)) return PTR_ERR(dport); + ret = get_genport_coordinates(match, dport); + if (ret) + dev_dbg(match, "Failed to get generic port perf coordinates.\n"); + return 0; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f1c43f02d65e..f7bbd57d9bcf 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -655,6 +655,7 @@ struct cxl_rcrb_info { * @rcrb: Data about the Root Complex Register Block layout * @rch: Indicate whether this dport was enumerated in RCH or VH mode * @port: reference to cxl_port that contains this downstream port + * @genport_coord: access coordinates (performance) from ACPI generic port * @coord: access coordinates (performance) for switch from CDAT * @link_latency: calculated PCIe downstream latency */ @@ -665,6 +666,7 @@ struct cxl_dport { struct cxl_rcrb_info rcrb; bool rch; struct cxl_port *port; + struct access_coordinate hb_access; struct access_coordinate coord; long link_latency; }; From patchwork Thu Oct 12 18:55:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419643 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6D342AB39 for ; Thu, 12 Oct 2023 18:55:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VkGrtSLm" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36193B7 for ; Thu, 12 Oct 2023 11:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136929; x=1728672929; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CsRJaJKNlMehuieIIo/z1J7SQNo1ckZtn/RT1MT43TY=; b=VkGrtSLmu8ox+RJQrQWhOoHgQxg+iSz98MsW1G0+ffpOnrtIUwOXCos0 /+MdGRPos9eflv22XqS7OnoZlJlF0WRj1+T/H1iPEUHIYUt9t7r/96FXY T8oIR9Yv8rh0Gu5ARJUwpHGdmJj20QP8SrBy7tCVHr10A4RwWQSfYqYWp Jtf9zMK+0omcKUA/rCbU0yu0TmUMtdMTrwPou/hqKRo/9ZW9XjFYtvU9K ktuTdAFcVdVusc9R1rGeIUXz9Fv+O4elSp/hJjdFBepAeSRreCDIDUCYs gns6NDjGT8MxxUk0nSY1RrTRWV8LfLYo28bOGqT6WPnO+YvFaz3pHNo81 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604234" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604234" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724593" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724593" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:19 -0700 Subject: [PATCH v11 18/22] cxl: Add helper function that calculate performance data for downstream ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:18 -0700 Message-ID: <169713691844.2205276.3210626628611814563.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The CDAT information from the switch, Switch Scoped Latency and Bandwidth Information Strucutre (SSLBIS), is parsed and stored under a cxl_dport based on the correlated downstream port id from the SSLBIS entry. Walk the entire CXL port paths and collect all the performance data. Also pick up the link latency number that's stored under the dports. The entire path PCIe bandwidth can be retrieved using the pcie_bandwidth_available() call. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/core/port.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 ++ 2 files changed, 69 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 3b45da2b425e..f2b4f9048f2e 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -2016,6 +2017,71 @@ bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, CXL); +static void combine_coordinates(struct access_coordinate *c1, + struct access_coordinate *c2) +{ + if (c2->write_bandwidth) + c1->write_bandwidth = min(c1->write_bandwidth, + c2->write_bandwidth); + c1->write_latency += c2->write_latency; + + if (c2->read_bandwidth) + c1->read_bandwidth = min(c1->read_bandwidth, + c2->read_bandwidth); + c1->read_latency += c2->read_latency; +} + +/** + * cxl_endpoint_get_perf_coordinates - Retrieve performance numbers stored in dports + * of CXL path + * @port: endpoint cxl_port + * @coord: output performance data + * + * Return: errno on failure, 0 on success. + */ +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord) +{ + struct access_coordinate c = { + .read_bandwidth = UINT_MAX, + .write_bandwidth = UINT_MAX, + }; + struct cxl_port *iter = port; + struct cxl_dport *dport; + struct pci_dev *pdev; + unsigned int bw; + + if (!is_cxl_endpoint(port)) + return -EINVAL; + + dport = iter->parent_dport; + while (iter && !is_cxl_root(iter)) { + combine_coordinates(&c, &dport->coord); + c.write_latency += dport->link_latency; + c.read_latency += dport->link_latency; + + combine_coordinates(&c, &dport->hb_access); + + iter = to_cxl_port(iter->dev.parent); + dport = iter->parent_dport; + } + + /* Get the calculated PCI paths bandwidth */ + pdev = to_pci_dev(port->uport_dev->parent); + bw = pcie_bandwidth_available(pdev, NULL, NULL, NULL); + if (bw == 0) + return -ENXIO; + bw /= BITS_PER_BYTE; + + c.write_bandwidth = min(c.write_bandwidth, bw); + c.read_bandwidth = min(c.read_bandwidth, bw); + + *coord = c; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_endpoint_get_perf_coordinates, CXL); + /* for user tooling to ensure port disable work has completed */ static ssize_t flush_store(const struct bus_type *bus, const char *buf, size_t count) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f7bbd57d9bcf..61b58b25834d 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -888,6 +888,9 @@ static inline int cxl_cdat_switch_process(struct cxl_port *port) } #endif +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. From patchwork Thu Oct 12 18:55:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419642 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18DDA358B5 for ; Thu, 12 Oct 2023 18:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="L8TEckoy" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D38CF83 for ; Thu, 12 Oct 2023 11:55:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136930; x=1728672930; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qpYA+mxsekueWxUrGQhYL9tjY3MrrmDar386K3k/iN8=; b=L8TEckoyMipY0AsWrVCaB70eYRlJwtUi0L9othf3v98sz/QLw2tkFAVe GQKgZF70k6rIizCLxrxET22UnMQ2GNYPRdvurYMllZ1HVrngiqG/XnIcG p7hkLi3xMl4HjHUUoKSGHxGu2Bwjwm0RAJ7PJ9EaHY2MQBuaqqxwGVBEF XNRoTDqaRTSqVwe7IVznLDVqncQYDjb4RfgwLyRfXGi/iDj3TE0SxPuy0 N6xCYlvalEG0WBivzRuK0m0autbXgnfPaH6a51/Euhu5JvsabduGjOXOE CQbNML3CApOkur1L70PBbf5kJaglkKswf/aYfuazs0OvPoEe9yM8iKkl0 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604267" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604267" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724621" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724621" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:24 -0700 Subject: [PATCH v11 19/22] cxl: Compute the entire CXL path latency and bandwidth data From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:24 -0700 Message-ID: <169713692442.2205276.676765260116439939.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net CXL Memory Device SW Guide [1] rev1.0 2.11.2 provides instruction on how to calculate latency and bandwidth for CXL memory device. Calculate minimum bandwidth and total latency for the path from the CXL device to the root port. The QTG id is retrieved by providing the performance data as input and calling the root port callback ->get_qos_class(). The retrieved id is stored with the cxl_port of the CXL device. For example for a device that is directly attached to a host bus: Total Latency = Device Latency (from CDAT) + Dev to Host Bus (HB) Link Latency + Generic Port Latency Min Bandwidth = Min bandwidth for link bandwidth between HB and CXL device, device CDAT bandwidth, and Generic Port Bandwidth For a device that has a switch in between host bus and CXL device: Total Latency = Device (CDAT) Latency + Dev to Switch Link Latency + Switch (CDAT) Latency + Switch to HB Link Latency + Generic Port Latency Min Bandwidth = Min bandwidth for link bandwidth between CXL device to CXL switch, CXL device CDAT bandwidth, CXL switch CDAT bandwidth, CXL switch to HB bandwidth, and Generic Port Bandwidth. [1]: https://cdrdv2-public.intel.com/643805/643805_CXL%20Memory%20Device%20SW%20Guide_Rev1p0.pdf Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v10: - Update to new get_qos_class() API call. Only retrieve 1 QTG ID since this is what the kernel is using. Can expand to retrieve additional IDs in the future when needed. (Dan) --- drivers/cxl/cxl.h | 4 ++++ drivers/cxl/port.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 59 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 61b58b25834d..9569ff70ea6e 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -859,12 +859,16 @@ static inline struct cxl_dax_region *to_cxl_dax_region(struct device *dev) } #endif + /* CDAT related bits */ struct dsmas_entry { struct list_head list; struct range dpa_range; u8 handle; struct access_coordinate coord; + + int entries; + int qos_class; }; #ifdef CONFIG_FIRMWARE_TABLE diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index f5e2be79f217..99a619360bc5 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -57,6 +57,54 @@ static int discover_region(struct device *dev, void *root) return 0; } +static int cxl_port_perf_data_calculate(struct cxl_port *port, + struct list_head *dsmas_list) +{ + struct access_coordinate c; + struct cxl_port *root_port; + struct cxl_root *cxl_root; + struct dsmas_entry *dent; + int valid_entries = 0; + int rc; + + rc = cxl_endpoint_get_perf_coordinates(port, &c); + if (rc) { + dev_dbg(&port->dev, "Failed to retrieve perf coordinates.\n"); + return rc; + } + + root_port = find_cxl_root(port); + cxl_root = to_cxl_root(root_port); + if (!cxl_root->ops || !cxl_root->ops->get_qos_class) + return -EOPNOTSUPP; + + list_for_each_entry(dent, dsmas_list, list) { + int qos_class; + + dent->coord.read_latency = dent->coord.read_latency + + c.read_latency; + dent->coord.write_latency = dent->coord.write_latency + + c.write_latency; + dent->coord.read_bandwidth = min_t(int, c.read_bandwidth, + dent->coord.read_bandwidth); + dent->coord.write_bandwidth = min_t(int, c.write_bandwidth, + dent->coord.write_bandwidth); + + dent->entries = 1; + rc = cxl_root->ops->get_qos_class(root_port, &dent->coord, 1, &qos_class); + if (rc != 1) + continue; + + valid_entries++; + dent->qos_class = qos_class; + } + + if (!valid_entries) + return -ENOENT; + + return 0; +} + static int cxl_switch_port_probe(struct cxl_port *port) { struct cxl_hdm *cxlhdm; @@ -146,10 +194,14 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) LIST_HEAD(dsmas_list); rc = cxl_cdat_endpoint_process(port, &dsmas_list); - if (rc < 0) + if (rc < 0) { dev_dbg(&port->dev, "Failed to parse CDAT: %d\n", rc); - - /* Performance data processing */ + } else { + rc = cxl_port_perf_data_calculate(port, &dsmas_list); + if (rc) + dev_dbg(&port->dev, + "Failed to do perf coord calculations.\n"); + } cxl_cdat_dsmas_list_destroy(&dsmas_list); } From patchwork Thu Oct 12 18:55:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419644 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E6F6358B5 for ; Thu, 12 Oct 2023 18:55:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="apw8YUBV" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05583C0 for ; Thu, 12 Oct 2023 11:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136932; x=1728672932; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7MhPfyiIXMaWQR9Wmb8Dak0RR8zuyj/7tltszk38rpQ=; b=apw8YUBVxNrQ5vvEd1BTf4ZcRrOHz1KzG10CHsfphu1XUwNX/EV19Ugh PQgj1/JYjrCcjcYq8YcYmec32L/00PzFIKfgk0lSDhbGCkEC29u2Y1DK4 EIJgaNo5lGfUEacC2tdzaeOqjlcdFbrkwoyKuwqmbZX337LbGKeSsF5P/ GQ25v0Tp07ibIMR15YL90C+8kGyo/V7BLvKddDDcziiBQKTdYBPJz3uks JNwieYaRHxKjfQfb7RKL/a2v+CVdenqBxA7PENCgxZ2qmYFzTajA036VI DZrCnH3U0AMA3uKvUYPzd7R+yIxzzJnlbZwLUZy4HBZBC3YH3yh5A6a6r w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604289" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604289" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724642" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724642" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:30 -0700 Subject: [PATCH v11 20/22] cxl: Store QTG IDs and related info to the CXL memory device context From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:30 -0700 Message-ID: <169713693022.2205276.8814476945721343862.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Once the QTG ID _DSM is executed successfully, the QTG ID is retrieved from the return package. Create a list of entries in the cxl_memdev context and store the QTG ID as qos_class token and the associated DPA range. This information can be exposed to user space via sysfs in order to help region setup for hot-plugged CXL memory devices. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v11: - Detected multiple entries and emit such case. (Jonathan) - Preserve first found entry if there are multiple entries. - Refactor dsmas processing paths in switch port (Jonathan) v10: - Store single qos_class value. (Dan) - Rename cxl_memdev_set_qtg() to cxl_memdev_set_qos_class() - Removed Jonathan's review tag due to code changes. --- drivers/cxl/core/mbox.c | 1 + drivers/cxl/cxlmem.h | 23 ++++++++++++++++++ drivers/cxl/port.c | 60 ++++++++++++++++++++++++++++++++++++++++++----- 3 files changed, 78 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 4df4f614f490..6193b8d57469 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1378,6 +1378,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mutex_init(&mds->event.log_lock); mds->cxlds.dev = dev; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; + INIT_LIST_HEAD(&mds->perf_list); return mds; } diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 706f8a6d1ef4..a310a51a3fa4 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -6,6 +6,7 @@ #include #include #include +#include #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */ @@ -388,6 +389,20 @@ enum cxl_devtype { CXL_DEVTYPE_CLASSMEM, }; +/** + * struct perf_prop - performance property entry + * @list - list entry + * @dpa_range - range for DPA address + * @coord - QoS performance data (i.e. latency, bandwidth) + * @qos_class - QoS Class cookies + */ +struct perf_prop_entry { + struct list_head list; + struct range dpa_range; + struct access_coordinate coord; + int qos_class; +}; + /** * struct cxl_dev_state - The driver device state * @@ -452,6 +467,9 @@ struct cxl_dev_state { * @security: security driver state info * @fw: firmware upload / activation state * @mbox_send: @dev specific transport for transmitting mailbox commands + * @ram_qos_class: QoS class cookies for volatile region + * @pmem_qos_class: QoS class cookies for persistent region + * @perf_list: performance data entries list * * See CXL 3.0 8.2.9.8.2 Capacity Configuration and Label Storage for * details on capacity parameters. @@ -472,6 +490,11 @@ struct cxl_memdev_state { u64 active_persistent_bytes; u64 next_volatile_bytes; u64 next_persistent_bytes; + + int ram_qos_class; + int pmem_qos_class; + struct list_head perf_list; + struct cxl_event_state event; struct cxl_poison_state poison; struct cxl_security_state security; diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 99a619360bc5..7eb26cefe2cb 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -105,6 +105,49 @@ static int cxl_port_perf_data_calculate(struct cxl_port *port, return 0; } +static void cxl_memdev_set_qos_class(struct cxl_dev_state *cxlds, + struct list_head *dsmas_list) +{ + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + struct range pmem_range = { + .start = cxlds->pmem_res.start, + .end = cxlds->pmem_res.end, + }; + struct range ram_range = { + .start = cxlds->ram_res.start, + .end = cxlds->ram_res.end, + }; + struct perf_prop_entry *perf; + struct dsmas_entry *dent; + + list_for_each_entry(dent, dsmas_list, list) { + perf = devm_kzalloc(cxlds->dev, sizeof(*perf), GFP_KERNEL); + if (!perf) + return; + + perf->dpa_range = dent->dpa_range; + perf->coord = dent->coord; + perf->qos_class = dent->qos_class; + list_add_tail(&perf->list, &mds->perf_list); + + if (resource_size(&cxlds->ram_res) && + range_contains(&ram_range, &dent->dpa_range)) { + if (mds->ram_qos_class == CXL_QOS_CLASS_INVALID) + mds->ram_qos_class = perf->qos_class; + else + dev_dbg(cxlds->dev, + "Multiple DSMAS entries for ram region.\n"); + } else if (resource_size(&cxlds->pmem_res) && + range_contains(&pmem_range, &dent->dpa_range)) { + if (mds->pmem_qos_class == CXL_QOS_CLASS_INVALID) + mds->pmem_qos_class = perf->qos_class; + else + dev_dbg(cxlds->dev, + "Multiple DSMAS entries for pmem region.\n"); + } + } +} + static int cxl_switch_port_probe(struct cxl_port *port) { struct cxl_hdm *cxlhdm; @@ -196,17 +239,22 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) rc = cxl_cdat_endpoint_process(port, &dsmas_list); if (rc < 0) { dev_dbg(&port->dev, "Failed to parse CDAT: %d\n", rc); - } else { - rc = cxl_port_perf_data_calculate(port, &dsmas_list); - if (rc) - dev_dbg(&port->dev, - "Failed to do perf coord calculations.\n"); + goto out; } + rc = cxl_port_perf_data_calculate(port, &dsmas_list); + if (rc) { + dev_dbg(&port->dev, + "Failed to do perf coord calculations.\n"); + goto out; + } + + cxl_memdev_set_qos_class(cxlds, &dsmas_list); +out: cxl_cdat_dsmas_list_destroy(&dsmas_list); } - return 0; + return rc; } static int cxl_port_probe(struct device *dev) From patchwork Thu Oct 12 18:55:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419645 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3609C2AB39 for ; Thu, 12 Oct 2023 18:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WWt4scM0" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54039BE for ; Thu, 12 Oct 2023 11:55:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136938; x=1728672938; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R0enbTvRz5kqpMulkqEk0Lfi71F21mkrstOjvIlamWY=; b=WWt4scM0AWhOYjLfiQ2n3oTNC2MAiyoqh3+INcMz7l8PTI2zEYgJMswB 0mm6Fxgx9L5ght9/QXRzgNXKVvYFLpq6hZuzfpo/o01daitKYIQKa89A6 kWME/lJzskehQtEnviAVMPAtOwRTkNeOJLIqKXrJyS9o2X7NRdPbmD5Io nw3OMnUhKTMp7PSIeEc6DmJddFxacV/F+PpiKH/ANl7LU7HVygAP7oQ6P pjoJDNOCD/tEq3yOm5zaBMwseG0wEiN0XG2XRRn0sd678PznH1OHMIZ65 hzDRC9bgOv8v32VRJN8+hRlctI960N8kE6WPyUXuIsWkCb3oYBNHEX+Jq w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604330" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604330" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724692" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724692" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:36 -0700 Subject: [PATCH v11 21/22] cxl: Export sysfs attributes for memory device QoS class From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Dan Williams , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:36 -0700 Message-ID: <169713693604.2205276.7697288344292256806.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Export qos_class sysfs attributes for the CXL memory device. The QoS clas should show up as /sys/bus/cxl/devices/memX/ram/qos_class for the volatile partition and /sys/bus/cxl/devices/memX/pmem/qos_class for the persistent partition. The QTG ID is retrieved via _DSM after supplying the calculated bandwidth and latency for the entire CXL path from device to the CPU. This ID is used to match up to the root decoder QoS class to determine which CFMWS the memory range of a hotplugged CXL mem device should be assigned under. While there may be multiple DSMAS exported by the device CDAT, the driver will only expose the first QTG ID per partition in sysfs for now. In the future when multiple QTG IDs are necessary, they can be exposed. [1] [1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab Suggested-by: Dan Williams Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v11: - Move qos_class0 to qos_class (Dan) v10: - Export only qos_class0, the first entry. Additional qos_class entries can be exported later as needed. (Dan) - Have the sysfs attrib return -ENOENT unless driver is attached. (Dan) - Removed Jonathan's review tag due to code changes. --- Documentation/ABI/testing/sysfs-bus-cxl | 34 +++++++++++++++++++++++++++++++ drivers/cxl/core/memdev.c | 34 +++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 44ffbbb36654..97c1e6bca888 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -28,6 +28,23 @@ Description: Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/ram/qos_class +Date: May, 2023 +KernelVersion: v6.7 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" + this attribute conveys a comma delimited list of platform + specific cookies that identifies a QoS performance class + for the volatile partition of the CXL mem device. These + class-ids can be compared against a similar "qos_class" + published for a root decoder. While it is not required + that the endpoints map their local memory-class to a + matching platform class, mismatches are not recommended + and there are platform specific performance related + side-effects that may result. First class-id is displayed. + + What: /sys/bus/cxl/devices/memX/pmem/size Date: December, 2020 KernelVersion: v5.12 @@ -38,6 +55,23 @@ Description: Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/pmem/qos_class +Date: May, 2023 +KernelVersion: v6.7 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" + this attribute conveys a comma delimited list of platform + specific cookies that identifies a QoS performance class + for the persistent partition of the CXL mem device. These + class-ids can be compared against a similar "qos_class" + published for a root decoder. While it is not required + that the endpoints map their local memory-class to a + matching platform class, mismatches are not recommended + and there are platform specific performance related + side-effects that may result. First class-id is displayed. + + What: /sys/bus/cxl/devices/memX/serial Date: January, 2022 KernelVersion: v5.18 diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 14b547c07f54..7a501fd25fe7 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -88,6 +88,22 @@ static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_ram_size = __ATTR(size, 0444, ram_size_show, NULL); +static ssize_t ram_qos_class_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + + if (!dev->driver) + return -ENOENT; + + return sysfs_emit(buf, "%d\n", mds->ram_qos_class); +} + +static struct device_attribute dev_attr_ram_qos_class = + __ATTR(qos_class, 0444, ram_qos_class_show, NULL); + static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -101,6 +117,22 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_pmem_size = __ATTR(size, 0444, pmem_size_show, NULL); +static ssize_t pmem_qos_class_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + + if (!dev->driver) + return -ENOENT; + + return sysfs_emit(buf, "%d\n", mds->pmem_qos_class); +} + +static struct device_attribute dev_attr_pmem_qos_class = + __ATTR(qos_class, 0444, pmem_qos_class_show, NULL); + static ssize_t serial_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -439,11 +471,13 @@ static struct attribute *cxl_memdev_attributes[] = { static struct attribute *cxl_memdev_pmem_attributes[] = { &dev_attr_pmem_size.attr, + &dev_attr_pmem_qos_class.attr, NULL, }; static struct attribute *cxl_memdev_ram_attributes[] = { &dev_attr_ram_size.attr, + &dev_attr_ram_qos_class.attr, NULL, }; From patchwork Thu Oct 12 18:55:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419646 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 859BE2AB39 for ; Thu, 12 Oct 2023 18:55:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TYfinP+i" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A16083 for ; Thu, 12 Oct 2023 11:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136944; x=1728672944; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Us4xbz1jEw5CIMuHHOMNuigmLVLkQCoc0iTk+tbOmYA=; b=TYfinP+iweWeeVwlYEFnVKa2ZG8Ii1GfFGBPwCOWfQHRMlKnOgEXNr/Z Mz9lTB7jA2WaRu6dOuhIRYkPUOFwavTMJ/Kx+0/MlgzuHJh8scSKtOMQX acMMXGYqcP/XUUusiwQQJPS1HnS4rYGW8x+SZFsSFgpqbkOGYFUd37N5M 9y4BeLUzKIcAKdGy2wlI+Hnm0QfcXah7+OIyU+kc3F3AirHQhhJYDnktG B63rpUEj49ckPZv36mRc8bOiP7QRhGpnOQlao8Kp6dbiYOPkSLjK/cI+a YKTEzHJq+W6JVT4UBEEbS1DZV51wY004eOSSW2R1yIy34R6M/6fOW4U8u w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="388880518" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="388880518" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="870711774" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="870711774" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:42 -0700 Subject: [PATCH v11 22/22] cxl: Check qos_class validity on memdev probe From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:41 -0700 Message-ID: <169713694184.2205276.9863653630713216825.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add a check to make sure the qos_class for the device will match one of the root decoders qos_class. If no match is found, then the qos_class for the device is set to invalid. Also add a check to ensure that the device's host bridge matches to one of the root decoder's downstream targets. Signed-off-by: Dave Jiang --- v11: - Return when matched in match function (Jonathan) - Don't return after matched in caller function, still need to check pmem. (Jonathan) - Fix copy/paste error for pmem_qos_class. (Jonathan) - Use device_for_each_child() instead of bus_for_each_dev(). (Dan) - Add match of host_bridge to a root decoder target. (Dan) --- drivers/cxl/mem.c | 121 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 317c7548e4e9..74f72e37ccfc 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -104,6 +104,123 @@ static int cxl_debugfs_poison_clear(void *data, u64 dpa) DEFINE_DEBUGFS_ATTRIBUTE(cxl_poison_clear_fops, NULL, cxl_debugfs_poison_clear, "%llx\n"); +struct qos_class_ctx { + bool matched; + int dev_qos_class; +}; + +static int match_cxlrd_qos_class(struct device *dev, void *data) +{ + struct qos_class_ctx *ctx = data; + struct cxl_root_decoder *cxlrd; + + if (!is_root_decoder(dev)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + if (cxlrd->qos_class == CXL_QOS_CLASS_INVALID || + ctx->dev_qos_class == CXL_QOS_CLASS_INVALID) + return 0; + + if (cxlrd->qos_class == ctx->dev_qos_class) { + ctx->matched = 1; + return 1; + } + + return 0; +} + +struct qos_hb_ctx { + bool matched; + struct device *host_bridge; +}; + +static int match_cxlrd_hb(struct device *dev, void *data) +{ + struct cxl_switch_decoder *cxlsd; + struct qos_hb_ctx *ctx = data; + struct cxl_root_decoder *cxlrd; + unsigned int seq; + + if (!is_root_decoder(dev)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + cxlsd = &cxlrd->cxlsd; + + do { + seq = read_seqbegin(&cxlsd->target_lock); + for (int i = 0; i < cxlsd->nr_targets; i++) { + if (ctx->host_bridge == + cxlsd->target[i]->dport_dev) { + ctx->matched = true; + return 1; + } + } + } while (read_seqretry(&cxlsd->target_lock, seq)); + + return 0; +} + +static int cxl_qos_class_verify(struct cxl_memdev *cxlmd) +{ + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + struct qos_class_ctx ctx; + struct qos_hb_ctx hbctx; + struct cxl_port *root_port; + int rc; + + root_port = find_cxl_root(cxlmd->endpoint); + if (!root_port) + return -ENODEV; + + /* Check that the QTG IDs are all sane between end device and root decoders */ + if (mds->ram_qos_class != CXL_QOS_CLASS_INVALID) { + ctx = (struct qos_class_ctx) { + .matched = false, + .dev_qos_class = mds->ram_qos_class, + }; + rc = device_for_each_child(&root_port->dev, &ctx, match_cxlrd_qos_class); + if (rc < 0) + goto out; + + if (!ctx.matched) + mds->ram_qos_class = CXL_QOS_CLASS_INVALID; + } + + if (mds->pmem_qos_class != CXL_QOS_CLASS_INVALID) { + ctx = (struct qos_class_ctx) { + .matched = false, + .dev_qos_class = mds->pmem_qos_class, + }; + rc = device_for_each_child(&root_port->dev, &ctx, match_cxlrd_qos_class); + if (rc < 0) + goto out; + + if (!ctx.matched) + mds->pmem_qos_class = CXL_QOS_CLASS_INVALID; + } + + /* Check to make sure that the device's host bridge is under a root decoder */ + hbctx = (struct qos_hb_ctx) { + .matched = false, + .host_bridge = cxlmd->endpoint->host_bridge, + }; + rc = device_for_each_child(&root_port->dev, &hbctx, match_cxlrd_hb); + if (rc < 0) + goto out; + + if (!hbctx.matched) { + mds->ram_qos_class = CXL_QOS_CLASS_INVALID; + mds->pmem_qos_class = CXL_QOS_CLASS_INVALID; + } + +out: + put_device(&root_port->dev); + return rc; +} + static int cxl_mem_probe(struct device *dev) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); @@ -173,6 +290,10 @@ static int cxl_mem_probe(struct device *dev) if (rc) return rc; + rc = cxl_qos_class_verify(cxlmd); + if (rc < 0) + return rc; + if (resource_size(&cxlds->pmem_res) && IS_ENABLED(CONFIG_CXL_PMEM)) { rc = devm_cxl_add_nvdimm(cxlmd); if (rc == -ENODEV)