From patchwork Thu Oct 12 18:55:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13419643 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6D342AB39 for ; Thu, 12 Oct 2023 18:55:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VkGrtSLm" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36193B7 for ; Thu, 12 Oct 2023 11:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697136929; x=1728672929; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CsRJaJKNlMehuieIIo/z1J7SQNo1ckZtn/RT1MT43TY=; b=VkGrtSLmu8ox+RJQrQWhOoHgQxg+iSz98MsW1G0+ffpOnrtIUwOXCos0 /+MdGRPos9eflv22XqS7OnoZlJlF0WRj1+T/H1iPEUHIYUt9t7r/96FXY T8oIR9Yv8rh0Gu5ARJUwpHGdmJj20QP8SrBy7tCVHr10A4RwWQSfYqYWp Jtf9zMK+0omcKUA/rCbU0yu0TmUMtdMTrwPou/hqKRo/9ZW9XjFYtvU9K ktuTdAFcVdVusc9R1rGeIUXz9Fv+O4elSp/hJjdFBepAeSRreCDIDUCYs gns6NDjGT8MxxUk0nSY1RrTRWV8LfLYo28bOGqT6WPnO+YvFaz3pHNo81 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="3604234" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="3604234" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10861"; a="824724593" X-IronPort-AV: E=Sophos;i="6.03,219,1694761200"; d="scan'208";a="824724593" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.15.16]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Oct 2023 11:55:19 -0700 Subject: [PATCH v11 18/22] cxl: Add helper function that calculate performance data for downstream ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Date: Thu, 12 Oct 2023 11:55:18 -0700 Message-ID: <169713691844.2205276.3210626628611814563.stgit@djiang5-mobl3> In-Reply-To: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> References: <169713674328.2205276.10184241477215488339.stgit@djiang5-mobl3> User-Agent: StGit/1.5 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The CDAT information from the switch, Switch Scoped Latency and Bandwidth Information Strucutre (SSLBIS), is parsed and stored under a cxl_dport based on the correlated downstream port id from the SSLBIS entry. Walk the entire CXL port paths and collect all the performance data. Also pick up the link latency number that's stored under the dports. The entire path PCIe bandwidth can be retrieved using the pcie_bandwidth_available() call. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- drivers/cxl/core/port.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 ++ 2 files changed, 69 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 3b45da2b425e..f2b4f9048f2e 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -2016,6 +2017,71 @@ bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, CXL); +static void combine_coordinates(struct access_coordinate *c1, + struct access_coordinate *c2) +{ + if (c2->write_bandwidth) + c1->write_bandwidth = min(c1->write_bandwidth, + c2->write_bandwidth); + c1->write_latency += c2->write_latency; + + if (c2->read_bandwidth) + c1->read_bandwidth = min(c1->read_bandwidth, + c2->read_bandwidth); + c1->read_latency += c2->read_latency; +} + +/** + * cxl_endpoint_get_perf_coordinates - Retrieve performance numbers stored in dports + * of CXL path + * @port: endpoint cxl_port + * @coord: output performance data + * + * Return: errno on failure, 0 on success. + */ +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord) +{ + struct access_coordinate c = { + .read_bandwidth = UINT_MAX, + .write_bandwidth = UINT_MAX, + }; + struct cxl_port *iter = port; + struct cxl_dport *dport; + struct pci_dev *pdev; + unsigned int bw; + + if (!is_cxl_endpoint(port)) + return -EINVAL; + + dport = iter->parent_dport; + while (iter && !is_cxl_root(iter)) { + combine_coordinates(&c, &dport->coord); + c.write_latency += dport->link_latency; + c.read_latency += dport->link_latency; + + combine_coordinates(&c, &dport->hb_access); + + iter = to_cxl_port(iter->dev.parent); + dport = iter->parent_dport; + } + + /* Get the calculated PCI paths bandwidth */ + pdev = to_pci_dev(port->uport_dev->parent); + bw = pcie_bandwidth_available(pdev, NULL, NULL, NULL); + if (bw == 0) + return -ENXIO; + bw /= BITS_PER_BYTE; + + c.write_bandwidth = min(c.write_bandwidth, bw); + c.read_bandwidth = min(c.read_bandwidth, bw); + + *coord = c; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_endpoint_get_perf_coordinates, CXL); + /* for user tooling to ensure port disable work has completed */ static ssize_t flush_store(const struct bus_type *bus, const char *buf, size_t count) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f7bbd57d9bcf..61b58b25834d 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -888,6 +888,9 @@ static inline int cxl_cdat_switch_process(struct cxl_port *port) } #endif +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/.