From patchwork Mon May 8 20:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13235060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8C68C77B7F for ; Mon, 8 May 2023 20:47:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232643AbjEHUrj (ORCPT ); Mon, 8 May 2023 16:47:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbjEHUri (ORCPT ); Mon, 8 May 2023 16:47:38 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90F9655A9 for ; Mon, 8 May 2023 13:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683578857; x=1715114857; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6X1f545mj1OVHPb/RNhze5zXFRbLTLVYEFvGxjtabeo=; b=ne28Im5RD5oo/iw4Kn7FUHxWV+4HbEZj7zBB4104jc6JLMWuI6mTA2Rq OJsr3fIcM/r6UrnWcarDp/q3w/heQ82ZsRwKmrSuAk5UuWdsTk0AQoIAU zBK33GqBNN+6iMcpIH3Jd7BtEWI4KGwm27hkhcMeeaVdz5VAZR/jhr+m2 REmFqLpacSjMCAtcFymjcTM0kNgVaDvqaWGYX759w5qMOWYM6wVSrkp/E aNpFYEsARJapRvtLK2IF1ZVbLT3He25Y2xtQT2f4UbYbMMef/7zDx7ItV Z7mA0SAI7gRzZnhen3X9Rzbt1s52oj4bSVeWQWnt+7SfhQnTEfWsL3Ml9 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10704"; a="348580605" X-IronPort-AV: E=Sophos;i="5.99,259,1677571200"; d="scan'208";a="348580605" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 May 2023 13:47:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10704"; a="822826906" X-IronPort-AV: E=Sophos;i="5.99,259,1677571200"; d="scan'208";a="822826906" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.213.172.228]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 May 2023 13:47:36 -0700 Subject: [PATCH v5 07/14] cxl: Add helper function that calculate performance data for downstream ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com Date: Mon, 08 May 2023 13:47:36 -0700 Message-ID: <168357885604.2756219.14660517574518073528.stgit@djiang5-mobl3> In-Reply-To: <168357873843.2756219.5839806150467356492.stgit@djiang5-mobl3> References: <168357873843.2756219.5839806150467356492.stgit@djiang5-mobl3> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The CDAT information from the switch, Switch Scoped Latency and Bandwidth Information Strucutre (SSLBIS), is parsed and stored under a cxl_dport based on the correlated downstream port id from the SSLBIS entry. Walk the entire CXL port paths and collect all the performance data. Also pick up the link latency number that's stored under the dports. The entire path PCIe bandwidth can be retrieved using the pcie_bandwidth_available() call. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- drivers/cxl/core/port.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 ++ 2 files changed, 72 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e4f75847b851..1111a3cebc8e 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -1950,6 +1951,74 @@ bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, CXL); +static void combine_coordinates(struct access_coordinate *c1, + struct access_coordinate *c2) +{ + if (c2->write_bandwidth) + c1->write_bandwidth = min_t(unsigned int, + c1->write_bandwidth, + c2->write_bandwidth); + c1->write_latency += c2->write_latency; + + if (c2->read_bandwidth) + c1->read_bandwidth = min_t(unsigned int, + c1->read_bandwidth, + c2->read_bandwidth); + c1->read_latency += c2->read_latency; +} + +/** + * cxl_port_get_perf_coordinates - Retrieve performance numbers stored in dports + * of CXL path + * @port: endpoint cxl_port + * @coord: output performance data + * + * Return: errno on failure, 0 on success. + */ +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord) +{ + struct access_coordinate c = { + .read_bandwidth = UINT_MAX, + .write_bandwidth = UINT_MAX, + }; + struct cxl_port *iter = port; + struct cxl_dport *dport; + struct pci_dev *pdev; + unsigned int bw; + + if (!is_cxl_endpoint(port)) + return -EINVAL; + + dport = iter->parent_dport; + while (iter && !is_cxl_root(iter)) { + combine_coordinates(&c, &dport->coord); + c.write_latency += dport->link_latency; + c.read_latency += dport->link_latency; + + if (dport->genport_coord) + combine_coordinates(&c, dport->genport_coord); + + iter = to_cxl_port(iter->dev.parent); + dport = iter->parent_dport; + } + + /* Get the calculated PCI paths bandwidth */ + pdev = to_pci_dev(port->uport->parent); + bw = pcie_bandwidth_available(pdev, NULL, NULL, NULL); + if (bw == 0) + return -ENXIO; + bw /= BITS_PER_BYTE; + + c.write_bandwidth = min_t(unsigned int, c.write_bandwidth, bw); + c.read_bandwidth = min_t(unsigned int, c.read_bandwidth, bw); + + *coord = c; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_endpoint_get_perf_coordinates, CXL); + /* for user tooling to ensure port disable work has completed */ static ssize_t flush_store(struct bus_type *bus, const char *buf, size_t count) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 033b822a20f2..0c8952e568cc 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -858,6 +858,9 @@ struct qtg_dsm_input { __le32 wr_bw; }; +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/.