From patchwork Fri Jun 16 21:42:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13283314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A7BEEB64D8 for ; Fri, 16 Jun 2023 21:42:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230432AbjFPVm3 (ORCPT ); Fri, 16 Jun 2023 17:42:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231846AbjFPVm2 (ORCPT ); Fri, 16 Jun 2023 17:42:28 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20D6330F7 for ; Fri, 16 Jun 2023 14:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686951747; x=1718487747; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FeKpjjGz7/fZTpc9THX+Osz8wEbsNEGudLIes6gzbTg=; b=f9j109dRDdP7jxchgzLUIwk9XIxn6IefwiAcnNEZEO/NRFnY1JdGgG6A EXTvT+tb6oJ4WSlHYA2LpGAFoc2Dvl/MUBoJWyCPjj78P49Lqggth5NVB mgGReix71k0y9M+/khvPcSbZOhj323Yz4337bukIoMRm27qm99nLv/k1s vvKgHY5F65YqU2UkfFkAQ2sPjmVwAXRo06R1HqAgi3JOV6hRVIrnoGNQt g34SgIfQYz2KJfjdsrLG0rkbElv9Rma5/vlEZVuUz0DsmfiOgOwimaLdG jnBeiPOUxAzYcoRqJFyGpr5bPjgJJqRRguRcV98o05fJEjpZtPM2QMcGU w==; X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="362733644" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="362733644" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 14:42:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="959757393" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="959757393" Received: from djiang5-mobl3.amr.corp.intel.com (HELO [192.168.1.177]) ([10.212.16.91]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 14:42:19 -0700 Subject: [PATCH v7 07/11] cxl: Add helper function that calculate performance data for downstream ports From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com Date: Fri, 16 Jun 2023 14:42:20 -0700 Message-ID: <168695174021.3031571.11686850224446622786.stgit@djiang5-mobl3> In-Reply-To: <168695160531.3031571.4875512229068707023.stgit@djiang5-mobl3> References: <168695160531.3031571.4875512229068707023.stgit@djiang5-mobl3> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The CDAT information from the switch, Switch Scoped Latency and Bandwidth Information Strucutre (SSLBIS), is parsed and stored under a cxl_dport based on the correlated downstream port id from the SSLBIS entry. Walk the entire CXL port paths and collect all the performance data. Also pick up the link latency number that's stored under the dports. The entire path PCIe bandwidth can be retrieved using the pcie_bandwidth_available() call. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v6: - Change min_t() to min(). (Jonathan) --- drivers/cxl/core/port.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 ++ 2 files changed, 69 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 73aa8ebb36e6..83fb2b4dc421 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -1948,6 +1949,71 @@ bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, CXL); +static void combine_coordinates(struct access_coordinate *c1, + struct access_coordinate *c2) +{ + if (c2->write_bandwidth) + c1->write_bandwidth = min(c1->write_bandwidth, + c2->write_bandwidth); + c1->write_latency += c2->write_latency; + + if (c2->read_bandwidth) + c1->read_bandwidth = min(c1->read_bandwidth, + c2->read_bandwidth); + c1->read_latency += c2->read_latency; +} + +/** + * cxl_endpoint_get_perf_coordinates - Retrieve performance numbers stored in dports + * of CXL path + * @port: endpoint cxl_port + * @coord: output performance data + * + * Return: errno on failure, 0 on success. + */ +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord) +{ + struct access_coordinate c = { + .read_bandwidth = UINT_MAX, + .write_bandwidth = UINT_MAX, + }; + struct cxl_port *iter = port; + struct cxl_dport *dport; + struct pci_dev *pdev; + unsigned int bw; + + if (!is_cxl_endpoint(port)) + return -EINVAL; + + dport = iter->parent_dport; + while (iter && !is_cxl_root(iter)) { + combine_coordinates(&c, &dport->coord); + c.write_latency += dport->link_latency; + c.read_latency += dport->link_latency; + + combine_coordinates(&c, &dport->hb_access); + + iter = to_cxl_port(iter->dev.parent); + dport = iter->parent_dport; + } + + /* Get the calculated PCI paths bandwidth */ + pdev = to_pci_dev(port->uport->parent); + bw = pcie_bandwidth_available(pdev, NULL, NULL, NULL); + if (bw == 0) + return -ENXIO; + bw /= BITS_PER_BYTE; + + c.write_bandwidth = min(c.write_bandwidth, bw); + c.read_bandwidth = min(c.read_bandwidth, bw); + + *coord = c; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_endpoint_get_perf_coordinates, CXL); + /* for user tooling to ensure port disable work has completed */ static ssize_t flush_store(const struct bus_type *bus, const char *buf, size_t count) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index e55a62c461fd..580f44587ab0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -867,6 +867,9 @@ static inline int cxl_cdat_switch_process(struct cxl_port *port) } #endif +int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, + struct access_coordinate *coord); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/.