From patchwork Sat Dec 17 01:33:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13075741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E740C4332F for ; Sat, 17 Dec 2022 01:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229865AbiLQBdf (ORCPT ); Fri, 16 Dec 2022 20:33:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229488AbiLQBde (ORCPT ); Fri, 16 Dec 2022 20:33:34 -0500 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F37B73325 for ; Fri, 16 Dec 2022 17:33:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1671240813; x=1702776813; h=subject:from:to:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=cOIZLngsvqV8wDzzAmvNVtCqhVbofpyAswiK7OaTYL8=; b=Q6ZRty3zS4P32rfQBUHwliA+UyqOeleyk3vZfXsiR+C7wDn92Wkis+BW 6Gjifc+1iJX2+nH8dKXaLAH2R8PTz15/sNo7pYnDdPJfSXgZVtSuvZtyr Z3weskN3FCHNvCPQfmkWm1M+BvDkY/0JuzyIq8Eq8MwY9SdvqVucKX6mg BypnVJWEGjbuExZsOLJJoWJ0+J2TYboX5BhVUjYAs01grU97zNdYd5WQY IU9r/ZEYvat28Se/8tacocqH8QJvvLntghPFwImQnYPGAwTCY5Ehf3hno +3BKownAUFMU9NmFMiEyoWmR0GkYJjDwwNLED9kmxSzdO628FDsQ3YEHq w==; X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="302513777" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="302513777" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:33 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="649993069" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="649993069" Received: from smukhe4-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.110.159]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:33 -0800 Subject: [PATCH 1/3] cxl/mem: Quiet port walking warning From: Dan Williams To: linux-cxl@vger.kernel.org Date: Fri, 16 Dec 2022 17:33:32 -0800 Message-ID: <167124081278.1626103.4792472728150764118.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> References: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The cxl_mem driver attempts to establish, or revalidate, the cxl_port hierarcy to attach a cxl_memdev to a CXL platform topology. There is a natural race (on ACPI platforms) between when the cxl_mem driver attempts to attach and when the cxl_acpi driver establishes the root of the topology. If cxl_mem_probe() runs first it will iterate to the top of the device topology without finding the CXL platform root. That situation is benign / expected, so stop warning about it. The cxl_acpi driver will poke cxl_mem_probe() to try again once the CXL platform root is established. Signed-off-by: Dan Williams --- drivers/cxl/core/port.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 810e60cc331c..6296d2bc909a 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1400,8 +1400,8 @@ int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd) uport_dev = dport_dev->parent; if (!uport_dev) { - dev_warn(dev, "at %s no parent for dport: %s\n", - dev_name(iter), dev_name(dport_dev)); + dev_dbg(dev, "at %s no parent for dport: %s\n", + dev_name(iter), dev_name(dport_dev)); return -ENXIO; } From patchwork Sat Dec 17 01:33:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13075742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36049C4332F for ; Sat, 17 Dec 2022 01:33:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbiLQBdl (ORCPT ); Fri, 16 Dec 2022 20:33:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229488AbiLQBdk (ORCPT ); Fri, 16 Dec 2022 20:33:40 -0500 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E080D7332C for ; Fri, 16 Dec 2022 17:33:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1671240818; x=1702776818; h=subject:from:to:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=jubWLOBUyOmhiJkbWEc3d2/yMX1W4/AWN5ZdeNYZllg=; b=AFTvg/tZ6PBNiH9kvGm43QeQM0vhpV6hWKs8cmim5qWBWrjL85nYuyEC UmP3M2L3sOwYzfVf+To/gI/VmdlnftdDKj3fXIzG6J6m0qEbBxXJ8mdAf mWyD973uvXJWDdCQoZ8YsVRKjRb3P02m9TA+f7ashrvkTp/v+TAfZvOd1 y9qZd49BoHlAA5FmdubxjL/aq7NaTjFfwRxyhyf1AlQSmizt4pAr10sA6 iZUMM8Ca10NKxYO5ONtXue15oSLFnQKTG07JPPKWP2foIBvvELgiXVHuI P5iihR45AdcVnEd+8YKo2ZgGKAzob5Prc1R+PuFK78uwq36KYcdUlIQcX Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="302513798" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="302513798" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:38 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="649993079" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="649993079" Received: from smukhe4-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.110.159]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:38 -0800 Subject: [PATCH 2/3] cxl/region: Clarify when a cxld->commit() callback is mandatory From: Dan Williams To: linux-cxl@vger.kernel.org Date: Fri, 16 Dec 2022 17:33:38 -0800 Message-ID: <167124081824.1626103.1555704405392757219.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> References: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Both cxl_switch_decoders() and cxl_endpoint_decoders() are considered by cxl_region_decode_commit(). Flag cases where cxl_switch_decoders with multiple targets, or cxl_endpoint_decoders do not have a commit callback set. The switch case is unlikely to happen since switches are only enumerated by the CXL core, but the endpoint case may support decoders defined by drivers outside of drivers/cxl, like accerator drivers. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- drivers/cxl/core/region.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c11a6ab5e48d..60828d01972a 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -156,6 +156,22 @@ static int cxl_region_decode_reset(struct cxl_region *cxlr, int count) return 0; } +static int commit_decoder(struct cxl_decoder *cxld) +{ + struct cxl_switch_decoder *cxlsd = NULL; + + if (cxld->commit) + return cxld->commit(cxld); + + if (is_switch_decoder(&cxld->dev)) + cxlsd = to_cxl_switch_decoder(&cxld->dev); + + if (dev_WARN_ONCE(&cxld->dev, !cxlsd || cxlsd->nr_targets > 1, + "->commit() is required\n")) + return -ENXIO; + return 0; +} + static int cxl_region_decode_commit(struct cxl_region *cxlr) { struct cxl_region_params *p = &cxlr->params; @@ -174,8 +190,7 @@ static int cxl_region_decode_commit(struct cxl_region *cxlr) iter = to_cxl_port(iter->dev.parent)) { cxl_rr = cxl_rr_load(iter, cxlr); cxld = cxl_rr->decoder; - if (cxld->commit) - rc = cxld->commit(cxld); + rc = commit_decoder(cxld); if (rc) break; } From patchwork Sat Dec 17 01:33:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13075743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17BDFC4332F for ; Sat, 17 Dec 2022 01:33:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229907AbiLQBdq (ORCPT ); Fri, 16 Dec 2022 20:33:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229939AbiLQBdp (ORCPT ); Fri, 16 Dec 2022 20:33:45 -0500 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CFDF73319 for ; Fri, 16 Dec 2022 17:33:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1671240824; x=1702776824; h=subject:from:to:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=yGN+M08U0rZUPTvJG3Yx5fNkmQnaUxvo5iUIr1ABqok=; b=O9igpik3RnJTK0DICxO7koF5vpzwszsk/biF2Uad1Fc99lGPc33Efiev snJSnOfkY0uRcRxUT6Ui00MbNt5X+19+y8N38YLk+DbXlPANlw6LwvwXM hsGZGF09W3YW017gcwuvpgiUA2qeU8pGIdABrPLvMd9D5GTSlZvKbuwVA WJY9MMb6YJv+FC+XVtK7sxe5fthDH9KLy7DwKDRsj91JpNbaSY00GTGGW Fl2V5GqbvwSKaXRwg9RyCbE+wDVXusN6DwFKBONYB3qPwUrhvf/Zqdexo QPr6Z71O6BQi7AlSWH/DEYpMkCnpiWwAq8Arjmct5gOX8MYYy8b2C/Lya Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="302513805" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="302513805" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:44 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10563"; a="649993083" X-IronPort-AV: E=Sophos;i="5.96,251,1665471600"; d="scan'208";a="649993083" Received: from smukhe4-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.110.159]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2022 17:33:44 -0800 Subject: [PATCH 3/3] cxl/port: Link the 'parent_dport' in portX/ and endpointX/ sysfs From: Dan Williams To: linux-cxl@vger.kernel.org Date: Fri, 16 Dec 2022 17:33:43 -0800 Message-ID: <167124082375.1626103.6047000000121298560.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> References: <167124080717.1626103.10654476222026614847.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Similar to the justification in: 1b58b4cac6fc ("cxl/port: Record parent dport when adding ports") ...userspace wants to know the routing information for ports for calculating the memdev order for region creation among other things. Cache the information the kernel discovers at enumeration time in a 'parent_dport' attribute to save userspace the time of trawling sysfs to recover the same information. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- Documentation/ABI/testing/sysfs-bus-cxl | 15 +++++++++++++++ drivers/cxl/core/port.c | 27 +++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 8494ef27e8d2..1b17c8cb48b5 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -90,6 +90,21 @@ Description: capability. +What: /sys/bus/cxl/devices/{port,endpoint}X/parent_dport +Date: October, 2022 +KernelVersion: v6.2 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) CXL port objects are instantiated for each upstream port in + a CXL/PCIe switch, and for each endpoint to map the + corresponding memory device into the CXL port hierarchy. When a + descendant CXL port (switch or endpoint) is enumerated it is + useful to know which 'dport' object in the parent CXL port + routes to this descendant. The 'parent_dport' symlink points to + the device representing the downstream port of a CXL switch that + routes to {port,endpoint}X. + + What: /sys/bus/cxl/devices/portX/dportY Date: June, 2021 KernelVersion: v5.14 diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 6296d2bc909a..729e4aab5308 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -586,6 +586,29 @@ static int devm_cxl_link_uport(struct device *host, struct cxl_port *port) return devm_add_action_or_reset(host, cxl_unlink_uport, port); } +static void cxl_unlink_parent_dport(void *_port) +{ + struct cxl_port *port = _port; + + sysfs_remove_link(&port->dev.kobj, "parent_dport"); +} + +static int devm_cxl_link_parent_dport(struct device *host, + struct cxl_port *port, + struct cxl_dport *parent_dport) +{ + int rc; + + if (!parent_dport) + return 0; + + rc = sysfs_create_link(&port->dev.kobj, &parent_dport->dport->kobj, + "parent_dport"); + if (rc) + return rc; + return devm_add_action_or_reset(host, cxl_unlink_parent_dport, port); +} + static struct lock_class_key cxl_port_key; static struct cxl_port *cxl_port_alloc(struct device *uport, @@ -695,6 +718,10 @@ static struct cxl_port *__devm_cxl_add_port(struct device *host, if (rc) return ERR_PTR(rc); + rc = devm_cxl_link_parent_dport(host, port, parent_dport); + if (rc) + return ERR_PTR(rc); + return port; err: