From patchwork Tue Jun 11 05:52:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Kobayashi (Fujitsu)" X-Patchwork-Id: 13693141 Received: from esa12.hc1455-7.c3s2.iphmx.com (esa12.hc1455-7.c3s2.iphmx.com [139.138.37.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 198938488 for ; Tue, 11 Jun 2024 05:50:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=139.138.37.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718085016; cv=none; b=L4rBnq6JZfo+bM1yMUVzbnGsewj2kb/fSVqekzrgXSOj96GsMRsMVm0dtQmBAhsrr+QbnAGg402RmBnx8HEZn3quvTIKQLapMLrhEJBGqiSwB3G22Lo04Yu7khlxU+LpdC07wsZDhyEJ13mql5L4d3Eba+8vCxTe7OcdfMzHWiU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718085016; c=relaxed/simple; bh=uvmavJZDL5TKC9vuCcLZG1Ye0hMJ1msMnMyAFdn/iUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tWIAEQeU3ilqcTrpLau0iPXPsRXq/dDsZ2zfhzcE5RNHArCj0XTn0DahN6BTjHUQWIWjR7HnczpveOZvSCOJ4RABwl4R737KWCQsQtgRZPzBj/VeckKQPlSmP0uaFFfzg03EKd/X1KsL436aSXBCI3t4YEUVP+JszwemjISWc0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com; spf=pass smtp.mailfrom=fujitsu.com; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b=CciYhd1M; arc=none smtp.client-ip=139.138.37.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b="CciYhd1M" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1718085014; x=1749621014; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uvmavJZDL5TKC9vuCcLZG1Ye0hMJ1msMnMyAFdn/iUU=; b=CciYhd1M/3BMlBVA9dhIyi3hV3SLGjMka5Ud0TheBBQkTiKBWP154TAj ZTVIItj177E5DwJuHzwAGKcbAjrsPHg+XsYV+TS0Ip7TIZcn4cWDrSBNw +m4WqoWRIHY7nq6duAt0LqzzdmEfnkSWRahRqGZ32fhZHgQQNkfgeENQg ZHDMA1JO7y03cpBV4scYfudIlB2ma+USYEdopyrlaylzPzkFFkaObE6UK 3+SZW/9+u3x3sAKG8eod9RXKor+zBmJJsC35ycSgBVHRniGcb9AH/HhPT hTQeDYVrdHjLBypnREuB52GVpsuJW2RduJAF46rEL0MqQejDj9eB8ql+f A==; X-IronPort-AV: E=McAfee;i="6600,9927,11099"; a="141696198" X-IronPort-AV: E=Sophos;i="6.08,229,1712588400"; d="scan'208";a="141696198" Received: from unknown (HELO oym-r1.gw.nic.fujitsu.com) ([210.162.30.89]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2024 14:50:04 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r1.gw.nic.fujitsu.com (Postfix) with ESMTP id B48BCD29EE for ; Tue, 11 Jun 2024 14:50:02 +0900 (JST) Received: from m3002.s.css.fujitsu.com (msm3.b.css.fujitsu.com [10.128.233.104]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 04BD6D7489 for ; Tue, 11 Jun 2024 14:50:02 +0900 (JST) Received: from cxl-test.. (unknown [10.118.236.45]) by m3002.s.css.fujitsu.com (Postfix) with ESMTP id B3458203ED5A; Tue, 11 Jun 2024 14:50:01 +0900 (JST) From: "Kobayashi,Daisuke" To: kobayashi.da-06@jp.fujitsu.com, linux-cxl@vger.kernel.org Cc: y-goto@fujitsu.com, mj@ucw.cz, dan.j.williams@intel.com, jonathan.cameron@huawei.com, "Kobayashi,Daisuke" Subject: [PATCH v10 1/2] cxl/core/regs: Add rcd_pcie_cap initialization at __rcrb_to_component() Date: Tue, 11 Jun 2024 14:52:53 +0900 Message-ID: <20240611055254.61203-2-kobayashi.da-06@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240611055254.61203-1-kobayashi.da-06@fujitsu.com> References: <20240611055254.61203-1-kobayashi.da-06@fujitsu.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 Add rcd_pcie_cap and its initialization at __rcrb_to_component() to cache the offset of cxl1.1 device link status information. By caching it, avoid the walking memory map area to find the offset when output the register value. Signed-off-by: "Kobayashi,Daisuke" --- drivers/cxl/core/core.h | 5 +++++ drivers/cxl/core/regs.c | 27 ++++++++++++++++++++++++++- drivers/cxl/cxl.h | 9 +++++++++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 3b64fb1b9ed0..66778c3ce3b7 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -75,6 +75,11 @@ resource_size_t __rcrb_to_component(struct device *dev, enum cxl_rcrb which); u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb); +#define PCI_RCRB_CAP_LIST_ID_MASK GENMASK(7, 0) +#define PCI_RCRB_CAP_HDR_ID_MASK GENMASK(7, 0) +#define PCI_RCRB_CAP_HDR_NEXT_MASK GENMASK(15, 8) +#define RCRB_PCIECAP_LEN 0x3c + extern struct rw_semaphore cxl_dpa_rwsem; extern struct rw_semaphore cxl_region_rwsem; diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 372786f80955..5ce831ca05ca 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -514,6 +514,8 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *ri u32 bar0, bar1; u16 cmd; u32 id; + u32 cap_hdr; + u16 offset; if (which == CXL_RCRB_UPSTREAM) rcrb += SZ_4K; @@ -537,6 +539,19 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *ri cmd = readw(addr + PCI_COMMAND); bar0 = readl(addr + PCI_BASE_ADDRESS_0); bar1 = readl(addr + PCI_BASE_ADDRESS_1); + offset = FIELD_GET(PCI_RCRB_CAP_LIST_ID_MASK, readw(addr + PCI_CAPABILITY_LIST)); + cap_hdr = readl(addr + offset); + while ((FIELD_GET(PCI_RCRB_CAP_HDR_ID_MASK, cap_hdr)) != PCI_CAP_ID_EXP) { + offset = FIELD_GET(PCI_RCRB_CAP_HDR_NEXT_MASK, cap_hdr); + if (offset == 0 || offset > SZ_4K) { + offset = 0; + break; + } + cap_hdr = readl(addr + offset); + } + if (offset) + ri->rcd_pcie_cap = offset; + iounmap(addr); release_mem_region(rcrb, SZ_4K); @@ -572,8 +587,18 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *ri resource_size_t cxl_rcd_component_reg_phys(struct device *dev, struct cxl_dport *dport) { + resource_size_t rcd_pcie_offset, ret; + if (!dport->rch) return CXL_RESOURCE_NONE; - return __rcrb_to_component(dev, &dport->rcrb, CXL_RCRB_UPSTREAM); + + ret = __rcrb_to_component(dev, &dport->rcrb, CXL_RCRB_UPSTREAM); + if (dport->rcrb.rcd_pcie_cap) { + rcd_pcie_offset = dport->rcrb.base + dport->rcrb.rcd_pcie_cap; + dport->regs.rcd_pcie_cap = devm_cxl_iomap_block(dev, rcd_pcie_offset, + sizeof(u8) * RCRB_PCIECAP_LEN); + } + + return ret; } EXPORT_SYMBOL_NS_GPL(cxl_rcd_component_reg_phys, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 003feebab79b..fc9e0dbd5932 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -230,6 +230,14 @@ struct cxl_regs { struct_group_tagged(cxl_rch_regs, rch_regs, void __iomem *dport_aer; ); + + /* + * RCD upstream port specific PCIe cap register + * @pcie_cap: CXL 3.0 8.2.1.2 RCD Upstream Port RCRB + */ + struct_group_tagged(cxl_rcd_regs, rcd_regs, + void __iomem *rcd_pcie_cap; + ); }; struct cxl_reg_map { @@ -646,6 +654,7 @@ cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dport_dev) struct cxl_rcrb_info { resource_size_t base; + u16 rcd_pcie_cap; u16 aer_cap; }; From patchwork Tue Jun 11 05:52:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Kobayashi (Fujitsu)" X-Patchwork-Id: 13693142 Received: from esa12.hc1455-7.c3s2.iphmx.com (esa12.hc1455-7.c3s2.iphmx.com [139.138.37.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC3328488 for ; Tue, 11 Jun 2024 05:50:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=139.138.37.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718085018; cv=none; b=efDvPrXe22OB6kdV6qBESbJ/l2p8uwSKtXfGnqslXoHZy/di0GpozIzkyX7VgXYSWjyzLDoJyxM/8Tgng6EEHMK4s4Lj4wnjcv7JUA+e2Ug6Fv12/8WlDuSX1Sel5iT5tu0Tj41Cw1A+uP8MGa2IzgC8DzCXJOqFfTIQhvS9+l4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718085018; c=relaxed/simple; bh=JH+u0z3/30U9zJTAGt4xwRTaIm+J3oVWDe8r5oiOw+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pzI1hvtUgevLP/7Utaoc86siuQOV+fwsF4I2Vvi+1qcHFxuZp7ZGLbvDungbSezZiJJTKfue/vDbRzlAcrYErJJ4+hyqKOZtdeIHKzaa5F0jA55+XDT9zU0YPt6nh8tBV4sa8X+bnXe8j+Ff7oFOwkIl2ctDPiWVIC59oAHi2M8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com; spf=pass smtp.mailfrom=fujitsu.com; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b=WcBdwBKe; arc=none smtp.client-ip=139.138.37.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b="WcBdwBKe" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1718085017; x=1749621017; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JH+u0z3/30U9zJTAGt4xwRTaIm+J3oVWDe8r5oiOw+w=; b=WcBdwBKezGKYpofUskA4TE1gGWSLl5ErbXLEq+WYFqdFxAqddy4IUe3x 9Ue8dHUyAehepEU7K10NDZTQqL/mR+oOduuS6N9YHOehJZrDuyTnjUl2l wYyZ5heGKScZjoergRYzkwBvPGAXBZ38nEErkNxS8KiMl6mFXnZNAhv/u nLAtyvb01bKg6BqeGWZAUcj1WNrhs67Nt+1jlrX0XdJFOwMXOv4WgLuTV AaUpy+7zEmqJx8U6+QkNSL8FE3U9+EuXt9VHtOrWa0ThxeVaYWvcctcyM ySA5Knvz7RM6Qc/2fqCmgJb0TuoebRw11LdSuhzBjC2aMnBPI8qvtIzbi w==; X-IronPort-AV: E=McAfee;i="6600,9927,11099"; a="141696210" X-IronPort-AV: E=Sophos;i="6.08,229,1712588400"; d="scan'208";a="141696210" Received: from unknown (HELO yto-r4.gw.nic.fujitsu.com) ([218.44.52.220]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2024 14:50:09 +0900 Received: from yto-m4.gw.nic.fujitsu.com (yto-nat-yto-m4.gw.nic.fujitsu.com [192.168.83.67]) by yto-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id EB5FFC932D for ; Tue, 11 Jun 2024 14:50:06 +0900 (JST) Received: from m3002.s.css.fujitsu.com (msm3.b.css.fujitsu.com [10.128.233.104]) by yto-m4.gw.nic.fujitsu.com (Postfix) with ESMTP id 3446DD3F14 for ; Tue, 11 Jun 2024 14:50:06 +0900 (JST) Received: from cxl-test.. (unknown [10.118.236.45]) by m3002.s.css.fujitsu.com (Postfix) with ESMTP id 0C122203ED51; Tue, 11 Jun 2024 14:50:06 +0900 (JST) From: "Kobayashi,Daisuke" To: kobayashi.da-06@jp.fujitsu.com, linux-cxl@vger.kernel.org Cc: y-goto@fujitsu.com, mj@ucw.cz, dan.j.williams@intel.com, jonathan.cameron@huawei.com, "Kobayashi,Daisuke" Subject: [PATCH v10 2/2] cxl/pci: Add sysfs attribute for CXL 1.1 device link status Date: Tue, 11 Jun 2024 14:52:54 +0900 Message-ID: <20240611055254.61203-3-kobayashi.da-06@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240611055254.61203-1-kobayashi.da-06@fujitsu.com> References: <20240611055254.61203-1-kobayashi.da-06@fujitsu.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 Add sysfs attribute for CXL 1.1 device link status to the cxl pci device. In CXL1.1, the link status of the device is included in the RCRB mapped to the memory mapped register area. Critically, that arrangement makes the link status and control registers invisible to existing PCI user tooling. Export those registers via sysfs with the expectation that PCI user tooling will alternatively look for these sysfs files when attempting to access to these CXL 1.1 endpoints registers. Signed-off-by: "Kobayashi,Daisuke" --- drivers/cxl/pci.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 2ff361e756d6..655616a16892 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -786,6 +786,103 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, return 0; } +static ssize_t rcd_pcie_cap_emitl(struct device *dev, u16 offset, char *buf) +{ + struct cxl_dev_state *cxlds = dev_get_drvdata(dev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *endpoint_parent; + struct cxl_dport *dport; + struct cxl_port *port; + + port = cxl_mem_find_port(cxlmd, &dport); + if (!port) + return -EINVAL; + + endpoint_parent = port->uport_dev; + if (!endpoint_parent) + return -ENXIO; + + guard(device)(endpoint_parent); + if (!endpoint_parent->driver) + return -ENXIO; + + if (dport->regs.rcd_pcie_cap == NULL) + return -EINVAL; + + return sysfs_emit(buf, "%x\n", readl(dport->regs.rcd_pcie_cap + offset)); +} + +static ssize_t rcd_pcie_cap_emitw(struct device *dev, u16 offset, char *buf) +{ + struct cxl_dev_state *cxlds = dev_get_drvdata(dev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *endpoint_parent; + struct cxl_dport *dport; + struct cxl_port *port; + + port = cxl_mem_find_port(cxlmd, &dport); + if (!port) + return -EINVAL; + + endpoint_parent = port->uport_dev; + if (!endpoint_parent) + return -ENXIO; + + guard(device)(endpoint_parent); + if (!endpoint_parent->driver) + return -ENXIO; + + if (dport->regs.rcd_pcie_cap == NULL) + return -EINVAL; + + return sysfs_emit(buf, "%x\n", readw(dport->regs.rcd_pcie_cap + offset)); +} + +static ssize_t rcd_link_cap_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return rcd_pcie_cap_emitl(dev, PCI_EXP_LNKCAP, buf); +} +static DEVICE_ATTR_RO(rcd_link_cap); + +static ssize_t rcd_link_ctrl_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return rcd_pcie_cap_emitw(dev, PCI_EXP_LNKCTL, buf); +} +static DEVICE_ATTR_RO(rcd_link_ctrl); + +static ssize_t rcd_link_status_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return rcd_pcie_cap_emitw(dev, PCI_EXP_LNKSTA, buf); +} +static DEVICE_ATTR_RO(rcd_link_status); + +static struct attribute *cxl_rcd_attrs[] = { + &dev_attr_rcd_link_cap.attr, + &dev_attr_rcd_link_ctrl.attr, + &dev_attr_rcd_link_status.attr, + NULL +}; + +static umode_t cxl_rcd_visible(struct kobject *kobj, struct attribute *a, int n) +{ + struct device *dev = kobj_to_dev(kobj); + struct pci_dev *pdev = to_pci_dev(dev); + + if (is_cxl_restricted(pdev)) + return a->mode; + + return 0; +} + +static struct attribute_group cxl_rcd_group = { + .attrs = cxl_rcd_attrs, + .is_visible = cxl_rcd_visible, +}; +__ATTRIBUTE_GROUPS(cxl_rcd); + static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); @@ -969,6 +1066,7 @@ static struct pci_driver cxl_pci_driver = { .id_table = cxl_mem_pci_tbl, .probe = cxl_pci_probe, .err_handler = &cxl_error_handlers, + .dev_groups = cxl_rcd_groups, .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, },