[v2,00/12] cxl: Add support for Restricted CXL hosts (RCD mode)

Message

Robert Richter Oct. 18, 2022, 1:23 p.m. UTC

In Restricted CXL Device (RCD) mode (formerly referred to as CXL 1.1)
the PCIe enumeration hierarchy is different from CXL VH Enumeration
(formerly referred to as 2.0, for both modes see CXL spec 3.0: 9.11
and 9.12, [1]). This series adds support for RCD mode. It implements
the detection of Restricted CXL Hosts (RCHs) and its corresponding
Restricted CXL Devices (RCDs). It does the necessary enumeration of
ports and connects the endpoints. With all the plumbing an RCH/RCD
pair is registered at the Linux CXL bus and becomes visible in sysfs
in the same way as CXL VH hosts and devices do already. RCDs are
brought up as CXL endpoints and bound to subsequent drivers such as
cxl_mem.

For CXL VH the host driver (cxl_acpi) starts host bridge discovery
once the ACPI0017 CXL root device is detected and then searches for
ACPI0016 host bridges to enable CXL. This implementation requires that
in RCD mode an ACPI0017 device exists, the host bridge must have the
ACPI0016 ID set. This implementation also expects the host's
downstream and upstream port RCRBs base address being reported by
firmware using the optional CEDT CHBS entry of the host bridge (see
CXL spec 3.0, 9.17.1.2).

RCD mode does not support hot-plug, so host discovery is at boot time
only.

Patches #1 to #5 are prerequisites of the series with fixes needed and
a rework of debug messages for port enumeration. Those are general
patches and could be applied earlier and independently from the rest
assuming there are no objections with them. Patches #6 to #12 contain
the actual implementation of RCD mode support.

[1] https://www.computeexpresslink.org/spec-landing

---

v2:
 * Reworked series to use add_host_bridge_dport() and
   add_host_bridge_uport(). There is a single cxl root device
   (ACPI0017) also for RCHs (must have a ACPI0016 id). (Dan)
 * Rebased onto 6.1-rc1.
 * Added a WARN_ON_ONCE() to CXL_RESOURCE_NONE check. Updated patch
   description with an example case. (Dan, Jonathan)
 * Added wrapper functions to devm_cxl_add_port() and
   devm_cxl_add_dport(). (Dan)
 * Dropped "PCI/ACPI: Link host bridge to its ACPI fw node" patch.
 * Updated spec refs to use 3.0. Added PCIe base spec refs. (Jonathan)
 * Reused UID detect code. (Dan)
 * Dropped "cxl/acpi: Specify module load order dependency for the
   cxl_acpi module" patch. Return -EINVAL if host not yet ready. (Dan)
 * Minor other changes.
 * Note: I haven't included most of the received Reviewed-by tags due
   to the major rework. In any case, thanks to all here.

Robert Richter (12):
  cxl/core: Remove duplicate declaration of devm_cxl_iomap_block()
  cxl/core: Check physical address before mapping it in
    devm_cxl_iomap_block()
  cxl: Unify debug messages when calling devm_cxl_add_port()
  cxl: Unify debug messages when calling devm_cxl_add_dport()
  cxl/acpi: Improve debug messages in cxl_acpi_probe()
  cxl/acpi: Extract component registers of restricted hosts from RCRB
  cxl: Remove dev_is_cxl_root_child() check in
    devm_cxl_enumerate_ports()
  cxl: Factor out code in devm_cxl_enumerate_ports() to
    find_port_attach_ep()
  cxl: Extend devm_cxl_enumerate_ports() to support restricted devices
    (RCDs)
  cxl: Do not ignore PCI config read errors in match_add_dports()
  cxl: Factor out code in match_add_dports() to pci_dev_add_dport()
  cxl: Extend devm_cxl_port_enumerate_dports() to support restricted
    hosts (RCH)

 drivers/cxl/acpi.c           |  98 ++++++++++++----
 drivers/cxl/core/pci.c       |  76 +++++++++---
 drivers/cxl/core/port.c      | 219 +++++++++++++++++++++++++----------
 drivers/cxl/core/regs.c      |   5 +
 drivers/cxl/cxl.h            |   2 -
 tools/testing/cxl/test/cxl.c |   8 +-
 6 files changed, 305 insertions(+), 103 deletions(-)


base-commit: 9abf2313adc1ca1b6180c508c25f22f9395cc780