[00/19] cxl: Device memory setup

Message

Dan Williams June 4, 2023, 11:31 p.m. UTC

There are 2 models for implementing CXL memory. To date the CXL enabling
has been focused on the common class-device case (Type-3). The
class-code mandates the implementation of standard mechanisms like the
mailbox and mandatory commands. The other model (Type-2) is
implementation specific memory typically associated with local device
memory for an accelerator. Start the support for Type-2 and take the
opportunity to better prepare the CXL core for "a la carte" enabling of
optional CXL features.

Now, to date there has not been any engagement on the list for an
accelerator driver that wants to reuse the CXL core, but I think it is
worth moving ahead with these patches for the following reasons:

1/ The refactoring of region creation is needed by Persistent Memory
   support where the kernel needs to create regions from labels, not sysfs
   input.

2/ The 'struct cxl_dev_state' object carries infrastructure that is
   optional outside of CXL memory-device class-code devices. That makes it
   difficult to even start the discussion with accelerator driver authors
   that want to evaluate what pieces of the CXL core are suitable to reuse.

3/ The example type-2 driver in cxl_test protects against
   type-3-exclusive assumptions from leaking back into the code base.

In other words it is difficult to start the "type-2" discussion when the
kernel is ~1500 lines of change from the baseline such a driver might
need, and the cleanups make the code more maintainable independent of an
immediate non-test user.

The first 9 patches are general cleanups, the last 10 are focused on
refactoring region creation in support of driver-instantiated CXL memory
regions.

---

Dan Williams (19):
      cxl/regs: Clarify when a 'struct cxl_register_map' is input vs output
      tools/testing/cxl: Remove unused @cxlds argument
      cxl/mbox: Move mailbox related driver state to its own data structure
      cxl/memdev: Make mailbox functionality optional
      cxl/port: Rename CXL_DECODER_{EXPANDER,ACCELERATOR} => {HOSTMEM,DEVMEM}
      cxl/hdm: Default CXL_DEVTYPE_DEVMEM decoders to CXL_DECODER_DEVMEM
      cxl/region: Manage decoder target_type at decoder-attach time
      cxl/port: Enumerate flit mode capability
      cxl/memdev: Formalize endpoint port linkage
      cxl/memdev: Indicate probe deferral
      cxl/region: Factor out construct_region_{begin,end} and drop_region() for reuse
      cxl/region: Factor out interleave ways setup
      cxl/region: Factor out interleave granularity setup
      cxl/region: Clarify locking requirements of cxl_region_attach()
      cxl/region: Specify host-only vs device memory at region creation time
      cxl/hdm: Define a driver interface for DPA allocation
      cxl/region: Define a driver interface for HPA free space enumeration
      cxl/region: Define a driver interface for region creation
      tools/testing/cxl: Emulate a CXL accelerator with local memory


 drivers/cxl/acpi.c           |    2 
 drivers/cxl/core/hdm.c       |  164 +++++++++++++---
 drivers/cxl/core/mbox.c      |  277 ++++++++++++++-------------
 drivers/cxl/core/memdev.c    |  108 +++++++++-
 drivers/cxl/core/pci.c       |   84 ++++++++
 drivers/cxl/core/pmem.c      |    2 
 drivers/cxl/core/port.c      |   19 +-
 drivers/cxl/core/region.c    |  437 ++++++++++++++++++++++++++++++++++++------
 drivers/cxl/core/regs.c      |    8 -
 drivers/cxl/cxl.h            |   21 ++
 drivers/cxl/cxlmem.h         |  124 ++++++++----
 drivers/cxl/cxlpci.h         |   25 ++
 drivers/cxl/mem.c            |   17 +-
 drivers/cxl/pci.c            |  114 ++++++-----
 drivers/cxl/pmem.c           |   35 ++-
 drivers/cxl/port.c           |    5 
 drivers/cxl/security.c       |   24 +-
 tools/testing/cxl/test/cxl.c |   20 ++
 tools/testing/cxl/test/mem.c |  214 ++++++++++++++-------
 19 files changed, 1245 insertions(+), 455 deletions(-)

base-commit: 9561de3a55bed6bdd44a12820ba81ec416e705a7