| Message ID | 20220624041950.559155-5-dan.j.williams@intel.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | CXL PMEM Region Provisioning | expand |
On Thu, 23 Jun 2022 21:19:34 -0700 Dan Williams <dan.j.williams@intel.com> wrote: > From: Ben Widawsky <bwidawsk@kernel.org> > > The region provisioning flow involves selecting interleave ways + > granularity settings for a region, and then programming the decoder > topology to meet those constraints, if possible. For example, root > decoders set the minimum interleave ways + granularity for any hosted > regions. > > Given decoder programming is not atomic and collisions can occur between > multiple requesting regions userpace will be resonsible for conflict > resolution and it needs these attributes to make those decisions. > > Signed-off-by: Ben Widawsky <bwidawsk@kernel.org> > [djbw: reword changelog, make read-only, add sysfs ABI documentaion] > Signed-off-by: Dan Williams <dan.j.williams@intel.com> some comments on docs. > --- > Documentation/ABI/testing/sysfs-bus-cxl | 23 +++++++++++++++++++++++ > drivers/cxl/core/port.c | 23 +++++++++++++++++++++++ > 2 files changed, 46 insertions(+) > > diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl > index 85844f9bc00b..2a4e4163879f 100644 > --- a/Documentation/ABI/testing/sysfs-bus-cxl > +++ b/Documentation/ABI/testing/sysfs-bus-cxl > @@ -215,3 +215,26 @@ Description: > allocations are enforced to occur in increasing 'decoderX.Y/id' > order and frees are enforced to occur in decreasing > 'decoderX.Y/id' order. > + > + > +What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways > +Date: May, 2022 > +KernelVersion: v5.20 > +Contact: linux-cxl@vger.kernel.org > +Description: > + (RO) The number of targets across which this decoder's host > + physical address (HPA) memory range is interleaved. The device > + maps every Nth block of HPA (of size == > + 'interleave_granularity') to consecutive DPA addresses. The > + decoder's position in the interleave is determined by the > + device's (endpoint or switch) switch ancestry. Perhaps make it clear what happens for host bridges (i.e. decoder position in interleave defined by fixed memory window. > + > + > +What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity > +Date: May, 2022 > +KernelVersion: v5.20 > +Contact: linux-cxl@vger.kernel.org > +Description: > + (RO) The number of consecutive bytes of host physical address > + space this decoder claims at address N before awaint the next awaint? > + address (N + interleave_granularity * intereleave_ways). interleave_ways Even knowing exactly what this is, I don't understand the docs so perhaps reword this :) > diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c > index c48f217e689a..08a380d20cf1 100644 > --- a/drivers/cxl/core/port.c > +++ b/drivers/cxl/core/port.c > @@ -260,10 +260,33 @@ static ssize_t dpa_size_store(struct device *dev, struct device_attribute *attr, > } > static DEVICE_ATTR_RW(dpa_size); > > +static ssize_t interleave_granularity_show(struct device *dev, > + struct device_attribute *attr, > + char *buf) > +{ > + struct cxl_decoder *cxld = to_cxl_decoder(dev); > + > + return sysfs_emit(buf, "%d\n", cxld->interleave_granularity); > +} > + > +static DEVICE_ATTR_RO(interleave_granularity); > + > +static ssize_t interleave_ways_show(struct device *dev, > + struct device_attribute *attr, char *buf) > +{ > + struct cxl_decoder *cxld = to_cxl_decoder(dev); > + > + return sysfs_emit(buf, "%d\n", cxld->interleave_ways); > +} > + > +static DEVICE_ATTR_RO(interleave_ways); > + > static struct attribute *cxl_decoder_base_attrs[] = { > &dev_attr_start.attr, > &dev_attr_size.attr, > &dev_attr_locked.attr, > + &dev_attr_interleave_granularity.attr, > + &dev_attr_interleave_ways.attr, > NULL, > }; >
Jonathan Cameron wrote: > On Thu, 23 Jun 2022 21:19:34 -0700 > Dan Williams <dan.j.williams@intel.com> wrote: > > > From: Ben Widawsky <bwidawsk@kernel.org> > > > > The region provisioning flow involves selecting interleave ways + > > granularity settings for a region, and then programming the decoder > > topology to meet those constraints, if possible. For example, root > > decoders set the minimum interleave ways + granularity for any hosted > > regions. > > > > Given decoder programming is not atomic and collisions can occur between > > multiple requesting regions userpace will be resonsible for conflict > > resolution and it needs these attributes to make those decisions. > > > > Signed-off-by: Ben Widawsky <bwidawsk@kernel.org> > > [djbw: reword changelog, make read-only, add sysfs ABI documentaion] > > Signed-off-by: Dan Williams <dan.j.williams@intel.com> > some comments on docs. > > > --- > > Documentation/ABI/testing/sysfs-bus-cxl | 23 +++++++++++++++++++++++ > > drivers/cxl/core/port.c | 23 +++++++++++++++++++++++ > > 2 files changed, 46 insertions(+) > > > > diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl > > index 85844f9bc00b..2a4e4163879f 100644 > > --- a/Documentation/ABI/testing/sysfs-bus-cxl > > +++ b/Documentation/ABI/testing/sysfs-bus-cxl > > @@ -215,3 +215,26 @@ Description: > > allocations are enforced to occur in increasing 'decoderX.Y/id' > > order and frees are enforced to occur in decreasing > > 'decoderX.Y/id' order. > > + > > + > > +What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways > > +Date: May, 2022 > > +KernelVersion: v5.20 > > +Contact: linux-cxl@vger.kernel.org > > +Description: > > + (RO) The number of targets across which this decoder's host > > + physical address (HPA) memory range is interleaved. The device > > + maps every Nth block of HPA (of size == > > + 'interleave_granularity') to consecutive DPA addresses. The > > + decoder's position in the interleave is determined by the > > + device's (endpoint or switch) switch ancestry. > > Perhaps make it clear what happens for host bridges (i.e. decoder position > in interleave defined by fixed memory window. Added: "For root decoders their interleave is specified by platform firmware and they only specify a downstream target order for host bridges". > > > + > > + > > +What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity > > +Date: May, 2022 > > +KernelVersion: v5.20 > > +Contact: linux-cxl@vger.kernel.org > > +Description: > > + (RO) The number of consecutive bytes of host physical address > > + space this decoder claims at address N before awaint the next > > awaint? Surprised checkpatch did not flag this, or that I missed the checkpatch flag. > > > + address (N + interleave_granularity * intereleave_ways). > > interleave_ways > > Even knowing exactly what this is, I don't understand the docs so > perhaps reword this :) Reworded to: (RO) The number of consecutive bytes of host physical address space this decoder claims at address N before the decode rotates to the next target in the interleave at address N + interleave_granularity (assuming N is aligned to interleave_granularity).
> > > > > + address (N + interleave_granularity * intereleave_ways). > > > > interleave_ways > > > > Even knowing exactly what this is, I don't understand the docs so > > perhaps reword this :) > > Reworded to: > > (RO) The number of consecutive bytes of host physical address space this > decoder claims at address N before the decode rotates to the next target > in the interleave at address N + interleave_granularity (assuming N is > aligned to interleave_granularity). LGTM
diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 85844f9bc00b..2a4e4163879f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -215,3 +215,26 @@ Description: allocations are enforced to occur in increasing 'decoderX.Y/id' order and frees are enforced to occur in decreasing 'decoderX.Y/id' order. + + +What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways +Date: May, 2022 +KernelVersion: v5.20 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) The number of targets across which this decoder's host + physical address (HPA) memory range is interleaved. The device + maps every Nth block of HPA (of size == + 'interleave_granularity') to consecutive DPA addresses. The + decoder's position in the interleave is determined by the + device's (endpoint or switch) switch ancestry. + + +What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity +Date: May, 2022 +KernelVersion: v5.20 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) The number of consecutive bytes of host physical address + space this decoder claims at address N before awaint the next + address (N + interleave_granularity * intereleave_ways). diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index c48f217e689a..08a380d20cf1 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -260,10 +260,33 @@ static ssize_t dpa_size_store(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RW(dpa_size); +static ssize_t interleave_granularity_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev); + + return sysfs_emit(buf, "%d\n", cxld->interleave_granularity); +} + +static DEVICE_ATTR_RO(interleave_granularity); + +static ssize_t interleave_ways_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev); + + return sysfs_emit(buf, "%d\n", cxld->interleave_ways); +} + +static DEVICE_ATTR_RO(interleave_ways); + static struct attribute *cxl_decoder_base_attrs[] = { &dev_attr_start.attr, &dev_attr_size.attr, &dev_attr_locked.attr, + &dev_attr_interleave_granularity.attr, + &dev_attr_interleave_ways.attr, NULL, };