| Message ID | 169698642598.1991735.4883136743132463123.stgit@djiang5-mobl3 |
|---|---|
| State | Superseded |
| Headers | show |
| Series | cxl: Add support for QTG ID retrieval for CXL subsystem | expand |
On Tue, 10 Oct 2023 18:07:06 -0700 Dave Jiang <dave.jiang@intel.com> wrote: > Export qos_class sysfs attributes for the CXL memory device. The QoS clas > should show up as /sys/bus/cxl/devices/memX/ram/qos_class0 for the volatile > partition and /sys/bus/cxl/devices/memX/pmem/qos_class0 for the persistent > partition. The QTG ID is retrieved via _DSM after supplying the > calculated bandwidth and latency for the entire CXL path from device to > the CPU. This ID is used to match up to the root decoder QoS class to > determine which CFMWS the memory range of a hotplugged CXL mem device > should be assigned under. > > While there may be multiple DSMAS exported by the device CDAT, the driver > will only expose the first QTG ID per partition in sysfs for now. In the > future when multiple QTG IDs are necessary, they can be exposed. [1] I'm not sure this will extent cleanly if we get a two dimensional set to describle 1) Multiple DSMAS entries for RAM (so multiple inputs to pass to the _DSM) One nice thing here might be to ensure we have the first one seen. So if in future we do need to extent it this corresponds to the 0th one described. 2) Want to describe less ideal QTG values from _DSM Maybe it's too early to come to any conclusion and the single 0 is enough. The cynic in me suggests we call it. qos_class0_0 though to give us the space. If we needs DSMAS ranges, then we describe those using first index, and second is the priority index if we have multiple answers from _DSM. For now it's always 0_0 Jonathan > > [1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Dave Jiang <dave.jiang@intel.com> > > --- > v10: > - Export only qos_class0, the first entry. Additional qos_class entries can be > exported later as needed. (Dan) > - Have the sysfs attrib return -ENOENT unless driver is attached. (Dan) > - Removed Jonathan's review tag due to code changes. > --- > Documentation/ABI/testing/sysfs-bus-cxl | 34 +++++++++++++++++++++++++++++++ > drivers/cxl/core/memdev.c | 34 +++++++++++++++++++++++++++++++ > 2 files changed, 68 insertions(+) > > diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl > index 44ffbbb36654..dd613f5987b5 100644 > --- a/Documentation/ABI/testing/sysfs-bus-cxl > +++ b/Documentation/ABI/testing/sysfs-bus-cxl > @@ -28,6 +28,23 @@ Description: > Payload in the CXL-2.0 specification. > > > +What: /sys/bus/cxl/devices/memX/ram/qos_class0 > +Date: May, 2023 > +KernelVersion: v6.7 > +Contact: linux-cxl@vger.kernel.org > +Description: > + (RO) For CXL host platforms that support "QoS Telemmetry" > + this attribute conveys a comma delimited list of platform > + specific cookies that identifies a QoS performance class > + for the volatile partition of the CXL mem device. These > + class-ids can be compared against a similar "qos_class" > + published for a root decoder. While it is not required > + that the endpoints map their local memory-class to a > + matching platform class, mismatches are not recommended > + and there are platform specific performance related > + side-effects that may result. First class-id is displayed. > + > + > What: /sys/bus/cxl/devices/memX/pmem/size > Date: December, 2020 > KernelVersion: v5.12 > @@ -38,6 +55,23 @@ Description: > Payload in the CXL-2.0 specification. > > > +What: /sys/bus/cxl/devices/memX/pmem/qos_class0 > +Date: May, 2023 > +KernelVersion: v6.7 > +Contact: linux-cxl@vger.kernel.org > +Description: > + (RO) For CXL host platforms that support "QoS Telemmetry" > + this attribute conveys a comma delimited list of platform > + specific cookies that identifies a QoS performance class > + for the persistent partition of the CXL mem device. These > + class-ids can be compared against a similar "qos_class" > + published for a root decoder. While it is not required > + that the endpoints map their local memory-class to a > + matching platform class, mismatches are not recommended > + and there are platform specific performance related > + side-effects that may result. First class-id is displayed. > + > +
On 10/11/23 06:26, Jonathan Cameron wrote: > On Tue, 10 Oct 2023 18:07:06 -0700 > Dave Jiang <dave.jiang@intel.com> wrote: > >> Export qos_class sysfs attributes for the CXL memory device. The QoS clas >> should show up as /sys/bus/cxl/devices/memX/ram/qos_class0 for the volatile >> partition and /sys/bus/cxl/devices/memX/pmem/qos_class0 for the persistent >> partition. The QTG ID is retrieved via _DSM after supplying the >> calculated bandwidth and latency for the entire CXL path from device to >> the CPU. This ID is used to match up to the root decoder QoS class to >> determine which CFMWS the memory range of a hotplugged CXL mem device >> should be assigned under. >> >> While there may be multiple DSMAS exported by the device CDAT, the driver >> will only expose the first QTG ID per partition in sysfs for now. In the >> future when multiple QTG IDs are necessary, they can be exposed. [1] > > I'm not sure this will extent cleanly if we get a two dimensional set to describle > 1) Multiple DSMAS entries for RAM (so multiple inputs to pass to the _DSM) > One nice thing here might be to ensure we have the first one seen. > So if in future we do need to extent it this corresponds to the 0th one > described. > 2) Want to describe less ideal QTG values from _DSM > > > Maybe it's too early to come to any conclusion and the single 0 is enough. > The cynic in me suggests we call it. qos_class0_0 though to give us the space. > If we needs DSMAS ranges, then we describe those using first index, > and second is the priority index if we have multiple answers from _DSM. > For now it's always 0_0 I talked to Dan and it seems he prefers the simplest form for the current version until we have a need to move towards something more complex. So qos_class0 -> qos_class. We can move to qos_classN_M when there is a need. > > > Jonathan > >> >> [1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab >> >> Suggested-by: Dan Williams <dan.j.williams@intel.com> >> Signed-off-by: Dave Jiang <dave.jiang@intel.com> >> >> --- >> v10: >> - Export only qos_class0, the first entry. Additional qos_class entries can be >> exported later as needed. (Dan) >> - Have the sysfs attrib return -ENOENT unless driver is attached. (Dan) >> - Removed Jonathan's review tag due to code changes. >> --- >> Documentation/ABI/testing/sysfs-bus-cxl | 34 +++++++++++++++++++++++++++++++ >> drivers/cxl/core/memdev.c | 34 +++++++++++++++++++++++++++++++ >> 2 files changed, 68 insertions(+) >> >> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl >> index 44ffbbb36654..dd613f5987b5 100644 >> --- a/Documentation/ABI/testing/sysfs-bus-cxl >> +++ b/Documentation/ABI/testing/sysfs-bus-cxl >> @@ -28,6 +28,23 @@ Description: >> Payload in the CXL-2.0 specification. >> >> >> +What: /sys/bus/cxl/devices/memX/ram/qos_class0 >> +Date: May, 2023 >> +KernelVersion: v6.7 >> +Contact: linux-cxl@vger.kernel.org >> +Description: >> + (RO) For CXL host platforms that support "QoS Telemmetry" >> + this attribute conveys a comma delimited list of platform >> + specific cookies that identifies a QoS performance class >> + for the volatile partition of the CXL mem device. These >> + class-ids can be compared against a similar "qos_class" >> + published for a root decoder. While it is not required >> + that the endpoints map their local memory-class to a >> + matching platform class, mismatches are not recommended >> + and there are platform specific performance related >> + side-effects that may result. First class-id is displayed. >> + >> + >> What: /sys/bus/cxl/devices/memX/pmem/size >> Date: December, 2020 >> KernelVersion: v5.12 >> @@ -38,6 +55,23 @@ Description: >> Payload in the CXL-2.0 specification. >> >> >> +What: /sys/bus/cxl/devices/memX/pmem/qos_class0 >> +Date: May, 2023 >> +KernelVersion: v6.7 >> +Contact: linux-cxl@vger.kernel.org >> +Description: >> + (RO) For CXL host platforms that support "QoS Telemmetry" >> + this attribute conveys a comma delimited list of platform >> + specific cookies that identifies a QoS performance class >> + for the persistent partition of the CXL mem device. These >> + class-ids can be compared against a similar "qos_class" >> + published for a root decoder. While it is not required >> + that the endpoints map their local memory-class to a >> + matching platform class, mismatches are not recommended >> + and there are platform specific performance related >> + side-effects that may result. First class-id is displayed. >> + >> + > >
On Wed, 11 Oct 2023 14:43:14 -0700 Dave Jiang <dave.jiang@intel.com> wrote: > On 10/11/23 06:26, Jonathan Cameron wrote: > > On Tue, 10 Oct 2023 18:07:06 -0700 > > Dave Jiang <dave.jiang@intel.com> wrote: > > > >> Export qos_class sysfs attributes for the CXL memory device. The QoS clas > >> should show up as /sys/bus/cxl/devices/memX/ram/qos_class0 for the volatile > >> partition and /sys/bus/cxl/devices/memX/pmem/qos_class0 for the persistent > >> partition. The QTG ID is retrieved via _DSM after supplying the > >> calculated bandwidth and latency for the entire CXL path from device to > >> the CPU. This ID is used to match up to the root decoder QoS class to > >> determine which CFMWS the memory range of a hotplugged CXL mem device > >> should be assigned under. > >> > >> While there may be multiple DSMAS exported by the device CDAT, the driver > >> will only expose the first QTG ID per partition in sysfs for now. In the > >> future when multiple QTG IDs are necessary, they can be exposed. [1] > > > > I'm not sure this will extent cleanly if we get a two dimensional set to describle > > 1) Multiple DSMAS entries for RAM (so multiple inputs to pass to the _DSM) > > One nice thing here might be to ensure we have the first one seen. > > So if in future we do need to extent it this corresponds to the 0th one > > described. > > 2) Want to describe less ideal QTG values from _DSM > > > > > > Maybe it's too early to come to any conclusion and the single 0 is enough. > > The cynic in me suggests we call it. qos_class0_0 though to give us the space. > > If we needs DSMAS ranges, then we describe those using first index, > > and second is the priority index if we have multiple answers from _DSM. > > For now it's always 0_0 > > I talked to Dan and it seems he prefers the simplest form for the current version until we have a need to move towards something more complex. So qos_class0 -> qos_class. We can move to qos_classN_M when there is a need. > > Ok. We would need to maintain the qos_class interface as whatever it is today, but that's fine. J > > > > > > Jonathan > > > >> > >> [1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab > >> > >> Suggested-by: Dan Williams <dan.j.williams@intel.com> > >> Signed-off-by: Dave Jiang <dave.jiang@intel.com> > >> > >> --- > >> v10: > >> - Export only qos_class0, the first entry. Additional qos_class entries can be > >> exported later as needed. (Dan) > >> - Have the sysfs attrib return -ENOENT unless driver is attached. (Dan) > >> - Removed Jonathan's review tag due to code changes. > >> --- > >> Documentation/ABI/testing/sysfs-bus-cxl | 34 +++++++++++++++++++++++++++++++ > >> drivers/cxl/core/memdev.c | 34 +++++++++++++++++++++++++++++++ > >> 2 files changed, 68 insertions(+) > >> > >> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl > >> index 44ffbbb36654..dd613f5987b5 100644 > >> --- a/Documentation/ABI/testing/sysfs-bus-cxl > >> +++ b/Documentation/ABI/testing/sysfs-bus-cxl > >> @@ -28,6 +28,23 @@ Description: > >> Payload in the CXL-2.0 specification. > >> > >> > >> +What: /sys/bus/cxl/devices/memX/ram/qos_class0 > >> +Date: May, 2023 > >> +KernelVersion: v6.7 > >> +Contact: linux-cxl@vger.kernel.org > >> +Description: > >> + (RO) For CXL host platforms that support "QoS Telemmetry" > >> + this attribute conveys a comma delimited list of platform > >> + specific cookies that identifies a QoS performance class > >> + for the volatile partition of the CXL mem device. These > >> + class-ids can be compared against a similar "qos_class" > >> + published for a root decoder. While it is not required > >> + that the endpoints map their local memory-class to a > >> + matching platform class, mismatches are not recommended > >> + and there are platform specific performance related > >> + side-effects that may result. First class-id is displayed. > >> + > >> + > >> What: /sys/bus/cxl/devices/memX/pmem/size > >> Date: December, 2020 > >> KernelVersion: v5.12 > >> @@ -38,6 +55,23 @@ Description: > >> Payload in the CXL-2.0 specification. > >> > >> > >> +What: /sys/bus/cxl/devices/memX/pmem/qos_class0 > >> +Date: May, 2023 > >> +KernelVersion: v6.7 > >> +Contact: linux-cxl@vger.kernel.org > >> +Description: > >> + (RO) For CXL host platforms that support "QoS Telemmetry" > >> + this attribute conveys a comma delimited list of platform > >> + specific cookies that identifies a QoS performance class > >> + for the persistent partition of the CXL mem device. These > >> + class-ids can be compared against a similar "qos_class" > >> + published for a root decoder. While it is not required > >> + that the endpoints map their local memory-class to a > >> + matching platform class, mismatches are not recommended > >> + and there are platform specific performance related > >> + side-effects that may result. First class-id is displayed. > >> + > >> + > > > > >
diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 44ffbbb36654..dd613f5987b5 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -28,6 +28,23 @@ Description: Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/ram/qos_class0 +Date: May, 2023 +KernelVersion: v6.7 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" + this attribute conveys a comma delimited list of platform + specific cookies that identifies a QoS performance class + for the volatile partition of the CXL mem device. These + class-ids can be compared against a similar "qos_class" + published for a root decoder. While it is not required + that the endpoints map their local memory-class to a + matching platform class, mismatches are not recommended + and there are platform specific performance related + side-effects that may result. First class-id is displayed. + + What: /sys/bus/cxl/devices/memX/pmem/size Date: December, 2020 KernelVersion: v5.12 @@ -38,6 +55,23 @@ Description: Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/pmem/qos_class0 +Date: May, 2023 +KernelVersion: v6.7 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" + this attribute conveys a comma delimited list of platform + specific cookies that identifies a QoS performance class + for the persistent partition of the CXL mem device. These + class-ids can be compared against a similar "qos_class" + published for a root decoder. While it is not required + that the endpoints map their local memory-class to a + matching platform class, mismatches are not recommended + and there are platform specific performance related + side-effects that may result. First class-id is displayed. + + What: /sys/bus/cxl/devices/memX/serial Date: January, 2022 KernelVersion: v5.18 diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 14b547c07f54..bfecc9da2561 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -88,6 +88,22 @@ static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_ram_size = __ATTR(size, 0444, ram_size_show, NULL); +static ssize_t ram_qos_class0_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + + if (!dev->driver) + return -ENOENT; + + return sysfs_emit(buf, "%d\n", mds->ram_qos_class); +} + +static struct device_attribute dev_attr_ram_qos_class0 = + __ATTR(qos_class0, 0444, ram_qos_class0_show, NULL); + static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -101,6 +117,22 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_pmem_size = __ATTR(size, 0444, pmem_size_show, NULL); +static ssize_t pmem_qos_class0_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds); + + if (!dev->driver) + return -ENOENT; + + return sysfs_emit(buf, "%d\n", mds->pmem_qos_class); +} + +static struct device_attribute dev_attr_pmem_qos_class0 = + __ATTR(qos_class0, 0444, pmem_qos_class0_show, NULL); + static ssize_t serial_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -439,11 +471,13 @@ static struct attribute *cxl_memdev_attributes[] = { static struct attribute *cxl_memdev_pmem_attributes[] = { &dev_attr_pmem_size.attr, + &dev_attr_pmem_qos_class0.attr, NULL, }; static struct attribute *cxl_memdev_ram_attributes[] = { &dev_attr_ram_size.attr, + &dev_attr_ram_qos_class0.attr, NULL, };
Export qos_class sysfs attributes for the CXL memory device. The QoS clas should show up as /sys/bus/cxl/devices/memX/ram/qos_class0 for the volatile partition and /sys/bus/cxl/devices/memX/pmem/qos_class0 for the persistent partition. The QTG ID is retrieved via _DSM after supplying the calculated bandwidth and latency for the entire CXL path from device to the CPU. This ID is used to match up to the root decoder QoS class to determine which CFMWS the memory range of a hotplugged CXL mem device should be assigned under. While there may be multiple DSMAS exported by the device CDAT, the driver will only expose the first QTG ID per partition in sysfs for now. In the future when multiple QTG IDs are necessary, they can be exposed. [1] [1]: https://lore.kernel.org/linux-cxl/167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local/T/#md2a47b1ead3e1ba08f50eab29a4af1aed1d215ab Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Dave Jiang <dave.jiang@intel.com> --- v10: - Export only qos_class0, the first entry. Additional qos_class entries can be exported later as needed. (Dan) - Have the sysfs attrib return -ENOENT unless driver is attached. (Dan) - Removed Jonathan's review tag due to code changes. --- Documentation/ABI/testing/sysfs-bus-cxl | 34 +++++++++++++++++++++++++++++++ drivers/cxl/core/memdev.c | 34 +++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+)