From patchwork Fri Jun 11 00:22:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 12314447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 497A8C48BD1 for ; Fri, 11 Jun 2021 00:22:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 25E2561222 for ; Fri, 11 Jun 2021 00:22:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231261AbhFKAY0 (ORCPT ); Thu, 10 Jun 2021 20:24:26 -0400 Received: from mga05.intel.com ([192.55.52.43]:34366 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230346AbhFKAY0 (ORCPT ); Thu, 10 Jun 2021 20:24:26 -0400 IronPort-SDR: 3MpT+X1LJ+sqeB40bkpATRbFbqci90x8pZjUWsXqONuqFyfQ/vuh/eR7a972gGPMGslhQ5BrQL dLAe6hwxr+0Q== X-IronPort-AV: E=McAfee;i="6200,9189,10011"; a="291064267" X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="291064267" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:29 -0700 IronPort-SDR: OiS29MLS+qKWLLIgV6ft7S7MuESkEeJ5yJnWhWmCnbGqwfrejkvXWB7uQYu+PmslNZGWRiwzqM tYaXtSziaiXw== X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="638530583" Received: from iweiny-desk2.sc.intel.com (HELO localhost) ([10.3.52.147]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:28 -0700 From: ira.weiny@intel.com To: Dan Williams Cc: Ira Weiny , Alison Schofield , Vishal Verma , Ben Widawsky , linux-cxl@vger.kernel.org Subject: [PATCH 1/3] cxl/pci: Store memory capacity values Date: Thu, 10 Jun 2021 17:22:22 -0700 Message-Id: <20210611002224.1594913-2-ira.weiny@intel.com> X-Mailer: git-send-email 2.28.0.rc0.12.gb6a658bd00c9 In-Reply-To: <20210611002224.1594913-1-ira.weiny@intel.com> References: <20210611002224.1594913-1-ira.weiny@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Ira Weiny The Identify Memory Device command returns information about the volatile and persistent memory capacities. Store those values in the cxl_mem structure for later use. While at it, reuse the calculation of the volatile and persistent memory byte values to calculate the ram and pmem ranges. Signed-off-by: Ira Weiny Acked-by: Ben Widawsky --- drivers/cxl/mem.h | 4 ++++ drivers/cxl/pci.c | 36 +++++++++++++++++++++++++++++++++--- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/mem.h b/drivers/cxl/mem.h index 13868ff7cadf..8bd0d0506b97 100644 --- a/drivers/cxl/mem.h +++ b/drivers/cxl/mem.h @@ -75,5 +75,9 @@ struct cxl_mem { struct range pmem_range; struct range ram_range; + u64 total_cap_bytes; + u64 volatile_cap_bytes; + u64 persistent_cap_bytes; + u64 partition_align_bytes; }; #endif /* __CXL_MEM_H__ */ diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 5a1705b52278..9995f97d3b28 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -57,6 +57,15 @@ enum opcode { CXL_MBOX_OP_MAX = 0x10000 }; +/* + * CXL 2.0 - Memory capacity multiplier + * See Section 8.2.9.5 + * + * Volatile, Persistent, and Partition capacities are specified to be in + * multiples of 256MB - define a multiplier to convert to/from bytes. + */ +#define CXL_CAPACITY_MULTIPLIER SZ_256M + /** * struct mbox_cmd - A command to be submitted to hardware. * @opcode: (input) The command set and command submitted to hardware. @@ -1542,16 +1551,37 @@ static int cxl_mem_identify(struct cxl_mem *cxlm) if (rc < 0) return rc; + cxlm->total_cap_bytes = le64_to_cpu(id.total_capacity); + cxlm->total_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + + cxlm->volatile_cap_bytes = le64_to_cpu(id.volatile_capacity); + cxlm->volatile_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + + cxlm->persistent_cap_bytes = le64_to_cpu(id.persistent_capacity); + cxlm->persistent_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + + cxlm->partition_align_bytes = le64_to_cpu(id.partition_align); + cxlm->partition_align_bytes *= CXL_CAPACITY_MULTIPLIER; + + dev_dbg(&cxlm->pdev->dev, "Identify Memory Device\n" + " total_cap_bytes = %#llx\n" + " volatile_cap_bytes = %#llx\n" + " persistent_cap_bytes = %#llx\n" + " partition_align_bytes = %#llx\n", + cxlm->total_cap_bytes, + cxlm->volatile_cap_bytes, + cxlm->persistent_cap_bytes, + cxlm->partition_align_bytes); + /* * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias. * For now, only the capacity is exported in sysfs */ cxlm->ram_range.start = 0; - cxlm->ram_range.end = le64_to_cpu(id.volatile_capacity) * SZ_256M - 1; + cxlm->ram_range.end = cxlm->volatile_cap_bytes - 1; cxlm->pmem_range.start = 0; - cxlm->pmem_range.end = - le64_to_cpu(id.persistent_capacity) * SZ_256M - 1; + cxlm->pmem_range.end = cxlm->persistent_cap_bytes - 1; cxlm->lsa_size = le32_to_cpu(id.lsa_size); memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision)); From patchwork Fri Jun 11 00:22:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 12314451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A98E2C48BE0 for ; Fri, 11 Jun 2021 00:22:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91DEC613C3 for ; Fri, 11 Jun 2021 00:22:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231381AbhFKAY1 (ORCPT ); Thu, 10 Jun 2021 20:24:27 -0400 Received: from mga05.intel.com ([192.55.52.43]:34366 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231322AbhFKAY1 (ORCPT ); Thu, 10 Jun 2021 20:24:27 -0400 IronPort-SDR: 9MT7D94f9MUacOnmyjIjZ4zGhVgAmOMgcSMYHSOvicywJH5mlXyRIv2KIwJpU9eEE8rrLM9EBg C2adhrm8p8/A== X-IronPort-AV: E=McAfee;i="6200,9189,10011"; a="291064272" X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="291064272" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:30 -0700 IronPort-SDR: Cuxhiakn1bazL3DtblVD3C6C/LJ863OtGpK+mWjcgmpMt+NonhxQA4UIcyBnoNUx3glrNjDS4J +GBtz03CqYZA== X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="638530587" Received: from iweiny-desk2.sc.intel.com (HELO localhost) ([10.3.52.147]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:29 -0700 From: ira.weiny@intel.com To: Dan Williams Cc: Ira Weiny , Alison Schofield , Vishal Verma , Ben Widawsky , linux-cxl@vger.kernel.org Subject: [PATCH 2/3] cxl/mem: Report correct ram/pmem size in sysfs Date: Thu, 10 Jun 2021 17:22:23 -0700 Message-Id: <20210611002224.1594913-3-ira.weiny@intel.com> X-Mailer: git-send-email 2.28.0.rc0.12.gb6a658bd00c9 In-Reply-To: <20210611002224.1594913-1-ira.weiny@intel.com> References: <20210611002224.1594913-1-ira.weiny@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Ira Weiny Memory devices may specify volatile only, persistent only, and partitionable space which when added together result in a total capacity. The partitionable space is configurable between volatile and persistent space. To account for the dynamic partitionable space the correct ram and pmem size information is reported in the Get Partition Info device command. Define cxl_mem_get_partition() and call it to retrieve the correct ram and pmem ranges sizes. Signed-off-by: Ira Weiny --- drivers/cxl/pci.c | 97 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 90 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 9995f97d3b28..bcc2829e4475 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1455,6 +1455,62 @@ static struct cxl_mbox_get_supported_logs *cxl_get_gsl(struct cxl_mem *cxlm) return ret; } +/** + * cxl_mem_get_partition_info - Set partition info + * @cxlm: The device to act on + * @active_volatile_cap_bytes: returned active volatile capacity; in bytes + * @active_persistent_cap_bytes: returned active persistent capacity; in bytes + * @next_volatile_cap_bytes: return next volatile capacity; in bytes + * @next_persistent_cap_bytes: return next persistent capacity; in bytes + * + * Retrieve the current partition info for the device specified. The active + * values are the current capacity in bytes. If not 0, the 'next' values are + * the pending values, in bytes, which take affect on next cold reset. + * + * Return: 0 if no error: or the result of the mailbox command. + * + * See CXL @8.2.9.5.2.1 Get Partition Info + */ +int cxl_mem_get_partition_info(struct cxl_mem *cxlm, + u64 *active_volatile_cap_bytes, + u64 *active_persistent_cap_bytes, + u64 *next_volatile_cap_bytes, + u64 *next_persistent_cap_bytes) +{ + struct cxl_mbox_get_partition_info { + u64 active_volatile_cap; + u64 active_persistent_cap; + u64 next_volatile_cap; + u64 next_persistent_cap; + } __packed pi; + int rc; + + /* On error report 0 */ + *active_volatile_cap_bytes = 0; + *active_persistent_cap_bytes = 0; + *next_volatile_cap_bytes = 0; + *next_persistent_cap_bytes = 0; + + rc = cxl_mem_mbox_send_cmd(cxlm, CXL_MBOX_OP_GET_PARTITION_INFO, + NULL, 0, &pi, sizeof(pi)); + + if (rc) + return rc; + + *active_volatile_cap_bytes = le64_to_cpu(pi.active_volatile_cap); + *active_persistent_cap_bytes = le64_to_cpu(pi.active_persistent_cap); + *next_volatile_cap_bytes = le64_to_cpu(pi.next_volatile_cap); + *next_persistent_cap_bytes = le64_to_cpu(pi.next_volatile_cap); + + *active_volatile_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + *active_persistent_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + *next_volatile_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + *next_persistent_cap_bytes *= CXL_CAPACITY_MULTIPLIER; + + return 0; +} +EXPORT_SYMBOL_GPL(cxl_mem_get_partition_info); + /** * cxl_mem_enumerate_cmds() - Enumerate commands for a device. * @cxlm: The device. @@ -1573,20 +1629,45 @@ static int cxl_mem_identify(struct cxl_mem *cxlm) cxlm->persistent_cap_bytes, cxlm->partition_align_bytes); + cxlm->lsa_size = le32_to_cpu(id.lsa_size); + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision)); + + return 0; +} + +static void cxl_mem_create_range_info(struct cxl_mem *cxlm) +{ + u64 active_volatile_cap_bytes; + u64 active_persistent_cap_bytes; + u64 next_volatile_cap_bytes; + u64 next_persistent_cap_bytes; + + if (cxl_mem_get_partition_info(cxlm, + &active_volatile_cap_bytes, + &active_persistent_cap_bytes, + &next_volatile_cap_bytes, + &next_persistent_cap_bytes)) + dev_err(&cxlm->pdev->dev, "Failed to query partition information\n"); + + dev_dbg(&cxlm->pdev->dev, "Get Partition Info\n" + " active_volatile_cap_bytes = %#llx\n" + " active_persistent_cap_bytes = %#llx\n" + " next_volatile_cap_bytes = %#llx\n" + " next_persistent_cap_bytes = %#llx\n", + active_volatile_cap_bytes, + active_persistent_cap_bytes, + next_volatile_cap_bytes, + next_persistent_cap_bytes); + /* * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias. * For now, only the capacity is exported in sysfs */ cxlm->ram_range.start = 0; - cxlm->ram_range.end = cxlm->volatile_cap_bytes - 1; + cxlm->ram_range.end = active_volatile_cap_bytes - 1; cxlm->pmem_range.start = 0; - cxlm->pmem_range.end = cxlm->persistent_cap_bytes - 1; - - cxlm->lsa_size = le32_to_cpu(id.lsa_size); - memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision)); - - return 0; + cxlm->pmem_range.end = active_persistent_cap_bytes - 1; } static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) @@ -1618,6 +1699,8 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + cxl_mem_create_range_info(cxlm); + return cxl_mem_add_memdev(cxlm); } From patchwork Fri Jun 11 00:22:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 12314453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BEF4C49360 for ; Fri, 11 Jun 2021 00:22:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 55B99613D5 for ; Fri, 11 Jun 2021 00:22:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231272AbhFKAY3 (ORCPT ); Thu, 10 Jun 2021 20:24:29 -0400 Received: from mga05.intel.com ([192.55.52.43]:34366 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231396AbhFKAY2 (ORCPT ); Thu, 10 Jun 2021 20:24:28 -0400 IronPort-SDR: /SYsY0Ib0NaCFgyXCGle6prNWVw+1jAYDZ/HVNDczKuP40sxEVuMUbd+TEi5CiiIAWF3Zcl8Za S+uoI9ABiDog== X-IronPort-AV: E=McAfee;i="6200,9189,10011"; a="291064275" X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="291064275" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:31 -0700 IronPort-SDR: PcsiBMx0cui7c74ucbclh7j8UrVuV0gF3hnz4N4LDj0Bz22jveN1h/7oOyht8y+KFOr364OqPB /3lc1PSIGQog== X-IronPort-AV: E=Sophos;i="5.83,264,1616482800"; d="scan'208";a="638530603" Received: from iweiny-desk2.sc.intel.com (HELO localhost) ([10.3.52.147]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jun 2021 17:22:31 -0700 From: ira.weiny@intel.com To: Dan Williams Cc: Ira Weiny , Alison Schofield , Vishal Verma , Ben Widawsky , linux-cxl@vger.kernel.org Subject: [PATCH 3/3] cxl/mem: Add partition information to sysfs Date: Thu, 10 Jun 2021 17:22:24 -0700 Message-Id: <20210611002224.1594913-4-ira.weiny@intel.com> X-Mailer: git-send-email 2.28.0.rc0.12.gb6a658bd00c9 In-Reply-To: <20210611002224.1594913-1-ira.weiny@intel.com> References: <20210611002224.1594913-1-ira.weiny@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Ira Weiny Partitionable space is added to the ram and pmem size sysfs attributes but this does not show the entire story. Add full partition information about the device under /partition. Signed-off-by: Ira Weiny --- drivers/cxl/pci.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index bcc2829e4475..7e4e9605e4ed 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1226,6 +1226,42 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_pmem_size = __ATTR(size, 0444, pmem_size_show, NULL); +static ssize_t part_vo_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + + return sysfs_emit(buf, "%#llx\n", cxlm->volatile_cap_bytes); +} + +static struct device_attribute dev_attr_part_vo = + __ATTR(volatile_only, 0444, part_vo_show, NULL); + +static ssize_t part_po_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + + return sysfs_emit(buf, "%#llx\n", cxlm->persistent_cap_bytes); +} + +static struct device_attribute dev_attr_part_po = + __ATTR(persistent_only, 0444, part_po_show, NULL); + +static ssize_t part_total_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_mem *cxlm = cxlmd->cxlm; + + return sysfs_emit(buf, "%#llx\n", cxlm->total_cap_bytes); +} + +static struct device_attribute dev_attr_part_total = + __ATTR(total, 0444, part_total_show, NULL); + static struct attribute *cxl_memdev_attributes[] = { &dev_attr_firmware_version.attr, &dev_attr_payload_max.attr, @@ -1243,6 +1279,13 @@ static struct attribute *cxl_memdev_ram_attributes[] = { NULL, }; +static struct attribute *cxl_memdev_part_attributes[] = { + &dev_attr_part_vo.attr, + &dev_attr_part_po.attr, + &dev_attr_part_total.attr, + NULL, +}; + static struct attribute_group cxl_memdev_attribute_group = { .attrs = cxl_memdev_attributes, }; @@ -1257,10 +1300,16 @@ static struct attribute_group cxl_memdev_pmem_attribute_group = { .attrs = cxl_memdev_pmem_attributes, }; +static struct attribute_group cxl_memdev_part_attribute_group = { + .name = "partition", + .attrs = cxl_memdev_part_attributes, +}; + static const struct attribute_group *cxl_memdev_attribute_groups[] = { &cxl_memdev_attribute_group, &cxl_memdev_ram_attribute_group, &cxl_memdev_pmem_attribute_group, + &cxl_memdev_part_attribute_group, NULL, };