From patchwork Fri Feb 10 09:05:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80882C636CD for ; Fri, 10 Feb 2023 09:06:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 132986B011D; Fri, 10 Feb 2023 04:06:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E2996B011F; Fri, 10 Feb 2023 04:06:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EECC16B0120; Fri, 10 Feb 2023 04:06:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CA4BC6B011D for ; Fri, 10 Feb 2023 04:06:15 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8170D16150C for ; Fri, 10 Feb 2023 09:06:15 +0000 (UTC) X-FDA: 80450800710.11.AFA1890 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id BC95EA000F for ; Fri, 10 Feb 2023 09:06:12 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fvyPPhsC; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019973; a=rsa-sha256; cv=none; b=JPPbJ7FGWrq9LMy65xiBMMAEUDXr87bo+fTTwpDaaVdSm+C/Mc8xkY3sMzAKWJ6k2044CX xm8gQA80Erzv45HWrOZUrG1QnNz29DuwG+5kpMr8K4+kkpZi8fEOBJ5xN5zlkHsIyT8+06 9YI1QymGw0pyhgdHmQUHhEEQWmMbdxs= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fvyPPhsC; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wwuxx7bh+7ePNt2vu7+dCEZZpWGglxPpYkNjrsP1BZ4=; b=np1+1vMt061XJRocbJ/eJKN3W2q/gMtbV+8UKOb1T5bCHrUrVW83gC+/f84PeXRFVJQIw6 fCQxEIcYgFtI/rfbV8hQK48UQ1TOeDpir1FSMOivBduWpBOy49qz49KLH3aVwpkGF/+j2f V3uml3a0JAnP1xoXQtJ/N+yRmIAb4sU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019973; x=1707555973; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g9WOYiZgLx/JR1gosHNg5WOS/EaNuj1iypU5OOVGJsQ=; b=fvyPPhsCH3Kd81pwOUvnvW1vUwNtZT5ANa+lZNe/rpykXGlPTBnouuNi YhqepTOj5cxWEI8hw6Mh/SHAN0t2aH7M+MIVhWolSYwipKkj03jWvMvc5 5gChsO2BL91D+X/gm5V3QgNw7hWF8M8IsN4jiIDg1VaKgGjnJhFEfJGmL 6i8YwVCCARWDPIPILcwxf7rIMryZ+Abub+C8j9e/d9YV90EyN/gRXQQt6 2rYQcyaMZDQFA0m1MoUWkXiCocP9AzRRZnwNj5q9tCRRSyAcnjNaC1GSd Izpe/Vw+0vvhu/W0SZG4MhBE6Id3UcAA4j9Zp3PRudRRjm6IZeD9CWGcM A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416600026" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416600026" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:58 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930220" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930220" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:58 -0800 Subject: [PATCH v2 06/20] cxl/region: Add volatile region creation support From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Gregory Price , Dave Jiang , Ira Weiny , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:57 -0800 Message-ID: <167601995775.1924368.352616146815830591.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: BC95EA000F X-Rspamd-Server: rspam01 X-Stat-Signature: c8gmzquen6qmwnac9xmtuz3fu6cyurhm X-HE-Tag: 1676019972-590851 X-HE-Meta: U2FsdGVkX187xrGWYPF6pef5vTHO9HQj5CB3aYD0W4Tio1l0ie9k7IuFRQJuaLBSxWuZv80ReGy51D8/mqmaY1831C/Dev2aR9EsYNqaqhTtBoKYV0ntCMW6kM26hOKNNpLAYHnIIxBvXSLG0VW6M+wlLxNqXG0yYsd+oQgcxWUNcTum5iuADGj1iRN6qrQTuLC/tBHu9+nxwIeNS97bM1tPxbPXQLcuMT09UsQeHhfr+2gN53AQfTvaEhYUcI6UNIEN21/tGhx6y+SAtCSjMGgBDCtoVb296U3xae17QF0+cC3j6Ws6JoP3M8tgANUd1bu3+A6opmuniibfigN+s4W9Ft93EDJiy6EXwQ2Sk0hBN3yduZcTkPJfTD8o+dp6bV+gy4o8QV4yt3LgyzRFQJHeTzcmyoL91wMaKxIXBzD8iuCI5yPapKci5EB3aqaneb51a9UA4PqX5PVwaKzK1nvx1n5QWEIVKhV/z0jDORlCEHLimJFVTh/d4thDJC2T82Dto/nqN7a43ue63hHFzS15EZ3zCfzOLOtlblB+oHDMnwXfqvHYCHFBxsD1jrB7FhpZL+ed7po1Mu9RxCKlL2U5tD+MwmYOgEEigOQzC8Y8EoxlTHqEVseh/FaV4oYbSjQtJpl4+7kkRGACd7rJIwssIbtXvVtwisLtueXAPSob2WwM6eJS6XH7PnfviOfgg9s+/5GbnkPmquiPycsG5jg2vd5tADiBxkUoiBcLWjEqxZaeSKz+qmtZ4OMy6mhri7XRfxIvBeGDbVCC5hlXUEI1chGRyxRy84WlxbYLujc5Hx/vd0epw3eIF/28PQ7uKbVRvULJtTeKR6VR5Da5E/F9J2ouT0X2ltRR+2/GIW2V/xyr7jHsoFRHLB5BSvXkHZOVgd+movhlflOO3d3Q6q/IWQGH8sdLs7FLltwhEmkABagYMcBIcpx4W4WXR5ucM/oSHem6gi+tHG3m6WH GfAobVnD 2AdrtkDV+Aajjyw8XhmBa4nYHVzyK+sBaWO+/6pMj0sD8hUXuBWAOJp6ghKjtHLN9Jq5/88yNiBrJfMlsnTA8kNuIOZhitKrxc5H+kpHehY47l2IBJwWax/x0XmAfZ+DX4dEZw+1KcJjFmJCZboQ6FLhOhCiLNWO5qecfVX5/lZ8tAenacck9F0BiCGv7osv66I0C6R5Oyz6COqTP0T3hTTtr8YTxOrddsj1KRJCw5B0W0zGCDttGz3Mjslj81c7FV9D4KMCv7Qp29/OiQlzQmu+MfV3lNu0nzJWB0RiCiM94QeZyig+GIpANl+/bVOOgkvZPa68+1L++cXEqTw037jNBm0ci2VkGQNxvs9kFNSJhucELf4aYIuRxqthKsn4xTAOD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Expand the region creation infrastructure to enable 'ram' (volatile-memory) regions. The internals of create_pmem_region_store() and create_pmem_region_show() are factored out into helpers __create_region() and __create_region_show() for the 'ram' case to reuse. Reviewed-by: Vishal Verma Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564537678.847146.4066579806086171091.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++----- drivers/cxl/core/core.h | 1 drivers/cxl/core/port.c | 14 ++++++ drivers/cxl/core/region.c | 71 +++++++++++++++++++++++++------ 4 files changed, 83 insertions(+), 25 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 4c4e1cbb1169..3acf2f17a73f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -285,20 +285,20 @@ Description: interleave_granularity). -What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region -Date: May, 2022 -KernelVersion: v6.0 +What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region +Date: May, 2022, January, 2023 +KernelVersion: v6.0 (pmem), v6.3 (ram) Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process - of defining a new persistent memory region (interleave-set) - within the decode range bounded by root decoder 'decoderX.Y'. - The value written must match the current value returned from - reading this attribute. An atomic compare exchange operation is - done on write to assign the requested id to a region and - allocate the region-id for the next creation attempt. EBUSY is - returned if the region name written does not match the current - cached value. + of defining a new persistent, or volatile memory region + (interleave-set) within the decode range bounded by root decoder + 'decoderX.Y'. The value written must match the current value + returned from reading this attribute. An atomic compare exchange + operation is done on write to assign the requested id to a + region and allocate the region-id for the next creation attempt. + EBUSY is returned if the region name written does not match the + current cached value. What: /sys/bus/cxl/devices/decoderX.Y/delete_region diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..5eb873da5a30 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -11,6 +11,7 @@ extern struct attribute_group cxl_base_attribute_group; #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; +extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1e541956f605..9e5df64ea6b5 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -294,6 +294,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, SET_CXL_REGION_ATTR(create_pmem_region) + SET_CXL_REGION_ATTR(create_ram_region) SET_CXL_REGION_ATTR(delete_region) NULL, }; @@ -305,6 +306,13 @@ static bool can_create_pmem(struct cxl_root_decoder *cxlrd) return (cxlrd->cxlsd.cxld.flags & flags) == flags; } +static bool can_create_ram(struct cxl_root_decoder *cxlrd) +{ + unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + + return (cxlrd->cxlsd.cxld.flags & flags) == flags; +} + static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute *a, int n) { struct device *dev = kobj_to_dev(kobj); @@ -313,7 +321,11 @@ static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute * if (a == CXL_REGION_ATTR(create_pmem_region) && !can_create_pmem(cxlrd)) return 0; - if (a == CXL_REGION_ATTR(delete_region) && !can_create_pmem(cxlrd)) + if (a == CXL_REGION_ATTR(create_ram_region) && !can_create_ram(cxlrd)) + return 0; + + if (a == CXL_REGION_ATTR(delete_region) && + !(can_create_pmem(cxlrd) || can_create_ram(cxlrd))) return 0; return a->mode; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 285835145e9b..e440db8611a4 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1689,6 +1689,15 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, struct device *dev; int rc; + switch (mode) { + case CXL_DECODER_RAM: + case CXL_DECODER_PMEM: + break; + default: + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + return ERR_PTR(-EINVAL); + } + cxlr = cxl_region_alloc(cxlrd, id); if (IS_ERR(cxlr)) return cxlr; @@ -1717,12 +1726,38 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(rc); } +static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) +{ + return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +} + static ssize_t create_pmem_region_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + return __create_region_show(to_cxl_root_decoder(dev), buf); +} - return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +static ssize_t create_ram_region_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return __create_region_show(to_cxl_root_decoder(dev), buf); +} + +static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, + enum cxl_decoder_mode mode, int id) +{ + int rc; + + rc = memregion_alloc(GFP_KERNEL); + if (rc < 0) + return ERR_PTR(rc); + + if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { + memregion_free(rc); + return ERR_PTR(-EBUSY); + } + + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, @@ -1731,29 +1766,39 @@ static ssize_t create_pmem_region_store(struct device *dev, { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); struct cxl_region *cxlr; - int id, rc; + int rc, id; rc = sscanf(buf, "region%d\n", &id); if (rc != 1) return -EINVAL; - rc = memregion_alloc(GFP_KERNEL); - if (rc < 0) - return rc; + cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id); + if (IS_ERR(cxlr)) + return PTR_ERR(cxlr); - if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { - memregion_free(rc); - return -EBUSY; - } + return len; +} +DEVICE_ATTR_RW(create_pmem_region); + +static ssize_t create_ram_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + struct cxl_region *cxlr; + int rc, id; - cxlr = devm_cxl_add_region(cxlrd, id, CXL_DECODER_PMEM, - CXL_DECODER_EXPANDER); + rc = sscanf(buf, "region%d\n", &id); + if (rc != 1) + return -EINVAL; + + cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); return len; } -DEVICE_ATTR_RW(create_pmem_region); +DEVICE_ATTR_RW(create_ram_region); static ssize_t region_show(struct device *dev, struct device_attribute *attr, char *buf)