From patchwork Thu Jul 14 17:01:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E0F1C433EF for ; Thu, 14 Jul 2022 17:02:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234258AbiGNRCD (ORCPT ); Thu, 14 Jul 2022 13:02:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbiGNRCC (ORCPT ); Thu, 14 Jul 2022 13:02:02 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4A59509DC for ; Thu, 14 Jul 2022 10:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818121; x=1689354121; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EpKSZRUg6s+nHr80Ig7EQknua3MGIHFjqj++oJFMvrE=; b=bsM53VRmGUjBSPY1XJqZea7YTv0bdXwSkyWduynr/LipKASDVT6YlDvd 9PgFK649AnPApoVQkVwfLBBuTCeHuVc4bHoyDv25Njeo5p0ydOZFBqvnr jtxotXz49PDQrQhSFARLSzKmGQnJoCDkMgGlb3ZTjEiRecZK3WruMsfca xwJ3Iae2yyJwdeTKU/ZwRCGX1tQ+DqeqrYKySh3IhwKsvQUynHY+Hsjw7 ajHsN+eowEZjs0lG50+XNeYzlsBVKNyIqjGM+C8zZ25PspVzgtxM9h0rj QAYzBMTphWMFcfOuJskqYmTdlJKdvSGOidMYrNakVwtrRZf23Cte+VtrP Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265987121" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="265987121" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:53 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772693773" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:53 -0700 Subject: [ndctl PATCH v2 01/12] cxl/list: Reformat option list From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:01:52 -0700 Message-ID: <165781811294.1555691.6271986101970794441.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Cleanup some spurious spaces and let clang-format re-layout the options. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/list.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/cxl/list.c b/cxl/list.c index 940782d33a10..1b5f58328047 100644 --- a/cxl/list.c +++ b/cxl/list.c @@ -36,8 +36,7 @@ static const struct option options[] = { "filter by CXL endpoint device name(s)"), OPT_BOOLEAN('E', "endpoints", ¶m.endpoints, "include CXL endpoint info"), - OPT_STRING('d', "decoder", ¶m.decoder_filter, - "decoder device name", + OPT_STRING('d', "decoder", ¶m.decoder_filter, "decoder device name", "filter by CXL decoder device name(s) / class"), OPT_BOOLEAN('D', "decoders", ¶m.decoders, "include CXL decoder info"), @@ -45,11 +44,11 @@ static const struct option options[] = { "include CXL target data with decoders or ports"), OPT_BOOLEAN('i', "idle", ¶m.idle, "include disabled devices"), OPT_BOOLEAN('u', "human", ¶m.human, - "use human friendly number formats "), + "use human friendly number formats"), OPT_BOOLEAN('H', "health", ¶m.health, - "include memory device health information "), + "include memory device health information"), OPT_BOOLEAN('I', "partition", ¶m.partition, - "include memory device partition information "), + "include memory device partition information"), #ifdef ENABLE_DEBUG OPT_BOOLEAN(0, "debug", &debug, "debug list walk"), #endif From patchwork Thu Jul 14 17:01:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36221C43334 for ; Thu, 14 Jul 2022 17:03:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240099AbiGNRDA (ORCPT ); Thu, 14 Jul 2022 13:03:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240107AbiGNRC4 (ORCPT ); Thu, 14 Jul 2022 13:02:56 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76D6A5E318 for ; Thu, 14 Jul 2022 10:02:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818171; x=1689354171; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AeWJ82o6BwFn1NlqdqKUFKk+NqL3Dqc98exzk9j2WHE=; b=bhN0cjFnwTz8cee97uwZp7SVT8Adcd/XfzXO9xBUSBix8Y+npjbW2zHn v0Qbp9qw2cnv1w+qs91NQAQZQn54DTcX2DSBWHLDvTcVo/YL36IwYMJD2 b7XGy/0bAqhjY5o9aBzHzdDwNMr9LmnBAeiHuLhCskT7VZ2pewEL0XQh+ /HKi0NnTMyqIMPYlg0Bbt2g4weky3OP8mdiCsW5DCPOcBFEeuXlzmB2EM nhN7O0EoFChgV8LjYdVLo2jTHjfXW2btO9+5Ol3wqFnulHoQTmwJyVCoL XQbwxQXNald6VUGLpvSIg5wxjg6s5ktDZuOK10wMDJp7Z+BD3U5VEJaYh g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="284331586" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="284331586" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:59 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="685643797" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:58 -0700 Subject: [ndctl PATCH v2 02/12] cxl/list: Emit endpoint decoders filtered by memdev From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:01:58 -0700 Message-ID: <165781811836.1555691.1997564050287016121.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org For example, dump all the endpoint decoders from memdev 'mem8'. cxl list -Di -m 8 -d endpoint Signed-off-by: Dan Williams --- cxl/filter.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/cxl/filter.c b/cxl/filter.c index 66fd7420144a..2f88a9d2f398 100644 --- a/cxl/filter.c +++ b/cxl/filter.c @@ -428,7 +428,9 @@ util_cxl_decoder_filter_by_memdev(struct cxl_decoder *decoder, const char *ident, const char *serial) { struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + struct cxl_endpoint *endpoint; struct cxl_memdev *memdev; + struct cxl_port *port; if (!ident && !serial) return decoder; @@ -438,6 +440,12 @@ util_cxl_decoder_filter_by_memdev(struct cxl_decoder *decoder, continue; if (cxl_decoder_get_target_by_memdev(decoder, memdev)) return decoder; + port = cxl_decoder_get_port(decoder); + if (!port || !cxl_port_is_endpoint(port)) + continue; + endpoint = cxl_port_to_endpoint(port); + if (cxl_endpoint_get_memdev(endpoint) == memdev) + return decoder; } return NULL; From patchwork Thu Jul 14 17:02:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03358CCA47B for ; Thu, 14 Jul 2022 17:02:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239063AbiGNRCS (ORCPT ); Thu, 14 Jul 2022 13:02:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238542AbiGNRCR (ORCPT ); Thu, 14 Jul 2022 13:02:17 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20F374E851 for ; Thu, 14 Jul 2022 10:02:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818137; x=1689354137; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NgVTcO/N8lNlztGxQoim/NbcYQnweRCnSkSjx/KcjhI=; b=GUNZPZ629Y11SkuPSrqSw84+kLpoDK4R+phTrLy3gzhuo3xCns6cMz8Y +mvyTC17Zc3tHIJRjmoudyAFS3Ctel0982RLAR0sCvfTVmBEzTHzF//T3 dlc4TijCn0D08GHloqhBuN1bU3Edyo7WJ2AbGDDjOkaVkmHwJu6Svtpzq 62pP1qeYZUgE9Q//zYJMEjdK66Q7ADa4z8gGTxnJHPpaJJdJBVewuKfbX M6H+ikvpeUdJoPNKaLAeF2g9SxBFX2gl5StmnEENUtTKLq8shYvjNnOiu S8eaj7tspArkb23wj4nlqZP5UYb5yB1NRRe8DTSCO7jSqs46RazcrKfK8 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="349542022" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="349542022" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:04 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772693904" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:04 -0700 Subject: [ndctl PATCH v2 03/12] cxl/list: Hide 0s in disabled decoder listings From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:04 -0700 Message-ID: <165781812427.1555691.5252994293073680408.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Trim some redundant information from decoder listings when they are disabled. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/json.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/cxl/json.c b/cxl/json.c index fdc6f73a86c1..a213fdad55fd 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -442,7 +442,7 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, const char *devname = cxl_decoder_get_devname(decoder); struct cxl_port *port = cxl_decoder_get_port(decoder); struct json_object *jdecoder, *jobj; - u64 val; + u64 val, size; jdecoder = json_object_new_object(); if (!jdecoder) @@ -452,21 +452,21 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, if (jobj) json_object_object_add(jdecoder, "decoder", jobj); + size = cxl_decoder_get_size(decoder); val = cxl_decoder_get_resource(decoder); - if (val < ULLONG_MAX) { + if (size && val < ULLONG_MAX) { jobj = util_json_object_hex(val, flags); if (jobj) json_object_object_add(jdecoder, "resource", jobj); } - val = cxl_decoder_get_size(decoder); - if (val < ULLONG_MAX) { - jobj = util_json_object_size(val, flags); + if (size && size < ULLONG_MAX) { + jobj = util_json_object_size(size, flags); if (jobj) json_object_object_add(jdecoder, "size", jobj); } - if (val == 0) { + if (size == 0) { jobj = json_object_new_string("disabled"); if (jobj) json_object_object_add(jdecoder, "state", jobj); From patchwork Thu Jul 14 17:02:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF94FC43334 for ; Thu, 14 Jul 2022 17:02:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238104AbiGNRCN (ORCPT ); Thu, 14 Jul 2022 13:02:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbiGNRCL (ORCPT ); Thu, 14 Jul 2022 13:02:11 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40A794E851 for ; Thu, 14 Jul 2022 10:02:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818131; x=1689354131; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NR3xEc9IRdzsHzkHzpnmoYhBLcYxBL8o1b4hDwqsDX8=; b=lTSvvi2wsAGPO5pG5OOrsJN+NNdYz8o7pWmIllck2l594QUK8OOahlz4 ee/Zg7yh/VsRwFP9JLxwQdjZyQx2JDa/wI5JyoILT52kJ3Jz5hD7Ie1s4 OmovZ59XPYCQeaR6nyryqJQmWJMav1Lgklck6MOxIqSE4/j4L9rV5tjGl aq0GWq4SpBDI0514Qyxc7m2hRtARh/CSwnX0mMPbOX9rvWlFCT7lNLbMM +NiEd2inxDlC/262OZIidzFDH8nvWx4Haik2tKzQ129iHOjMTT+i2KaOj GkWgL9oLqCnpOiwuLhfwIUPXUy/jVPlf4/V+Qs9w0HWwatRJsaLgkrz1u w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265364104" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="265364104" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:10 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="663862091" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:10 -0700 Subject: [ndctl PATCH v2 04/12] cxl/list: Add DPA span to endpoint decoder listings From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:09 -0700 Message-ID: <165781812967.1555691.4685129673233918478.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Optionally include in decoder listings the device local address space for endpoint decoders with active / allocated capacity. Signed-off-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 2 ++ cxl/json.c | 18 ++++++++++++++++ cxl/lib/libcxl.c | 43 +++++++++++++++++++++++++++++++++++++- cxl/lib/libcxl.sym | 6 +++++ cxl/lib/private.h | 2 ++ cxl/libcxl.h | 2 ++ 6 files changed, 72 insertions(+), 1 deletion(-) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index f8f0e668ab59..2aef489e8e12 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -392,6 +392,8 @@ more CXL decoder objects. ---- unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); int cxl_decoder_get_id(struct cxl_decoder *decoder); int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder); diff --git a/cxl/json.c b/cxl/json.c index a213fdad55fd..3f52d3bbff45 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -472,6 +472,24 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add(jdecoder, "state", jobj); } + if (cxl_port_is_endpoint(port)) { + size = cxl_decoder_get_dpa_size(decoder); + val = cxl_decoder_get_dpa_resource(decoder); + if (size && val < ULLONG_MAX) { + jobj = util_json_object_hex(val, flags); + if (jobj) + json_object_object_add(jdecoder, "dpa_resource", + jobj); + } + + if (size && size < ULLONG_MAX) { + jobj = util_json_object_size(size, flags); + if (jobj) + json_object_object_add(jdecoder, "dpa_size", + jobj); + } + } + if (cxl_port_is_root(port) && cxl_decoder_is_mem_capable(decoder)) { if (cxl_decoder_is_pmem_capable(decoder)) { jobj = json_object_new_boolean(true); diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index c988ce2ddea9..f36edcfc735a 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -955,8 +955,19 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) decoder->size = strtoull(buf, NULL, 0); switch (port->type) { - case CXL_PORT_SWITCH: case CXL_PORT_ENDPOINT: + sprintf(path, "%s/dpa_resource", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) < 0) + decoder->dpa_resource = ULLONG_MAX; + else + decoder->dpa_resource = strtoull(buf, NULL, 0); + sprintf(path, "%s/dpa_size", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) < 0) + decoder->dpa_size = ULLONG_MAX; + else + decoder->dpa_size = strtoull(buf, NULL, 0); + + case CXL_PORT_SWITCH: decoder->pmem_capable = true; decoder->volatile_capable = true; decoder->mem_capable = true; @@ -1113,6 +1124,36 @@ CXL_EXPORT unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder) return decoder->size; } +CXL_EXPORT unsigned long long +cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return ULLONG_MAX; + } + + return decoder->dpa_resource; +} + +CXL_EXPORT unsigned long long +cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return ULLONG_MAX; + } + + return decoder->dpa_size; +} + CXL_EXPORT enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index dffcb60b8dd0..8e2fc75557f9 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -167,3 +167,9 @@ global: cxl_cmd_new_set_partition; cxl_cmd_partition_set_mode; } LIBCXL_1; + +LIBCXL_3 { +global: + cxl_decoder_get_dpa_resource; + cxl_decoder_get_dpa_size; +} LIBCXL_2; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index c6d88f7140f2..24a2ae6787be 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -101,6 +101,8 @@ struct cxl_decoder { struct cxl_ctx *ctx; u64 start; u64 size; + u64 dpa_resource; + u64 dpa_size; void *dev_buf; size_t buf_len; char *dev_path; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 0007f4d9bcee..76aebe3efda8 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -129,6 +129,8 @@ struct cxl_decoder *cxl_decoder_get_first(struct cxl_port *port); struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); From patchwork Thu Jul 14 17:02:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B9CFC43334 for ; Thu, 14 Jul 2022 17:02:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238542AbiGNRCZ (ORCPT ); Thu, 14 Jul 2022 13:02:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239298AbiGNRCX (ORCPT ); Thu, 14 Jul 2022 13:02:23 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0568E5508F for ; Thu, 14 Jul 2022 10:02:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818141; x=1689354141; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eEjIKuScu5PsNnJ6b00Knr0ddNcwkSwfM5aF9xtCMgY=; b=Kik4GQCZshrxtNKFJsOsUB2JJjf49IPcypRhyyzU3leYxQ4E3U+iNrjk THNZXMam/bEcx8z0NN3WhFC1XVYTSF2+6P7o2rq6jUFfxmMXYo3kky9NK 7I1C/mU3zInsrbKxmXnfzz6uOS4QfNNF6vxjHGaCNm4WIMg/ca8zBiJDo 6GBGAV7iO8VsVovK9TWmaiGtxz2OnKzrs5FgAIMAZyO9PYMAp0unROrg4 HzNb24PD3yw1bOXcQnGGR0qte9/vhFHLxjpWqMQyqwyOr39ctdK5pAtUN LrksKV34IH8ayvz9t37HnFU3Iau8sGJqUXviq74K4BXM/xUTpzjui3+pw g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265987301" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="265987301" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:16 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="685643981" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:16 -0700 Subject: [ndctl PATCH v2 05/12] ccan/list: Import latest list helpers From: Dan Williams To: vishal.l.verma@intel.com Cc: Ira Weiny , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:15 -0700 Message-ID: <165781813572.1555691.15909358688944168922.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Pick up the definition of list_add_{before,after} and other updates from ccan at commit 52b86922f846 ("ccan/base64: fix GCC warning."). Reported-by: Ira Weiny Signed-off-by: Dan Williams Reviewed-by: Ira Weiny --- ccan/list/list.h | 258 +++++++++++++++++++++++++++++++++++++++++++++------- ndctl/lib/inject.c | 1 util/list.h | 40 -------- 3 files changed, 222 insertions(+), 77 deletions(-) delete mode 100644 util/list.h diff --git a/ccan/list/list.h b/ccan/list/list.h index 3ebd1b23dc0f..15f5fb7b34eb 100644 --- a/ccan/list/list.h +++ b/ccan/list/list.h @@ -95,8 +95,8 @@ struct list_node *list_check_node(const struct list_node *n, #define list_debug(h, loc) list_check((h), loc) #define list_debug_node(n, loc) list_check_node((n), loc) #else -#define list_debug(h, loc) (h) -#define list_debug_node(n, loc) (n) +#define list_debug(h, loc) ((void)loc, h) +#define list_debug_node(n, loc) ((void)loc, n) #endif /** @@ -111,7 +111,7 @@ struct list_node *list_check_node(const struct list_node *n, * Example: * static struct list_head my_list = LIST_HEAD_INIT(my_list); */ -#define LIST_HEAD_INIT(name) { { &name.n, &name.n } } +#define LIST_HEAD_INIT(name) { { &(name).n, &(name).n } } /** * LIST_HEAD - define and initialize an empty list_head @@ -145,6 +145,48 @@ static inline void list_head_init(struct list_head *h) h->n.next = h->n.prev = &h->n; } +/** + * list_node_init - initialize a list_node + * @n: the list_node to link to itself. + * + * You don't need to use this normally! But it lets you list_del(@n) + * safely. + */ +static inline void list_node_init(struct list_node *n) +{ + n->next = n->prev = n; +} + +/** + * list_add_after - add an entry after an existing node in a linked list + * @h: the list_head to add the node to (for debugging) + * @p: the existing list_node to add the node after + * @n: the new list_node to add to the list. + * + * The existing list_node must already be a member of the list. + * The new list_node does not need to be initialized; it will be overwritten. + * + * Example: + * struct child c1, c2, c3; + * LIST_HEAD(h); + * + * list_add_tail(&h, &c1.list); + * list_add_tail(&h, &c3.list); + * list_add_after(&h, &c1.list, &c2.list); + */ +#define list_add_after(h, p, n) list_add_after_(h, p, n, LIST_LOC) +static inline void list_add_after_(struct list_head *h, + struct list_node *p, + struct list_node *n, + const char *abortstr) +{ + n->next = p->next; + n->prev = p; + p->next->prev = n; + p->next = n; + (void)list_debug(h, abortstr); +} + /** * list_add - add an entry at the start of a linked list. * @h: the list_head to add the node to @@ -163,10 +205,34 @@ static inline void list_add_(struct list_head *h, struct list_node *n, const char *abortstr) { - n->next = h->n.next; - n->prev = &h->n; - h->n.next->prev = n; - h->n.next = n; + list_add_after_(h, &h->n, n, abortstr); +} + +/** + * list_add_before - add an entry before an existing node in a linked list + * @h: the list_head to add the node to (for debugging) + * @p: the existing list_node to add the node before + * @n: the new list_node to add to the list. + * + * The existing list_node must already be a member of the list. + * The new list_node does not need to be initialized; it will be overwritten. + * + * Example: + * list_head_init(&h); + * list_add_tail(&h, &c1.list); + * list_add_tail(&h, &c3.list); + * list_add_before(&h, &c3.list, &c2.list); + */ +#define list_add_before(h, p, n) list_add_before_(h, p, n, LIST_LOC) +static inline void list_add_before_(struct list_head *h, + struct list_node *p, + struct list_node *n, + const char *abortstr) +{ + n->next = p; + n->prev = p->prev; + p->prev->next = n; + p->prev = n; (void)list_debug(h, abortstr); } @@ -185,11 +251,7 @@ static inline void list_add_tail_(struct list_head *h, struct list_node *n, const char *abortstr) { - n->next = &h->n; - n->prev = h->n.prev; - h->n.prev->next = n; - h->n.prev = n; - (void)list_debug(h, abortstr); + list_add_before_(h, &h->n, n, abortstr); } /** @@ -229,6 +291,21 @@ static inline bool list_empty_nodebug(const struct list_head *h) } #endif +/** + * list_empty_nocheck - is a list empty? + * @h: the list_head + * + * If the list is empty, returns true. This doesn't perform any + * debug check for list consistency, so it can be called without + * locks, racing with the list being modified. This is ok for + * checks where an incorrect result is not an issue (optimized + * bail out path for example). + */ +static inline bool list_empty_nocheck(const struct list_head *h) +{ + return h->n.next == &h->n; +} + /** * list_del - delete an entry from an (unknown) linked list. * @n: the list_node to delete from the list. @@ -237,7 +314,7 @@ static inline bool list_empty_nodebug(const struct list_head *h) * another list, but not deleted again. * * See also: - * list_del_from() + * list_del_from(), list_del_init() * * Example: * list_del(&child->list); @@ -255,6 +332,27 @@ static inline void list_del_(struct list_node *n, const char* abortstr) #endif } +/** + * list_del_init - delete a node, and reset it so it can be deleted again. + * @n: the list_node to be deleted. + * + * list_del(@n) or list_del_init() again after this will be safe, + * which can be useful in some cases. + * + * See also: + * list_del_from(), list_del() + * + * Example: + * list_del_init(&child->list); + * parent->num_children--; + */ +#define list_del_init(n) list_del_init_(n, LIST_LOC) +static inline void list_del_init_(struct list_node *n, const char *abortstr) +{ + list_del_(n, abortstr); + list_node_init(n); +} + /** * list_del_from - delete an entry from a known linked list. * @h: the list_head the node is in. @@ -285,6 +383,39 @@ static inline void list_del_from(struct list_head *h, struct list_node *n) list_del(n); } +/** + * list_swap - swap out an entry from an (unknown) linked list for a new one. + * @o: the list_node to replace from the list. + * @n: the list_node to insert in place of the old one. + * + * Note that this leaves @o in an undefined state; it can be added to + * another list, but not deleted/swapped again. + * + * See also: + * list_del() + * + * Example: + * struct child x1, x2; + * LIST_HEAD(xh); + * + * list_add(&xh, &x1.list); + * list_swap(&x1.list, &x2.list); + */ +#define list_swap(o, n) list_swap_(o, n, LIST_LOC) +static inline void list_swap_(struct list_node *o, + struct list_node *n, + const char* abortstr) +{ + (void)list_debug_node(o, abortstr); + *n = *o; + n->next->prev = n; + n->prev->next = n; +#ifdef CCAN_LIST_DEBUG + /* Catch use-after-del. */ + o->next = o->prev = NULL; +#endif +} + /** * list_entry - convert a list_node back into the structure containing it. * @n: the list_node @@ -406,9 +537,29 @@ static inline const void *list_tail_(const struct list_head *h, size_t off) * printf("Name: %s\n", child->name); */ #define list_for_each_rev(h, i, member) \ - for (i = container_of_var(list_debug(h, LIST_LOC)->n.prev, i, member); \ - &i->member != &(h)->n; \ - i = container_of_var(i->member.prev, i, member)) + list_for_each_rev_off(h, i, list_off_var_(i, member)) + +/** + * list_for_each_rev_safe - iterate through a list backwards, + * maybe during deletion + * @h: the list_head + * @i: the structure containing the list_node + * @nxt: the structure containing the list_node + * @member: the list_node member of the structure + * + * This is a convenient wrapper to iterate @i over the entire list backwards. + * It's a for loop, so you can break and continue as normal. The extra + * variable * @nxt is used to hold the next element, so you can delete @i + * from the list. + * + * Example: + * struct child *next; + * list_for_each_rev_safe(&parent->children, child, next, list) { + * printf("Name: %s\n", child->name); + * } + */ +#define list_for_each_rev_safe(h, i, nxt, member) \ + list_for_each_rev_safe_off(h, i, nxt, list_off_var_(i, member)) /** * list_for_each_safe - iterate through a list, maybe during deletion @@ -422,7 +573,6 @@ static inline const void *list_tail_(const struct list_head *h, size_t off) * @nxt is used to hold the next element, so you can delete @i from the list. * * Example: - * struct child *next; * list_for_each_safe(&parent->children, child, next, list) { * list_del(&child->list); * parent->num_children--; @@ -537,10 +687,28 @@ static inline void list_prepend_list_(struct list_head *to, list_head_init(from); } +/* internal macros, do not use directly */ +#define list_for_each_off_dir_(h, i, off, dir) \ + for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.dir, \ + (off)); \ + list_node_from_off_((void *)i, (off)) != &(h)->n; \ + i = list_node_to_off_(list_node_from_off_((void *)i, (off))->dir, \ + (off))) + +#define list_for_each_safe_off_dir_(h, i, nxt, off, dir) \ + for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.dir, \ + (off)), \ + nxt = list_node_to_off_(list_node_from_off_(i, (off))->dir, \ + (off)); \ + list_node_from_off_(i, (off)) != &(h)->n; \ + i = nxt, \ + nxt = list_node_to_off_(list_node_from_off_(i, (off))->dir, \ + (off))) + /** * list_for_each_off - iterate through a list of memory regions. * @h: the list_head - * @i: the pointer to a memory region wich contains list node data. + * @i: the pointer to a memory region which contains list node data. * @off: offset(relative to @i) at which list node data resides. * * This is a low-level wrapper to iterate @i over the entire list, used to @@ -548,12 +716,12 @@ static inline void list_prepend_list_(struct list_head *to, * so you can break and continue as normal. * * WARNING! Being the low-level macro that it is, this wrapper doesn't know - * nor care about the type of @i. The only assumtion made is that @i points + * nor care about the type of @i. The only assumption made is that @i points * to a chunk of memory that at some @offset, relative to @i, contains a - * properly filled `struct node_list' which in turn contains pointers to - * memory chunks and it's turtles all the way down. Whith all that in mind + * properly filled `struct list_node' which in turn contains pointers to + * memory chunks and it's turtles all the way down. With all that in mind * remember that given the wrong pointer/offset couple this macro will - * happilly churn all you memory untill SEGFAULT stops it, in other words + * happily churn all you memory until SEGFAULT stops it, in other words * caveat emptor. * * It is worth mentioning that one of legitimate use-cases for that wrapper @@ -567,17 +735,24 @@ static inline void list_prepend_list_(struct list_head *to, * printf("Name: %s\n", child->name); */ #define list_for_each_off(h, i, off) \ - for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.next, \ - (off)); \ - list_node_from_off_((void *)i, (off)) != &(h)->n; \ - i = list_node_to_off_(list_node_from_off_((void *)i, (off))->next, \ - (off))) + list_for_each_off_dir_((h),(i),(off),next) + +/** + * list_for_each_rev_off - iterate through a list of memory regions backwards + * @h: the list_head + * @i: the pointer to a memory region which contains list node data. + * @off: offset(relative to @i) at which list node data resides. + * + * See list_for_each_off for details + */ +#define list_for_each_rev_off(h, i, off) \ + list_for_each_off_dir_((h),(i),(off),prev) /** * list_for_each_safe_off - iterate through a list of memory regions, maybe * during deletion * @h: the list_head - * @i: the pointer to a memory region wich contains list node data. + * @i: the pointer to a memory region which contains list node data. * @nxt: the structure containing the list_node * @off: offset(relative to @i) at which list node data resides. * @@ -590,15 +765,26 @@ static inline void list_prepend_list_(struct list_head *to, * printf("Name: %s\n", child->name); */ #define list_for_each_safe_off(h, i, nxt, off) \ - for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.next, \ - (off)), \ - nxt = list_node_to_off_(list_node_from_off_(i, (off))->next, \ - (off)); \ - list_node_from_off_(i, (off)) != &(h)->n; \ - i = nxt, \ - nxt = list_node_to_off_(list_node_from_off_(i, (off))->next, \ - (off))) + list_for_each_safe_off_dir_((h),(i),(nxt),(off),next) +/** + * list_for_each_rev_safe_off - iterate backwards through a list of + * memory regions, maybe during deletion + * @h: the list_head + * @i: the pointer to a memory region which contains list node data. + * @nxt: the structure containing the list_node + * @off: offset(relative to @i) at which list node data resides. + * + * For details see `list_for_each_rev_off' and `list_for_each_rev_safe' + * descriptions. + * + * Example: + * list_for_each_rev_safe_off(&parent->children, child, + * next, offsetof(struct child, list)) + * printf("Name: %s\n", child->name); + */ +#define list_for_each_rev_safe_off(h, i, nxt, off) \ + list_for_each_safe_off_dir_((h),(i),(nxt),(off),prev) /* Other -off variants. */ #define list_entry_off(n, type, off) \ diff --git a/ndctl/lib/inject.c b/ndctl/lib/inject.c index d61c02c176e2..3486ffefc40a 100644 --- a/ndctl/lib/inject.c +++ b/ndctl/lib/inject.c @@ -2,7 +2,6 @@ // Copyright (C) 2014-2020, Intel Corporation. All rights reserved. #include #include -#include #include #include #include diff --git a/util/list.h b/util/list.h deleted file mode 100644 index 1ea9c59b9f0c..000000000000 --- a/util/list.h +++ /dev/null @@ -1,40 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* Copyright (C) 2015-2020 Intel Corporation. All rights reserved. */ -#ifndef _NDCTL_LIST_H_ -#define _NDCTL_LIST_H_ - -#include - -/** - * list_add_after - add an entry after the given node in the linked list. - * @h: the list_head to add the node to - * @l: the list_node after which to add to - * @n: the list_node to add to the list. - * - * The list_node does not need to be initialized; it will be overwritten. - * Example: - * struct child *child = malloc(sizeof(*child)); - * - * child->name = "geoffrey"; - * list_add_after(&parent->children, &child1->list, &child->list); - * parent->num_children++; - */ -#define list_add_after(h, l, n) list_add_after_(h, l, n, LIST_LOC) -static inline void list_add_after_(struct list_head *h, - struct list_node *l, - struct list_node *n, - const char *abortstr) -{ - if (l->next == &h->n) { - /* l is the last element, this becomes a list_add_tail */ - list_add_tail(h, n); - return; - } - n->next = l->next; - n->prev = l; - l->next->prev = n; - l->next = n; - (void)list_debug(h, abortstr); -} - -#endif /* _NDCTL_LIST_H_ */ From patchwork Thu Jul 14 17:02:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75C5DC433EF for ; Thu, 14 Jul 2022 17:03:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232491AbiGNRDI (ORCPT ); Thu, 14 Jul 2022 13:03:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240107AbiGNRDC (ORCPT ); Thu, 14 Jul 2022 13:03:02 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 732CE5887E for ; Thu, 14 Jul 2022 10:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818181; x=1689354181; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8pHDEH3ziJVotJ1oYBiooCwlZBWAUamzVCenaWfGOWU=; b=HpsTAeq/H1emT2unVVQ1mq5RqtsU1ZrhGsRX4g3jqzJI7kG7WFd1A4hK BxtztTEyv/X6DEjqTX+yVG9m4+kqsRtaToqqQaIEBVSEANqXfNIpOmfmG hes61MU2xLGNPFotXBksbNGA92qHaMCcioTwcOyXvGghUFkGSJTPRzVpt eShx5ou9pHdSdBpK7tOdYBqONgJatIdx3lgLcq7M0Ga7ACmw1Pb05sEsE oJ660p9Km6eCJBNVkKV0PjaUeKEQEPHJ/SCKpcY0a/ZWIg36h9BOVygL5 jlGza9e/HNuHoYaSaMtMggzQ5FJB4JgwkB4rcmzeZtXt8A8bBRKZ/Kz28 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="371889671" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="371889671" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:31 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="842214200" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:22 -0700 Subject: [ndctl PATCH v2 06/12] cxl/lib: Maintain decoders in id order From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:21 -0700 Message-ID: <165781814167.1555691.14895625637451030942.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Given that decoder instance order is fundamental to the DPA translation sequence for endpoint decoders, enforce that cxl_decoder_for_each() returns decoders in instance order. Otherwise, they show up in readddir() order which is not predictable. Add a list_add_sorted() to generically handle inserting into a sorted list. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/lib/libcxl.c | 8 +++++++- util/list.h | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 util/list.h diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index f36edcfc735a..e4c5d3819e88 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -908,6 +909,11 @@ cxl_endpoint_get_memdev(struct cxl_endpoint *endpoint) return NULL; } +static int decoder_id_cmp(struct cxl_decoder *d1, struct cxl_decoder *d2) +{ + return d1->id - d2->id; +} + static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) { const char *devname = devpath_to_devname(cxldecoder_base); @@ -1049,7 +1055,7 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) return decoder_dup; } - list_add(&port->decoders, &decoder->list); + list_add_sorted(&port->decoders, decoder, list, decoder_id_cmp); free(path); return decoder; diff --git a/util/list.h b/util/list.h new file mode 100644 index 000000000000..cb7727123ea8 --- /dev/null +++ b/util/list.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2022 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_LIST_H_ +#define _NDCTL_LIST_H_ + +#include + +#define list_add_sorted(head, n, node, cmp) \ + do { \ + struct list_head *__head = (head); \ + typeof(n) __iter, __next; \ + typeof(n) __new = (n); \ + \ + if (list_empty(__head)) { \ + list_add(__head, &__new->node); \ + break; \ + } \ + \ + list_for_each (__head, __iter, node) { \ + if (cmp(__new, __iter) < 0) { \ + list_add_before(__head, &__iter->node, \ + &__new->node); \ + break; \ + } \ + __next = list_next(__head, __iter, node); \ + if (!__next) { \ + list_add_after(__head, &__iter->node, \ + &__new->node); \ + break; \ + } \ + if (cmp(__new, __next) < 0) { \ + list_add_before(__head, &__next->node, \ + &__new->node); \ + break; \ + } \ + } \ + } while (0) + +#endif /* _NDCTL_LIST_H_ */ From patchwork Thu Jul 14 17:02:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 817E4CCA47B for ; Thu, 14 Jul 2022 17:02:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239526AbiGNRCg (ORCPT ); Thu, 14 Jul 2022 13:02:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbiGNRCg (ORCPT ); Thu, 14 Jul 2022 13:02:36 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D553B57250 for ; Thu, 14 Jul 2022 10:02:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818154; x=1689354154; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z2SMMm41uIRd5DJN5abWRnM6MsMByqAH5tJ7qm43uuc=; b=bTVPbIVbHwSqS58F7uDIaMM6e0asAtR1FPu73AJeHCD/eX/yuVolADxx bYgOmTltJedm8J0lVahx7TDchoaXajo5TEJUxdcNEG/47S7oO9iRM2ePB 4ao53qnJur+56Q5EgJGa9Gaz+uYXAMB3irHM1Nz6Ofxgt7O5qFzuh8AIy IIu+t/Zfgwfy9Rq8vVNbLx4WwI1Y64hARgKuU1cMAl96d7ydu01k6cTxG JNOf5M1M8n5yrib3yQ3qQ6NS51qr3dTk24QrCbT0SfAsx5X6h3DiegY0p M/AwcAA2gKM5s6Q4HSi7IZVt8jH0I7mvUqbylvv7kNYVtj1Owwm7wrgyJ g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268602692" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="268602692" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:28 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="653953199" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:27 -0700 Subject: [ndctl PATCH v2 07/12] cxl/memdev: Fix json for multi-device partitioning From: Dan Williams To: vishal.l.verma@intel.com Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:27 -0700 Message-ID: <165781814737.1555691.889129128205037941.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org In the case when someone partitions several devices at once, collect all the affected memdevs into a json array. With the move to use util_display_json_array() that also requires a set of flags to be specifiied. Apply the UTIL_JSON_HUMAN flag for all interactive command result output to bring this command in line with other tools. Cc: Alison Schofield Signed-off-by: Dan Williams --- cxl/memdev.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/cxl/memdev.c b/cxl/memdev.c index 91d914db5af6..9fcd8ae5724b 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -19,6 +19,7 @@ struct action_context { FILE *f_out; FILE *f_in; + struct json_object *jdevs; }; static struct parameters { @@ -339,12 +340,13 @@ out: } static int action_setpartition(struct cxl_memdev *memdev, - struct action_context *actx) + struct action_context *actx) { const char *devname = cxl_memdev_get_devname(memdev); enum cxl_setpart_type type = CXL_SETPART_PMEM; unsigned long long size = ULLONG_MAX; struct json_object *jmemdev; + unsigned long flags; struct cxl_cmd *cmd; int rc; @@ -396,10 +398,12 @@ out_err: if (rc) log_err(&ml, "%s error: %s\n", devname, strerror(-rc)); - jmemdev = util_cxl_memdev_to_json(memdev, UTIL_JSON_PARTITION); - if (jmemdev) - printf("%s\n", json_object_to_json_string_ext(jmemdev, - JSON_C_TO_STRING_PRETTY)); + flags = UTIL_JSON_PARTITION; + if (actx->f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + jmemdev = util_cxl_memdev_to_json(memdev, flags); + if (actx->jdevs && jmemdev) + json_object_array_add(actx->jdevs, jmemdev); return rc; } @@ -446,6 +450,9 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, err++; } + if (action == action_setpartition) + actx.jdevs = json_object_new_array(); + if (err == argc) { usage_with_options(u, options); return -EINVAL; @@ -528,6 +535,15 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, if (actx.f_in != stdin) fclose(actx.f_in); + if (actx.jdevs) { + unsigned long flags = 0; + + if (actx.f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + util_display_json_array(actx.f_out, actx.jdevs, flags); + } + + out_close_fout: if (actx.f_out != stdout) fclose(actx.f_out); From patchwork Thu Jul 14 17:02:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0882C43334 for ; Thu, 14 Jul 2022 17:02:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239599AbiGNRCk (ORCPT ); Thu, 14 Jul 2022 13:02:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239569AbiGNRCj (ORCPT ); Thu, 14 Jul 2022 13:02:39 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BEE85B799 for ; Thu, 14 Jul 2022 10:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818158; x=1689354158; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R/HiYanKlcDyL1XhX9Xghwwfcblt81s6EZC3V86Gj74=; b=HzSK1lsGdpX7Uxez03jnQMc2fzZWn2QcuqjQ2YxBpe/k7qcV7TxyHAa8 RxJp8zSn7m7xAHmRqyrRUZLgyxhTV8e+FPdySRhR5rmhdinGmxkI9s9vD glqPdptJiv2TmQhWUQgpel7j5WoJrl8Yx6Rm1avdUsDchomN+0Z5n+26R 8s396N8+PZ8l6QQmZttoCSY52XL8TFfOt2bRfOyH1RuUjEg51V37PewaV ysKdYPgaY+0FXT3OasBpf85TpGUnEtd0mbeIruIQ3fOtogp69piZ/CTHF R+j0vH91F8xahI1hCCofMbVNWSEFjQGE1MrJGMI30TTTUKWr+/AW1mCu+ w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268602734" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="268602734" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:33 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="653953217" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:33 -0700 Subject: [ndctl PATCH v2 08/12] cxl/list: Emit 'mode' for endpoint decoder objects From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:33 -0700 Message-ID: <165781815306.1555691.17541956592287631419.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The 'mode' property of an endpoint decoder indicates the access properties of the DPA (device physical address) mapped into HPA (host physical address) by the decoder. Where the modes are 'none' (decoder-disabled), 'ram' (voltaile memory), 'pmem' (persistent memory), and 'mixed' (an unexpected, but possible, case where the decoder straddles a mode / partition boundary). Signed-off-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 9 +++++++++ cxl/json.c | 8 ++++++++ cxl/lib/libcxl.c | 30 ++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 1 + cxl/lib/private.h | 1 + cxl/libcxl.h | 23 +++++++++++++++++++++++ 6 files changed, 72 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 2aef489e8e12..90fe33887821 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -405,6 +405,15 @@ enum cxl_decoder_target_type { }; cxl_decoder_get_target_type(struct cxl_decoder *decoder); + +enum cxl_decoder_mode { + CXL_DECODER_MODE_NONE, + CXL_DECODER_MODE_MIXED, + CXL_DECODER_MODE_PMEM, + CXL_DECODER_MODE_RAM, +}; +enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); + bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder); diff --git a/cxl/json.c b/cxl/json.c index 3f52d3bbff45..ae9c8126f14f 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -473,6 +473,8 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, } if (cxl_port_is_endpoint(port)) { + enum cxl_decoder_mode mode = cxl_decoder_get_mode(decoder); + size = cxl_decoder_get_dpa_size(decoder); val = cxl_decoder_get_dpa_resource(decoder); if (size && val < ULLONG_MAX) { @@ -488,6 +490,12 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add(jdecoder, "dpa_size", jobj); } + + if (mode > CXL_DECODER_MODE_NONE) { + jobj = json_object_new_string(cxl_decoder_mode_name(mode)); + if (jobj) + json_object_object_add(jdecoder, "mode", jobj); + } } if (cxl_port_is_root(port) && cxl_decoder_is_mem_capable(decoder)) { diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index e4c5d3819e88..a4709175678d 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -960,6 +960,21 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) else decoder->size = strtoull(buf, NULL, 0); + sprintf(path, "%s/mode", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) == 0) { + if (strcmp(buf, "ram") == 0) + decoder->mode = CXL_DECODER_MODE_RAM; + else if (strcmp(buf, "pmem") == 0) + decoder->mode = CXL_DECODER_MODE_PMEM; + else if (strcmp(buf, "mixed") == 0) + decoder->mode = CXL_DECODER_MODE_MIXED; + else if (strcmp(buf, "none") == 0) + decoder->mode = CXL_DECODER_MODE_NONE; + else + decoder->mode = CXL_DECODER_MODE_MIXED; + } else + decoder->mode = CXL_DECODER_MODE_NONE; + switch (port->type) { case CXL_PORT_ENDPOINT: sprintf(path, "%s/dpa_resource", cxldecoder_base); @@ -1160,6 +1175,21 @@ cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) return decoder->dpa_size; } +CXL_EXPORT enum cxl_decoder_mode +cxl_decoder_get_mode(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return CXL_DECODER_MODE_NONE; + } + + return decoder->mode; +} + CXL_EXPORT enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 8e2fc75557f9..88c5a7edd33e 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -172,4 +172,5 @@ LIBCXL_3 { global: cxl_decoder_get_dpa_resource; cxl_decoder_get_dpa_size; + cxl_decoder_get_mode; } LIBCXL_2; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 24a2ae6787be..f6d4573757fd 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -108,6 +108,7 @@ struct cxl_decoder { char *dev_path; int nr_targets; int id; + enum cxl_decoder_mode mode; bool pmem_capable; bool volatile_capable; bool mem_capable; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 76aebe3efda8..1436dc4601cc 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -127,10 +127,33 @@ struct cxl_dport *cxl_port_get_dport_by_memdev(struct cxl_port *port, struct cxl_decoder; struct cxl_decoder *cxl_decoder_get_first(struct cxl_port *port); struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder); +struct cxl_decoder *cxl_decoder_get_last(struct cxl_port *port); +struct cxl_decoder *cxl_decoder_get_prev(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); +enum cxl_decoder_mode { + CXL_DECODER_MODE_NONE, + CXL_DECODER_MODE_MIXED, + CXL_DECODER_MODE_PMEM, + CXL_DECODER_MODE_RAM, +}; +static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) +{ + static const char *names[] = { + [CXL_DECODER_MODE_NONE] = "none", + [CXL_DECODER_MODE_MIXED] = "mixed", + [CXL_DECODER_MODE_PMEM] = "pmem", + [CXL_DECODER_MODE_RAM] = "ram", + }; + + if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_RAM) + mode = CXL_DECODER_MODE_NONE; + return names[mode]; +} + +enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); From patchwork Thu Jul 14 17:02:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABEF0C43334 for ; Thu, 14 Jul 2022 17:03:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239848AbiGNRDH (ORCPT ); Thu, 14 Jul 2022 13:03:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240133AbiGNRDC (ORCPT ); Thu, 14 Jul 2022 13:03:02 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A83D75C37C for ; Thu, 14 Jul 2022 10:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818181; x=1689354181; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d6ApsR6hlkYxBPeMgixmyfChQYOVY7n3l02N3ta4pdo=; b=LXyivYRETwIMT62m+GpTOJjWmp5cN0vokS4zjHzpdgJWeAog+gVr160l 58R2VIUK5TUjb+bzJJObVDYtH0opGY0Tj9MjcTsoWOGWEs3AIJCWp8F8i vtq8AlmW8N5HlifxwWBdKv9PMX5qnwSUvjESBK8J5TildIG0T1A6pvgyY /KkmKRGWgN58UabUE12Td2Dd6umEDlnc02cBhJAnB1VYhawn+LW5EyJzE X2d8LPBcIwBzEee5xbAelhPIBmP2EiznaqoBT3WvyVnvM1ctKuba4nd8o IIOaJ4VXiemMWCQQnJgP+a0ddfexl6TcNfMr8DC/win3W+mKQ2ohluhnU A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="286316862" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="286316862" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:39 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="593438298" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:39 -0700 Subject: [ndctl PATCH v2 09/12] cxl/set-partition: Accept 'ram' as an alias for 'volatile' From: Dan Williams To: vishal.l.verma@intel.com Cc: Alison Schofield , Davidlohr Bueso , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:38 -0700 Message-ID: <165781815878.1555691.12251226240559355924.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org 'ram' is a more convenient shorthand for volatile memory. Cc: Alison Schofield Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- Documentation/cxl/cxl-set-partition.txt | 2 +- cxl/memdev.c | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Documentation/cxl/cxl-set-partition.txt b/Documentation/cxl/cxl-set-partition.txt index 1e548af77da2..f0126daf808b 100644 --- a/Documentation/cxl/cxl-set-partition.txt +++ b/Documentation/cxl/cxl-set-partition.txt @@ -37,7 +37,7 @@ include::memdev-option.txt[] -t:: --type=:: - Type of partition, 'pmem' or 'volatile', to modify. + Type of partition, 'pmem' or 'ram' (volatile), to modify. Default: 'pmem' -s:: diff --git a/cxl/memdev.c b/cxl/memdev.c index 9fcd8ae5724b..1cecad2dba4b 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -65,7 +65,7 @@ OPT_BOOLEAN('f', "force", ¶m.force, \ #define SET_PARTITION_OPTIONS() \ OPT_STRING('t', "type", ¶m.type, "type", \ - "'pmem' or 'volatile' (Default: 'pmem')"), \ + "'pmem' or 'ram' (volatile) (Default: 'pmem')"), \ OPT_STRING('s', "size", ¶m.size, "size", \ "size in bytes (Default: all available capacity)"), \ OPT_BOOLEAN('a', "align", ¶m.align, \ @@ -355,6 +355,8 @@ static int action_setpartition(struct cxl_memdev *memdev, /* default */; else if (strcmp(param.type, "volatile") == 0) type = CXL_SETPART_VOLATILE; + else if (strcmp(param.type, "ram") == 0) + type = CXL_SETPART_VOLATILE; else { log_err(&ml, "invalid type '%s'\n", param.type); return -EINVAL; From patchwork Thu Jul 14 17:02:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C04F8C43334 for ; Thu, 14 Jul 2022 17:02:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239980AbiGNRCt (ORCPT ); Thu, 14 Jul 2022 13:02:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239848AbiGNRCr (ORCPT ); Thu, 14 Jul 2022 13:02:47 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CEC554CAF for ; Thu, 14 Jul 2022 10:02:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818165; x=1689354165; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u7l+qE05EyPGBjp2Bg9ejGd8ChU7Kpr9hP8IMycY+WY=; b=UFsUgmidm0iTOAiCFJNrTfFA51qaWb0+RcX82WcutQ0LHTSuLaXeBbhp j3MHpHkGOV6iF14O0gkejbLljFmlhO0bqOk1/MA52Av5s1FBFNL/BgHv1 34+yQbfWpq0a/ryI9J3sY+5RlDWKx4G/ZsYxNzXJ/GBAfjixqD66/Xu65 IU0xtUUMuyz1SVGiIWO4MS7AclcFd0FGKQle67Ku1V11kL0nl9s8BEwvy UeagfXI9M70r1SMCgW+Gb8OaHQOKt+aQOw8KpRfh0a/I2dJaWy0epZTsS goCOaRfmtsVMWMHY8mBIGwJVIvNsyj/nxH9ccAFS6ci0SYIF4wEA5strl A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265987492" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="265987492" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:45 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772694169" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:44 -0700 Subject: [ndctl PATCH v2 10/12] cxl/memdev: Add {reserve,free}-dpa commands From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:44 -0700 Message-ID: <165781816425.1555691.17958897857798325111.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Add helper commands for managing allocations of DPA (device physical address) capacity on a set of CXL memory devices. The main convenience this command affords is automatically picking the next decoder to allocate per-memdev. For example, to allocate 256MiB from all endpoints that are covered by a given root decoder, and collect those resulting endpoint-decoders into an array: readarray -t mem < <(cxl list -M -d $decoder | jq -r ".[].memdev") readarray -t endpoint < <(cxl reserve-dpa -t pmem ${mem[*]} -s $((256<<20)) | jq -r ".[] | .decoder.decoder") Signed-off-by: Dan Williams --- .clang-format | 1 Documentation/cxl/cxl-free-dpa.txt | 53 ++++++ Documentation/cxl/cxl-reserve-dpa.txt | 67 ++++++++ Documentation/cxl/lib/libcxl.txt | 2 Documentation/cxl/meson.build | 2 cxl/builtin.h | 2 cxl/cxl.c | 2 cxl/filter.c | 4 cxl/filter.h | 2 cxl/lib/libcxl.c | 86 ++++++++++ cxl/lib/libcxl.sym | 4 cxl/libcxl.h | 9 + cxl/memdev.c | 280 +++++++++++++++++++++++++++++++++ 13 files changed, 511 insertions(+), 3 deletions(-) create mode 100644 Documentation/cxl/cxl-free-dpa.txt create mode 100644 Documentation/cxl/cxl-reserve-dpa.txt diff --git a/.clang-format b/.clang-format index 6aabcb68e6a9..7254a1b15738 100644 --- a/.clang-format +++ b/.clang-format @@ -81,6 +81,7 @@ ForEachMacros: - 'cxl_bus_foreach' - 'cxl_port_foreach' - 'cxl_decoder_foreach' + - 'cxl_decoder_foreach_reverse' - 'cxl_target_foreach' - 'cxl_dport_foreach' - 'cxl_endpoint_foreach' diff --git a/Documentation/cxl/cxl-free-dpa.txt b/Documentation/cxl/cxl-free-dpa.txt new file mode 100644 index 000000000000..73fb048fdee3 --- /dev/null +++ b/Documentation/cxl/cxl-free-dpa.txt @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 + +cxl-free-dpa(1) +=============== + +NAME +---- +cxl-free-dpa - release device-physical address space + +SYNOPSIS +-------- +[verse] +'cxl free-dpa' [..] [] + +The CXL region provisioning process proceeds in multiple steps. One of +the steps is identifying and reserving the DPA span that each member of +the interleave-set (region) contributes in advance of attaching that +allocation to a region. For development, test, and debug purposes this +command is a helper to find the last allocated decoder on a device and +zero-out / free its DPA allocation. + +OPTIONS +------- +:: +include::memdev-option.txt[] + +-d:: +--decoder:: + Specify the decoder to free. The CXL specification + mandates that DPA must be released in the reverse order it was + allocated. See linkcxl:cxl-reserve-dpa[1] + +-t:: +--type:: + Constrain the search for "last allocated decoder" to decoders targeting + the given partition. + +-f:: +--force:: + The kernel enforces CXL DPA ordering constraints on deallocation events, + and the tool anticipates those and fails operations that are expected to + fail without sending them to the kernel. For test purposes, continue to + attempt "expected to fail" operations to exercise the driver. + +-v:: + Turn on verbose debug messages in the library (if libcxl was built with + logging and debug enabled). + +include::../copyright.txt[] + +SEE ALSO +-------- +linkcxl:cxl-reserve-dpa[1] diff --git a/Documentation/cxl/cxl-reserve-dpa.txt b/Documentation/cxl/cxl-reserve-dpa.txt new file mode 100644 index 000000000000..560daf0cb277 --- /dev/null +++ b/Documentation/cxl/cxl-reserve-dpa.txt @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0 + +cxl-reserve-dpa(1) +================== + +NAME +---- +cxl-reserve-dpa - allocate device-physical address space + +SYNOPSIS +-------- +[verse] +'cxl reserve-dpa' [..] [] + +The CXL region provisioning process proceeds in multiple steps. One of +the steps is identifying and reserving the DPA span that each member of +the interleave-set (region) contributes in advance of attaching that +allocation to a region. For development, test, and debug purposes this +command is a helper to find the next available decoder on endpoint +(memdev) and mark a span of DPA as busy. + +OPTIONS +------- +:: +include::memdev-option.txt[] + +-d:: +--decoder:: + Specify the decoder to attempt the allocation. The CXL specification + mandates that allocations must be ordered by DPA and decoder instance. + I.e. the lowest DPA allocation on the device is covered by deocder0, and + the last / highest DPA allocation is covered by the last decoder. This + ordering is enforced by the kernel. By default the tool picks the 'next + available' decoder. + +-t:: +--type:: + Select the partition for the allocation. CXL devices implement a + partition that divdes 'ram' and 'pmem' capacity, where 'pmem' capacity + consumes the higher DPA capacity above the partition boundary. The type + defaults to 'pmem'. Note that given CXL DPA allocation constraints, once + any 'pmem' allocation is established then all remaining 'ram' capacity + becomes reserved (skipped). + +-f:: +--force:: + The kernel enforces CXL DPA allocation ordering constraints, and + the tool anticipates those and fails operations that are expected to + fail without sending them to the kernel. For test purposes, continue to + attempt "expected to fail" operations to exercise the driver. + +-s:: +--size:: + Specify the size of the allocation. This option supports the suffixes + "k" or "K" for KiB, "m" or "M" for MiB, "g" or "G" for GiB and "t" or + "T" for TiB. This defaults to "all available capacity of the specified + type". + +-v:: + Turn on verbose debug messages in the library (if libcxl was built with + logging and debug enabled). + +include::../copyright.txt[] + +SEE ALSO +-------- +linkcxl:cxl-free-dpa[1] diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 90fe33887821..7a38ce4a54e2 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -394,6 +394,7 @@ unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); +int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, unsigned long long size); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); int cxl_decoder_get_id(struct cxl_decoder *decoder); int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder); @@ -413,6 +414,7 @@ enum cxl_decoder_mode { CXL_DECODER_MODE_RAM, }; enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); +int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); diff --git a/Documentation/cxl/meson.build b/Documentation/cxl/meson.build index 974a5a41d169..d019dfc23659 100644 --- a/Documentation/cxl/meson.build +++ b/Documentation/cxl/meson.build @@ -36,6 +36,8 @@ cxl_manpages = [ 'cxl-disable-port.txt', 'cxl-disable-bus.txt', 'cxl-set-partition.txt', + 'cxl-reserve-dpa.txt', + 'cxl-free-dpa.txt', ] foreach man : cxl_manpages diff --git a/cxl/builtin.h b/cxl/builtin.h index a437bc314a30..9e6fc624e86c 100644 --- a/cxl/builtin.h +++ b/cxl/builtin.h @@ -12,6 +12,8 @@ int cmd_init_labels(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_check_labels(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_disable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); +int cmd_reserve_dpa(int argc, const char **argv, struct cxl_ctx *ctx); +int cmd_free_dpa(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_disable_port(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_port(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx); diff --git a/cxl/cxl.c b/cxl/cxl.c index aa4ce61b7c87..ef4cda9e7c23 100644 --- a/cxl/cxl.c +++ b/cxl/cxl.c @@ -66,6 +66,8 @@ static struct cmd_struct commands[] = { { "write-labels", .c_fn = cmd_write_labels }, { "disable-memdev", .c_fn = cmd_disable_memdev }, { "enable-memdev", .c_fn = cmd_enable_memdev }, + { "reserve-dpa", .c_fn = cmd_reserve_dpa }, + { "free-dpa", .c_fn = cmd_free_dpa }, { "disable-port", .c_fn = cmd_disable_port }, { "enable-port", .c_fn = cmd_enable_port }, { "set-partition", .c_fn = cmd_set_partition }, diff --git a/cxl/filter.c b/cxl/filter.c index 2f88a9d2f398..e5fab19ec47f 100644 --- a/cxl/filter.c +++ b/cxl/filter.c @@ -380,8 +380,8 @@ struct cxl_port *util_cxl_port_filter_by_memdev(struct cxl_port *port, return NULL; } -static struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, - const char *__ident) +struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, + const char *__ident) { struct cxl_port *port = cxl_decoder_get_port(decoder); int pid, did; diff --git a/cxl/filter.h b/cxl/filter.h index 955794366d5c..c913dafd85ba 100644 --- a/cxl/filter.h +++ b/cxl/filter.h @@ -50,6 +50,8 @@ struct cxl_target *util_cxl_target_filter_by_memdev(struct cxl_target *target, struct cxl_dport *util_cxl_dport_filter_by_memdev(struct cxl_dport *dport, const char *ident, const char *serial); +struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, + const char *__ident); int cxl_filter_walk(struct cxl_ctx *ctx, struct cxl_filter_params *param); bool cxl_filter_has(const char *needle, const char *__filter); #endif /* _CXL_UTIL_FILTER_H_ */ diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index a4709175678d..be6bc2c65483 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -1120,6 +1120,20 @@ CXL_EXPORT struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder) return list_next(&port->decoders, decoder, list); } +CXL_EXPORT struct cxl_decoder *cxl_decoder_get_last(struct cxl_port *port) +{ + cxl_decoders_init(port); + + return list_tail(&port->decoders, struct cxl_decoder, list); +} + +CXL_EXPORT struct cxl_decoder *cxl_decoder_get_prev(struct cxl_decoder *decoder) +{ + struct cxl_port *port = decoder->port; + + return list_prev(&port->decoders, decoder, list); +} + CXL_EXPORT struct cxl_ctx *cxl_decoder_get_ctx(struct cxl_decoder *decoder) { return decoder->ctx; @@ -1175,6 +1189,78 @@ cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) return decoder->dpa_size; } +CXL_EXPORT int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, + unsigned long long size) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + char *path = decoder->dev_buf; + int len = decoder->buf_len, rc; + char buf[SYSFS_ATTR_SIZE]; + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return -EINVAL; + } + + if (snprintf(path, len, "%s/dpa_size", decoder->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + cxl_decoder_get_devname(decoder)); + return -ENOMEM; + } + + sprintf(buf, "%#llx\n", size); + rc = sysfs_write_attr(ctx, path, buf); + if (rc < 0) + return rc; + + decoder->dpa_size = size; + return 0; +} + +CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + char *path = decoder->dev_buf; + int len = decoder->buf_len, rc; + char buf[SYSFS_ATTR_SIZE]; + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return -EINVAL; + } + + switch (mode) { + case CXL_DECODER_MODE_PMEM: + sprintf(buf, "pmem"); + break; + case CXL_DECODER_MODE_RAM: + sprintf(buf, "ram"); + break; + default: + err(ctx, "%s: unsupported mode: %d\n", + cxl_decoder_get_devname(decoder), mode); + return -EINVAL; + } + + if (snprintf(path, len, "%s/mode", decoder->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + cxl_decoder_get_devname(decoder)); + return -ENOMEM; + } + + rc = sysfs_write_attr(ctx, path, buf); + if (rc < 0) + return rc; + + decoder->mode = mode; + return 0; +} + CXL_EXPORT enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 88c5a7edd33e..7712de0c5010 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -173,4 +173,8 @@ global: cxl_decoder_get_dpa_resource; cxl_decoder_get_dpa_size; cxl_decoder_get_mode; + cxl_decoder_get_last; + cxl_decoder_get_prev; + cxl_decoder_set_dpa_size; + cxl_decoder_set_mode; } LIBCXL_2; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 1436dc4601cc..33a216ee3a4c 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -139,6 +139,7 @@ enum cxl_decoder_mode { CXL_DECODER_MODE_PMEM, CXL_DECODER_MODE_RAM, }; + static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) { static const char *names[] = { @@ -154,6 +155,10 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) } enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); +int cxl_decoder_set_mode(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode); +int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, + unsigned long long size); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); @@ -182,6 +187,10 @@ bool cxl_decoder_is_locked(struct cxl_decoder *decoder); for (decoder = cxl_decoder_get_first(port); decoder != NULL; \ decoder = cxl_decoder_get_next(decoder)) +#define cxl_decoder_foreach_reverse(port, decoder) \ + for (decoder = cxl_decoder_get_last(port); decoder != NULL; \ + decoder = cxl_decoder_get_prev(decoder)) + struct cxl_target; struct cxl_target *cxl_target_get_first(struct cxl_decoder *decoder); struct cxl_target *cxl_target_get_next(struct cxl_target *target); diff --git a/cxl/memdev.c b/cxl/memdev.c index 1cecad2dba4b..e42f554326ec 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -33,6 +33,7 @@ static struct parameters { bool align; const char *type; const char *size; + const char *decoder_filter; } param; static struct log_ctx ml; @@ -71,6 +72,19 @@ OPT_STRING('s', "size", ¶m.size, "size", \ OPT_BOOLEAN('a', "align", ¶m.align, \ "auto-align --size per device's requirement") +#define RESERVE_DPA_OPTIONS() \ +OPT_STRING('s', "size", ¶m.size, "size", \ + "size in bytes (Default: all available capacity)") + +#define DPA_OPTIONS() \ +OPT_STRING('d', "decoder", ¶m.decoder_filter, \ + "decoder instance id", \ + "override the automatic decoder selection"), \ +OPT_STRING('t', "type", ¶m.type, "type", \ + "'pmem' or 'ram' (volatile) (Default: 'pmem')"), \ +OPT_BOOLEAN('f', "force", ¶m.force, \ + "Attempt 'expected to fail' operations") + static const struct option read_options[] = { BASE_OPTIONS(), LABEL_OPTIONS(), @@ -108,6 +122,242 @@ static const struct option set_partition_options[] = { OPT_END(), }; +static const struct option reserve_dpa_options[] = { + BASE_OPTIONS(), + RESERVE_DPA_OPTIONS(), + DPA_OPTIONS(), + OPT_END(), +}; + +static const struct option free_dpa_options[] = { + BASE_OPTIONS(), + DPA_OPTIONS(), + OPT_END(), +}; + +enum reserve_dpa_mode { + DPA_ALLOC, + DPA_FREE, +}; + +static int __reserve_dpa(struct cxl_memdev *memdev, + enum reserve_dpa_mode alloc_mode, + struct action_context *actx) +{ + struct cxl_decoder *decoder, *auto_target = NULL, *target = NULL; + struct cxl_endpoint *endpoint = cxl_memdev_get_endpoint(memdev); + const char *devname = cxl_memdev_get_devname(memdev); + unsigned long long avail_dpa, size; + enum cxl_decoder_mode mode; + struct cxl_port *port; + char buf[256]; + int rc; + + if (param.type) { + if (strcmp(param.type, "ram") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "volatile") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "ram") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "pmem") == 0) + mode = CXL_DECODER_MODE_PMEM; + else { + log_err(&ml, "%s: unsupported type: %s\n", devname, + param.type); + return -EINVAL; + } + } else + mode = CXL_DECODER_MODE_RAM; + + if (!endpoint) { + log_err(&ml, "%s: CXL operation disabled\n", devname); + return -ENXIO; + } + + port = cxl_endpoint_get_port(endpoint); + + if (mode == CXL_DECODER_MODE_RAM) + avail_dpa = cxl_memdev_get_ram_size(memdev); + else + avail_dpa = cxl_memdev_get_pmem_size(memdev); + + cxl_decoder_foreach(port, decoder) { + size = cxl_decoder_get_dpa_size(decoder); + if (size == ULLONG_MAX) + continue; + if (cxl_decoder_get_mode(decoder) != mode) + continue; + + if (size > avail_dpa) { + log_err(&ml, "%s: capacity accounting error\n", + devname); + return -ENXIO; + } + avail_dpa -= size; + } + + if (!param.size) + if (alloc_mode == DPA_ALLOC) { + size = avail_dpa; + if (!avail_dpa) { + log_err(&ml, "%s: no available capacity\n", + devname); + return -ENOSPC; + } + } else + size = 0; + else { + size = parse_size64(param.size); + if (size == ULLONG_MAX) { + log_err(&ml, "%s: failed to parse size option '%s'\n", + devname, param.size); + return -EINVAL; + } + if (size > avail_dpa) { + log_err(&ml, "%s: '%s' exceeds available capacity\n", + devname, param.size); + if (!param.force) + return -ENOSPC; + } + } + + /* + * Find next free decoder, assumes cxl_decoder_foreach() is in + * hardware instance-id order + */ + if (alloc_mode == DPA_ALLOC) + cxl_decoder_foreach(port, decoder) { + /* first 0-dpa_size is our target */ + if (cxl_decoder_get_dpa_size(decoder) == 0) { + auto_target = decoder; + break; + } + } + else + cxl_decoder_foreach_reverse(port, decoder) { + /* nothing to free? */ + if (!cxl_decoder_get_dpa_size(decoder)) + continue; + /* + * Active decoders can't be freed, and by definition all + * previous decoders must also be active + */ + if (cxl_decoder_get_size(decoder)) + break; + /* first dpa_size > 0 + disabled decoder is our target */ + if (cxl_decoder_get_dpa_size(decoder) < ULLONG_MAX) { + auto_target = decoder; + break; + } + } + + if (param.decoder_filter) { + unsigned long id; + char *end; + + id = strtoul(param.decoder_filter, &end, 0); + /* allow for standalone ordinal decoder ids */ + if (*end == '\0') + rc = snprintf(buf, sizeof(buf), "decoder%d.%ld", + cxl_port_get_id(port), id); + else + rc = snprintf(buf, sizeof(buf), "%s", + param.decoder_filter); + + if (rc >= (int)sizeof(buf)) { + log_err(&ml, "%s: decoder filter '%s' too long\n", + devname, param.decoder_filter); + return -EINVAL; + } + + if (alloc_mode == DPA_ALLOC) + cxl_decoder_foreach(port, decoder) { + target = util_cxl_decoder_filter(decoder, buf); + if (target) + break; + } + else + cxl_decoder_foreach_reverse(port, decoder) { + target = util_cxl_decoder_filter(decoder, buf); + if (target) + break; + } + + if (!target) { + log_err(&ml, "%s: no match for decoder: '%s'\n", + devname, param.decoder_filter); + return -ENXIO; + } + + if (target != auto_target) { + log_err(&ml, "%s: %s is out of sequence\n", devname, + cxl_decoder_get_devname(target)); + if (!param.force) + return -EINVAL; + } + } + + if (!target) + target = auto_target; + + if (!target) { + log_err(&ml, "%s: no suitable decoder found\n", devname); + return -ENXIO; + } + + if (cxl_decoder_get_mode(target) != mode) { + rc = cxl_decoder_set_dpa_size(target, 0); + if (rc) { + log_err(&ml, + "%s: %s: failed to clear allocation to set mode\n", + devname, cxl_decoder_get_devname(target)); + return rc; + } + rc = cxl_decoder_set_mode(target, mode); + if (rc) { + log_err(&ml, "%s: %s: failed to set %s mode\n", devname, + cxl_decoder_get_devname(target), + mode == CXL_DECODER_MODE_PMEM ? "pmem" : "ram"); + return rc; + } + } + + rc = cxl_decoder_set_dpa_size(target, size); + if (rc) + log_err(&ml, "%s: %s: failed to set dpa allocation\n", devname, + cxl_decoder_get_devname(target)); + else { + struct json_object *jdev, *jdecoder; + unsigned long flags = 0; + + if (actx->f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + jdev = util_cxl_memdev_to_json(memdev, flags); + jdecoder = util_cxl_decoder_to_json(target, flags); + if (!jdev || !jdecoder) { + json_object_put(jdev); + json_object_put(jdecoder); + } else { + json_object_object_add(jdev, "decoder", jdecoder); + json_object_array_add(actx->jdevs, jdev); + } + } + return rc; +} + +static int action_reserve_dpa(struct cxl_memdev *memdev, + struct action_context *actx) +{ + return __reserve_dpa(memdev, DPA_ALLOC, actx); +} + +static int action_free_dpa(struct cxl_memdev *memdev, + struct action_context *actx) +{ + return __reserve_dpa(memdev, DPA_FREE, actx); +} + static int action_disable(struct cxl_memdev *memdev, struct action_context *actx) { if (!cxl_memdev_is_enabled(memdev)) @@ -452,7 +702,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, err++; } - if (action == action_setpartition) + if (action == action_setpartition || action == action_reserve_dpa || + action == action_free_dpa) actx.jdevs = json_object_new_array(); if (err == argc) { @@ -495,6 +746,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, count = 0; for (i = 0; i < argc; i++) { + bool found = false; + cxl_memdev_foreach(ctx, memdev) { const char *memdev_filter = NULL; const char *serial_filter = NULL; @@ -507,6 +760,7 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, if (!util_cxl_memdev_filter(memdev, memdev_filter, serial_filter)) continue; + found = true; if (action == action_write) { single = memdev; @@ -519,6 +773,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, else if (rc && !err) err = rc; } + if (!found) + log_info(&ml, "no memdev matches %s\n", argv[i]); } rc = err; @@ -622,3 +878,25 @@ int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx) return count >= 0 ? 0 : EXIT_FAILURE; } + +int cmd_reserve_dpa(int argc, const char **argv, struct cxl_ctx *ctx) +{ + int count = memdev_action( + argc, argv, ctx, action_reserve_dpa, reserve_dpa_options, + "cxl reserve-dpa [..] []"); + log_info(&ml, "reservation completed on %d mem device%s\n", + count >= 0 ? count : 0, count > 1 ? "s" : ""); + + return count >= 0 ? 0 : EXIT_FAILURE; +} + +int cmd_free_dpa(int argc, const char **argv, struct cxl_ctx *ctx) +{ + int count = memdev_action( + argc, argv, ctx, action_free_dpa, free_dpa_options, + "cxl free-dpa [..] []"); + log_info(&ml, "reservation release completed on %d mem device%s\n", + count >= 0 ? count : 0, count > 1 ? "s" : ""); + + return count >= 0 ? 0 : EXIT_FAILURE; +} From patchwork Thu Jul 14 17:02:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAE92C433EF for ; Thu, 14 Jul 2022 17:03:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239941AbiGNRC7 (ORCPT ); Thu, 14 Jul 2022 13:02:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240099AbiGNRC4 (ORCPT ); Thu, 14 Jul 2022 13:02:56 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0A835C9EF for ; Thu, 14 Jul 2022 10:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818170; x=1689354170; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ScYbThIZfT4n55eM8DUbm+cfiTaDRHUaL5wTJC8Cy8k=; b=MKa4J3sSEOUMVbejXDqZhWSAF+qm2S9apugEX6DF+c54kLSCijlpuGhQ 56GBIk/y/dQ0JZG4stncpe+xUla1BlnVXuFLRUtxLmhDU9OP1Va5L12mz t4mt/laPmxT0mH99+LjkOTxr2JqP2v51ic5uxC+y8K4A7m4tDL2+4IlHS P+LwfH2Gzy1Tq5pRAAImLRw7YQCSX5hgdHi4ki/IqBUYZVrI+cCFcdqjF x8JHA+rkbvqI2d+jwv7XYADg3Im1hKDp84dA4VMv3gOzOB78ln6oCF52G UMIBRZQh+jiSI0QJSSuizwssC2+DizwQ4/hHiHgcp99zaPXbOl6XiVLZg g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="286712672" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="286712672" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:50 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="596163896" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:49 -0700 Subject: [ndctl PATCH v2 11/12] cxl/test: Update CXL memory parameters From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:49 -0700 Message-ID: <165781816971.1555691.18362747345754213762.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org In support of testing CXL region configurations cxl_test changed the size of its root decoders and endpoints. Use the size of the first root decoder to determine if this is an updated kernel. Signed-off-by: Dan Williams --- test/cxl-topology.sh | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/test/cxl-topology.sh b/test/cxl-topology.sh index ff11614f4f14..2583005fef26 100644 --- a/test/cxl-topology.sh +++ b/test/cxl-topology.sh @@ -64,14 +64,9 @@ switch[2]=$(jq -r ".[] | .[\"ports:${bridge[1]}\"] | $port_sort | .[0].host" <<< switch[3]=$(jq -r ".[] | .[\"ports:${bridge[1]}\"] | $port_sort | .[1].host" <<< $json) -# check that all 8 cxl_test memdevs are enabled by default and have a -# pmem size of 256M -json=$($CXL list -b cxl_test -M) -count=$(jq "map(select(.pmem_size == $((256 << 20)))) | length" <<< $json) -((count == 8)) || err "$LINENO" - - # validate the expected properties of the 4 root decoders +# use the size of the first decoder to determine the cxl_test version / +# properties json=$($CXL list -b cxl_test -D -d root) port_id=${root:4} port_id_len=${#port_id} @@ -80,26 +75,41 @@ count=$(jq "[ $decoder_sort | .[0] | select(.volatile_capable == true) | select(.size == $((256 << 20))) | select(.nr_targets == 1) ] | length" <<< $json) -((count == 1)) || err "$LINENO" + +if [ $count -eq 1 ]; then + decoder_base_size=$((256 << 20)) + pmem_size=$((256 << 20)) +else + decoder_base_size=$((1 << 30)) + pmem_size=$((1 << 30)) +fi count=$(jq "[ $decoder_sort | .[1] | select(.volatile_capable == true) | - select(.size == $((512 << 20))) | + select(.size == $((decoder_base_size * 2))) | select(.nr_targets == 2) ] | length" <<< $json) ((count == 1)) || err "$LINENO" count=$(jq "[ $decoder_sort | .[2] | select(.pmem_capable == true) | - select(.size == $((256 << 20))) | + select(.size == $decoder_base_size) | select(.nr_targets == 1) ] | length" <<< $json) ((count == 1)) || err "$LINENO" count=$(jq "[ $decoder_sort | .[3] | select(.pmem_capable == true) | - select(.size == $((512 << 20))) | + select(.size == $((decoder_base_size * 2))) | select(.nr_targets == 2) ] | length" <<< $json) ((count == 1)) || err "$LINENO" + +# check that all 8 cxl_test memdevs are enabled by default and have a +# pmem size of 256M, or 1G +json=$($CXL list -b cxl_test -M) +count=$(jq "map(select(.pmem_size == $pmem_size)) | length" <<< $json) +((count == 8)) || err "$LINENO" + + # check that switch ports disappear after all of their memdevs have been # disabled, and return when the memdevs are enabled. for s in ${switch[@]} From patchwork Thu Jul 14 17:02:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36080C43334 for ; Thu, 14 Jul 2022 17:03:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239950AbiGNRDO (ORCPT ); Thu, 14 Jul 2022 13:03:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239659AbiGNRDO (ORCPT ); Thu, 14 Jul 2022 13:03:14 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AF1E57279 for ; Thu, 14 Jul 2022 10:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818192; x=1689354192; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SXHtWpv2HNpUil832VHGH8Vd9M5eM5+D7bANZc7UQtE=; b=DmBMbNb/1qbFt//UHtH1HVzoRF7eCABjjCF358LS2rnb4sjuRnkNjTgJ 9gfwFRPDwT+j/U93NfHyrvq5tCaNGdK784bSgX8/5rckeOU7zhPZGjVtx 5hs38mmHH03dVIe/0NrUOswGF12YW7Bt+WE7T/UNKe/b/nWO0oJSwNRHT Hcpje5xQCF6R4wFTTY5Z5By/KM4IlzjtfaLwRZNc3Lbp8oN0eQx4va+xV 2/pqJ7j5EoqCqfxj54NimEW3yG2e7JWAgwCc9SDtyOhVOcI89tIOCMTia fIbwkMmVbbaT8qUGRM1qnPQMkXmtYHkntvGom2fACK+vQ0jkxv8wjt+95 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="371889777" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="371889777" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:55 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772694252" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:55 -0700 Subject: [ndctl PATCH v2 12/12] cxl/test: Checkout region setup/teardown From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:55 -0700 Message-ID: <165781817516.1555691.3557156570639615515.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Exercise the fundamental region provisioning sysfs mechanisms of discovering available DPA capacity, allocating DPA to a region, and programming HDM decoders. Signed-off-by: Dan Williams --- test/cxl-region-sysfs.sh | 122 ++++++++++++++++++++++++++++++++++++++++++++++ test/meson.build | 2 + 2 files changed, 124 insertions(+) create mode 100644 test/cxl-region-sysfs.sh diff --git a/test/cxl-region-sysfs.sh b/test/cxl-region-sysfs.sh new file mode 100644 index 000000000000..2582edb3f306 --- /dev/null +++ b/test/cxl-region-sysfs.sh @@ -0,0 +1,122 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 Intel Corporation. All rights reserved. + +. $(dirname $0)/common + +rc=1 + +set -ex + +trap 'err $LINENO' ERR + +check_prereq "jq" + +modprobe -r cxl_test +modprobe cxl_test +udevadm settle + +# THEORY OF OPERATION: Create a x8 interleave across the pmem capacity +# of the 8 endpoints defined by cxl_test, commit the decoders (which +# just stubs out the actual hardware programming aspect, but updates the +# driver state), and then tear it all down again. As with other cxl_test +# tests if the CXL topology in tools/testing/cxl/test/cxl.c ever changes +# then the paired update must be made to this test. + +# find the root decoder that spans both test host-bridges and support pmem +decoder=$($CXL list -b cxl_test -D -d root | jq -r ".[] | + select(.pmem_capable == true) | + select(.nr_targets == 2) | + .decoder") + +# find the memdevs mapped by that decoder +readarray -t mem < <($CXL list -M -d $decoder | jq -r ".[].memdev") + +# ask cxl reserve-dpa to allocate pmem capacity from each of those memdevs +readarray -t endpoint < <($CXL reserve-dpa -t pmem ${mem[*]} -s $((256<<20)) | + jq -r ".[] | .decoder.decoder") + +# instantiate an empty region +region=$(cat /sys/bus/cxl/devices/$decoder/create_pmem_region) +echo $region > /sys/bus/cxl/devices/$decoder/create_pmem_region +uuidgen > /sys/bus/cxl/devices/$region/uuid + +# setup interleave geometry +nr_targets=${#endpoint[@]} +echo $nr_targets > /sys/bus/cxl/devices/$region/interleave_ways +g=$(cat /sys/bus/cxl/devices/$decoder/interleave_granularity) +echo $g > /sys/bus/cxl/devices/$region/interleave_granularity +echo $((nr_targets * (256<<20))) > /sys/bus/cxl/devices/$region/size + +# grab the list of memdevs grouped by host-bridge interleave position +port_dev0=$($CXL list -T -d $decoder | jq -r ".[] | + .targets | .[] | select(.position == 0) | .target") +port_dev1=$($CXL list -T -d $decoder | jq -r ".[] | + .targets | .[] | select(.position == 1) | .target") +readarray -t mem_sort0 < <($CXL list -M -p $port_dev0 | jq -r ".[] | .memdev") +readarray -t mem_sort1 < <($CXL list -M -p $port_dev1 | jq -r ".[] | .memdev") + +# TODO: add a cxl list option to list memdevs in valid region provisioning +# order, hardcode for now. +mem_sort=() +mem_sort[0]=${mem_sort0[0]} +mem_sort[1]=${mem_sort1[0]} +mem_sort[2]=${mem_sort0[2]} +mem_sort[3]=${mem_sort1[2]} +mem_sort[4]=${mem_sort0[1]} +mem_sort[5]=${mem_sort1[1]} +mem_sort[6]=${mem_sort0[3]} +mem_sort[7]=${mem_sort1[3]} + +# TODO: use this alternative memdev ordering to validate a negative test for +# specifying invalid positions of memdevs +#mem_sort[2]=${mem_sort0[0]} +#mem_sort[1]=${mem_sort1[0]} +#mem_sort[0]=${mem_sort0[2]} +#mem_sort[3]=${mem_sort1[2]} +#mem_sort[4]=${mem_sort0[1]} +#mem_sort[5]=${mem_sort1[1]} +#mem_sort[6]=${mem_sort0[3]} +#mem_sort[7]=${mem_sort1[3]} + +# re-generate the list of endpoint decoders in region position programming order +endpoint=() +for i in ${mem_sort[@]} +do + readarray -O ${#endpoint[@]} -t endpoint < <($CXL list -Di -d endpoint -m $i | jq -r ".[] | + select(.mode == \"pmem\") | .decoder") +done + +# attach all endpoint decoders to the region +pos=0 +for i in ${endpoint[@]} +do + echo $i > /sys/bus/cxl/devices/$region/target$pos + pos=$((pos+1)) +done +echo "$region added ${#endpoint[@]} targets: ${endpoint[@]}" + +# walk up the topology and commit all decoders +echo 1 > /sys/bus/cxl/devices/$region/commit + +# walk down the topology and de-commit all decoders +echo 0 > /sys/bus/cxl/devices/$region/commit + +# remove endpoints from the region +pos=0 +for i in ${endpoint[@]} +do + echo "" > /sys/bus/cxl/devices/$region/target$pos + pos=$((pos+1)) +done + +# release DPA capacity +readarray -t endpoint < <($CXL free-dpa -t pmem ${mem[*]} | + jq -r ".[] | .decoder.decoder") +echo "$region released ${#endpoint[@]} targets: ${endpoint[@]}" + +# validate no WARN or lockdep report during the run +log=$(journalctl -r -k --since "-$((SECONDS+1))s") +grep -q "Call Trace" <<< $log && err "$LINENO" + +modprobe -r cxl_test diff --git a/test/meson.build b/test/meson.build index 210dcb0b5ff1..3203d9cea243 100644 --- a/test/meson.build +++ b/test/meson.build @@ -151,6 +151,7 @@ max_extent = find_program('max_available_extent_ns.sh') pfn_meta_errors = find_program('pfn-meta-errors.sh') track_uuid = find_program('track-uuid.sh') cxl_topo = find_program('cxl-topology.sh') +cxl_sysfs = find_program('cxl-region-sysfs.sh') tests = [ [ 'libndctl', libndctl, 'ndctl' ], @@ -176,6 +177,7 @@ tests = [ [ 'pfn-meta-errors.sh', pfn_meta_errors, 'ndctl' ], [ 'track-uuid.sh', track_uuid, 'ndctl' ], [ 'cxl-topology.sh', cxl_topo, 'cxl' ], + [ 'cxl-region-sysfs.sh', cxl_sysfs, 'cxl' ], ] if get_option('destructive').enabled()