From patchwork Wed Oct 30 21:54:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857248 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BEF11CF7B1 for ; Wed, 30 Oct 2024 21:55:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325310; cv=none; b=dsP8r9tALqRSyivs9qWF+cMeLF+xi11Hs8BQhUk2NktKRfynOgsZycnF2lAZf18Rav0mitmchBvyEn5oe0G+290DnKvFIkLG07HeX6Yex8L1CUJrAtahF5J6FVSDkVIrJYq+hwnwhuZOJCHjfqpxAAKt5YuNfFLu3KjH+hFfVqM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325310; c=relaxed/simple; bh=bnA5MH3Ut/o2XkdYerkFVpg3U06Fxf8NVmDS1qWQDyU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bpVqcbJqWeBXz6Kba9puYY+OeLX17fBkJjCH1dMYvlzxtAb+JpMbk94AFxbJmbL75MOHVETad7M7A0x+ogGC2+U3ft2hFOc/lWjb+wwoL7uecg5XukIyfsWQyxXL7vK7zUH8qGAp6tErwULEgLtmV0p35elU4O/iEXOlA4ZhgGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mQJfZvCT; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mQJfZvCT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325308; x=1761861308; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=bnA5MH3Ut/o2XkdYerkFVpg3U06Fxf8NVmDS1qWQDyU=; b=mQJfZvCTsCMii/vpZgh9iLCFQ/dxQCIoba8Yij+LTmCXJFlfFx7DWP0L koNDPWpf4akDQPOL4v6VM1yezsFDQOMt9Gec44UtnNSaoKfdD+8Iyb7lm cz51tfXkRDQl66xkZF1WwWw69teHGKM6TeAcJ/Z1QDnFFescpJSfeeu+G fEdGgiK7mMsHQ9uT7CjT0C2Mj80TEcr/I5EizOWLBpDE1HR4aowH3EWwJ 6M9BZtsjtKFQYE0pMC98aRCAC38hmhIAULjTontyTqEXmo0R9aSj7zETe +kmt5z6alrAoOAgT6IdUtR7I1AU0qaHtVKPq6mOTEpTixRzKVkW168vsQ g==; X-CSE-ConnectionGUID: n39eXuqNQ1WRjRn31lPrrw== X-CSE-MsgGUID: S55+OW1AQHeQ751QaWnuHA== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620516" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620516" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:07 -0700 X-CSE-ConnectionGUID: HtVLIGyhTBOeFp4sGGTRyQ== X-CSE-MsgGUID: anaIG9rLS/yLhYlVpvRwUA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899958" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:06 -0700 From: Ira Weiny Date: Wed, 30 Oct 2024 16:54:44 -0500 Subject: [ndctl PATCH 1/6] ndctl/cxl-events: Don't fail test until event counts are reported Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-1-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=1446; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=bnA5MH3Ut/o2XkdYerkFVpg3U06Fxf8NVmDS1qWQDyU=; b=2LHLcye7C1q/k2ff3Tle0TanynHZFAlrIrZb1/odOlKFg9NhciiCGZwU3KfuxfpD1n7zCEAt3 HfeF81rrCzfCuMYo971bKWs0+tSsYBFlETLEUwEFcGyHdtLRKFFzVnZ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= In testing DCD event modifications a failed cxl-event test lacked details on the event counts. This was because the greps were failing the test rather than the check against the counts. Suppress the grep failure and rely on event count checks for pass/fail of the test. Signed-off-by: Ira Weiny --- test/cxl-events.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/cxl-events.sh b/test/cxl-events.sh index ff4f3fdff1d8f6fd80f093126a27bf14b52d167f..c216d6aa9148c938a649cb22656127b3df440039 100644 --- a/test/cxl-events.sh +++ b/test/cxl-events.sh @@ -71,10 +71,10 @@ echo 0 > /sys/kernel/tracing/tracing_on echo "TEST: Events seen" trace_out=$(cat /sys/kernel/tracing/trace) -num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}") -num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}") -num_failure=$(grep -c "log=Failure" <<< "${trace_out}") -num_info=$(grep -c "log=Informational" <<< "${trace_out}") +num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}" || true) +num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}" || true) +num_failure=$(grep -c "log=Failure" <<< "${trace_out}" || true) +num_info=$(grep -c "log=Informational" <<< "${trace_out}" || true) echo " LOG (Expected) : (Found)" echo " overflow ($num_overflow_expected) : $num_overflow" echo " Fatal ($num_fatal_expected) : $num_fatal" From patchwork Wed Oct 30 21:54:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857249 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB4191DBB36 for ; Wed, 30 Oct 2024 21:55:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325310; cv=none; b=CyrQy4urGbrYwqodxjT9+XsGNVnh3GZAEdsULS5z3iC13jfD8iNf86HcejSTY+62zUVs8cdYty0WiSMAjVaWTtz6qYrdRF2YCQmH2cwgiH9fHpCo8Bw/0ZWbt1/SO1j16gWmtpY9esN+RWnK9rM6Pu5zuEk6Esbzn4mDxfzQSMM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325310; c=relaxed/simple; bh=ie1bgarTdW2m0OhCx2ixlHunBdnNwnRrR7sxjGL0b9I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=thZKYqGb/mgom0PA6++6EgYafAiIt3jNPffZW+6KqhPVuH4DPjXx5CRye//jXOCfYtL3bFgFhA7de1/O6ijrBXH2bPBLqBknnAXKHd+PWXBsKDHoU1fi8e22aAjZg38fSloxAe0y/JZrB9enHPMn1SbPsXOhmRxD2oS9G9S4OW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=K+hnXUc2; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="K+hnXUc2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325309; x=1761861309; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=ie1bgarTdW2m0OhCx2ixlHunBdnNwnRrR7sxjGL0b9I=; b=K+hnXUc2AGbw9V0DmN9921+wauDwMacEgqcGBYbMMYfLycK0hXQDWN5c ZHRfTuNCIXEWOjTpy0R493+XubD0txtnI/MtBrtvxGvXkkZ6MSUFqC8uZ GfhSIZujluWlLhpcAKs/KSpTpomxk4pDRfZh9OxMH1eBT4rahy5CrVykx VZcRpTn6d15LFHFZddLGANwAmmmiyZHKO2yShN5tr7GF7SBZ/XOzLzCAn 3QtAwoJhGaNj8jlH2avXZRdWkYA0YPLGyF9I2n0sFBlkZ2ZDEqSCpRcc6 riesP2A0xG2m7W3VDoKC4YsHasNlRPVThwAwo1xPj0oiEdFWyHcMp7s+P Q==; X-CSE-ConnectionGUID: QfpZoDpTSzC4leQUStDZXw== X-CSE-MsgGUID: RtTtZZGaQW2yHja3b9Z4jQ== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620521" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620521" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:09 -0700 X-CSE-ConnectionGUID: 6Te4gJDZRQeFrDkxPoWf6Q== X-CSE-MsgGUID: SmOYxaVWTxG9n6q6h0tzcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899962" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:08 -0700 From: Ira Weiny Date: Wed, 30 Oct 2024 16:54:45 -0500 Subject: [ndctl PATCH 2/6] ndctl/cxl/region: Report max size for region creation Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-2-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=969; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=ie1bgarTdW2m0OhCx2ixlHunBdnNwnRrR7sxjGL0b9I=; b=SEow2WPOpIniAZ5Mgcv0yx6x5d70R6gsolbiHhmajL24qlEROwAP/dc+2s5PitJ/Hi1lVUA22 e7/1/06C+OxCjoNAPY10VgoONwbnCaAm5OPZO1C24FGaz5JWuDyCSR8 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= When creating a region if the size exceeds the max an error is printed. However, the max available space is not reported which makes it harder to determine what is wrong. Add the max size available to the output error. Signed-off-by: Ira Weiny Reviewed-by: Fan Ni --- cxl/region.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cxl/region.c b/cxl/region.c index 96aa5931d2281c7577679b7f6165218964fa0425..207cf2d003148992255c715f286bc0f38de2ca84 100644 --- a/cxl/region.c +++ b/cxl/region.c @@ -677,8 +677,8 @@ static int create_region(struct cxl_ctx *ctx, int *count, } if (!default_size && size > max_extent) { log_err(&rl, - "%s: region size %#lx exceeds max available space\n", - cxl_decoder_get_devname(p->root_decoder), size); + "%s: region size %#lx exceeds max available space (%#lx)\n", + cxl_decoder_get_devname(p->root_decoder), size, max_extent); return -ENOSPC; } From patchwork Wed Oct 30 21:54:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857250 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D02D1EB9FD for ; Wed, 30 Oct 2024 21:55:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325313; cv=none; b=JFF7ibTUJOgmrbDvcEX7784tp66F7L5kxb1RCCsX9P/7xJmhHKskoS3X2IyHI+oJq5sHxCs2jDA7S9XMO+Owf65UumaEVjEUIvpfhHc94awrsXhG75ACts40HLrYjzo1KmWYaYoTWVYnV3Vpak018f2RDjvNaLMve9BYvYz8QxY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325313; c=relaxed/simple; bh=LIQ+4ujocGInLdjTo8MJ1n7kTR/xhH67XvAW1aV3zdQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mcCip9j/9GM3RVnLjeCJ8Tc9TsWCNRpOl14l9Wbdn0L2YOfC1ULEOaJ38tdG6C8s04SsfHjBpngt6xYvDAbRLgqlTuj/w0Lcsqch9GEPCy9OGLdbo0jS923yhIBpxC6AV5D+hTRqhX9qppxl+zEfC1MMAGvFZQuNc0S2j5Ia2XU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QAqb/Eh7; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QAqb/Eh7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325311; x=1761861311; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=LIQ+4ujocGInLdjTo8MJ1n7kTR/xhH67XvAW1aV3zdQ=; b=QAqb/Eh7+ZZd2UOhkrGqYa2ospjw6EW+NxNd5euzRsU5fE63Q9QypXIL 2xal3TovvqGxI4yOTxZHKk0KburOBN0voHysb3JPtZg9T5evcA8AdSWzw e1rTQQSpTfe4vGadulgjIZ+foF6JuJvd/1pOOy+FA72feDk+GRZ0Td+3E b3ZDbWIbJPPNoaPnWDdhZLMgJueuXFCzotxpLztLy+29eJaYCX20P0ncR cEAg6kASfSDOtdo8OwFrYJhx3OuwuUf7OWI3TWRqHHilZD2sWLXalQ7/y JifqF3e764hQjn9wCHtPLbLqdNun5DLxJnv67zM+r24KpJ2H+8gs8MKvz Q==; X-CSE-ConnectionGUID: XLs8BfHSTJirxn41e8J5Xw== X-CSE-MsgGUID: kRHl/vDoRh2h3Tul60Ll1g== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620528" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620528" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:11 -0700 X-CSE-ConnectionGUID: h2gBF1f1SoethfUt+UF6Sg== X-CSE-MsgGUID: Z2r08cdPSFGNK5vr7CfHuA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899966" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:10 -0700 From: Ira Weiny Date: Wed, 30 Oct 2024 16:54:46 -0500 Subject: [ndctl PATCH 3/6] ndctl: Separate region mode from decoder mode Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-3-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=6625; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=LIQ+4ujocGInLdjTo8MJ1n7kTR/xhH67XvAW1aV3zdQ=; b=HLDVM13q1qCNmMaX+0HzFPTdvUp6l7EG1d5U8Jfb3eZ1NCvk995Tk+FU3FOJ5CA65vpwKOcHV 4AYEVBQg9xxAnehEr76kld18F07iHxGfjWT0CBlNcVTlxZIOTppvmfV X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= With the introduction of DCD, region mode and decoder mode no longer remain a 1:1 relation. An interleaved region may be made up of Dynamic Capacity partitions with different indexes on each of the target devices. Introduce a new region mode enumeration and access function. To maintain compatibility with existing software the region mode values are defined the same as the current decoder mode. In addition cxl_region_get_mode() is retained. Signed-off-by: Ira Weiny --- cxl/json.c | 6 +++--- cxl/lib/libcxl.c | 15 ++++++++++----- cxl/lib/libcxl.sym | 1 + cxl/lib/private.h | 2 +- cxl/libcxl.h | 35 +++++++++++++++++++++++++++++++++++ 5 files changed, 50 insertions(+), 9 deletions(-) diff --git a/cxl/json.c b/cxl/json.c index 5066d3bed13f8fcc36ab8f0ea127685c246d94d7..dcd3cc28393faf7e8adf299a857531ecdeaac50a 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -1147,7 +1147,7 @@ void util_cxl_mappings_append_json(struct json_object *jregion, struct json_object *util_cxl_region_to_json(struct cxl_region *region, unsigned long flags) { - enum cxl_decoder_mode mode = cxl_region_get_mode(region); + enum cxl_region_mode mode = cxl_region_get_region_mode(region); const char *devname = cxl_region_get_devname(region); struct json_object *jregion, *jobj; u64 val; @@ -1174,8 +1174,8 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, json_object_object_add(jregion, "size", jobj); } - if (mode != CXL_DECODER_MODE_NONE) { - jobj = json_object_new_string(cxl_decoder_mode_name(mode)); + if (mode != CXL_REGION_MODE_NONE) { + jobj = json_object_new_string(cxl_region_mode_name(mode)); if (jobj) json_object_object_add(jregion, "type", jobj); } diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 63aa4ef3acdc2fb3c4ec6c13be5feb802e817d0d..5cbfb3e7d466b491ef87ea285f7e50d3bac230db 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -431,10 +431,10 @@ CXL_EXPORT bool cxl_region_qos_class_mismatch(struct cxl_region *region) if (!memdev) continue; - if (region->mode == CXL_DECODER_MODE_RAM) { + if (region->mode == CXL_REGION_MODE_RAM) { if (root_decoder->qos_class != memdev->ram_qos_class) return true; - } else if (region->mode == CXL_DECODER_MODE_PMEM) { + } else if (region->mode == CXL_REGION_MODE_PMEM) { if (root_decoder->qos_class != memdev->pmem_qos_class) return true; } @@ -619,9 +619,9 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base) sprintf(path, "%s/mode", cxlregion_base); if (sysfs_read_attr(ctx, path, buf) < 0) - region->mode = CXL_DECODER_MODE_NONE; + region->mode = CXL_REGION_MODE_NONE; else - region->mode = cxl_decoder_mode_from_ident(buf); + region->mode = cxl_region_mode_from_ident(buf); sprintf(path, "%s/modalias", cxlregion_base); if (sysfs_read_attr(ctx, path, buf) == 0) @@ -748,11 +748,16 @@ CXL_EXPORT unsigned long long cxl_region_get_resource(struct cxl_region *region) return region->start; } -CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region) +CXL_EXPORT enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region) { return region->mode; } +CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region) +{ + return (enum cxl_decoder_mode)cxl_region_get_region_mode(region); +} + CXL_EXPORT unsigned int cxl_region_get_interleave_ways(struct cxl_region *region) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 0c155a40ad4765106f0eab1745281d462af782fe..b5d9bdcc38e09812f26afc1cb0e804f86784b8e6 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -287,4 +287,5 @@ LIBECXL_8 { global: cxl_memdev_trigger_poison_list; cxl_region_trigger_poison_list; + cxl_region_get_region_mode; } LIBCXL_7; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index b6cd910e93359b53cac34427acfe84c7abcb78b0..0f45be89b6a00477d13fb6d7f1906213a3073c48 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -171,7 +171,7 @@ struct cxl_region { unsigned int interleave_ways; unsigned int interleave_granularity; enum cxl_decode_state decode_state; - enum cxl_decoder_mode mode; + enum cxl_region_mode mode; struct daxctl_region *dax_region; struct kmod_module *module; struct list_head mappings; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 0a5fd0e13cc24e0032d4a83d780278fbe0038d32..06b87a0924faafec6c80eca83ea7551d4e117256 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -303,6 +303,39 @@ int cxl_memdev_is_enabled(struct cxl_memdev *memdev); for (endpoint = cxl_endpoint_get_first(port); endpoint != NULL; \ endpoint = cxl_endpoint_get_next(endpoint)) +enum cxl_region_mode { + CXL_REGION_MODE_NONE = CXL_DECODER_MODE_NONE, + CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, + CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, + CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, +}; + +static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) +{ + static const char *names[] = { + [CXL_REGION_MODE_NONE] = "none", + [CXL_REGION_MODE_MIXED] = "mixed", + [CXL_REGION_MODE_PMEM] = "pmem", + [CXL_REGION_MODE_RAM] = "ram", + }; + + if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM) + mode = CXL_REGION_MODE_NONE; + return names[mode]; +} + +static inline enum cxl_region_mode +cxl_region_mode_from_ident(const char *ident) +{ + if (strcmp(ident, "ram") == 0) + return CXL_REGION_MODE_RAM; + else if (strcmp(ident, "volatile") == 0) + return CXL_REGION_MODE_RAM; + else if (strcmp(ident, "pmem") == 0) + return CXL_REGION_MODE_PMEM; + return CXL_REGION_MODE_NONE; +} + struct cxl_region; struct cxl_region *cxl_region_get_first(struct cxl_decoder *decoder); struct cxl_region *cxl_region_get_next(struct cxl_region *region); @@ -318,6 +351,8 @@ const char *cxl_region_get_devname(struct cxl_region *region); void cxl_region_get_uuid(struct cxl_region *region, uuid_t uu); unsigned long long cxl_region_get_size(struct cxl_region *region); unsigned long long cxl_region_get_resource(struct cxl_region *region); +enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region); +/* Deprecated: use cxl_region_get_region_mode() */ enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region); unsigned int cxl_region_get_interleave_ways(struct cxl_region *region); unsigned int cxl_region_get_interleave_granularity(struct cxl_region *region); From patchwork Wed Oct 30 21:54:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857251 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 802201D0E0D for ; Wed, 30 Oct 2024 21:55:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325317; cv=none; b=qTQKHjVjcStIB+Yi7BY/AG+NpiiQi+138aDJ3zQDUP4YytYfBACbPs7FpsdawpGbmeaTHEVxOt51e085AB3/4sx1QATzJ1UPy3ppThQycaYz9Znvj5QXUomameq8AkKT5ygu0rURcfTf+GYgYwAA+dQFoieGzyoPJ5eOPvOmdOs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325317; c=relaxed/simple; bh=9a1sAnRX7zf6QJZHCa5uRKk1d/kP28qRgid/slWPCtw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=T5z0jQDyGt1OtMDFdjEfnqMR+q80eyCFRJb9yBleCCUyfuFtZOZT3dBa/wjrConSejTUhutcy+zfXYroD1c8JFhCspwDnGkmiVkAmilmz6zBUgel5Ee+GSUze/QoX843IhU/D7fbChkN99L586wp81phVFJafXEDm0npqktJnv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jyzGauJx; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jyzGauJx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325314; x=1761861314; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=9a1sAnRX7zf6QJZHCa5uRKk1d/kP28qRgid/slWPCtw=; b=jyzGauJxS2/IoeFX3IN0xZLm5P/zjWPqlMVLpbWaAu5Y2OnFKoPI9C39 AFo8sCKnUto0CG6LMiSQ8ZhTdLmy6y9tzcuUoEippXw+d1yJq0kCOSXR2 0o8Ki3wBlFVqPueUXGJSumTS6Tz4CqnaCyMObcoZq5bSHQJpzd+3JcR2m r4uw9VwSJ+F5hvYO8zE+n/wjEINKFVBoHfyuIng+o4qlVT1sIuWfyha65 3+kNqejTBSgzsGVS8tZwrLvnq5G5FWaTtkX/fo1r64LbSoC1IDI/3M23k g1khUbtz8BVLUEf32dw/q4/XZt0uQ4SijqCiIbLpBmPg3Q4SrWuqwNnvD w==; X-CSE-ConnectionGUID: +bXnFXSSRaKc3XF4Lu1FIA== X-CSE-MsgGUID: 2QFrB/D2TAW5b1d7JtH5Lw== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620539" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620539" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:13 -0700 X-CSE-ConnectionGUID: H7UK86xTScW7QbWVr3GA9A== X-CSE-MsgGUID: hPwBkY75SoCi0y6WiO9xEg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899977" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:12 -0700 From: ira.weiny@intel.com Date: Wed, 30 Oct 2024 16:54:47 -0500 Subject: [ndctl PATCH 4/6] cxl/region: Add creation of Dynamic capacity regions Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-4-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny , Sushant1 Kumar X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=21119; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=FVsSpGStbX5yclkjrSSbkTX6IUp5xgqEI5oxJhk5glM=; b=qx2V9YpQzdPFDdBjgFCRSysfE9UdyJAg3gFZafIZL7gENY1qDzNhRZU8witSkScr3XEUFnpoi 0KHm2eTcNs9DC4pmpr4WIZfZ9wenby0P1SPJOsqb1LdlOd+VaDzkM2t X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= From: Navneet Singh CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be spare and defined as dynamic capacity (dc). Add support for DCD devices. Query for DCD capabilities. Add the ability to add DC partitions to a CXL DC region. Signed-off-by: Navneet Singh Co-authored-by: Sushant1 Kumar Signed-off-by: Sushant1 Kumar Co-authored-by: Ira Weiny Signed-off-by: Ira Weiny --- Changes: [iweiny: adjust to new sysfs interface.] [iweiny: Rebase to latest pending] [iweiny: Adjust DCD region code to new upstream sysfs entries] [iweiny: Ensure backwards compatibility for non-DC kernels] [iweiny: fixup help message to show DC type] [iweiny: don't double declare decoder mode is dc] [iweiny: simplify __reserve_dpa() with decoder mode to index] [iweiny: Adjust to the new region mode] --- cxl/json.c | 26 +++++++++++++++ cxl/lib/libcxl.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++++++- cxl/lib/libcxl.sym | 3 ++ cxl/lib/private.h | 6 +++- cxl/libcxl.h | 55 +++++++++++++++++++++++++++++-- cxl/memdev.c | 7 +++- cxl/region.c | 49 ++++++++++++++++++++++++++-- 7 files changed, 234 insertions(+), 7 deletions(-) diff --git a/cxl/json.c b/cxl/json.c index dcd3cc28393faf7e8adf299a857531ecdeaac50a..4276b9678d7e03eaf2aec581a08450f2a0b857f2 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -754,10 +754,12 @@ err_free: return jpoison; } +#define DC_SIZE_NAME_LEN 64 struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, unsigned long flags) { const char *devname = cxl_memdev_get_devname(memdev); + char size_name[DC_SIZE_NAME_LEN]; struct json_object *jdev, *jobj; unsigned long long serial, size; const char *fw_version; @@ -800,6 +802,17 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, } } + for (int index; index < MAX_NUM_DC_REGIONS; index++) { + size = cxl_memdev_get_dc_size(memdev, index); + if (size) { + jobj = util_json_object_size(size, flags); + if (jobj) { + sprintf(size_name, "dc%d_size", index); + json_object_object_add(jdev, + size_name, jobj); + } + } + } if (flags & UTIL_JSON_HEALTH) { jobj = util_cxl_memdev_health_to_json(memdev, flags); if (jobj) @@ -948,11 +961,13 @@ struct json_object *util_cxl_bus_to_json(struct cxl_bus *bus, return jbus; } +#define DC_CAPABILITY_NAME_LEN 16 struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, unsigned long flags) { const char *devname = cxl_decoder_get_devname(decoder); struct cxl_port *port = cxl_decoder_get_port(decoder); + char dc_capable_name[DC_CAPABILITY_NAME_LEN]; struct json_object *jdecoder, *jobj; struct cxl_region *region; u64 val, size; @@ -1059,6 +1074,17 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add( jdecoder, "volatile_capable", jobj); } + for (int index = 0; index < MAX_NUM_DC_REGIONS; index++) { + if (cxl_decoder_is_dc_capable(decoder, index)) { + jobj = json_object_new_boolean(true); + if (jobj) { + sprintf(dc_capable_name, "dc%d_capable", index); + json_object_object_add(jdecoder, + dc_capable_name, + jobj); + } + } + } } if (cxl_port_is_root(port) && diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 5cbfb3e7d466b491ef87ea285f7e50d3bac230db..4caa2d02313bf71960971c4eaa67fa42cea08d55 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -1267,7 +1267,6 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base) char buf[SYSFS_ATTR_SIZE]; struct stat st; char *host; - if (!path) return NULL; dbg(ctx, "%s: base: \'%s\'\n", devname, cxlmem_base); @@ -1304,6 +1303,19 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base) else memdev->ram_qos_class = atoi(buf); + memdev->dc_partition_count = 0; + for (int partition = 0; partition < MAX_NUM_DC_REGIONS; partition++) { + sprintf(path, "%s/dc%d/size", cxlmem_base, partition); + if (sysfs_read_attr(ctx, path, buf) < 0) + continue; + memdev->dc_size[partition] = strtoull(buf, NULL, 0); + memdev->dc_partition_count++; + sprintf(path, "%s/dc%d/qos_class", cxlmem_base, partition); + if (sysfs_read_attr(ctx, path, buf) < 0) + continue; + memdev->dc_qos_class[partition] = strtoull(buf, NULL, 0); + } + sprintf(path, "%s/payload_max", cxlmem_base); if (sysfs_read_attr(ctx, path, buf) == 0) { memdev->payload_max = strtoull(buf, NULL, 0); @@ -1540,6 +1552,14 @@ CXL_EXPORT int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev) return memdev->ram_qos_class; } +CXL_EXPORT unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, int index) +{ + if (index >= 0 && index < MAX_NUM_DC_REGIONS) + return memdev->dc_size[index]; + + return 0; +} + CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev) { return memdev->firmware_version; @@ -2275,6 +2295,22 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) decoder->mode = CXL_DECODER_MODE_RAM; else if (strcmp(buf, "pmem") == 0) decoder->mode = CXL_DECODER_MODE_PMEM; + else if (strcmp(buf, "dc0") == 0) + decoder->mode = CXL_DECODER_MODE_DC0; + else if (strcmp(buf, "dc1") == 0) + decoder->mode = CXL_DECODER_MODE_DC1; + else if (strcmp(buf, "dc2") == 0) + decoder->mode = CXL_DECODER_MODE_DC2; + else if (strcmp(buf, "dc3") == 0) + decoder->mode = CXL_DECODER_MODE_DC3; + else if (strcmp(buf, "dc4") == 0) + decoder->mode = CXL_DECODER_MODE_DC4; + else if (strcmp(buf, "dc5") == 0) + decoder->mode = CXL_DECODER_MODE_DC5; + else if (strcmp(buf, "dc6") == 0) + decoder->mode = CXL_DECODER_MODE_DC6; + else if (strcmp(buf, "dc7") == 0) + decoder->mode = CXL_DECODER_MODE_DC7; else if (strcmp(buf, "mixed") == 0) decoder->mode = CXL_DECODER_MODE_MIXED; else if (strcmp(buf, "none") == 0) @@ -2318,6 +2354,14 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) case CXL_PORT_SWITCH: decoder->pmem_capable = true; decoder->volatile_capable = true; + decoder->dc_capable[0] = true; + decoder->dc_capable[1] = true; + decoder->dc_capable[2] = true; + decoder->dc_capable[3] = true; + decoder->dc_capable[4] = true; + decoder->dc_capable[5] = true; + decoder->dc_capable[6] = true; + decoder->dc_capable[7] = true; decoder->mem_capable = true; decoder->accelmem_capable = true; sprintf(path, "%s/locked", cxldecoder_base); @@ -2341,6 +2385,14 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) { "cap_type2", &decoder->accelmem_capable }, { "cap_type3", &decoder->mem_capable }, { "cap_ram", &decoder->volatile_capable }, + { "cap_ram", &decoder->dc_capable[0] }, + { "cap_ram", &decoder->dc_capable[1] }, + { "cap_ram", &decoder->dc_capable[2] }, + { "cap_ram", &decoder->dc_capable[3] }, + { "cap_ram", &decoder->dc_capable[4] }, + { "cap_ram", &decoder->dc_capable[5] }, + { "cap_ram", &decoder->dc_capable[6] }, + { "cap_ram", &decoder->dc_capable[7] }, { "cap_pmem", &decoder->pmem_capable }, { "locked", &decoder->locked }, }; @@ -2592,6 +2644,30 @@ CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder, case CXL_DECODER_MODE_RAM: sprintf(buf, "ram"); break; + case CXL_DECODER_MODE_DC0: + sprintf(buf, "dc0"); + break; + case CXL_DECODER_MODE_DC1: + sprintf(buf, "dc1"); + break; + case CXL_DECODER_MODE_DC2: + sprintf(buf, "dc2"); + break; + case CXL_DECODER_MODE_DC3: + sprintf(buf, "dc3"); + break; + case CXL_DECODER_MODE_DC4: + sprintf(buf, "dc4"); + break; + case CXL_DECODER_MODE_DC5: + sprintf(buf, "dc5"); + break; + case CXL_DECODER_MODE_DC6: + sprintf(buf, "dc6"); + break; + case CXL_DECODER_MODE_DC7: + sprintf(buf, "dc7"); + break; default: err(ctx, "%s: unsupported mode: %d\n", cxl_decoder_get_devname(decoder), mode); @@ -2648,6 +2724,14 @@ CXL_EXPORT bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder) return decoder->mem_capable; } +CXL_EXPORT bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, int index) +{ + if (index >= 0 && index < MAX_NUM_DC_REGIONS) + return decoder->dc_capable[index]; + + return 0; +} + CXL_EXPORT bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder) { return decoder->accelmem_capable; @@ -2717,6 +2801,8 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder, sprintf(path, "%s/create_pmem_region", decoder->dev_path); else if (mode == CXL_DECODER_MODE_RAM) sprintf(path, "%s/create_ram_region", decoder->dev_path); + else if (cxl_decoder_mode_is_dc(mode)) + sprintf(path, "%s/create_dc_region", decoder->dev_path); rc = sysfs_read_attr(ctx, path, buf); if (rc < 0) { @@ -2768,6 +2854,13 @@ cxl_decoder_create_ram_region(struct cxl_decoder *decoder) return cxl_decoder_create_region(decoder, CXL_DECODER_MODE_RAM); } +CXL_EXPORT struct cxl_region * +cxl_decoder_create_dc_region(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode) +{ + return cxl_decoder_create_region(decoder, mode); +} + CXL_EXPORT int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder) { return decoder->nr_targets; diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index b5d9bdcc38e09812f26afc1cb0e804f86784b8e6..351da7512e05080d847fd87740488d613462dbc9 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -19,6 +19,7 @@ global: cxl_memdev_get_ctx; cxl_memdev_get_pmem_size; cxl_memdev_get_ram_size; + cxl_memdev_get_dc_size; cxl_memdev_get_firmware_verison; cxl_cmd_get_devname; cxl_cmd_new_raw; @@ -247,6 +248,8 @@ LIBCXL_5 { global: cxl_region_get_mode; cxl_decoder_create_ram_region; + cxl_decoder_is_dc_capable; + cxl_decoder_create_dc_region; cxl_region_get_daxctl_region; cxl_port_get_parent_dport; } LIBCXL_4; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 0f45be89b6a00477d13fb6d7f1906213a3073c48..10abfa63dfc759b1589f9f039da1b920f8eb605e 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -12,7 +12,6 @@ #include #define CXL_EXPORT __attribute__ ((visibility("default"))) - struct cxl_pmem { int id; void *dev_buf; @@ -47,6 +46,9 @@ struct cxl_memdev { struct list_node list; unsigned long long pmem_size; unsigned long long ram_size; + unsigned long long dc_size[MAX_NUM_DC_REGIONS]; + unsigned long long dc_qos_class[MAX_NUM_DC_REGIONS]; + int dc_partition_count; int ram_qos_class; int pmem_qos_class; int payload_max; @@ -111,6 +113,7 @@ struct cxl_endpoint { struct cxl_memdev *memdev; }; + struct cxl_target { struct list_node list; struct cxl_decoder *decoder; @@ -140,6 +143,7 @@ struct cxl_decoder { bool pmem_capable; bool volatile_capable; bool mem_capable; + bool dc_capable[MAX_NUM_DC_REGIONS]; bool accelmem_capable; bool locked; enum cxl_decoder_target_type target_type; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 06b87a0924faafec6c80eca83ea7551d4e117256..17ed682548b970d57f016942badc76dce61bdeaf 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -72,6 +72,7 @@ int cxl_memdev_get_minor(struct cxl_memdev *memdev); struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev); +unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, int index); int cxl_memdev_get_pmem_qos_class(struct cxl_memdev *memdev); int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev); const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev); @@ -191,11 +192,20 @@ unsigned long long cxl_decoder_get_max_available_extent(struct cxl_decoder *decoder); int cxl_root_decoder_get_qos_class(struct cxl_decoder *decoder); +#define MAX_NUM_DC_REGIONS 8 enum cxl_decoder_mode { CXL_DECODER_MODE_NONE, CXL_DECODER_MODE_MIXED, CXL_DECODER_MODE_PMEM, CXL_DECODER_MODE_RAM, + CXL_DECODER_MODE_DC0, + CXL_DECODER_MODE_DC1, + CXL_DECODER_MODE_DC2, + CXL_DECODER_MODE_DC3, + CXL_DECODER_MODE_DC4, + CXL_DECODER_MODE_DC5, + CXL_DECODER_MODE_DC6, + CXL_DECODER_MODE_DC7, }; static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) @@ -205,9 +215,17 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) [CXL_DECODER_MODE_MIXED] = "mixed", [CXL_DECODER_MODE_PMEM] = "pmem", [CXL_DECODER_MODE_RAM] = "ram", + [CXL_DECODER_MODE_DC0] = "dc0", + [CXL_DECODER_MODE_DC1] = "dc1", + [CXL_DECODER_MODE_DC2] = "dc2", + [CXL_DECODER_MODE_DC3] = "dc3", + [CXL_DECODER_MODE_DC4] = "dc4", + [CXL_DECODER_MODE_DC5] = "dc5", + [CXL_DECODER_MODE_DC6] = "dc6", + [CXL_DECODER_MODE_DC7] = "dc7", }; - if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_RAM) + if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_DC7) mode = CXL_DECODER_MODE_NONE; return names[mode]; } @@ -221,9 +239,35 @@ cxl_decoder_mode_from_ident(const char *ident) return CXL_DECODER_MODE_RAM; else if (strcmp(ident, "pmem") == 0) return CXL_DECODER_MODE_PMEM; + else if (strcmp(ident, "dc0") == 0) + return CXL_DECODER_MODE_DC0; + else if (strcmp(ident, "dc1") == 0) + return CXL_DECODER_MODE_DC1; + else if (strcmp(ident, "dc2") == 0) + return CXL_DECODER_MODE_DC2; + else if (strcmp(ident, "dc3") == 0) + return CXL_DECODER_MODE_DC3; + else if (strcmp(ident, "dc4") == 0) + return CXL_DECODER_MODE_DC4; + else if (strcmp(ident, "dc5") == 0) + return CXL_DECODER_MODE_DC5; + else if (strcmp(ident, "dc6") == 0) + return CXL_DECODER_MODE_DC6; + else if (strcmp(ident, "dc7") == 0) + return CXL_DECODER_MODE_DC7; return CXL_DECODER_MODE_NONE; } +static inline bool cxl_decoder_mode_is_dc(enum cxl_decoder_mode mode) +{ + return (mode >= CXL_DECODER_MODE_DC0 && mode <= CXL_DECODER_MODE_DC7); +} + +static inline int cxl_decoder_dc_mode_to_index(enum cxl_decoder_mode mode) +{ + return mode - CXL_DECODER_MODE_DC0; +} + enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); @@ -248,6 +292,7 @@ enum cxl_decoder_target_type { enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder); bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); +bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, int index); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder); @@ -258,6 +303,8 @@ unsigned int cxl_decoder_get_interleave_ways(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_get_region(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_create_pmem_region(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_create_ram_region(struct cxl_decoder *decoder); +struct cxl_region *cxl_decoder_create_dc_region(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode); struct cxl_decoder *cxl_decoder_get_by_name(struct cxl_ctx *ctx, const char *ident); struct cxl_memdev *cxl_decoder_get_memdev(struct cxl_decoder *decoder); @@ -308,6 +355,7 @@ enum cxl_region_mode { CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, + CXL_REGION_MODE_DC, }; static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) @@ -317,9 +365,10 @@ static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) [CXL_REGION_MODE_MIXED] = "mixed", [CXL_REGION_MODE_PMEM] = "pmem", [CXL_REGION_MODE_RAM] = "ram", + [CXL_REGION_MODE_DC] = "dc", }; - if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM) + if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_DC) mode = CXL_REGION_MODE_NONE; return names[mode]; } @@ -333,6 +382,8 @@ cxl_region_mode_from_ident(const char *ident) return CXL_REGION_MODE_RAM; else if (strcmp(ident, "pmem") == 0) return CXL_REGION_MODE_PMEM; + else if (strcmp(ident, "dc") == 0) + return CXL_REGION_MODE_DC; return CXL_REGION_MODE_NONE; } diff --git a/cxl/memdev.c b/cxl/memdev.c index 6e44d1578d03b6af998502e54714635b8f31b556..0a7d350efe9e612cd67d32328cca286dcdcb2991 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -269,8 +269,13 @@ static int __reserve_dpa(struct cxl_memdev *memdev, if (mode == CXL_DECODER_MODE_RAM) avail_dpa = cxl_memdev_get_ram_size(memdev); - else + else if (mode == CXL_DECODER_MODE_PMEM) avail_dpa = cxl_memdev_get_pmem_size(memdev); + else if (cxl_decoder_mode_is_dc(mode)) { + int i = cxl_decoder_dc_mode_to_index(mode); + + avail_dpa = cxl_memdev_get_dc_size(memdev, i); + } cxl_decoder_foreach(port, decoder) { size = cxl_decoder_get_dpa_size(decoder); diff --git a/cxl/region.c b/cxl/region.c index 207cf2d003148992255c715f286bc0f38de2ca84..310694ae07fae25f13d032a30c130bf7d3394388 100644 --- a/cxl/region.c +++ b/cxl/region.c @@ -78,7 +78,7 @@ OPT_INTEGER('w', "ways", ¶m.ways, \ OPT_INTEGER('g', "granularity", ¶m.granularity, \ "granularity of the interleave set"), \ OPT_STRING('t', "type", ¶m.type, \ - "region type", "region type - 'pmem' or 'ram'"), \ + "region type", "region type - 'pmem', 'ram', 'dcX'"), \ OPT_STRING('U', "uuid", ¶m.uuid, \ "region uuid", "uuid for the new region (default: autogenerate)"), \ OPT_BOOLEAN('m', "memdevs", ¶m.memdevs, \ @@ -400,9 +400,22 @@ static int parse_region_options(int argc, const char **argv, } } +static int dc_mode_to_region_index(enum cxl_decoder_mode mode) +{ + int index = 0; + + for (unsigned int i = CXL_DECODER_MODE_DC0; i <= CXL_DECODER_MODE_DC7; i++) { + if (mode == i) + return index; + index++; + } + + return -EINVAL; +} + static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p) { - int i; + int i, index; for (i = 0; i < p->ways; i++) { struct json_object *jobj = @@ -417,6 +430,10 @@ static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p) case CXL_DECODER_MODE_PMEM: size = cxl_memdev_get_pmem_size(memdev); break; + case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7: + index = dc_mode_to_region_index(p->mode); + size = cxl_memdev_get_dc_size(memdev, index); + break; default: /* Shouldn't ever get here */ ; } @@ -473,6 +490,7 @@ static int validate_decoder(struct cxl_decoder *decoder, struct parsed_params *p) { const char *devname = cxl_decoder_get_devname(decoder); + int index; int rc; switch(p->mode) { @@ -488,6 +506,13 @@ static int validate_decoder(struct cxl_decoder *decoder, return -EINVAL; } break; + case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7: + index = dc_mode_to_region_index(p->mode); + if (!cxl_decoder_is_dc_capable(decoder, index)) { + log_err(&rl, "%s is not dc%d capable\n", devname, index); + return -EINVAL; + } + break; default: log_err(&rl, "unknown type: %s\n", param.type); return -EINVAL; @@ -502,12 +527,25 @@ static int validate_decoder(struct cxl_decoder *decoder, return 0; } +static enum cxl_decoder_mode dc_region_index_to_mode(int index) +{ + return (CXL_DECODER_MODE_DC0 + index); +} + static void set_type_from_decoder(struct cxl_ctx *ctx, struct parsed_params *p) { /* if param.type was explicitly specified, nothing to do here */ if (param.type) return; + /* Only chose DC if it is the only type available */ + for (int index = 0; index < MAX_NUM_DC_REGIONS; index++) { + if (cxl_decoder_is_dc_capable(p->root_decoder, index)) { + p->mode = dc_region_index_to_mode(index); + break; + } + } + /* * default to pmem if both types are set, otherwise the single * capability dominates. @@ -699,6 +737,13 @@ static int create_region(struct cxl_ctx *ctx, int *count, param.root_decoder); return -ENXIO; } + } else if (cxl_decoder_mode_is_dc(p->mode)) { + region = cxl_decoder_create_dc_region(p->root_decoder, p->mode); + if (!region) { + log_err(&rl, "failed to create region under %s\n", + param.root_decoder); + return -ENXIO; + } } else { log_err(&rl, "region type '%s' is not supported\n", param.type); From patchwork Wed Oct 30 21:54:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857252 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 391EB1DC1A2 for ; Wed, 30 Oct 2024 21:55:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325317; cv=none; b=nNguwIws+ZWjW2+6io60/1wIcdCv8D7ZvsYTFtnuYXo0JLZWqHbSl7Es6D29JNQ2dDrkKMs0skwgWShYbQVgaBEM2o1Y73oX/a9qwI1WlWG0f+umgZbOfhm4hlSl7KmM74gc8sSFdz6pwggQOqCHpXpVYUdN0ENQUM47Gwm37jE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325317; c=relaxed/simple; bh=hNg5R9mU2h3DU1+K9LO/y4ujrMoMCFDqEMxAASaRVp4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Kz4YRlsQiS+H7zkOUp5e2rS3/0RdFDj/yJyxkeALP3QL9bApOhL8HWmZBpxDCdNiohCc352twwGK48/G640ba2anabDGkjRRVRZEh0+9nVabRn/4va857Sy7omCQcAhUY4OzqQ2BXle6nFg4GUmRj0fwn51sMgHxvgp+yxytGo0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mG+BAjRk; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mG+BAjRk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325315; x=1761861315; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=hNg5R9mU2h3DU1+K9LO/y4ujrMoMCFDqEMxAASaRVp4=; b=mG+BAjRkUOB22dHVbYZ49idn2yke2iMkdhHh16qK8QAQw9/o2cx7175K flFV9Z1m2Bnm/rpr+GU4BIdj+01lJRIC7CDE+2Glfxwn5AnEi/37aH5tr 5TUC4Pstszm7//Bt/NexPMoVJl+xR6XvTVq+t6tER18I6+K+bllnyZcGU J3vBtsVHnx9Iu3pAQEiiAILb0tANeTybr6ZaBAGIg64EsjOF5vDtfS1X8 OJiyg/+PJzVCMWuwNfM4e1akr3E5HjXKGBJp0S+dTOw+r6WId8ZGa1XUV 2f5QlNaFF9WlEhbklqtl4o6V1SVuMmQ1tqFeoOh8Bs2soiOfVXQDviS/e Q==; X-CSE-ConnectionGUID: qYYfrMbCReqkwn94wz1Ndg== X-CSE-MsgGUID: 5PLe9GrBSO29T/IinhKtaA== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620545" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620545" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:15 -0700 X-CSE-ConnectionGUID: lpkXNkHGTOith2cdtvswhg== X-CSE-MsgGUID: DGgBbKG7Q32UyQ7SK3L7mA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899982" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:14 -0700 From: Ira Weiny Date: Wed, 30 Oct 2024 16:54:48 -0500 Subject: [ndctl PATCH 5/6] ndctl/cxl: Add extent output to region query Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-5-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=12704; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=hNg5R9mU2h3DU1+K9LO/y4ujrMoMCFDqEMxAASaRVp4=; b=5r1axWgAQ1gT1N3AGx+Epze41HZVk9IuTzcBfa7BXFr4/okrYf/CDLuBW8Yb0iL3O67e+5SCw 4OSPWr2NZxRB+s1CExcECaGTTL/ZzWmmHtrYpsFxrcq3E1TJy89oRU1 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= DCD regions have 0 or more extents. The ability to list those and their properties is useful to end users. Add extent output to region queries. Signed-off-by: Ira Weiny Tested-by: Fan Ni --- Documentation/cxl/cxl-list.txt | 4 ++ cxl/filter.h | 3 + cxl/json.c | 47 ++++++++++++++ cxl/json.h | 3 + cxl/lib/libcxl.c | 138 +++++++++++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 5 ++ cxl/lib/private.h | 11 ++++ cxl/libcxl.h | 11 ++++ cxl/list.c | 3 + util/json.h | 1 + 10 files changed, 226 insertions(+) diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt index 9a9911e7dd9bba561c6202784017db1bb4b9f4bd..71fd313cfec2509c79f8ad1e0f64857d0d804c13 100644 --- a/Documentation/cxl/cxl-list.txt +++ b/Documentation/cxl/cxl-list.txt @@ -411,6 +411,10 @@ OPTIONS } ---- +-N:: +--extents:: + Extend Dynamic Capacity region listings extent information. + -r:: --region:: Specify CXL region device name(s), or device id(s), to filter the listing. diff --git a/cxl/filter.h b/cxl/filter.h index 956a46e0c7a9f05abf696cce97a365164e95e50d..a31b80c87ccac407bd4ff98b302a23b33cbe413c 100644 --- a/cxl/filter.h +++ b/cxl/filter.h @@ -31,6 +31,7 @@ struct cxl_filter_params { bool alert_config; bool dax; bool media_errors; + bool extents; int verbose; struct log_ctx ctx; }; @@ -91,6 +92,8 @@ static inline unsigned long cxl_filter_to_flags(struct cxl_filter_params *param) flags |= UTIL_JSON_DAX | UTIL_JSON_DAX_DEVS; if (param->media_errors) flags |= UTIL_JSON_MEDIA_ERRORS; + if (param->extents) + flags |= UTIL_JSON_EXTENTS; return flags; } diff --git a/cxl/json.c b/cxl/json.c index 4276b9678d7e03eaf2aec581a08450f2a0b857f2..9708ecd340d8c337a548909474ab2763ff3125da 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -1170,6 +1170,50 @@ void util_cxl_mappings_append_json(struct json_object *jregion, json_object_object_add(jregion, "mappings", jmappings); } +void util_cxl_extents_append_json(struct json_object *jregion, + struct cxl_region *region, + unsigned long flags) +{ + struct json_object *jextents; + struct cxl_region_extent *extent; + + jextents = json_object_new_array(); + if (!jextents) + return; + + cxl_extent_foreach(region, extent) { + struct json_object *jextent, *jobj; + unsigned long long val; + char tag_str[40]; + uuid_t tag; + + jextent = json_object_new_object(); + if (!jextent) + continue; + + val = cxl_extent_get_offset(extent); + jobj = util_json_object_hex(val, flags); + if (jobj) + json_object_object_add(jextent, "offset", jobj); + + val = cxl_extent_get_length(extent); + jobj = util_json_object_size(val, flags); + if (jobj) + json_object_object_add(jextent, "length", jobj); + + cxl_extent_get_tag(extent, tag); + uuid_unparse(tag, tag_str); + jobj = json_object_new_string(tag_str); + if (jobj) + json_object_object_add(jextent, "tag", jobj); + + json_object_array_add(jextents, jextent); + json_object_set_userdata(jextent, extent, NULL); + } + + json_object_object_add(jregion, "extents", jextents); +} + struct json_object *util_cxl_region_to_json(struct cxl_region *region, unsigned long flags) { @@ -1256,6 +1300,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, } } + if (flags & UTIL_JSON_EXTENTS) + util_cxl_extents_append_json(jregion, region, flags); + if (cxl_region_qos_class_mismatch(region)) { jobj = json_object_new_boolean(true); if (jobj) diff --git a/cxl/json.h b/cxl/json.h index eb7572be4106baf0469ba9243a9a767d07df8882..f9c07ab41a337838b75ffee4486f6c48ddc99863 100644 --- a/cxl/json.h +++ b/cxl/json.h @@ -20,6 +20,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, void util_cxl_mappings_append_json(struct json_object *jregion, struct cxl_region *region, unsigned long flags); +void util_cxl_extents_append_json(struct json_object *jregion, + struct cxl_region *region, + unsigned long flags); void util_cxl_targets_append_json(struct json_object *jdecoder, struct cxl_decoder *decoder, const char *ident, const char *serial, diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 4caa2d02313bf71960971c4eaa67fa42cea08d55..8ebb100df0c6078630bbe45fbed270709dfb4a5f 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -568,6 +568,7 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base) region->ctx = ctx; region->decoder = decoder; list_head_init(®ion->mappings); + list_head_init(®ion->extents); region->dev_path = strdup(cxlregion_base); if (!region->dev_path) @@ -1178,6 +1179,143 @@ cxl_mapping_get_next(struct cxl_memdev_mapping *mapping) return list_next(®ion->mappings, mapping, list); } +static void cxl_extents_init(struct cxl_region *region) +{ + const char *devname = cxl_region_get_devname(region); + struct cxl_ctx *ctx = cxl_region_get_ctx(region); + char *extent_path, *dax_region_path; + struct dirent *de; + DIR *dir = NULL; + + if (region->extents_init) + return; + region->extents_init = 1; + + dbg(ctx, "Checking extents: %s\n", region->dev_path); + + dax_region_path = calloc(1, strlen(region->dev_path) + 64); + if (!dax_region_path) { + err(ctx, "%s: allocation failure\n", devname); + return; + } + + extent_path = calloc(1, strlen(region->dev_path) + 100); + if (!extent_path) { + err(ctx, "%s: allocation failure\n", devname); + free(dax_region_path); + return; + } + + sprintf(dax_region_path, "%s/dax_region%d", + region->dev_path, region->id); + dir = opendir(dax_region_path); + if (!dir) { + err(ctx, "no extents found: %s\n", dax_region_path); + free(extent_path); + free(dax_region_path); + return; + } + + while ((de = readdir(dir)) != NULL) { + struct cxl_region_extent *extent; + char buf[SYSFS_ATTR_SIZE]; + u64 offset, length; + int id, region_id; + + if (sscanf(de->d_name, "extent%d.%d", ®ion_id, &id) != 2) + continue; + + sprintf(extent_path, "%s/extent%d.%d/offset", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/offset\n", + devname, region_id, id); + continue; + } + + offset = strtoull(buf, NULL, 0); + if (offset == ERANGE) { + err(ctx, "%s extent%d.%d: failed to read offset\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/length", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/length\n", + devname, region_id, id); + continue; + } + + length = strtoull(buf, NULL, 0); + if (length == ERANGE) { + err(ctx, "%s extent%d.%d: failed to read length\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/tag", + dax_region_path, region_id, id); + buf[0] = '\0'; + if (sysfs_read_attr(ctx, extent_path, buf) != 0) + dbg(ctx, "%s extent%d.%d: failed to read tag\n", + devname, region_id, id); + + extent = calloc(1, sizeof(*extent)); + if (!extent) { + err(ctx, "%s extent%d.%d: allocation failure\n", + devname, region_id, id); + continue; + } + if (strlen(buf) && uuid_parse(buf, extent->tag) < 0) + err(ctx, "%s:%s\n", extent_path, buf); + extent->region = region; + extent->offset = offset; + extent->length = length; + + list_node_init(&extent->list); + list_add(®ion->extents, &extent->list); + } + free(dax_region_path); + free(extent_path); + closedir(dir); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_first(struct cxl_region *region) +{ + cxl_extents_init(region); + + return list_top(®ion->extents, struct cxl_region_extent, list); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_next(struct cxl_region_extent *extent) +{ + struct cxl_region *region = extent->region; + + return list_next(®ion->extents, extent, list); +} + +CXL_EXPORT unsigned long long +cxl_extent_get_offset(struct cxl_region_extent *extent) +{ + return extent->offset; +} + +CXL_EXPORT unsigned long long +cxl_extent_get_length(struct cxl_region_extent *extent) +{ + return extent->length; +} + +CXL_EXPORT void +cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag) +{ + memcpy(tag, extent->tag, sizeof(uuid_t)); +} + CXL_EXPORT struct cxl_decoder * cxl_mapping_get_decoder(struct cxl_memdev_mapping *mapping) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 351da7512e05080d847fd87740488d613462dbc9..37c3531115c73cdb69b96fa47bc88bbbb901f085 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -291,4 +291,9 @@ global: cxl_memdev_trigger_poison_list; cxl_region_trigger_poison_list; cxl_region_get_region_mode; + cxl_extent_get_first; + cxl_extent_get_next; + cxl_extent_get_offset; + cxl_extent_get_length; + cxl_extent_get_tag; } LIBCXL_7; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 10abfa63dfc759b1589f9f039da1b920f8eb605e..5b50b3f778a66a2266d6d5ee69e2a72cdad54a70 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -164,6 +164,7 @@ struct cxl_region { struct cxl_decoder *decoder; struct list_node list; int mappings_init; + int extents_init; struct cxl_ctx *ctx; void *dev_buf; size_t buf_len; @@ -179,6 +180,7 @@ struct cxl_region { struct daxctl_region *dax_region; struct kmod_module *module; struct list_head mappings; + struct list_head extents; }; struct cxl_memdev_mapping { @@ -188,6 +190,15 @@ struct cxl_memdev_mapping { struct list_node list; }; +#define CXL_REGION_EXTENT_TAG 0x10 +struct cxl_region_extent { + struct cxl_region *region; + u64 offset; + u64 length; + uuid_t tag; + struct list_node list; +}; + enum cxl_cmd_query_status { CXL_CMD_QUERY_NOT_RUN = 0, CXL_CMD_QUERY_OK, diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 17ed682548b970d57f016942badc76dce61bdeaf..b7c85a67224c86d17a41376c147364e1f88db080 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -448,6 +448,17 @@ unsigned int cxl_mapping_get_position(struct cxl_memdev_mapping *mapping); mapping != NULL; \ mapping = cxl_mapping_get_next(mapping)) +struct cxl_region_extent; +struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region); +struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent); +#define cxl_extent_foreach(region, extent) \ + for (extent = cxl_extent_get_first(region); \ + extent != NULL; \ + extent = cxl_extent_get_next(extent)) +unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent); +unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent); +void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag); + struct cxl_cmd; const char *cxl_cmd_get_devname(struct cxl_cmd *cmd); struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode); diff --git a/cxl/list.c b/cxl/list.c index 0b25d78248d5f4f529fd2c2e073e43895c722568..47d135166212b87449f960e94ee75657f7040ca9 100644 --- a/cxl/list.c +++ b/cxl/list.c @@ -59,6 +59,8 @@ static const struct option options[] = { "include alert configuration information"), OPT_BOOLEAN('L', "media-errors", ¶m.media_errors, "include media-error information "), + OPT_BOOLEAN('N', "extents", ¶m.extents, + "include extent information (Dynamic Capacity regions only)"), OPT_INCR('v', "verbose", ¶m.verbose, "increase output detail"), #ifdef ENABLE_DEBUG OPT_BOOLEAN(0, "debug", &debug, "debug list walk"), @@ -135,6 +137,7 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx) param.decoders = true; param.targets = true; param.regions = true; + param.extents = true; /*fallthrough*/ case 0: break; diff --git a/util/json.h b/util/json.h index 560f845c6753ee176f7c64b4310fe1f9b1ce6d39..79ae3240e7ce151be75f6666fcaba0ba90aba7fc 100644 --- a/util/json.h +++ b/util/json.h @@ -21,6 +21,7 @@ enum util_json_flags { UTIL_JSON_TARGETS = (1 << 11), UTIL_JSON_PARTITION = (1 << 12), UTIL_JSON_ALERT_CONFIG = (1 << 13), + UTIL_JSON_EXTENTS = (1 << 14), }; void util_display_json_array(FILE *f_out, struct json_object *jarray, From patchwork Wed Oct 30 21:54:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13857253 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 771471E0B66 for ; Wed, 30 Oct 2024 21:55:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325320; cv=none; b=O68aEwavWCaivXagmaBo12Kx83d1htwy9uaM5+QZa1sOZrdMt0L03xxqSv1eBvqXrAxMczzbOZHS1BXK9qbAFYqdtKJT7MiC2Jf8+FC2xDvssCd2di+YCw5OVxVusE8pdZdX5sbX7SsTRtWLwW3SCBfCpmXQFBjk8qek9FjaioQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730325320; c=relaxed/simple; bh=X3tSiPjMY0InL7je0ERdPbs9YjGKNmwj5fy9TRxpMsQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tPSxsxhlX1+ksDYdlhuKm5iy4yOfmkv0BFdhq30jbvkmDQl/+D2EPPK3hUCvax8saAq1E0zLs4ahLmlsGthUtfvIihW6Gc/KDiVHgsYwYQR/yDWaHPHIsDo7x/J3HaUaO+YJC4CQvmYf/MUYhH57I3FSchrM6AM8CtxJs913GY8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=C36ioHEf; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="C36ioHEf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730325318; x=1761861318; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=X3tSiPjMY0InL7je0ERdPbs9YjGKNmwj5fy9TRxpMsQ=; b=C36ioHEfsJ0XnbKoAxz0XRUAGirQWc2jpxmU6itHd4O70CJKdaBJvMFQ Vgax8xzNkA9cXQgdqxsZfvdvniwxh7Ej4YH6VpAS0k4j3d5fSWfd1FwJt hZgpbBo/6i+dposzMi9faKu1kXjVcTSuoIriZTG5/ZKyYtACBTizQwJTE PPZLPDemR0f3FRxfzjAp/QjglqOyv+6NBTP1UBeTMOOhn5uhpxfialv7R 6JrRli4ftphNdaBrvKB95OlAxKj9px9Gdd1tMIsCC8kkyb8HbPlZQhPAE AvKQvE40mtcHnoIrBG1wQNn4LEIKiqJe76tt5mRr0fIsXCdyI4Bx+LhCf Q==; X-CSE-ConnectionGUID: IAcalL/5RcmsMIEDnnVk9g== X-CSE-MsgGUID: rMxFexCHRqm2cZcCqjOD+g== X-IronPort-AV: E=McAfee;i="6700,10204,11241"; a="40620549" X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="40620549" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:17 -0700 X-CSE-ConnectionGUID: 34OIqzloRS6nYOhb4TedLA== X-CSE-MsgGUID: /XNGq3h7Rbirild4/lDp9g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,246,1725346800"; d="scan'208";a="119899992" Received: from msatwood-mobl.amr.corp.intel.com (HELO localhost) ([10.125.108.161]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Oct 2024 14:55:16 -0700 From: Ira Weiny Date: Wed, 30 Oct 2024 16:54:49 -0500 Subject: [ndctl PATCH 6/6] ndctl/cxl/test: Add Dynamic Capacity tests Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241030-dcd-region2-v1-6-04600ba2b48e@intel.com> References: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> In-Reply-To: <20241030-dcd-region2-v1-0-04600ba2b48e@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730325302; l=22154; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=X3tSiPjMY0InL7je0ERdPbs9YjGKNmwj5fy9TRxpMsQ=; b=An4F4rAv+CvS44mq8d+SFn0cDqg0U7cTak7J5y8D8SXLA1YqI2mgkcceqic680q/Mnfsp2KBQ fv7oXY3kZc8ARWowI6ptzhphMcfUGWaiXANiHjseI2NkftJeSppy1ie X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= cxl_test provides a good way to ensure quick smoke and regression testing. The complexity of DCD and the new sparse DAX regions required to use them benefits greatly with a series of smoke tests. The only part of the kernel stack which must be bypassed is the actual irq of DCD events. However, the event processing itself can be tested via cxl_test calling directly the event processing function directly. In this way the rest of the stack; management of sparse regions, the extent device lifetimes, and the dax device operations can be tested. Add Dynamic Capacity Device tests for kernels which have DCD support in cxl_test. Signed-off-by: Ira Weiny --- Changes; [iweiny: remove tag injection from test] [iweiny: add test for 2 regions in 1 DCD partition] [iweiny: better document region/extent layout expected for each test] [iweiny: Add more bit support and test] [iweiny: Enhance region check to look at the region size.] [iweiny: test allow create region on different dc partitions: default dc0] [iweiny: fix pre-existing extent comments] [iweiny: use new extent output from ndctl to check extents] [iweiny: fix returning the region name from create_dcd_region] [iweiny: add tag testing] --- test/cxl-dcd.sh | 656 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ test/meson.build | 2 + 2 files changed, 658 insertions(+) diff --git a/test/cxl-dcd.sh b/test/cxl-dcd.sh new file mode 100644 index 0000000000000000000000000000000000000000..f5248fce4f3899acdf646ace4f0eb531ea2286d3 --- /dev/null +++ b/test/cxl-dcd.sh @@ -0,0 +1,656 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2024 Intel Corporation. All rights reserved. + +. "$(dirname "$0")/common" + +rc=77 +set -ex + +trap 'err $LINENO' ERR + +check_prereq "jq" + +modprobe -r cxl_test +modprobe cxl_test +rc=1 + +dev_path="/sys/bus/platform/devices" +cxl_path="/sys/bus/cxl/devices" + +# a test extent tag +test_tag=dc-test-tag + +# +# The test devices have 2G of non DC capacity. A single DC reagion of 1G is +# added beyond that. +# +# The testing centers around 3 extents. Two are pre-existing on test load +# called pre-existing. The other is created within this script alone called +# base. + +# +# | 2G non- | DC region (1G) | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| |----------| |----------| | +# | | (base) | | (pre)- | | (pre2)- | | +# | | | | existing | | existing | | + +dcsize="" + +base_dpa=0x80000000 + +# base extent at dpa 2G - 64M long +base_ext_offset=0x0 +base_ext_dpa=$(($base_dpa + $base_ext_offset)) +base_ext_length=0x4000000 + +# pre existing extent base + 128M, 64M length +# 0x00000088000000-0x0000008bffffff +pre_ext_offset=0x8000000 +pre_ext_dpa=$(($base_dpa + $pre_ext_offset)) +pre_ext_length=0x4000000 + +# pre2 existing extent base + 256M, 64M length +# 0x00000090000000-0x00000093ffffff +pre2_ext_offset=0x10000000 +pre2_ext_dpa=$(($base_dpa + $pre2_ext_offset)) +pre2_ext_length=0x4000000 + +mem="" +bus="" +device="" +decoder="" + +create_dcd_region() +{ + mem="$1" + decoder="$2" + reg_size_string="" + if [ "$3" != "" ]; then + reg_size_string="-s $3" + fi + dcd_partition="dc0" + if [ "$4" != "" ]; then + dcd_partition="$4" + fi + + # create region + rc=$($CXL create-region -t ${dcd_partition} -d "$decoder" -m "$mem" ${reg_size_string} | jq -r ".region") + + if [[ ! $rc ]]; then + echo "create-region failed for $decoder / $mem" + err "$LINENO" + fi + + echo ${rc} +} + +check_region() +{ + search=$1 + region_size=$2 + + result=$($CXL list -r "$search" | jq -r ".[].region") + if [ "$result" != "$search" ]; then + echo "check region failed to find $search" + err "$LINENO" + fi + + result=$($CXL list -r "$search" | jq -r ".[].size") + if [ "$result" != "$region_size" ]; then + echo "check region failed invalid size $result != $region_size" + err "$LINENO" + fi +} + +check_not_region() +{ + search=$1 + + result=$($CXL list -r "$search" | jq -r ".[].region") + if [ "$result" == "$search" ]; then + echo "check not region failed; $search found" + err "$LINENO" + fi +} + +destroy_region() +{ + local region=$1 + $CXL disable-region $region + $CXL destroy-region $region +} + +inject_extent() +{ + device="$1" + dpa="$2" + length="$3" + tag="$4" + + more="0" + if [ "$5" != "" ]; then + more="1" + fi + + echo ${dpa}:${length}:${tag}:${more} > "${dev_path}/${device}/dc_inject_extent" +} + +remove_extent() +{ + device="$1" + dpa="$2" + length="$3" + + echo ${dpa}:${length} > "${dev_path}/${device}/dc_del_extent" +} + +create_dax_dev() +{ + reg="$1" + + dax_dev=$($DAXCTL create-device -r $reg | jq -er '.[].chardev') + + echo ${dax_dev} +} + +fail_create_dax_dev() +{ + reg="$1" + + set +e + result=$($DAXCTL create-device -r $reg) + set -e + if [ "$result" == "0" ]; then + echo "FAIL device created" + err "$LINENO" + fi +} + +shrink_dax_dev() +{ + dev="$1" + new_size="$2" + + $DAXCTL disable-device $dev + $DAXCTL reconfigure-device $dev -s $new_size + $DAXCTL enable-device $dev +} + +destroy_dax_dev() +{ + dev="$1" + + $DAXCTL disable-device $dev + $DAXCTL destroy-device $dev +} + +check_dax_dev() +{ + search="$1" + size="$2" + + result=$($DAXCTL list -d $search | jq -er '.[].chardev') + if [ "$result" != "$search" ]; then + echo "check dax device failed to find $search" + err "$LINENO" + fi + result=$($DAXCTL list -d $search | jq -er '.[].size') + if [ "$result" -ne "$size" ]; then + echo "check dax device failed incorrect size $result; exp $size" + err "$LINENO" + fi +} + +# check that the dax device is not there. +check_not_dax_dev() +{ + reg="$1" + search="$2" + result=$($DAXCTL list -r $reg -D | jq -er '.[].chardev') + if [ "$result" == "$search" ]; then + echo "FAIL found $search" + err "$LINENO" + fi +} + +check_extent() +{ + region=$1 + offset=$(($2)) + length=$(($3)) + + result=$($CXL list -r "$region" -N | jq -r ".[].extents[] | select(.offset == ${offset}) | .length") + if [[ $result != $length ]]; then + echo "FAIL region $1 could not find extent @ $offset ($length)" + err "$LINENO" + fi +} + +check_extent_cnt() +{ + region=$1 + count=$(($2)) + + result=$($CXL list -r $region -N | jq -r '.[].extents[].offset' | wc -l) + if [[ $result != $count ]]; then + echo "FAIL region $1: found wrong number of extents $result; expect $count" + err "$LINENO" + fi +} + +readarray -t memdevs < <("$CXL" list -b cxl_test -Mi | jq -r '.[].memdev') + +for mem in ${memdevs[@]}; do + dcsize=$($CXL list -m $mem | jq -r '.[].dc0_size') + if [ "$dcsize" == "null" ]; then + continue + fi + decoder=$($CXL list -b cxl_test -D -d root -m "$mem" | + jq -r ".[] | + select(.dc0_capable == true) | + select(.nr_targets == 1) | + select(.max_available_extent >= ${dcsize}) | + .decoder") + if [[ $decoder ]]; then + bus=`"$CXL" list -b cxl_test -m ${mem} | jq -r '.[].bus'` + device=$($CXL list -m $mem | jq -r '.[].host') + break + fi +done + +echo "TEST: DCD test device bus:${bus} decoder:${decoder} mem:${mem} device:${device} size:${dcsize}" + +if [ "$decoder" == "" ] || [ "$device" == "" ] || [ "$dcsize" == "" ]; then + echo "No mem device/decoder found with DCD support" + exit 77 +fi + +echo "" +echo "Test: pre-existing extent" +echo "" +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +# should contain pre-created extents +check_extent ${region} ${pre_ext_offset} ${pre_ext_length} +check_extent ${region} ${pre2_ext_offset} ${pre2_ext_length} + + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | | |----------| |----------| | +# | | | (pre)- | | (pre2)- | | +# | | | existing | | existing | | + +# Remove the pre-created test extent out from under dax device +# stack should hold ref until dax device deleted +echo "" +echo "Test: Remove extent from under DAX dev" +echo "" +dax_dev=$(create_dax_dev ${region}) +check_extent_cnt ${region} 2 +remove_extent ${device} $pre_ext_dpa $pre_ext_length +length="$(($pre_ext_length + $pre2_ext_length))" +check_dax_dev ${dax_dev} $length +check_extent_cnt ${region} 2 +destroy_dax_dev ${dax_dev} +check_not_dax_dev ${region} ${dax_dev} + +# In-use extents are not released. Remove after use. +check_extent_cnt ${region} 2 +remove_extent ${device} $pre_ext_dpa $pre_ext_length +remove_extent ${device} $pre2_ext_dpa $pre2_ext_length +check_extent_cnt ${region} 0 + +echo "" +echo "Test: Create dax device spanning 2 extents" +echo "" +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +check_extent ${region} ${pre_ext_offset} ${pre_ext_length} +inject_extent ${device} $base_ext_dpa $base_ext_length "" +check_extent ${region} ${base_ext_offset} ${base_ext_length} + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| |----------| | +# | | (base) | | (pre)- | | +# | | | | existing | | + +check_extent_cnt ${region} 2 +dax_dev=$(create_dax_dev ${region}) + +echo "" +echo "Test: dev dax is spanning sparse extents" +echo "" +ext_sum_length="$(($base_ext_length + $pre_ext_length))" +check_dax_dev ${dax_dev} $ext_sum_length + + +echo "" +echo "Test: Remove extents under sparse dax device" +echo "" +remove_extent ${device} $base_ext_dpa $base_ext_length +check_extent_cnt ${region} 2 +remove_extent ${device} $pre_ext_dpa $pre_ext_length +check_extent_cnt ${region} 2 +destroy_dax_dev ${dax_dev} +check_not_dax_dev ${region} ${dax_dev} + +# In-use extents are not released. Remove after use. +check_extent_cnt ${region} 2 +remove_extent ${device} $base_ext_dpa $base_ext_length +remove_extent ${device} $pre_ext_dpa $pre_ext_length +check_extent_cnt ${region} 0 + +echo "" +echo "Test: inject without/with tag" +echo "" +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +check_extent ${region} ${pre_ext_offset} ${pre_ext_length} +inject_extent ${device} $base_ext_dpa $base_ext_length "" +check_extent ${region} ${base_ext_offset} ${base_ext_length} +remove_extent ${device} $base_ext_dpa $base_ext_length +remove_extent ${device} $pre_ext_dpa $pre_ext_length +check_extent_cnt ${region} 0 + + +echo "" +echo "Test: partial extent remove" +echo "" +inject_extent ${device} $base_ext_dpa $base_ext_length "" +dax_dev=$(create_dax_dev ${region}) + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| | +# | | (base) | | +# | | |---| | +# Partial + +partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))" +partial_ext_length="$(($base_ext_length / 2))" +echo "Removing Partial : $partial_ext_dpa $partial_ext_length" +remove_extent ${device} $partial_ext_dpa $partial_ext_length +check_extent_cnt ${region} 1 +destroy_dax_dev ${dax_dev} +check_not_dax_dev ${region} ${dax_dev} + +# In-use extents are not released. Remove after use. +check_extent_cnt ${region} 1 +remove_extent ${device} $partial_ext_dpa $partial_ext_length +check_extent_cnt ${region} 0 + +# Test multiple extent remove +echo "" +echo "Test: multiple extent remove with single extent remove command" +echo "" +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +inject_extent ${device} $base_ext_dpa $base_ext_length "" +check_extent_cnt ${region} 2 +dax_dev=$(create_dax_dev ${region}) + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| |-------------------| | +# | | (base) | | (pre)-existing | | +# |------------------| +# Partial + +partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))" +partial_ext_length="$(($pre_ext_dpa - $base_ext_dpa))" +echo "Removing multiple in span : $partial_ext_dpa $partial_ext_length" +remove_extent ${device} $partial_ext_dpa $partial_ext_length +check_extent_cnt ${region} 2 +destroy_dax_dev ${dax_dev} +check_not_dax_dev ${region} ${dax_dev} + + +echo "" +echo "Test: Destroy region without extent removal" +echo "" + +# In-use extents are not released. +check_extent_cnt ${region} 2 +destroy_region ${region} +check_not_region ${region} + + +echo "" +echo "Test: Destroy region with extents and dax devices" +echo "" +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +check_extent_cnt ${region} 0 +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | | |----------| | +# | | | (pre)- | | +# | | | existing | | + +check_extent_cnt ${region} 1 +dax_dev=$(create_dax_dev ${region}) +destroy_region ${region} +check_not_region ${region} + +echo "" +echo "Test: Fail sparse dax dev creation without space" +echo "" +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | | |-------------------| | +# | | | (pre)-existing | | + +check_extent_cnt ${region} 1 + +# | | | dax0.1 | | + +dax_dev=$(create_dax_dev ${region}) +check_dax_dev ${dax_dev} $pre_ext_length +fail_create_dax_dev ${region} + +echo "" +echo "Test: Resize sparse dax device" +echo "" + +# Shrink +# | | | dax0.1 | | +resize_ext_length=$(($pre_ext_length / 2)) +shrink_dax_dev ${dax_dev} $resize_ext_length +check_dax_dev ${dax_dev} $resize_ext_length + +# Fill +# | | | dax0.1 | dax0.2 | | +dax_dev=$(create_dax_dev ${region}) +check_dax_dev ${dax_dev} $resize_ext_length +destroy_region ${region} +check_not_region ${region} + + +# 2 extent +# create dax dev +# resize into 1st extent +# create dev on rest of 1st and all of second +# Ensure both devices are correct + +echo "" +echo "Test: Resize sparse dax device across extents" +echo "" +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +inject_extent ${device} $base_ext_dpa $base_ext_length "" + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| |-------------------| | +# | | (base) | | (pre)-existing | | + +check_extent_cnt ${region} 2 +dax_dev=$(create_dax_dev ${region}) +ext_sum_length="$(($base_ext_length + $pre_ext_length))" + +# | | dax0.1 | | dax0.1 | | + +check_dax_dev ${dax_dev} $ext_sum_length +resize_ext_length=33554432 # 32MB + +# | | D1 | | + +shrink_dax_dev ${dax_dev} $resize_ext_length +check_dax_dev ${dax_dev} $resize_ext_length + +# | | D1 | D2| | dax0.2 | | + +dax_dev=$(create_dax_dev ${region}) +remainder_length=$((ext_sum_length - $resize_ext_length)) +check_dax_dev ${dax_dev} $remainder_length + +# | | D1 | D2| | dax0.2 | | + +remainder_length=$((remainder_length / 2)) +shrink_dax_dev ${dax_dev} $remainder_length +check_dax_dev ${dax_dev} $remainder_length + +# | | D1 | D2| | dax0.2 | dax0.3 | | + +dax_dev=$(create_dax_dev ${region}) +check_dax_dev ${dax_dev} $remainder_length +destroy_region ${region} +check_not_region ${region} + + +echo "" +echo "Test: Rejecting overlapping extents" +echo "" + +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + +# | 2G non- | DC region | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | | |-------------------| | +# | | | (pre)-existing | | + +check_extent_cnt ${region} 1 + +# Attempt overlapping extent +# +# | | |-----------------| | +# | | | overlapping | | + +partial_ext_dpa="$(($base_ext_dpa + ($pre_ext_dpa / 2)))" +partial_ext_length=$pre_ext_length +inject_extent ${device} $partial_ext_dpa $partial_ext_length "" + +# Should only see the original ext +check_extent_cnt ${region} 1 +destroy_region ${region} +check_not_region ${region} + + +echo "" +echo "Test: create 2 regions in the same DC partition" +echo "" +region_size=$(($dcsize / 2)) +region=$(create_dcd_region ${mem} ${decoder} ${region_size} dc1) +check_region ${region} ${region_size} + +region_two=$(create_dcd_region ${mem} ${decoder} ${region_size} dc1) +check_region ${region_two} ${region_size} + +destroy_region ${region_two} +check_not_region ${region_two} +destroy_region ${region} +check_not_region ${region} + + +echo "" +echo "Test: More bit" +echo "" +region=$(create_dcd_region ${mem} ${decoder}) +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" 1 +# More bit should hold off surfacing extent until the more bit is 0 +check_extent_cnt ${region} 0 +inject_extent ${device} $base_ext_dpa $base_ext_length "" +check_extent_cnt ${region} 2 +destroy_region ${region} +check_not_region ${region} + + +# Create a new region for driver tests +region=$(create_dcd_region ${mem} ${decoder}) + +echo "" +echo "Test: driver remove tear down" +echo "" +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +check_extent ${region} ${pre_ext_offset} ${pre_ext_length} +dax_dev=$(create_dax_dev ${region}) +# remove driver releases extents +modprobe -r dax_cxl +check_extent_cnt ${region} 0 + +# leave region up, driver removed. +echo "" +echo "Test: no driver inject ok" +echo "" +check_region ${region} ${dcsize} +inject_extent ${device} $pre_ext_dpa $pre_ext_length "" +check_extent_cnt ${region} 1 +modprobe dax_cxl +check_extent_cnt ${region} 1 + +destroy_region ${region} +check_not_region ${region} + + +# Test event reporting +# results expected +num_dcd_events_expected=2 + +echo "Test: Prep event trace" +echo "" > /sys/kernel/tracing/trace +echo 1 > /sys/kernel/tracing/events/cxl/enable +echo 1 > /sys/kernel/tracing/tracing_on + +inject_extent ${device} $base_ext_dpa $base_ext_length "" +remove_extent ${device} $base_ext_dpa $base_ext_length + +echo 0 > /sys/kernel/tracing/tracing_on + +echo "Test: Events seen" +trace_out=$(cat /sys/kernel/tracing/trace) + +# Look for DCD events +num_dcd_events=$(grep -c "cxl_dynamic_capacity" <<< "${trace_out}") +echo " LOG (Expected) : (Found)" +echo " DCD events ($num_dcd_events_expected) : $num_dcd_events" + +if [ "$num_dcd_events" -ne $num_dcd_events_expected ]; then + err "$LINENO" +fi + +modprobe -r cxl_test + +check_dmesg "$LINENO" + +exit 0 diff --git a/test/meson.build b/test/meson.build index d871e28e17ce512cd1e7b43f3ec081729fe5e03a..1cfcb60d16e05272893ae1c67820aa8614281505 100644 --- a/test/meson.build +++ b/test/meson.build @@ -161,6 +161,7 @@ cxl_sanitize = find_program('cxl-sanitize.sh') cxl_destroy_region = find_program('cxl-destroy-region.sh') cxl_qos_class = find_program('cxl-qos-class.sh') cxl_poison = find_program('cxl-poison.sh') +cxl_dcd = find_program('cxl-dcd.sh') tests = [ [ 'libndctl', libndctl, 'ndctl' ], @@ -194,6 +195,7 @@ tests = [ [ 'cxl-destroy-region.sh', cxl_destroy_region, 'cxl' ], [ 'cxl-qos-class.sh', cxl_qos_class, 'cxl' ], [ 'cxl-poison.sh', cxl_poison, 'cxl' ], + [ 'cxl-dcd.sh', cxl_dcd, 'cxl' ], ] if get_option('destructive').enabled()