From patchwork Sun Dec 15 02:58:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908635 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C20748C07 for ; Sun, 15 Dec 2024 02:58:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231516; cv=none; b=P2SRAK+Qh/jw8FnFq9edQUTmaOO531H6F/D79jR2rTdH32PXdMVMqhkOceNMDgeXf6kd+9wgcZtpYIXgFg98/GRmF8cc5DWJyesg23PP38/hIpL7O8AWMxNzoyLpY+Vf9OVNFXNj9hULsPunrHroxlYBOBAsnMKg13PCmftgM+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231516; c=relaxed/simple; bh=2kDR2F9vKHIh6gm7iGCw+sZX6KqNubTsGoJ+dHRxaag=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MT45+D4hhbHJ+mjMrOFllSoLE6UW0AI2HZiG8u2T1wrrvmDd1s80tz3ULmZ+4ykfW28qSGK5SAMV21juAnrcBQMD9G/FjQepFMvVg/8l2PW0zqV7izKH5o5HDhtbHODGfgFatsh6Ti/1SlG6Iq7Jkpy+29mx4fUc6OJLEfrbqkE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=cQTcI6/C; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="cQTcI6/C" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231514; x=1765767514; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=2kDR2F9vKHIh6gm7iGCw+sZX6KqNubTsGoJ+dHRxaag=; b=cQTcI6/C845BNWTC2TD7w/FgtcXFtz6BQoqxs7kFM69fvHKKxtmnF9yo aFIQUtC9EDONeWWPRo9jICxLj4iwCh7LlEd3LdJcdvWamO6zKt6R+dYfV hY6XYljHEPisolaysURz57f5Z0U+omYp+lHy4wdKBl1i6bPGtPZuJibGJ fCFGIm3E9Uqggz6Lrj7hDvUcEgNJ33VlvqMR9kxuyEsftLUxbmJlLc6ow oxBB/Haet6unkAEmLNtJaw7rZAEgsOE3QH04nm2JVhG0nBcMnDw+ExVaK BuZemRDGQOcZ8G56x5f3BtH8usKVnoVKJpZtBB3WYcSUUrBL6gV9NhjEK Q==; X-CSE-ConnectionGUID: jBes6lbKSByXY57WbK7OIw== X-CSE-MsgGUID: BzVaiK1FRhGi1bTwpsPb/A== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="52166935" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="52166935" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:34 -0800 X-CSE-ConnectionGUID: 7YNHhoA+RLKyTMM7ySgHZA== X-CSE-MsgGUID: b6IZyFoKRcKKsitReDdRLA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="97309308" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by orviesa007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:33 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:28 -0600 Subject: [ndctl PATCH v4 1/9] ndctl/cxl-events: Don't fail test until event counts are reported Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-1-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=1494; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=2kDR2F9vKHIh6gm7iGCw+sZX6KqNubTsGoJ+dHRxaag=; b=wb8HxjQsB/wHATb4wKKaGox+Y7O4ht4Ig3/yZ7ougZYUDZo1OaOVD8WveV75dnUfTT119qk3l ccZot/Z/udmDkI1iPEXmIRUGQnGFq9/ergipHvJqsLCJInrOehV98N1 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= In testing DCD event modifications a failed cxl-event test lacked details on the event counts. This was because the greps were failing the test rather than the check against the counts. Suppress the grep failure and rely on event count checks for pass/fail of the test. Reviewed-by: Dave Jiang Signed-off-by: Ira Weiny --- test/cxl-events.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/cxl-events.sh b/test/cxl-events.sh index ff4f3fdff1d8f6fd80f093126a27bf14b52d167f..c216d6aa9148c938a649cb22656127b3df440039 100644 --- a/test/cxl-events.sh +++ b/test/cxl-events.sh @@ -71,10 +71,10 @@ echo 0 > /sys/kernel/tracing/tracing_on echo "TEST: Events seen" trace_out=$(cat /sys/kernel/tracing/trace) -num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}") -num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}") -num_failure=$(grep -c "log=Failure" <<< "${trace_out}") -num_info=$(grep -c "log=Informational" <<< "${trace_out}") +num_overflow=$(grep -c "cxl_overflow" <<< "${trace_out}" || true) +num_fatal=$(grep -c "log=Fatal" <<< "${trace_out}" || true) +num_failure=$(grep -c "log=Failure" <<< "${trace_out}" || true) +num_info=$(grep -c "log=Informational" <<< "${trace_out}" || true) echo " LOG (Expected) : (Found)" echo " overflow ($num_overflow_expected) : $num_overflow" echo " Fatal ($num_fatal_expected) : $num_fatal" From patchwork Sun Dec 15 02:58:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908636 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A54F49454 for ; Sun, 15 Dec 2024 02:58:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231518; cv=none; b=sbPB3Vk+f4zNJa+siFCxA0tm5ahF+Mjce9XKKiJdunARXlB/dTPlsGqPZ4R6ACI/1abRrYRLNpDPoPO3K0kL3BwBB95/PtgYHboQqRNlsOz9E6AQrT2XIuocZr1UPi+GAi1SQcdnNK9Gao+VoH36pbG104qPM8kjlTHMxhp76I4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231518; c=relaxed/simple; bh=KEzX8VvlIPq+LG+kp76tQaRhTCaEVv6DrTm3JB2Q+Z4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rmR3XGSCNdw/uvXDSDiv9yXq1JhvpbZj/KkBDQeQPR47ptFeGfPlEYWdnYCVkv3HRJe3y58GrHBbQRLdy+V8SoOAo8+ULAHroznp+riFtHgf+sMfAOGas6qpJlrDz0A3mg1WZ30WO5G7L0KG+OVe8LplUMcgyLWv/RmHTF5FDo4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bMKG5VfU; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bMKG5VfU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231516; x=1765767516; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=KEzX8VvlIPq+LG+kp76tQaRhTCaEVv6DrTm3JB2Q+Z4=; b=bMKG5VfUE0qMmFrfERkdxSD76AfcCWXcuij19ZXAd0xWjusXFGixUQyZ Xf+SBJ/bh8SSjmj5Zb45BKW2511H6iZRGDfJrO7ddDrLKFu5PdC8PzQNl kb+X4aYQT/O85pOM0e0qjuG1U7QRVmLGScdBLeqguuegQjY/eNSIoKQoz 9OOXl6t7tFn4chHqj7pkDZikhyuooykLYE1mds20B92AZrbRdEpEttNc7 KnTBFa4tsOBQzr226QGvVSXNAiEi6u1CkMDe8HvBpV8BO9HR+Ur+ZUEop IHvKtElLcOAD+Szppb40ogoVG+fuH4eopdDK1qf11/B0eB7g7420L/c0v w==; X-CSE-ConnectionGUID: ImyfI/9fQfaWeQjqiMbVIw== X-CSE-MsgGUID: m9mM7e3DQm+8HCZP4MQnZQ== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="52166940" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="52166940" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:36 -0800 X-CSE-ConnectionGUID: mR9ufJZIRd2HwnK0VpkU/w== X-CSE-MsgGUID: 2BOb2w7BQdGFE+MqSmU9uQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="97309313" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by orviesa007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:35 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:29 -0600 Subject: [ndctl PATCH v4 2/9] ndctl/cxl/region: Report max size for region creation Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-2-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=1059; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=KEzX8VvlIPq+LG+kp76tQaRhTCaEVv6DrTm3JB2Q+Z4=; b=xySbMWePvvsY6iygYZnDjPocl8XLdj88xQk9YHrTpa4ZhG7wi+ScFSk5yCdvyNC2N9wInJDyX xMjGHWQXC9pAlplh60TUKssaPEmDby34oU0i345tIzt6cmS98oCSDmY X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= When creating a region if the size exceeds the max an error is printed. However, the max available space is not reported which makes it harder to determine what is wrong. Add the max size available to the output error. Reviewed-by: Dave Jiang Reviewed-by: Fan Ni Signed-off-by: Ira Weiny --- cxl/region.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cxl/region.c b/cxl/region.c index 96aa5931d2281c7577679b7f6165218964fa0425..207cf2d003148992255c715f286bc0f38de2ca84 100644 --- a/cxl/region.c +++ b/cxl/region.c @@ -677,8 +677,8 @@ static int create_region(struct cxl_ctx *ctx, int *count, } if (!default_size && size > max_extent) { log_err(&rl, - "%s: region size %#lx exceeds max available space\n", - cxl_decoder_get_devname(p->root_decoder), size); + "%s: region size %#lx exceeds max available space (%#lx)\n", + cxl_decoder_get_devname(p->root_decoder), size, max_extent); return -ENOSPC; } From patchwork Sun Dec 15 02:58:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908637 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5302BBA4A for ; Sun, 15 Dec 2024 02:58:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231520; cv=none; b=ZV9gWTcM2IOkW6HjFVDwpSregWHCG8QT+3cGTC8dR8rWzObUrSo9a1ocxhQ4yEc5cAIwfPz7EG71h25IJY6IYoKLfTQwi4Oi7NyJIleiiFM45VXEU0WKX3iSlFGJ7xNnxy55jyTftLEhnW/4xrJRoIe59s66dJIIekqcEg1bFSA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231520; c=relaxed/simple; bh=gYuIf8iFWgCstPhZQ8M3bKFjkVl0enZVlIXva/K1lgw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PlNh8Ag7l8EZybzRGswi+iKB0+TPDxXg6ldglhIZm8APMkUgRNSHVvzNs9Pu8mTxtNg++yyFkgZT1+vuzWhcBO7FMKCC6LCfxvVZZdwAJzJPffSr2TCjEcQamCQJLNm+r0p1Q2HPIJQ75nFoA6WOWwMD7I97wc56NLmbABsqpVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fYnA8/BT; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fYnA8/BT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231518; x=1765767518; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=gYuIf8iFWgCstPhZQ8M3bKFjkVl0enZVlIXva/K1lgw=; b=fYnA8/BTJmb+oEtvtyKi568YJWwuzzdgbg7qDrB0Y9kNapDeihxd7UjO 8KoqXsQiotOD4HT/pPEhRXk1aAr0YTD3/fmF0ybSv3sDDMe4RS7KBGu69 dmF9JWVp/fMztYG2aQhChnL5/gXdL7u3gWu2DLwUqxvgW3RXFyf+nXDy4 cDo7mOsqiwcQ6C2n2IrJUCJi8+s51HT9NAd7HvlXPsFXk4mG0gx+qXoNf OO+eIwQByfzrKXJlwRLKuISjYsEmiEAE3dMlailjEHuQp1nldl4XEKhL6 lxC3YZvUyCy8ZX+qiSHMwZvtmWY++itAaX2cFHgPWRMAOQTc69haWocKA w==; X-CSE-ConnectionGUID: ERZxMqfaTRKOyyo9+X80Fg== X-CSE-MsgGUID: kowNGL/5RxCT5mFuTxgP4w== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="52166944" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="52166944" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:37 -0800 X-CSE-ConnectionGUID: G2GMZz7pRX26LN3k9VA3rg== X-CSE-MsgGUID: ol4njKxwSbSLhjzjlY9SdQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="97309317" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by orviesa007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:37 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:30 -0600 Subject: [ndctl PATCH v4 3/9] libcxl: Separate region mode from decoder mode Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-3-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=9543; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=gYuIf8iFWgCstPhZQ8M3bKFjkVl0enZVlIXva/K1lgw=; b=uaEAmxvbsiQ8pdn2I8soQQZPn1ieVv654yQn9FKokrXqrkQG7zU3Q6GkXQ9/3PlWjEs4p0Wxu KPE7H1NsQGAAgF6nPvqZACG2cdS+PZwErOHawvIvEPIVOmG4MX5V60/ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= With the introduction of DCD, region mode and decoder mode no longer remain a 1:1 relation. An interleaved region may be made up of Dynamic Capacity partitions with different indexes on each of the target devices. Introduce a new region mode enumeration and access function to libcxl. To maintain compatibility with existing software the region mode values are defined the same as the current decoder mode. In addition cxl_region_get_mode() is retained. Signed-off-by: Ira Weiny --- Documentation/cxl/lib/libcxl.txt | 20 ++++++++++++++++++-- cxl/lib/libcxl.c | 25 +++++++++++++++---------- cxl/lib/libcxl.sym | 5 +++++ cxl/lib/private.h | 2 +- cxl/libcxl.h | 35 +++++++++++++++++++++++++++++++++++ 5 files changed, 74 insertions(+), 13 deletions(-) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 40598a08b9f4840f79e3ab43f62f412d8b2136ed..d5c3558aacecb08d7f5754fdcc77d6e743560601 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -553,11 +553,20 @@ struct cxl_region *cxl_region_get_next(struct cxl_region *region); ===== REGION: Attributes ---- +enum cxl_region_mode { + CXL_REGION_MODE_NONE = CXL_DECODER_MODE_NONE, + CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, + CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, + CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, +}; +const char *cxl_region_mode_name(enum cxl_region_mode mode); +enum cxl_region_mode cxl_region_mode_from_ident(const char *ident); +enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region); + int cxl_region_get_id(struct cxl_region *region); const char *cxl_region_get_devname(struct cxl_region *region); void cxl_region_get_uuid(struct cxl_region *region, uuid_t uu); unsigned long long cxl_region_get_size(struct cxl_region *region); -enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region); unsigned long long cxl_region_get_resource(struct cxl_region *region); unsigned int cxl_region_get_interleave_ways(struct cxl_region *region); unsigned int cxl_region_get_interleave_granularity(struct cxl_region *region); @@ -576,8 +585,12 @@ int cxl_region_clear_all_targets(struct cxl_region *region); int cxl_region_decode_commit(struct cxl_region *region); int cxl_region_decode_reset(struct cxl_region *region); struct daxctl_region *cxl_region_get_daxctl_region(struct cxl_region *region); ----- +DEPRECATED: + +enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region); + +---- A region's resource attribute is the Host Physical Address at which the region's address space starts. The region's address space is a subset of the parent root decoder's address space. @@ -601,6 +614,9 @@ where its properties can be interrogated by daxctl. The helper cxl_region_get_daxctl_region() returns an 'struct daxctl_region *' that can be used with other libdaxctl APIs. +Regions now have a mode distinct from decoders. cxl_region_get_mode() is +deprecated in favor of cxl_region_get_region_mode(). + include::../../copyright.txt[] SEE ALSO diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 63aa4ef3acdc2fb3c4ec6c13be5feb802e817d0d..35a40091e8f5813c1b3ef2ffb931c9ec584b02ad 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -431,10 +431,10 @@ CXL_EXPORT bool cxl_region_qos_class_mismatch(struct cxl_region *region) if (!memdev) continue; - if (region->mode == CXL_DECODER_MODE_RAM) { + if (region->mode == CXL_REGION_MODE_RAM) { if (root_decoder->qos_class != memdev->ram_qos_class) return true; - } else if (region->mode == CXL_DECODER_MODE_PMEM) { + } else if (region->mode == CXL_REGION_MODE_PMEM) { if (root_decoder->qos_class != memdev->pmem_qos_class) return true; } @@ -619,9 +619,9 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base) sprintf(path, "%s/mode", cxlregion_base); if (sysfs_read_attr(ctx, path, buf) < 0) - region->mode = CXL_DECODER_MODE_NONE; + region->mode = CXL_REGION_MODE_NONE; else - region->mode = cxl_decoder_mode_from_ident(buf); + region->mode = cxl_region_mode_from_ident(buf); sprintf(path, "%s/modalias", cxlregion_base); if (sysfs_read_attr(ctx, path, buf) == 0) @@ -748,11 +748,16 @@ CXL_EXPORT unsigned long long cxl_region_get_resource(struct cxl_region *region) return region->start; } -CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region) +CXL_EXPORT enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region) { return region->mode; } +CXL_EXPORT enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region) +{ + return (enum cxl_decoder_mode)cxl_region_get_region_mode(region); +} + CXL_EXPORT unsigned int cxl_region_get_interleave_ways(struct cxl_region *region) { @@ -2700,7 +2705,7 @@ cxl_decoder_get_region(struct cxl_decoder *decoder) } static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder, - enum cxl_decoder_mode mode) + enum cxl_region_mode mode) { struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); char *path = decoder->dev_buf; @@ -2708,9 +2713,9 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder, struct cxl_region *region; int rc; - if (mode == CXL_DECODER_MODE_PMEM) + if (mode == CXL_REGION_MODE_PMEM) sprintf(path, "%s/create_pmem_region", decoder->dev_path); - else if (mode == CXL_DECODER_MODE_RAM) + else if (mode == CXL_REGION_MODE_RAM) sprintf(path, "%s/create_ram_region", decoder->dev_path); rc = sysfs_read_attr(ctx, path, buf); @@ -2754,13 +2759,13 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder, CXL_EXPORT struct cxl_region * cxl_decoder_create_pmem_region(struct cxl_decoder *decoder) { - return cxl_decoder_create_region(decoder, CXL_DECODER_MODE_PMEM); + return cxl_decoder_create_region(decoder, CXL_REGION_MODE_PMEM); } CXL_EXPORT struct cxl_region * cxl_decoder_create_ram_region(struct cxl_decoder *decoder) { - return cxl_decoder_create_region(decoder, CXL_DECODER_MODE_RAM); + return cxl_decoder_create_region(decoder, CXL_REGION_MODE_RAM); } CXL_EXPORT int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder) diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 0c155a40ad4765106f0eab1745281d462af782fe..17a660f508ad1e053af2992824535ccf7ce877b2 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -288,3 +288,8 @@ global: cxl_memdev_trigger_poison_list; cxl_region_trigger_poison_list; } LIBCXL_7; + +LIBCXL_9 { +global: + cxl_region_get_region_mode; +} LIBECXL_8; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index b6cd910e93359b53cac34427acfe84c7abcb78b0..0f45be89b6a00477d13fb6d7f1906213a3073c48 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -171,7 +171,7 @@ struct cxl_region { unsigned int interleave_ways; unsigned int interleave_granularity; enum cxl_decode_state decode_state; - enum cxl_decoder_mode mode; + enum cxl_region_mode mode; struct daxctl_region *dax_region; struct kmod_module *module; struct list_head mappings; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 0a5fd0e13cc24e0032d4a83d780278fbe0038d32..06b87a0924faafec6c80eca83ea7551d4e117256 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -303,6 +303,39 @@ int cxl_memdev_is_enabled(struct cxl_memdev *memdev); for (endpoint = cxl_endpoint_get_first(port); endpoint != NULL; \ endpoint = cxl_endpoint_get_next(endpoint)) +enum cxl_region_mode { + CXL_REGION_MODE_NONE = CXL_DECODER_MODE_NONE, + CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, + CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, + CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, +}; + +static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) +{ + static const char *names[] = { + [CXL_REGION_MODE_NONE] = "none", + [CXL_REGION_MODE_MIXED] = "mixed", + [CXL_REGION_MODE_PMEM] = "pmem", + [CXL_REGION_MODE_RAM] = "ram", + }; + + if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM) + mode = CXL_REGION_MODE_NONE; + return names[mode]; +} + +static inline enum cxl_region_mode +cxl_region_mode_from_ident(const char *ident) +{ + if (strcmp(ident, "ram") == 0) + return CXL_REGION_MODE_RAM; + else if (strcmp(ident, "volatile") == 0) + return CXL_REGION_MODE_RAM; + else if (strcmp(ident, "pmem") == 0) + return CXL_REGION_MODE_PMEM; + return CXL_REGION_MODE_NONE; +} + struct cxl_region; struct cxl_region *cxl_region_get_first(struct cxl_decoder *decoder); struct cxl_region *cxl_region_get_next(struct cxl_region *region); @@ -318,6 +351,8 @@ const char *cxl_region_get_devname(struct cxl_region *region); void cxl_region_get_uuid(struct cxl_region *region, uuid_t uu); unsigned long long cxl_region_get_size(struct cxl_region *region); unsigned long long cxl_region_get_resource(struct cxl_region *region); +enum cxl_region_mode cxl_region_get_region_mode(struct cxl_region *region); +/* Deprecated: use cxl_region_get_region_mode() */ enum cxl_decoder_mode cxl_region_get_mode(struct cxl_region *region); unsigned int cxl_region_get_interleave_ways(struct cxl_region *region); unsigned int cxl_region_get_interleave_granularity(struct cxl_region *region); From patchwork Sun Dec 15 02:58:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908638 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEBD89454 for ; Sun, 15 Dec 2024 02:58:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231521; cv=none; b=kUjOGp2xptqCej84fUTsaLbhrMdRU/moELQlONlwUdCbgv8+7jPKeQdYvOffj88OfC7qUCefa77YPxst1mP32zNWsZBj5eG5z5nZrNbCZOnVvRcFpqXs3sswlHhLdWbl49BMRFMYngxNKK9SX0GXHazhU+RdT77b+vao8G+EPKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231521; c=relaxed/simple; bh=xwh5QIFiEZatH/op9hzPR9Y0hPludvE5/P/bczp99iw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=W+FWQjToEsZ+3sknEdLj2ZvCDsK1YIhUsZb6LNoqURYYkQJ29T9lcFyuIvzjAP7fHxqwzZ2me1IdFjat5EeR3crVA9Y6D/EREjbQKG77hhXEvpkTxEUNukGejbxOkzK0TyHGtCSh1Dd+Twu4LGWRAaCZp77GLN2mERpwNHCf4uw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aGXl+tR1; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aGXl+tR1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231519; x=1765767519; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=xwh5QIFiEZatH/op9hzPR9Y0hPludvE5/P/bczp99iw=; b=aGXl+tR1J58KlrcB5UjC7JtlDPYbcpXmi61eqVh9eJs/VIvgU4YiKQ/g tcE5PNwq6VsizegEWVWtZEHssKVNehIT1CA7dJvxHa8ZLx5cCz7f3YT28 3FpU3y+Krm1xITVtqkZNjQSFUTMEkk2H4vWYhr5f1x33G8mRi14/kfumi H9Q511UJLCJpkhlQrDsCb8tXXxThAwP4vLQZ9kuQdK0BIbvrz+NLucHRL zXpM//42d+ut8nYiJZepc2QfB4oP88RV11mx3UZB1eR0nTSX/BbUHp2Yo ClKOFzLa+2RZ4z8mSO3X67gUHdbNe+VZRdf0dRN3kWyNYe5ysHvBQMjjR w==; X-CSE-ConnectionGUID: gm70fCDASoOHWvYoeOFRCw== X-CSE-MsgGUID: hMWg6ZMeSrOtc5ff9rs+HA== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="52166949" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="52166949" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:39 -0800 X-CSE-ConnectionGUID: hwqGEvIRQzq1o+uKMnYgQw== X-CSE-MsgGUID: QZXMFqrVQ6GatnSHehobdA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="97309321" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by orviesa007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:38 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:31 -0600 Subject: [ndctl PATCH v4 4/9] cxl/region: Use new region mode in cxl-cli Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-4-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=6754; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=xwh5QIFiEZatH/op9hzPR9Y0hPludvE5/P/bczp99iw=; b=d4Qpf8+1pwVcJ9sb4yT8SJ4sNrtVTbcbFWCjtKIXimjPx9z2maO8vibaZvbf8vnRwEhrzxiVN H7Qy+7J8A1lCtfFC0pIh0KZDAdvXxmcJaO2RKBIudcjyNgN+G5ZeUja X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= With the introduction of DCD, region mode and decoder mode no longer remain a 1:1 relation. An interleaved region may be made up of Dynamic Capacity partitions with different indexes on each of the target devices. Modify cxl-cli to use the new region mode interface from libcxl. Modify parameter processing and variable name changes for clarity in the future. Functionality remains the same. Signed-off-by: Ira Weiny --- cxl/json.c | 6 +++--- cxl/region.c | 50 ++++++++++++++++++++++++++++++++------------------ 2 files changed, 35 insertions(+), 21 deletions(-) diff --git a/cxl/json.c b/cxl/json.c index 5066d3bed13f8fcc36ab8f0ea127685c246d94d7..dcd3cc28393faf7e8adf299a857531ecdeaac50a 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -1147,7 +1147,7 @@ void util_cxl_mappings_append_json(struct json_object *jregion, struct json_object *util_cxl_region_to_json(struct cxl_region *region, unsigned long flags) { - enum cxl_decoder_mode mode = cxl_region_get_mode(region); + enum cxl_region_mode mode = cxl_region_get_region_mode(region); const char *devname = cxl_region_get_devname(region); struct json_object *jregion, *jobj; u64 val; @@ -1174,8 +1174,8 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, json_object_object_add(jregion, "size", jobj); } - if (mode != CXL_DECODER_MODE_NONE) { - jobj = json_object_new_string(cxl_decoder_mode_name(mode)); + if (mode != CXL_REGION_MODE_NONE) { + jobj = json_object_new_string(cxl_region_mode_name(mode)); if (jobj) json_object_object_add(jregion, "type", jobj); } diff --git a/cxl/region.c b/cxl/region.c index 207cf2d003148992255c715f286bc0f38de2ca84..527bd6708b162815068a95ddb360fce3914347de 100644 --- a/cxl/region.c +++ b/cxl/region.c @@ -49,7 +49,8 @@ struct parsed_params { int argc; const char **argv; struct cxl_decoder *root_decoder; - enum cxl_decoder_mode mode; + enum cxl_decoder_mode decoder_mode; + enum cxl_region_mode region_mode; bool enforce_qos; }; @@ -301,19 +302,28 @@ static int parse_create_options(struct cxl_ctx *ctx, int count, return -ENXIO; p->num_memdevs = json_object_array_length(p->memdevs); + p->region_mode = CXL_REGION_MODE_NONE; if (param.type) { - p->mode = cxl_decoder_mode_from_ident(param.type); - if (p->mode == CXL_DECODER_MODE_RAM && param.uuid) { + p->region_mode = cxl_region_mode_from_ident(param.type); + if (p->region_mode == CXL_REGION_MODE_RAM && param.uuid) { log_err(&rl, "can't set UUID for ram / volatile regions"); goto err; } - if (p->mode == CXL_DECODER_MODE_NONE) { + if (p->region_mode == CXL_REGION_MODE_NONE) { log_err(&rl, "unsupported type: %s\n", param.type); goto err; } - } else { - p->mode = CXL_DECODER_MODE_PMEM; + } + + switch (p->region_mode) { + case CXL_REGION_MODE_RAM: + p->decoder_mode = CXL_DECODER_MODE_RAM; + break; + case CXL_REGION_MODE_PMEM: + default: + p->decoder_mode = CXL_DECODER_MODE_PMEM; + break; } if (param.size) { @@ -410,7 +420,7 @@ static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p) struct cxl_memdev *memdev = json_object_get_userdata(jobj); u64 size = 0; - switch(p->mode) { + switch(p->decoder_mode) { case CXL_DECODER_MODE_RAM: size = cxl_memdev_get_ram_size(memdev); break; @@ -446,7 +456,7 @@ static int create_region_validate_qos_class(struct parsed_params *p) json_object_array_get_idx(p->memdevs, i); struct cxl_memdev *memdev = json_object_get_userdata(jobj); - if (p->mode == CXL_DECODER_MODE_RAM) + if (p->decoder_mode == CXL_DECODER_MODE_RAM) qos_class = cxl_memdev_get_ram_qos_class(memdev); else qos_class = cxl_memdev_get_pmem_qos_class(memdev); @@ -475,7 +485,7 @@ static int validate_decoder(struct cxl_decoder *decoder, const char *devname = cxl_decoder_get_devname(decoder); int rc; - switch(p->mode) { + switch(p->decoder_mode) { case CXL_DECODER_MODE_RAM: if (!cxl_decoder_is_volatile_capable(decoder)) { log_err(&rl, "%s is not volatile capable\n", devname); @@ -512,10 +522,14 @@ static void set_type_from_decoder(struct cxl_ctx *ctx, struct parsed_params *p) * default to pmem if both types are set, otherwise the single * capability dominates. */ - if (cxl_decoder_is_volatile_capable(p->root_decoder)) - p->mode = CXL_DECODER_MODE_RAM; - if (cxl_decoder_is_pmem_capable(p->root_decoder)) - p->mode = CXL_DECODER_MODE_PMEM; + if (cxl_decoder_is_volatile_capable(p->root_decoder)) { + p->decoder_mode = CXL_DECODER_MODE_RAM; + p->region_mode = CXL_REGION_MODE_RAM; + } + if (cxl_decoder_is_pmem_capable(p->root_decoder)) { + p->decoder_mode = CXL_DECODER_MODE_PMEM; + p->region_mode = CXL_REGION_MODE_PMEM; + } } static int create_region_validate_config(struct cxl_ctx *ctx, @@ -685,14 +699,14 @@ static int create_region(struct cxl_ctx *ctx, int *count, if (size > max_extent) size = ALIGN_DOWN(max_extent, SZ_256M * p->ways); - if (p->mode == CXL_DECODER_MODE_PMEM) { + if (p->region_mode == CXL_REGION_MODE_PMEM) { region = cxl_decoder_create_pmem_region(p->root_decoder); if (!region) { log_err(&rl, "failed to create region under %s\n", param.root_decoder); return -ENXIO; } - } else if (p->mode == CXL_DECODER_MODE_RAM) { + } else if (p->region_mode == CXL_REGION_MODE_RAM) { region = cxl_decoder_create_ram_region(p->root_decoder); if (!region) { log_err(&rl, "failed to create region under %s\n", @@ -714,7 +728,7 @@ static int create_region(struct cxl_ctx *ctx, int *count, try(cxl_region, set_interleave_granularity, region, granularity); try(cxl_region, set_interleave_ways, region, p->ways); - if (p->mode == CXL_DECODER_MODE_PMEM) { + if (p->region_mode == CXL_REGION_MODE_PMEM) { if (!param.uuid) uuid_generate(p->uuid); try(cxl_region, set_uuid, region, p->uuid); @@ -732,14 +746,14 @@ static int create_region(struct cxl_ctx *ctx, int *count, rc = -ENXIO; goto out; } - if (cxl_decoder_get_mode(ep_decoder) != p->mode) { + if (cxl_decoder_get_mode(ep_decoder) != p->decoder_mode) { /* * The cxl_memdev_find_decoder() helper returns a free * decoder whose size has been checked for 0. * Thus it is safe to change the mode here if needed. */ try(cxl_decoder, set_dpa_size, ep_decoder, 0); - try(cxl_decoder, set_mode, ep_decoder, p->mode); + try(cxl_decoder, set_mode, ep_decoder, p->decoder_mode); } try(cxl_decoder, set_dpa_size, ep_decoder, size/p->ways); rc = cxl_region_set_target(region, i, ep_decoder); From patchwork Sun Dec 15 02:58:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908639 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91854BE65 for ; Sun, 15 Dec 2024 02:58:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231524; cv=none; b=cczkB18Krd+b9/y2TaD53qaVtHdt0ba+wED7jF7e+8F0kwKH484FGh66Gf+qPh9eunUBDwawz2EMEoMUjsDjdFWEmlHXYfeJKeW+xQ31J/+7EnRqPoPwNa+C4pK/gOHpwCFJd+ywW74n6k7+wfjnhTeyQhgGNxetSxHu0j9j+rM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231524; c=relaxed/simple; bh=DxlFe2XpImRBjLSwTUNOMwQ0foFv/lQ7Ye4CyP2kZ6k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nDifOi3edEtXlGmlN5TCIaH4UuxYdcu/WfaieCUYC6aKCFLjDupPtUsVD5oxXxk5XZ0aRw65Cp7koWyhtzspwpZ/QPu2XiV01Uj6LAohGP+icggaagjiYX/TaP2x+7suBx6+mpgj1ACAedgtYunkS3ORR3eY4cL49CAnUEL8Ays= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ISKaNn3x; arc=none smtp.client-ip=198.175.65.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ISKaNn3x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231523; x=1765767523; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=DxlFe2XpImRBjLSwTUNOMwQ0foFv/lQ7Ye4CyP2kZ6k=; b=ISKaNn3xGg6NCbSgeLsiT9XggSMfJh61qMgzLcqQCHAFOuB0xDqBekSq M1WN2zxDVpFlV4RPs49k/dGKV/gHmTjS6hOFGS78i7RT1gXSBwTkiV0cz /x9IVPJzEWCp9RsfBzlpLaUaDhZiW92gKaVcGjhahoma5wu+FOXOOsp0k 8eWTmkkuhov6iGjYXyxV6gug66WhcHTMysYtvXGT8d6oVcJ9TmquD4ssS qjcjF4XCpSBpbJFeD/MTfebwE0pvsh0hTuL9RoHg9HemzuCcqcYS/eMya Ro/SyuIEhAp4CRUj5i30st/TFRRULHmbh2AF8ANkEtGiJlEDxF5c3CgBw Q==; X-CSE-ConnectionGUID: rjxQTcaXT7GZAHXj7eBMBA== X-CSE-MsgGUID: mmEY1X05Sra+OVOKTakSZQ== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="38424555" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="38424555" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:43 -0800 X-CSE-ConnectionGUID: 1csNoKsxTHyXzKrEa+Jlwg== X-CSE-MsgGUID: LrB6idgxTRSARtEqSW7K2A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="101462502" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:41 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:32 -0600 Subject: [ndctl PATCH v4 5/9] libcxl: Add Dynamic Capacity region support Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-5-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=16050; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=DxlFe2XpImRBjLSwTUNOMwQ0foFv/lQ7Ye4CyP2kZ6k=; b=1wzGXCqJXsMSQ2eDvc8SH7LN746zyW9mAP1JnT13K604Y8/se5KjQQhx32u/6c6VgT2MoVqRK 7td0RVi3kIYDpicv++4c9zScTf5ZHd10J03LNo5R+LxCmjW+bECLblw X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be sparse and defined as dynamic capacity (dc). Add support for DCD devices and regions to libcxl. Add documentation for the new interfaces. Based on an original patch from Navneet Singh. Signed-off-by: Sushant1 Kumar Co-developed-by: Ira Weiny Signed-off-by: Ira Weiny --- Documentation/cxl/lib/libcxl.txt | 17 ++++++- cxl/lib/libcxl.c | 98 ++++++++++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 3 ++ cxl/lib/private.h | 4 ++ cxl/libcxl.h | 52 ++++++++++++++++++++- 5 files changed, 171 insertions(+), 3 deletions(-) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index d5c3558aacecb08d7f5754fdcc77d6e743560601..abca08fc81e6e84d176facafad6decae2f875880 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -68,6 +68,7 @@ int cxl_memdev_get_major(struct cxl_memdev *memdev); int cxl_memdev_get_minor(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev); +unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, enum cxl_decoder_mode mode); const char *cxl_memdev_get_firmware_version(struct cxl_memdev *memdev); size_t cxl_memdev_get_label_size(struct cxl_memdev *memdev); int cxl_memdev_nvdimm_bridge_active(struct cxl_memdev *memdev); @@ -422,11 +423,23 @@ enum cxl_decoder_mode { CXL_DECODER_MODE_MIXED, CXL_DECODER_MODE_PMEM, CXL_DECODER_MODE_RAM, + CXL_DECODER_MODE_DC0, + CXL_DECODER_MODE_DC1, + CXL_DECODER_MODE_DC2, + CXL_DECODER_MODE_DC3, + CXL_DECODER_MODE_DC4, + CXL_DECODER_MODE_DC5, + CXL_DECODER_MODE_DC6, + CXL_DECODER_MODE_DC7, }; enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); +const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode); +enum cxl_decoder_mode cxl_decoder_mode_from_ident(const char *ident); +bool cxl_decoder_mode_is_dc(enum cxl_decoder_mode mode); int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); +bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder); @@ -558,6 +571,7 @@ enum cxl_region_mode { CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, + CXL_REGION_MODE_DC, }; const char *cxl_region_mode_name(enum cxl_region_mode mode); enum cxl_region_mode cxl_region_mode_from_ident(const char *ident); @@ -615,7 +629,8 @@ cxl_region_get_daxctl_region() returns an 'struct daxctl_region *' that can be used with other libdaxctl APIs. Regions now have a mode distinct from decoders. cxl_region_get_mode() is -deprecated in favor of cxl_region_get_region_mode(). +deprecated in favor of cxl_region_get_region_mode(). Dynamic capacity regions +require the use of cxl_region_get_region_mode(). include::../../copyright.txt[] diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 35a40091e8f5813c1b3ef2ffb931c9ec584b02ad..df250db9dbacb2f0f34e8a592ce194159584fe4f 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -1304,6 +1304,19 @@ static void *add_cxl_memdev(void *parent, int id, const char *cxlmem_base) else memdev->ram_qos_class = atoi(buf); + memdev->dc_partition_count = 0; + for (int partition = 0; partition < MAX_NUM_DC_REGIONS; partition++) { + sprintf(path, "%s/dc%d/size", cxlmem_base, partition); + if (sysfs_read_attr(ctx, path, buf) < 0) + continue; + memdev->dc_size[partition] = strtoull(buf, NULL, 0); + memdev->dc_partition_count++; + sprintf(path, "%s/dc%d/qos_class", cxlmem_base, partition); + if (sysfs_read_attr(ctx, path, buf) < 0) + continue; + memdev->dc_qos_class[partition] = strtoull(buf, NULL, 0); + } + sprintf(path, "%s/payload_max", cxlmem_base); if (sysfs_read_attr(ctx, path, buf) == 0) { memdev->payload_max = strtoull(buf, NULL, 0); @@ -1540,6 +1553,23 @@ CXL_EXPORT int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev) return memdev->ram_qos_class; } +static int cxl_decoder_dc_mode_to_index(enum cxl_decoder_mode mode) +{ + if (mode < CXL_DECODER_MODE_DC0 || mode > CXL_DECODER_MODE_DC7) + return -EINVAL; + return mode - CXL_DECODER_MODE_DC0; +} + +CXL_EXPORT unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, + enum cxl_decoder_mode mode) +{ + int index = cxl_decoder_dc_mode_to_index(mode); + + if (index < 0) + return 0; + return memdev->dc_size[index]; +} + CXL_EXPORT const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev) { return memdev->firmware_version; @@ -2275,6 +2305,22 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) decoder->mode = CXL_DECODER_MODE_RAM; else if (strcmp(buf, "pmem") == 0) decoder->mode = CXL_DECODER_MODE_PMEM; + else if (strcmp(buf, "dc0") == 0) + decoder->mode = CXL_DECODER_MODE_DC0; + else if (strcmp(buf, "dc1") == 0) + decoder->mode = CXL_DECODER_MODE_DC1; + else if (strcmp(buf, "dc2") == 0) + decoder->mode = CXL_DECODER_MODE_DC2; + else if (strcmp(buf, "dc3") == 0) + decoder->mode = CXL_DECODER_MODE_DC3; + else if (strcmp(buf, "dc4") == 0) + decoder->mode = CXL_DECODER_MODE_DC4; + else if (strcmp(buf, "dc5") == 0) + decoder->mode = CXL_DECODER_MODE_DC5; + else if (strcmp(buf, "dc6") == 0) + decoder->mode = CXL_DECODER_MODE_DC6; + else if (strcmp(buf, "dc7") == 0) + decoder->mode = CXL_DECODER_MODE_DC7; else if (strcmp(buf, "mixed") == 0) decoder->mode = CXL_DECODER_MODE_MIXED; else if (strcmp(buf, "none") == 0) @@ -2318,6 +2364,8 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) case CXL_PORT_SWITCH: decoder->pmem_capable = true; decoder->volatile_capable = true; + for (unsigned index = 0; index < MAX_NUM_DC_REGIONS; index++) + decoder->dc_capable[index] = true; decoder->mem_capable = true; decoder->accelmem_capable = true; sprintf(path, "%s/locked", cxldecoder_base); @@ -2341,6 +2389,14 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) { "cap_type2", &decoder->accelmem_capable }, { "cap_type3", &decoder->mem_capable }, { "cap_ram", &decoder->volatile_capable }, + { "cap_ram", &decoder->dc_capable[0] }, + { "cap_ram", &decoder->dc_capable[1] }, + { "cap_ram", &decoder->dc_capable[2] }, + { "cap_ram", &decoder->dc_capable[3] }, + { "cap_ram", &decoder->dc_capable[4] }, + { "cap_ram", &decoder->dc_capable[5] }, + { "cap_ram", &decoder->dc_capable[6] }, + { "cap_ram", &decoder->dc_capable[7] }, { "cap_pmem", &decoder->pmem_capable }, { "locked", &decoder->locked }, }; @@ -2592,6 +2648,30 @@ CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder, case CXL_DECODER_MODE_RAM: sprintf(buf, "ram"); break; + case CXL_DECODER_MODE_DC0: + sprintf(buf, "dc0"); + break; + case CXL_DECODER_MODE_DC1: + sprintf(buf, "dc1"); + break; + case CXL_DECODER_MODE_DC2: + sprintf(buf, "dc2"); + break; + case CXL_DECODER_MODE_DC3: + sprintf(buf, "dc3"); + break; + case CXL_DECODER_MODE_DC4: + sprintf(buf, "dc4"); + break; + case CXL_DECODER_MODE_DC5: + sprintf(buf, "dc5"); + break; + case CXL_DECODER_MODE_DC6: + sprintf(buf, "dc6"); + break; + case CXL_DECODER_MODE_DC7: + sprintf(buf, "dc7"); + break; default: err(ctx, "%s: unsupported mode: %d\n", cxl_decoder_get_devname(decoder), mode); @@ -2648,6 +2728,16 @@ CXL_EXPORT bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder) return decoder->mem_capable; } +CXL_EXPORT bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode) +{ + int index = cxl_decoder_dc_mode_to_index(mode); + + if (index < 0) + return false; + return decoder->dc_capable[index]; +} + CXL_EXPORT bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder) { return decoder->accelmem_capable; @@ -2717,6 +2807,8 @@ static struct cxl_region *cxl_decoder_create_region(struct cxl_decoder *decoder, sprintf(path, "%s/create_pmem_region", decoder->dev_path); else if (mode == CXL_REGION_MODE_RAM) sprintf(path, "%s/create_ram_region", decoder->dev_path); + else if (mode == CXL_REGION_MODE_DC) + sprintf(path, "%s/create_dc_region", decoder->dev_path); rc = sysfs_read_attr(ctx, path, buf); if (rc < 0) { @@ -2768,6 +2860,12 @@ cxl_decoder_create_ram_region(struct cxl_decoder *decoder) return cxl_decoder_create_region(decoder, CXL_REGION_MODE_RAM); } +CXL_EXPORT struct cxl_region * +cxl_decoder_create_dc_region(struct cxl_decoder *decoder) +{ + return cxl_decoder_create_region(decoder, CXL_REGION_MODE_DC); +} + CXL_EXPORT int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder) { return decoder->nr_targets; diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 17a660f508ad1e053af2992824535ccf7ce877b2..fdb227789985443a13c72751bbd42ab383db5f97 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -292,4 +292,7 @@ global: LIBCXL_9 { global: cxl_region_get_region_mode; + cxl_memdev_get_dc_size; + cxl_decoder_is_dc_capable; + cxl_decoder_create_dc_region; } LIBECXL_8; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 0f45be89b6a00477d13fb6d7f1906213a3073c48..3efa230bfb632e6c6048aadd18f799b07d4bdfd3 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -47,6 +47,9 @@ struct cxl_memdev { struct list_node list; unsigned long long pmem_size; unsigned long long ram_size; + unsigned long long dc_size[MAX_NUM_DC_REGIONS]; + unsigned long long dc_qos_class[MAX_NUM_DC_REGIONS]; + int dc_partition_count; int ram_qos_class; int pmem_qos_class; int payload_max; @@ -140,6 +143,7 @@ struct cxl_decoder { bool pmem_capable; bool volatile_capable; bool mem_capable; + bool dc_capable[MAX_NUM_DC_REGIONS]; bool accelmem_capable; bool locked; enum cxl_decoder_target_type target_type; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 06b87a0924faafec6c80eca83ea7551d4e117256..d7f8a37816f236acd71fc834eae70a7a17a2721a 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -72,6 +72,9 @@ int cxl_memdev_get_minor(struct cxl_memdev *memdev); struct cxl_ctx *cxl_memdev_get_ctx(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_pmem_size(struct cxl_memdev *memdev); unsigned long long cxl_memdev_get_ram_size(struct cxl_memdev *memdev); +enum cxl_decoder_mode; +unsigned long long cxl_memdev_get_dc_size(struct cxl_memdev *memdev, + enum cxl_decoder_mode mode); int cxl_memdev_get_pmem_qos_class(struct cxl_memdev *memdev); int cxl_memdev_get_ram_qos_class(struct cxl_memdev *memdev); const char *cxl_memdev_get_firmware_verison(struct cxl_memdev *memdev); @@ -191,11 +194,20 @@ unsigned long long cxl_decoder_get_max_available_extent(struct cxl_decoder *decoder); int cxl_root_decoder_get_qos_class(struct cxl_decoder *decoder); +#define MAX_NUM_DC_REGIONS 8 enum cxl_decoder_mode { CXL_DECODER_MODE_NONE, CXL_DECODER_MODE_MIXED, CXL_DECODER_MODE_PMEM, CXL_DECODER_MODE_RAM, + CXL_DECODER_MODE_DC0, + CXL_DECODER_MODE_DC1, + CXL_DECODER_MODE_DC2, + CXL_DECODER_MODE_DC3, + CXL_DECODER_MODE_DC4, + CXL_DECODER_MODE_DC5, + CXL_DECODER_MODE_DC6, + CXL_DECODER_MODE_DC7, }; static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) @@ -205,9 +217,17 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) [CXL_DECODER_MODE_MIXED] = "mixed", [CXL_DECODER_MODE_PMEM] = "pmem", [CXL_DECODER_MODE_RAM] = "ram", + [CXL_DECODER_MODE_DC0] = "dc0", + [CXL_DECODER_MODE_DC1] = "dc1", + [CXL_DECODER_MODE_DC2] = "dc2", + [CXL_DECODER_MODE_DC3] = "dc3", + [CXL_DECODER_MODE_DC4] = "dc4", + [CXL_DECODER_MODE_DC5] = "dc5", + [CXL_DECODER_MODE_DC6] = "dc6", + [CXL_DECODER_MODE_DC7] = "dc7", }; - if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_RAM) + if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_DC7) mode = CXL_DECODER_MODE_NONE; return names[mode]; } @@ -221,9 +241,30 @@ cxl_decoder_mode_from_ident(const char *ident) return CXL_DECODER_MODE_RAM; else if (strcmp(ident, "pmem") == 0) return CXL_DECODER_MODE_PMEM; + else if (strcmp(ident, "dc0") == 0) + return CXL_DECODER_MODE_DC0; + else if (strcmp(ident, "dc1") == 0) + return CXL_DECODER_MODE_DC1; + else if (strcmp(ident, "dc2") == 0) + return CXL_DECODER_MODE_DC2; + else if (strcmp(ident, "dc3") == 0) + return CXL_DECODER_MODE_DC3; + else if (strcmp(ident, "dc4") == 0) + return CXL_DECODER_MODE_DC4; + else if (strcmp(ident, "dc5") == 0) + return CXL_DECODER_MODE_DC5; + else if (strcmp(ident, "dc6") == 0) + return CXL_DECODER_MODE_DC6; + else if (strcmp(ident, "dc7") == 0) + return CXL_DECODER_MODE_DC7; return CXL_DECODER_MODE_NONE; } +static inline bool cxl_decoder_mode_is_dc(enum cxl_decoder_mode mode) +{ + return (mode >= CXL_DECODER_MODE_DC0 && mode <= CXL_DECODER_MODE_DC7); +} + enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); @@ -248,6 +289,8 @@ enum cxl_decoder_target_type { enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder); bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); +bool cxl_decoder_is_dc_capable(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_accelmem_capable(struct cxl_decoder *decoder); @@ -258,6 +301,7 @@ unsigned int cxl_decoder_get_interleave_ways(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_get_region(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_create_pmem_region(struct cxl_decoder *decoder); struct cxl_region *cxl_decoder_create_ram_region(struct cxl_decoder *decoder); +struct cxl_region *cxl_decoder_create_dc_region(struct cxl_decoder *decoder); struct cxl_decoder *cxl_decoder_get_by_name(struct cxl_ctx *ctx, const char *ident); struct cxl_memdev *cxl_decoder_get_memdev(struct cxl_decoder *decoder); @@ -308,6 +352,7 @@ enum cxl_region_mode { CXL_REGION_MODE_MIXED = CXL_DECODER_MODE_MIXED, CXL_REGION_MODE_PMEM = CXL_DECODER_MODE_PMEM, CXL_REGION_MODE_RAM = CXL_DECODER_MODE_RAM, + CXL_REGION_MODE_DC, }; static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) @@ -317,9 +362,10 @@ static inline const char *cxl_region_mode_name(enum cxl_region_mode mode) [CXL_REGION_MODE_MIXED] = "mixed", [CXL_REGION_MODE_PMEM] = "pmem", [CXL_REGION_MODE_RAM] = "ram", + [CXL_REGION_MODE_DC] = "dc", }; - if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_RAM) + if (mode < CXL_REGION_MODE_NONE || mode > CXL_REGION_MODE_DC) mode = CXL_REGION_MODE_NONE; return names[mode]; } @@ -333,6 +379,8 @@ cxl_region_mode_from_ident(const char *ident) return CXL_REGION_MODE_RAM; else if (strcmp(ident, "pmem") == 0) return CXL_REGION_MODE_PMEM; + else if (strcmp(ident, "dc") == 0) + return CXL_REGION_MODE_DC; return CXL_REGION_MODE_NONE; } From patchwork Sun Dec 15 02:58:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908640 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C92438F40 for ; Sun, 15 Dec 2024 02:58:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231526; cv=none; b=iRMxQgfNLfdse7QiFDFn5SdTwMo0INLlgXMQMJDpM4f6V3NZraS9zPMeMe5y9G0qadpzx6sCQ3x0FdhZqHtFMgYJzgHFy1LlCES+E9kyEZ0e6LYdtslxOxXZVsYJVrJ6v8O5QXxyFWVmlIG5/YKmnVP8bvM2tWILcMVztu5MgFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231526; c=relaxed/simple; bh=dOGgszzotdkdWVvZZ5AhmzaK70eLbYHLQuUG5XbtZbM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=N/FFlX2J2GCzmEoxx/cndWapjSFyvixMYwYPGAHFgKlCMc9XY1tm1ehKMbN+qG1ygbGYzEZ9mL0bru7jJvGpxQEPYWmzFCAQSSJQ7w7WrZdc3RGs4wLmjbsSLYKPxfFkg2fFhnIhncH87PJY4OzTaBX7udE2zLCdzi1O5sP6q6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nYGCKZRm; arc=none smtp.client-ip=198.175.65.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nYGCKZRm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231525; x=1765767525; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=dOGgszzotdkdWVvZZ5AhmzaK70eLbYHLQuUG5XbtZbM=; b=nYGCKZRmndyNatPe1zrdEC7uxG3T47zIURS5s1/FL22hShy5Kx9b4ilh x0xUC47FwMejU7UtHY90RWAMQhaHMGMlsuCtR9wlF7chE3xRcy3+CX9Tr mOjvnAaasTkhtbKEr3+MJvPGVCBvjCngZSx4lWLVFqa4y/sW1AqhYMMVc r0HQg+XMfIKBpmPlTeQhorKR8vJ2N8s5DIGYUpF62X543oywBtuxvI1xh ip8Ul4+bPF6n3U7h3pFuJoaMEa1Gg43MJaLn8QmJINxjQHWNsK79vZ7S2 CgQg0JFZhtJyRNKNE0gXccii3SL4+fL6fgi75vvss5g5x+sY1+fIU+wZw g==; X-CSE-ConnectionGUID: KegH4bwRSqS4wciIct2kwA== X-CSE-MsgGUID: tvTPw0KnSBek+0gzney0wQ== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="38424559" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="38424559" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:45 -0800 X-CSE-ConnectionGUID: 3zceaKm3RcGOC/UO/A20MQ== X-CSE-MsgGUID: EeXNmb4DRGymvbUemLCabg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="101462506" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:43 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:33 -0600 Subject: [ndctl PATCH v4 6/9] cxl/region: Add cxl-cli support for DCD regions Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-6-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=9116; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=dOGgszzotdkdWVvZZ5AhmzaK70eLbYHLQuUG5XbtZbM=; b=sWymmsEi6Um3rtZD+Cr1lnjFk/y/+t39vCIFMacLPvlqWgwGHw41alBBk6/OwhjPLBwybry5n e5VB/AvVXJSD9v6xzpTyRBNGg+LNW4P5BNgz37e+dItv8fKEkanGM7J X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= CXL Dynamic Capacity Devices (DCDs) optionally support dynamic capacity with up to eight partitions (Regions) (dc0-dc7). CXL regions can now be sparse and defined as dynamic capacity (dc). DCD region creation requires a specific partition, or decoder mode, to be supplied. Introduce a required option for dc regions to specify the decoder mode. Add support for dynamic capacity region creation. Based on an original patch from Navneet Singh. Signed-off-by: Sushant1 Kumar Co-developed-by: Ira Weiny Signed-off-by: Ira Weiny --- Documentation/cxl/cxl-create-region.txt | 11 ++++++++-- cxl/json.c | 27 ++++++++++++++++++++++- cxl/memdev.c | 4 +++- cxl/region.c | 39 ++++++++++++++++++++++++++++++++- 4 files changed, 76 insertions(+), 5 deletions(-) diff --git a/Documentation/cxl/cxl-create-region.txt b/Documentation/cxl/cxl-create-region.txt index b244af60b8a63281ed63d0d6f4027ea729ad51b0..a12cc8d3f19fa582376599ecc8512640f15ce42c 100644 --- a/Documentation/cxl/cxl-create-region.txt +++ b/Documentation/cxl/cxl-create-region.txt @@ -75,8 +75,9 @@ include::bus-option.txt[] -t:: --type=:: - Specify the region type - 'pmem' or 'ram'. Default to root decoder - capability, and if that is ambiguous, default to 'pmem'. + Specify the region type - 'pmem', 'ram', or 'dc'. Default to root + decoder capability including the first of any DC partition found. If + the decoder capability is ambiguous, default to 'pmem'. -U:: --uuid=:: @@ -105,6 +106,12 @@ include::bus-option.txt[] supplied, the first cross-host bridge (if available), decoder that supports the largest interleave will be chosen. +-M:: +--decoder-mode=:: + For a 'dc' region type, specify the dynamic capacity partition to be + used on each device, 'dcX' [where X is 0-7]. Ignored for region type + of 'ram' or 'pmem'. + -Q:: --enforce-qos:: Parameter to enforce qos_class mismatch failure. Region create operation diff --git a/cxl/json.c b/cxl/json.c index dcd3cc28393faf7e8adf299a857531ecdeaac50a..c5391be84fba51da57fc15ece7c1f94cce139276 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -754,12 +754,15 @@ err_free: return jpoison; } +#define DC_SIZE_NAME_LEN 64 struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, unsigned long flags) { const char *devname = cxl_memdev_get_devname(memdev); + char size_name[DC_SIZE_NAME_LEN]; struct json_object *jdev, *jobj; unsigned long long serial, size; + enum cxl_decoder_mode mode; const char *fw_version; int numa_node; int qos_class; @@ -800,6 +803,16 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, } } + for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) { + size = cxl_memdev_get_dc_size(memdev, mode); + if (!size) + continue; + jobj = util_json_object_size(size, flags); + if (!jobj) + continue; + sprintf(size_name, "%s_size", cxl_decoder_mode_name(mode)); + json_object_object_add(jdev, size_name, jobj); + } if (flags & UTIL_JSON_HEALTH) { jobj = util_cxl_memdev_health_to_json(memdev, flags); if (jobj) @@ -948,12 +961,15 @@ struct json_object *util_cxl_bus_to_json(struct cxl_bus *bus, return jbus; } +#define DC_CAPABILITY_NAME_LEN 16 struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, unsigned long flags) { const char *devname = cxl_decoder_get_devname(decoder); struct cxl_port *port = cxl_decoder_get_port(decoder); + char dc_capable_name[DC_CAPABILITY_NAME_LEN]; struct json_object *jdecoder, *jobj; + enum cxl_decoder_mode mode; struct cxl_region *region; u64 val, size; @@ -1013,7 +1029,7 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, } if (cxl_port_is_endpoint(port)) { - enum cxl_decoder_mode mode = cxl_decoder_get_mode(decoder); + mode = cxl_decoder_get_mode(decoder); size = cxl_decoder_get_dpa_size(decoder); val = cxl_decoder_get_dpa_resource(decoder); @@ -1059,6 +1075,15 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add( jdecoder, "volatile_capable", jobj); } + for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) { + if (!cxl_decoder_is_dc_capable(decoder, mode)) + continue; + jobj = json_object_new_boolean(true); + if (!jobj) + continue; + sprintf(dc_capable_name, "%s_capable", cxl_decoder_mode_name(mode)); + json_object_object_add(jdecoder, dc_capable_name, jobj); + } } if (cxl_port_is_root(port) && diff --git a/cxl/memdev.c b/cxl/memdev.c index 6e44d1578d03b6af998502e54714635b8f31b556..b132527d7c21e99c9da75ea7cbf1497fd924a142 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -269,8 +269,10 @@ static int __reserve_dpa(struct cxl_memdev *memdev, if (mode == CXL_DECODER_MODE_RAM) avail_dpa = cxl_memdev_get_ram_size(memdev); - else + else if (mode == CXL_DECODER_MODE_PMEM) avail_dpa = cxl_memdev_get_pmem_size(memdev); + else if (cxl_decoder_mode_is_dc(mode)) + avail_dpa = cxl_memdev_get_dc_size(memdev, mode); cxl_decoder_foreach(port, decoder) { size = cxl_decoder_get_dpa_size(decoder); diff --git a/cxl/region.c b/cxl/region.c index 527bd6708b162815068a95ddb360fce3914347de..79f434b0c99545523f3b8209f90ff2f02111881c 100644 --- a/cxl/region.c +++ b/cxl/region.c @@ -26,6 +26,7 @@ static struct region_params { const char *uuid; const char *root_decoder; const char *region; + const char *decoder_mode; int ways; int granularity; bool memdevs; @@ -79,9 +80,11 @@ OPT_INTEGER('w', "ways", ¶m.ways, \ OPT_INTEGER('g', "granularity", ¶m.granularity, \ "granularity of the interleave set"), \ OPT_STRING('t', "type", ¶m.type, \ - "region type", "region type - 'pmem' or 'ram'"), \ + "region type", "region type - 'pmem', 'ram', or 'dc'"), \ OPT_STRING('U', "uuid", ¶m.uuid, \ "region uuid", "uuid for the new region (default: autogenerate)"), \ +OPT_STRING('M', "decoder-mode", ¶m.decoder_mode, "decoder mode", \ + "decoder mode for dc regions - 'dcX' [where X is 0-7]"), \ OPT_BOOLEAN('m', "memdevs", ¶m.memdevs, \ "non-option arguments are memdevs"), \ OPT_BOOLEAN('u', "human", ¶m.human, "use human friendly number formats"), \ @@ -314,12 +317,19 @@ static int parse_create_options(struct cxl_ctx *ctx, int count, log_err(&rl, "unsupported type: %s\n", param.type); goto err; } + if (p->region_mode == CXL_REGION_MODE_DC && !param.decoder_mode) { + log_err(&rl, "dc type requires a decoder mode\n"); + goto err; + } } switch (p->region_mode) { case CXL_REGION_MODE_RAM: p->decoder_mode = CXL_DECODER_MODE_RAM; break; + case CXL_REGION_MODE_DC: + p->decoder_mode = cxl_decoder_mode_from_ident(param.decoder_mode); + break; case CXL_REGION_MODE_PMEM: default: p->decoder_mode = CXL_DECODER_MODE_PMEM; @@ -427,6 +437,9 @@ static void collect_minsize(struct cxl_ctx *ctx, struct parsed_params *p) case CXL_DECODER_MODE_PMEM: size = cxl_memdev_get_pmem_size(memdev); break; + case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7: + size = cxl_memdev_get_dc_size(memdev, p->decoder_mode); + break; default: /* Shouldn't ever get here */ ; } @@ -498,6 +511,13 @@ static int validate_decoder(struct cxl_decoder *decoder, return -EINVAL; } break; + case CXL_DECODER_MODE_DC0 ... CXL_DECODER_MODE_DC7: + if (!cxl_decoder_is_dc_capable(decoder, p->decoder_mode)) { + log_err(&rl, "%s is not %s capable\n", devname, + cxl_decoder_mode_name(p->decoder_mode)); + return -EINVAL; + } + break; default: log_err(&rl, "unknown type: %s\n", param.type); return -EINVAL; @@ -514,10 +534,20 @@ static int validate_decoder(struct cxl_decoder *decoder, static void set_type_from_decoder(struct cxl_ctx *ctx, struct parsed_params *p) { + enum cxl_decoder_mode mode; + /* if param.type was explicitly specified, nothing to do here */ if (param.type) return; + for (mode = CXL_DECODER_MODE_DC0; mode <= CXL_DECODER_MODE_DC7; mode++) { + if (cxl_decoder_is_dc_capable(p->root_decoder, mode)) { + p->decoder_mode = mode; + p->region_mode = CXL_REGION_MODE_DC; + break; + } + } + /* * default to pmem if both types are set, otherwise the single * capability dominates. @@ -713,6 +743,13 @@ static int create_region(struct cxl_ctx *ctx, int *count, param.root_decoder); return -ENXIO; } + } else if (p->region_mode == CXL_REGION_MODE_DC) { + region = cxl_decoder_create_dc_region(p->root_decoder); + if (!region) { + log_err(&rl, "failed to create region under %s\n", + param.root_decoder); + return -ENXIO; + } } else { log_err(&rl, "region type '%s' is not supported\n", param.type); From patchwork Sun Dec 15 02:58:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908641 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AE2C8C07 for ; Sun, 15 Dec 2024 02:58:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231528; cv=none; b=fqN+gKNfCPsbcrKqjduUvklLsRakxR2bdCLTKz6uX/WPgZusK+hnxLcc2nkUJsKHe7cAmpr1rOemCbe4kqUjm/VMBrorl5ZWVJIyFW1QeOQtBijpqOU8y7aRfGCxXCvTlmenXlsp86Lml+uQHjdBv9RsHKZ6oI3EvtE17hFosto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231528; c=relaxed/simple; bh=6Nyr7xJsAkzjD9LFKGHwcxKx7vxO4IVRyLaJMOUJZ8s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GlEENioC6DDIW9nNjQSwJSV4Ein/LntRSzohcEhZ1KMfgRakLvAMnVnU5xhNucNSCfXgbYl81HdMbisp5aO7ss6CIGkzY9S7wit7NVqgDCDRmJe5uatjD8/Tq0zmXqLdK3/MmGJTim4+uetp2LQM2Ez9USP0fDG9gt+zsIaEsv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BDvYBNvK; arc=none smtp.client-ip=198.175.65.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BDvYBNvK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231528; x=1765767528; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=6Nyr7xJsAkzjD9LFKGHwcxKx7vxO4IVRyLaJMOUJZ8s=; b=BDvYBNvKJbKZiDG+dYRRk9KVpurr79nZIj14yuz4gwZedjILfehZCBq5 ASVIOU3g97TbKBfUVlg8i9IQKjdFOGtgn0Xp7dP1cjgWiI1KynFayjxXZ gPaaDdOr/nrCwjPUaKQ/4YUq4+vFHWiYuOyj1+7gnRpm0QePdd59NPR7l LibncYqupDU9AS0nnztkad8jYq7/2/zMHJHK1pbnZrJqFRCd5FV1beZES reA5CxdSwhR10575QJBfTqRAkHXuF8SqPbs7WnDMq4nEQy/nN6R2Py9QC di7AF6DZwORPXHob+EUIFGN7tvzPSZQSOxbGGAZIccLUfiSzc2CuTf1ha w==; X-CSE-ConnectionGUID: 4/xGLx6SSQK+dXW+Buc27Q== X-CSE-MsgGUID: Xa7L+5lERtePtR+j9iT0XQ== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="38424563" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="38424563" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:48 -0800 X-CSE-ConnectionGUID: 8nu/DKNmQayOlgvzo9pAPA== X-CSE-MsgGUID: q1d3HvuKSq6z2wHfQ76ZbQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="101462511" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:45 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:34 -0600 Subject: [ndctl PATCH v4 7/9] libcxl: Add extent functionality to DC regions Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-7-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=8952; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=6Nyr7xJsAkzjD9LFKGHwcxKx7vxO4IVRyLaJMOUJZ8s=; b=wWlSm1k9+RYSkpUri1Q4qloqbzXECesrln7Psz5Ir7aqq7WiiMYBEp+m6JRa0ThjC732jih0i E+VyeyzlynIAE7iQiutGpMdaUj3uAr+phOpoFZMi3m1rfm08npU+D0A X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= DCD regions have 0 or more extents. The ability to list those and their properties is useful to end users. Add extent scanning and reporting functionality to libcxl. Signed-off-by: Ira Weiny --- Documentation/cxl/lib/libcxl.txt | 27 ++++++++ cxl/lib/libcxl.c | 138 +++++++++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 5 ++ cxl/lib/private.h | 11 ++++ cxl/libcxl.h | 11 ++++ 5 files changed, 192 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index abca08fc81e6e84d176facafad6decae2f875880..0b53cf9a3a09a3e8c9059f796823b52d22d1077f 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -632,6 +632,33 @@ Regions now have a mode distinct from decoders. cxl_region_get_mode() is deprecated in favor of cxl_region_get_region_mode(). Dynamic capacity regions require the use of cxl_region_get_region_mode(). +EXTENTS +------- + +=== EXTENT: Enumeration +---- +struct cxl_region_extent; +struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region); +struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent); +#define cxl_extent_foreach(region, extent) \ + for (extent = cxl_extent_get_first(region); \ + extent != NULL; \ + extent = cxl_extent_get_next(extent)) + +---- + +=== EXTENT: Attributes +---- +unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent); +unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent); +void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag); +---- + +Extents represent available memory within a dynamic capacity region. Extent +objects are available for informational purposes to aid in allocation of +memory. + + include::../../copyright.txt[] SEE ALSO diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index df250db9dbacb2f0f34e8a592ce194159584fe4f..a029b14dcccf038b02b28d05df6f0dc71557df5e 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -568,6 +568,7 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base) region->ctx = ctx; region->decoder = decoder; list_head_init(®ion->mappings); + list_head_init(®ion->extents); region->dev_path = strdup(cxlregion_base); if (!region->dev_path) @@ -1178,6 +1179,143 @@ cxl_mapping_get_next(struct cxl_memdev_mapping *mapping) return list_next(®ion->mappings, mapping, list); } +static void cxl_extents_init(struct cxl_region *region) +{ + const char *devname = cxl_region_get_devname(region); + struct cxl_ctx *ctx = cxl_region_get_ctx(region); + char *extent_path, *dax_region_path; + struct dirent *de; + DIR *dir = NULL; + + if (region->extents_init) + return; + region->extents_init = 1; + + dax_region_path = calloc(1, strlen(region->dev_path) + 64); + if (!dax_region_path) { + err(ctx, "%s: allocation failure\n", devname); + return; + } + + extent_path = calloc(1, strlen(region->dev_path) + 100); + if (!extent_path) { + err(ctx, "%s: allocation failure\n", devname); + free(dax_region_path); + return; + } + + sprintf(dax_region_path, "%s/dax_region%d", + region->dev_path, region->id); + dir = opendir(dax_region_path); + if (!dir) { + err(ctx, "no extents found (%s): %s\n", + strerror(errno), dax_region_path); + free(extent_path); + free(dax_region_path); + return; + } + + while ((de = readdir(dir)) != NULL) { + struct cxl_region_extent *extent; + char buf[SYSFS_ATTR_SIZE]; + u64 offset, length; + int id, region_id; + + if (sscanf(de->d_name, "extent%d.%d", ®ion_id, &id) != 2) + continue; + + sprintf(extent_path, "%s/extent%d.%d/offset", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/offset\n", + devname, region_id, id); + continue; + } + + offset = strtoull(buf, NULL, 0); + if (offset == ULLONG_MAX) { + err(ctx, "%s extent%d.%d: failed to read offset\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/length", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/length\n", + devname, region_id, id); + continue; + } + + length = strtoull(buf, NULL, 0); + if (length == ULLONG_MAX) { + err(ctx, "%s extent%d.%d: failed to read length\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/tag", + dax_region_path, region_id, id); + buf[0] = '\0'; + if (sysfs_read_attr(ctx, extent_path, buf) != 0) + dbg(ctx, "%s extent%d.%d: failed to read tag\n", + devname, region_id, id); + + extent = calloc(1, sizeof(*extent)); + if (!extent) { + err(ctx, "%s extent%d.%d: allocation failure\n", + devname, region_id, id); + continue; + } + if (strlen(buf) && uuid_parse(buf, extent->tag) < 0) + err(ctx, "%s:%s\n", extent_path, buf); + extent->region = region; + extent->offset = offset; + extent->length = length; + + list_node_init(&extent->list); + list_add(®ion->extents, &extent->list); + dbg(ctx, "%s added extent%d.%d\n", devname, region_id, id); + } + free(dax_region_path); + free(extent_path); + closedir(dir); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_first(struct cxl_region *region) +{ + cxl_extents_init(region); + + return list_top(®ion->extents, struct cxl_region_extent, list); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_next(struct cxl_region_extent *extent) +{ + struct cxl_region *region = extent->region; + + return list_next(®ion->extents, extent, list); +} + +CXL_EXPORT unsigned long long +cxl_extent_get_offset(struct cxl_region_extent *extent) +{ + return extent->offset; +} + +CXL_EXPORT unsigned long long +cxl_extent_get_length(struct cxl_region_extent *extent) +{ + return extent->length; +} + +CXL_EXPORT void +cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag) +{ + memcpy(tag, extent->tag, sizeof(uuid_t)); +} + CXL_EXPORT struct cxl_decoder * cxl_mapping_get_decoder(struct cxl_memdev_mapping *mapping) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index fdb227789985443a13c72751bbd42ab383db5f97..d8e8dbc7e091792fe48faa4657ab7cf1d795efdd 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -295,4 +295,9 @@ global: cxl_memdev_get_dc_size; cxl_decoder_is_dc_capable; cxl_decoder_create_dc_region; + cxl_extent_get_first; + cxl_extent_get_next; + cxl_extent_get_offset; + cxl_extent_get_length; + cxl_extent_get_tag; } LIBECXL_8; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 3efa230bfb632e6c6048aadd18f799b07d4bdfd3..62278ec79963c198dcca490015e4c3f7621109b2 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -164,6 +164,7 @@ struct cxl_region { struct cxl_decoder *decoder; struct list_node list; int mappings_init; + int extents_init; struct cxl_ctx *ctx; void *dev_buf; size_t buf_len; @@ -179,6 +180,7 @@ struct cxl_region { struct daxctl_region *dax_region; struct kmod_module *module; struct list_head mappings; + struct list_head extents; }; struct cxl_memdev_mapping { @@ -188,6 +190,15 @@ struct cxl_memdev_mapping { struct list_node list; }; +#define CXL_REGION_EXTENT_TAG 0x10 +struct cxl_region_extent { + struct cxl_region *region; + u64 offset; + u64 length; + uuid_t tag; + struct list_node list; +}; + enum cxl_cmd_query_status { CXL_CMD_QUERY_NOT_RUN = 0, CXL_CMD_QUERY_OK, diff --git a/cxl/libcxl.h b/cxl/libcxl.h index d7f8a37816f236acd71fc834eae70a7a17a2721a..1d294ac0278d798214acb2f62e98aaaccaf60ea5 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -445,6 +445,17 @@ unsigned int cxl_mapping_get_position(struct cxl_memdev_mapping *mapping); mapping != NULL; \ mapping = cxl_mapping_get_next(mapping)) +struct cxl_region_extent; +struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region); +struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent); +#define cxl_extent_foreach(region, extent) \ + for (extent = cxl_extent_get_first(region); \ + extent != NULL; \ + extent = cxl_extent_get_next(extent)) +unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent); +unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent); +void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag); + struct cxl_cmd; const char *cxl_cmd_get_devname(struct cxl_cmd *cmd); struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode); From patchwork Sun Dec 15 02:58:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908642 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6C5AC133 for ; Sun, 15 Dec 2024 02:58:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231531; cv=none; b=AddCG8j8cn6G4z/6ED0KijzJ2SVsBUHQnZneVLCIymIs6ToChlC5dFx7+M58ArJzParBhDIErldkd2OY3tO2DQtvXcq2Eg0UWvsqXmMKRn/ghuB4Za3kzJY6onHVrPAUddi00hxZtU0zfcuPMwPrUBZQy2NDCwKbW58g3CRAmUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231531; c=relaxed/simple; bh=sm9ZEyQlWPLDP0W5A+dsrr3jMpJLe/ZSc4iBQSQg6i0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=E0O7u42QlzwQYMKaOlPZRk92bwyJctQ7aZjGFClqxsrCroOo6uVYy1wZeXb1M3Bfjz0GVG6vbhAsY9VUtDlinrDTbyC2oqIZfnCtNde7FByu98P3rCCFZ22FsKZKYL3llxQzbnLrDAJlb6+5LFuorDHucvzppgYA2VNnq5tphbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UrNOjDT8; arc=none smtp.client-ip=198.175.65.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UrNOjDT8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231530; x=1765767530; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=sm9ZEyQlWPLDP0W5A+dsrr3jMpJLe/ZSc4iBQSQg6i0=; b=UrNOjDT8mII1HALrH6s6ZIWIaopKYJSjbTN4basrF+nUXuePkEUEfHOx NMEq6dGzQdD4r11f2+MHd+D5Ul/uw5uagUOUuSEDytYjXTrEHKDRjlkSS cfxbuec24luiakDhyn+LNQvUJdqLPq3docozY+T+CLFB9rmlBOf8nbPrp ZCGQuqbJSZiyWhPUIKhoWs9MHfyhNBNnC4/Rqm4QeEkZ3CaxJ+tVqZ7Ia t54l0a86IDkAn6UluwX0IJjOziW/itAGG7FQ4jAwzf5FxHLqNitXvgKw6 g828upRC+BkBN0LcVMrtshfrdlmsFFNJQUxo8CkY3ylq+iQLXcFPgBZHH g==; X-CSE-ConnectionGUID: lhV1IQ5XQuqkert41tL/PQ== X-CSE-MsgGUID: Mg1aTSDeQta+zQX0j0kNeQ== X-IronPort-AV: E=McAfee;i="6700,10204,11286"; a="38424568" X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="38424568" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:50 -0800 X-CSE-ConnectionGUID: MZQOdjXkQFqT5lJyikHJ1g== X-CSE-MsgGUID: o/D8OPIYRZGfiST0dLakuw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="101462516" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:48 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:35 -0600 Subject: [ndctl PATCH v4 8/9] cxl/region: Add extent output to region query Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-8-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=6589; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=sm9ZEyQlWPLDP0W5A+dsrr3jMpJLe/ZSc4iBQSQg6i0=; b=XhUJg4SCi2SiSKk2dkOhkvi12aIjOGvLfiS5odedKclDEFGYo+KjhpsJYp/7BGb4WRu9IaIBQ mz0y7ojrzl4DML19iJEBMhYfD5HBlQZlFaCKu9zN2oRHuFavGCC3Te1 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= DCD regions have 0 or more extents. The ability to list those and their properties is useful to end users. Add an option for extent output to region queries. An example of this is: $ ./build/cxl/cxl list -r 8 -Nu { "region":"region8", ... "type":"dc", ... "extents":[ { "offset":"0x10000000", "length":"64.00 MiB (67.11 MB)", "tag":"00000000-0000-0000-0000-000000000000" }, { "offset":"0x8000000", "length":"64.00 MiB (67.11 MB)", "tag":"00000000-0000-0000-0000-000000000000" } ] } Signed-off-by: Ira Weiny --- Documentation/cxl/cxl-list.txt | 29 ++++++++++++++++++++++++++ cxl/filter.h | 3 +++ cxl/json.c | 47 ++++++++++++++++++++++++++++++++++++++++++ cxl/json.h | 3 +++ cxl/list.c | 3 +++ util/json.h | 1 + 6 files changed, 86 insertions(+) diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt index 9a9911e7dd9bba561c6202784017db1bb4b9f4bd..43453cc72245f586070f8c4f31b3ee475e3c6cd2 100644 --- a/Documentation/cxl/cxl-list.txt +++ b/Documentation/cxl/cxl-list.txt @@ -411,6 +411,35 @@ OPTIONS } ---- +-N:: +--extents:: + Append Dynamic Capacity extent information. +---- +13:34:28 > ./build/cxl/cxl list -r 8 -Nu +{ + "region":"region8", + "resource":"0xf030000000", + "size":"512.00 MiB (536.87 MB)", + "type":"dc", + "interleave_ways":1, + "interleave_granularity":256, + "decode_state":"commit", + "extents":[ + { + "offset":"0x10000000", + "length":"64.00 MiB (67.11 MB)", + "tag":"00000000-0000-0000-0000-000000000000" + }, + { + "offset":"0x8000000", + "length":"64.00 MiB (67.11 MB)", + "tag":"00000000-0000-0000-0000-000000000000" + } + ] +} +---- + + -r:: --region:: Specify CXL region device name(s), or device id(s), to filter the listing. diff --git a/cxl/filter.h b/cxl/filter.h index 956a46e0c7a9f05abf696cce97a365164e95e50d..a31b80c87ccac407bd4ff98b302a23b33cbe413c 100644 --- a/cxl/filter.h +++ b/cxl/filter.h @@ -31,6 +31,7 @@ struct cxl_filter_params { bool alert_config; bool dax; bool media_errors; + bool extents; int verbose; struct log_ctx ctx; }; @@ -91,6 +92,8 @@ static inline unsigned long cxl_filter_to_flags(struct cxl_filter_params *param) flags |= UTIL_JSON_DAX | UTIL_JSON_DAX_DEVS; if (param->media_errors) flags |= UTIL_JSON_MEDIA_ERRORS; + if (param->extents) + flags |= UTIL_JSON_EXTENTS; return flags; } diff --git a/cxl/json.c b/cxl/json.c index c5391be84fba51da57fc15ece7c1f94cce139276..450e62243ecdfec1aa011241ff7257ac3b37196f 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -1169,6 +1169,50 @@ void util_cxl_mappings_append_json(struct json_object *jregion, json_object_object_add(jregion, "mappings", jmappings); } +void util_cxl_extents_append_json(struct json_object *jregion, + struct cxl_region *region, + unsigned long flags) +{ + struct json_object *jextents; + struct cxl_region_extent *extent; + + jextents = json_object_new_array(); + if (!jextents) + return; + + cxl_extent_foreach(region, extent) { + struct json_object *jextent, *jobj; + unsigned long long val; + char tag_str[40]; + uuid_t tag; + + jextent = json_object_new_object(); + if (!jextent) + continue; + + val = cxl_extent_get_offset(extent); + jobj = util_json_object_hex(val, flags); + if (jobj) + json_object_object_add(jextent, "offset", jobj); + + val = cxl_extent_get_length(extent); + jobj = util_json_object_size(val, flags); + if (jobj) + json_object_object_add(jextent, "length", jobj); + + cxl_extent_get_tag(extent, tag); + uuid_unparse(tag, tag_str); + jobj = json_object_new_string(tag_str); + if (jobj) + json_object_object_add(jextent, "tag", jobj); + + json_object_array_add(jextents, jextent); + json_object_set_userdata(jextent, extent, NULL); + } + + json_object_object_add(jregion, "extents", jextents); +} + struct json_object *util_cxl_region_to_json(struct cxl_region *region, unsigned long flags) { @@ -1255,6 +1299,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, } } + if (flags & UTIL_JSON_EXTENTS) + util_cxl_extents_append_json(jregion, region, flags); + if (cxl_region_qos_class_mismatch(region)) { jobj = json_object_new_boolean(true); if (jobj) diff --git a/cxl/json.h b/cxl/json.h index eb7572be4106baf0469ba9243a9a767d07df8882..f9c07ab41a337838b75ffee4486f6c48ddc99863 100644 --- a/cxl/json.h +++ b/cxl/json.h @@ -20,6 +20,9 @@ struct json_object *util_cxl_region_to_json(struct cxl_region *region, void util_cxl_mappings_append_json(struct json_object *jregion, struct cxl_region *region, unsigned long flags); +void util_cxl_extents_append_json(struct json_object *jregion, + struct cxl_region *region, + unsigned long flags); void util_cxl_targets_append_json(struct json_object *jdecoder, struct cxl_decoder *decoder, const char *ident, const char *serial, diff --git a/cxl/list.c b/cxl/list.c index 0b25d78248d5f4f529fd2c2e073e43895c722568..47d135166212b87449f960e94ee75657f7040ca9 100644 --- a/cxl/list.c +++ b/cxl/list.c @@ -59,6 +59,8 @@ static const struct option options[] = { "include alert configuration information"), OPT_BOOLEAN('L', "media-errors", ¶m.media_errors, "include media-error information "), + OPT_BOOLEAN('N', "extents", ¶m.extents, + "include extent information (Dynamic Capacity regions only)"), OPT_INCR('v', "verbose", ¶m.verbose, "increase output detail"), #ifdef ENABLE_DEBUG OPT_BOOLEAN(0, "debug", &debug, "debug list walk"), @@ -135,6 +137,7 @@ int cmd_list(int argc, const char **argv, struct cxl_ctx *ctx) param.decoders = true; param.targets = true; param.regions = true; + param.extents = true; /*fallthrough*/ case 0: break; diff --git a/util/json.h b/util/json.h index 560f845c6753ee176f7c64b4310fe1f9b1ce6d39..79ae3240e7ce151be75f6666fcaba0ba90aba7fc 100644 --- a/util/json.h +++ b/util/json.h @@ -21,6 +21,7 @@ enum util_json_flags { UTIL_JSON_TARGETS = (1 << 11), UTIL_JSON_PARTITION = (1 << 12), UTIL_JSON_ALERT_CONFIG = (1 << 13), + UTIL_JSON_EXTENTS = (1 << 14), }; void util_display_json_array(FILE *f_out, struct json_object *jarray, From patchwork Sun Dec 15 02:58:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13908643 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E945D299 for ; Sun, 15 Dec 2024 02:58:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231534; cv=none; b=Ah7fL2esrp6Vmpwi8fzRVt0NdreJD5Qv8WOz2xnVO1ZpEkvmWRkCsJWbG5kiC8oHvE2W+EfM4ehX4pEl3YYgZBLsNMoNNgGywNIW4xDHnlk9BKdhEz2Mz6t1OTPPPsl7dt03UvodwyBczHXHojk6EEeO+CM3buKx+ybhHx4N0kc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734231534; c=relaxed/simple; bh=WVidWGTjmWjDdbIW2Fn6txJR6NX0YCqXC+P4lMmVwps=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cSpoO+Yzin8Q43LxamuJjBDth6tdjfzCxboXzr/LS3ud/0A28cU+DAON11uvWv9KxdGqu0Cz5wgyNtwgOrbHOh8FyD8ZOYnZWYF2n6MfuHofrKGwTtsKxtmTW2ALT2YuQd1NI81OoeFI5D4MIrTiJWmX6uLHT57zRdcmBc1HkOg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XaXd8RQs; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XaXd8RQs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1734231532; x=1765767532; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=WVidWGTjmWjDdbIW2Fn6txJR6NX0YCqXC+P4lMmVwps=; b=XaXd8RQsJV7Hlsf35BTqTExV3fNQGcT+ZZKNAXxp8occHw4knVvYAdYT 5i+26KWGF3kXEUPVY8DPzUk97PFVQHwAgXsXB4O56gFC/xkUOMcXck+Z6 yHU+1IX5uYgwLBKbXzv2QbWyrxIGHHv43F58aH0UdZsivEC0qCg1Yxs74 xRlC3S+iUvdvpvr5DHu997UQWctnLJFjgx/AooZALdBtYIyDkNhaKGYt7 DsUFXpWp504YMCERkt2jUb7LjohvdV9BNiO+Fd75NqcIQXD9e0+lE8qEF bIJyMqaU7kzdscYpxYC7E+e/OTcFJV3llG6xO+1PjKjQvTLrr7sLISW+q A==; X-CSE-ConnectionGUID: o8YHCcsdTMeyzR+tZjRwVg== X-CSE-MsgGUID: 4GJT4lyBTOWstpbh+WABLA== X-IronPort-AV: E=McAfee;i="6700,10204,11278"; a="46039329" X-IronPort-AV: E=Sophos;i="6.12,214,1728975600"; d="scan'208";a="46039329" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:52 -0800 X-CSE-ConnectionGUID: kRO854DuSVyJu9B6jxdQCw== X-CSE-MsgGUID: gc4kb/svRgOBR3YaoFRCGQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,235,1728975600"; d="scan'208";a="97286790" Received: from rchatre-mobl4.amr.corp.intel.com (HELO localhost) ([10.125.111.42]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2024 18:58:51 -0800 From: Ira Weiny Date: Sat, 14 Dec 2024 20:58:36 -0600 Subject: [ndctl PATCH v4 9/9] cxl/test: Add Dynamic Capacity tests Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241214-dcd-region2-v4-9-36550a97f8e2@intel.com> References: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> In-Reply-To: <20241214-dcd-region2-v4-0-36550a97f8e2@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Sushant1 Kumar , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1734231510; l=29791; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=WVidWGTjmWjDdbIW2Fn6txJR6NX0YCqXC+P4lMmVwps=; b=LJMf4bl2CqT0WE9XQfzeSoKH25MA745HKLeQxiN3Og3R/t88ZAMWv1fB4pe0eWR/i4swzj0Al whquZI0Wno3B3U6MR1Ld1VB9oWaSZx+gEgc22vMXheHKv9KDy0lDij+ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= cxl_test provides a good way to ensure quick smoke and regression testing. The complexity of DCD and the new sparse DAX regions required to use them benefits greatly with a series of smoke tests. The only part of the kernel stack which must be bypassed is the actual irq of DCD events. However, the event processing itself can be tested via cxl_test calling directly into the event processing. In this way the rest of the stack; management of sparse regions, the extent device lifetimes, and the dax device operations can be tested. Add Dynamic Capacity Device tests for kernels which have DCD support. Signed-off-by: Ira Weiny --- Changes: [iweiny: Fix checks for dax devices] [iweiny: Fix documentation on tests] --- test/cxl-dcd.sh | 879 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ test/meson.build | 2 + 2 files changed, 881 insertions(+) diff --git a/test/cxl-dcd.sh b/test/cxl-dcd.sh new file mode 100644 index 0000000000000000000000000000000000000000..193b71923b2f44142ab258e1d74b88361f6b7a5e --- /dev/null +++ b/test/cxl-dcd.sh @@ -0,0 +1,879 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2024 Intel Corporation. All rights reserved. + +. "$(dirname "$0")/common" + +rc=77 +set -ex + +trap 'err $LINENO' ERR + +check_prereq "jq" + +rc=1 + +dev_path="/sys/bus/platform/devices" +cxl_path="/sys/bus/cxl/devices" + +# a test extent tag +test_tag=dc-test-tag + +# +# The test devices have 2G of non DC capacity. A single DC reagion of 1G is +# added beyond that. +# +# The testing centers around 3 extents. Two are "pre-existing" on test load +# called pre-ext and pre2-ext. The other is created within this script alone +# called base. + +# +# | 2G non- | DC region (1G) | +# | DC cap | | +# | ... |-------------------------------------------------------| +# | |--------| |----------| |----------| | +# | | (base) | |(pre-ext) | |(pre2-ext)| | + +dc0_size="" +dc1_size="" + +base_dpa=0x80000000 + +# base extent at dpa 2G - 64M long +base_ext_offset=0x0 +base_ext_dpa=$(($base_dpa + $base_ext_offset)) +base_ext_length=0x4000000 + +# pre existing extent base + 128M, 64M length +# 0x00000088000000-0x0000008bffffff +pre_ext_offset=0x8000000 +pre_ext_dpa=$(($base_dpa + $pre_ext_offset)) +pre_ext_length=0x4000000 + +# pre2 existing extent base + 256M, 64M length +# 0x00000090000000-0x00000093ffffff +pre2_ext_offset=0x10000000 +pre2_ext_dpa=$(($base_dpa + $pre2_ext_offset)) +pre2_ext_length=0x4000000 + +mem="" +bus="" +device="" +decoder="" + +# ======================================================================== +# Support functions +# ======================================================================== + +create_dcd_region() +{ + mem="$1" + decoder="$2" + reg_size_string="" + if [ "$3" != "" ]; then + reg_size_string="-s $3" + fi + + dcd_partition="dc0" + if [ "$4" != "" ]; then + dcd_partition="$4" + fi + + # create region + rc=$($CXL create-region -t dc -M ${dcd_partition} -d "$decoder" -m "$mem" ${reg_size_string} | jq -r ".region") + + if [[ ! $rc ]]; then + echo "create-region failed for $decoder / $mem" + err "$LINENO" + fi + + echo ${rc} +} + +check_region() +{ + search=$1 + region_size=$2 + + result=$($CXL list -r "$search" | jq -r ".[].region") + if [ "$result" != "$search" ]; then + echo "check region failed to find $search" + err "$LINENO" + fi + + result=$($CXL list -r "$search" | jq -r ".[].size") + if [ "$result" != "$region_size" ]; then + echo "check region failed invalid size $result != $region_size" + err "$LINENO" + fi +} + +check_not_region() +{ + search=$1 + + result=$($CXL list -r "$search" | jq -r ".[].region") + if [ "$result" == "$search" ]; then + echo "check not region failed; $search found" + err "$LINENO" + fi +} + +destroy_region() +{ + local region=$1 + $CXL disable-region $region + $CXL destroy-region $region +} + +inject_extent() +{ + device="$1" + dpa="$2" + length="$3" + tag="$4" + + more="0" + if [ "$5" != "" ]; then + more="1" + fi + + echo ${dpa}:${length}:${tag}:${more} > "${dev_path}/${device}/dc_inject_extent" +} + +remove_extent() +{ + device="$1" + dpa="$2" + length="$3" + + echo ${dpa}:${length} > "${dev_path}/${device}/dc_del_extent" +} + +create_dax_dev() +{ + reg="$1" + + dax_dev=$($DAXCTL create-device -r $reg | jq -er '.[].chardev') + + echo ${dax_dev} +} + +fail_create_dax_dev() +{ + reg="$1" + + set +e + result=$($DAXCTL create-device -r $reg) + set -e + if [ "$result" == "0" ]; then + echo "FAIL device created" + err "$LINENO" + fi +} + +shrink_dax_dev() +{ + dev="$1" + new_size="$2" + + $DAXCTL disable-device $dev + $DAXCTL reconfigure-device $dev -s $new_size + $DAXCTL enable-device $dev +} + +destroy_dax_dev() +{ + dev="$1" + + $DAXCTL disable-device $dev + $DAXCTL destroy-device $dev +} + +check_dax_dev() +{ + search="$1" + size=$(($2)) + + result=$($DAXCTL list -d $search | jq -er '.[].chardev') + if [ "$result" != "$search" ]; then + echo "check dax device failed to find $search" + err "$LINENO" + fi + result=$($DAXCTL list -d $search | jq -er '.[].size') + if [ "$result" -ne "$size" ]; then + echo "check dax device failed incorrect size $result; exp $size" + err "$LINENO" + fi +} + +# check that the dax device is not there. +check_not_dax_dev() +{ + reg="$1" + search="$2" + result=$($DAXCTL list -r $reg -D | jq -r '.[].chardev') + if [ "$result" == "$search" ]; then + echo "FAIL found $search" + err "$LINENO" + fi +} + +check_extent() +{ + region=$1 + offset=$(($2)) + length=$(($3)) + + result=$($CXL list -r "$region" -N | jq -r ".[].extents[] | select(.offset == ${offset}) | .length") + if [[ $result != $length ]]; then + echo "FAIL region $1 could not find extent @ $offset ($length)" + err "$LINENO" + fi +} + +check_extent_cnt() +{ + region=$1 + count=$(($2)) + + result=$($CXL list -r $region -N | jq -r '.[].extents[].offset' | wc -l) + if [[ $result != $count ]]; then + echo "FAIL region $1: found wrong number of extents $result; expect $count" + err "$LINENO" + fi +} + + +# ======================================================================== +# Tests +# ======================================================================== + +# testing pre existing extents must be called first as the extents were created +# by cxl-test being loaded +test_pre_existing_extents() +{ + echo "" + echo "Test: pre-existing extent" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | |----------| |----------| | + # | | |(pre-ext) | |(pre2-ext)| | + check_region ${region} ${dc0_size} + # should contain pre-created extents + check_extent ${region} ${pre_ext_offset} ${pre_ext_length} + check_extent ${region} ${pre2_ext_offset} ${pre2_ext_length} + + remove_extent ${device} $pre_ext_dpa $pre_ext_length + # | | |----------| | + # | | |(pre2-ext)| | + remove_extent ${device} $pre2_ext_dpa $pre2_ext_length + # | | | + # | | | + check_extent_cnt ${region} 0 + destroy_region ${region} + check_not_region ${region} +} + +test_remove_extent_under_dax_device() +{ + # Remove the pre-created test extent out from under dax device + # stack should hold ref until dax device deleted + echo "" + echo "Test: Remove extent from under DAX dev" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | | + # | | | + + + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + # | | |----------| | + # | | |(pre-ext) | | + check_extent ${region} ${pre_ext_offset} ${pre_ext_length} + + dax_dev=$(create_dax_dev ${region}) + # | | |----------| | + # | | |(pre-ext) | | + # | | | daxX.1 | | + check_extent_cnt ${region} 1 + remove_extent ${device} $pre_ext_dpa $pre_ext_length + # In-use extents are not released. + check_dax_dev ${dax_dev} $pre_ext_length + + check_extent_cnt ${region} 1 + destroy_dax_dev ${dax_dev} + # | | |----------| | + # | | |(pre-ext) | | + check_not_dax_dev ${region} ${dax_dev} + + check_extent_cnt ${region} 1 + # Remove after use + remove_extent ${device} $pre_ext_dpa $pre_ext_length + # | | | + # | | | + check_extent_cnt ${region} 0 + destroy_region ${region} + check_not_region ${region} +} + +test_remove_extents_in_use() +{ + echo "" + echo "Test: Remove extents under sparse dax device" + echo "" + remove_extent ${device} $base_ext_dpa $base_ext_length + check_extent_cnt ${region} 2 + remove_extent ${device} $pre_ext_dpa $pre_ext_length + check_extent_cnt ${region} 2 +} + +test_create_dax_dev_spanning_two_extents() +{ + echo "" + echo "Test: Create dax device spanning 2 extents" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + check_extent ${region} ${pre_ext_offset} ${pre_ext_length} + # | | |----------| | + # | | |(pre-ext) | | + inject_extent ${device} $base_ext_dpa $base_ext_length "" + check_extent ${region} ${base_ext_offset} ${base_ext_length} + # | |--------| |----------| | + # | | (base) | |(pre-ext) | | + + check_extent_cnt ${region} 2 + dax_dev=$(create_dax_dev ${region}) + # | |--------| |----------| | + # | | (base) | |(pre-ext) | | + # | | daxX.1 | | daxX.1 | | + + echo "Checking if dev dax is spanning sparse extents" + ext_sum_length="$(($base_ext_length + $pre_ext_length))" + check_dax_dev ${dax_dev} $ext_sum_length + + test_remove_extents_in_use + + destroy_dax_dev ${dax_dev} + check_not_dax_dev ${region} ${dax_dev} + + # In-use extents were not released. Check they can be removed after the + # dax device is removed. + check_extent_cnt ${region} 2 + remove_extent ${device} $base_ext_dpa $base_ext_length + remove_extent ${device} $pre_ext_dpa $pre_ext_length + check_extent_cnt ${region} 0 + destroy_region ${region} + check_not_region ${region} +} + +test_inject_tag_support() +{ + echo "" + echo "Test: inject without/with tag" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + check_extent ${region} ${pre_ext_offset} ${pre_ext_length} + inject_extent ${device} $base_ext_dpa $base_ext_length "ira" + + # extent with tag should be rejected + check_extent_cnt ${region} 1 + remove_extent ${device} $pre_ext_dpa $pre_ext_length + check_extent_cnt ${region} 0 + + destroy_region ${region} + check_not_region ${region} +} + +test_partial_extent_remove () +{ + echo "" + echo "Test: partial extent remove" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + + inject_extent ${device} $base_ext_dpa $base_ext_length "" + # | ... |-------------------------------------------------------| + # | |--------| | + # | | (base) | | + + dax_dev=$(create_dax_dev ${region}) + + # | ... |-------------------------------------------------------| + # | |--------| | + # | | (base) | | + # | | daxX.1 | | + + partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))" + partial_ext_length="$(($base_ext_length / 2))" + echo "Removing Partial : $partial_ext_dpa $partial_ext_length" + + # | | |---| | + # Partial + + remove_extent ${device} $partial_ext_dpa $partial_ext_length + # In-use extents are not released. + check_extent_cnt ${region} 1 + + # | ... |-------------------------------------------------------| + # | |--------| | + # | | (base) | | + # | | daxX.1 | | + + destroy_dax_dev ${dax_dev} + check_not_dax_dev ${region} ${dax_dev} + + # | ... |-------------------------------------------------------| + # | |--------| | + # | | (base) | | + + # Partial results in whole extent removal + check_extent_cnt ${region} 1 + remove_extent ${device} $partial_ext_dpa $partial_ext_length + # | | |---| | + # Partial + check_extent_cnt ${region} 0 + + # | ... |-------------------------------------------------------| + destroy_region ${region} + check_not_region ${region} +} + +test_multiple_extent_remove () +{ + # Test multiple extent remove + echo "" + echo "Test: multiple extent remove with single extent remove command" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + inject_extent ${device} $base_ext_dpa $base_ext_length "" + # | ... |-------------------------------------------------------| + # | |--------| |-------------------| | + # | | (base) | | (pre)-existing | | + + check_extent_cnt ${region} 2 + dax_dev=$(create_dax_dev ${region}) + + # | ... |-------------------------------------------------------| + # | |--------| |-------------------| | + # | | (base) | | (pre)-existing | | + # | | daxX.1 | | daxX.1 | | + + partial_ext_dpa="$(($base_ext_dpa + ($base_ext_length / 2)))" + partial_ext_length="$(($pre_ext_dpa - $base_ext_dpa))" + echo "Removing multiple in span : $partial_ext_dpa $partial_ext_length" + # |------------------| + # Partial + remove_extent ${device} $partial_ext_dpa $partial_ext_length + + # | ... |-------------------------------------------------------| + # | |--------| |-------------------| | + # | | (base) | | (pre)-existing | | + # | | daxX.1 | | daxX.1 | | + + # In-use extents are not released. + check_extent_cnt ${region} 2 + + destroy_dax_dev ${dax_dev} + check_not_dax_dev ${region} ${dax_dev} + + # | ... |-------------------------------------------------------| + # | |--------| |-------------------| | + # | | (base) | | (pre)-existing | | + + # Remove both extents + check_extent_cnt ${region} 2 + # |------------------| + # Partial + remove_extent ${device} $partial_ext_dpa $partial_ext_length + # | ... |-------------------------------------------------------| + check_extent_cnt ${region} 0 + + destroy_region ${region} + check_not_region ${region} +} + +test_destroy_region_without_extent_removal () +{ + echo "" + echo "Test: Destroy region without extent removal" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + inject_extent ${device} $base_ext_dpa $base_ext_length "" + check_extent_cnt ${region} 2 + destroy_region ${region} + check_not_region ${region} +} + +test_destroy_with_extent_and_dax () +{ + echo "" + echo "Test: Destroy region with extents and dax devices" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + check_extent_cnt ${region} 0 + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | |----------| | + # | | |(pre-ext) | | + + check_extent_cnt ${region} 1 + dax_dev=$(create_dax_dev ${region}) + # | | | | | + check_dax_dev ${dax_dev} ${pre_ext_length} + destroy_region ${region} + check_not_region ${region} + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | | + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + check_extent_cnt ${region} 0 + destroy_region ${region} + check_not_region ${region} +} + +test_dax_device_ops () +{ + echo "" + echo "Test: Fail sparse dax dev creation without space" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | |-------------------| | + # | | | (pre)-existing | | + + check_extent_cnt ${region} 1 + + # | | | daxX.1 | | + + dax_dev=$(create_dax_dev ${region}) + check_dax_dev ${dax_dev} $pre_ext_length + fail_create_dax_dev ${region} + + echo "" + echo "Test: Resize sparse dax device" + echo "" + + # Shrink + # | | | daxX.1 | | + resize_ext_length=$(($pre_ext_length / 2)) + shrink_dax_dev ${dax_dev} $resize_ext_length + check_dax_dev ${dax_dev} $resize_ext_length + + # Fill + # | | | daxX.1 | daxX.2 | | + dax_dev=$(create_dax_dev ${region}) + check_dax_dev ${dax_dev} $resize_ext_length + destroy_region ${region} + check_not_region ${region} + + + # 2 extent + # create dax dev + # resize into 1st extent + # create dev on rest of 1st and all of second + # Ensure both devices are correct + + echo "" + echo "Test: Resize sparse dax device across extents" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + inject_extent ${device} $base_ext_dpa $base_ext_length "" + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | |--------| |-------------------| | + # | | (base) | | (pre)-existing | | + + check_extent_cnt ${region} 2 + dax_dev=$(create_dax_dev ${region}) + ext_sum_length="$(($base_ext_length + $pre_ext_length))" + + # | | daxX.1 | | daxX.1 | | + + check_dax_dev ${dax_dev} $ext_sum_length + resize_ext_length=33554432 # 32MB + + # | | D1 | | + + shrink_dax_dev ${dax_dev} $resize_ext_length + check_dax_dev ${dax_dev} $resize_ext_length + + # | | D1 | D2| | daxX.2 | | + + dax_dev=$(create_dax_dev ${region}) + remainder_length=$((ext_sum_length - $resize_ext_length)) + check_dax_dev ${dax_dev} $remainder_length + + # | | D1 | D2| | daxX.2 | | + + remainder_length=$((remainder_length / 2)) + shrink_dax_dev ${dax_dev} $remainder_length + check_dax_dev ${dax_dev} $remainder_length + + # | | D1 | D2| | daxX.2 | daxX.3 | | + + dax_dev=$(create_dax_dev ${region}) + check_dax_dev ${dax_dev} $remainder_length + destroy_region ${region} + check_not_region ${region} +} + +test_reject_overlapping () +{ + echo "" + echo "Test: Rejecting overlapping extents" + echo "" + + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + + # | 2G non- | DC region | + # | DC cap | | + # | ... |-------------------------------------------------------| + # | | |-------------------| | + # | | | (pre)-existing | | + + check_extent_cnt ${region} 1 + + # Attempt overlapping extent + # + # | | |-----------------| | + # | | | overlapping | | + + partial_ext_dpa="$(($base_ext_dpa + ($pre_ext_dpa / 2)))" + partial_ext_length=$pre_ext_length + inject_extent ${device} $partial_ext_dpa $partial_ext_length "" + + # Should only see the original ext + check_extent_cnt ${region} 1 + destroy_region ${region} + check_not_region ${region} +} + +test_two_regions() +{ + echo "" + echo "Test: create 2 regions in the same DC partition" + echo "" + region_size=$(($dc1_size / 2)) + region=$(create_dcd_region ${mem} ${decoder_dc1} ${region_size} dc1) + check_region ${region} ${region_size} + + region_two=$(create_dcd_region ${mem} ${decoder_dc1} ${region_size} dc1) + check_region ${region_two} ${region_size} + + destroy_region ${region_two} + check_not_region ${region_two} + destroy_region ${region} + check_not_region ${region} +} + +test_more_bit() +{ + echo "" + echo "Test: More bit" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" 1 + # More bit should hold off surfacing extent until the more bit is 0 + check_extent_cnt ${region} 0 + inject_extent ${device} $base_ext_dpa $base_ext_length "" + check_extent_cnt ${region} 2 + destroy_region ${region} + check_not_region ${region} +} + +test_driver_tear_down() +{ + echo "" + echo "Test: driver remove tear down" + echo "" + region=$(create_dcd_region ${mem} ${decoder}) + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + check_extent ${region} ${pre_ext_offset} ${pre_ext_length} + dax_dev=$(create_dax_dev ${region}) + # remove driver releases extents + modprobe -r dax_cxl + check_extent_cnt ${region} 0 +} + +test_driver_bring_up() +{ + # leave region up, driver removed. + echo "" + echo "Test: no driver inject ok" + echo "" + check_region ${region} ${dc0_size} + inject_extent ${device} $pre_ext_dpa $pre_ext_length "" + check_extent_cnt ${region} 1 + + modprobe dax_cxl + check_extent_cnt ${region} 1 + + destroy_region ${region} + check_not_region ${region} +} + +test_driver_reload() +{ + test_driver_tear_down + test_driver_bring_up +} + +test_event_reporting() +{ + # Test event reporting + # results expected + num_dcd_events_expected=2 + + echo "Test: Prep event trace" + echo "" > /sys/kernel/tracing/trace + echo 1 > /sys/kernel/tracing/events/cxl/enable + echo 1 > /sys/kernel/tracing/tracing_on + + inject_extent ${device} $base_ext_dpa $base_ext_length "" + remove_extent ${device} $base_ext_dpa $base_ext_length + + echo 0 > /sys/kernel/tracing/tracing_on + + echo "Test: Events seen" + trace_out=$(cat /sys/kernel/tracing/trace) + + # Look for DCD events + num_dcd_events=$(grep -c "cxl_dynamic_capacity" <<< "${trace_out}") + echo " LOG (Expected) : (Found)" + echo " DCD events ($num_dcd_events_expected) : $num_dcd_events" + + if [ "$num_dcd_events" -ne $num_dcd_events_expected ]; then + err "$LINENO" + fi +} + + +# ======================================================================== +# main() +# ======================================================================== + +modprobe -r cxl_test +modprobe cxl_test + +readarray -t memdevs < <("$CXL" list -b cxl_test -Mi | jq -r '.[].memdev') + +for mem in ${memdevs[@]}; do + dc0_size=$($CXL list -m $mem | jq -r '.[].dc0_size') + if [ "$dc0_size" == "null" ]; then + continue + fi + dc1_size=$($CXL list -m $mem | jq -r '.[].dc1_size') + if [ "$dc1_size" == "null" ]; then + continue + fi + decoder=$($CXL list -b cxl_test -D -d root -m "$mem" | + jq -r ".[] | + select(.dc0_capable == true) | + select(.nr_targets == 1) | + select(.max_available_extent >= ${dc0_size}) | + .decoder") + decoder_dc1=$($CXL list -b cxl_test -D -d root -m "$mem" | + jq -r ".[] | + select(.dc1_capable == true) | + select(.nr_targets == 1) | + select(.max_available_extent >= ${dc1_size}) | + .decoder") + if [[ $decoder ]]; then + bus=`"$CXL" list -b cxl_test -m ${mem} | jq -r '.[].bus'` + device=$($CXL list -m $mem | jq -r '.[].host') + break + fi +done + +echo "TEST: DCD test device bus:${bus} decoder:${decoder} mem:${mem} device:${device} size:${dc0_size}" + +if [ "$decoder" == "" ] || [ "$device" == "" ] || [ "$dc0_size" == "" ]; then + echo "No mem device/decoder found with DCD support" + exit 77 +fi + +if [ "$decoder_dc1" == "" ]; then + echo "insufficient DC capability for ${mem}/${device}" + exit 77 +fi + +# testing pre existing extents must be called first as the extents were created +# by cxl-test being loaded +test_pre_existing_extents +test_remove_extent_under_dax_device +test_create_dax_dev_spanning_two_extents +test_inject_tag_support +test_partial_extent_remove +test_multiple_extent_remove +test_destroy_region_without_extent_removal +test_destroy_with_extent_and_dax +test_dax_device_ops +test_reject_overlapping +test_two_regions +test_more_bit +test_driver_reload +test_event_reporting + +modprobe -r cxl_test + +check_dmesg "$LINENO" + +exit 0 diff --git a/test/meson.build b/test/meson.build index d871e28e17ce512cd1e7b43f3ec081729fe5e03a..1cfcb60d16e05272893ae1c67820aa8614281505 100644 --- a/test/meson.build +++ b/test/meson.build @@ -161,6 +161,7 @@ cxl_sanitize = find_program('cxl-sanitize.sh') cxl_destroy_region = find_program('cxl-destroy-region.sh') cxl_qos_class = find_program('cxl-qos-class.sh') cxl_poison = find_program('cxl-poison.sh') +cxl_dcd = find_program('cxl-dcd.sh') tests = [ [ 'libndctl', libndctl, 'ndctl' ], @@ -194,6 +195,7 @@ tests = [ [ 'cxl-destroy-region.sh', cxl_destroy_region, 'cxl' ], [ 'cxl-qos-class.sh', cxl_qos_class, 'cxl' ], [ 'cxl-poison.sh', cxl_poison, 'cxl' ], + [ 'cxl-dcd.sh', cxl_dcd, 'cxl' ], ] if get_option('destructive').enabled()