From patchwork Wed Sep 25 02:46:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13811554 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0869F148FF5; Wed, 25 Sep 2024 02:47:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232434; cv=none; b=O2hVggE7U/QyCEvJXfwRxNpz2RpoGlmxu1raBUfGdcEXfvrkKOYsIWprapt1KR+srvbhEfi/SXQawPkxiJs0S7Y4kJLUHcE0NWGotC1ai156owYTYz30bA1obuGYB9IdIAzY3hcmIEWEDH+IS2A/BC9ib6W2q214KOuw3tnT4s8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232434; c=relaxed/simple; bh=KaE7JObUOq0Hcir5tFk/BG6islLlrFQy8Kf+EuSeDvI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Vw5obGzsu832Jhm2fzafqiGxqXzp7b+iwVlSoH49yZVYXGzXqfx0QEQfCSQM6tFOo6/hhMz+pac31OOCYCCtlCfIRMiipnvaTjCUXK0HkGqPPNrvxrFjWZTCOoxHJsphN/32ogZuSF7J7E/3ILkwRk2X5eXc7bZgq5EocpEG6vg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Qv8/0POU; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Qv8/0POU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727232432; x=1758768432; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KaE7JObUOq0Hcir5tFk/BG6islLlrFQy8Kf+EuSeDvI=; b=Qv8/0POU7ce61vC3gni/NTpbxhuLmtJrk1kO7VzuNB8tc1npxd27GVo6 F8HF7YVVtsbFeFl63Pnz4gE3Ntvh56ORW9VXxyY/lYbYjp+6OLfZh44+Q hKor7TOPKWFw9wVECs1vg+q511HFIyIrENSdpehj032Q5O2w0/752D6fT UyQFw/h//YatvGQqUCxhyRezyYw+jNOdNTuvtWIntMQ3JRI/YT4wMZn+M /VWZF2VZqC60+vypq59r6t0VQadWpuRQyLoV8ZN5QsnmJzldpJ6qlMVO2 L7JZBkdPXb2qnw3A4mansB4n1YHuWhbIBG4pr34/NC/UgBiX8l6QK031O w==; X-CSE-ConnectionGUID: 7b5VOZPkRP6FxzMdSW2AEA== X-CSE-MsgGUID: u3JKQ+VpSkWmQdoVFfPf6Q== X-IronPort-AV: E=McAfee;i="6700,10204,11205"; a="26132026" X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="26132026" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:12 -0700 X-CSE-ConnectionGUID: SSEpI4rNQSmayvsP5CKZtg== X-CSE-MsgGUID: 1Jpq3+vxR+q0bv6wWzRlqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="102388969" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:09 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Jonathan Cameron , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero Subject: [RFC 1/5] cxl: Rename ACPI_CEDT_CFMWS_RESTRICT_TYPE2/TYPE3 Date: Wed, 25 Sep 2024 10:46:43 +0800 Message-Id: <20240925024647.46735-2-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240925024647.46735-1-ying.huang@intel.com> References: <20240925024647.46735-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to the description of the "Window Restrictions" field of "CFMWS Structure" in the CXL spec v3.1 section 9.18.1.3: CXL Fixed Memory Window Structure (CFMWS), the bit 0 of "Window Restrictions" is formerly known as "CXL Type 2 Memory" and renamed to "Device Coherent", while the bit 1 is formerly known as "CXL Type 3 Memory" and renamed to "Host-only Coherent". Because type 3 memory can only be host-only coherent before, while it can be host-only coherent or device coherent with "Back-Invalidate" now. To avoid confusing about type 3 memory and host-only coherent in Linux kernel, we rename corresponding bit definition from ACPI_CEDT_CFMWS_RESTRICT_TYPE2/TYPE3 to ACPI_CEDT_CFMWS_RESTRICT_DEVCOH/HOSTONLYCOH. This makes the kernel code consistent with the spec too. Also rename the corresponding cxl_decoder flags CXL_DECODER_F_TYPE2/TYPE3 to CXL_DECODER_F_DEVCOH/HOSTONLYCOH. No functionality change is expected, because we just rename the flags constant definition. Signed-off-by: "Huang, Ying" Suggested-by: Jonathan Cameron Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Reviewed-by: Davidlohr Bueso Reviewed-by: Gregory Price --- drivers/cxl/acpi.c | 8 ++++---- drivers/cxl/core/port.c | 8 ++++---- drivers/cxl/cxl.h | 14 +++++++------- include/acpi/actbl1.h | 10 +++++----- tools/testing/cxl/test/cxl.c | 18 +++++++++--------- 5 files changed, 29 insertions(+), 29 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 82b78e331d8e..3115f246273b 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -115,10 +115,10 @@ static unsigned long cfmws_to_decoder_flags(int restrictions) { unsigned long flags = CXL_DECODER_F_ENABLE; - if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_TYPE2) - flags |= CXL_DECODER_F_TYPE2; - if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_TYPE3) - flags |= CXL_DECODER_F_TYPE3; + if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_DEVCOH) + flags |= CXL_DECODER_F_DEVCOH; + if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH) + flags |= CXL_DECODER_F_HOSTONLYCOH; if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_VOLATILE) flags |= CXL_DECODER_F_RAM; if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_PMEM) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1d5007e3795a..67a8dc4d7868 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -125,8 +125,8 @@ static DEVICE_ATTR_RO(name) CXL_DECODER_FLAG_ATTR(cap_pmem, CXL_DECODER_F_PMEM); CXL_DECODER_FLAG_ATTR(cap_ram, CXL_DECODER_F_RAM); -CXL_DECODER_FLAG_ATTR(cap_type2, CXL_DECODER_F_TYPE2); -CXL_DECODER_FLAG_ATTR(cap_type3, CXL_DECODER_F_TYPE3); +CXL_DECODER_FLAG_ATTR(cap_type2, CXL_DECODER_F_DEVCOH); +CXL_DECODER_FLAG_ATTR(cap_type3, CXL_DECODER_F_HOSTONLYCOH); CXL_DECODER_FLAG_ATTR(locked, CXL_DECODER_F_LOCK); static ssize_t target_type_show(struct device *dev, @@ -326,14 +326,14 @@ static struct attribute *cxl_decoder_root_attrs[] = { static bool can_create_pmem(struct cxl_root_decoder *cxlrd) { - unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_PMEM; + unsigned long flags = CXL_DECODER_F_HOSTONLYCOH | CXL_DECODER_F_PMEM; return (cxlrd->cxlsd.cxld.flags & flags) == flags; } static bool can_create_ram(struct cxl_root_decoder *cxlrd) { - unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + unsigned long flags = CXL_DECODER_F_HOSTONLYCOH | CXL_DECODER_F_RAM; return (cxlrd->cxlsd.cxld.flags & flags) == flags; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9afb407d438f..28c8783d3c14 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -315,13 +315,13 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, * Additionally indicate whether decoder settings were autodetected, * user customized. */ -#define CXL_DECODER_F_RAM BIT(0) -#define CXL_DECODER_F_PMEM BIT(1) -#define CXL_DECODER_F_TYPE2 BIT(2) -#define CXL_DECODER_F_TYPE3 BIT(3) -#define CXL_DECODER_F_LOCK BIT(4) -#define CXL_DECODER_F_ENABLE BIT(5) -#define CXL_DECODER_F_MASK GENMASK(5, 0) +#define CXL_DECODER_F_RAM BIT(0) +#define CXL_DECODER_F_PMEM BIT(1) +#define CXL_DECODER_F_DEVCOH BIT(2) +#define CXL_DECODER_F_HOSTONLYCOH BIT(3) +#define CXL_DECODER_F_LOCK BIT(4) +#define CXL_DECODER_F_ENABLE BIT(5) +#define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { CXL_DECODER_DEVMEM = 2, diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index 841ef9f22795..2b38455e0f13 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -551,11 +551,11 @@ struct acpi_cedt_cfmws_target_element { /* Values for Restrictions field above */ -#define ACPI_CEDT_CFMWS_RESTRICT_TYPE2 (1) -#define ACPI_CEDT_CFMWS_RESTRICT_TYPE3 (1<<1) -#define ACPI_CEDT_CFMWS_RESTRICT_VOLATILE (1<<2) -#define ACPI_CEDT_CFMWS_RESTRICT_PMEM (1<<3) -#define ACPI_CEDT_CFMWS_RESTRICT_FIXED (1<<4) +#define ACPI_CEDT_CFMWS_RESTRICT_DEVCOH (1) +#define ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH (1<<1) +#define ACPI_CEDT_CFMWS_RESTRICT_VOLATILE (1<<2) +#define ACPI_CEDT_CFMWS_RESTRICT_PMEM (1<<3) +#define ACPI_CEDT_CFMWS_RESTRICT_FIXED (1<<4) /* 2: CXL XOR Interleave Math Structure */ diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 90d5afd52dd0..3982d292d286 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -209,7 +209,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -224,7 +224,7 @@ static struct { }, .interleave_ways = 1, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -239,7 +239,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -254,7 +254,7 @@ static struct { }, .interleave_ways = 1, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -269,7 +269,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -284,7 +284,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M, @@ -301,7 +301,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -317,7 +317,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 1, .granularity = 0, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -333,7 +333,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 2, .granularity = 0, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 16UL, From patchwork Wed Sep 25 02:46:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13811555 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E68614A4D0; Wed, 25 Sep 2024 02:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232437; cv=none; b=unGZJAesjKp30wl4VHaueiA7BNkIPrfdm52+2c7fF/k+NfXP0irIoBKxfJMavJB+zcm07fIwpvEwlEJKq4d30Sq7dCnPDsvZLuMNoHceU4rcEYnzXtmbBjAENi2OISo5D0X9HPo5k2aylDkdYQZMYGDxlhSQYW/9+CNbI4sIG9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232437; c=relaxed/simple; bh=m4ookucFEI6FwnJQ40I6imhYRVPguRTN6n1jzM6Aa0s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pz/Yv04FYJ3aX299AlAts5cjTR02aCxtASffMz/V8mGdf8RMUUa3P7AyUmB5g0Ixz9iTZ+QdqrvozQetrahAI70e2n+xQTUGdqTFqftdnUEppAkAz1SDQnY4ORkXOf0PKeAiwKwDs+OErGWvccOs83H0nRK8V7co/vfedtpab4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MEpaEu88; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MEpaEu88" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727232436; x=1758768436; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m4ookucFEI6FwnJQ40I6imhYRVPguRTN6n1jzM6Aa0s=; b=MEpaEu88f/ll52LhdUxU1KKUz6RWCPBVWLB2OYkRdathoV1eb/TBo0AX 3quqEesLq3tDvhs/1Ehl1Yd6jTzcFFyAl+/EDROhc3KwtcD0MZ2Bx/wTt UaGzwq0Fy49tKGkvVVdGsikkGVK8ArLs/qh4yYrzTnn43UO0sFJ2A8A6h WPWgcdRM3+X+HLmK//waB75cVG+pVjuMqt4JBKpBwXcylh+mX8o/pd6oF npuNCHRJBnUhqHiNSipht2l4GZ/pXezBnFoKK1/sl3dMC0WrLOxWmCrPH eV1uM4MGGHii49oOn1RnN++aPAJJyfZN3xNFixEF9WDwh1vOSHsKQVjKU Q==; X-CSE-ConnectionGUID: cxZxsGuISFO19hkJxKye3A== X-CSE-MsgGUID: EEYX+r9XSZmYMz8prswKbw== X-IronPort-AV: E=McAfee;i="6700,10204,11205"; a="26132043" X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="26132043" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:15 -0700 X-CSE-ConnectionGUID: qZl88NFBTcO8pjcd3NZHSw== X-CSE-MsgGUID: zQoAT6HISImexiM+KQgT1A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="102388982" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:12 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Jonathan Cameron , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero Subject: [RFC 2/5] cxl: Rename CXL_DECODER_HOSTONLYMEM/DEVMEM Date: Wed, 25 Sep 2024 10:46:44 +0800 Message-Id: <20240925024647.46735-3-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240925024647.46735-1-ying.huang@intel.com> References: <20240925024647.46735-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Previously, CXL type3 devices (memory expanders) use hostonly coherence (HDM-H), while CXL type2 devices (accelerators) use dev coherence (HDM-D). So the target device type of a cxl decoder is named as CXL_DECODER_HOSTONLYMEM for type3 devices and CXL_DECODER_DEVMEM for type2 devices. However, this isn't true anymore. CXL type3 devices can use dev coherence + back invalidation (HDM-DB) too. To avoid confusing between the device type and coherence, in this patch, CXL_DECODER_HOSTONLYMEM/DEVMEM is renamed to CXL_DECODER_EXPANDER/ACCEL. No functionality change is expected in this patch. Signed-off-by: "Huang, Ying" Cc: Jonathan Cameron Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Reviewed-by: Davidlohr Bueso Reviewed-by: Gregory Price --- drivers/cxl/acpi.c | 2 +- drivers/cxl/core/hdm.c | 16 ++++++++-------- drivers/cxl/core/port.c | 6 +++--- drivers/cxl/core/region.c | 2 +- drivers/cxl/cxl.h | 4 ++-- tools/testing/cxl/test/cxl.c | 6 +++--- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 3115f246273b..21486e471305 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -382,7 +382,7 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, cxld = &cxlrd->cxlsd.cxld; cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions); - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = cfmws->base_hpa, .end = cfmws->base_hpa + cfmws->window_size - 1, diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..57b54ecdb000 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -572,7 +572,7 @@ static void cxld_set_interleave(struct cxl_decoder *cxld, u32 *ctrl) static void cxld_set_type(struct cxl_decoder *cxld, u32 *ctrl) { u32p_replace_bits(ctrl, - !!(cxld->target_type == CXL_DECODER_HOSTONLYMEM), + !!(cxld->target_type == CXL_DECODER_EXPANDER), CXL_HDM_DECODER0_CTRL_HOSTONLY); } @@ -771,7 +771,7 @@ static int cxl_setup_hdm_decoder_from_dvsec( if (!len) return -ENOENT; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->commit = NULL; cxld->reset = NULL; cxld->hpa_range = info->dvsec_range[which]; @@ -847,9 +847,9 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; else - cxld->target_type = CXL_DECODER_DEVMEM; + cxld->target_type = CXL_DECODER_ACCEL; guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -876,16 +876,16 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, * more precision. */ if (cxlds->type == CXL_DEVTYPE_CLASSMEM) - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; else - cxld->target_type = CXL_DECODER_DEVMEM; + cxld->target_type = CXL_DECODER_ACCEL; } else { /* To be overridden by region type at commit time */ - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; } if (!FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl) && - cxld->target_type == CXL_DECODER_HOSTONLYMEM) { + cxld->target_type == CXL_DECODER_EXPANDER) { ctrl |= CXL_HDM_DECODER0_CTRL_HOSTONLY; writel(ctrl, hdm + CXL_HDM_DECODER0_CTRL_OFFSET(which)); } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 67a8dc4d7868..47ad6d9329db 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -135,9 +135,9 @@ static ssize_t target_type_show(struct device *dev, struct cxl_decoder *cxld = to_cxl_decoder(dev); switch (cxld->target_type) { - case CXL_DECODER_DEVMEM: + case CXL_DECODER_ACCEL: return sysfs_emit(buf, "accelerator\n"); - case CXL_DECODER_HOSTONLYMEM: + case CXL_DECODER_EXPANDER: return sysfs_emit(buf, "expander\n"); } return -ENXIO; @@ -1768,7 +1768,7 @@ static int cxl_decoder_init(struct cxl_port *port, struct cxl_decoder *cxld) /* Pre initialize an "empty" decoder */ cxld->interleave_ways = 1; cxld->interleave_granularity = PAGE_SIZE; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = 0, .end = -1, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21ad5f242875..8229e8a0072d 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2545,7 +2545,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } - return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM); + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 28c8783d3c14..55b8c32f8d72 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -324,8 +324,8 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, #define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { - CXL_DECODER_DEVMEM = 2, - CXL_DECODER_HOSTONLYMEM = 3, + CXL_DECODER_ACCEL = 2, + CXL_DECODER_EXPANDER = 3, }; /* diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 3982d292d286..352a62c745c6 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -724,7 +724,7 @@ static void default_mock_decoder(struct cxl_decoder *cxld) cxld->interleave_ways = 1; cxld->interleave_granularity = 256; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->commit = mock_decoder_commit; cxld->reset = mock_decoder_reset; } @@ -798,7 +798,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld) cxld->interleave_ways = 2; eig_to_granularity(window->granularity, &cxld->interleave_granularity); - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->flags = CXL_DECODER_F_ENABLE; cxled->state = CXL_DECODER_STATE_AUTO; port->commit_end = cxld->id; @@ -831,7 +831,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld) } else cxlsd->target[0] = dport; cxld = &cxlsd->cxld; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->flags = CXL_DECODER_F_ENABLE; iter->commit_end = 0; /* From patchwork Wed Sep 25 02:46:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13811556 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C506E14AD32; Wed, 25 Sep 2024 02:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232440; cv=none; b=nus475/NYHtzFHtQuqk5cB6/ubJXUiROHO60apMVOJIwPJtQZiROL/KAQ3LYftUozwBTeF1V+FOmE2+1GTVxLgcJndZpoAzuIvxAmIHfqUhf13f2/eJsYG6kVNNDtLmYuotdnObpaacei9drK0DAHkXxMu9/k3YOkj2t2ziRVTU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232440; c=relaxed/simple; bh=KR5JtIBjk9XQKtGLonFY5l2vNf8jHSzObyuuvpIkJNI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cHW5RxbCKMWREwJ9kP2XO+0mRjcSz/fQkQzjK+recWz+7U/v8p6g0q99qlSjOFjg3LyrrACnmjRbHo9KJEvImO5/kYDwQHlN7a/kQldRfs2ouSbEUwMhq99i1JZL7YbqTcjxefZLx8JOiFjDlBLGIBhko8KxkfjWVdb4EICrdFg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KOLhb5mw; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KOLhb5mw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727232439; x=1758768439; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KR5JtIBjk9XQKtGLonFY5l2vNf8jHSzObyuuvpIkJNI=; b=KOLhb5mwBcxAZ+jaguVrbrz2NlCjsiaWkjiam4TRSEZ392oLymhJLkui V8N/GlUUB/uUOaK7w83xK9NF5YbwMASMRqivzDGpm4pIhyNMDOI7I6UvE 19X26F1521EGCx0w54/4VbEz+SphwPqt7KG0vyOVoh20qqQtcVjteLpSO Y/0zNZ6aI7B9D8nAVe82sszoYqHniXBAioBZn11iQ3Jlr7gItPVJU44En dfPq9jNCgsrzoaZ2vzLSZV41pIII4scLVMEeKjJekKpG7h+OHbUDDN6G4 5pJxFprAHu2u7CAYb1suSJ0M3b45kO4vGOc9L07CWtMovy0UkSaixTlFv w==; X-CSE-ConnectionGUID: Sd/ISAdsRUmJRnwcVNAdpQ== X-CSE-MsgGUID: 2hJW1xw2RjesuEne2fMzhQ== X-IronPort-AV: E=McAfee;i="6700,10204,11205"; a="26132055" X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="26132055" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:19 -0700 X-CSE-ConnectionGUID: 8gzOwi7XS3qjgWvHfYU97w== X-CSE-MsgGUID: ZvikI7uxTWuV9Dyc4rNCrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="102389003" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:16 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Jonathan Cameron , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero Subject: [RFC 3/5] cxl: Separate coherence from target type Date: Wed, 25 Sep 2024 10:46:45 +0800 Message-Id: <20240925024647.46735-4-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240925024647.46735-1-ying.huang@intel.com> References: <20240925024647.46735-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Previously, target type (expander or accelerator) and coherence (HOSTONLY (HDM-H) or DEV (HDM-D/DB)) are synonym. So target type is used to designate coherence too. However, it's possible for expanders to use HDM-DB now. So, we need to separate coherence from target type. Accordingly, the HOSTONLY field of decoder ctrl register (CXL_HDM_DECODER0_CTRL_HOSTONLY) need to be set according to coherence. The coherence of decoders can not be determined via target type too. So, accelerator/expander device drivers need to specify coherence explicitly via newly added coherence field in struct cxl_dev_state. The coherence of each end points in a region need to be same. So, the coherence of the first end point is recorded in struct region. Which will be checked against the coherence of all other end points. Signed-off-by: "Huang, Ying" Cc: Jonathan Cameron Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Reviewed-by: Gregory Price --- drivers/cxl/core/hdm.c | 22 +++++++++++++++------- drivers/cxl/core/mbox.c | 1 + drivers/cxl/core/port.c | 1 + drivers/cxl/core/region.c | 37 ++++++++++++++++++++++++++++++++++--- drivers/cxl/cxl.h | 9 +++++++++ drivers/cxl/cxlmem.h | 11 +++++++++++ 6 files changed, 71 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 57b54ecdb000..478fb6691759 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -569,10 +569,10 @@ static void cxld_set_interleave(struct cxl_decoder *cxld, u32 *ctrl) *ctrl |= CXL_HDM_DECODER0_CTRL_COMMIT; } -static void cxld_set_type(struct cxl_decoder *cxld, u32 *ctrl) +static void cxld_set_coherence(struct cxl_decoder *cxld, u32 *ctrl) { u32p_replace_bits(ctrl, - !!(cxld->target_type == CXL_DECODER_EXPANDER), + !!(cxld->coherence == CXL_DECODER_HOSTONLYCOH), CXL_HDM_DECODER0_CTRL_HOSTONLY); } @@ -667,7 +667,7 @@ static int cxl_decoder_commit(struct cxl_decoder *cxld) /* common decoder settings */ ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); cxld_set_interleave(cxld, &ctrl); - cxld_set_type(cxld, &ctrl); + cxld_set_coherence(cxld, &ctrl); base = cxld->hpa_range.start; size = range_len(&cxld->hpa_range); @@ -846,10 +846,13 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, cxld->flags |= CXL_DECODER_F_ENABLE; if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; - if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) { cxld->target_type = CXL_DECODER_EXPANDER; - else + cxld->coherence = CXL_DECODER_HOSTONLYCOH; + } else { cxld->target_type = CXL_DECODER_ACCEL; + cxld->coherence = CXL_DECODER_DEVCOH; + } guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -879,13 +882,18 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, cxld->target_type = CXL_DECODER_EXPANDER; else cxld->target_type = CXL_DECODER_ACCEL; + if (cxlds->coherence == CXL_DEVCOH_HOSTONLY) + cxld->coherence = CXL_DECODER_HOSTONLYCOH; + else + cxld->coherence = CXL_DECODER_DEVCOH; } else { - /* To be overridden by region type at commit time */ + /* To be overridden by region type/coherence at commit time */ cxld->target_type = CXL_DECODER_EXPANDER; + cxld->coherence = CXL_DECODER_HOSTONLYCOH; } if (!FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl) && - cxld->target_type == CXL_DECODER_EXPANDER) { + cxld->coherence == CXL_DECODER_HOSTONLYCOH) { ctrl |= CXL_HDM_DECODER0_CTRL_HOSTONLY; writel(ctrl, hdm + CXL_HDM_DECODER0_CTRL_OFFSET(which)); } diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index e5cdeafdf76e..3635a0a402b7 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1424,6 +1424,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mds->cxlds.reg_map.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; + mds->cxlds.coherence = CXL_DEVCOH_HOSTONLY; mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID; mds->pmem_perf.qos_class = CXL_QOS_CLASS_INVALID; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 47ad6d9329db..2dee78e9b90c 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1769,6 +1769,7 @@ static int cxl_decoder_init(struct cxl_port *port, struct cxl_decoder *cxld) cxld->interleave_ways = 1; cxld->interleave_granularity = PAGE_SIZE; cxld->target_type = CXL_DECODER_EXPANDER; + cxld->coherence = CXL_DECODER_HOSTONLYCOH; cxld->hpa_range = (struct range) { .start = 0, .end = -1, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 8229e8a0072d..cec7d08b6f44 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1005,9 +1005,10 @@ static int cxl_rr_alloc_decoder(struct cxl_port *port, struct cxl_region *cxlr, } /* - * Endpoints should already match the region type, but backstop that - * assumption with an assertion. Switch-decoders change mapping-type - * based on what is mapped when they are assigned to a region. + * Endpoints should already match the region type/coherence, but + * backstop that assumption with an assertion. Switch-decoders change + * mapping-type/coherence based on what is mapped when they are assigned + * to a region. */ dev_WARN_ONCE(&cxlr->dev, port == cxled_to_port(cxled) && @@ -1016,6 +1017,13 @@ static int cxl_rr_alloc_decoder(struct cxl_port *port, struct cxl_region *cxlr, dev_name(&cxled_to_memdev(cxled)->dev), dev_name(&cxld->dev), cxld->target_type, cxlr->type); cxld->target_type = cxlr->type; + dev_WARN_ONCE(&cxlr->dev, + port == cxled_to_port(cxled) && + cxld->coherence != cxlr->coherence, + "%s:%s mismatch decoder coherence %d -> %d\n", + dev_name(&cxled_to_memdev(cxled)->dev), + dev_name(&cxld->dev), cxld->coherence, cxlr->coherence); + cxld->coherence = cxlr->coherence; cxl_rr->decoder = cxld; return 0; } @@ -1925,6 +1933,29 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } + /* Set the coherence of region to that of the first endpoint */ + if (cxlr->coherence == CXL_DECODER_INVALIDCOH) { + unsigned long flags = cxlrd->cxlsd.cxld.flags; + enum cxl_decoder_coherence coherence = cxled->cxld.coherence; + + cxlr->coherence = coherence; + if ((coherence == CXL_DECODER_HOSTONLYCOH && + !(flags & CXL_DECODER_F_HOSTONLYCOH)) || + (coherence == CXL_DECODER_DEVCOH && + !(flags & CXL_DECODER_F_DEVCOH))) { + dev_dbg(&cxlr->dev, +"%s:%s endpoint coherence: %d isn't supported by root decoder: %#lx\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + coherence, flags); + return -ENXIO; + } + } else if (cxled->cxld.coherence != cxlr->coherence) { + dev_dbg(&cxlr->dev, "%s:%s coherence mismatch: %d vs %d\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + cxled->cxld.coherence, cxlr->coherence); + return -ENXIO; + } + if (!cxled->dpa_res) { dev_dbg(&cxlr->dev, "%s:%s: missing DPA allocation.\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev)); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 55b8c32f8d72..99398c868d82 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -328,6 +328,12 @@ enum cxl_decoder_type { CXL_DECODER_EXPANDER = 3, }; +enum cxl_decoder_coherence { + CXL_DECODER_INVALIDCOH, + CXL_DECODER_HOSTONLYCOH, + CXL_DECODER_DEVCOH, +}; + /* * Current specification goes up to 8, double that seems a reasonable * software max for the foreseeable future @@ -356,6 +362,7 @@ struct cxl_decoder { int interleave_ways; int interleave_granularity; enum cxl_decoder_type target_type; + enum cxl_decoder_coherence coherence; struct cxl_region *region; unsigned long flags; int (*commit)(struct cxl_decoder *cxld); @@ -517,6 +524,7 @@ struct cxl_region_params { * @id: This region's id. Id is globally unique across all regions * @mode: Endpoint decoder allocation / access mode * @type: Endpoint decoder target type + * @coherence: Endpoint decoder coherence * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge * @flags: Region state flags @@ -530,6 +538,7 @@ struct cxl_region { int id; enum cxl_decoder_mode mode; enum cxl_decoder_type type; + enum cxl_decoder_coherence coherence; struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_pmem_region *cxlr_pmem; unsigned long flags; diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index afb53d058d62..cc4880286134 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -393,6 +393,16 @@ enum cxl_devtype { CXL_DEVTYPE_CLASSMEM, }; +/* + * enum cxl_devcoherence - the coherence of the cxl device + * @CXL_DEVCOH_DEV - HDM-D or HDM-DB + * @CXL_DEVCOH_HOSTONLY - HDM-H + */ +enum cxl_devcoherence { + CXL_DEVCOH_DEV, + CXL_DEVCOH_HOSTONLY, +}; + /** * struct cxl_dpa_perf - DPA performance property entry * @dpa_range: range for DPA address @@ -438,6 +448,7 @@ struct cxl_dev_state { struct resource ram_res; u64 serial; enum cxl_devtype type; + enum cxl_devcoherence coherence; }; /** From patchwork Wed Sep 25 02:46:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13811557 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3879914B955; Wed, 25 Sep 2024 02:47:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232444; cv=none; b=sXL46e+DFNF4NkMRl3t2Zf9T39pU8t52jogGvaR1TMcWuaKBLQM8pI02kbZ9Lbc3+KVxcZfQctBa8E5862TZp2sfIpEoPVr3qW4/OHmsP5MyBGW28Yj9Euin/0/6c8YRuwUHvlMIns2Rc1J9dxV7EN21bNfBYuxDa1dAqTRVOyk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232444; c=relaxed/simple; bh=KJXo8CquvIp6jpXXQM1ww2sW6uLXMf0+6SQWjmsIFBU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SlSR4YXgLn6UnZwFi1of2b3h8K2nfmuv6CAwq/dXmt9HpvpSlyEhQ9/En9UC/rD7Be5NFZtXnGldxtXvveMlafF7mCb00wem0ftyN6KJQkhBfTXFVDCu6lEPoJOUeWMfeg5gfJXhxrjSll0QfueFN6RYK7I6dT4jOnup4AYckLo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iy3bP5UL; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iy3bP5UL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727232442; x=1758768442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KJXo8CquvIp6jpXXQM1ww2sW6uLXMf0+6SQWjmsIFBU=; b=iy3bP5ULHJkyEI3LSyu24+ZIJp+u5F0gE3NFJahReZu8f0hCAHb//4zh CMu2Rqzok6tgJ1UfXOh0+LKlMiFnid5mnCDS4XYWe33b2Jb3HAII/g7nf uRSAJokIihuLVqC8ovrdNqb2WcbjD7KzIq7QHdAG5Xl7zGfmuCR74GrFV /UD7LMEXrY9dc2ezFdzaO14qHj7k2pDyl6XTa6mlOx/zNQpNQwIX1kBPx btdDOqo/xjMiG7KgcnFROhGLsxYuSaGjHIemwRp6NNjtC40+LorpvBkdd fmz3ChdKi09EP+QUAqgJrbTg6TSsATSPcl9d5zCOGEfsEpWiwENmWfmGh w==; X-CSE-ConnectionGUID: +AG9weyuTbexeDTVjkiR8w== X-CSE-MsgGUID: NVupepx/QZS8sGyCtQcqPw== X-IronPort-AV: E=McAfee;i="6700,10204,11205"; a="26132067" X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="26132067" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:22 -0700 X-CSE-ConnectionGUID: QRzszYR1TuCJBAgyXM6yNg== X-CSE-MsgGUID: XYpsNtclRxqCawOIFDuwSQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="102389036" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:19 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Jonathan Cameron , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero Subject: [RFC 4/5] cxl: Set type of region to that of the first endpoint Date: Wed, 25 Sep 2024 10:46:46 +0800 Message-Id: <20240925024647.46735-5-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240925024647.46735-1-ying.huang@intel.com> References: <20240925024647.46735-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The type of region is hard-coded as type 3 expander now, because this is the only supported device type. As a preparation to support type 2 accelerators, we set the type of region to that of the first endpoint. Then, we will check whether the type of region is same as the type of other endpoints of the region. Because what we really need is to make sure the type of all endpoints of a region is same. The target type of endpoint devices comes from expander/accelerator device drivers via struct cxl_dev_state. Signed-off-by: "Huang, Ying" Cc: Jonathan Cameron Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Reviewed-by: Gregory Price --- drivers/cxl/acpi.c | 1 - drivers/cxl/core/hdm.c | 28 +++++++++++++--------------- drivers/cxl/core/port.c | 2 ++ drivers/cxl/core/region.c | 13 +++++++------ drivers/cxl/cxl.h | 1 + 5 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 21486e471305..29c2a44b122c 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -382,7 +382,6 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, cxld = &cxlrd->cxlsd.cxld; cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions); - cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = cfmws->base_hpa, .end = cfmws->base_hpa + cfmws->window_size - 1, diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 478fb6691759..c9accf8be71f 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -841,18 +841,25 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, .end = base + size - 1, }; + if (cxled) { + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + if (cxlds->type == CXL_DEVTYPE_CLASSMEM) + cxld->target_type = CXL_DECODER_EXPANDER; + else + cxld->target_type = CXL_DECODER_ACCEL; + } + /* decoders are enabled if committed */ if (committed) { cxld->flags |= CXL_DECODER_F_ENABLE; if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; - if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) { - cxld->target_type = CXL_DECODER_EXPANDER; + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) cxld->coherence = CXL_DECODER_HOSTONLYCOH; - } else { - cxld->target_type = CXL_DECODER_ACCEL; + else cxld->coherence = CXL_DECODER_DEVCOH; - } guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -874,21 +881,12 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); struct cxl_dev_state *cxlds = cxlmd->cxlds; - /* - * Default by devtype until a device arrives that needs - * more precision. - */ - if (cxlds->type == CXL_DEVTYPE_CLASSMEM) - cxld->target_type = CXL_DECODER_EXPANDER; - else - cxld->target_type = CXL_DECODER_ACCEL; if (cxlds->coherence == CXL_DEVCOH_HOSTONLY) cxld->coherence = CXL_DECODER_HOSTONLYCOH; else cxld->coherence = CXL_DECODER_DEVCOH; } else { - /* To be overridden by region type/coherence at commit time */ - cxld->target_type = CXL_DECODER_EXPANDER; + /* To be overridden by region coherence at commit time */ cxld->coherence = CXL_DECODER_HOSTONLYCOH; } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 2dee78e9b90c..5633b7316cb3 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -139,6 +139,8 @@ static ssize_t target_type_show(struct device *dev, return sysfs_emit(buf, "accelerator\n"); case CXL_DECODER_EXPANDER: return sysfs_emit(buf, "expander\n"); + default: + break; } return -ENXIO; } diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index cec7d08b6f44..9c68ec445128 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1926,7 +1926,10 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } - if (cxled->cxld.target_type != cxlr->type) { + /* Set the type of region to that of the first endpoint */ + if (cxlr->type == CXL_DECODER_INVALID) { + cxlr->type = cxled->cxld.target_type; + } else if (cxled->cxld.target_type != cxlr->type) { dev_dbg(&cxlr->dev, "%s:%s type mismatch: %d vs %d\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), cxled->cxld.target_type, cxlr->type); @@ -2482,7 +2485,6 @@ static int cxl_region_calculate_adistance(struct notifier_block *nb, * @cxlrd: root decoder * @id: memregion id to create, or memregion_free() on failure * @mode: mode for the endpoint decoders of this region - * @type: select whether this is an expander or accelerator (type-2 or type-3) * * This is the second step of region initialization. Regions exist within an * address space which is mapped by a @cxlrd. @@ -2492,8 +2494,7 @@ static int cxl_region_calculate_adistance(struct notifier_block *nb, */ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, int id, - enum cxl_decoder_mode mode, - enum cxl_decoder_type type) + enum cxl_decoder_mode mode) { struct cxl_port *port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent); struct cxl_region *cxlr; @@ -2504,7 +2505,7 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, if (IS_ERR(cxlr)) return cxlr; cxlr->mode = mode; - cxlr->type = type; + cxlr->type = CXL_DECODER_INVALID; dev = &cxlr->dev; rc = dev_set_name(dev, "region%d", id); @@ -2576,7 +2577,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } - return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); + return devm_cxl_add_region(cxlrd, id, mode); } static ssize_t create_pmem_region_store(struct device *dev, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 99398c868d82..2a2d2c483654 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -324,6 +324,7 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, #define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { + CXL_DECODER_INVALID, CXL_DECODER_ACCEL = 2, CXL_DECODER_EXPANDER = 3, }; From patchwork Wed Sep 25 02:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13811558 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 973C8149DF7; Wed, 25 Sep 2024 02:47:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232447; cv=none; b=q981fXf05DynEuGHMG0bmOmucseLK6g+SNPYbWENtaOJJwgm5Sv8mrBl487YDCIHezxesSDcKT/OTeCdwt3+MQVd9OmRaRo98lZMe9NT06ARmDZpYN4K+aNDP380uK2fdFVZ8TLlDt2jLEHqS+1qhmY/nZbo6NIsRfL+5sSdlLU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727232447; c=relaxed/simple; bh=fiHK4fG65Xon9kkcXibHaO9Hg0EgB8moOgev/gWVaq0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QbFfEMDeRBVdfOARzZWootiO15dQWIDlFN4XQIfq1bFb6cQxWP+gFcpAfagYt/6fMMzQrdHT9RVzX9AG6lj3yhluEg1MHNAHzeWQcMOSynnPYpYels0NjhEluGU2nFSn1NHzW8PzDTXS7UgTsFo+3IRmPa/tg11iN3lJ6KkOqc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=F01CkFB8; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F01CkFB8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727232446; x=1758768446; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fiHK4fG65Xon9kkcXibHaO9Hg0EgB8moOgev/gWVaq0=; b=F01CkFB8UgT9Zdz+YaGj0yIfv+REVvgYkSMrDj371N+LUaZp2WsotZIL ZndZkLvkhrsMX6hWTCo/h9fQJ+IprkdrB0iyGabrCeIDKSo6jYQteb/Rn CeSJiWwIXkyIqDmMASV2WmEXEAh+AeekPYH7DE5TzBeM0HgJa0LwF1xFt /PIzn87kfPHVV9vRyk8YHrM2yZSGPf98J4uHc5RSMx/rPuIKKoiVSkuZa qGyBH+0FMSlj/uuirPOPGhUza3ppq45v/adb5nQqnwrINR0IMF2auwZBb xxiueLPZhqY8BdIAJrPofGiAuUpXfqDoQctOtIE+Mu43cZ49ItPTwU+B8 Q==; X-CSE-ConnectionGUID: hDIZOXMpTlGBwS7Jo/BI6w== X-CSE-MsgGUID: vIOnRRAyQ361nmUje2JJeg== X-IronPort-AV: E=McAfee;i="6700,10204,11205"; a="26132074" X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="26132074" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:25 -0700 X-CSE-ConnectionGUID: moQoaoE4S9m+ypWf8ECacg== X-CSE-MsgGUID: 6WX5nXaVQaOAq5hfkePxAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,256,1719903600"; d="scan'208";a="102389059" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2024 19:47:22 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero Subject: [RFC 5/5] cxl: Avoid to create dax regions for type2 accelerators Date: Wed, 25 Sep 2024 10:46:47 +0800 Message-Id: <20240925024647.46735-6-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240925024647.46735-1-ying.huang@intel.com> References: <20240925024647.46735-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The memory range of a type2 accelerator should be managed by the type2 accelerator specific driver instead of the common dax region drivers, as discussed in [1]. [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/ So, in this patch, we skip dax regions creation for type2 accelerator device memory regions. Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/ Signed-off-by: "Huang, Ying" Co-developed-by: Dan Williams Signed-off-by: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Reviewed-by: Gregory Price --- drivers/cxl/core/region.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 9c68ec445128..b276752c38da 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3466,6 +3466,14 @@ static int cxl_region_probe(struct device *dev) p->res->start, p->res->end, cxlr, is_system_ram) > 0) return 0; + /* + * Accelerator regions have specific usage, skip + * device-dax registration. + */ + if (cxlr->type == CXL_DECODER_ACCEL) + return 0; + + /* Expander routes to device-dax */ return devm_cxl_add_dax_region(cxlr); default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",