From patchwork Sun Apr 13 22:52:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049469 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2EED1A5B89 for ; Sun, 13 Apr 2025 22:52:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584744; cv=fail; b=q6O8NPszz2tH8RIoIElwewk2Rwp3WCPcX+K/65g+P2iLwa6syal5yH0yDqGwkhOaJPUm1XCmDbo4CdRqnrVtUCiPmQhaQJYFGX7kBCv8QqFzsht30du4qezeFsdeNqcYhGXVCGeZKRQ8wl2L6tbN1Yfs1sKyco+aFVhRaPnxsN4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584744; c=relaxed/simple; bh=ObXjYpSXhyIOJclFZabMVwT7Iw3oZ5eWVtgb3/GUyCU=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=Mh1O3g0ejq4P+vPd/bXDzvftJI4E8kPDRU2BKKnjtVTsKrEqZncDP1q7vgwOqyHMVI5RMt2gpLhaD1ePiM3LFMwv/Jz8qNms9dbzPmirpiVfHWhFOu0s2JzoYZ2mIH9QxEUuo33HYtTlEmyDAHZovdr4rZTg6luS2omvt8VpjdQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=heGP9f7o; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="heGP9f7o" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584742; x=1776120742; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=ObXjYpSXhyIOJclFZabMVwT7Iw3oZ5eWVtgb3/GUyCU=; b=heGP9f7oul1v85wN0FHpMAvhgHKcBsryqOAQ5cgIL5hd7fcE330FRYzI 3H9AUx9b0eEhvlXqtPloe+fRATzGv8zp2cJNLawDOJzmtgVup/EOdeO6u bs5dTtzPMbtE6eKKPi4D08Wy17Z9aETWw5vLy8YwXf9Hof3knAVz6Duju ZALGz1NDhF+atlWLk31w/aNaoDTIkqI2KZvRTauz6k30g2GRZMrp3k+gM UoyizCV90Kdp/nkSVfgPHE7iSvzsyz0Esb3tZrpOm5u7Gdb62NcaYtCMD tTY8JTpXyjnYs9kfyMGLorRWIJUzNBztbIAIt1qdG2migX3IGuYWEuVII w==; X-CSE-ConnectionGUID: zwbzGrVmTK+OLbrkY6Ia6g== X-CSE-MsgGUID: /G6KWr+iSb+MyPv2xkmPxA== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280945" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280945" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:21 -0700 X-CSE-ConnectionGUID: 16kwmHiDQC6UNkrmnfJrhQ== X-CSE-MsgGUID: CRdSWK5pSWi/agsQajn9RA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657624" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:21 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:20 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:20 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.48) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=keMmtVMywCKOCDsuiNgbnq+gsDFcJo8UEkPouXWK3dC5Gv3EmBnJ0MhohN5M1V5ii2J5u3Z0Oa25dYwsKJsXXYFU2taetWPVQhDnnV9zLnOpWc+BCtv5VHHVkid/2V3IvfNjoUo9WLzFde9nPAWzCCTcvMxrh5ZahgpdVMGl0dEelVqND9zZPmlcdDQd703x1NfYTopDTPG4Xu/AZHdJPAEF2wT94kwCpBeoKAIJfNlg7Exr/JOUgJ36ea6fOzLzaCnJhY8syeFE3cpqu3Aq5elYAQ/BvDo6xjswCtMMIcJpEUnbwUbA5sRPPFlwZyTzsPcQunZBeiQdjj08MWZb6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Rnp5qmlW0f1okaqStrG2jxKpSYTSKZide7t8m+Fd8ww=; b=gV78LOJbqc+uXBxEalRuNG1xa4o/vReZkA3KH0iaNEcI2D9DdCcaBRP4i65mLT3qKXcJOw2nOzWYehKwm6RNujTR1jfuPnfel+i9hpSRyvMySdwPmrccOJOqJForiJo9aJlyQhsH0IqeW0If9D3kAbzrQEbBYvIqe0+i/nw6xKXNBj191P7EU+JkSfNM+cm0wDKZmQWOPVJ6EtsZMwx2K2LU1VjdNDj44SthXkbjadM1zoU89PMKeUlu1XIaFPcVbYOV1L0yZ+dLKO74H1th/z4rEETRyn/kNPmHwRxi7v+MNFrVrQ/WDYNYM1tUrTSV/+mSw8vrhskBQojibIHdbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:44 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:43 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:09 -0500 Subject: [PATCH v9 01/19] cxl/mbox: Flag support for Dynamic Capacity Devices (DCD) Message-ID: <20250413-dcd-type2-upstream-v9-1-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=4586; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=ObXjYpSXhyIOJclFZabMVwT7Iw3oZ5eWVtgb3/GUyCU=; b=2CkaSSQAtgE6kCVfvIHhyyHvMDIIb4PkrgLRXi9PTyvh6B5OOscV117iFWBWpzulQYXQ15+ZB TLulNVfBTcJAn/0JcJ3cT3iGXIzqAZRZ7GQBpjinDbm3LUrrtKt0G4d X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: 21404f0d-096c-4e56-bda3-08dd7addc322 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?zjAZXPDMBBGfo0GcwiR9tVhb13KQHVp?= =?utf-8?q?u+zAnFKGh+AmeXEQl4UmdU5w0Le2QVF2/Q02giMzrfhPrsNPPVCoF2ozvRR2jBg4y?= =?utf-8?q?KIZLXGXMyW+ZTWBcbUYK69U6Qb0Hy7tM3HnRuJAST+j8aaKVzH3JJm/X5ptVLIziy?= =?utf-8?q?HndeBJwTmOCFWDgtGFl9/E6qyc/eL5zwKETp0OzJG/k7M3jvAsbFdb8Akid4sGkdD?= =?utf-8?q?MFpYgGWN/JYSHjbzyrncW1ZdFopeIUi6oNoa9eda7YKrfmcvJBsPrcejqkjU4b88P?= =?utf-8?q?giTEmTdc4YJ3fryhtPdguAPyuanMdBnpM5/xcuOrU1el5ypBi3AtBigAyInl7z5bC?= =?utf-8?q?Wm1tkd955VOYAkkmZMpZxe2756OH3mQBx/4zYSgQdwtjel3b8Mo+IXEms+kWbpp6z?= =?utf-8?q?6+StwBxApfKtXKl3l3LlvFMpxMYRJkiuEgHdNozEJldeHsiE04sKye3ikYASvjzpV?= =?utf-8?q?dEjCn5iyGyEnBP2Duav/nSxbeeFKN7e0Ap7hTHM1wTa01sycr+9wLp/807eUFriV2?= =?utf-8?q?1iQ5WJiSvHaRMcq45WarRVJ+JCzdVxo0EEHcZFg9/A1GMEplEr3u3Gs3HoTqAZ/P1?= =?utf-8?q?1NeIe4I/raEczk/y6WP7t4S7ZE/pbB3tM2QVJ8cZTiJfrjtDKhyun3tfpDnP51kMT?= =?utf-8?q?Z0guqPyPgvYqQ85pYAL/8lNzmhtEWp0HQIqTvwpIEfaeJR2xve0LW81RsfVXcRDjJ?= =?utf-8?q?ep/nAnh6N2k+jLIF35OmIz9aOoYUDz5bi94Xl+WHfb7d2I2tJx0LFFjwAyoipwW8M?= =?utf-8?q?hVed09UXlssRJZ0Z5mY9USaaPGUvAXEgcsIA74t0RGU1+CKUSO4XiDgeaTFj9euj8?= =?utf-8?q?lomKYz1jZExCJvoal6L6BHk3gxtbPEhgyKjNTRn7uCRQTQkPoOToGZNrJOH32A6QD?= =?utf-8?q?m2s43cygQqzuvpJrI1iVtr/rJGV4IZvAGRYKPY+V7KetJ0cFQaLU0UP7IkVwfYCVr?= =?utf-8?q?uGGYxfSKKtT5UJQ/wxJfdNz8MLFYTWfkDo7qLp7UfmLAwUzK79v4/b8JUOv07+2nz?= =?utf-8?q?NiLwfBZzCLbJHqDH/Z+Aem+Lv8H9oUa1BXwE67swAWvEinBTte0ojrVxW4DUhRADt?= =?utf-8?q?W6ZXsTT45GQDiamZUO7DdawUR2HoT4yfNURzIRx6rdXj9pdOJHxZ2UIKhfb+/UUrT?= =?utf-8?q?d7kRMCses3GIOAPcrxcLEk6nFJp1Z+2VL/1CugTLmz8bxWgIJC65A1Gh3uoqc2EJV?= =?utf-8?q?/5exKs3nUDojulL8YP6u2IdNp3hHlK8zoLJH8tFH5kHtR8+W9L1xvdcf1opKbUI6V?= =?utf-8?q?F+MUu6Q++bfnG1j2tk9/yAUfcuB6h2R4mOWivuy4KSGTDop/VyJM3l8iJ8SI7z7XR?= =?utf-8?q?phh27vOJAkH77t075eZOSttrvwwk5voNm0PrlAf99CP2OoHUrFbrR6Y=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Ti3y2BmUXzf7R8ezFRniGmRBfH+7?= =?utf-8?q?7fYzdLV1R3aoXnWhDTaFrfTSC1Zw/yyP71TBDuQ7ZhDBnwh43+4pww1r5whItmgvK?= =?utf-8?q?tC7PvwOF+k13XjKW7c3mK+hWs5WTJ+MsstkEaKK07zWHkvYELWU74zdJQzaO6laX+?= =?utf-8?q?A4B7+YTbZZCcZrhU8QohxVC7/VaI3wOZwP6dKS9uQ7/dymT0aLyBJsTPMLhdPyxUa?= =?utf-8?q?WKr4h5YlxPzcx98NBd8o2YWHbCaZiMYbMPo6PmNllkgKvSSdPByryns3XYNrxCIvp?= =?utf-8?q?bfI+19j12xgbjxniOjhASG8CrY8LqbMwUcR1A4YCIptCVaERGQErwOIi4w3iDAjy4?= =?utf-8?q?r0wBjwU+sYFRWjO8frEXmJQ6ABYNbjnSosfmcde2JQS24SRI+ep7vs2YBH3krHcYd?= =?utf-8?q?gnJ3bhawm1rYSVUi1aAj7KFdrAXxiu+1XL92/TO2YAnnUmJaDT6obRvXlHoRDFKZY?= =?utf-8?q?h1CgvpkBHxLAVhBTLU7kbLLiYLdYmhGcWI+sPlS53yT8Z4594RfdNxiRy78wA8hSw?= =?utf-8?q?VPNENGMz0zgkU8A4MAoJr2YDB+ZmF9ryuas/Nui777q825Q5JeH2/mULwwJHBAsGw?= =?utf-8?q?4SF/Uw9siaw1lNSV0f3xHpS2TaFBJ/Pc5/Q8xvuAEueaCmFkMWgGV+b0w4JuVtIvv?= =?utf-8?q?wbVRJW4espnTP1sN2rfhvxyg41w0rXbqxveap3oLwSBtyaifjrtTYtxa5aVHtyYt/?= =?utf-8?q?QlYsTR838KQeEEcTuG8kNGb27NaeHmAx++r9YfLWNmn7YQ+0OSbo6WCS8FJNGJtu0?= =?utf-8?q?yGkHNG4oEh/MuEvJ0E69a1AuFwQlntkdvdbhGlvDLL4ANy3yjEByh5jzt3799Rlcx?= =?utf-8?q?3lzxcciesvFxuQANFFwwqgCSnWbgVfAvJkIIl8WqiaELU74aK5ld35hw/8PG86kEB?= =?utf-8?q?fPULfTepsJ+jmq65wPf/1sohi8+Eq8hPkAN2oMSJXUHDqMp6Edkv6ojKorqmM9tqF?= =?utf-8?q?SeeMrvtppbUOTaNCVg9sMFEuF4JGYreAgoY7tjTN8aKywKGBJR7uklUOkIDukfm4v?= =?utf-8?q?2cOxtz5MoMOhKhKyrLmSM2MI5jP3T6a+vZQx9G0HyplhYaU2M28WTJ9W+aQgL5vTA?= =?utf-8?q?/BwQOyjcCM5pTGGA/pUZ+Ju/BvIO9O0K69sYc05jdn2vt6M5HSLEARv3yCOBq2NcP?= =?utf-8?q?No5cPKtf+q3PNlCmKz+FE5gR/ZMBhwIJGwbCS4aI01uqUxJCy1n0wMJeloSg+kF1I?= =?utf-8?q?tID3nSaXHd3+l6bXV3oGuwqmlQT37dKTki81ZsAXCP4tXqaUUFBmus/8kbIwRlvOD?= =?utf-8?q?+5b/+5xdDwsmlL/Q9nx7A0gq3aD0XlEh7dNIjeX3P++K7o4zYItuX6tcl4ZEkA3H3?= =?utf-8?q?cWVIISGS6GnpptVghu1AAJHdYGqdN+3qEwNrHQDDN+iEEygH2TND+O8nKSEsGlbg0?= =?utf-8?q?aFbOrhSsRnUJy+S24nKI7qjCsS70iJrZ7Vr2bmmudtzU4zGR/pH/nhk6q8nKYhyV3?= =?utf-8?q?cWehCMqo1qV1+mryPsxvYV1L/kZR5n2cdBh8t09AD5XzpfO3YM+kPcBQKZ4TYtCUv?= =?utf-8?q?hlyp9XBL3GrE?= X-MS-Exchange-CrossTenant-Network-Message-Id: 21404f0d-096c-4e56-bda3-08dd7addc322 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:43.9083 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3DTvJt7lzTMXsv5u+7CPJkCiXHKbu1G0+/zYed5TkaqxDo/u8TByoTtjv/mCxapPQNXBg51pRI0e+tJEjAl37Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Per the CXL 3.1 specification software must check the Command Effects Log (CEL) for dynamic capacity command support. Detect support for the DCD commands while reading the CEL, including: Get DC Config Get DC Extent List Add DC Response Release DC Based on an original patch by Navneet Singh. Signed-off-by: Ira Weiny --- Changes: [iweiny: rebased] [iweiny: remove tags] [djbw: remove dcd_cmds bitmask from mds] --- drivers/cxl/core/mbox.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 15 +++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index d72764056ce6..58d378400a4b 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -165,6 +165,43 @@ static void cxl_set_security_cmd_enabled(struct cxl_security_state *security, } } +static bool cxl_is_dcd_command(u16 opcode) +{ +#define CXL_MBOX_OP_DCD_CMDS 0x48 + + return (opcode >> 8) == CXL_MBOX_OP_DCD_CMDS; +} + +static void cxl_set_dcd_cmd_enabled(struct cxl_memdev_state *mds, u16 opcode, + unsigned long *cmd_mask) +{ + switch (opcode) { + case CXL_MBOX_OP_GET_DC_CONFIG: + set_bit(CXL_DCD_ENABLED_GET_CONFIG, cmd_mask); + break; + case CXL_MBOX_OP_GET_DC_EXTENT_LIST: + set_bit(CXL_DCD_ENABLED_GET_EXTENT_LIST, cmd_mask); + break; + case CXL_MBOX_OP_ADD_DC_RESPONSE: + set_bit(CXL_DCD_ENABLED_ADD_RESPONSE, cmd_mask); + break; + case CXL_MBOX_OP_RELEASE_DC: + set_bit(CXL_DCD_ENABLED_RELEASE, cmd_mask); + break; + default: + break; + } +} + +static bool cxl_verify_dcd_cmds(struct cxl_memdev_state *mds, unsigned long *cmds_seen) +{ + DECLARE_BITMAP(all_cmds, CXL_DCD_ENABLED_MAX); + DECLARE_BITMAP(dst, CXL_DCD_ENABLED_MAX); + + bitmap_fill(all_cmds, CXL_DCD_ENABLED_MAX); + return bitmap_and(dst, cmds_seen, all_cmds, CXL_DCD_ENABLED_MAX); +} + static bool cxl_is_poison_command(u16 opcode) { #define CXL_MBOX_OP_POISON_CMDS 0x43 @@ -750,6 +787,7 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_cel_entry *cel_entry; const int cel_entries = size / sizeof(*cel_entry); + DECLARE_BITMAP(dcd_cmds, CXL_DCD_ENABLED_MAX); struct device *dev = mds->cxlds.dev; int i, ro_cmds = 0, wr_cmds = 0; @@ -778,11 +816,17 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) enabled++; } + if (cxl_is_dcd_command(opcode)) { + cxl_set_dcd_cmd_enabled(mds, opcode, dcd_cmds); + enabled++; + } + dev_dbg(dev, "Opcode 0x%04x %s\n", opcode, enabled ? "enabled" : "unsupported by driver"); } set_features_cap(cxl_mbox, ro_cmds, wr_cmds); + mds->dcd_supported = cxl_verify_dcd_cmds(mds, dcd_cmds); } static struct cxl_mbox_get_supported_logs *cxl_get_gsl(struct cxl_memdev_state *mds) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 3ec6b906371b..394a776954f4 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -216,6 +216,15 @@ struct cxl_event_state { struct mutex log_lock; }; +/* Device enabled DCD commands */ +enum dcd_cmd_enabled_bits { + CXL_DCD_ENABLED_GET_CONFIG, + CXL_DCD_ENABLED_GET_EXTENT_LIST, + CXL_DCD_ENABLED_ADD_RESPONSE, + CXL_DCD_ENABLED_RELEASE, + CXL_DCD_ENABLED_MAX +}; + /* Device enabled poison commands */ enum poison_cmd_enabled_bits { CXL_POISON_ENABLED_LIST, @@ -472,6 +481,7 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox) * @partition_align_bytes: alignment size for partition-able capacity * @active_volatile_bytes: sum of hard + soft volatile * @active_persistent_bytes: sum of hard + soft persistent + * @dcd_supported: all DCD commands are supported * @event: event log driver state * @poison: poison driver state info * @security: security driver state info @@ -491,6 +501,7 @@ struct cxl_memdev_state { u64 partition_align_bytes; u64 active_volatile_bytes; u64 active_persistent_bytes; + bool dcd_supported; struct cxl_event_state event; struct cxl_poison_state poison; @@ -551,6 +562,10 @@ enum cxl_opcode { CXL_MBOX_OP_UNLOCK = 0x4503, CXL_MBOX_OP_FREEZE_SECURITY = 0x4504, CXL_MBOX_OP_PASSPHRASE_SECURE_ERASE = 0x4505, + CXL_MBOX_OP_GET_DC_CONFIG = 0x4800, + CXL_MBOX_OP_GET_DC_EXTENT_LIST = 0x4801, + CXL_MBOX_OP_ADD_DC_RESPONSE = 0x4802, + CXL_MBOX_OP_RELEASE_DC = 0x4803, CXL_MBOX_OP_MAX = 0x10000 }; From patchwork Sun Apr 13 22:52:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049458 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 751F718FDA5 for ; Sun, 13 Apr 2025 22:51:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584722; cv=fail; b=MH0aWY833bAgyAA1FPAkiBsoGrKc+2k4jp4gPIaXPIJ/52yYDtdhwj/33aDv43DGbO8sICwVKhh654tvFiY6nlQq3i0ioiTwOpjFBKNUyYD3S+q/1RwvOB8nE8YxMMQspcIWHOTocTcTMBy5VRZn2ASUp425Dw4eJRHlu7g6TpA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584722; c=relaxed/simple; bh=yKu5GzCSwhClZskfPSul/l1KNzhV4Imp85VjVWq1kVA=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=eAxEXTPb9UKKJUD/XqpdCW/mB0U2pMjTC5sFmQuk9GxoFsXE0jpNQT9lecm77p4Koa1/JHGj14ovu0Ip1n9L24nl7uV3pMQIgxSs2SkjCR4h+bdRuBz9ZyqbqZuEr6vW7BYe0+PIB/dtNfCq+zdzgj3WGTr2OJw76WcGtXoyr8Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U0ADUzrF; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U0ADUzrF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584720; x=1776120720; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=yKu5GzCSwhClZskfPSul/l1KNzhV4Imp85VjVWq1kVA=; b=U0ADUzrFS6ljStSHzzHtt4NPQwvKbqrGR5/EOaS8WjmQp8dsMfhYbpA8 eKStF8B4wZQuF+Uh6uRReX8R9dUD3MaUp06IBpFw23IDB2Io4iL5gTJOR vyiJh+wvkuPgRkiciohDRgs78W5+zblCYk0vIhmQ+A01rIIp8FiHYFtNE gf7P0uJfpbYzt6bkq562dmq1a7ZjhEnNxHDc0zTDMlJiPo26MFlGzJHZO 75wUzcs8PT4UReU5v/tP3Dwlf52COh0ObzMZWtD+UOdHikec+X3Bl8j9W RTnnyHN29jIH/Z3oJko2CtoAm/0L3tQgW8UtP4j2MZ5TsaW1vK85Pvw8M A==; X-CSE-ConnectionGUID: dBRovmfaR6agA5qfOZo7Iw== X-CSE-MsgGUID: Rjim1ztnQuu6JIONr1X6Sw== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280885" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280885" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:51:59 -0700 X-CSE-ConnectionGUID: N4SWL+r6QBiYshQ4t4sEgA== X-CSE-MsgGUID: inuCNsY4SvaZ4XrBUEuOMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657424" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:51:58 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:51:58 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:51:57 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:51:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=khs8704fxCoppVAEvlPzgBiMVxbxiTioxn5zLThlzDm+MSLTb1qftLut4Tk6kUGPhvyzKXjrT+NLNd8ZNtbo4QQVYEwpSACqMydCBdfU99lNYxSCn/WtwqwdETH/cZAkrWn6VikzeThsm7qYHAn2nFuOryVa15lAVTiXlVXM6/7sVgiMxYR61srmcVCOMk7gvkle8lWaV6iAoI/Db9V4t23EogJaBK1R+0Ur9h7RW2PpxVgpZf6mmoC6E0QQEu4OY6zeCCJ4r6pBBrkBJDY66bP1qu7Ocsmu+YinpiPgCZzQg2EdacE20+KBSOpGgJx0EtcFyjKoAYPQiZElSTpJQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QkoHo8DOZ66Z7EiSbnZBmZz8Tw+rUGijfRBcpfBpfYU=; b=iM/nXwdp649Tev/uEMXzMA1tnIEOkAeRNiOuvoqeKNK+03o+SR207AfdQ6p9kKcseJnYZKXeQrQOJuVS2aDTquUUlmderPcXk8upQKWI9Gp5oh4xgckFeiDw9KHjOrxchOj6bFwWil7Zx7YN+inQ1UdxIOsDhv5MY9PsG0oEI5KqsU0XHWtFI1MX0eTgQS/KgYwZv+fRlLLLymx2ROlRBE+dwPKSIXTki8Akjs4nsxW6rb2ox7p/ABQcqqqecoF0PlwArbOQoweHigHKtJAt59obg+IfXGjNFFv4z0UtsJ+rJeAnh9PQYIHduZ3pJpCw/P7dIAnjxnlddha1bssA7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:45 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:45 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:10 -0500 Subject: [PATCH v9 02/19] cxl/mem: Read dynamic capacity configuration from the device Message-ID: <20250413-dcd-type2-upstream-v9-2-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=12514; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=yKu5GzCSwhClZskfPSul/l1KNzhV4Imp85VjVWq1kVA=; b=bN1/2MH5VstkPjHZQa4RmCVO2wG0EP+Ci0FetDpq9W9nIkT5q8Ja/2+TnNzC5vVldDEtCfEqr XNP4trGWR3FBHv9R4V5auPkBSOwcxjZlGX9mAqPB3JEccOcGuNqXI/8 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: a10f9a32-c0d7-46cf-3d7f-08dd7addc41e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?oudKczqH45Gi4VFxhHarvxxp8uVYNv4?= =?utf-8?q?d61I7Pz+reHYZrwjnPkOPWjrz3KUu+S45kjwYkYX9Ui8a9Tu/nsdXks+O+GTtCjPA?= =?utf-8?q?y+rhlzNWCrrUQP7gUKtiuIps8/xh9OlLzeYDz6gyLWSJ1Yar2uGCMetx4RTIgRKx3?= =?utf-8?q?SHEApt7SEzr1pV7NXHYMYGqkh5czoV2GFk3XHd1kTBEpiJPBSeMhPc7YT3FWKSPeO?= =?utf-8?q?pCA7H/WvBaYkF3lMVuo3xq0ER1/tXVey+UfOw2yDbu50LOy7P/UxQYvHTC9ZeLT/Q?= =?utf-8?q?mmN3NVZb5wM51C3ZGYSBrNa7pri5sADEqQpDsNoR3NykCRNbsOcdXtigJd9xE3M2t?= =?utf-8?q?H0tbHpTRVLwKmj3aKMJr2V47UKE7XIP5o8BuIl+lzFmg6qmSwQZqehgTNEu+Srht7?= =?utf-8?q?r8nloIC2oUKO9LqPeHISlCnPF2ESzpMUEuClaiVorg0Vfx972OBhtbZ+69YwTcg2K?= =?utf-8?q?U86U2DI83ENQ0AoBKsAcBOyC97NpHyYoGVVul9XBS3jTQa0j6JDZLA8pcSDP9JETK?= =?utf-8?q?FJm3zveOlSLkUOFj5tOGagyeQhxX8tB/acX2JMe6DBscPTdD4o9vhyn76fmCPI0DJ?= =?utf-8?q?/9zA3cpaY0HZO6Sl3Tcx1TYBMIuRdtPlgY1xPEt6fhh6SJ8CrmBv4Hrs99gG+RZuR?= =?utf-8?q?Lk2koAlQvXPUVqap3N4MMwSughTOB2tpDucre0E9Kb+gcixBI1hblUsViZFh9IXW8?= =?utf-8?q?T+Pj8H1Uv7rpUfHfiQGJhu8ofg99ME9/hNP6OrM04pkkTxWg1zSmUeecFoFryZkcB?= =?utf-8?q?/zyIZ6HqfCIANdOHqWgN3G77H28waZoNyja+2Fqpj/MCQQnGYse8snaVMIzdW4v15?= =?utf-8?q?9oDIhgX3M989Ax25OKPi4J6yIUfGOkqEa0yWjvYDaIC5iMYgbFyzNZV5DRSlHw45w?= =?utf-8?q?Zhv6+YcUoaHQFySLPepaiBqteWXA0vA5kSGxhg8fbluHNrHe459CC6UHDVaNOjfH+?= =?utf-8?q?aZ3Feso9MK53GTS5KdIYznSTDjrcoeOoQo+GQLRj4DUe8VJK3Cq7FYK29mI0v5nh8?= =?utf-8?q?6q0gvxUCD1JfOkt1e6Zn1w+qUgF+wsDi9jHgdnJsa+wZz09PY2VIYTR9ldw60SpVf?= =?utf-8?q?cHYMibmhamtoBnpi5VvRPdMO0OIQVR2NKdd4oznZUfG7sw7fFW4Uydi9c23qJD94F?= =?utf-8?q?Y1aWhyYU3ioH3pynscb6OJ6B1rgTyOaKaw8QqVZ/qH8entwvUmp6+e3sgNQ6v5uwk?= =?utf-8?q?RIuH6MbEsT+h8DpiT1mbfUaLcMjtq4T3W3GHdOSGCzM1XZY22lYZtxiuwQtoNjm+U?= =?utf-8?q?FmKIGqYb4abf188cb+tf6iVZlSeswVVP+8CQScGiCpw2MJuV8GLmrVNdPVcVwN2mP?= =?utf-8?q?8SakwbLgc2T5uIBf85v3gI19nUiVQ5QTtLofCJTrZqzYkgUwHC8vnto=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?SjWAGw3vjEP5qj1vr43eBcInh9E1?= =?utf-8?q?NAhh8ZgasDPlNX3n8hPSlNx9yafNYHk2RF5FSisKMbpjS3HmuoeCyepQS22JMLVQI?= =?utf-8?q?Z0As6CZa8N9Qy9iBPN1BB1NXgL2lZTTyW8BuO4c9yt9w8EjoELbYV+f/PKL5YUqz0?= =?utf-8?q?djw6i1QAjGBSb7xTXQDoCgHAeqQi4r5iCxDQXkqLx2mAE6Of5Ed/05f9dyUzwVwbc?= =?utf-8?q?6xjVLyE255VPmjKqqRAj1Ca1immQ2n2DUBF8bIEs4Oku12hI7jQbe6Y/uFtjmqpok?= =?utf-8?q?pQFyjETr7EF4pTVVZUTmWchGHe5qSXTeaBzbsqlIQ5VpR2iZoTXwAQXH2v/zcM/0Y?= =?utf-8?q?UJdMe60S6e9DswByC/PX/okSmMwB5TloLEXYDv00LmQDdCsdClV0DRWV3V5rWoeN3?= =?utf-8?q?dzN+sNSEx5ezmojiGZvU39N+rd1iqAc3l94Avtm7LfUZW0pMSZWsgkhH/haltDc25?= =?utf-8?q?eHEFT0zf9L2Z8c+MZ9UvkinNDIyJbhENJxQhF4eA2m8F1H9m2VOrrOa2aHn3IpPiS?= =?utf-8?q?QF2M9gZcY2PspQGizblZ5kwQD6SgXOjBigHq9vrEerZizUUgX69RRiO6/lWWEHadf?= =?utf-8?q?ubfRyLxyeytiO/fNwn3moRGDf8YzixXLjmoPkAdfe+lgRM7WNEiTqxDWccHBiLtWN?= =?utf-8?q?XVb26EfJVBC49xAg0DJLpnz2nwWVdZMAVVKuKLNEs9sFsLC+eZugLdPvW6xTMUFR5?= =?utf-8?q?DxTwMZq6Ycg2Xm86h1GgRzlJrp+6WnNrz62wi5S5tSZUYsoraq4xJCdsQFlRXToba?= =?utf-8?q?d37xF8lAaNFWEYbrALuKBsFYEYI7OwAzi1dmD3xY1EIrpk8yL338hcYffLDqQ0HIQ?= =?utf-8?q?LIdR+2BnJQoi5nCgeeUfVe0zSG+JTU9yxHYnJPLNfv/RGCazLNxRUK39Whj2WpSNC?= =?utf-8?q?z1uOm+441lr7nQdIx70P0kTe+O1FC4ks3xB465MO61AYqee/O8Fl9eBf/xZM8QkNx?= =?utf-8?q?qVfK5WTL4ju9cCPxnosGS1ZXVNnuAf1ilvLfR7TU1iHjnau4ul6xabuN0ynhL/zor?= =?utf-8?q?j/mLStyPfPSuMdnf69VA2brtSOgf2FFsr+vLPtpPkcIXRc9QlQqS9vLSrb9FtCE3L?= =?utf-8?q?g+y0FUtxjoRFcbkMt6JPI/xALSeOr5tXQumGHrNalFCSm6dZ6r6CkKlWbyT3zq+Pe?= =?utf-8?q?spaiznS1owySBFt5k9cUs9b3+U+7EpDEJYfRq6TtJLcOgP6fpqRdMpER5GJ7tq0wm?= =?utf-8?q?qs+2MZW24M9bE1H+Q+yiHS2M41ReQjMVrJrAaJRLx7if14Rvsdxer+kBUX/B5kxZa?= =?utf-8?q?ljdBV8fmGdk7sxyGTfn0XMqOXRn+oFXPJwLrJgCWHNbmqfgnslrUoKBmoa0Jur8Of?= =?utf-8?q?cBa6unSqaqgIgRUTSzxyegFh0H4sp0Vg/ZL412RMs1uP8110wDRtZMVZD/gfMTMwh?= =?utf-8?q?rPXYw4+VQEBQYHDV1+7JhShox+1C85AQWgkVUjMZ9wTjUZoGo74Moc4Oh2OZ2T8YI?= =?utf-8?q?VEAGS2UKwmewItP023rnhKs51LlN8vtvkEvIwdK52tdilu9RUhKJFYVw7/t+V3mP1?= =?utf-8?q?NwwRqHp48llZ?= X-MS-Exchange-CrossTenant-Network-Message-Id: a10f9a32-c0d7-46cf-3d7f-08dd7addc41e X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:45.5504 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C4NCX1kzs+mwobyjZI19BQIhbLzxVpPV55BVg8UeiW1OK5bzASyyrDaEpFXh09tpObOy4Fx3vP8DFJKjHfNf/A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Devices which optionally support Dynamic Capacity (DC) are configured via mailbox commands. CXL 3.2 section 9.13.3 requires the host to issue the Get DC Configuration command in order to properly configure DCDs. Without the Get DC Configuration command DCD can't be supported. Implement the DC mailbox commands as specified in CXL 3.2 section 8.2.10.9.9 (opcodes 48XXh) to read and store the DCD configuration information. Disable DCD if an invalid configuration is found. Linux has no support for more than one dynamic capacity partition. Read and validate all the partitions but configure only the first partition as 'dynamic ram A'. Additional partitions can be added in the future if such a device ever materializes. Additionally is it anticipated that no skips will be present from the end of the pmem partition. Check for an disallow this configuration as well. Linux has no use for the trailing fields of the Get Dynamic Capacity Configuration Output Payload (Total number of supported extents, number of available extents, total number of supported tags, and number of available tags). Avoid defining those fields to use the more useful dynamic C array. Based on an original patch by Navneet Singh. Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase] [iweiny: Update spec references to 3.2] [djbw: Limit to 1 partition] [djbw: Avoid inter-partition skipping] [djbw: s/region/partition/] [djbw: remove cxl_dc_region[partition]_info->name] [iweiny: adjust to lack of dcd_cmds in mds] [iweiny: remove extra 'region' from names] [iweiny: remove unused CXL_DYNAMIC_CAPACITY_SANITIZE_ON_RELEASE_FLAG] --- drivers/cxl/core/hdm.c | 2 + drivers/cxl/core/mbox.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 1 + drivers/cxl/cxlmem.h | 54 ++++++++++++++- drivers/cxl/pci.c | 3 + 5 files changed, 238 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 70cae4ebf8a4..c5f8a17d00f1 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -459,6 +459,8 @@ static const char *cxl_mode_name(enum cxl_partition_mode mode) return "ram"; case CXL_PARTMODE_PMEM: return "pmem"; + case CXL_PARTMODE_DYNAMIC_RAM_A: + return "dynamic_ram_a"; default: return ""; }; diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 58d378400a4b..866a423d6125 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1313,6 +1313,153 @@ int cxl_mem_sanitize(struct cxl_memdev *cxlmd, u16 cmd) return -EBUSY; } +static int cxl_dc_check(struct device *dev, struct cxl_dc_partition_info *part_array, + u8 index, struct cxl_dc_partition *dev_part) +{ + size_t blk_size, len; + + part_array[index].start = le64_to_cpu(dev_part->base); + part_array[index].size = le64_to_cpu(dev_part->decode_length); + part_array[index].size *= CXL_CAPACITY_MULTIPLIER; + len = le64_to_cpu(dev_part->length); + blk_size = le64_to_cpu(dev_part->block_size); + + /* Check partitions are in increasing DPA order */ + if (index > 0) { + struct cxl_dc_partition_info *prev_part = &part_array[index - 1]; + + if ((prev_part->start + prev_part->size) > + part_array[index].start) { + dev_err(dev, + "DPA ordering violation for DC partition %d and %d\n", + index - 1, index); + return -EINVAL; + } + } + + if (!IS_ALIGNED(part_array[index].start, SZ_256M) || + !IS_ALIGNED(part_array[index].start, blk_size)) { + dev_err(dev, "DC partition %d invalid start %zu blk size %zu\n", + index, part_array[index].start, blk_size); + return -EINVAL; + } + + if (part_array[index].size == 0 || len == 0 || + part_array[index].size < len || !IS_ALIGNED(len, blk_size)) { + dev_err(dev, "DC partition %d invalid length; size %zu len %zu blk size %zu\n", + index, part_array[index].size, len, blk_size); + return -EINVAL; + } + + if (blk_size == 0 || blk_size % CXL_DCD_BLOCK_LINE_SIZE || + !is_power_of_2(blk_size)) { + dev_err(dev, "DC partition %d invalid block size; %zu\n", + index, blk_size); + return -EINVAL; + } + + dev_dbg(dev, "DC partition %d start %zu start %zu size %zu\n", + index, part_array[index].start, part_array[index].size, + blk_size); + + return 0; +} + +/* Returns the number of partitions in dc_resp or -ERRNO */ +static int cxl_get_dc_config(struct cxl_mailbox *mbox, u8 start_partition, + struct cxl_mbox_get_dc_config_out *dc_resp, + size_t dc_resp_size) +{ + struct cxl_mbox_get_dc_config_in get_dc = (struct cxl_mbox_get_dc_config_in) { + .partition_count = CXL_MAX_DC_PARTITIONS, + .start_partition_index = start_partition, + }; + struct cxl_mbox_cmd mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_DC_CONFIG, + .payload_in = &get_dc, + .size_in = sizeof(get_dc), + .size_out = dc_resp_size, + .payload_out = dc_resp, + .min_out = 1, + }; + int rc; + + rc = cxl_internal_send_cmd(mbox, &mbox_cmd); + if (rc < 0) + return rc; + + dev_dbg(mbox->host, "Read %d/%d DC partitions\n", + dc_resp->partitions_returned, dc_resp->avail_partition_count); + return dc_resp->partitions_returned; +} + +/** + * cxl_dev_dc_identify() - Reads the dynamic capacity information from the + * device. + * @mbox: Mailbox to query + * @dc_info: The dynamic partition information to return + * + * Read Dynamic Capacity information from the device and return the partition + * information. + * + * Return: 0 if identify was executed successfully, -ERRNO on error. + * on error only dynamic_bytes is left unchanged. + */ +int cxl_dev_dc_identify(struct cxl_mailbox *mbox, + struct cxl_dc_partition_info *dc_info) +{ + struct cxl_dc_partition_info partitions[CXL_MAX_DC_PARTITIONS]; + size_t dc_resp_size = mbox->payload_size; + struct device *dev = mbox->host; + u8 start_partition; + u8 num_partitions; + + struct cxl_mbox_get_dc_config_out *dc_resp __free(kfree) = + kvmalloc(dc_resp_size, GFP_KERNEL); + if (!dc_resp) + return -ENOMEM; + + /* Read and check all partition information for validity and potential + * debugging; see debug output in cxl_dc_check() */ + start_partition = 0; + do { + int rc, i, j; + + rc = cxl_get_dc_config(mbox, start_partition, dc_resp, dc_resp_size); + if (rc < 0) { + dev_err(dev, "Failed to get DC config: %d\n", rc); + return rc; + } + + num_partitions += rc; + + if (num_partitions < 1 || num_partitions > CXL_MAX_DC_PARTITIONS) { + dev_err(dev, "Invalid num of dynamic capacity partitions %d\n", + num_partitions); + return -EINVAL; + } + + for (i = start_partition, j = 0; i < num_partitions; i++, j++) { + rc = cxl_dc_check(dev, partitions, i, + &dc_resp->partition[j]); + if (rc) + return rc; + } + + start_partition = num_partitions; + + } while (num_partitions < dc_resp->avail_partition_count); + + /* Return 1st partition */ + dc_info->start = partitions[0].start; + dc_info->size = partitions[0].size; + dev_dbg(dev, "Returning partition 0 %zu size %zu\n", + dc_info->start, dc_info->size); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_dev_dc_identify, "CXL"); + static void add_part(struct cxl_dpa_info *info, u64 start, u64 size, enum cxl_partition_mode mode) { int i = info->nr_partitions; @@ -1383,6 +1530,38 @@ int cxl_get_dirty_count(struct cxl_memdev_state *mds, u32 *count) } EXPORT_SYMBOL_NS_GPL(cxl_get_dirty_count, "CXL"); +void cxl_configure_dcd(struct cxl_memdev_state *mds, struct cxl_dpa_info *info) +{ + struct cxl_dc_partition_info dc_info = { 0 }; + struct device *dev = mds->cxlds.dev; + size_t skip; + int rc; + + rc = cxl_dev_dc_identify(&mds->cxlds.cxl_mbox, &dc_info); + if (rc) { + dev_warn(dev, + "Failed to read Dynamic Capacity config: %d\n", rc); + cxl_disable_dcd(mds); + return; + } + + /* Skips between pmem and the dynamic partition are not supported */ + skip = dc_info.start - info->size; + if (skip) { + dev_warn(dev, + "Dynamic Capacity skip from pmem not supported: %zu\n", + skip); + cxl_disable_dcd(mds); + return; + } + + info->size += dc_info.size; + dev_dbg(dev, "Adding dynamic ram partition A; %zu size %zu\n", + dc_info.start, dc_info.size); + add_part(info, dc_info.start, dc_info.size, CXL_PARTMODE_DYNAMIC_RAM_A); +} +EXPORT_SYMBOL_NS_GPL(cxl_configure_dcd, "CXL"); + int cxl_arm_dirty_shutdown(struct cxl_memdev_state *mds) { struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index be8a7dc77719..a9d42210e8a3 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -485,6 +485,7 @@ struct cxl_region_params { enum cxl_partition_mode { CXL_PARTMODE_RAM, CXL_PARTMODE_PMEM, + CXL_PARTMODE_DYNAMIC_RAM_A, }; /* diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 394a776954f4..057933128d2c 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -97,7 +97,7 @@ int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, resource_size_t base, resource_size_t len, resource_size_t skipped); -#define CXL_NR_PARTITIONS_MAX 2 +#define CXL_NR_PARTITIONS_MAX 3 struct cxl_dpa_info { u64 size; @@ -380,6 +380,7 @@ enum cxl_devtype { CXL_DEVTYPE_CLASSMEM, }; +#define CXL_MAX_DC_PARTITIONS 8 /** * struct cxl_dpa_perf - DPA performance property entry * @dpa_range: range for DPA address @@ -722,6 +723,31 @@ struct cxl_mbox_set_shutdown_state_in { u8 state; } __packed; +/* See CXL 3.2 Table 8-178 get dynamic capacity config Input Payload */ +struct cxl_mbox_get_dc_config_in { + u8 partition_count; + u8 start_partition_index; +} __packed; + +/* See CXL 3.2 Table 8-179 get dynamic capacity config Output Payload */ +struct cxl_mbox_get_dc_config_out { + u8 avail_partition_count; + u8 partitions_returned; + u8 rsvd[6]; + /* See CXL 3.2 Table 8-180 */ + struct cxl_dc_partition { + __le64 base; + __le64 decode_length; + __le64 length; + __le64 block_size; + __le32 dsmad_handle; + u8 flags; + u8 rsvd[3]; + } __packed partition[] __counted_by(partitions_returned); + /* Trailing fields unused */ +} __packed; +#define CXL_DCD_BLOCK_LINE_SIZE 0x40 + /* Set Timestamp CXL 3.0 Spec 8.2.9.4.2 */ struct cxl_mbox_set_timestamp_in { __le64 timestamp; @@ -845,9 +871,24 @@ enum { int cxl_internal_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); int cxl_dev_state_identify(struct cxl_memdev_state *mds); + +struct cxl_mem_dev_info { + u64 total_bytes; + u64 volatile_bytes; + u64 persistent_bytes; +}; + +struct cxl_dc_partition_info { + size_t start; + size_t size; +}; + +int cxl_dev_dc_identify(struct cxl_mailbox *mbox, + struct cxl_dc_partition_info *dc_info); int cxl_await_media_ready(struct cxl_dev_state *cxlds); int cxl_enumerate_cmds(struct cxl_memdev_state *mds); int cxl_mem_dpa_fetch(struct cxl_memdev_state *mds, struct cxl_dpa_info *info); +void cxl_configure_dcd(struct cxl_memdev_state *mds, struct cxl_dpa_info *info); struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev); void set_exclusive_cxl_commands(struct cxl_memdev_state *mds, unsigned long *cmds); @@ -860,6 +901,17 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd, const uuid_t *uuid, union cxl_event *evt); int cxl_get_dirty_count(struct cxl_memdev_state *mds, u32 *count); int cxl_arm_dirty_shutdown(struct cxl_memdev_state *mds); + +static inline bool cxl_dcd_supported(struct cxl_memdev_state *mds) +{ + return mds->dcd_supported; +} + +static inline void cxl_disable_dcd(struct cxl_memdev_state *mds) +{ + mds->dcd_supported = false; +} + int cxl_set_timestamp(struct cxl_memdev_state *mds); int cxl_poison_state_init(struct cxl_memdev_state *mds); int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len, diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 7b14a154463c..bc40cf6e2fe9 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -998,6 +998,9 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + if (cxl_dcd_supported(mds)) + cxl_configure_dcd(mds, &range_info); + rc = cxl_dpa_setup(cxlds, &range_info); if (rc) return rc; From patchwork Sun Apr 13 22:52:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049459 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA70419539F for ; Sun, 13 Apr 2025 22:52:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584723; cv=fail; b=Kxy3/bbOzV3heBwx1/ETmCsMgHTQqxOYnFZu4po7f8DuwuliJl95TPjS+4P8w8nubkmY36vhgJQcyaWQBDdHc9okJaphntAZIHQjr3tkaNY720EiNmW/TNyedPaDL5gonX1RCf43aZMXFwReYTOF8nHz4r/RfRa7RjEPO6bdOAs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584723; c=relaxed/simple; bh=V3fWJr3Em3iZQf2yytjyz4d5nf/SXSyrd7uDfepAKP4=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=Y9i/5esGhXQMkUUyNYV/dXZYs/CE6mz1pmCJeLPA0EP06/NpVTFwydXDZ84izZKvo83T8X18TjupkDMH04dyURUEa2DN8ibke5v69rSoFp1dxQtDm2jrYvgKQtWhn4gsx7Al2eQNcuLEMHzLUFVragAZpgTRUjJEzIlpnGA6AjA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MqS2Zc9T; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MqS2Zc9T" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584721; x=1776120721; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=V3fWJr3Em3iZQf2yytjyz4d5nf/SXSyrd7uDfepAKP4=; b=MqS2Zc9Tq3OuGCNV7/PK21yuqjeIi3f2lYIhV0aKdt3+UewZ+7Wvl9w8 7us+RAsqFZome5OPUEhw6lupFbyczkNrQTR+TqeLyvf+ARMx5WlQ8ZNxj x1pT3by9R1q128e+5yE0UkhUWtgT/QktsoEQj4SJQ+74tAznWmfwS0ZKA yUp6ZMH/YF2rlUTiBng2hgdYj+M70pZ3KKqaKnoodtIwqRHellH6OJVGm cHYcT+cWS2S5Pm3n5c/+x/tpzE/ce8kgof1OXISiNNMbBffo1QVKndekx 34rv4YcyfLbhYF8PW1D1ChxPV6joPw/4bmP+b/q9gTh3Kw0S9wtHSBLLO A==; X-CSE-ConnectionGUID: uX5eDUv8QreMr0MOHl3zeg== X-CSE-MsgGUID: IiZHN0BpSbm3e5ahDMRxXA== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280891" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280891" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:00 -0700 X-CSE-ConnectionGUID: xVHgEbcBRbWcSny/L7WzzA== X-CSE-MsgGUID: 64Ik891lQGq1Y9+T1Ul4BA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657430" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:51:59 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:51:58 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:51:58 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:51:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DWEJtkC5NvYYRQS+rchlsHU3+eIR4vYAeMxNmlXurb5oBWbf41FW/liNyXdFNLVJq1RMtfH2fQKeoLg/yKDan7eNO+gll7MAPbNlmQGkDgllHh/yZsUXV7k22rUqPq7zPyeFaWV8e9AOFnDH5yEtbWuqjSN86JWlJzfURt2zn6py6FNuUnqzyP8ZMgsRAUnzWuKxR+C4QLcmyOf8jE/Gidd3g01cMyul6/nSoobQTgle/e8/NjsyWwcEHHrHfM+1AM9UdDS/pN5CYyXLBcviCOO+2HIWhi9zLNLD/XOmeXqrMKTASnCeBv+ls/VQioDNjeOFv40ZEMdRjqbQrmSZnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xKw2/RAcdXZWQN02V1z9yzRy7CXyZCgrhub8YtJUy/o=; b=org9c99KH4OuPNz/QYzuwrnrHVWv62+veCeA12J+cOPoOTm8iQ+z6i84ekUjL2yclaYhaBT1Xv/1YFwE4vfayBuhbL6eYNl8t3XrOq6s3PuSaZTo6sBW38Btyd/F6Jw7zWnd0hBBFEGEFYIO5BLqbuG2B4xGl4eexm/54IR08jslhmBgtipl5Xs7aff6NkbziIawV38V17TAX1fJJn7hxvo0L3mK12WRLHa4hNwZDmFEWTdh3N9oWiUlx0naQpGeMESx/PivPt0X2Wb/c0hD2Rp7GvPIVK2PbeZO/m0FNnWeYsu7yS9pR0JeICwlByXyVfIEAcqze0lkL8oV+0gh4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:47 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:47 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:11 -0500 Subject: [PATCH v9 03/19] cxl/cdat: Gather DSMAS data for DCD partitions Message-ID: <20250413-dcd-type2-upstream-v9-3-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=4875; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=V3fWJr3Em3iZQf2yytjyz4d5nf/SXSyrd7uDfepAKP4=; b=AyU5Sbgydhbm2qFavE2D0NLOq67cFT0qHgTrjJZn5EyonDuA4LqaYh1vyvPtk899fCRRyPfSq VJ4h4uy1+yoCH/MA3Qz5CTwwJOlkY4s5S2wdbdVLxqhprOY0ofY97Z7 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: 2b275f08-8b81-4bd9-6ea2-08dd7addc517 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?fi9fdLw+omWWJTRZP+G7CP9YoJ7DX8m?= =?utf-8?q?YKbnhGtQCNhHG1x+hKStCCOgSzS0Set0e5MZ8UUI3moFbo0W53HMjLU+9wMpsIeBU?= =?utf-8?q?kg0w/jV5GI1622pp1rD1nTix19xWptKZoTos0K29w9hgikl1oshcbRtWov2zDaVtG?= =?utf-8?q?2Pnel5GPTvBxAv6HIWxXPZREdUhzI0VktbYtO2UfvhKkp+ArDpbel4w+YSU06O2Rx?= =?utf-8?q?aPNMqYcPvn4yesBhZMAMwmgWpLVuXQHK22mfrta4JO4F0aGMSGNCmeHlt12yMPELi?= =?utf-8?q?M2V/F3IR8mvH/kteWpUQuR5l+UsITX6MxrmVP+oWcqeJq7LNxByU7G0MxMJYOMr/B?= =?utf-8?q?avq8CEKf3Sl+zJ0KSAZChUcFVh6qSelu6sFi4dhMl4Y5xh9zCV2wdAxv65ImD7580?= =?utf-8?q?IFWJwpPIlN73tKsQHgOoSJhTKj7yD9XRUdDvblL7pXxz+zNafU0008sIvtR16cIFG?= =?utf-8?q?rWTRzzGwakjyVgt0y11i6XbOQykHQrZ6Go/C2xhCYlq0XajNhpxLwEJ6oOxCFlKA6?= =?utf-8?q?+2yCE5z6+vKzGBayp3xAKuj2vOswqkZHwREck362cFlLgTDNYLu3Gm2nLCuIwZriT?= =?utf-8?q?Zb01w3nr/W6x7IIoQzwUvtHiCM5i8u01hI+f9gnidOUuYf3JFyW3+QcHgK5XcZXWy?= =?utf-8?q?Eh1CDB4FB/AOunGyhWaEttcnizHGfZVRjS8hIzt0FHsZqa+UnPfZV6jIATV2iZ9T7?= =?utf-8?q?C83T+OPM7Y0CgQyuqU5glDDyf2Vdm+bIH5mg3F/kHqwxdrqa5E3twDXR6MbX548We?= =?utf-8?q?egblbkKCOq9sI4/PeRHitsGUBE4m//ggzh/kb0PJNGXA3Kw7qR6lHLlbuqD1ESpul?= =?utf-8?q?hLMLHNw89LJQiCkW/QJ94BYwgrAWnJ2YF8bIKwmZfUlhlbrLlsAiLIlIKtaLWd0Gb?= =?utf-8?q?ZapuKxekJHy1HyvirkA/MpXO8YoTArBWl8HKwIdL0vUZSJIxoMLAK/OwnZ3qIj9GP?= =?utf-8?q?fBZnC7w6QDiQvLIkrtYw0nVAZJghjD5A3qDg28GQa9WHKg5gbmjDPP5m4aPCdX70Z?= =?utf-8?q?EEwayOGyYIjsFehRLYItv94hrBON6TlG30sVupwp8n8H+YVbXd1K49PHBDTgB79pH?= =?utf-8?q?1e48mdzRj2cfm/qkRR9iHNwocjaDW4GmPpk3YYGiPtzo0r9eaeAdXzvsXgvA57LhX?= =?utf-8?q?Qtj5xNZNHs6K+y5ivq8XHah9NiLoiplT/AnurKqHc+3jxKad/25/FhQl3qSFPDjRN?= =?utf-8?q?wHxq4UaGilf7jmKtImg3aC8TGw1Az7UN5q/AUkpBy84JAKGG77iBLQWp2pAPLbwSK?= =?utf-8?q?+xR99rfDstV/IR6hsPY/JxMYOxGJq6FS0qI1wdRCLwUxjFPpyL+3TwKhQhYyg1oeU?= =?utf-8?q?DvR/H028/QCCmGBrSh+PuY/8hiYciTr9Ug=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?6XKmOqi5DEBwRDmZoqRZjgkRbCDn?= =?utf-8?q?VFYKD86oXBdV+/qZ8ci5YNgYtyNPtwmZ3YhMNQ1Fh545SBA4KfGGLBVb7MRi8Gs84?= =?utf-8?q?Jr8lPb0UVvUzeepyCtUcy1YrjjBaOZCUAEQ+gPsH9KcZQ+3paZRR2sIF1Iy+vy937?= =?utf-8?q?Xx2DRvBskZikMszkwr2CduqxZMb66yc3BCVWz/HagCI7DGknqHDxZU7NpKBFbPnMS?= =?utf-8?q?xArl6Ix5DhU4XBom/10sf7E3CGbeLwlsEmdjausdTP44mQgIpsbtj032h5TADPI0X?= =?utf-8?q?Lv/v9yezhxc2otIma5x85qFF+scauu0OPRR9/RsALlP0FVNJSlY039D9htQPA3Szr?= =?utf-8?q?kieBuB/CrzCx45Gl7LC4G6/K/6yO6GbwMVvSC43kbgEgkPcv2+FDxcJxhesiCUpqH?= =?utf-8?q?kQx3XgF+RiLJA+d94iY5J7W01l7uMZ/CMeTfFtU9HLqqrxQDHrKLIrhVUEguYhAWo?= =?utf-8?q?YQY1sSGchmj1zV+lPDX1NuVT34d3hf79d+k1MlPWD/voerwgqdp8VN+jzNcTmat/u?= =?utf-8?q?pgpJu7nW154JEHYsaXxJjkTVCb5BJAFq4T+trFjOrVe3zwClN4O6Vkit8wWpJbn4X?= =?utf-8?q?dDmS28ZoOJ/59pBvgMw59GRlMKP8wi4+PMp1UnFTmO+QWKrwQtkAgBtcrhQDe9cmn?= =?utf-8?q?vtR2j+jSWjWHXi5yDAU5D5oL5MGKP6DEicmdFWixKbzCrNjyjwoKA+/6QlUeqIL0A?= =?utf-8?q?ElvOMqcf9BO2uactgC/srXr3utsorUgH+FXtFBwvxs1VLa6CJOxXgFOYcsmWct9F+?= =?utf-8?q?fddFls9v2yfG46jwFS+uJYI+kuXtv902OL2WYtCSFWcpZ700g4NtZYBCie/ylskHy?= =?utf-8?q?rgmw9jnXoAL97xFg2gta9Y7GxHpRGIqX4cK3k7gUcOMe3hTmir2l9002kWvPU+2UZ?= =?utf-8?q?WUcRUVLUJCVALoRlblRifiz4xN2o6GhYBQ/UhPaDoeWOn5EUl3Wr0NZukoB/jNWAp?= =?utf-8?q?vzibIsyRAMd9t2gtVSdqW4WEGP4n9vpmUpyvxvg86iAKwbv56BOZTGtySRK3NDWhT?= =?utf-8?q?6+X+HmTdzvZBp3aU13vItzg/3Qpz9acKUCWXS/sJXI88bc6B6di2xeO/uHCchHJMD?= =?utf-8?q?HZ60GivhFtA/JM6HLDhycUicEw+CJRm3w/xDREIY9lua3r3gsJsqbGaN7VlGCWxFg?= =?utf-8?q?BFeuefxwsDRjlQMSHrwjuE+sQlsulnv/MHHUHkhEJD4dPZRsk1QJYxbCaRZTPtQcp?= =?utf-8?q?V2pFbEwBHFs78gvoI4VuagHzAjB5s30HRyJAi6d5331ThsdnDyl8sW/pVyuGQRc+K?= =?utf-8?q?FSLWW4oA4issgOsJFF/66RiGojcohgG0xf8xGBtHoNGY9swciJjwOMLEv7SJ8MpwQ?= =?utf-8?q?a5ZOasGYyI0jXtcgVT46Lrul+2CQtGk8uM/tY9XW0CGRFXp/C4fM5BykSH+gOOKjG?= =?utf-8?q?XViDSsmzj5W9hRg5PWXJcRGEgv6y3adhSPuHczwNgSvwVjpj9LAo1A0WSVFFOMWMG?= =?utf-8?q?FRjR7bXREO+QPH+ONfctxdkBuG3oO7JY6T3PP+YEjaf4srb3pvXQRqOwwchRp5k3l?= =?utf-8?q?OjszT1xEH0nQ?= X-MS-Exchange-CrossTenant-Network-Message-Id: 2b275f08-8b81-4bd9-6ea2-08dd7addc517 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:47.2554 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tnWNTg5dVhoU2PWzio5UvYtxgSzapntFAx2J5o3z5HYKkRnwpr3+CDCpwZnHdb9ImOxNrU1PW/04+wjrCKejaw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Additional DCD partition (AKA region) information is contained in the DSMAS CDAT tables, including performance, read only, and shareable attributes. Match DCD partitions with DSMAS tables and store the meta data. Signed-off-by: Ira Weiny --- Changes: [iweiny: Adjust for new perf/partition infrastructure] --- drivers/cxl/core/cdat.c | 11 +++++++++++ drivers/cxl/core/mbox.c | 2 ++ drivers/cxl/cxlmem.h | 6 ++++++ 3 files changed, 19 insertions(+) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index edb4f41eeacc..ad93713f4364 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -17,6 +17,7 @@ struct dsmas_entry { struct access_coordinate cdat_coord[ACCESS_COORDINATE_MAX]; int entries; int qos_class; + bool shareable; }; static u32 cdat_normalize(u16 entry, u64 base, u8 type) @@ -74,6 +75,7 @@ static int cdat_dsmas_handler(union acpi_subtable_headers *header, void *arg, return -ENOMEM; dent->handle = dsmas->dsmad_handle; + dent->shareable = dsmas->flags & ACPI_CDAT_DSMAS_SHAREABLE; dent->dpa_range.start = le64_to_cpu((__force __le64)dsmas->dpa_base_address); dent->dpa_range.end = le64_to_cpu((__force __le64)dsmas->dpa_base_address) + le64_to_cpu((__force __le64)dsmas->dpa_length) - 1; @@ -244,6 +246,7 @@ static void update_perf_entry(struct device *dev, struct dsmas_entry *dent, dpa_perf->coord[i] = dent->coord[i]; dpa_perf->cdat_coord[i] = dent->cdat_coord[i]; } + dpa_perf->shareable = dent->shareable; dpa_perf->dpa_range = dent->dpa_range; dpa_perf->qos_class = dent->qos_class; dev_dbg(dev, @@ -266,13 +269,21 @@ static void cxl_memdev_set_qos_class(struct cxl_dev_state *cxlds, bool found = false; for (int i = 0; i < cxlds->nr_partitions; i++) { + enum cxl_partition_mode mode = cxlds->part[i].mode; struct resource *res = &cxlds->part[i].res; + u8 handle = cxlds->part[i].handle; struct range range = { .start = res->start, .end = res->end, }; if (range_contains(&range, &dent->dpa_range)) { + if (mode == CXL_PARTMODE_DYNAMIC_RAM_A && + dent->handle != handle) + dev_warn(dev, + "Dynamic RAM perf mismatch; %pra (%u) vs %pra (%u)\n", + &range, handle, &dent->dpa_range, dent->handle); + update_perf_entry(dev, dent, &cxlds->part[i].perf); found = true; diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 866a423d6125..c589d8a330bb 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1321,6 +1321,7 @@ static int cxl_dc_check(struct device *dev, struct cxl_dc_partition_info *part_a part_array[index].start = le64_to_cpu(dev_part->base); part_array[index].size = le64_to_cpu(dev_part->decode_length); part_array[index].size *= CXL_CAPACITY_MULTIPLIER; + part_array[index].handle = le32_to_cpu(dev_part->dsmad_handle) & 0xFF; len = le64_to_cpu(dev_part->length); blk_size = le64_to_cpu(dev_part->block_size); @@ -1453,6 +1454,7 @@ int cxl_dev_dc_identify(struct cxl_mailbox *mbox, /* Return 1st partition */ dc_info->start = partitions[0].start; dc_info->size = partitions[0].size; + dc_info->handle = partitions[0].handle; dev_dbg(dev, "Returning partition 0 %zu size %zu\n", dc_info->start, dc_info->size); diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 057933128d2c..96d8edaa5003 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -104,6 +104,7 @@ struct cxl_dpa_info { struct cxl_dpa_part_info { struct range range; enum cxl_partition_mode mode; + u8 handle; } part[CXL_NR_PARTITIONS_MAX]; int nr_partitions; }; @@ -387,12 +388,14 @@ enum cxl_devtype { * @coord: QoS performance data (i.e. latency, bandwidth) * @cdat_coord: raw QoS performance data from CDAT * @qos_class: QoS Class cookies + * @shareable: Is the range sharable */ struct cxl_dpa_perf { struct range dpa_range; struct access_coordinate coord[ACCESS_COORDINATE_MAX]; struct access_coordinate cdat_coord[ACCESS_COORDINATE_MAX]; int qos_class; + bool shareable; }; /** @@ -400,11 +403,13 @@ struct cxl_dpa_perf { * @res: shortcut to the partition in the DPA resource tree (cxlds->dpa_res) * @perf: performance attributes of the partition from CDAT * @mode: operation mode for the DPA capacity, e.g. ram, pmem, dynamic... + * @handle: DMASS handle intended to represent this partition */ struct cxl_dpa_partition { struct resource res; struct cxl_dpa_perf perf; enum cxl_partition_mode mode; + u8 handle; }; /** @@ -881,6 +886,7 @@ struct cxl_mem_dev_info { struct cxl_dc_partition_info { size_t start; size_t size; + u8 handle; }; int cxl_dev_dc_identify(struct cxl_mailbox *mbox, From patchwork Sun Apr 13 22:52:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049460 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68981199230 for ; Sun, 13 Apr 2025 22:52:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584724; cv=fail; b=WhyvYMKmZrNIWeHEksmf7IAYONPAC5WYBfkK/9sLcLCMeFEAifgh1lS+GTH4Df/XTmiVDRwbTGmDuPBDbetXP67ZJYoVp6xDOtuaToCRry1Hru/F7YfmtnokaY/Ldak54SgHxx6t+2fsmTjxxOFCw59xtX7KYAtoB/0fD2bW8UE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584724; c=relaxed/simple; bh=QWIUOdTsybkdMzgUkQnUMwA8ER+kALdCrOEkHqc97M0=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=vGHilzZb0SkPfN48RWdvtPgaerQ70S7BE8NFJz+RPRhwAuk3hDT6v0j+UkoqsuvolXvBcHZ6AFVqvMKV+BwpwKM2rC2XKuec1YR3d/maudVzSEE2b0NlDY+1aME06EgwKSnUFAWxuPTmm5kPsjX87vOkT/M58T1gt7MNL78Fa7c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=d15mz+r+; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="d15mz+r+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584723; x=1776120723; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=QWIUOdTsybkdMzgUkQnUMwA8ER+kALdCrOEkHqc97M0=; b=d15mz+r+xUwzBS6sFU3K2pRkPiYzCXrMAU+kyVQPFhAv8pMedSOrq72T gN+EKh7UsiJI953JeguqWETuN1dJbNAeGGnDGHgFTmX1Qf2ckICCOf//c OCfHapnpZNRB6z6UvPpDCsJYVRea60aQHeo4emglCFKnbOTGAsrnFjnAw 7W3EYAs4B3C6iuVsqD8aQz9CFj7wWHrCVKfHNliLcYYTKyzXTjW/j7qEf bwECXv9dcf7hClXj+2iyN1948UKN8ykpLLzKeT2bPPgfm/Df8lWr5bViM ZgDVmyb4z5PCH1jEFUWLkbGB0E0n/PI1zLM0B+b+KClq3CsfyGi/lX9yM Q==; X-CSE-ConnectionGUID: +4DdVGUmQgy4vfaCq277Nw== X-CSE-MsgGUID: 1Rbobwu6SZ6vcl1l2lPM5A== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280896" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280896" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:00 -0700 X-CSE-ConnectionGUID: xV8WZal4TcSenoAGXnnouA== X-CSE-MsgGUID: jwunai7NSvWhWqVLmlkJkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657436" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:00 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:51:59 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:51:59 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:51:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ULAItf1T2IQmu6+PeAAh3UhxpOXxsZdw2axXLN0AqE+s8w8s6pIKbKLr59xtfcqV/xM4eA2mnsUaws1+wUHUlnZf+GWj4pLwpx9sgRghqFvf7w6haRSsZHgNxcfMt5q8yYNh/IqWuj2TLqUBH8ZsDcLwtR0Ozn3c4NEVmZaFdJMjSYSIbU4qX9/CPwRtVVTiI0IsUSm1tUXXztwFS1WjJo0yeqJfPar3UpSXk8tzqgGPVm/lYRc2Uwfyr4p0qeWKHNy3nll2Bodq6NXsbKfrK+EqyEouLJYZPckgC9fOI3Sqb9RMJXvXtQRN4tSay9myNvByrDZbCZBFQjUcfhYo9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XRCSEVRqzMVLB9sUkTAqkxwHRS2MdCyEZXOrYPgN78Q=; b=xetbZdd/lvK62NBnL0hkUa4YP5Wc/4KUUJ2QFfuUY/ggDWIZOJmxDSx0SWnTmMNsYMB2u8X0VcyU5/QMZudp5hLYlIVJ8R4mmbl4IPWWbUsq1T5RmMAk7nvF+Cc73KQnm85VG/tDMupSdngjhrXqOjlZEGTxFIJzb7fnO2Ov4aLJr55xO4pGBZqjLP5g9Jzm+tHI47yMVEwQp8z82YrO/4KCQz1TSu6BKxVtX03+d4SDTzkDnH7KrM8XyxR0mJyPPz2UQ4EEjnIi1eKPmdlfDunpETQOV+rXkeyRyHPuEpQsR3UXBSfMSSdV+YZCN9HWYzG/o9+8FoaiVJqSaT+wkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:48 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:48 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:12 -0500 Subject: [PATCH v9 04/19] cxl/core: Enforce partition order/simplify partition calls Message-ID: <20250413-dcd-type2-upstream-v9-4-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=6891; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=QWIUOdTsybkdMzgUkQnUMwA8ER+kALdCrOEkHqc97M0=; b=3xtF7gtSQBOSRiRuM4Pmou0rdAEf9ybrI8OOhE/D44AoGI4QrLYY9hv+/tlyPnGPaZITIvnzy zsQIqH6chMFDryjPBTBQ3WeHnzsUBEQiETeCCbhnYaVncOZgFSxDbQd X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: cde2e068-9156-4a3a-6ff7-08dd7addc611 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?gtllh8LDU4XwBSPagI1Aikl8Ywk3DbA?= =?utf-8?q?xrVnDY3f2IA2bEQ8KVbJ/W8E6T1BX/abJM5LQReC0A8c2BQ/kJVhXNrrhAQT//ldm?= =?utf-8?q?zXrEnYOEYVJb4jk3b2VO+5njD1Wch3KhTbbObs5W/bbuz4irVD9okEHFNLr6WpOgx?= =?utf-8?q?0r5uz9u9zi9jL6Hof6BCskVYZyyfQ/jsNVc5NajSPdEn9dfkAPVPE9AQS+UawZwe0?= =?utf-8?q?rMnk9oTtbbUxDL3qRvCaw1BdOdDcs4671Tl79bR+bl6uhDj5Gu78bOqCj/sXPLi1h?= =?utf-8?q?K6xWZJ3SEhjS294dKRxLWGV+q02SMdgVnKdxzaMuDi4yesChqvpFQm7YHkVaFPZy1?= =?utf-8?q?W1eEJyAdf5yW8Ov1Zl0J1IP1RH0LSkW/bx3JFAZ6wzvwlH0Z0EDll6uh3G+6zHlyH?= =?utf-8?q?PykkPJvKjotkkRdY3VLXuzCfAAES1ILMOXe90qgnlW/w+/AB7eEgsYMLHFvWZ7LZ3?= =?utf-8?q?f8H1dMyWYZ9zjAG9BFcJZINFzCtc3F5BYkqicxwCR/09liQvMElg/7YQPto4+/XcB?= =?utf-8?q?FnucTzdsWwX51BSCsHfqQ+hjkiL27EVHY+NW3GuN8E8e91f+fXHUfbQsuESrotL/4?= =?utf-8?q?/3xHQ/Bxf87vs2AKNhPwJ/zdqup/igq7z2kRNnICYcYP6/8nWQLiJNPH7rks5WXRR?= =?utf-8?q?eRXDnuGKhVyDv9njErebY3jt1sB8ZgDWw1rVPwA5oYUBrqliszk5KiW2tzBxHEaWN?= =?utf-8?q?RQTmtelKagwtigmuMOxbKCjqoFSPeotAFbWEIx7ELWhH4rhMmfzk+YTiXP8ZCh74x?= =?utf-8?q?+Khh8s5AC83/xNNjyXDUx4kYjKTcgOzotilX7bR6+npUliONWH+jPxABbzUI9tTUn?= =?utf-8?q?JhHzINt5OmtZV+j4PZ7412TDJKdRSofnB6RjsUU/VW4rfnv298SjryDsGcGKJqKzr?= =?utf-8?q?NLMYteQu2/63RfEdwh2GIID8tfvY1Fzoge/L8hOW1Bm+SqmYYdaHxNW5wa6lcRLgf?= =?utf-8?q?PZq2qr2oC0UsHSafpnwsFLm5c40y+wWk9gGIK8Bw9Pc6YUclFlkkJuDrn1UeGtcOf?= =?utf-8?q?xGqdIQoKj9aUPMXSNk9a8mmPXiC90bKjX3Vr85nf+bHDv7wJ5L8y8KrsR84TBC3/7?= =?utf-8?q?kkErcnPfXV+lkduE8t/r8ew0Q1oLULgbTEr/Ix5GQt7ObokReaJBnqaARlKDSt8cV?= =?utf-8?q?ogBubcpPRRZ1lZx+pjYgRjA6SpZ7GA2ZHrBWgI4twg+Sp22W7nRiJgPOdCsuT2ucG?= =?utf-8?q?ikYXp7SCsvW61+WUpLVUVQ2mX+5JclVICJP5YrDLK2TKc6pHpm/c+2XVNEsKg3Q2v?= =?utf-8?q?746tZW6NbgQQD36NYXGHVWxNQuPADkYBAHuONhsm/hjUHtPXfXdfB2BNLsMdHPJ9Y?= =?utf-8?q?zuznNLdSwZy+oBvhf+nGWkI7g8wmdHovohWAnPd9sNY+JEPU82XVzzU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?70xTudAAfVesLBSA/GN3ZFfjVCu1?= =?utf-8?q?a76ybkrNO65bHxAodz9FkNAh1PiNsbQ2Jhoz1Oit8aEZZtWEg4GbHYUB7JmMydJ9Q?= =?utf-8?q?Ia6QPh/1W/a30zqPOGFzbfh2cOA+nl/6qYvbolksmBstD/bkwykwUx9nj4ZLEZCqE?= =?utf-8?q?Vpm8nPPNg91lJrNPNRsIkcNE2LjUdumyYqExZdzF9HwPOrnz8DXT2Qp17oPKpEAe1?= =?utf-8?q?+Ciwq+Ws4qo0Z2RDLzaAYqIjD9XP7tiEtqy9tzIZkN2B6OJth2phnCsAu//3laoCk?= =?utf-8?q?UHVYOkKe9yhiIJVpDGOKnOoc8ogcGakNr70F9aGMjXPnQofELEvjlNkjh5gjrb4Tz?= =?utf-8?q?LaDK43BC4hkWeH6Vq0a9qT/Q3Rpqfj5pUsC91FBLD0yG30f3yJqJQESEh5tFc9O5R?= =?utf-8?q?/sxiRzorUDPnIw+CtPmycIBV7loIXIwKgsGgrt6X3aXSXqbH8asfiXJyEEWGOihH1?= =?utf-8?q?VRkPR/SDKzHzTZwOHaWXllwYG93xT5sL+tJJxCZ2WDAHzDniRxqK7e12FR/NqzXiF?= =?utf-8?q?MycmS9qADtOAPMsv+V5SY4bwuAsBqL0biSCwxvgj92tU4q2zVhUkwJEJHkfhRB1qu?= =?utf-8?q?42yReBGGvRpq4iDjjcbPZsZzk4Ka4YqTqcZHrCWtXC2vhDjNne3UKmOII+1lgU9hO?= =?utf-8?q?9N2+2PPQo6+IH12eiZ3EK3EFgt6r6It5ZRBJbAJvfc2aW0mN6lzjGt5Jh1o31EnLx?= =?utf-8?q?oNimvoptJ/G/k0i9WRhEXGYoI5qt0FOO84QPXv82F5RbnjTcaBmL6kmSsp8oZzOWw?= =?utf-8?q?xuPPByUmwRfHkpk1S4pvs2PMZlgJ8++BRTncC1AZUCohh6UADxaFhsgj1qh+0FQIO?= =?utf-8?q?t2tzSLwdA8R7VIy/CL3rmPY75xnVyqj1jOmb7nWBlKlMf4tpg00vBbjWRTFEfEhL7?= =?utf-8?q?4+NaRVEUfgr2ekCk4yR3Yo1Zadhn7VWUTfJRlw0xSB3o9jZ22Ok+ydAM//Oc4URPz?= =?utf-8?q?9dEw7FU+5pW46xTSxmseWIW8XLwc+SzamneywdEnMucK6sahLWxzcP5ra0E4QEozF?= =?utf-8?q?yxLr+L/0ujHACLTRe8c9jDs1oXwyLM8hRS0+vDHsZqXFdRM0VyghlPEggzvsz423k?= =?utf-8?q?xWjaWfrFyn4Y8APlTQiP52ADRxD8hWHjOk7xvYixkEyNDZOh4dQ2olcaeuFYk2EAI?= =?utf-8?q?lD5X7M8h4z8tlu795LWNhfXkoVregJNsVA1qPWS+UTV6EqSVyuUktAs2a/yaSBsY0?= =?utf-8?q?qRFW5uOMkSukjlTqEI57RpCbJ29mkyqg9yIscbMYJZdiJYLftM2VXKdJwfzn7SNqd?= =?utf-8?q?ZVbqT2yINQDNe6v+L595qNRzFZ2uWsbbneWksz30l9WWVikCwSWQx/g5OX0EJT3CM?= =?utf-8?q?+aFHAPoTskEsomh2N7+SsZiQEn4w/ad62MWf3mqci3wXOpU6KdN+X1p8s8THyy3na?= =?utf-8?q?b8NY6uIlcvz4Mv4yEX+QrtVrWAV3KMpRaI9a5dPojysjiuB87eIY8TphjVtOOwdgT?= =?utf-8?q?OlGPqTFy8WXIbUgKR+Ss2meML1cJZOH/zBmYlhc33OnRJMBZWhXxhFi9ygGlUxA0t?= =?utf-8?q?Wh7gUm4RHoHB?= X-MS-Exchange-CrossTenant-Network-Message-Id: cde2e068-9156-4a3a-6ff7-08dd7addc611 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:48.8167 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pOB0zDvmq8rRIT6CZVl4SsF+r8siwZkL4OwBQhdjgKKpHL3OtGMSizu7+ivivNqP9CCq7t0WBHOw3PgYjNJG1A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Device partitions have an implied order which is made more complex by the addition of a dynamic partition. Remove the ram special case information calls in favor of generic calls with a check ahead of time to ensure the preservation of the implied partition order. Signed-off-by: Ira Weiny Reviewed-by: Jonathan Cameron --- drivers/cxl/core/hdm.c | 11 ++++++++++- drivers/cxl/core/memdev.c | 32 +++++++++----------------------- drivers/cxl/cxl.h | 1 + drivers/cxl/cxlmem.h | 9 +++------ drivers/cxl/mem.c | 2 +- 5 files changed, 24 insertions(+), 31 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index c5f8a17d00f1..92e1a24e2109 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -470,6 +470,7 @@ static const char *cxl_mode_name(enum cxl_partition_mode mode) int cxl_dpa_setup(struct cxl_dev_state *cxlds, const struct cxl_dpa_info *info) { struct device *dev = cxlds->dev; + int i; guard(rwsem_write)(&cxl_dpa_rwsem); @@ -482,9 +483,17 @@ int cxl_dpa_setup(struct cxl_dev_state *cxlds, const struct cxl_dpa_info *info) return 0; } + /* Verify partitions are in expected order. */ + for (i = 1; i < info->nr_partitions; i++) { + if (cxlds->part[i].mode < cxlds->part[i-1].mode) { + dev_err(dev, "Partition order mismatch\n"); + return 0; + } + } + cxlds->dpa_res = DEFINE_RES_MEM(0, info->size); - for (int i = 0; i < info->nr_partitions; i++) { + for (i = 0; i < info->nr_partitions; i++) { const struct cxl_dpa_part_info *part = &info->part[i]; int rc; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index a16a5886d40a..9d6f8800e37a 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -75,20 +75,12 @@ static ssize_t label_storage_size_show(struct device *dev, } static DEVICE_ATTR_RO(label_storage_size); -static resource_size_t cxl_ram_size(struct cxl_dev_state *cxlds) -{ - /* Static RAM is only expected at partition 0. */ - if (cxlds->part[0].mode != CXL_PARTMODE_RAM) - return 0; - return resource_size(&cxlds->part[0].res); -} - static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr, char *buf) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; - unsigned long long len = cxl_ram_size(cxlds); + unsigned long long len = cxl_part_size(cxlds, CXL_PARTMODE_RAM); return sysfs_emit(buf, "%#llx\n", len); } @@ -101,7 +93,7 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; - unsigned long long len = cxl_pmem_size(cxlds); + unsigned long long len = cxl_part_size(cxlds, CXL_PARTMODE_PMEM); return sysfs_emit(buf, "%#llx\n", len); } @@ -407,10 +399,11 @@ static struct attribute *cxl_memdev_attributes[] = { NULL, }; -static struct cxl_dpa_perf *to_pmem_perf(struct cxl_dev_state *cxlds) +static struct cxl_dpa_perf *part_perf(struct cxl_dev_state *cxlds, + enum cxl_partition_mode mode) { for (int i = 0; i < cxlds->nr_partitions; i++) - if (cxlds->part[i].mode == CXL_PARTMODE_PMEM) + if (cxlds->part[i].mode == mode) return &cxlds->part[i].perf; return NULL; } @@ -421,7 +414,7 @@ static ssize_t pmem_qos_class_show(struct device *dev, struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; - return sysfs_emit(buf, "%d\n", to_pmem_perf(cxlds)->qos_class); + return sysfs_emit(buf, "%d\n", part_perf(cxlds, CXL_PARTMODE_PMEM)->qos_class); } static struct device_attribute dev_attr_pmem_qos_class = @@ -433,20 +426,13 @@ static struct attribute *cxl_memdev_pmem_attributes[] = { NULL, }; -static struct cxl_dpa_perf *to_ram_perf(struct cxl_dev_state *cxlds) -{ - if (cxlds->part[0].mode != CXL_PARTMODE_RAM) - return NULL; - return &cxlds->part[0].perf; -} - static ssize_t ram_qos_class_show(struct device *dev, struct device_attribute *attr, char *buf) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; - return sysfs_emit(buf, "%d\n", to_ram_perf(cxlds)->qos_class); + return sysfs_emit(buf, "%d\n", part_perf(cxlds, CXL_PARTMODE_RAM)->qos_class); } static struct device_attribute dev_attr_ram_qos_class = @@ -482,7 +468,7 @@ static umode_t cxl_ram_visible(struct kobject *kobj, struct attribute *a, int n) { struct device *dev = kobj_to_dev(kobj); struct cxl_memdev *cxlmd = to_cxl_memdev(dev); - struct cxl_dpa_perf *perf = to_ram_perf(cxlmd->cxlds); + struct cxl_dpa_perf *perf = part_perf(cxlmd->cxlds, CXL_PARTMODE_RAM); if (a == &dev_attr_ram_qos_class.attr && (!perf || perf->qos_class == CXL_QOS_CLASS_INVALID)) @@ -501,7 +487,7 @@ static umode_t cxl_pmem_visible(struct kobject *kobj, struct attribute *a, int n { struct device *dev = kobj_to_dev(kobj); struct cxl_memdev *cxlmd = to_cxl_memdev(dev); - struct cxl_dpa_perf *perf = to_pmem_perf(cxlmd->cxlds); + struct cxl_dpa_perf *perf = part_perf(cxlmd->cxlds, CXL_PARTMODE_PMEM); if (a == &dev_attr_pmem_qos_class.attr && (!perf || perf->qos_class == CXL_QOS_CLASS_INVALID)) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index a9d42210e8a3..4bb0ff4d8f5f 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -482,6 +482,7 @@ struct cxl_region_params { resource_size_t cache_size; }; +/* Modes should be in the implied DPA order */ enum cxl_partition_mode { CXL_PARTMODE_RAM, CXL_PARTMODE_PMEM, diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 96d8edaa5003..a74ac2d70d8d 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -453,14 +453,11 @@ struct cxl_dev_state { #endif }; -static inline resource_size_t cxl_pmem_size(struct cxl_dev_state *cxlds) +static inline resource_size_t cxl_part_size(struct cxl_dev_state *cxlds, + enum cxl_partition_mode mode) { - /* - * Static PMEM may be at partition index 0 when there is no static RAM - * capacity. - */ for (int i = 0; i < cxlds->nr_partitions; i++) - if (cxlds->part[i].mode == CXL_PARTMODE_PMEM) + if (cxlds->part[i].mode == mode) return resource_size(&cxlds->part[i].res); return 0; } diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 9675243bd05b..b58b915708f9 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -152,7 +152,7 @@ static int cxl_mem_probe(struct device *dev) return -ENXIO; } - if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { + if (cxl_part_size(cxlds, CXL_PARTMODE_PMEM) && IS_ENABLED(CONFIG_CXL_PMEM)) { rc = devm_cxl_add_nvdimm(parent_port, cxlmd); if (rc) { if (rc == -ENODEV) From patchwork Sun Apr 13 22:52:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049461 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 088F919C558 for ; Sun, 13 Apr 2025 22:52:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584725; cv=fail; b=NakTxLkhTvNWZLvcCeHCgKX9ryGaNWcQ56TTv6xGqC4NzY/2rfoT6mBV7g2mUL8BBXC21xWOTgsYPnN+nlARJz0T1YZQRzVk1uUvc9fAFuPcigglDpFvBVRS99ivIt0Pxewkq7XTPlpAlDB4jesNLg3EIu7qymfoWAXv2+xAFTU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584725; c=relaxed/simple; bh=TdJg89owSQ09YHez2+5qf+RnBC3Budv6uZ6C5fSYAY0=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=CD3EGM4xodYY9pisEb4dxl5SLlZ1hBqoW5eVe3szNo9MZ+Dqxy6/5usnH1+l+RpnrtcusbGAFEPn/Zc9vUOWgEJDyemP5vG1YNQCG4XCfQ2l8DRzsy0Xr/9P+DmvLnibC5hhjAWzKfhBXF5Oxa9cnZJCEmdJoJbGOjL9FUFpDgU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hYffbEKi; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hYffbEKi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584723; x=1776120723; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=TdJg89owSQ09YHez2+5qf+RnBC3Budv6uZ6C5fSYAY0=; b=hYffbEKi645vUf6v/TvdkYDXgLoDUDOQZAgmh+Wo3pmQm6ELarjJWlWm 6mbYHJIjE3mMbQnm5xY3lumZCpQodAicKSwTYDZVvKI5zzYKay06/Dlep mGvaTsXDeWdUUL3XLyweywKACgpIIr5f/7oK+oAs8Db8Dmj/5oA3W7+Ui z74qYPU7YwFlwbbMaGRO6vcWMXX5A4Tk0p6suvQXpL9NIT6ACkJI/vgVT 7DfIwWZoexlET8sn8CI62z+0RR3/gEBJGl8Wa2SgHMzU41LT5SdzI8haZ 873X4PvsgftixXO1SBXkJliIKJK1HYu6QBDfsg1iaN3snRgwZIbRl0UPY A==; X-CSE-ConnectionGUID: QdHIq2jmRjyC6UW/tlhxXA== X-CSE-MsgGUID: kYkqfBkjTJaMARwq8THesg== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280901" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280901" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:01 -0700 X-CSE-ConnectionGUID: cw9qAucuTI2MxsEE9TmFfw== X-CSE-MsgGUID: R7Vezj7yQ8ijVOd8oT602Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657443" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:00 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:00 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:00 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:51:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eBQaFRjOMLZaAKTugbGzGxbqDEMSIEevlin4ZoQ69Skwzg6WfIihHzEvfS1cIL4KmSGF+Q8QRiPyErSo7m/cotiDMaSLS8vnHQIeEFY7aODDnhmv3X5TNywPa0ucv6AAh522BH8qcRMsM7Ij54Ah8YZ0k//s7CDP0lrujKNrGjaZ3BzCn+dzPSiIVqOBm3k2owyeOGWdJCezV/Q2ohVo/KwcwazS0aDlWmvn8ffA91V0UXK/P1M0bH/pqzWUEkE8fyx+msAnxfCdxDgK10W3HlbzqPbmKtcHK5WffLVGcCCULalo8xr/oyqO/eGS5dLJLns8//u6GsnyTlf6Y0HDBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zpWif/CnuOIudWRzPn5nWqW7axZb6SRcXPd+Zv+hJJo=; b=M1BjPQ1leSW9dUq652ZbZIWjq+EN5BjJE6osGwBIy5mRoqa/99l31NF/M1AYaMgwzKYKRsV6A8PiLCvH5zFTYAKZ8TeA3crMQmDKEKraOKTLc/ne93pRNVqQy2f+n2FsEfzo2yg6aZRgEQH4RNVGEMDbdcNKf7IF5M7yy/0g1S8w0ewLeEtyNpIs9djoRAWw80ZHOg2QTfu/1m3lBbJeQnNL8b7HtiBwRHENw2BVkbmxie4vBaav8qjgrthZ1VnFaIM14KC5Km3R1wjYdeUxmDyCMy2NUWNWSUmXLVMXAkgjYpVYzVyYqI/pV4+hxzNhpIpyFU9ktdjKUypQlc+EGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:50 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:50 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:13 -0500 Subject: [PATCH v9 05/19] cxl/mem: Expose dynamic ram A partition in sysfs Message-ID: <20250413-dcd-type2-upstream-v9-5-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=5374; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=TdJg89owSQ09YHez2+5qf+RnBC3Budv6uZ6C5fSYAY0=; b=bYaZ8kzJIjb8F8TRq5BXse1Q94wKep+lL9bHykZ8N+OJ0h4nKYHFt6wGA7a7K6RPoLilfcLqD R47OVHjjVT4BOdGO4lMq24TVqNMRpIdQCV7KvHoihEDvEA3JoSLld4q X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: 22ed960e-0423-4f3c-0fa4-08dd7addc71e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?c0ktvlXs894Nr/KHhNms2Tn8X4NcNyG?= =?utf-8?q?lHkCvGzhJ/I8JgClL0PiMFQNRgBptn6i0IIEG1Jt3+VQL1EWF9uxTj/T6RXoYAiCe?= =?utf-8?q?rtYPUfcUR+MqxpfE6g6zlPnUZ19REox8GI8aVWEQrf3A6cNl3g6lFROQ5lkKkZtwM?= =?utf-8?q?BH/7RiJVTY5qgwTP3TsbR02+xMBAoU2mkrxmQnWQrTHoYM/ogD12ieXYEq1E2ehRo?= =?utf-8?q?vTZ3HGc5X258XjspDaMZMT2H+1tXvnpJQtlZYUKAK33l6DumHzEqqD0lz0TWcfIoi?= =?utf-8?q?h7s4rdI9gfQnvOYDE5gLvZwdYVC8H26U5skHKVS1zDqqiXj54yCd6OygfIG+P1Uzz?= =?utf-8?q?1/ANYkpRtAdIIjSN6L96WIVkde08Ogbu/uyhJFwwN7yvvKZHYnGvBOoI3brYA7zrL?= =?utf-8?q?zyZiXb7dXvXEGGPDKYN0Vl70W9vCCeaeZzClOE0dLpfWdXI/m0FCwmyNpaH3tXt66?= =?utf-8?q?On5PABMd3StEpGmZc9sRn/lHBj3Wr06T6lQo6DbisPLojMCHvBMfQc6Eyl9OCuCqr?= =?utf-8?q?ZkW4Ubin3pfP+o1ITYCV3LefDjI8ZBwRePntvAn0nnDQfLPx8vm63/ju6dq37526g?= =?utf-8?q?8chIzYqe0nfDbfFGLzErjtIVPbsn/m36+DRofus8AVtBRCbNPQyUdozSpq5vA4jxL?= =?utf-8?q?s/oPBj6PXF0jimuHwfQ3DZMguhKOhuV8QiK8TkJbNrwatwucUVn6Lj64SGEw1rXfi?= =?utf-8?q?a0ExFOtNS9AK5+a2ECVggwE2vS3BhdT7ZzEJeAaaO+lyZM3l90a1X48b4BtkSdXr/?= =?utf-8?q?lpEktwJm9mnp0Y547A674y2mctlz2AuYkB0UacrM1ulWKuN3cCHSVRa6q9d3cfSOL?= =?utf-8?q?mklvyNvtFbECqYNxyC6CBh4hKXWIAs7hJGZk9K7X1BoR+rCQmcNYGBiJkFcSJ2xic?= =?utf-8?q?9dAF/jR64sWM+9LpM/Y1ahfdt9irNKJqvBohCscuRQFHeeRzqtVNbX8xQX4wCPv8g?= =?utf-8?q?N7QClFtEXN/E6qHfx48ERJg+HlAV8/mPDrmH+oc33NAUIMgssaKXE4EyRGhwb9IBQ?= =?utf-8?q?Vk0sa7AfeYccyREJzJFk3U9FpPBq6lR4Ulm8CeBfMgvuZ4UdpLOEKUAgyZvkGzHr3?= =?utf-8?q?tjhrZqxBbEPb1a4oOHEvAXg1eq73wJc13mIV71fWXmJ/WmpAnKzVkWXRBOxf5LlLJ?= =?utf-8?q?uyqHG6cwqmppGJS61hTKeNk8IsNprIFwHMtYHmEMrela2NUTbNJD8Pgj27ZtTzXBi?= =?utf-8?q?b0KGO6RCy5JylYHMYOtH73vcb00ROv9SSY+gAo5rskE52isYydBhR1rnb9aZs7CgA?= =?utf-8?q?KJV813z2V27A90/ECfkIRoiIUem2jbl7Rm/oMp7+/zeqV9IF67c+krkG6i4QQjKmO?= =?utf-8?q?BCRhvMOu2zSEWFYLELdu1dMBTZgj5jI9i/yS6rBzRDv6OrtUc/+5Hyc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?hB/6+vn7AskUyEsp/IWPNjpwenH3?= =?utf-8?q?QLUZkYeVvFVaPPmdhLJ0CGh2uWj+r46o8W0oEYVKcsNJR1qNY9mOaNRn17bv3Tica?= =?utf-8?q?XZO3LhXixzQIcZMWl30VZj2mLN9uK8jle5IZXDoGeNyujt4RBk/0ueg/0RC5lXWlY?= =?utf-8?q?K0L1/SblQVhF7HMEhqA1r+GHMyjIn4AFRaOwjwBCafNckFcVcqc4h2xaq0tpJ9pqe?= =?utf-8?q?W+K0KaGSF4+FkugMivUpW/gujqEEo3DZ6C7Wll4pZxnAAGZIBHWhOUX0AEixgeHAa?= =?utf-8?q?sai+8NEyVx7gX0eqj17aonmwVjGgrauiqT+XX7fJBYqglPeYFbSU75zw0R+Of3VUw?= =?utf-8?q?18NeOpi2DbeGxI58egbajRw1740srMaWvS1R4Ft5cjzUCEHN7ZXdHz/RwSEitmnja?= =?utf-8?q?TI0pmFK+1QZ0ueC5Myo+6mIMZlZ7INzlKq6JyRDoS6TEDliQ10Uff4L1QDCq8vwNc?= =?utf-8?q?0SKC1OAu+iKTM8ornXzQdSLZ0ElJK1kWwG0tRKUHEfaNkdoS3KCOvO6S68llL+6Dt?= =?utf-8?q?jgZ6eHBLVfVb1I4cHVvN2Q70zaGpsk0OeysiPMioE5Rk3QTVObMvMtvzhjIuaouGu?= =?utf-8?q?EXOvPZ0cH8T2ZBLJDaS/iok0oNMUxAiQNwpqN7tC1dA61TQ5W2Y0gHOWCEllUTUbK?= =?utf-8?q?2t4d7M7xuc8MdJr5ijhHDtC5PwONAbDCeWQih+d52trXwIf2zsY+QCCR9qIdfXIAo?= =?utf-8?q?TX0FKX5O3+oz+6i0wkqbbAT49M9Io22U3DvlC1bLBQbmrdTfuUIXRZxeuxExjGBBL?= =?utf-8?q?zNbVuUPrzLs+rTTE95fCMmLErJDlHUVK0P8bBsUzygME41xga4HetlwKEiUWkVRoB?= =?utf-8?q?CNS38Z+AyGll0zTEOqXX/ZttZUqZIFDq2sFPiioBvf9zcmVqiHyzSZsSEe3Qch5F8?= =?utf-8?q?2FmXPOzSeV6hlXVLZg08UXcrfQdzrJ0JmeEpXLXu9Pjx9Lm6XjC+pQIAJdMNgG278?= =?utf-8?q?H2A8VJ4ZebNIsu9ToBB3MVPvDXO5Oq0fKFMR00+r65Bku2fTEjA+G2vaN6emed2LU?= =?utf-8?q?47yciJ1HHHrggOAuzlu52L2qhJjGS89Ti8SYxnvm0E90QyLlPxKadpanBo/sdvwDz?= =?utf-8?q?bP5A7Mm7O8zUHCxRTdVHNQEE7COAsVv57B+svukb2SB8HUu6imYjVIlPR28pQcdLs?= =?utf-8?q?bW5Ed1WogIEJWY8uGK2nhjBoUhw+RcAaNIUKz1U0qhrfcjDMJoELAFXnSfIFj0MGW?= =?utf-8?q?NeVGYGJPiELmFx4kD+YtPGrZFHFXvOHhaxbA9gESHkyEIeb9+iArh83YXXns5Cs97?= =?utf-8?q?uDlCf39VWY96cd8N0oWLQv1to8sS5Any1Ego51M76anED6cxqGjijm2UJzTSzRoio?= =?utf-8?q?mwzRwSzT94KL/LyKf5UZpflyXNTCGOMHk3dAXD/+4IUQUHde45SakRVsyocfcoU61?= =?utf-8?q?xCmBgYMXOXqXC1t4F3HgFrNL7hctRoTk3aavdzA9WC7uog+1e52x0NLFJuv7fy9E5?= =?utf-8?q?gMXBE39txe7JS+nNKuG6eybj41Kc8JRSM/gPZ35dP2k+frKAidhMJEwShftjc9ycB?= =?utf-8?q?c2hlNWziK7Hd?= X-MS-Exchange-CrossTenant-Network-Message-Id: 22ed960e-0423-4f3c-0fa4-08dd7addc71e X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:50.5782 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: s89zLihhIQAAQ/wHPymdfRXuw1qTU4e5TIr+nG/ncRTMlKTlWrno2qPgke8hv7OqQaJ2o673ztDowHLCyV9ZKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com To properly configure CXL regions user space will need to know the details of the dynamic ram partition. Expose the first dynamic ram partition through sysfs. Signed-off-by: Ira Weiny Reviewed-by: Jonathan Cameron --- Changes: [iweiny: Complete rewrite of the old patch.] --- Documentation/ABI/testing/sysfs-bus-cxl | 24 ++++++++++++++ drivers/cxl/core/memdev.c | 57 +++++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 99bb3faf7a0e..2b59041bb410 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -89,6 +89,30 @@ Description: and there are platform specific performance related side-effects that may result. First class-id is displayed. +What: /sys/bus/cxl/devices/memX/dynamic_ram_a/size +Date: May, 2025 +KernelVersion: v6.16 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) The first Dynamic RAM partition capacity as bytes. + + +What: /sys/bus/cxl/devices/memX/dynamic_ram_a/qos_class +Date: May, 2025 +KernelVersion: v6.16 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) For CXL host platforms that support "QoS Telemmetry" + this attribute conveys a comma delimited list of platform + specific cookies that identifies a QoS performance class + for the persistent partition of the CXL mem device. These + class-ids can be compared against a similar "qos_class" + published for a root decoder. While it is not required + that the endpoints map their local memory-class to a + matching platform class, mismatches are not recommended + and there are platform specific performance related + side-effects that may result. First class-id is displayed. + What: /sys/bus/cxl/devices/memX/serial Date: January, 2022 diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 9d6f8800e37a..063a14c1973a 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -101,6 +101,19 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_pmem_size = __ATTR(size, 0444, pmem_size_show, NULL); +static ssize_t dynamic_ram_a_size_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + unsigned long long len = cxl_part_size(cxlds, CXL_PARTMODE_DYNAMIC_RAM_A); + + return sysfs_emit(buf, "%#llx\n", len); +} + +static struct device_attribute dev_attr_dynamic_ram_a_size = + __ATTR(size, 0444, dynamic_ram_a_size_show, NULL); + static ssize_t serial_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -426,6 +439,25 @@ static struct attribute *cxl_memdev_pmem_attributes[] = { NULL, }; +static ssize_t dynamic_ram_a_qos_class_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + return sysfs_emit(buf, "%d\n", + part_perf(cxlds, CXL_PARTMODE_DYNAMIC_RAM_A)->qos_class); +} + +static struct device_attribute dev_attr_dynamic_ram_a_qos_class = + __ATTR(qos_class, 0444, dynamic_ram_a_qos_class_show, NULL); + +static struct attribute *cxl_memdev_dynamic_ram_a_attributes[] = { + &dev_attr_dynamic_ram_a_size.attr, + &dev_attr_dynamic_ram_a_qos_class.attr, + NULL, +}; + static ssize_t ram_qos_class_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -502,6 +534,29 @@ static struct attribute_group cxl_memdev_pmem_attribute_group = { .is_visible = cxl_pmem_visible, }; +static umode_t cxl_dynamic_ram_a_visible(struct kobject *kobj, struct attribute *a, int n) +{ + struct device *dev = kobj_to_dev(kobj); + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dpa_perf *perf = part_perf(cxlmd->cxlds, CXL_PARTMODE_DYNAMIC_RAM_A); + + if (a == &dev_attr_dynamic_ram_a_qos_class.attr && + (!perf || perf->qos_class == CXL_QOS_CLASS_INVALID)) + return 0; + + if (a == &dev_attr_dynamic_ram_a_size.attr && + (!cxl_part_size(cxlmd->cxlds, CXL_PARTMODE_DYNAMIC_RAM_A))) + return 0; + + return a->mode; +} + +static struct attribute_group cxl_memdev_dynamic_ram_a_attribute_group = { + .name = "dynamic_ram_a", + .attrs = cxl_memdev_dynamic_ram_a_attributes, + .is_visible = cxl_dynamic_ram_a_visible, +}; + static umode_t cxl_memdev_security_visible(struct kobject *kobj, struct attribute *a, int n) { @@ -530,6 +585,7 @@ static const struct attribute_group *cxl_memdev_attribute_groups[] = { &cxl_memdev_attribute_group, &cxl_memdev_ram_attribute_group, &cxl_memdev_pmem_attribute_group, + &cxl_memdev_dynamic_ram_a_attribute_group, &cxl_memdev_security_attribute_group, NULL, }; @@ -538,6 +594,7 @@ void cxl_memdev_update_perf(struct cxl_memdev *cxlmd) { sysfs_update_group(&cxlmd->dev.kobj, &cxl_memdev_ram_attribute_group); sysfs_update_group(&cxlmd->dev.kobj, &cxl_memdev_pmem_attribute_group); + sysfs_update_group(&cxlmd->dev.kobj, &cxl_memdev_dynamic_ram_a_attribute_group); } EXPORT_SYMBOL_NS_GPL(cxl_memdev_update_perf, "CXL"); From patchwork Sun Apr 13 22:52:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049462 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 869071A314E for ; Sun, 13 Apr 2025 22:52:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584725; cv=fail; b=gZno8HhAfvcxqKVtHgBh/sBYnlOJhjqyZcbkP3YhQsUYhYVsbjPNtrs4YIk/i52R8zcqBXeD8jPxsGm4LkNSAFuJPSBJWY0hunmo/5X6KK3dQfaHoYMe4N+FDGx7ldWkAn8+ZC/qoTGR8MeuKkxJBerb2SDr8Ww7G1Hcuf0ld+c= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584725; c=relaxed/simple; bh=ubkszuk8OOj91HHaqwNgXvdiZweiDuzDaFgiJyN6lDo=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=n085m+kwF4tySG8e5ZGylGl0KANmlJPizYWzJ57GwbdcwXTB/R9dWzaAJrZQnCBAKinYIEWVoKh+DU//XLYi0GCm+D5/fE7gomIKFLBkSPicM5/U+DXZ3vzh50wLa73s1sMzju/Ug49xOCs9IK8deUCavdTCWZXD7cqPxg7FZuk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aUK0rlOs; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aUK0rlOs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584724; x=1776120724; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=ubkszuk8OOj91HHaqwNgXvdiZweiDuzDaFgiJyN6lDo=; b=aUK0rlOshktwjIYx0iSfbYu8lHU6qfL3U32+EKSGTM7j75ujoevPrAYU cuKnPDFCYfQdCg11rTALXCnapLWEJbZgAEFJKcT00vMj7VpRAbP6vNrg4 h7omMhAfauBeQn/IDxXJtq/aDqZoZYU7KO53qfaQXqxApQCjxschDRcvc zNXXJ7CgZ2gp/6VaCZzS3d4BIA95PCctoJDhI3SMb0xkZpgWPBLc4n/9o cRUDvzey2mB/keU81wYTdVEjcobYAljhn44JTbk/ZOUk8zCQc2ca1PeCL I+vqIOSI300U0oI1D6nHsJCsN0mDZqGq3FB1IbjGOlpoDGvy/OUZgw+80 w==; X-CSE-ConnectionGUID: PFmHxcsIRMqmLBm61JSSPw== X-CSE-MsgGUID: LJ3wwJtrTt+VOCjHo9cPMA== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280906" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280906" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:02 -0700 X-CSE-ConnectionGUID: Xh9v8XCLSDC5H+iLXuSEHg== X-CSE-MsgGUID: tplrpwWXRu+e+PeQyBnsDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657453" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:01 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:00 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:00 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jZ87yonip5Zz+nSBkTMAQ1A08Ic1A3xjIjSc+uwBUMC8Z7Xye9uCtHgbOo9Mai970eCbCvDJGY5RRUd9Vywp09NCrF+wIlPZdrxL5isKZu8grSflZpD3VTz3RaxDOhh21qQLamZa6qB4F4wgPtapo99XTEf/KySYhxAYixEcAD8Dsq4YIOLtV3Zv156F8uZ71UfU9hHQhNngDbgSMIkEb3Bf8PBiyeLLlcvq1OJ0Y4PKcbzW9V3xP6CdFX71D7Ua7vlhhHpCF8Aof8GJtDWSbtVGtlj7Tcuffngv6oD1ghNwTr3Fp3bT9nl+KWOGOWHZ7HmNzBlHL9bKD55vnKlucg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rQK0+O4GqSKPKOs9Sr/fszqYqvtsWCSqVUxvStUxO3E=; b=AzmljhdD6N+PfDkBOMNznhWXnGuM723cqfxOdsr9HWTfHBwiEzT9AuLvN5f5FLZFjV27p9Q/DUPtdT1PrflKuUpdgFT3ZIoGTCcxRXqsUAAL00FVMziaTLxlx2en2lCqsS9tyoNo7C8mg+1dMj7FZn8nC6GaohBLeh4yf4h+PaFVIJiDbjLHj9+o6Ynx+d5dva3JyIaPU2AtP+XVtLTgAD9thGBK8Ytl7ePOEWzWly0uZhhvI7V/pwTuRZaPV6i0QaDR5+65iPRUZPceNnTqZsp6PuTnMRG/4lStrW2oq/3NWfbWMwE3LxNeFX1VBVoEeqGErFIqwtCJRy9f72wrVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:52 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:52 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:14 -0500 Subject: [PATCH v9 06/19] cxl/port: Add 'dynamic_ram_a' to endpoint decoder mode Message-ID: <20250413-dcd-type2-upstream-v9-6-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=3257; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=ubkszuk8OOj91HHaqwNgXvdiZweiDuzDaFgiJyN6lDo=; b=PqrpuoOgqfEFIv5Kmfg2/Jpcf6p/+KUEk1ohlJOdVsWF+mMLc/Dtjg2VYYzjPMSOZX5GjI+pY XENW8xJ4pD/BPaI+4OsCHUdn12rI5HJccuek6nFn/yuQs+M8l0hIKy2 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: d62e3194-dadb-4df5-6767-08dd7addc807 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?eYGXQOB8XKmi9V+2ETOIr/VJsA8b31L?= =?utf-8?q?qldjAUKfOhC8QLE9WhYv98Yk7nItL0cgWFtLRXB+EK4hYbNL9h9dNNzwCoNNzQRLA?= =?utf-8?q?/p7QmHYYTHgTDZ8hLux2z9KwVkF+LxO9dw3gbj/xIQpodvGq26JpEiBqqQj63iOQc?= =?utf-8?q?eRylsbnOwazahClq1cNHQQXnxhdpfgXK1Z0DxfEVrQ2D0pbsWBsqxqwa4GL3Md2Vv?= =?utf-8?q?9+pVd9Cq7e3uPdiQDT04hjfsmG4+2T02V/ctk4YNQusgiwTgj8eKDAhYELr1JbGFk?= =?utf-8?q?OzuI7X9KBM7HxITh8O0VaUp9jiD48Ebu7ruEdeasZDzlWgo3tIG3v4NU8wt2NCjyV?= =?utf-8?q?sLyxv046yjBA1Kh8UXhrddjq9lIB8SOKKBLTteeNpnK9v2MvTZzLMZxhAZvvTZHJW?= =?utf-8?q?tTPW1CAns7TnolexXU0tex8XudXhKj0BSeP/3jSN2hehlSoEAi6PzL9GjjuWMrXA0?= =?utf-8?q?qpvd5KVxL/yBhkVmjAXDPIhCcFGBk5NF5fKccAjGd2lhO6d8Yw0icdRqf3MqrlOX5?= =?utf-8?q?iExTWfzzk8XytV8Zc26fnizNaHrnNkDX9gZGCYNUU3mq35L7hWpX+DBGtDQIsHAUJ?= =?utf-8?q?LuGFDGWn/B52qokJZ+wrFoO+7BOw0a7QOUAdG3zhLLQT3yf3HtYMdOluQx8sC4H4E?= =?utf-8?q?GLeJMT0PO1+rUh9khJBZTexUBX259TAbDHivqc4M54u5mQmrtBc6XursPu07z1E1t?= =?utf-8?q?JUjVHl2QuUsGsGftNobEVJuBi/wjsWqnf3SeGhMjqqDIBhoUsHHOiqKn3HpxGmroe?= =?utf-8?q?hx4QsPskbuaXT/0yuf+aPD6IJPbUWJpSNy1ECjtEHQjnHsURmipcAYVoHFvOoYXPz?= =?utf-8?q?4cleS+lkQPmSVBdq/B6TqOZVRleLBlHiIrcyCfiRTqyPAVnOgG9+1OaagKZ6fSpsz?= =?utf-8?q?0/XsXukSiYmbN+izZs3NxNhbWQkClzcisbTOlKiGjbBbQdDGsNlcbriPZfn7zbXRF?= =?utf-8?q?uRJBRYNDWBomwhR5YnzUhb0adQmZoXLXZyivKtPBS+smjjIihrBQ1eea8/UTBl+it?= =?utf-8?q?3joUILgHri1ZsBBS2cxICbSlmuO6UFXdC7GkPlk2mJIKxxUzJzUCLFevNIhe5/VDF?= =?utf-8?q?r93Eu7ScwkIBUmM/MivXUWGBQZexaGBMkxF0oG4ddweYFuEY5lwXS66+BV4TOdvJa?= =?utf-8?q?EL/3BFeZhtp8/QQDBFti7AQzAJVrY9RF2xaLSbuBraPR4BHenmeDOiA7oyB0Bevwf?= =?utf-8?q?DFba+kdDSmXn13YsYsj0/x72PEYKBl5hUecCUzQUiBAvH9V3IMZololMm+55UQ6LP?= =?utf-8?q?LRQzIEZeweqDUw3WylPozgFg3gziMATQhmZz9MND8GLjCLcSSL+Z24rRronmnqlam?= =?utf-8?q?XYYM9bh9MGUMuwIZvTxB6Hz+OqwwaomQqKuNneiv/HOhHF7R/bei6Zo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?3InJo41hAEvWgKOE4y1Fl9x/gwas?= =?utf-8?q?Ifp2uOFhC+pA5jWUkFYNb+qiVNWIC2EUTDWjFalo9WPARgHUm7axDKwibnIwY+YND?= =?utf-8?q?aF8L2az/hNwX5nWgQusbTrjS+8S5OVErPaDcTg57Y7hhIezYzhVNLQCcECve8kW/r?= =?utf-8?q?iGGaca0YLRs6Szwy5nKznRNule95oxQvaQ4ApSWKk58BgaygGmWbIQwwLT0KZRvZx?= =?utf-8?q?lg1lVVkO6Jzllkj5GjbxrgHapnZO5GD06n6BmGs5+NCXCNLNi6GNvubgP0XXJM13T?= =?utf-8?q?53uMPqV70m9CQ74hIm8CIzmz5PbH9CMioRTPMHkUqxIABqMQel2kEm/8h8LQ+i19B?= =?utf-8?q?U9KHkV2lSzqEncKGAb71mF2MQDgl9yxIzaMkKGgw2SzCbfocDs8N28iYN4mZbSnFZ?= =?utf-8?q?IAZo6Dn47clx1yom0QYZ8wxPnkIRuAGyaD+aNJcfJnOblGkaKB7J6HHnRXeV5ZTtx?= =?utf-8?q?rICT7GJYmHSk002girrJMKakaroGA4cjorJPKB02Fo283Pdu2qtset0pi0DDCYbaf?= =?utf-8?q?nPeeif/ZxpW2gNpm0pATPKwh2MP8oV3yJTwTgYcrngU9gjZxbiCJ2bxe02PMT5kki?= =?utf-8?q?zeKuKRqRzaK6yD7wpFirddqsHlIEUeD2enoxCh7jorAO6c5AI2arKBnyNWliExuON?= =?utf-8?q?NgrpG7tPBX/rcCZtwi5dwUHlJGlslYoKprLcPjBbfcMrouZiYkGQ4X3jRfJ0dG6s3?= =?utf-8?q?S4S8rmWsG6lfwy5S92CisTUkeMIqh32kBQJG/7rtxwZF5oa9OluZUnw3z3/gdn0ep?= =?utf-8?q?4jfJImLGDQmcubryWTlS/mPutswHBxSQzZVyYRSXp8SMYumQD9l7xIC42Hb+0LB4o?= =?utf-8?q?2H4zAtQDqtO+vSfQSCIdt6QqD4Cd2OuvRiG0Z8BeR6K7TO56zoWWjRbPfSKSjDO0O?= =?utf-8?q?OaTMk6GkROlF9GfBIodfWMQ+xP9HQ/elYYZe3+bdBGd3ev1bRxsvglhqbIEoUrlQ+?= =?utf-8?q?xkIWM5SRVn3cPkrpbl8up7v1Pj72w78SWvrsV/FVygYuU7f6h4kGw52a691gs95iD?= =?utf-8?q?N7y1L719o31vpjk8uHAYWzaRQmOyL3vjPlu057dg2BpguD+x8W5HEZmGVBiLYgK+G?= =?utf-8?q?QN0EH0N7C+bWWz+iqM4zhe4k7XPRubhQfRQaxX67vyjFCjcpfis0Dlwl6kptu/jDz?= =?utf-8?q?1Caa6dOEX8ht1q9XQQIYw7b04orU+7e6Oni5Hu53GCfGNBtoefZ+Wtgtl4fx9jzFc?= =?utf-8?q?5AsnYx+lJwaMhuu5PAjFBbetx73Ci6/XcECH0eEj/Qgb3+U1mpvClQQLxROgJdHe3?= =?utf-8?q?OsOg7Ol1xTuNb5sKIKbFCvAxIyTOrCEBA3J+HneIZm54v6IoZteFaF+couRSmJ1Du?= =?utf-8?q?FrxTDf7llrvzwHi1E+4TtOkCeiwnSB7Q2VySlatUfHa87Fu2A5XFvQa2u1D/9Rcfo?= =?utf-8?q?ocBsffsyBxOMNw2vuyZXjAA5flqfaZ5JazWVmcIVyM2FdU3Vs6FTFlXPx9o+n6SSO?= =?utf-8?q?5Uxpc4NXcbEyqY19zC7zLv7fUMT/3Gq49s2C247sOBnqJkQKRNnu75Si/VnDXYEkx?= =?utf-8?q?b3gqT7GcTb8p?= X-MS-Exchange-CrossTenant-Network-Message-Id: d62e3194-dadb-4df5-6767-08dd7addc807 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:52.0536 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vCzSVqp9hV1BqoZUQxT3PydRuagBpIaCVKOdyh+ZaSk8XeRan1Or+kWfSL81p0FPIWLz31bCYAribZPavJOy5g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Endpoints can now support a single dynamic ram partition following the persistent memory partition. Expand the mode to allow a decoder to point to the first dynamic ram partition. Signed-off-by: Ira Weiny Reviewed-by: Jonathan Cameron --- Changes: [iweiny: completely re-written] --- Documentation/ABI/testing/sysfs-bus-cxl | 18 +++++++++--------- drivers/cxl/core/port.c | 4 ++++ 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 2b59041bb410..b2754e6047ca 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -358,22 +358,22 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/mode -Date: May, 2022 -KernelVersion: v6.0 +Date: May, 2022, May 2025 +KernelVersion: v6.0, v6.16 (dynamic_ram_a) Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it translates from a host physical address range, to a device local address range. Device-local address ranges are further - split into a 'ram' (volatile memory) range and 'pmem' - (persistent memory) range. The 'mode' attribute emits one of - 'ram', 'pmem', or 'none'. The 'none' indicates the decoder is - not actively decoding, or no DPA allocation policy has been - set. + split into a 'ram' (volatile memory) range, 'pmem' (persistent + memory), and 'dynamic_ram_a' (first Dynamic RAM) range. The + 'mode' attribute emits one of 'ram', 'pmem', 'dynamic_ram_a' or + 'none'. The 'none' indicates the decoder is not actively + decoding, or no DPA allocation policy has been set. 'mode' can be written, when the decoder is in the 'disabled' - state, with either 'ram' or 'pmem' to set the boundaries for the - next allocation. + state, with either 'ram', 'pmem', or 'dynamic_ram_a' to set the + boundaries for the next allocation. What: /sys/bus/cxl/devices/decoderX.Y/dpa_resource diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 0fd6646c1a2e..e98605bd39b4 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -125,6 +125,7 @@ static DEVICE_ATTR_RO(name) CXL_DECODER_FLAG_ATTR(cap_pmem, CXL_DECODER_F_PMEM); CXL_DECODER_FLAG_ATTR(cap_ram, CXL_DECODER_F_RAM); +CXL_DECODER_FLAG_ATTR(cap_dynamic_ram_a, CXL_DECODER_F_RAM); CXL_DECODER_FLAG_ATTR(cap_type2, CXL_DECODER_F_TYPE2); CXL_DECODER_FLAG_ATTR(cap_type3, CXL_DECODER_F_TYPE3); CXL_DECODER_FLAG_ATTR(locked, CXL_DECODER_F_LOCK); @@ -219,6 +220,8 @@ static ssize_t mode_store(struct device *dev, struct device_attribute *attr, mode = CXL_PARTMODE_PMEM; else if (sysfs_streq(buf, "ram")) mode = CXL_PARTMODE_RAM; + else if (sysfs_streq(buf, "dynamic_ram_a")) + mode = CXL_PARTMODE_DYNAMIC_RAM_A; else return -EINVAL; @@ -324,6 +327,7 @@ static struct attribute_group cxl_decoder_base_attribute_group = { static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_pmem.attr, &dev_attr_cap_ram.attr, + &dev_attr_cap_dynamic_ram_a.attr, &dev_attr_cap_type2.attr, &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, From patchwork Sun Apr 13 22:52:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049463 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B9A11E5B70 for ; Sun, 13 Apr 2025 22:52:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; cv=fail; b=ZVATsUQX6/4Uv+PSqCg192d5AMj9oWCB1qiBPKq5MlLbtymGBegl1kqA7ySMBvW9e5XWqVk2/M3uXJA/y/hny98jmsI+1tjREWmwfs011jpUOQYsEDF0ZQFRjcSnUN8etaxNgoOEtXbNc4GMgR2msgoKG4fo3uX7uZLkSmv6DaY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; c=relaxed/simple; bh=Ue9nxkCBAbAQFWbu0CQ3m+slfJ5d9+PWGNBQeO6mizY=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=cn6Rn04ET9egIYjKjxedgVDn+qCQ5QKmTtTu0PvyelP+t0gV4F2kCSeTVM4dETaqvNhU3J5tnMnHzEcA8rxFbL7EdGOPOigD3U/Dbi0484QoVyojIoBUFMMmQa5FMfBTXInZM/qT+dau7KSZsvambTtz47viEYYVi2fUwUjjlQE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CvekO5WV; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CvekO5WV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584725; x=1776120725; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=Ue9nxkCBAbAQFWbu0CQ3m+slfJ5d9+PWGNBQeO6mizY=; b=CvekO5WVAuh5vDJfqah554r+9vaU7Lg/6DdiKn1u5sZALBNqqhY/hk+5 vVNss2h0p7laJZia64xbwibScY6FFZ9Y1nOXZJjyKvR5/Ye0ExhQtwC3H NgqJHpvQiJb3iRlirMpZCHMLViXapC22GTtuYyskepViU1whIab+l2ujr incRoHfRO1nBE7kL7SB7KVTVkS+T00J0hoF9bmX4CYo/FrrnJB08M6lKF o6iboZeiXDl4N4DESWPw9HBP9OkiFJrYKSPwSzceQ9sKW/rQ+Bpthxrb1 93PDzSlevLhwnpTsmo9Vn6y2T6KuioWBGr7u8/EILqPxxKm2gsnc+42Xp w==; X-CSE-ConnectionGUID: dolVbyvIQS6JmBgDATUbKg== X-CSE-MsgGUID: m94c6+QeQ/WxyKoL7cvvrg== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280911" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280911" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:02 -0700 X-CSE-ConnectionGUID: G/R6Q/J1REeJWdnJ3PK/FA== X-CSE-MsgGUID: zcLim1KxSfS87+lZuIhUrg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657463" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:02 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:01 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:01 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:01 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UnYaRK2K5joiLzCXU0ypa4E9N4OwIWcVmCTCtTjC5ZfPatbfoA9dGT5YmVZrtZGWoRC5+vdWiEZp5OAc6xpLefcMTNtLZmrScgmlJWiSjyBe9NCn5meGRVhdutAWEexDfPn2yCteusFGyciBTfBIaq0gEMv+Y/RUIgFSQbWG6mh2geCjBLB2k+Xdl+Kv9DnYgtdEtwfUR4MhGmncduY0ddOv4cWaUWCjhRph6ySETtEPBi1f4gWpfXd4Qik+GSceRbNsOt/xTAejjkBfG3bYqmYXycvU0j13tUaokO1ulnF2+DleF53/K8nB03Y0im5MDjG4tJKZEtC6XbHyTEGNWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YXIxguaIDgMBQhxvppqbWNbriOR5oVLyqgq47Upy2I8=; b=C8k2gxPizLy3UFq6q6jO8+lZgGvdpkSYXtRhmAEDXy3yfvNSMZo+2O+H8jzL8Fs4U273UmaDi29qnnovouhOkWnp/r9G6RBcPsbnPmAwHsUim+8ikN6fJUB+dLucgb3aNEJhVIyoWSjeL1AMWvrHI22GeElJK/BsQngEB50AHJ7AIIS4AfdBevIA0ivJ6NkPXoj1cmhaQuGVT3mv3NdkhFMFbB1wIyynLbhQz2bmwvmsHbGbNorzR6hwIKRVm9YQ+r7KA4olVW0nj+rdPKix4/2mcmvqfgZx84F+3txKfVsKx+V/OxC6/a3qD6rIfGYMuzxBZ607VH1VnE1fzCCwKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:53 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:53 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:15 -0500 Subject: [PATCH v9 07/19] cxl/region: Add sparse DAX region support Message-ID: <20250413-dcd-type2-upstream-v9-7-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=10968; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=Ue9nxkCBAbAQFWbu0CQ3m+slfJ5d9+PWGNBQeO6mizY=; b=tngxmK7sPNauggRF7uFSLXG+oUAuWS0LuNdDUKS4cOUwk3n8EByXRCtN5zVBrkFzpz7Y7IVMP cWCd+/VxCJMBlj15chVHMOGxulh4Bcy8If4whOwQEiQPDJBOhTmTGId X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: 049bf665-13bd-48e2-f354-08dd7addc8ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?q?UcgCmU4bnQf89yxW+cnXFKmR2iHHC9w?= =?utf-8?q?FrP22d8Jp8d3QH2fLn81h8Bz4x/+fd02jspH34qNX6uY1y69O6LaLC9J0Amw4wAfe?= =?utf-8?q?iwPwHCv6a5pEdAY/nHRXqMi/BW30lvWARZXVizHqHV7DtqdnoktoufMrWQFUdH2rN?= =?utf-8?q?PBBNqWHvTzSolEU1quihXeiV1HN4RHh7+DXfw9gVHkRb+10S9giXIC3cgfrQIm/JI?= =?utf-8?q?Lplm4YnhKT1nX2BWjt8Zv/tSGRAUEPgktgcujrTnnTxVkN88FvumLWfcZGdBqJHLe?= =?utf-8?q?zlFVJ7c5oJgX1pPoAMCtgMkO9g+o0YcAJKCtVxz4EQImtBX+NLU/R4J9B6Da6yEKe?= =?utf-8?q?eCjgbwwutsaEdc+JWu+tK9p59LSCM19QesRbyeDAvr0IJj3+XtY9/Hq7flwZnrBhe?= =?utf-8?q?R5f+ojhmrS5eM2kbroRELoMxH/dlYmtnYz3OT8vninAKTMEUilUVNWQWsTU3HTXBT?= =?utf-8?q?rBuD1hbpZmPowx4ZwQ0ycOMxXBNoi0nuIKvDtX45PIvRQNdmnu40Zu8XFYfHFQjri?= =?utf-8?q?9w/I5kVaf3luxjmDC/sgA4xEb0A9gFUHmNd92PC62j68FjkQhbCK5VfI9YSdhCqaC?= =?utf-8?q?EqwE8WOLdtRvecTfNFt9CbITLpiSTOPNR+msNIm2zQ8UjSKgG1z61dxiR+1zQPjaL?= =?utf-8?q?ZrfsOIUVhZOtHeXUXz32QXij/AINCRMkjAuWK5N/WsIypUnBq59/Qb/jN5juXx7b2?= =?utf-8?q?obsQU2Y5B580obGA8Psa1Z8KLUcNMV6fp4IKboDSOST0DVNZdylsxOoPYf8SkFgfU?= =?utf-8?q?aVRjKaPt3FBfDsPLeE11h7G/Ew9C6KGPgNlVNZHFj2ELuo2w5m/tNIflRv7ATyMS4?= =?utf-8?q?R+9u3qRzCsSVNjUfcyqGwy4/jBagMwXJRN80XucT0VL/D4wByMdwXdq1/67861fM3?= =?utf-8?q?EqOMjWHgBiMre3Tu32PHZUsXMTwYffZBMKI6lzkaZMV8ETlrKlOB5/Xv3CNkf0PG7?= =?utf-8?q?Dmr/DJs/2NXylMtom8JDUFNKi31mD29x9GzXtf4KDhd+XE5OyR8XehgaklHehFSiO?= =?utf-8?q?yqRkTYJmMTTmqHycQXtC0Ph62qd1l2dP2tPutDKw+kZb4YClvTbsVB3J2T+D1pK17?= =?utf-8?q?x9CLnfjs1gKDjYwWELMx1F62o8iPvAPucmQrVnH4JEsHMmPVD/OIifU2lowZs9M62?= =?utf-8?q?OryTXMGQXSYQo7gWztBBIK8Zt+T8nSLEblyBSdQwAWm8w0cSnewzxl5mzaIxpJIGx?= =?utf-8?q?QIcJjavtzd+apAOlwpjZzOTUog/Ovi20HyAT+nsW9ihqRAVUO7xOOdrGyjXh6oy4U?= =?utf-8?q?TgMJ6wfmqxhLZ4qyqUG+t38ERazUWxgdGuP78dqlJzv3TSYx8f1qAGqLtayz57alQ?= =?utf-8?q?zzXQOEUUaNdonbXLW7jrTIjzF3ZMM00Szs7PWSoWuKzQv9vjj2abAUg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?4hpfkoeGUBnwt4pvNtMZ1YLahHYo?= =?utf-8?q?y/f/QxCDQ9NX50KIhAL1s+Onsuc6eDMBWatf8FX7PPtHuIcb0sbitTf88oXiM0soD?= =?utf-8?q?9joRjP9Y8nYJi7PP6UVbBLTw6wFcahrDaj5NpdRbwMi/syBBJIjuckNI3nov1u12E?= =?utf-8?q?rNqWpU1ElQSZ782uZTPayyQSZngxuUZEzxkOnuJ20BNU2ndbviur7JtJHSI+0T4g0?= =?utf-8?q?LrnL1lbuPap0IXHZQxE6pbKyylSZTCmpoyZbG3JVAiNF460cx5Ce9LdZJhp2RkMXQ?= =?utf-8?q?UM/FAemsb9uGT2iKHZK0eYPEZl9BOyS6OoqRQIR8vZkqEEYLHx9sTmv1+r74xpqpX?= =?utf-8?q?2bOmRfdMMvVX8CSaLL4xOg/llgm+4pm6DHEnZAn0/2rAa7PrOLLUYO1cHUnbSRtiD?= =?utf-8?q?EzqMqA16E6waqdVIMZ0TbOTCRI8NX8T4jD52/Cq8wjk9rZXIOfQtt1wAE0C1sZF71?= =?utf-8?q?MskniytiLvGE4kYTXnkbA4Mbioz7wZIk8dBSaIbCzsTf4JGpQhPQQisVjM8Ihsh4b?= =?utf-8?q?ZkjdpSHsL+VwluZGFg3gOaHGfBihBFYYSYVaE6MbKdEKA58e+Pe5mLY+19d1VMSWs?= =?utf-8?q?bgzUWYAZyUCtuuYnGDWvxLfpCFI2rD6BQxl+muXuMzGd5UNVtYTZ5cwvmswRacFDu?= =?utf-8?q?RN0CMlDLKAo9tQMpR4Cr2SMsiIwv4CIlRrdA3wS3XyTUqBMw71+4K6qGdHPKu+QJt?= =?utf-8?q?Y8pKNooCcGf0LcMWH1K82HKIOVmmBlr/4OSPYx7xVp4dj5MuwzN0UtMbZhKMElZ08?= =?utf-8?q?J3bgQWzXa9wzQ14JGB3IbmEoXXxqydVhT/42jyUKF+ViFo2LcriiCzk9tHmA7LdOc?= =?utf-8?q?wGLeDJ8zhu3i01lKNOZkhsTulhJrAGOyPVkGUpOaUlQQ6zkM1dblBobGUJF/XoU1Y?= =?utf-8?q?FoYP41KFxRJyfnoimifr2IALwAOLCqIXJhaLVccI9RP0Lda+fBhmNd1Amg7DUQJ/s?= =?utf-8?q?kQqKfQe8AXCjhuAMlnoiz4fp6tiTR41gauSjehVsmD0mF1t4W84C52YlmvTolfYqF?= =?utf-8?q?GT6DIEeyyOpHvicZzfoDJh0ln08rKK1mglu5564CWIctga8F2zGCjibG8ZwJ+Aw9N?= =?utf-8?q?gpJq1fmK7vnkBhAVt+VirAkpChFqK3orYs2lMq5kVouZ6A+WRwHYzxOkRhriJ2KCC?= =?utf-8?q?pRtU2YrjYloVvEzVqLxdxIUzX08ElhCjgHx1QyVAFlHlKXvqV7i6XOvmWAmbQ/Mar?= =?utf-8?q?9fLi8/EmNf07nr/xsU148ml+uMeEfcw+cHJjS6l8Xn8Mv96Tl8KOBIGoS8jMPvX5S?= =?utf-8?q?J+zjt8Uba8VLiqWbvAZcWncjr6DE5aKxXDxzHc5YKRFavwJVcaKxsk9gJ+EiUfMpU?= =?utf-8?q?d41g69vSSW1ITbP6CI8xON2Ak2Ic7eCmPTuyTWS0MCWuELwydyz292rCHYyvwjrCR?= =?utf-8?q?14DDIl+lY0O8Mocgnk3T8E+GJlUyr1WCbmF7AchjDu1bCyNk4VK9pCFPTxdt5NWii?= =?utf-8?q?EbFj1TqJVIv89GhZy07Yql2XyvCRMuKPz5Psb5WVcn+oWtjOl+AZ/N/MbFn4QBtlX?= =?utf-8?q?7d0kLlW/+OYn?= X-MS-Exchange-CrossTenant-Network-Message-Id: 049bf665-13bd-48e2-f354-08dd7addc8ed X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:53.5872 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AmZ2zILsHNw2NLrMFaEVkCnLh79qg+VHw66fMAv48+kxaIW4CUHsnFfIQvIuqOFduBJPEt6afMYl9zcwaRQG8Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Dynamic Capacity CXL regions must allow memory to be added or removed dynamically. In addition to the quantity of memory available the location of the memory within a DC partition is dynamic based on the extents offered by a device. CXL DAX regions must accommodate the sparseness of this memory in the management of DAX regions and devices. Introduce the concept of a sparse DAX region. Introduce create_dynamic_ram_a_region() sysfs entry to create such regions. Special case dynamic capable regions to create a 0 sized seed DAX device to maintain compatibility which requires a default DAX device to hold a region reference. Indicate 0 byte available capacity until such time that capacity is added. Sparse regions complicate the range mapping of dax devices. There is no known use case for range mapping on sparse regions. Avoid the complication by preventing range mapping of dax devices on sparse regions. Interleaving is deferred for now. Add checks. Based on an original patch by Navneet Singh. Signed-off-by: Ira Weiny Reviewed-by: Jonathan Cameron --- Changes: [iweiny: adjust to new partition mode and new singular dynamic ram partition] --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++++++---------- drivers/cxl/core/core.h | 11 ++++++++++ drivers/cxl/core/port.c | 1 + drivers/cxl/core/region.c | 38 +++++++++++++++++++++++++++++++-- drivers/dax/bus.c | 10 +++++++++ drivers/dax/bus.h | 1 + drivers/dax/cxl.c | 16 ++++++++++++-- 7 files changed, 84 insertions(+), 15 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index b2754e6047ca..2e26d95ac66f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -434,20 +434,20 @@ Description: interleave_granularity). -What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region -Date: May, 2022, January, 2023 -KernelVersion: v6.0 (pmem), v6.3 (ram) +What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram,dynamic_ram_a}_region +Date: May, 2022, January, 2023, May 2025 +KernelVersion: v6.0 (pmem), v6.3 (ram), v6.16 (dynamic_ram_a) Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process - of defining a new persistent, or volatile memory region - (interleave-set) within the decode range bounded by root decoder - 'decoderX.Y'. The value written must match the current value - returned from reading this attribute. An atomic compare exchange - operation is done on write to assign the requested id to a - region and allocate the region-id for the next creation attempt. - EBUSY is returned if the region name written does not match the - current cached value. + of defining a new persistent, volatile, or dynamic RAM memory + region (interleave-set) within the decode range bounded by root + decoder 'decoderX.Y'. The value written must match the current + value returned from reading this attribute. An atomic compare + exchange operation is done on write to assign the requested id + to a region and allocate the region-id for the next creation + attempt. EBUSY is returned if the region name written does not + match the current cached value. What: /sys/bus/cxl/devices/decoderX.Y/delete_region diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 15699299dc11..08facbc2d270 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -5,6 +5,7 @@ #define __CXL_CORE_H__ #include +#include extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; @@ -12,9 +13,19 @@ extern const struct device_type cxl_pmu_type; extern struct attribute_group cxl_base_attribute_group; +static inline struct cxl_memdev_state * +cxled_to_mds(struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + return container_of(cxlds, struct cxl_memdev_state, cxlds); +} + #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; extern struct device_attribute dev_attr_create_ram_region; +extern struct device_attribute dev_attr_create_dynamic_ram_a_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e98605bd39b4..b2bd24437484 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -334,6 +334,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_qos_class.attr, SET_CXL_REGION_ATTR(create_pmem_region) SET_CXL_REGION_ATTR(create_ram_region) + SET_CXL_REGION_ATTR(create_dynamic_ram_a_region) SET_CXL_REGION_ATTR(delete_region) NULL, }; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c3f4dc244df7..716d33140ee8 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -480,6 +480,11 @@ static ssize_t interleave_ways_store(struct device *dev, if (rc) return rc; + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A && val != 1) { + dev_err(dev, "Interleaving and DCD not supported\n"); + return -EINVAL; + } + rc = ways_to_eiw(val, &iw); if (rc) return rc; @@ -2198,6 +2203,7 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, if (sysfs_streq(buf, "\n")) rc = detach_target(cxlr, pos); else { + struct cxl_endpoint_decoder *cxled; struct device *dev; dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf); @@ -2209,8 +2215,13 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, goto out; } - rc = attach_target(cxlr, to_cxl_endpoint_decoder(dev), pos, - TASK_INTERRUPTIBLE); + cxled = to_cxl_endpoint_decoder(dev); + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A && + !cxl_dcd_supported(cxled_to_mds(cxled))) { + dev_dbg(dev, "DCD unsupported\n"); + return -EINVAL; + } + rc = attach_target(cxlr, cxled, pos, TASK_INTERRUPTIBLE); out: put_device(dev); } @@ -2555,6 +2566,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, switch (mode) { case CXL_PARTMODE_RAM: case CXL_PARTMODE_PMEM: + case CXL_PARTMODE_DYNAMIC_RAM_A: break; default: dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); @@ -2607,6 +2619,21 @@ static ssize_t create_ram_region_store(struct device *dev, } DEVICE_ATTR_RW(create_ram_region); +static ssize_t create_dynamic_ram_a_region_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return __create_region_show(to_cxl_root_decoder(dev), buf); +} + +static ssize_t create_dynamic_ram_a_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + return create_region_store(dev, buf, len, CXL_PARTMODE_DYNAMIC_RAM_A); +} +DEVICE_ATTR_RW(create_dynamic_ram_a_region); + static ssize_t region_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -3173,6 +3200,12 @@ static int devm_cxl_add_dax_region(struct cxl_region *cxlr) struct device *dev; int rc; + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A && + cxlr->params.interleave_ways != 1) { + dev_err(&cxlr->dev, "Interleaving DC not supported\n"); + return -EINVAL; + } + cxlr_dax = cxl_dax_region_alloc(cxlr); if (IS_ERR(cxlr_dax)) return PTR_ERR(cxlr_dax); @@ -3539,6 +3572,7 @@ static int cxl_region_probe(struct device *dev) case CXL_PARTMODE_PMEM: return devm_cxl_add_pmem_region(cxlr); case CXL_PARTMODE_RAM: + case CXL_PARTMODE_DYNAMIC_RAM_A: /* * The region can not be manged by CXL if any portion of * it is already online as 'System RAM' diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index fde29e0ad68b..d8cb5195a227 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -178,6 +178,11 @@ static bool is_static(struct dax_region *dax_region) return (dax_region->res.flags & IORESOURCE_DAX_STATIC) != 0; } +static bool is_sparse(struct dax_region *dax_region) +{ + return (dax_region->res.flags & IORESOURCE_DAX_SPARSE_CAP) != 0; +} + bool static_dev_dax(struct dev_dax *dev_dax) { return is_static(dev_dax->region); @@ -301,6 +306,9 @@ static unsigned long long dax_region_avail_size(struct dax_region *dax_region) lockdep_assert_held(&dax_region_rwsem); + if (is_sparse(dax_region)) + return 0; + for_each_dax_region_resource(dax_region, res) size -= resource_size(res); return size; @@ -1373,6 +1381,8 @@ static umode_t dev_dax_visible(struct kobject *kobj, struct attribute *a, int n) return 0; if (a == &dev_attr_mapping.attr && is_static(dax_region)) return 0; + if (a == &dev_attr_mapping.attr && is_sparse(dax_region)) + return 0; if ((a == &dev_attr_align.attr || a == &dev_attr_size.attr) && is_static(dax_region)) return 0444; diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index cbbf64443098..783bfeef42cc 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -13,6 +13,7 @@ struct dax_region; /* dax bus specific ioresource flags */ #define IORESOURCE_DAX_STATIC BIT(0) #define IORESOURCE_DAX_KMEM BIT(1) +#define IORESOURCE_DAX_SPARSE_CAP BIT(2) struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c index 13cd94d32ff7..88b051cea755 100644 --- a/drivers/dax/cxl.c +++ b/drivers/dax/cxl.c @@ -13,19 +13,31 @@ static int cxl_dax_region_probe(struct device *dev) struct cxl_region *cxlr = cxlr_dax->cxlr; struct dax_region *dax_region; struct dev_dax_data data; + resource_size_t dev_size; + unsigned long flags; if (nid == NUMA_NO_NODE) nid = memory_add_physaddr_to_nid(cxlr_dax->hpa_range.start); + flags = IORESOURCE_DAX_KMEM; + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A) + flags |= IORESOURCE_DAX_SPARSE_CAP; + dax_region = alloc_dax_region(dev, cxlr->id, &cxlr_dax->hpa_range, nid, - PMD_SIZE, IORESOURCE_DAX_KMEM); + PMD_SIZE, flags); if (!dax_region) return -ENOMEM; + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A) + /* Add empty seed dax device */ + dev_size = 0; + else + dev_size = range_len(&cxlr_dax->hpa_range); + data = (struct dev_dax_data) { .dax_region = dax_region, .id = -1, - .size = range_len(&cxlr_dax->hpa_range), + .size = dev_size, .memmap_on_memory = true, }; From patchwork Sun Apr 13 22:52:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049464 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DB681F30BB for ; Sun, 13 Apr 2025 22:52:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; cv=fail; b=C+SroxEXNGxxYpWE3I8HQb2L3drGzRX6LoOR3dEikgrTWmXhCRjjmIGSvFzpZuhIpTQ4YBu9FXEiwAPfAqRwgozC16ScpiXgvLnds8UC5jcr/OX2C+3urXeYecu7Smcij8EK/UVeezZfXzTjW96HppaqDgqEOnlTp9S6EzL0ip8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; c=relaxed/simple; bh=iluJfpsU++oIIgnjHnQQGAMSzSadvi007rxv3Yc4G6U=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=rUaxvstP91WRM0KZTLvEdR3073SCfOqukswdURYdSZA3uLXgLsg2MicdJYv+pNGipNgxz7891chHszWUbdgxiKRG1WeDpbRMmjA6FZaaQ8j4w4X7k4GfvBbSd1BzOd1t7f7YjD35HVekfIVTVI7MmDzLa/6KgsxlEMREko1MafQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=b+QnFiB5; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="b+QnFiB5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584725; x=1776120725; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=iluJfpsU++oIIgnjHnQQGAMSzSadvi007rxv3Yc4G6U=; b=b+QnFiB5ckAKcjfXsnEoEdJSiVCT8qm+/qIB3X63XTtYNsXUPbWQn6Jo xOC1otsWZDdFsngKF2APhQzDcDhBXqYMZuJwBft1Lc9Cjq0Xdj9L7fwcu ABx26vl0Z8OBH7i+0/f9UZZyBpOlyIoaoq9odsZxVvAIOS92krslJziLr C2+frVD3i9SrGoWS4rqoyqQInBTxIZqbn4eZj1WLwNReZIZPm2ieAsVx4 OlVqsvds3rhfgu/c/S2No2v9OO8FIqU1rVIRAsGe3ItCd5A4Cd/aqOUhi JDst0h3MW327hPxMJjSF2Ml79UlSd7yVgOlHOODrH01qPJTBCFI3YcBOJ w==; X-CSE-ConnectionGUID: zmqNjOUaTmSmsrDAHmaUCw== X-CSE-MsgGUID: AvaSmOYtS6CqTFVf/Izx5A== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280914" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280914" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:03 -0700 X-CSE-ConnectionGUID: RSqqwHBiTEOjVt3h32dqdQ== X-CSE-MsgGUID: pAmz1rqtRuuZy+xcQ3PAPQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657471" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:02 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:02 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:02 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=I1D3WTP2TykNmNbIS4YsN27uUoWPjtQZit3EQUUooVODRuFNej7gGwykRtRWzuUd5+BNMTLmj/5ocTEzJmiqafFqQ/zVMlHM3LkuXuC8JGp7HN6Nxi6eaWs3sqtws9tWL3FtmdM7Cyqpmo98kGpTHK13PJeo3BkIxwHRCO8ORQfix+AY4veUPOKBFqjw6wYKwGuvvUr1pO4XvoERMTm6Fxf3yZBpT8Tg61FrHu4tHQAb+tNjzLS6AJ64sz2W15p2utZVW8rmQAukOVNstNm20BYAS+m3vdM/sQKv8pLzsvpZ43LoSSzhUaXc99bJiok0Pudf4AbVUxLiiT6G8U3GwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YouiFuSf4pyrHsNUvpkk19Of+rZKju0AJBO992m58ZM=; b=BQq2vJglEjs/1GVHsH51zrYYPrtsFjbX9sHMaZFO3zGacbcnTaTi8ksCPtXbBSYVvCZYPWJRFMjSsg1Es9qB+jzrXeqcRExxYyxA4sOnmC2ZAGpoGAzWAaLlrLqJXSW2hphnYjrV5Y+ZWj1l8q3mYKEKQnla+kKY2WzLUYY7rH88u1FBa7uFRFu5QcvUM64wbSaF2ZtKdHH+kblfi2zsfr5DF8Zz90BSQLKGbxtnN9c4v4NcRa81u7hatRiayMRwKqrBi70XpcvU5MopX9IESjOuzTG0HuYq5LTIMQtcmUBo9R3V8/3SJUe8AoPVs7bOOd1qDEWW/TGfpLfQRvYY/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:55 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:55 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:16 -0500 Subject: [PATCH v9 08/19] cxl/events: Split event msgnum configuration from irq setup Message-ID: <20250413-dcd-type2-upstream-v9-8-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , , Li Ming X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=2631; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=iluJfpsU++oIIgnjHnQQGAMSzSadvi007rxv3Yc4G6U=; b=4ksVC3g/3tEirb+WH7N3h465FU+jX1FXL+aIj5QnuVEdZ+lAUir9vifMLKND8+PDnUSr+/tPI Y21mG9OA/XBDj3poiE+6mVzsupZPTUEcljr3cmX+yrxnZUKLQaP7t79 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: a88790ab-a0db-4891-b932-08dd7addc9e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?OnbEFM3LLd8ze3a8o1MSGYcA9erBl+L?= =?utf-8?q?tTtinWBH/yowOSZoYorPjPKuHgcqB2x03zH5pi3sav8HBnTfoRDuz+jmlqNoT3Skq?= =?utf-8?q?Ldz51N7pyloC3nHGCGnRhmRtQVWq4SXZTFq9EFF+Kp/POEvX9Qub8UwMJOiEV88WK?= =?utf-8?q?8i4ZqE9P+MZgML6e7ngie7d1nKxwXVN79ZesNzABZcxU4QmrLV3UG6x2+pj0XhQMn?= =?utf-8?q?Q9drpTzuNc3iUURtFT0PmbC5nHb+WFvFunmSxe0tOeL7jGlUkrGVAdO2KHzgC+NY3?= =?utf-8?q?C2+wbaAAxFpRPFdHwYmpP1hWXkOCmpxVJy0CnP+dmKBzMXVgjIcwnxNodNg+6ne+H?= =?utf-8?q?csjxB227KckjVXQJKed9DbHtTKhPrmjySb4C9ffZfevg1Y7ebmaWzPawP2xpcP5oK?= =?utf-8?q?vPXDgQYcT0N8DJatiXkbkt8R0w1PZYVNIYyVVv/TEQ6sYqAMB7645inLK8LpD4Ijc?= =?utf-8?q?S+LOJ7q4lZEzGsJA+pJhpESd9lcz3hGIexqo1dAOP2iYIUMD1agpsJPJ6kYXRZofM?= =?utf-8?q?50VnFQiNoC3JNgGsA+AJYR14gviXa+zJbQ6sDR6g1JKr4hrYLsJtglK92alwehdbG?= =?utf-8?q?VhuGlasfl/TzqXf3vF6vXZiENVUinwSqr9Trw/9egDi5R2rePeivp2vtheX9HY0IP?= =?utf-8?q?t5DlNm4yYnpVsc0fhU9ID2+61VfnLXqZTZoLWQN38Yb8b8T4D4hddyPNGc+twWkfK?= =?utf-8?q?gEF8RoGPuNdkjPCQ02O2GqH6wou1k5vved09zXYENGiM69fFgnHkaSvZPdv44xLQw?= =?utf-8?q?OgRsPNj8H2CEf90UAZTBdimipljoAnhSA/ihlRZNN7H3iTLE4jVIYodbfFglCOcFs?= =?utf-8?q?yAeujr/MebdBsqQ0/mbh4Q1/11bpMvjuEPAL3ChQfkUg3/P88JrssZnmKOQ2SBkU3?= =?utf-8?q?OYLVKkUPS+yqqj6h+R5rwvLPlnlP1N6VwXreNFicOAmsyUnQPUj37xEVCe8RRrO18?= =?utf-8?q?JSrF6cUd9QSQYInGT7yD1h5SZv40LRLsAEMlUlpDBjvu9z59OAMVhKjSicZGFXW88?= =?utf-8?q?hjT+qAz41DYyh8tKu0ms/KDluEnxjSM74Fxh/AmYXph/14Qr+zMxX7C+iCJYeZaSd?= =?utf-8?q?cAfmnymiGCJyNo99gr1+vgnnnyjUPBTZgk9pCWe6z+dd2uVRdSHyVzWkEE8he4Nsf?= =?utf-8?q?LhaO9xRshaVdliUZONZP//tsJmTB9nyF/6y4Gv/HJAW9M05yqBZGCn1ZzTvBXn/Ff?= =?utf-8?q?40SUqrRwCCkYpfc6viYPUa2EPst95tJCCrBC7uo6xyUfGEOBJqtDeST6/Uk4mRRqj?= =?utf-8?q?8yvtREQ/7EUF2y/RPjqs+3hhu+0NNk5AHANx9TohDVKRKZXeQCoj332tqjNxiIGUQ?= =?utf-8?q?z0sXEcA5r6zy1EQrTTULvZA3gADFVkq9pPSRgusQD+OVo6I5jafosvQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?z55sNs5ky6DoTpKQFVmMjaub5X6V?= =?utf-8?q?NTPJNPTogTjk/0I9oZrZQUTgGdphqcUI9hNAlZCBkEGJ1JGuPoeOYtowrJ4NO87OG?= =?utf-8?q?UJVTHcQc2cKwZ5jfBY/MxwvvTXJLPag7YPlX2cv95tE9bj8WNkxM0qmnlTnchvhKb?= =?utf-8?q?B2A92I4J8EtfXljow1WOp1Pg3gYz5Bu+wash/twwNltuAotJai/l18CW40iVk0Gbq?= =?utf-8?q?6OTwuMZZW+vlh9JZv52P8qq9bXyC3/D03AuC+eo8zBZu8yXYa1i5eCf2+Jfh8LchH?= =?utf-8?q?QdfL2skR0zaE1jC2CqYBHYMwHvq9+vXqvCsA2MlGNQ+f/DFpelgGjiCR/DZUiM5tM?= =?utf-8?q?eflC9ges+YAREnnFb2GuDDSaQmu/8N/CHILzin0yy8JZrYRQnNVwAh/SO23gJurTt?= =?utf-8?q?/xkNblyytkCR3e6O+yJPMwSL/qzNIy55byB7QlUOnEAqIgE0Bg7sWjxvCtEUTEbzD?= =?utf-8?q?Jd8SLM/zywaPBl+Bj4/voTp1NSgjUiuzAYs4hu4Dd/Aa5F91Nr3VUfr3vBZUS8LLd?= =?utf-8?q?Ym0awaSbJVh+FhBpNhORm8kmoKWDcRovnoVgdD4gROllcWr3C3WcpIVMgdBqN5AeI?= =?utf-8?q?j2JayiD7molaw8D2smDkI8IACrB4TUdAqz0Nvg5IcANMV39JYcOKDHe0+bP8ZGSYH?= =?utf-8?q?F1d/yXGqDPI1SXWUXXpTVVawRn3KoJm3oV4nN7KwN6eCthVLAp+7/IUNFjONsuQDR?= =?utf-8?q?xuDdWAd2mKACupXtaFcgcSI8ZnBI4gVG2hX6W1q/W/tH0pa7FPeviSteDaZMeIArk?= =?utf-8?q?08iVcF1CrCzhHkYqKxoThCUs8QUl82hRqYotmZhJGyR2nTXd0416V6tUCS696YbbV?= =?utf-8?q?RGnWFKHPjzzoWoDuVSy/dAcw9EYmyIG1wegrI7WnooMvXi5W3Jbp8Hi9+bDDU5l/7?= =?utf-8?q?T3dngNRUUIaqNokPrrJ0AB7qPosKMVSYkHiy/BvVCK1TG07NE0N0j2JZnzP+N2QGZ?= =?utf-8?q?1Q04mwMJ8asEByJ0i23DtHFMhwVmFRfFDTA0tExh4eXFvuUycxly+m1oEFH4U2Syc?= =?utf-8?q?Csy9Zvvx8oMT5fZXBmfNv5Pb/xDAGz5fZ6vRtRJlHfEHcb/MVawlK6Sp13QLMRyYM?= =?utf-8?q?r9VteewznX1tshKfe0oj0XQk67EhhNI0OSpd0s8bNh6ZnFxrGSVxw9Uc7Ay6St5Jc?= =?utf-8?q?Fg/Bf9nXlns6qR6luz493cLYJU2NsiAGNsRb4XrjTUeXVnlNTt2fojng9gvAPqXbA?= =?utf-8?q?aimhqjm2SGPS2IHHxbZdlcjQBStbfhpzhUzWeu23y2lthE9Sh9EX1PmA3JBIgf6NX?= =?utf-8?q?pFB/UeCnDpaGCQGDMiU6oeE7sR2HE1mHTE1j0pfUZx4LWa5Aao+Ad2PyprjkZhzKg?= =?utf-8?q?HPHbSVq72ZNVyZQLoWop6oo+NJ7+dAKH6Wt9x5LbbuqlvUGWu3UYRNA3BGapjeRnS?= =?utf-8?q?zjV2VJlX7Ss0eLNxNKpwVuBnx5XsQPGpaaN0znwl1vx9pBbPafnx53nP8YRyNW5dM?= =?utf-8?q?4Xtt4DScLzfORcrZBoUhlXKVEtUjD+LouGRioazljgD1G4xYkfqn2bOUBl+3FOY6t?= =?utf-8?q?NMU7a9+EB5ay?= X-MS-Exchange-CrossTenant-Network-Message-Id: a88790ab-a0db-4891-b932-08dd7addc9e5 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:55.1929 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Lij58BCKBgJ8UPZLXHCFCp080M+PmQMYbvZ3PoNFy/UznMfZBTMeQBFuGJjHFNV0ges1iqCkV9t98/4Krg8vbA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Dynamic Capacity Devices (DCD) require event interrupts to process memory addition or removal. BIOS may have control over non-DCD event processing. DCD interrupt configuration needs to be separate from memory event interrupt configuration. Split cxl_event_config_msgnums() from irq setup in preparation for separate DCD interrupts configuration. Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Reviewed-by: Dave Jiang Reviewed-by: Li Ming Signed-off-by: Ira Weiny --- drivers/cxl/pci.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index bc40cf6e2fe9..308b05bbb82d 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -715,35 +715,31 @@ static int cxl_event_config_msgnums(struct cxl_memdev_state *mds, return cxl_event_get_int_policy(mds, policy); } -static int cxl_event_irqsetup(struct cxl_memdev_state *mds) +static int cxl_event_irqsetup(struct cxl_memdev_state *mds, + struct cxl_event_interrupt_policy *policy) { struct cxl_dev_state *cxlds = &mds->cxlds; - struct cxl_event_interrupt_policy policy; int rc; - rc = cxl_event_config_msgnums(mds, &policy); - if (rc) - return rc; - - rc = cxl_event_req_irq(cxlds, policy.info_settings); + rc = cxl_event_req_irq(cxlds, policy->info_settings); if (rc) { dev_err(cxlds->dev, "Failed to get interrupt for event Info log\n"); return rc; } - rc = cxl_event_req_irq(cxlds, policy.warn_settings); + rc = cxl_event_req_irq(cxlds, policy->warn_settings); if (rc) { dev_err(cxlds->dev, "Failed to get interrupt for event Warn log\n"); return rc; } - rc = cxl_event_req_irq(cxlds, policy.failure_settings); + rc = cxl_event_req_irq(cxlds, policy->failure_settings); if (rc) { dev_err(cxlds->dev, "Failed to get interrupt for event Failure log\n"); return rc; } - rc = cxl_event_req_irq(cxlds, policy.fatal_settings); + rc = cxl_event_req_irq(cxlds, policy->fatal_settings); if (rc) { dev_err(cxlds->dev, "Failed to get interrupt for event Fatal log\n"); return rc; @@ -790,11 +786,15 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, return -EBUSY; } + rc = cxl_event_config_msgnums(mds, &policy); + if (rc) + return rc; + rc = cxl_mem_alloc_event_buf(mds); if (rc) return rc; - rc = cxl_event_irqsetup(mds); + rc = cxl_event_irqsetup(mds, &policy); if (rc) return rc; From patchwork Sun Apr 13 22:52:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049465 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C11B81F3D31 for ; Sun, 13 Apr 2025 22:52:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; cv=fail; b=RiaFHLldZqkbSZX+LWWPOoqAgl3wEaEG/S4FS+UiTNfe+c28VVxI0zPT8MVVtMTqy516CCPuiwKOGpCdwtdhrXWYiICvbR2BG1IgIo/txSU8Z7V+wMgErd7g2Kkr4fsOuTVSI7z8MIgObOtjarITNlOVFtRua4rwYTK8+k7z970= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584727; c=relaxed/simple; bh=I4hV71hOvjOspfYk0aPxzUTtPYw5QAMg7ynPu+boSlw=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=X9FTxZzhBrpk6E2rNEsXBzbaNDE+398Qo74c4HE+AXVcvrDVtSgEsFpgL9pi/GwiKH17s5yMuCzcJqBaUyskMDJcmCaWHjYiMxwSwcy3U/K3jCtChX6wSERYC8QIyuxj1luwhadsijuS6CXRnAhjveOqmzTgtkIG6MyC1rcMCAE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nRCriQeS; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nRCriQeS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584726; x=1776120726; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=I4hV71hOvjOspfYk0aPxzUTtPYw5QAMg7ynPu+boSlw=; b=nRCriQeSJmcpv2I7wXiX98rQaDu/aAmzsRzkKCliTesTL1J5Qc9GZmnC fiOfYi6xU+k0QdfWXZwkC6bNC8CjXIuM/y8cNWaajki9RlaOODrrBRU9E b+Lsfr3XN3xdZanwmhJsA2lJ0KKSe1M3xcUialyZxZgB1EK13pVb3pkqs hMjeSiJe2CY2jkSmPm06ZAJNU11+uvvV5A9DjcgZULZoT36kJ99tH7swZ 0V6RHcreO77tHitWIMCMr4yegYK5SertCzxrZob7eDouo4voc68Zyq7L1 3ozPVOBneU8Hp3GnvTPWKUKgTU+NAJzBOhVj8MN+LbH+POPO84TGeTkRc Q==; X-CSE-ConnectionGUID: mvaNDcF4RhWi3iAX66OXeQ== X-CSE-MsgGUID: 93EU6PtbRWme9YCQJuOfXw== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280920" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280920" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:04 -0700 X-CSE-ConnectionGUID: r3hxRt6/T2unMfPH1JfjjQ== X-CSE-MsgGUID: LsSwU6+oRpSPyz9S6AFDMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657482" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:03 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:02 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:02 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pM/b3wRSEVPTKVKnkPc79uHWwAPyUQGsho0i0yVHBk/LmUFf1I6tv8NaD9nfGGPYroq2weck/V9pac95oEo2VPV1G57TuzAP7LjBYUkBgq1Dqau2+gzau0R8aIJAX3NrpEeL0Qp4DRa+niQJo6VbkpTS9zNdO60N1ItyKTPSwJ/a2jSNvT22wUQTNi+qgSUp5ggC4ozJJssbApKbLDhgH54FmnLW7BphirdaEgh+f4Hj1JP1gcXJ3fLGREL0oQZVwnGonO4NoAws+ornw0qajgi/ECZnm0xfeF5YdxnQ+hEMPdDr9YB0+wE7mH4YRdtrz1d1zMmvRsRCupelvcFxhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a/GNGXtpj/hr3QtK7ujbvV1FA1d76rRqvbSPhLn3p38=; b=WX9VI3Y8Y8v79eWLnsDSEH8DKgQuiuYbMiPYTQqQGtAJ/8l5uw+95G6NkiV/tYgYzBG10j8+729O4jDzx5hnh7doz62TArDT6uvHaF6ibnNYQe/jnadV9PijW2US3EWDu1ZWgE7Oc3WaZmvtGMgSMDWr38EAeHy4Mow5HjAQK2uoZVqHSGW8tQFP3mna3GSp22m8eJ9tmziG/cqpHE5vwLGLZTOxff3NxDHVjQmzyesJ4xRUI2CSXwyO2K61XuTvkX12DrHvDKGFU8gTUACjYTCNQAZqeHT/qRhR5OUbZjZoq8qYOfCme7woDBQHOSv9ubvjPLxOF6r/htfSuuHwjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:57 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:57 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:17 -0500 Subject: [PATCH v9 09/19] cxl/pci: Factor out interrupt policy check Message-ID: <20250413-dcd-type2-upstream-v9-9-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , , Li Ming X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=2242; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=I4hV71hOvjOspfYk0aPxzUTtPYw5QAMg7ynPu+boSlw=; b=OSXoDp9OrvabwhRjCQQxUU7QdFJTUEigulwgAulz8q2Z4G3jHv58yquDPvV6PQ3Zc97iAdS7h SoBRvDF1Sq/CU1fNHJd+273EBhAsp+88JJtvwQauTdtOmKA1Jzo3vgN X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: d8b96218-9bd6-4755-3933-08dd7addcae4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?0pfJq0hKIniEscmfeirvP8i0JzXt5i/?= =?utf-8?q?FfFpSYlThsIf99Fn1QGHX0W2RGoEpDug9uSaPsBKVST6L03olP4N39AHVcvVcBFM6?= =?utf-8?q?o3PaNZpFz6BqH5QEZIE4QJBKMDsTUxlKndAAYbivZUDY2bLDV/hT9y40OUFfYjvQV?= =?utf-8?q?OSqm+h15DYUdeFq6kNQzIN7VBaR7+EzFUOTb61/caO1hzT/MImK+CM+M3vvO6/Qvc?= =?utf-8?q?ox2u0i9xk21u6Q1UlCWthUQHvME6LpLc/GgOTssPAJz8/Fc9UYkpTc+pL10B/ViB6?= =?utf-8?q?2FpaPbsPqWsWOsZuVAvVaTC2jFg2ltMq9ahpe1wvO4Acx58Xm+hfGO18FOFElkg6+?= =?utf-8?q?59w9Wm+Ia9uY4CEoj8J8SBbbg38c2oCKIG16+cf/zR8xzr52rMHxH8VDvlZM3elZw?= =?utf-8?q?4oRKQYlCNpRWAPqJ4s67GRG5oauZW0CwVu+MQK51nvXd2/Qn1oPpgPgMNQvMXdmDX?= =?utf-8?q?hr8n5oOX/CNDA3cfdhgBCjYJosiw8xwavZam3qjEmoRcAOUVYbcPgcXSW3XBNOVi8?= =?utf-8?q?ufZVeS/KG8LDc4GUz8QMDh9WeWa4gu51vUUl5t6S+TWby7cQn3gII0DKFzPIEv05R?= =?utf-8?q?ZGEjh1Mqw7lHAceiHyjKU890ucP6L+N8tnX6/rt0+K6yMeiXMrm1y6u+Qfxlc/uLO?= =?utf-8?q?eEvHkoajpqZDqBJhcez1iS23b06hnCE/MjZRTRRqj4kFeZoNl2Z9ojQnXw+jH7yHr?= =?utf-8?q?cPp5WWWRR78gpZv7j7PqWSjvBTgfMovnHDhGknEv/TParzJaNtyZIp83jaO5Cf96/?= =?utf-8?q?ZIbm1c7cK7kFA8Sgjkp//O239uIGo6stBqmWsNgWHxvILZ/ZIZ+VHo6PGFcFt3+M+?= =?utf-8?q?bfwqXxk7Pqfifb6XQwG3PMWX+JNH/6cnVA4LQSCAPoKPWiPNT8OZdm9oYrYgbxzKf?= =?utf-8?q?JN855G9i9S5a/CSV/WZKIwsLlxiav4uW5bVpBNIZY7C5KOzppn8u4FKLDOqWhnqEd?= =?utf-8?q?cE+iFOKdAP9/xKNm2XD9EXjCFFEUprFdfZ5IwZAwjQBNeLmaDdz0VQJrHki8j8Zhn?= =?utf-8?q?TJShLy887HI+EebVZF7rXK3T2dh5CT5qbeMCG9Y7fMISSh5YUpvUsOahV7/OhTAW1?= =?utf-8?q?IP6BgNJqM5jI4M/zGIJcCQXjl1A6Pu0xcWWh/kXt80wLwUKRkcjLBKpxaHuS8fNbE?= =?utf-8?q?6XPXkzJiCL/JiEybobTIgF5YFfoZLoGtjZshT7ChfKYkrjvMVo+5V6f6C/ce6+b6f?= =?utf-8?q?dTgiSSbEH/YMFCYwYSndXwPLzCp6PL2d3AA0MdWAYODezC6xQ1bZeXEdvDDswh/TX?= =?utf-8?q?fIuxcH0pWyP8TTKzb2L3x8aO4uu/uMyd7VaXFfZ9Q8CqsgQ1ZyjMrs4INN6Jkq2iH?= =?utf-8?q?LFL1p7xS1MYPdUL8lpYFiJYSyLZzcRvPpVr8VPA+xg3q5uHPMEUlK38=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?FOINHu59HZP3uagi5lntjel5LyCG?= =?utf-8?q?vFm8WUqz4+Xg3P6t0827fHGZsGJUxbrkqCa1wQXEaz1ekAtWicuV0n1aMHKCtp96P?= =?utf-8?q?GMK5aB/z5qFIHPAdz8a8vo2f1fk+7Q9Z6h3TzYiyJcevzeV/679QMowj5isjemNth?= =?utf-8?q?90kytjbbM6W/UHD3EvxxjVOubSe1ryuntm1EoiIJWbhE8kHMcTedJAmTS86pg/Z0I?= =?utf-8?q?lo9dJva+0GTWeG80mnMJu4ppiEntw2VK7ZFrMpP92SboVSwQoOpBaOqon7Gz+ugYe?= =?utf-8?q?qInkbD/kbAIV+0t3qIRd6Fc/Paek663YOH+EC3qJZzT16H3RplircUGp98GLXMrlA?= =?utf-8?q?EGNQ1PIDrM7vHS3ovBbcZoiw3pWvSZto+4YUohJKoBLOmgtwjdDYbQn4Emr6EVjo1?= =?utf-8?q?r/yXWZO/zYcp9xDwdkhXQIpwG60mexWEUlL6mLWcsJkFlanVImNxgF5h9NFVrbFUc?= =?utf-8?q?bPTobsCjMyV0nir71Wzjl5Q+RCxnzrlxrANhlG7aXmHrda0Vptxi3oFl3KZVwF0FH?= =?utf-8?q?76KDSCNPd/fhIwMgpPrxflgpY94YGxY5uAeUmtmg7N/w9qB/ayPJfyiIItGU6jUvW?= =?utf-8?q?ANxFiGIeMnF5KWIesvkj80flDGCJUWnCoUxvXXeZ3KRg4i0ECAejYDkVfCDX/ZPT6?= =?utf-8?q?zATX98HsB9sw3n5lmxJLoCyNFrl0W5HR1+2MQ2yTUwOzAO3zuLICeeZyK3MEnKwZr?= =?utf-8?q?gq2NBMZRwxvrJtG+xMNDpPQFE7T1qblaNfysoPrt/pFxGKbUuxP73oQDAH31KHH73?= =?utf-8?q?mxtgi1DEhW8uuHaKeMMDSb5E2ua42SXz5rAgKbYyqnE2jRmLBHESCZ89yVWQJ9Ieb?= =?utf-8?q?hvFbU/f7VTrfkV5s7OX7ZNaIIZLkp+iXfD0tIQeA192Vjju6T1fjGqPi/cqDkTMHI?= =?utf-8?q?6Viu4/z6iI2hCbEdxq52+TVitNxPmLnYGdmBv5Tv/be/bDwcoS+boWpR5iDnrLVqu?= =?utf-8?q?nN4LlBJRLYznJ3QKuScIncTkCbOZe9YEekP4nQHqW2mz7PShS+wTaiKTbu/avZ/CT?= =?utf-8?q?jcgAUo6cVv4XiM1j+6JufJIjATusNqreOUcboOkACacvfM/6HOe5GUB1tGduI5DgL?= =?utf-8?q?Wb788yHPFh8NC2bo5BNPQ1C05aWLu6lAHBwSJEVFz7UUAzqge3hk0Vjdc0tWLI9//?= =?utf-8?q?EzMCE9m9a7bnDPn5HQHBxZZOhvPxF7q4UtXJCUMIOSnPUsA5TmAGDq4IEewUJLPJa?= =?utf-8?q?bTISqpjx/2K5oJBty+c1AIyDE6gVHaS6GNHDbMY3qrqDiP/LpUOHQ2vC98qaxK6Gk?= =?utf-8?q?/uHKGv6neuUIjtcUi76hpXpf6LPdmDa06iQVgcjQiPsBLzWv7nq3oQKmhPwasYxLv?= =?utf-8?q?rYdBNJhIYB1f4DV3rwNOKKlThTCyS7hLDs1pKrLBZlU8IhwClbfoqV5qyWMTP4z6v?= =?utf-8?q?J7T92/EC0xefcSvc7m6dUPU9Uf0eFbmSYhpyd71W4dQJHq2iB/fCb5E3qeBOnFRUp?= =?utf-8?q?8yLhTSFyolp+3YVzwGcYb+6GqS+eCwynBiTTCpdhttGjdV3xw2gmCZKmXwjg/iq48?= =?utf-8?q?r4n6tyFhSXW5?= X-MS-Exchange-CrossTenant-Network-Message-Id: d8b96218-9bd6-4755-3933-08dd7addcae4 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:56.9538 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TA69BM0KOA1HMaZ9Sn/WWD7aKXQ5F30ADnuICwpiAXgoqrU4tZSLAFRYCbQwNLULiYGeTcQ/eVEXzdDIi+7GfQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Dynamic Capacity Devices (DCD) require event interrupts to process memory addition or removal. BIOS may have control over non-DCD event processing. DCD interrupt configuration needs to be separate from memory event interrupt configuration. Factor out event interrupt setting validation. Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Reviewed-by: Li Ming Link: https://lore.kernel.org/all/663922b475e50_d54d72945b@dwillia2-xfh.jf.intel.com.notmuch/ [1] Suggested-by: Dan Williams Signed-off-by: Ira Weiny --- drivers/cxl/pci.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 308b05bbb82d..36d031d66dec 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -755,6 +755,21 @@ static bool cxl_event_int_is_fw(u8 setting) return mode == CXL_INT_FW; } +static bool cxl_event_validate_mem_policy(struct cxl_memdev_state *mds, + struct cxl_event_interrupt_policy *policy) +{ + if (cxl_event_int_is_fw(policy->info_settings) || + cxl_event_int_is_fw(policy->warn_settings) || + cxl_event_int_is_fw(policy->failure_settings) || + cxl_event_int_is_fw(policy->fatal_settings)) { + dev_err(mds->cxlds.dev, + "FW still in control of Event Logs despite _OSC settings\n"); + return false; + } + + return true; +} + static int cxl_event_config(struct pci_host_bridge *host_bridge, struct cxl_memdev_state *mds, bool irq_avail) { @@ -777,14 +792,8 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, if (rc) return rc; - if (cxl_event_int_is_fw(policy.info_settings) || - cxl_event_int_is_fw(policy.warn_settings) || - cxl_event_int_is_fw(policy.failure_settings) || - cxl_event_int_is_fw(policy.fatal_settings)) { - dev_err(mds->cxlds.dev, - "FW still in control of Event Logs despite _OSC settings\n"); + if (!cxl_event_validate_mem_policy(mds, &policy)) return -EBUSY; - } rc = cxl_event_config_msgnums(mds, &policy); if (rc) From patchwork Sun Apr 13 22:52:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049466 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C4B21F4281 for ; Sun, 13 Apr 2025 22:52:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584729; cv=fail; b=fUIzG/tqjN1OatryX538bI1919NmEbr3Icpi6emFXWE6l5n0atA7mSwFG4xYRSgRCEcb0FAv3gq5ON130NEnS1FEhPsGIfDxKfmPmsbLQ7+Uv19IItyuJod8qQRAQTptVTajZNDlx14KgjGclb+8ltZbrnmHKIKDY1ixLDGHNjU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584729; c=relaxed/simple; bh=DweLy/iFYuJN2tVNOKyxcg0lNjkjC4dzRFmq2hss15o=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=QArz6fcMcLDDlR2Cb4XfTEf/CzP/3Qu5kQm7GeADwoWeLlTRKNurTHg4hyprp/jZYzbR6c2Ju8z70ylLMnH1u8D+nkzKVLGsxDbzu9eIU9CrsThIThcuS6/aF9eAEtd07jIBi+2WjquYw9mB7VjR11BpK2vymARbh/8Z08i7WT4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aFDLrK6p; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aFDLrK6p" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584727; x=1776120727; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=DweLy/iFYuJN2tVNOKyxcg0lNjkjC4dzRFmq2hss15o=; b=aFDLrK6pLK9II205XHRUswU9+5Vpo8qeZdjTzqmFs6ot2AB8UWQZWSQC lbJmY5vB1CO+VM/eGgb/Ew6Rj0YGUmVnXrvYg/iKiYpp44gXACaKrIobl eEQGTLdFNj+seIb3swsgmWCOWhdxg6tueODF38ui4+M3x7Fzjka6ti8it v84ErjYXlXULDD4pDxDRX7F0+4fP4RdfnXJChkt2QFhyjGuKFXAb/58xq U+/UCpcHgWEqtRsGKdrZ1y3es79ApKB1AoEj2ovhSk/cPownBWeGqAj9Z bhRL7NhnX0LJ33hUGqzLEAHT8GlZZJipzJ6n22bgO5SrVWEm7JYkNxkBU Q==; X-CSE-ConnectionGUID: BmPQZXxjRAmoOrNfFkfMAw== X-CSE-MsgGUID: ScFYeDswQi6k5NHOyORECw== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280927" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280927" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:05 -0700 X-CSE-ConnectionGUID: sMRIQHZZTXKTL3F5Xbl3rg== X-CSE-MsgGUID: cSUTNWydTj28tZzYQJ0V+A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657488" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:04 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:03 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:03 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:03 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vl+/f8aHDGJ8kSTMrOJRG9ZdTPZgSNxgZZF/PLW7M8vMFxfQQNqhu6SCd7TGbw8KCO6X/2ajTbu5fNobyvUpJtk5vdfornqFAE0nWCrEkM/T+8wz3ybLntqV4d4qLaaKODuml0OgH92QEg3wCiZwdOBIyWba9aBK5dVjmooAt1U6bLmtX5QBCwjxyAbQkQrpWKeIuzVik2U8oBXZW5p1fdNKPL/shwH8uGWvdIBlbX6LG/g4d/8rkjuLVyJqSpICDWpO0HignBYH6maplyCT1caMfI1CWSAxiBHysLJNdwotqsmRP+fjBoNjxPnuXkjQBIQepZZGXKmt3B6HwZTVfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YdhHm7MPN9whPt26xmtzjGZbBz7yKJXcWBNKSmyB8kQ=; b=s9vMVIzuTYVeZkk/2dr77MO6EmMcltVEFuO4EViqcxj04kLJzRN7q+L3jACOIlZekMyk4TTBifCu52o0u43guvi8FQpEw47GfSOemV2z4TYEs2kftN6rkpuEWsRadrG2rCPeq/qIZ+RRZ9uHnXyVbkYsTHfBRh5X1DDNQX6xfNR6jgG5uE7fh9hAvXhhf6gRSojGWOcberqo54RSS4gdCjfG/KJ4OGMorPAKO/Vga5ULMKh4G+21MtZDvJukfqX1w5M1oV+0H1gU99fN3uKPFJ0fLgfmzOYiA0llJweKAsCjtc15x6SyvlYu9Dw3a0W6rhV4WrUNkWZwKHNO5rG4IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:51:59 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:51:59 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:18 -0500 Subject: [PATCH v9 10/19] cxl/mem: Configure dynamic capacity interrupts Message-ID: <20250413-dcd-type2-upstream-v9-10-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , , Li Ming X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=5555; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=DweLy/iFYuJN2tVNOKyxcg0lNjkjC4dzRFmq2hss15o=; b=n+ag5TvRrreHjgrTSvROpgnVQw5WZXYglhSyUnemZTrMXYOBep823DwswAWmBMAaGW2gN1TlB jAUlP8JegpWCY/kVn8N+3y0jvmKaEyO9KUoPBBoLvApcLf4oE0zcwGh X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: 3436bfc4-62a0-4485-5e4d-08dd7addcc1d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?embjKDHGhmoE6xC21ITnGg5gBHejPpz?= =?utf-8?q?vNbvmIOwEarj+LkVTf89yMBshnKfHR6IHrr/7wcPBqlIW+jNcd0MIqBFJvDXpq266?= =?utf-8?q?npgc1i2/sMcVCA5rm/EcPCnzAQCF0eZaiP2V1SOZugYf8Z2Oz907yNVcOZl2GDofR?= =?utf-8?q?O1aiAkLcFX3ge1A16y+fybjxbOayZvBMonrAqq98uNzDt4Y1A13KNpnsVbmQNc9zC?= =?utf-8?q?kdSNNhHpF+6WCk2tZypZtSQ6nZdGKipBE48wOB5DbL9H6WT7XLROeKJrR4pNfIZ6U?= =?utf-8?q?NWFZmIUiYCL0TsojKUJMJUxAvnaVLxGxINtnsvh6KqUlsVYgPlzqJZ6GVZYdKLBmj?= =?utf-8?q?fDB8cFfucfA4ID2LoHgUdlrCFt1gvwITegM3Nt1ziLysFc6/6kellfZ3ngDbyKfiL?= =?utf-8?q?Ga4QRH4mr6wgFW/GRlnVo0KWVxpIwH39zHPbt6LfWoLV1oNFESr3h/EfNzsFYH4JJ?= =?utf-8?q?h5yNS35WgFaqYzEJQbSEaNAv+6m+2k5wvA8ifwI6B7ewMak7jT39x7MzuYFXj4AB8?= =?utf-8?q?KVm3JmzvtUdWXAafQJwT2Z4llt7btzmLbrIKNFeBF4TkyrdF3ABwcvhGzokQmc0n+?= =?utf-8?q?015aAvirVE6a3PBFmjpvTYF9EDJwTQigyuWXeTQh9nlE2esSojHxMUowxp7XPUkcX?= =?utf-8?q?qpPtITjxcbIGAiWJ+8W9GfReptu3WKbFnj/6Q575ypbqCmthVH8lmO7/jR8nryrVt?= =?utf-8?q?vCWdzImht/c34HdIw8+CuwRvhlwiFKtZZhiFn2wzna1rWSz5rfUtT7b5gbdI4gfpK?= =?utf-8?q?kgzuXLItSQKDCimMdrPR6hni8hG0+cqsLA24R4IoPyqA+KxoA6HCtOlX1exE308Iz?= =?utf-8?q?C2KM0Hl02N9IoOYYfUpwl2InVODEugDvESldcy4YqsoCYsPpx6xALVAnCx/arxx7x?= =?utf-8?q?tdjRK7BYdax0z1feNwv1IiiS6LmMblW3GXFrSZ4WF3e6LQkJj9qmbpK5oMJphXlN8?= =?utf-8?q?+shxWYI0m/zjPi8yKTS0SozDY6+B5ntUlAkNd8LzS61dgrR9kZ0NyiuevclFpTLXd?= =?utf-8?q?MrkhdSnKyH317dgbiZNObifvLmcZpHVNUJTIvJZJhQC0VMFzyVnyyaCXNjQTbB77D?= =?utf-8?q?yz21Tc7+mc2feGA0EjWDloD3dtvVVTWVNWztsq6ZfNugcolk28FMnan64B65PxgGX?= =?utf-8?q?9vljHr7W/m7/iu1FZJncS7xeISMCK1FzbygEV/kSTUB5KevXgzwngsDEQoGM+Bz/g?= =?utf-8?q?Q97xg5608TZXVg3Dxpx8CaWafqBWH6X2LKs/OVfjZcurfbTQkKgiB6r3gO9QbCNtp?= =?utf-8?q?H5iBDaMOqmk71EUob8I6r1GMnAeP6kmTBQfw0mbR2RkYrMSmOhavDJghMEJPUrUh/?= =?utf-8?q?v6kr2gqMk4Z5LonpdtqUZn7Eti/bbHZqgk7wFRlysFhRvbg40KKoqPc=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?+Vu+ekcB7O5dW/3pzIimjw51jsnb?= =?utf-8?q?D0x4tMUfdTaJLhLHUtPdTmZy0p4XFEdUGP8aPYQJEhcRdc+7Dx9xEsf2kR7SmXTe5?= =?utf-8?q?zQtEhXV3lSLlf2bQ8Tcr9o/CTyuSuyBKJfvKY7Y20T5bhBW5oQrzoURLLp0CfuXsL?= =?utf-8?q?YyOgRo6xzkdFkx+4uWZTgUbywXLwMW9O3denlOHSfpoU0L5yalMMg6lnGVQuLXV+l?= =?utf-8?q?3Va8BX1y7Cff+hhV3l+3GbaFM/Pwj6NcEsG9IdtQVpglEMjVcAt/j8FivHkvNUynE?= =?utf-8?q?2cuj8R9eRszL7io7Ki0yNxp2hM33j9bFcBiYQsQBEitrmTITYb4NPb89J87oY3JUw?= =?utf-8?q?Sx2q0NPzvlvPqSzCIlPXnNZD5DbZRdMgSY1Ft2swKUdSJ3z4683QzxH+mHWrH4bWF?= =?utf-8?q?aZdm91raPyt8zoHXDWpr5PPjuVlYIkcRnADoSAgYI0yOxyeZxRfj+QQIEtvfPQu6b?= =?utf-8?q?fzmvNRsVzBa1U94dLivSo7VNYH3Sgxvq5SCYF2joX3gpMTXyMQcLoyihONW4MKTVz?= =?utf-8?q?9WRohaJSZGlT0U0/k6WZpasWMwCGfgzzJpH18N6XG+W+2vTuWO7/aErrGEjKmQl//?= =?utf-8?q?7g7fXWoqV0v/OL9pYPsG7TCfsXKCIYQr3TE5XW3aOLCrt6DbiyFODq7ObCy4K2v4m?= =?utf-8?q?B13vUovlzPUUhISp6e2/AhETiyEVWP1t4Cx4TW0QpuZeUolbOdlAG8t1haEHqiymF?= =?utf-8?q?Vg4qfF+AvHO4yJIMvmi2RpV6vPF+UR1NrMV8BqXItca6652yZ3COt8YRVFEW/gpVN?= =?utf-8?q?i6RsLOpb+9j4XnMXPgyV8LO5kXxROW55p9A0sT2LN39fqt1xM+iWxqpjNVjLR85Qu?= =?utf-8?q?Ne6Ofy/Fs/v+hFjVW0X+F1vvh7Mjfbv5++dATfB/qbQh0tlVPk3nlUd0vjOOA6sZa?= =?utf-8?q?JMd+uUMSJj/tigKeLCOch+1mx/NGMBHcYoe8gTrgZpLQ3VCWRw4OCbF/RJxPs9aZq?= =?utf-8?q?IRDzfaA9/sCKvSYvzvigp0OhE8Tqji7y25nkznqDMD7JuuX4rKwL+7gXlzD78DtVl?= =?utf-8?q?2lLkwTAWJjbDksug9wAneVPTJcFWZol57qfcSyC6BP++N5PTsrTzdTEY3UeS6tmtB?= =?utf-8?q?jPE3Q1jWPxzt6HEAhl4PCyO8QiawuNqtUSIb8cPv7VAitpbM0P20jstOE4XCu3Sh7?= =?utf-8?q?FtByw7h1iS1ZJOpWk5ElhkKeSdvkJt8LOfWUqbAUZVy4GiFkQ/hSepOaUuFBnLL67?= =?utf-8?q?T+OZkavWbp0MycSxeknah9sw9MYARHh/YP3igmToTZ/ESaFQvSDERC2Dltrdy9s0f?= =?utf-8?q?eoJz4ArWfMiLLi0j0JeyXOZ02QqibGHmLSztXJyXH41ojsiAyxzRL1iG10uckqLlg?= =?utf-8?q?K0+Zf5J7UVG4GrN4CIxQPtCuwN7K58Q8BM/TJ+2w1w1f8Taf+CFkX6S8ix7O1ZV9W?= =?utf-8?q?r32FA3UG8Ei21Y6V5yOvVX6BDfCFF3K2PyCRze4knq8WB/KwhGjNA3IjiFM/AGVyA?= =?utf-8?q?N/A2AyINXpKm4Kln8K5tkQ49irraMQkL5U7EFiDNLvzu7mT6nH7UI79MmYnT569/F?= =?utf-8?q?C3DfaCnTh9Lq?= X-MS-Exchange-CrossTenant-Network-Message-Id: 3436bfc4-62a0-4485-5e4d-08dd7addcc1d X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:51:58.9221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2s0j6NgTAbayS5kR4XNUkK6taXnc5s7WyXlq16B+cltOy1hIiEb0dk1ht9UuZRfuXpFuWa8DkdwNymXdou06wA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com Dynamic Capacity Devices (DCD) support extent change notifications through the event log mechanism. The interrupt mailbox commands were extended in CXL 3.1 to support these notifications. Firmware can't configure DCD events to be FW controlled but can retain control of memory events. Configure DCD event log interrupts on devices supporting dynamic capacity. Disable DCD if interrupts are not supported. Care is taken to preserve the interrupt policy set by the FW if FW first has been selected by the BIOS. Based on an original patch by Navneet Singh. Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Li Ming Reviewed-by: Fan Ni Signed-off-by: Ira Weiny --- drivers/cxl/cxlmem.h | 2 ++ drivers/cxl/pci.c | 73 ++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 62 insertions(+), 13 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index a74ac2d70d8d..34a606c5ead0 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -204,7 +204,9 @@ struct cxl_event_interrupt_policy { u8 warn_settings; u8 failure_settings; u8 fatal_settings; + u8 dcd_settings; } __packed; +#define CXL_EVENT_INT_POLICY_BASE_SIZE 4 /* info, warn, failure, fatal */ /** * struct cxl_event_state - Event log driver state diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 36d031d66dec..c8a315bbf012 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -685,23 +685,34 @@ static int cxl_event_get_int_policy(struct cxl_memdev_state *mds, } static int cxl_event_config_msgnums(struct cxl_memdev_state *mds, - struct cxl_event_interrupt_policy *policy) + struct cxl_event_interrupt_policy *policy, + bool native_cxl) { struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + size_t size_in = CXL_EVENT_INT_POLICY_BASE_SIZE; struct cxl_mbox_cmd mbox_cmd; int rc; - *policy = (struct cxl_event_interrupt_policy) { - .info_settings = CXL_INT_MSI_MSIX, - .warn_settings = CXL_INT_MSI_MSIX, - .failure_settings = CXL_INT_MSI_MSIX, - .fatal_settings = CXL_INT_MSI_MSIX, - }; + /* memory event policy is left if FW has control */ + if (native_cxl) { + *policy = (struct cxl_event_interrupt_policy) { + .info_settings = CXL_INT_MSI_MSIX, + .warn_settings = CXL_INT_MSI_MSIX, + .failure_settings = CXL_INT_MSI_MSIX, + .fatal_settings = CXL_INT_MSI_MSIX, + .dcd_settings = 0, + }; + } + + if (cxl_dcd_supported(mds)) { + policy->dcd_settings = CXL_INT_MSI_MSIX; + size_in += sizeof(policy->dcd_settings); + } mbox_cmd = (struct cxl_mbox_cmd) { .opcode = CXL_MBOX_OP_SET_EVT_INT_POLICY, .payload_in = policy, - .size_in = sizeof(*policy), + .size_in = size_in, }; rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); @@ -748,6 +759,30 @@ static int cxl_event_irqsetup(struct cxl_memdev_state *mds, return 0; } +static int cxl_irqsetup(struct cxl_memdev_state *mds, + struct cxl_event_interrupt_policy *policy, + bool native_cxl) +{ + struct cxl_dev_state *cxlds = &mds->cxlds; + int rc; + + if (native_cxl) { + rc = cxl_event_irqsetup(mds, policy); + if (rc) + return rc; + } + + if (cxl_dcd_supported(mds)) { + rc = cxl_event_req_irq(cxlds, policy->dcd_settings); + if (rc) { + dev_err(cxlds->dev, "Failed to get interrupt for DCD event log\n"); + cxl_disable_dcd(mds); + } + } + + return 0; +} + static bool cxl_event_int_is_fw(u8 setting) { u8 mode = FIELD_GET(CXLDEV_EVENT_INT_MODE_MASK, setting); @@ -773,18 +808,26 @@ static bool cxl_event_validate_mem_policy(struct cxl_memdev_state *mds, static int cxl_event_config(struct pci_host_bridge *host_bridge, struct cxl_memdev_state *mds, bool irq_avail) { - struct cxl_event_interrupt_policy policy; + struct cxl_event_interrupt_policy policy = { 0 }; + bool native_cxl = host_bridge->native_cxl_error; int rc; /* * When BIOS maintains CXL error reporting control, it will process * event records. Only one agent can do so. + * + * If BIOS has control of events and DCD is not supported skip event + * configuration. */ - if (!host_bridge->native_cxl_error) + if (!native_cxl && !cxl_dcd_supported(mds)) return 0; if (!irq_avail) { dev_info(mds->cxlds.dev, "No interrupt support, disable event processing.\n"); + if (cxl_dcd_supported(mds)) { + dev_info(mds->cxlds.dev, "DCD requires interrupts, disable DCD\n"); + cxl_disable_dcd(mds); + } return 0; } @@ -792,10 +835,10 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, if (rc) return rc; - if (!cxl_event_validate_mem_policy(mds, &policy)) + if (native_cxl && !cxl_event_validate_mem_policy(mds, &policy)) return -EBUSY; - rc = cxl_event_config_msgnums(mds, &policy); + rc = cxl_event_config_msgnums(mds, &policy, native_cxl); if (rc) return rc; @@ -803,12 +846,16 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, if (rc) return rc; - rc = cxl_event_irqsetup(mds, &policy); + rc = cxl_irqsetup(mds, &policy, native_cxl); if (rc) return rc; cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); + dev_dbg(mds->cxlds.dev, "Event config : %s DCD %s\n", + native_cxl ? "OS" : "BIOS", + cxl_dcd_supported(mds) ? "supported" : "not supported"); + return 0; } From patchwork Sun Apr 13 22:52:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049473 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 048A41FECBA for ; Sun, 13 Apr 2025 22:52:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584747; cv=fail; b=WHKFreBLdzoSsaNh5+u8O377q554SFjvT3IZqKGBo//FbHQBafaz2ZHGb7qmoSyUdHy73PEB4aGTN7in4VQPsfPTUBBuUwCJvR+Xq23KvwwkqLNGBEtc3FbmL5gXoVyQNWIg9C9hUqtR2FMlWTt/Uvgi0BJJIv0K52UjgcFKghA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584747; c=relaxed/simple; bh=wkudF2XNTcQS8VPUDMQ571w4CvWff5yrKjJtNZtBW2s=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=U3szl7pzePmDN7/w4+C7EPUPiYCSprx0nL5B8mzga6Kdfi50DkrtZnQuahKCYikuIKpWyrbyokjjXO0cOrFqVQpqyTZRjkL8z32mSxq8cTVVOw8Rhj9fxj1am/Q4+6xgxlpH7KP2WvVyjk8bdMWZzNhLgW0H63yxEkB2TuRRMlg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DUBXH5ca; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DUBXH5ca" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584745; x=1776120745; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=wkudF2XNTcQS8VPUDMQ571w4CvWff5yrKjJtNZtBW2s=; b=DUBXH5cai9lmW2iG6HcWGmdk6toSJwz4DBH+QHvYDNe7kUe0uhMwjfmw Ae4pSN2CUMm1358NW1N7LQ1LyKkuXWKlr6Y0ARGZntpQk1JJzTSrhGk8L oUvRk/3Oapqdd9ijtnWQhB/2uxycLxSiur6zySJbxsxOMEPQG1CgpZ3X+ /9Z2ZGHb/9jIc0sJRQz8vxepphK3eoy11+npJWWpJaUT3VjHt6JEhAFi5 oze5+7Qv3WC9HQglaRhQrZTebbrsg9eKxf9fQZaus2n03G34bldum52X4 ZdeDYijAXQHkQejCPEegtV8gGmefUU39edKE9EIO51lb/jr43AazE/sfD g==; X-CSE-ConnectionGUID: 8bsd7F2gTyCeUGD6amR0yQ== X-CSE-MsgGUID: 1ZA9Oq36TbKcoXKJ5HWqdA== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280947" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280947" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:22 -0700 X-CSE-ConnectionGUID: 63yr2e9gTqqcH3hTpgEfDw== X-CSE-MsgGUID: /cihgiucQ8W8OupWJpRtgg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657627" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:22 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:21 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:21 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.48) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WaEqpOAEbVU9uT/MCZX+XZ0VJKOIoVg5TugwxWSDMkuZpAimSXRmDbtGw64BWAxMWzpbMX5sLkNht73POhbxOUw8uliHhRfzUeZPAoJuPxyRl+rZmQK2eMgAgUaSGAJ+r0NCQBZ/wgmUgkJNhmTTaPiVOWdvqWXuSk7AktlLSvHpl1GHG06Fv29m8/TrZDVJbzE/RFxNNUgvh6oc222Ar0PlQbYZqWfT7fxmO/Bhaksez6jE4wcsn4H+6ih8DdgoNhfp7AMFE821Q0Uq/EZngptpCU5oqbSQTGczjLijeRExVJ6rstvkeNrHcQzrRKg2Jua0WjupC+5FkIjXgKavIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xHChTg9LEV/qJAb1pJTzHBnMQyur4fyxjR7vjkIpdhk=; b=IBJelhyhXSOjQcUVOngq8tQEm2FT8+jx1RsGZAglEXIcQYTBc453ETWaG3yQGzmTZcvsrdNZ9Za52ih1eDfLgMr7/ARXhgxcLT7Nrl+JZyKASoPIKdzBvLYDypUvrOHt40kw+CrbFEiZlHnewMXU/9D5s8SmjNuM3+VJA+OpTSon+3cEsACInz3YokPd/J8f8BuHCiMHrUy+nFNWZMq3hILKluwOrcgWv3d8WEt1EarU9ffNrD6C70nmPINZ6etzfoLoDMQoWhI94JgH28Y16ukKCulEvOBt08omRrxAEN3Xk9conjWcDa5drIe/DWQ5bVGoWAFoYpJElyzfkjCRcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by DM4PR11MB6042.namprd11.prod.outlook.com (2603:10b6:8:61::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.28; Sun, 13 Apr 2025 22:52:00 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:00 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:19 -0500 Subject: [PATCH v9 11/19] cxl/core: Return endpoint decoder information from region search Message-ID: <20250413-dcd-type2-upstream-v9-11-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , , Li Ming X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=4478; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=wkudF2XNTcQS8VPUDMQ571w4CvWff5yrKjJtNZtBW2s=; b=qhzG4PaAzvVurrvwp6PNMi09MOnwGo2di6304YuKuEHO3itK6l3RaQvXu6NjdjeEvBwwQzhxv 4ZGYHmnYlMJAkvmhyDnvAe+Kx2gnwgxxb71xVol0j6jm5WWesbpJ9TU X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|DM4PR11MB6042:EE_ X-MS-Office365-Filtering-Correlation-Id: d89e3e88-eafe-4c1d-c85f-08dd7addcd11 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?Q7gYb76sM/D2VKlxlVEi6D8OAjb39fQ?= =?utf-8?q?siWTDRMAspl4nZws6ab2P17t0csaC30QFL6hjbV71FI9DAitPqCLn+VJ6Ipw+XihT?= =?utf-8?q?BmBSNCDLYOgNH6jraDuGzypQdprhBD0PA5+RzNjPISEqq+hUZRZK/k6Le6YQU9NK1?= =?utf-8?q?lhhFqdLhBE3T1ucd3629FCDlFz562FMhhliRvbY5duoy1Ju93CftqqG64IURbYIcY?= =?utf-8?q?IQNnoASLTWTo4iTmRUdqQQHt5CDOTAKehOBzu0ok8X4bS8KkOgOoAxJClpvwcN1tm?= =?utf-8?q?jzjaHp2jzztnzm1Yio8xDcDkn6eaKxNulnOpiEmeosszsMZT2DrW45GQ20dQOXpEk?= =?utf-8?q?EMdNfxeBFbkBGwTR4e/waT+cZIToNscr5/3pQ7CXAuW6YEjAafx0u6AIgavT9d+E2?= =?utf-8?q?FeBkfg3wOD+Ils+U92EXQ8YEkm7wrWgrKtqc7iX1k81dv4gp8OSb87k52z16RiiWr?= =?utf-8?q?yXIupgvhVgiRdwoYY+u1kWLe1zsmQGymQgH82AaTfgJRyPWTPYm88XpDouXIzDmfC?= =?utf-8?q?vki7ohBmNDLqhN4ehAIgnVjiPpizM9ldnxxXT5mRiFQFmOKr/UG1OtjhxDPn6rDEA?= =?utf-8?q?CYDZeW6LtaBWNodjxvqub/zTibFr49u34WUHPPGOrZRkXLAv23BDWJJ9ZhPisAQNd?= =?utf-8?q?xlacraB1Z8Zu31Tn0Wtc/EmYu1LK5ip/XXolbQyzwv2HXr8rf8fWQLjSj4aoX2gv+?= =?utf-8?q?GpEjP6THmF4usa/dc8Cq36b504bcQlPWl+UcL7HdxJdgG0ZfHCl82EcsqnrjAcYV1?= =?utf-8?q?G9hKfHIGFylKtVoxU98vS5kCgONsIqFvpsOBfPZLxdE0ValNFRwVEgF+fHHDtZl+Q?= =?utf-8?q?/BNszRV+SCg29D3z+egLlOd395wuMV5ia6fs5k5BUJV5ngdk23MOcVpAEYhK3f7EQ?= =?utf-8?q?bs5pmPokRj3zd7iqe7qZ+Q7uYv6lZcLcIInqYFdaO5LhhweBgI7PQrtFwcQ7882Pm?= =?utf-8?q?FaUruK7B5xutNi9y5gqebzEpvdvBj9GrFjHVYMby/KEZvWAdD0+vw/6Fw2/ggt1DJ?= =?utf-8?q?kbL5Oa0svG/oEzCMMxCkx1sLOpTlGLPvHmNdBAjXI9cNCrByjnQw5tgrKoUee6wAs?= =?utf-8?q?PRoyl0yhA8xV2LfdwnGEtEXAchdmr2U2WFeedqPWYrL08TefwsY1YbiCe1+vSFtUZ?= =?utf-8?q?8JHbApDSsrnUuRmzPK2vf+CDxVeWWCSwX7nPWY5fUF2L9qLuAwjj8iVQWJhIGinFg?= =?utf-8?q?fK1usqqZu5day3f+ZrdTj1IMAswK3m4bflP6b5EcI+0XMCTU/vIgvGma6Cocqiqh5?= =?utf-8?q?fEatDleqTiljpV0r3XxJ02GO9Mv+PPA5rxXBa8pukIoNSy+YUg989j50EiMgKfEFo?= =?utf-8?q?nba6wWM6RWTEfBcs98eA4G7dweMRPBXQjefpyJd6Dzm/h8X33D9SQbU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?PTF34IQ7MEgAfeONWA5ezFkxNwXW?= =?utf-8?q?t224MkRzN0JfqN+QC8ZvlAst/T30aMSXgMJ3gmjstayu0Mt02+XjkV4ozFh4a+jvP?= =?utf-8?q?fZyKFUMyF8ZYDI+dD3NEfBNWbgz2mv95mX+y512wlpb+K24RONYBtiKrqmGQlZYwT?= =?utf-8?q?Cj6wmNptw3hBdm+jDFrMdhyYzqE+D74vPEUqphB4YX3rb3WJ9oYw30yHMHIySTl5l?= =?utf-8?q?2pZNgpyScy7IPm146GN0KANPU6wfw2g9xE3H/t+r4sbqH6vispi9I/mWAjhfjKoXf?= =?utf-8?q?1tAwGl7ueiFQdxArG+wxyno2AT+RYpNbQxrVwtIfK1KoPV0fLXGi2zyFg8iX5nDM6?= =?utf-8?q?8qU9y80R/mlWIx1bYUC+TsRVadfm845UM8juFVGTyFyyXwkUarlyZE4PmmKST7uMf?= =?utf-8?q?N+SyqRFZFQG9AfvsNcZffCc85SoGHrs6nz60p7FbR3bZpOV5gteRLMyQOMbxfeq9D?= =?utf-8?q?Jneq5uKUVERUA1mL0giGDc44gHVqg5XlGRCdAIZy2JyZK+2VN0bizEIKTDs1v0lLR?= =?utf-8?q?mdTOkM0HiXRRqRS30j3dGei4TbkuYNREqqffrzSdcrrN1HprZYbjKt3zkassSD4E4?= =?utf-8?q?p+vVyK2uAS0RggwOOZzw/AswKemvcEb+0mB7KA1vNOezGoAO9Pt/FiG+O9y6ERIe0?= =?utf-8?q?yaFKu2os/dbuDvNAJmZVlj3puk9/D+0CXCcuTsBBbQZCRiYRAHqHhV5P7iTPHM52N?= =?utf-8?q?6DiH3BqIpV0BZytjh3OTS3BWPB8On4EyMZlp8hVRFPvOtl+eFiyA+7uFtMgNLepZd?= =?utf-8?q?tWeZxy337eeMMO/ac6TsyEfWuxqSs4cfbxpww3W2OdXBw0nD5HdDSNVY3/F44v2re?= =?utf-8?q?KxgEIDu1opGWqM0MKM3WXxu4snTKApcucnkGOyHV/OOeoMi7cGKodWUAc87XkSsSt?= =?utf-8?q?DZ6iytOM3kVgZR/k1+XBUnnbZMxaM9rLeLTtEqBMcZNmwSWW5+LvQ+kXfa2iP6tce?= =?utf-8?q?n9gBd2B+H/eJ2sEf85Y2YMnfnBbF8gg43qPjDcfT5jkkmBusQdlqkhKScW5iqOxXV?= =?utf-8?q?wKG9vEQETT5jx/MfTDJYGflRzZXmVxzrhPi93TBuNpFVht4Zi23bBY3KfGaofhZ1s?= =?utf-8?q?5wCKpH8BXnOBBRcTyxg8UcupF0F3pPGnujY1KUtY5SvUrQy/PxK5eQd5Y/QFgwNBI?= =?utf-8?q?TVTbGp5L1ryMLWmMbmDERK+KCbpWlDel2/ZIOaQSU1+iIL85hiul4d9hIKdFv7sEf?= =?utf-8?q?IgRvj7vTcJewI9xTJW9jbF0euihmNNeEJV7M2Fx3bmxiyKxL8lVx/I3jniLeu6Mph?= =?utf-8?q?lTmnOoTl54BEW4RyyrCBL83hvRYqdbuJntDeZADOKpqXQAM0fuA/tGxXHj8GoLoQV?= =?utf-8?q?0VpbIHi8FoU/k9i0Pz+xuJuU2LPdkuF8iOtwt3T5/9jQJ+usYXH2jeqw0x13mqTDf?= =?utf-8?q?gegoKm+ZeDzVIIO/WLpZkp385BzwVJ9VWduhsVc6CenRz/WYaetsqPktPt4OiH9Un?= =?utf-8?q?Zmhn/RIfpOIgaoqRA/uxPpkhJ1JJFX4xY3K3lbs3Fb+15LQuwMbF6grNKALy5IZCH?= =?utf-8?q?9a89Go4scNgy?= X-MS-Exchange-CrossTenant-Network-Message-Id: d89e3e88-eafe-4c1d-c85f-08dd7addcd11 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:00.6300 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0jQckqDImRY5pMabHSSwuD9R8CB/bEhYKrCQFvsUvCtaCyJgncQRbC6uB2Hn3BhmW+2YhukkjTIrR6p2YBKgOQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6042 X-OriginatorOrg: intel.com cxl_dpa_to_region() finds the region from a tuple. The search involves finding the device endpoint decoder as well. Dynamic capacity extent processing uses the endpoint decoder HPA information to calculate the HPA offset. In addition, well behaved extents should be contained within an endpoint decoder. Return the endpoint decoder found to be used in subsequent DCD code. Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Reviewed-by: Dave Jiang Reviewed-by: Li Ming Reviewed-by: Alison Schofield Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase] --- drivers/cxl/core/core.h | 6 ++++-- drivers/cxl/core/mbox.c | 2 +- drivers/cxl/core/memdev.c | 4 ++-- drivers/cxl/core/region.c | 8 +++++++- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 08facbc2d270..76e23ec03fb4 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -40,7 +40,8 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled); int cxl_region_init(void); void cxl_region_exit(void); int cxl_get_poison_by_endpoint(struct cxl_port *port); -struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa); +struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa, + struct cxl_endpoint_decoder **cxled); u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd, u64 dpa); @@ -51,7 +52,8 @@ static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, return ULLONG_MAX; } static inline -struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa) +struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa, + struct cxl_endpoint_decoder **cxled) { return NULL; } diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index c589d8a330bb..b3dd119d166a 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -957,7 +957,7 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd, guard(rwsem_read)(&cxl_dpa_rwsem); dpa = le64_to_cpu(evt->media_hdr.phys_addr) & CXL_DPA_MASK; - cxlr = cxl_dpa_to_region(cxlmd, dpa); + cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL); if (cxlr) { u64 cache_size = cxlr->params.cache_size; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 063a14c1973a..d3555d1f13c6 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -320,7 +320,7 @@ int cxl_inject_poison(struct cxl_memdev *cxlmd, u64 dpa) if (rc) goto out; - cxlr = cxl_dpa_to_region(cxlmd, dpa); + cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL); if (cxlr) dev_warn_once(cxl_mbox->host, "poison inject dpa:%#llx region: %s\n", dpa, @@ -384,7 +384,7 @@ int cxl_clear_poison(struct cxl_memdev *cxlmd, u64 dpa) if (rc) goto out; - cxlr = cxl_dpa_to_region(cxlmd, dpa); + cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL); if (cxlr) dev_warn_once(cxl_mbox->host, "poison clear dpa:%#llx region: %s\n", dpa, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 716d33140ee8..9c573e8d6ed7 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2839,6 +2839,7 @@ int cxl_get_poison_by_endpoint(struct cxl_port *port) struct cxl_dpa_to_region_context { struct cxl_region *cxlr; u64 dpa; + struct cxl_endpoint_decoder *cxled; }; static int __cxl_dpa_to_region(struct device *dev, void *arg) @@ -2872,11 +2873,13 @@ static int __cxl_dpa_to_region(struct device *dev, void *arg) dev_name(dev)); ctx->cxlr = cxlr; + ctx->cxled = cxled; return 1; } -struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa) +struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa, + struct cxl_endpoint_decoder **cxled) { struct cxl_dpa_to_region_context ctx; struct cxl_port *port; @@ -2888,6 +2891,9 @@ struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa) if (port && is_cxl_endpoint(port) && cxl_num_decoders_committed(port)) device_for_each_child(&port->dev, &ctx, __cxl_dpa_to_region); + if (cxled) + *cxled = ctx.cxled; + return ctx.cxlr; } From patchwork Sun Apr 13 22:52:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049468 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C65561FECAA for ; Sun, 13 Apr 2025 22:52:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584743; cv=fail; b=mQF3LXxLjxDkMgWXvLGKC9KpG3tqUpLNp28UZcftuncEJbJT4rZ/XM1aKzIx97DPHyugTuQoPtD6sDBRA+zLR7IjHau9c5A6NgK0KXvVHbYgqB4zenbm7+Fjd0NhkTuQdydvsGBJGe60xrf5++eFK9qUyhx1yGW8nmdmMfxs95A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584743; c=relaxed/simple; bh=dCYaC5ZWrBF0vEquEYwJKxfuBnfC5MhdfQ924/0X7fs=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=cnkIOAVv0AeZXxDRLqTQSVrdtC7qMaq1kBERgjt6BXnMnToE4ZWSDl5WfaV/SeU1sQpEwz7FcV0SvT8OKopKNNixBDRtQoqBwdyZp4fccm6Xyr4O2ZKHyvnLZxRumi58zYVAkSrpCFTr5VrW7TnMLDvdA7NwgpWgkSDBc/4Do/4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=a8vdKp4z; arc=fail smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="a8vdKp4z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584739; x=1776120739; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=dCYaC5ZWrBF0vEquEYwJKxfuBnfC5MhdfQ924/0X7fs=; b=a8vdKp4zevryp/St1FC/dKOtWi1dGCe4gcqH+zJBjzTIc5PK/eOYlfGK 51YH9w6rWHR+kmpBf21apxqCXfoic3Ww8RB7grKOna0YZ+Zl/Oh/g2Veh w9a2N9xJ1oObFj+xXaX08MOQ9y2Vd4+FRvaR06SvlO9tPopq40M6SSAHf RkaKBMwz2iiWcX6GXYSQ7CC0UrSCfLxNJjKV2taWMSqJDM4VfYmU/qDme HxMxGMU+BVlci2fH7DO+Q6HwWVligl7EQT3gGE/nbwQOGU7mZRbd24+Pv oTDHP00v6mF8hnYagj40ViiYNlZuJ+CwRgFYQu/j7eNrpvdt2+N3agWB0 g==; X-CSE-ConnectionGUID: BuNRbhaNQI65kzfIuRxV5w== X-CSE-MsgGUID: L9ZvZL2mTGCUghFfXAqLqw== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="71431120" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="71431120" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:19 -0700 X-CSE-ConnectionGUID: iGdH7j6pTMm1kr09iDUwog== X-CSE-MsgGUID: 3RMnPZcBRQ2whXn3tcqVjw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="134405565" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa005.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:19 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:18 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:18 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:18 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HZqrdEpxJjKtvuA0wKO25q9MA90BHPDMXLsTvgc/fiW1vyL5LtCQ6NCdj4KkSK6WTk+hqCAtQJ8SpQmAAc3EWfplbWVmAEYcyOMlJgikX7Wf0z4dFD0MhAEiYsedGECuR0dkx8unqH3EMXboZh7HLra7MoG99uAVIte1L00h/nerB5iSwpHAYa3MJq+yMuKcEQWj0RjQcnW/BV0DaPNgl4X0WeDCJ1RAGsosciVMpFH3mdMIQSUHK3oGThmXs/S2syNB+NxbB2MYI/6/wMWXvdhQb8mrdkWiyIAfD1kDZgUj95wg0h6jKGEUBRFYqt6XwZNAtn/mLhKtBc6EVn8yiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ba2T0ZfLf8iYDHabMe8oJgPLPkgGNb9IiUzF9GEIk4M=; b=ZYTK2+kYFBYkruJqi29Hres6GAFpBYR1cXT0X+EAErLJcgBV4E6S90s5b7ZAeSnp3R4DaHM80H7h1J0VtSKwoCRF5YoSAGMnj3MF+tiE1v8tgUuJ3nxhgIL14zOOL3yhgpdpqtRMJwy+b7xRIuoFDY7dw8mHFzB7neILsyrR2MKGfrH77d93e5qh4TTHBO6g2Mkp3RQRnYfserHYjXGcg8zS4TmZ1v2VeIhhirdd5ySbI4wZb8WtmGHLoJAc6LZ1eN5vIkxZTM8YouSWcXcgEYCPL40urzFOfHLHwgTE68yV/IlHRdOyreGPCuJqG7pYnUoXgSATWThjmNQcwhaTgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:02 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:02 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:20 -0500 Subject: [PATCH v9 12/19] cxl/extent: Process dynamic partition events and realize region extents Message-ID: <20250413-dcd-type2-upstream-v9-12-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , , Li Ming X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=35115; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=dCYaC5ZWrBF0vEquEYwJKxfuBnfC5MhdfQ924/0X7fs=; b=ewzwB3JiprOktVaqQTE2Z82vmmw0rqQfCZ2eIU1WE+uGQ4BmrippwoS+Ztw5us/NtfaSOrUnm 7h3Kp8O/BsfAUG95z/9xzTnWAhARQbGQ3AaYZ+P0V7iuuvepLgyvxgZ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: 78094907-f15b-41e5-147b-08dd7addce29 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?D36w4yyBx5iiLTEDZ4NrmykgGVTXbf8?= =?utf-8?q?4nOu64q+ZG7Ywd4WkO24aWfnrQ9J2yLWBcDh+39KH4+0a0qG9dFtQh41qXdF/BJ5h?= =?utf-8?q?kxqnvjtvTEfimpJReyxUwlmrXsfHI1A9eDiwZmkoPPSq8cGGhtku3N5sfxp0O1e8O?= =?utf-8?q?HmHswKnoo/kqNPoFTHQaKqjn5kD0xFtm5jsVwRg3/Ry6kYhSqQtlIiVTP+ShZHnA0?= =?utf-8?q?ny2tgpbTbJ5yVrJkLmb/UELZ/rQgKhOe3l79gAjRccHsCG8Vm+22FNjl6sZwqGyds?= =?utf-8?q?qJD3lDPidlqwnuyqg4uS5k5AojEECWWl0YFyF7M2sAiA5g5MuVQnAQfbIyoAIdNm9?= =?utf-8?q?dXhDeFq/HhmYMpmr+1gl3WpnyC9mbR1nmDVmHRRjSoiOGjerhUNffD4ncu3fYCWGZ?= =?utf-8?q?uZfFSXd2wKOsL9WXFUmwte49cV2PRGWuKDoG1H4QhTZpwpfgAVDhxeWShP08pd2sT?= =?utf-8?q?fih0SE90dJM+AsuP5RPRTwO92q9OyZ0dIuF1xtFaAFx7A7jdziG3k/1qb2FG8O/kL?= =?utf-8?q?d3RWnNWfgP9i43T0aguDwfSgSqp8HQmT8xWnG52iD5WAullfNlI+GF8qc3WPzgiKd?= =?utf-8?q?lq1z7E4r29lppJAyHLMSM7ZHH+qo57qNH0ubNf0i6k7kOlFz1UvGUBS+y3Wxq5dz8?= =?utf-8?q?luEXXxEMJH93c27gJSx1425ENA3Ncov1KJVOXZAO4q+qDNWawItErdgtfDiWaG1/p?= =?utf-8?q?zbDYcmqpzS6PwARAG2RywRS2TIlaH9G9TysJcz2YEHxtZKYTkOmYmbrEA/zmy3F2/?= =?utf-8?q?eQWZ5SkFXMkodvZ8GOyIYT0/QssnUbL5N9TBQ264R8W3sAaI1Ld4JpCu8u0zHFpum?= =?utf-8?q?N2VCJxWH/gVImzgOkgmLm4AtEUVJPhfPFgZoewceCXkd/axaeX1h/aMcvuAI4mCTx?= =?utf-8?q?gQPq5KPtOu+DQ1B9i1Pm1uzjOufz+UVwIO7ZHjQol2mTWzPg+w7VSuwHJhsILSPEa?= =?utf-8?q?BsjSx4+ARQaX0mXZPNAg75heCCr1LACNha4DIaBoDjR7BdzX4rifrphM59LKtSql2?= =?utf-8?q?BGcX0z7zAc1QbeyTzWT20Ze3yZSr7CXVSmQ2RVZnPDS3J0VFAniiyapXkrGquvzZe?= =?utf-8?q?+CuS4Lsg+i+Azk14O76/1tHOwhTEC9tWt8mIP+bxgJ3mHoYhQpks2h0xJ7dig7pvW?= =?utf-8?q?o3r9gqr3f94CjRw2XInmdB89CzCqIZ0uaODyriFSUcHMciomKj5we5Dxaowu7sYyl?= =?utf-8?q?TbORXziexwyMQTUwTp14PcwqLfSOjkXRX9D9mQ8VtNMFcvVNkCmwK4fEVcqawJC/f?= =?utf-8?q?Hh/O0aEgj5F7ObH6ar/pNNYbbi9VTkPsqEbGKfRvV4F2st8iNiHQOah56S/zB1kjq?= =?utf-8?q?G2oyKH3WpejCnnyiXMRa6lLxpWop/wOuO6ObBNsuJjhLlHy7o4UXGkE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?aPYnxwjzf1BKNjEcw03uMBJVG7/b?= =?utf-8?q?7oeJOK02MkIbgF+2PfM4X5i94pvk9P1R/oPvHG2mPOe07JDmqO9M6qTDQi+Etjso2?= =?utf-8?q?KcCHBQii7/meyF9+G1KOhQ1I8jhRPT9Q4YYmObQudVDytLgsKagqRSnMl84xsLLtj?= =?utf-8?q?AUua0y2nyP9CvRirNmrpBLMUVvgS5bZ4gJgMJ1zqiy9E8xNl86ndV2e1eYAEo76ek?= =?utf-8?q?rIpnZzHZ/r9QO9fSwl04rcJ2UUeYtUby06TGf0fzs+hYjV5OarqQajmJ5kP7XKvpC?= =?utf-8?q?a5WohjhVwD7enOD8Qi5XTcXFx3jBFVu1adM0fAnEWeEkHE3wWAF5GaxqSEC6UHPEU?= =?utf-8?q?pm4sLrG2Ksuimr4kMdM8TWQHtNSizYGyH2UbXsFqY2dk4pKZHE5izPduiiU8lz/n9?= =?utf-8?q?H//qkd6iuIrGUAnLEVAbCFyUYOxHEJ8i4a5sHXewl51B3rwJ4Ja2bQC6vaw2CiUnJ?= =?utf-8?q?4G/ALXF5iW4NaAdb9Ao+1zwR5T3QsfCG8Dyd5Eq0WEB5mefUYW4LRLwDrjy2qjfzJ?= =?utf-8?q?O0LHd5hoJT7hWIEgV/eWRs00XONh5OF1SyXYzl789nzZ3kUDesrVY6oAtsc15Czum?= =?utf-8?q?spA/4GVtmuDOAHyVmWnLmYnjKUNB/2CUYCfaELJUTDE8U/B60HPc8mdchftvipShO?= =?utf-8?q?aW6pX6C8zwQyQ1a2qLtrdjxPn1BCgsC7v1XrSbtR6Xu4nj/2rs51f+eQcAx+e7ycJ?= =?utf-8?q?SH+KwLhO164NZVDp93YlJWTKuuslB6+kojGHbO0Sf04v4xjkMeLCI27AhnCR5p497?= =?utf-8?q?F11HnbG6XHxp6grff6lPgBwLYsf54ue/zBTzfnSD3ZofbEyKDUGhG1GasAYL6OAHw?= =?utf-8?q?zuL1CAmZqI6aezJjE8eni7/srjM6PBBkiMp28NgFKGwbbEvxyDzQXXsANX1rw2kXR?= =?utf-8?q?OYeXzJvkLU1CvOi3cnC/LOjNcPna3aJNNovXdYc//iFs/fC3bCtt1KqO/gpzCVkT6?= =?utf-8?q?YQikfOmnBJcDKQ2xnenrP+1AXBZNrt9LZlECSIw50HmCsWENOwBwrNGfHK7jt7BW+?= =?utf-8?q?v1WIAJn+1RYE1hGQ+jUKqHoIEFRMQcJSR845dm/yuJMDpxOWzgwxM2MuCmxyX1y9Y?= =?utf-8?q?BcExKIwjMK2pBdEgR7W6WMKK7rL1lSSatPQKMBPDXu8BXF1J5i/vYdylupumu5Yll?= =?utf-8?q?REdoknBI7aARMqF2NquYoX0TNzSgpP17SYmqe0iUqL4zuh/jNKP07+pChj//w+JMu?= =?utf-8?q?djvgr9k9avRh2UbGu1Q37i0UgzSqrLAZ1Qs8PW0w94epnaDi0AHTMujXQ4obg5LNr?= =?utf-8?q?sDTGRtsfD4bOUBdAaI0Cf86X7hIE77mb1FF2LEJzpFF9cR/ZOMc/Xzg4fbG3pbyJu?= =?utf-8?q?8cMw03ZcctTgGZorykHqnrTq4S/sbu+CsW9NmvCP8TS7uPVTNVa0XGX0mKxCKJiYm?= =?utf-8?q?Ls8tK6MzpY9R+lQAo8M2qs1EWlyu9JKnoms19ynEkTv0kjwuwGbMNMH3Eft3dJYMu?= =?utf-8?q?SRThuIcQMzI3wU4IvuWVdYzkTxBa69omxO3yrb/1Kv2sdOOdK5vNxd6VbfbCUWfQi?= =?utf-8?q?c15DO0t+2fLa?= X-MS-Exchange-CrossTenant-Network-Message-Id: 78094907-f15b-41e5-147b-08dd7addce29 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:02.4914 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 48gQn+0XunOhVU+MqRSjub0KGHGKAfPj6ZMfWybXiTLlPYD31NrbC2qWhPBbQJVxVujBZfndmNZ2e/El4ay0tA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com A dynamic capacity device (DCD) sends events to signal the host for changes in the availability of Dynamic Capacity (DC) memory. These events contain extents describing a DPA range and meta data for memory to be added or removed. Events may be sent from the device at any time. Three types of events can be signaled, Add, Release, and Force Release. On add, the host may accept or reject the memory being offered. If no region exists, or the extent is invalid, the extent should be rejected. Add extent events may be grouped by a 'more' bit which indicates those extents should be processed as a group. On remove, the host can delay the response until the host is safely not using the memory. If no region exists the release can be sent immediately. The host may also release extents (or partial extents) at any time. Thus the 'more' bit grouping of release events is of less value and can be ignored in favor of sending multiple release capacity responses for groups of release events. Force removal is intended as a mechanism between the FM and the device and intended only when the host is unresponsive, out of sync, or otherwise broken. Purposely ignore force removal events. Regions are made up of one or more devices which may be surfacing memory to the host. Once all devices in a region have surfaced an extent the region can expose a corresponding extent for the user to consume. Without interleaving a device extent forms a 1:1 relationship with the region extent. Immediately surface a region extent upon getting a device extent. Per the specification the device is allowed to offer or remove extents at any time. However, anticipated use cases can expect extents to be offered, accepted, and removed in well defined chunks. Simplify extent tracking with the following restrictions. 1) Flag for removal any extent which overlaps a requested release range. 2) Refuse the offer of extents which overlap already accepted memory ranges. 3) Accept again a range which has already been accepted by the host. Eating duplicates serves three purposes. 3a) This simplifies the code if the device should get out of sync with the host. And it should be safe to acknowledge the extent again. 3b) This simplifies the code to process existing extents if the extent list should change while the extent list is being read. 3c) Duplicates for a given partition which are seen during a race between the hardware surfacing an extent and the cxl dax driver scanning for existing extents will be ignored. NOTE: Processing existing extents is done in a later patch. Management of the region extent devices must be synchronized with potential uses of the memory within the DAX layer. Create region extent devices as children of the cxl_dax_region device such that the DAX region driver can co-drive them and synchronize with the DAX layer. Synchronization and management is handled in a subsequent patch. Tag support within the DAX layer is not yet supported. To maintain compatibility with legacy DAX/region processing only tags with a value of 0 are allowed. This defines existing DAX devices as having a 0 tag which makes the most logical sense as a default. Process DCD events and create region devices. Based on an original patch by Navneet Singh. Reviewed-by: Dave Jiang Reviewed-by: Li Ming Reviewed-by: Jonathan Cameron Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase] [djbw: s/region/partition/] [iweiny: Adapt to new partition arch] [iweiny: s/tag/uuid/ throughout the code] --- drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 13 ++ drivers/cxl/core/extent.c | 366 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/mbox.c | 292 +++++++++++++++++++++++++++++++++++- drivers/cxl/core/region.c | 3 + drivers/cxl/cxl.h | 53 ++++++- drivers/cxl/cxlmem.h | 27 ++++ include/cxl/event.h | 31 ++++ tools/testing/cxl/Kbuild | 3 +- 9 files changed, 786 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 086df97a0fcf..792ac799f39d 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -17,6 +17,6 @@ cxl_core-y += cdat.o cxl_core-y += ras.o cxl_core-y += acpi.o cxl_core-$(CONFIG_TRACING) += trace.o -cxl_core-$(CONFIG_CXL_REGION) += region.o +cxl_core-$(CONFIG_CXL_REGION) += region.o extent.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 76e23ec03fb4..1272be497926 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -45,12 +45,24 @@ struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa, u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd, u64 dpa); +int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent); +int cxl_rm_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent); #else static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd, u64 dpa) { return ULLONG_MAX; } +static inline int cxl_add_extent(struct cxl_memdev_state *mds, + struct cxl_extent *extent) +{ + return 0; +} +static inline int cxl_rm_extent(struct cxl_memdev_state *mds, + struct cxl_extent *extent) +{ + return 0; +} static inline struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa, struct cxl_endpoint_decoder **cxled) @@ -129,6 +141,7 @@ int cxl_update_hmat_access_coordinates(int nid, struct cxl_region *cxlr, bool cxl_need_node_perf_attrs_update(int nid); int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port, struct access_coordinate *c); +void memdev_release_extent(struct cxl_memdev_state *mds, struct range *range); int cxl_ras_init(void); void cxl_ras_exit(void); diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c new file mode 100644 index 000000000000..6df277caf974 --- /dev/null +++ b/drivers/cxl/core/extent.c @@ -0,0 +1,366 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Intel Corporation. All rights reserved. */ + +#include +#include + +#include "core.h" + +static void cxled_release_extent(struct cxl_endpoint_decoder *cxled, + struct cxled_extent *ed_extent) +{ + struct cxl_memdev_state *mds = cxled_to_mds(cxled); + struct device *dev = &cxled->cxld.dev; + + dev_dbg(dev, "Remove extent %pra (%pU)\n", + &ed_extent->dpa_range, &ed_extent->uuid); + memdev_release_extent(mds, &ed_extent->dpa_range); + kfree(ed_extent); +} + +static void free_region_extent(struct region_extent *region_extent) +{ + struct cxled_extent *ed_extent; + unsigned long index; + + /* + * Remove from each endpoint decoder the extent which backs this region + * extent + */ + xa_for_each(®ion_extent->decoder_extents, index, ed_extent) + cxled_release_extent(ed_extent->cxled, ed_extent); + xa_destroy(®ion_extent->decoder_extents); + ida_free(®ion_extent->cxlr_dax->extent_ida, region_extent->dev.id); + kfree(region_extent); +} + +static void region_extent_release(struct device *dev) +{ + struct region_extent *region_extent = to_region_extent(dev); + + free_region_extent(region_extent); +} + +static const struct device_type region_extent_type = { + .name = "extent", + .release = region_extent_release, +}; + +bool is_region_extent(struct device *dev) +{ + return dev->type == ®ion_extent_type; +} +EXPORT_SYMBOL_NS_GPL(is_region_extent, "CXL"); + +static void region_extent_unregister(void *ext) +{ + struct region_extent *region_extent = ext; + + dev_dbg(®ion_extent->dev, "DAX region rm extent HPA %pra\n", + ®ion_extent->hpa_range); + device_unregister(®ion_extent->dev); +} + +static void region_rm_extent(struct region_extent *region_extent) +{ + struct device *region_dev = region_extent->dev.parent; + + devm_release_action(region_dev, region_extent_unregister, region_extent); +} + +static struct region_extent * +alloc_region_extent(struct cxl_dax_region *cxlr_dax, struct range *hpa_range, + uuid_t *uuid) +{ + int id; + + struct region_extent *region_extent __free(kfree) = + kzalloc(sizeof(*region_extent), GFP_KERNEL); + if (!region_extent) + return ERR_PTR(-ENOMEM); + + id = ida_alloc(&cxlr_dax->extent_ida, GFP_KERNEL); + if (id < 0) + return ERR_PTR(-ENOMEM); + + region_extent->hpa_range = *hpa_range; + region_extent->cxlr_dax = cxlr_dax; + uuid_copy(®ion_extent->uuid, uuid); + region_extent->dev.id = id; + xa_init(®ion_extent->decoder_extents); + return no_free_ptr(region_extent); +} + +static int online_region_extent(struct region_extent *region_extent) +{ + struct cxl_dax_region *cxlr_dax = region_extent->cxlr_dax; + struct device *dev = ®ion_extent->dev; + int rc; + + device_initialize(dev); + device_set_pm_not_required(dev); + dev->parent = &cxlr_dax->dev; + dev->type = ®ion_extent_type; + rc = dev_set_name(dev, "extent%d.%d", cxlr_dax->cxlr->id, dev->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + dev_dbg(dev, "region extent HPA %pra\n", ®ion_extent->hpa_range); + return devm_add_action_or_reset(&cxlr_dax->dev, region_extent_unregister, + region_extent); + +err: + dev_err(&cxlr_dax->dev, "Failed to initialize region extent HPA %pra\n", + ®ion_extent->hpa_range); + + put_device(dev); + return rc; +} + +struct match_data { + struct cxl_endpoint_decoder *cxled; + struct range *new_range; +}; + +static int match_contains(struct device *dev, const void *data) +{ + struct region_extent *region_extent = to_region_extent(dev); + const struct match_data *md = data; + struct cxled_extent *entry; + unsigned long index; + + if (!region_extent) + return 0; + + xa_for_each(®ion_extent->decoder_extents, index, entry) { + if (md->cxled == entry->cxled && + range_contains(&entry->dpa_range, md->new_range)) + return 1; + } + return 0; +} + +static bool extents_contain(struct cxl_dax_region *cxlr_dax, + struct cxl_endpoint_decoder *cxled, + struct range *new_range) +{ + struct match_data md = { + .cxled = cxled, + .new_range = new_range, + }; + + struct device *extent_device __free(put_device) + = device_find_child(&cxlr_dax->dev, &md, match_contains); + if (!extent_device) + return false; + + return true; +} + +static int match_overlaps(struct device *dev, const void *data) +{ + struct region_extent *region_extent = to_region_extent(dev); + const struct match_data *md = data; + struct cxled_extent *entry; + unsigned long index; + + if (!region_extent) + return 0; + + xa_for_each(®ion_extent->decoder_extents, index, entry) { + if (md->cxled == entry->cxled && + range_overlaps(&entry->dpa_range, md->new_range)) + return 1; + } + + return 0; +} + +static bool extents_overlap(struct cxl_dax_region *cxlr_dax, + struct cxl_endpoint_decoder *cxled, + struct range *new_range) +{ + struct match_data md = { + .cxled = cxled, + .new_range = new_range, + }; + + struct device *extent_device __free(put_device) + = device_find_child(&cxlr_dax->dev, &md, match_overlaps); + if (!extent_device) + return false; + + return true; +} + +static void calc_hpa_range(struct cxl_endpoint_decoder *cxled, + struct cxl_dax_region *cxlr_dax, + struct range *dpa_range, + struct range *hpa_range) +{ + resource_size_t dpa_offset, hpa; + + dpa_offset = dpa_range->start - cxled->dpa_res->start; + hpa = cxled->cxld.hpa_range.start + dpa_offset; + + hpa_range->start = hpa - cxlr_dax->hpa_range.start; + hpa_range->end = hpa_range->start + range_len(dpa_range) - 1; +} + +static int cxlr_rm_extent(struct device *dev, void *data) +{ + struct region_extent *region_extent = to_region_extent(dev); + struct range *region_hpa_range = data; + + if (!region_extent) + return 0; + + /* + * Any extent which 'touches' the released range is removed. + */ + if (range_overlaps(region_hpa_range, ®ion_extent->hpa_range)) { + dev_dbg(dev, "Remove region extent HPA %pra\n", + ®ion_extent->hpa_range); + region_rm_extent(region_extent); + } + return 0; +} + +int cxl_rm_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent) +{ + u64 start_dpa = le64_to_cpu(extent->start_dpa); + struct cxl_memdev *cxlmd = mds->cxlds.cxlmd; + struct cxl_endpoint_decoder *cxled; + struct range hpa_range, dpa_range; + struct cxl_region *cxlr; + + dpa_range = (struct range) { + .start = start_dpa, + .end = start_dpa + le64_to_cpu(extent->length) - 1, + }; + + guard(rwsem_read)(&cxl_region_rwsem); + cxlr = cxl_dpa_to_region(cxlmd, start_dpa, &cxled); + if (!cxlr) { + /* + * No region can happen here for a few reasons: + * + * 1) Extents were accepted and the host crashed/rebooted + * leaving them in an accepted state. On reboot the host + * has not yet created a region to own them. + * + * 2) Region destruction won the race with the device releasing + * all the extents. Here the release will be a duplicate of + * the one sent via region destruction. + * + * 3) The device is confused and releasing extents for which no + * region ever existed. + * + * In all these cases make sure the device knows we are not + * using this extent. + */ + memdev_release_extent(mds, &dpa_range); + return -ENXIO; + } + + calc_hpa_range(cxled, cxlr->cxlr_dax, &dpa_range, &hpa_range); + + /* Remove region extents which overlap */ + return device_for_each_child(&cxlr->cxlr_dax->dev, &hpa_range, + cxlr_rm_extent); +} + +static int cxlr_add_extent(struct cxl_dax_region *cxlr_dax, + struct cxl_endpoint_decoder *cxled, + struct cxled_extent *ed_extent) +{ + struct region_extent *region_extent; + struct range hpa_range; + int rc; + + calc_hpa_range(cxled, cxlr_dax, &ed_extent->dpa_range, &hpa_range); + + region_extent = alloc_region_extent(cxlr_dax, &hpa_range, &ed_extent->uuid); + if (IS_ERR(region_extent)) + return PTR_ERR(region_extent); + + rc = xa_insert(®ion_extent->decoder_extents, (unsigned long)ed_extent, + ed_extent, GFP_KERNEL); + if (rc) { + free_region_extent(region_extent); + return rc; + } + + /* device model handles freeing region_extent */ + return online_region_extent(region_extent); +} + +/* Callers are expected to ensure cxled has been attached to a region */ +int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent) +{ + u64 start_dpa = le64_to_cpu(extent->start_dpa); + struct cxl_memdev *cxlmd = mds->cxlds.cxlmd; + struct cxl_endpoint_decoder *cxled; + struct range ed_range, ext_range; + struct cxl_dax_region *cxlr_dax; + struct cxled_extent *ed_extent; + struct cxl_region *cxlr; + struct device *dev; + + ext_range = (struct range) { + .start = start_dpa, + .end = start_dpa + le64_to_cpu(extent->length) - 1, + }; + + guard(rwsem_read)(&cxl_region_rwsem); + cxlr = cxl_dpa_to_region(cxlmd, start_dpa, &cxled); + if (!cxlr) + return -ENXIO; + + cxlr_dax = cxled->cxld.region->cxlr_dax; + dev = &cxled->cxld.dev; + ed_range = (struct range) { + .start = cxled->dpa_res->start, + .end = cxled->dpa_res->end, + }; + + dev_dbg(&cxled->cxld.dev, "Checking ED (%pr) for extent %pra\n", + cxled->dpa_res, &ext_range); + + if (!range_contains(&ed_range, &ext_range)) { + dev_err_ratelimited(dev, + "DC extent DPA %pra (%pU) is not fully in ED %pra\n", + &ext_range, extent->uuid, &ed_range); + return -ENXIO; + } + + /* + * Allowing duplicates or extents which are already in an accepted + * range simplifies extent processing, especially when dealing with the + * cxl dax driver scanning for existing extents. + */ + if (extents_contain(cxlr_dax, cxled, &ext_range)) { + dev_warn_ratelimited(dev, "Extent %pra exists; accept again\n", + &ext_range); + return 0; + } + + if (extents_overlap(cxlr_dax, cxled, &ext_range)) + return -ENXIO; + + ed_extent = kzalloc(sizeof(*ed_extent), GFP_KERNEL); + if (!ed_extent) + return -ENOMEM; + + ed_extent->cxled = cxled; + ed_extent->dpa_range = ext_range; + import_uuid(&ed_extent->uuid, extent->uuid); + + dev_dbg(dev, "Add extent %pra (%pU)\n", &ed_extent->dpa_range, &ed_extent->uuid); + + return cxlr_add_extent(cxlr_dax, cxled, ed_extent); +} diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index b3dd119d166a..de01c6684530 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -930,6 +930,60 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds) } EXPORT_SYMBOL_NS_GPL(cxl_enumerate_cmds, "CXL"); +static int cxl_validate_extent(struct cxl_memdev_state *mds, + struct cxl_extent *extent) +{ + struct cxl_dev_state *cxlds = &mds->cxlds; + struct device *dev = mds->cxlds.dev; + u64 start, length; + + start = le64_to_cpu(extent->start_dpa); + length = le64_to_cpu(extent->length); + + struct range ext_range = (struct range){ + .start = start, + .end = start + length - 1, + }; + + if (le16_to_cpu(extent->shared_extn_seq) != 0) { + dev_err_ratelimited(dev, + "DC extent DPA %pra (%pU) can not be shared\n", + &ext_range, extent->uuid); + return -ENXIO; + } + + if (!uuid_is_null((const uuid_t *)extent->uuid)) { + dev_err_ratelimited(dev, + "DC extent DPA %pra (%pU); tags not supported\n", + &ext_range, extent->uuid); + return -ENXIO; + } + + /* Extents must be within the DC partition boundary */ + for (int i = 0; i < cxlds->nr_partitions; i++) { + struct cxl_dpa_partition *part = &cxlds->part[i]; + + if (part->mode != CXL_PARTMODE_DYNAMIC_RAM_A) + continue; + + struct range partition_range = (struct range) { + .start = part->res.start, + .end = part->res.end, + }; + + if (range_contains(&partition_range, &ext_range)) { + dev_dbg(dev, "DC extent DPA %pra (DCR:%pra)(%pU)\n", + &ext_range, &partition_range, extent->uuid); + return 0; + } + } + + dev_err_ratelimited(dev, + "DC extent DPA %pra (%pU) is not in a valid DC partition\n", + &ext_range, extent->uuid); + return -ENXIO; +} + void cxl_event_trace_record(const struct cxl_memdev *cxlmd, enum cxl_event_log_type type, enum cxl_event_type event_type, @@ -1064,6 +1118,221 @@ static int cxl_clear_event_record(struct cxl_memdev_state *mds, return rc; } +static int send_one_response(struct cxl_mailbox *cxl_mbox, + struct cxl_mbox_dc_response *response, + int opcode, u32 extent_list_size, u8 flags) +{ + struct cxl_mbox_cmd mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = opcode, + .size_in = struct_size(response, extent_list, extent_list_size), + .payload_in = response, + }; + + response->extent_list_size = cpu_to_le32(extent_list_size); + response->flags = flags; + return cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); +} + +static int cxl_send_dc_response(struct cxl_memdev_state *mds, int opcode, + struct xarray *extent_array, int cnt) +{ + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + struct cxl_mbox_dc_response *p; + struct cxl_extent *extent; + unsigned long index; + u32 pl_index; + + size_t pl_size = struct_size(p, extent_list, cnt); + u32 max_extents = cnt; + + /* May have to use more bit on response. */ + if (pl_size > cxl_mbox->payload_size) { + max_extents = (cxl_mbox->payload_size - sizeof(*p)) / + sizeof(struct updated_extent_list); + pl_size = struct_size(p, extent_list, max_extents); + } + + struct cxl_mbox_dc_response *response __free(kfree) = + kzalloc(pl_size, GFP_KERNEL); + if (!response) + return -ENOMEM; + + if (cnt == 0) + return send_one_response(cxl_mbox, response, opcode, 0, 0); + + pl_index = 0; + xa_for_each(extent_array, index, extent) { + response->extent_list[pl_index].dpa_start = extent->start_dpa; + response->extent_list[pl_index].length = extent->length; + pl_index++; + + if (pl_index == max_extents) { + u8 flags = 0; + int rc; + + if (pl_index < cnt) + flags |= CXL_DCD_EVENT_MORE; + rc = send_one_response(cxl_mbox, response, opcode, + pl_index, flags); + if (rc) + return rc; + cnt -= pl_index; + pl_index = 0; + } + } + + if (!pl_index) /* nothing more to do */ + return 0; + return send_one_response(cxl_mbox, response, opcode, pl_index, 0); +} + +void memdev_release_extent(struct cxl_memdev_state *mds, struct range *range) +{ + struct device *dev = mds->cxlds.dev; + struct xarray extent_list; + + struct cxl_extent extent = { + .start_dpa = cpu_to_le64(range->start), + .length = cpu_to_le64(range_len(range)), + }; + + dev_dbg(dev, "Release response dpa %pra\n", &range); + + xa_init(&extent_list); + if (xa_insert(&extent_list, 0, &extent, GFP_KERNEL)) { + dev_dbg(dev, "Failed to release %pra\n", &range); + goto destroy; + } + + if (cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC, &extent_list, 1)) + dev_dbg(dev, "Failed to release %pra\n", &range); + +destroy: + xa_destroy(&extent_list); +} + +static int validate_add_extent(struct cxl_memdev_state *mds, + struct cxl_extent *extent) +{ + int rc; + + rc = cxl_validate_extent(mds, extent); + if (rc) + return rc; + + return cxl_add_extent(mds, extent); +} + +static int cxl_add_pending(struct cxl_memdev_state *mds) +{ + struct device *dev = mds->cxlds.dev; + struct cxl_extent *extent; + unsigned long cnt = 0; + unsigned long index; + int rc; + + xa_for_each(&mds->pending_extents, index, extent) { + if (validate_add_extent(mds, extent)) { + /* + * Any extents which are to be rejected are omitted from + * the response. An empty response means all are + * rejected. + */ + dev_dbg(dev, "unconsumed DC extent DPA:%#llx LEN:%#llx\n", + le64_to_cpu(extent->start_dpa), + le64_to_cpu(extent->length)); + xa_erase(&mds->pending_extents, index); + kfree(extent); + continue; + } + cnt++; + } + rc = cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE, + &mds->pending_extents, cnt); + xa_for_each(&mds->pending_extents, index, extent) { + xa_erase(&mds->pending_extents, index); + kfree(extent); + } + return rc; +} + +static int handle_add_event(struct cxl_memdev_state *mds, + struct cxl_event_dcd *event) +{ + struct device *dev = mds->cxlds.dev; + struct cxl_extent *extent; + + extent = kmemdup(&event->extent, sizeof(*extent), GFP_KERNEL); + if (!extent) + return -ENOMEM; + + if (xa_insert(&mds->pending_extents, (unsigned long)extent, extent, + GFP_KERNEL)) { + kfree(extent); + return -ENOMEM; + } + + if (event->flags & CXL_DCD_EVENT_MORE) { + dev_dbg(dev, "more bit set; delay the surfacing of extent\n"); + return 0; + } + + /* extents are removed and free'ed in cxl_add_pending() */ + return cxl_add_pending(mds); +} + +static char *cxl_dcd_evt_type_str(u8 type) +{ + switch (type) { + case DCD_ADD_CAPACITY: + return "add"; + case DCD_RELEASE_CAPACITY: + return "release"; + case DCD_FORCED_CAPACITY_RELEASE: + return "force release"; + default: + break; + } + + return ""; +} + +static void cxl_handle_dcd_event_records(struct cxl_memdev_state *mds, + struct cxl_event_record_raw *raw_rec) +{ + struct cxl_event_dcd *event = &raw_rec->event.dcd; + struct cxl_extent *extent = &event->extent; + struct device *dev = mds->cxlds.dev; + uuid_t *id = &raw_rec->id; + int rc; + + if (!uuid_equal(id, &CXL_EVENT_DC_EVENT_UUID)) + return; + + dev_dbg(dev, "DCD event %s : DPA:%#llx LEN:%#llx\n", + cxl_dcd_evt_type_str(event->event_type), + le64_to_cpu(extent->start_dpa), le64_to_cpu(extent->length)); + + switch (event->event_type) { + case DCD_ADD_CAPACITY: + rc = handle_add_event(mds, event); + break; + case DCD_RELEASE_CAPACITY: + rc = cxl_rm_extent(mds, &event->extent); + break; + case DCD_FORCED_CAPACITY_RELEASE: + dev_err_ratelimited(dev, "Forced release event ignored.\n"); + rc = 0; + break; + default: + rc = -EINVAL; + break; + } + + if (rc) + dev_err_ratelimited(dev, "dcd event failed: %d\n", rc); +} + static void cxl_mem_get_records_log(struct cxl_memdev_state *mds, enum cxl_event_log_type type) { @@ -1100,9 +1369,13 @@ static void cxl_mem_get_records_log(struct cxl_memdev_state *mds, if (!nr_rec) break; - for (i = 0; i < nr_rec; i++) + for (i = 0; i < nr_rec; i++) { __cxl_event_trace_record(cxlmd, type, &payload->records[i]); + if (type == CXL_EVENT_TYPE_DCD) + cxl_handle_dcd_event_records(mds, + &payload->records[i]); + } if (payload->flags & CXL_GET_EVENT_FLAG_OVERFLOW) trace_cxl_overflow(cxlmd, type, payload); @@ -1134,6 +1407,8 @@ void cxl_mem_get_event_records(struct cxl_memdev_state *mds, u32 status) { dev_dbg(mds->cxlds.dev, "Reading event logs: %x\n", status); + if (cxl_dcd_supported(mds) && (status & CXLDEV_EVENT_STATUS_DCD)) + cxl_mem_get_records_log(mds, CXL_EVENT_TYPE_DCD); if (status & CXLDEV_EVENT_STATUS_FATAL) cxl_mem_get_records_log(mds, CXL_EVENT_TYPE_FATAL); if (status & CXLDEV_EVENT_STATUS_FAIL) @@ -1709,6 +1984,17 @@ int cxl_mailbox_init(struct cxl_mailbox *cxl_mbox, struct device *host) } EXPORT_SYMBOL_NS_GPL(cxl_mailbox_init, "CXL"); +static void clear_pending_extents(void *_mds) +{ + struct cxl_memdev_state *mds = _mds; + struct cxl_extent *extent; + unsigned long index; + + xa_for_each(&mds->pending_extents, index, extent) + kfree(extent); + xa_destroy(&mds->pending_extents); +} + struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) { struct cxl_memdev_state *mds; @@ -1726,6 +2012,10 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mds->cxlds.cxl_mbox.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; + xa_init(&mds->pending_extents); + rc = devm_add_action_or_reset(dev, clear_pending_extents, mds); + if (rc) + return ERR_PTR(rc); rc = devm_cxl_register_mce_notifier(dev, &mds->mce_notifier); if (rc == -EOPNOTSUPP) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 9c573e8d6ed7..3106df6f3636 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3048,6 +3048,7 @@ static void cxl_dax_region_release(struct device *dev) { struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + ida_destroy(&cxlr_dax->extent_ida); kfree(cxlr_dax); } @@ -3097,6 +3098,8 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr) dev = &cxlr_dax->dev; cxlr_dax->cxlr = cxlr; + cxlr->cxlr_dax = cxlr_dax; + ida_init(&cxlr_dax->extent_ida); device_initialize(dev); lockdep_set_class(&dev->mutex, &cxl_dax_region_key); device_set_pm_not_required(dev); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 4bb0ff4d8f5f..d027432b1572 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -11,6 +11,8 @@ #include #include #include +#include +#include extern const struct nvdimm_security_ops *cxl_security_ops; @@ -169,11 +171,13 @@ static inline int ways_to_eiw(unsigned int ways, u8 *eiw) #define CXLDEV_EVENT_STATUS_WARN BIT(1) #define CXLDEV_EVENT_STATUS_FAIL BIT(2) #define CXLDEV_EVENT_STATUS_FATAL BIT(3) +#define CXLDEV_EVENT_STATUS_DCD BIT(4) #define CXLDEV_EVENT_STATUS_ALL (CXLDEV_EVENT_STATUS_INFO | \ CXLDEV_EVENT_STATUS_WARN | \ CXLDEV_EVENT_STATUS_FAIL | \ - CXLDEV_EVENT_STATUS_FATAL) + CXLDEV_EVENT_STATUS_FATAL | \ + CXLDEV_EVENT_STATUS_DCD) /* CXL rev 3.0 section 8.2.9.2.4; Table 8-52 */ #define CXLDEV_EVENT_INT_MODE_MASK GENMASK(1, 0) @@ -381,6 +385,18 @@ enum cxl_decoder_state { CXL_DECODER_STATE_AUTO, }; +/** + * struct cxled_extent - Extent within an endpoint decoder + * @cxled: Reference to the endpoint decoder + * @dpa_range: DPA range this extent covers within the decoder + * @uuid: uuid from device for this extent + */ +struct cxled_extent { + struct cxl_endpoint_decoder *cxled; + struct range dpa_range; + uuid_t uuid; +}; + /** * struct cxl_endpoint_decoder - Endpoint / SPA to DPA decoder * @cxld: base cxl_decoder_object @@ -512,6 +528,7 @@ enum cxl_partition_mode { * @type: Endpoint decoder target type * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge + * @cxlr_dax: (for DC regions) cached copy of CXL DAX bridge * @flags: Region state flags * @params: active + config params for the region * @coord: QoS access coordinates for the region @@ -525,6 +542,7 @@ struct cxl_region { enum cxl_decoder_type type; struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_pmem_region *cxlr_pmem; + struct cxl_dax_region *cxlr_dax; unsigned long flags; struct cxl_region_params params; struct access_coordinate coord[ACCESS_COORDINATE_MAX]; @@ -566,12 +584,45 @@ struct cxl_pmem_region { struct cxl_pmem_region_mapping mapping[]; }; +/* See CXL 3.1 8.2.9.2.1.6 */ +enum dc_event { + DCD_ADD_CAPACITY, + DCD_RELEASE_CAPACITY, + DCD_FORCED_CAPACITY_RELEASE, + DCD_REGION_CONFIGURATION_UPDATED, +}; + struct cxl_dax_region { struct device dev; struct cxl_region *cxlr; struct range hpa_range; + struct ida extent_ida; }; +/** + * struct region_extent - CXL DAX region extent + * @dev: device representing this extent + * @cxlr_dax: back reference to parent region device + * @hpa_range: HPA range of this extent + * @uuid: uuid of the extent + * @decoder_extents: Endpoint decoder extents which make up this region extent + */ +struct region_extent { + struct device dev; + struct cxl_dax_region *cxlr_dax; + struct range hpa_range; + uuid_t uuid; + struct xarray decoder_extents; +}; + +bool is_region_extent(struct device *dev); +static inline struct region_extent *to_region_extent(struct device *dev) +{ + if (!is_region_extent(dev)) + return NULL; + return container_of(dev, struct region_extent, dev); +} + /** * struct cxl_port - logical collection of upstream port devices and * downstream port devices to construct a CXL memory diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 34a606c5ead0..63a38e449454 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include "cxl.h" @@ -487,6 +488,7 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox) * @active_volatile_bytes: sum of hard + soft volatile * @active_persistent_bytes: sum of hard + soft persistent * @dcd_supported: all DCD commands are supported + * @pending_extents: array of extents pending during more bit processing * @event: event log driver state * @poison: poison driver state info * @security: security driver state info @@ -507,6 +509,7 @@ struct cxl_memdev_state { u64 active_volatile_bytes; u64 active_persistent_bytes; bool dcd_supported; + struct xarray pending_extents; struct cxl_event_state event; struct cxl_poison_state poison; @@ -582,6 +585,21 @@ enum cxl_opcode { UUID_INIT(0x5e1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, \ 0x40, 0x3d, 0x86) +/* + * Add Dynamic Capacity Response + * CXL rev 3.1 section 8.2.9.9.9.3; Table 8-168 & Table 8-169 + */ +struct cxl_mbox_dc_response { + __le32 extent_list_size; + u8 flags; + u8 reserved[3]; + struct updated_extent_list { + __le64 dpa_start; + __le64 length; + u8 reserved[8]; + } __packed extent_list[]; +} __packed; + struct cxl_mbox_get_supported_logs { __le16 entries; u8 rsvd[6]; @@ -644,6 +662,14 @@ struct cxl_mbox_identify { UUID_INIT(0xfe927475, 0xdd59, 0x4339, 0xa5, 0x86, 0x79, 0xba, 0xb1, \ 0x13, 0xb7, 0x74) +/* + * Dynamic Capacity Event Record + * CXL rev 3.1 section 8.2.9.2.1; Table 8-43 + */ +#define CXL_EVENT_DC_EVENT_UUID \ + UUID_INIT(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f, 0x95, 0x26, 0x8e, \ + 0x10, 0x1a, 0x2a) + /* * Get Event Records output payload * CXL rev 3.0 section 8.2.9.2.2; Table 8-50 @@ -669,6 +695,7 @@ enum cxl_event_log_type { CXL_EVENT_TYPE_WARN, CXL_EVENT_TYPE_FAIL, CXL_EVENT_TYPE_FATAL, + CXL_EVENT_TYPE_DCD, CXL_EVENT_TYPE_MAX }; diff --git a/include/cxl/event.h b/include/cxl/event.h index f9ae1796da85..0c159eac4337 100644 --- a/include/cxl/event.h +++ b/include/cxl/event.h @@ -108,11 +108,42 @@ struct cxl_event_mem_module { u8 reserved[0x2a]; } __packed; +/* + * CXL rev 3.1 section 8.2.9.2.1.6; Table 8-51 + */ +struct cxl_extent { + __le64 start_dpa; + __le64 length; + u8 uuid[UUID_SIZE]; + __le16 shared_extn_seq; + u8 reserved[0x6]; +} __packed; + +/* + * Dynamic Capacity Event Record + * CXL rev 3.1 section 8.2.9.2.1.6; Table 8-50 + */ +#define CXL_DCD_EVENT_MORE BIT(0) +struct cxl_event_dcd { + struct cxl_event_record_hdr hdr; + u8 event_type; + u8 validity_flags; + __le16 host_id; + u8 partition_index; + u8 flags; + u8 reserved1[0x2]; + struct cxl_extent extent; + u8 reserved2[0x18]; + __le32 num_avail_extents; + __le32 num_avail_tags; +} __packed; + union cxl_event { struct cxl_event_generic generic; struct cxl_event_gen_media gen_media; struct cxl_event_dram dram; struct cxl_event_mem_module mem_module; + struct cxl_event_dcd dcd; /* dram & gen_media event header */ struct cxl_event_media_hdr media_hdr; } __packed; diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index 387f3df8b988..916f2b30e2f3 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -64,7 +64,8 @@ cxl_core-y += $(CXL_CORE_SRC)/cdat.o cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-y += $(CXL_CORE_SRC)/acpi.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o -cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o +cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o \ + $(CXL_CORE_SRC)/extent.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-y += config_check.o From patchwork Sun Apr 13 22:52:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049474 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B7B51F3FEB for ; Sun, 13 Apr 2025 22:52:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584757; cv=fail; b=EiwpNkwmz5eH/ITVhWLW5ihsmofNb2bNxqEp364LvyxZeiG3tWFGWYRZ7We7G0CgGlTN2i2c+hyVR7MbT7jRPTowmhyX3iL9xySvvtGfPZtLnoadJadgPXUJpbXW6dWBP9lmrI7OUF+pCJ0DqAwjBQUyYOtDKQFZEwwAM1b4qRQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584757; c=relaxed/simple; bh=4QxktrWxnejqz2TUfcmWO6HRceAt1/3c56ZBgEUdUaI=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=DmVYoSBFlnqtZNJhjhrBs4sHz8g254EuFgW+3X1J40VPDgbB9UV3fknu6wp/SBrt+2EqhfwHlaL69jCedxRiltrjw35FQo3w0HAI8sp5+s8NN17cWJC22+0C4N/matvg/D7NuRCsdZ1U7E7i1ux4tQadILkn1Rr6d/6nwGgp63E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gzCWUHOo; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gzCWUHOo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584756; x=1776120756; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=4QxktrWxnejqz2TUfcmWO6HRceAt1/3c56ZBgEUdUaI=; b=gzCWUHOo9wYeIXeBMvuOMzBpZ7lv6UWdJTZHKxZ0oWz8KgeXA3bcMxnr 5hOUslJJn4Oc1MLHNKrlsAaBbC3xB2z5MwWlTsZ2wMF7WHHKiEViu9uso RMGHoDSZfTry6mkeYnoNBVGL9IDGKbZhtAqVSA/Y0TymZoAzvu9BLtAcq 4K9eM1PkkorPEs6LSzprdbav5Nl0PFuIHhRskettpd/5yJ4wczaDPdgbA lveI8q4h+aeBtoyoB3vlV3xJe+Mhi0PEVZ+94Cid60RZdxTH+x+kDFWdT xy3HRFhZf39a2CMxPmtCr8vAwcuRtekLKX53z8TXkfv1tsGXsSiyu4zvj A==; X-CSE-ConnectionGUID: 8g+T8kqoRHCDx9cnBQjkng== X-CSE-MsgGUID: RAM0OeLTRPi9k9SfW59zoQ== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280960" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280960" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:35 -0700 X-CSE-ConnectionGUID: hucgneI/QBy8+4zmjQaRCw== X-CSE-MsgGUID: 42k62g37RrK55hw3QiNPzA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657634" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:35 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:34 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:34 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.57.44) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:33 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pa2G21XfKcJlz/lYAe7jXxcoEBrGy+x6wRpJ2eTHZZHWkYb8ftLIkj8QaYFC9wQQHUb+TAK9fUtGm+UZ8nyNo+0KIbYpBm4bLF2Wa2eeWEeSrBAdesR9DjQ2Gg4+MLVVbJh84qLXEshow+iOTfXTE1U0MwJSN1xhXea5iI9KDfiI2lts4e+iApiTMG6cJh50pj4IVEooiydw/XVG8hSC7DSc1L+rrqTe+sqY4HM/fGmiFiXPSmn49+JC3QsQRJdrV/6lsT59F6g1rDfyKUFqPmnQEX+KpnCCyX/dE2lCd+jos3gcTglMkUNV9bbuy3Akn1RM18DehMZFsyeQAG4KEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MvYgRMhAktNXF4t/3pfUAOXXg8C+qZBgHgfffXR6mRU=; b=oLGmzMkDhxuJRsxN22dijTAcd+1BV5bSF97qISlJRnmAezbYOFYl9eQ25HSr+htClXSKh+XB6ZxLp17oGErknFK8a1zL7nscXUDlAjsRpuaR5TMulwA2TfRCGDiI7jNLDoP6K4CzDiXuTE0p1TWgKPskUZqX9/c3BmWo1vYuaOtIbzW/xZxYJynuY1X5PVb8vkDqYB8choKDnCTfgzHiZsDTYpW5X6w2eddFRhhqRoFdYHsuuSCdXxcfRy7neiDoGzlGzGHX+t5lv2uQPJS7jh4/qbgqD6M3afnl1YjHPK/5z5jNJqsp+FFipT9efqoDfdAqdcUz5oOE9UzpB9rMog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:04 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:04 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:21 -0500 Subject: [PATCH v9 13/19] cxl/region/extent: Expose region extent information in sysfs Message-ID: <20250413-dcd-type2-upstream-v9-13-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=4964; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=4QxktrWxnejqz2TUfcmWO6HRceAt1/3c56ZBgEUdUaI=; b=KXsGvbveT4s3jlccnLfPtcpIne+PH6RDv1qMtHp9JjK0/d5oC6VwfYnbnUeflSx+a8NH3ma80 338X1M1ZU7mCB8fIijUF6yGiBMv2LLb6V6KkQqxWnWE7f7mHaSathc1 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: 434ae3cb-3d29-4c63-ebd5-08dd7addcf28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?Xl6zUccs8sUkmFuabAMTC82mOVGeQXB?= =?utf-8?q?5iiOD4QXfVXeENY5em7aDDxu07OSMUoNRvvN/FMTeWIT/0heoQl/nrr18JZasBjK6?= =?utf-8?q?bhHr91hNIu9H5RVfb3xIHQftLiOtJYjH1MVfM9aj1tkCqMCfJ1M+3Q4aiRccR8gza?= =?utf-8?q?6YrAqBF5X1oNvESzz8VNgPDLajQkVAUQQExWpXjENb/Jyw1KOQTLd/WKjKtXc4anY?= =?utf-8?q?dSRS0eYxfWkaqIcbtmwbjEAJ+R20C74Qq5AoEA8OTL0qwvgxjG3Z5uEr5uDgIPOs+?= =?utf-8?q?wXjDHMX+dDhhJzqKY2CAcoW5jLpcCr0PnL/09kjEOMSsVkphNLqW6akB5g2HsfYhi?= =?utf-8?q?PqGAtXKbm/4U/qaXz48UfqGumZvPsxigYVRZ7ZoHGde1yoAC5DKzWnuRGuN0uhPEu?= =?utf-8?q?1CNjt/sT+5ji8npN2TIp1D/HOZFyKaOx6tHdTUz67c0eb14DJ55lA6EUJOL+QHZAE?= =?utf-8?q?hs8AgkcF/PNddx9RA64+AFu5x4tyv3OiYZ9FIDuqCt7ZQ5GpqZeqfmRBJZmsn5Ara?= =?utf-8?q?JRQRPqD/vLQCXhtYhLlp+DSV4suU0ShrOLQ35xAn+tBgAvsPHlVsEuNT4rhoSmg33?= =?utf-8?q?qWJE7ZC34m+UeYLlD1WXU0WG2dfNqVoJcE3c6HWrcVulwzlCNFObLyyUTmQufQ/mR?= =?utf-8?q?7kLF34/VEokhg8hm+NGVZwtdXsTJ6L/9XRxCBPSLNXiccDfT2SXImFNLWPAD88V3K?= =?utf-8?q?+kMlQzoWEVUL8fRH6F866NHoV9sJZSMjHPhISKn8+il1EAObQco7A8//dhzxkYFTj?= =?utf-8?q?4w7VjioMb+AXZEPFvxKMXOMH24Lue6BUybipPv9wurW+cwWYYm94/6PzNwpusGJHy?= =?utf-8?q?1HXNPHF36KiwIYgn1PUS3sTvz+5LhF4GB5MFLV9i1ge56hZVev7UWhHly6/7o95+9?= =?utf-8?q?Va5E21Oa5cIhaigfZkhGb1Q6tstXyq4xZjh0RDfguxZASyhBHyh/JyKlnTp6aYS/u?= =?utf-8?q?VkO4xeVdWi3ZsWPizq3rhLuz4CCChg50+Zf7XMYWu4l7BrBqAwJr5FsPtl5HpXd/A?= =?utf-8?q?sq+8NGJKsj6eDCnG/sxG9uJxG1SqfaizNolErfXKNZAQbtSc6nVRq2uPmlKyCHNjc?= =?utf-8?q?an7ncd0HEYQsW7bzxgSFBEPQsb8+ZhiqGtsy+PhsHjiWPEp0W5zXCNPx0QUD0zcZV?= =?utf-8?q?kcYsEd4Z3jGZ880HIHddZd7+kK2doK5jiWf5S7mLN7FvV+6W3dVzIIGlHLm8I4rGH?= =?utf-8?q?9KX/fMfGFgDZaI+UWWeDBrhNMxVIgyRfo95y1bYO9aIxuJbGOe2JeYXBDHln9bdCv?= =?utf-8?q?T8tFzkj6MG39tVSFRKZf2EDp7ZiwyLGkJ6xn6EGkEHPnF5MPyVtYied45lxp7pY0U?= =?utf-8?q?3sqpTbJRI5rmQd2HbfjYx8hAaiMM0NZDlagu57MCzOqWZtaG0xIk2ZQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?LKUmmMiH9ge5T8xNg7Rtt8ZJGc4s?= =?utf-8?q?8gZUww9Fa7KCr5XsKE1BsKBlCMonyYVUIC8krR41av4Ryhk3i+EXt+748DFojrBMm?= =?utf-8?q?wRycmQaaMuudeOo8E9CgN3bHjDlAHwvN1ThU0Y7BQ875/3I/gIKnU9z+Wb7YioDgL?= =?utf-8?q?t80rLy7/FuvcEk3+iuVDRZTrb7LzG8p8D6QBJfNgGQOiMmRKLKZpfBr4+JFT0eMLZ?= =?utf-8?q?Z1XssRrK/tb2FT34T370YwabewsgMqjRMj9m4LEcIxFPi2DDqNnt8uQqZ8OxKO6Al?= =?utf-8?q?AC07zt5dHnG/C2LnSlPX3t/nS7OsqIUSZfal34y9SW+cb3fdkYLsLSG5UTkmX0V2O?= =?utf-8?q?8KZ0KhNdlUsyQU0RM1jnuM1N34iFxupPNTn8H+nBh/b6aoTfRnJUG8H6ZDWkkO8zb?= =?utf-8?q?ucEl1x51NtNZxOZeI5l38xNA81t4ypLVepgYPsNqDpcbZDGJ2fRsY8jTf6CocjSXm?= =?utf-8?q?Y0Z2UopGe9XMMz2kQYaISzX8++yrRaUal21jQb4vldCnjSV6lPG389OEkbf+7npzg?= =?utf-8?q?Xio7JLDDapzHKJnPsvCV4NeDMT1U1SQti9WOoDSCfpKkb0m7GvhWoZwzV172ufGTt?= =?utf-8?q?arih1Sr8uEyLf3mS0kO+a8yWUfJJzmxwAJWt28HdhzlcHdhHWlE2r6yZWzyKsGT6j?= =?utf-8?q?vgEk4m8wm3Eb1babS3+wCzByxjslD8A9RTiMyTxN67Oox3Vuijnlm0mzoGJd98gCj?= =?utf-8?q?gXG0rWQiRR7hSlLePpPPMQtQRq9TGR/A7wSSoH8biuAi63+l3y9JYZ9KPiSn841iw?= =?utf-8?q?GZYSk1VChNngGg5dTSzAkgfgdrBakuDrpM0WaSWZnKwMXl5yUE1lKa6Emgw3gPZio?= =?utf-8?q?i0HD0oD4wHZ/s7pGaFKL2bJKnVqs92AaMpIp2MWWwRutlEqUiMuS0IlriZ9nbafjx?= =?utf-8?q?c+OpqJaLgLvDEfvLN2Rx5C8WPZ8o3x/EIfFUC0GIAomWkZUbRgYa8inpBjKyZZMaz?= =?utf-8?q?Eyl/Z2zKMcpvvCs/w7EeA9qVZ6+Aaenwqc6qbvVVSSCW4uPtPSSPowGWrdakyO54c?= =?utf-8?q?/MxhIa/x6RcXqcyF/TlKFIsDZRS4RngKW/CnnNsmR2Usz4FRm2YLcIanv/obtZXUc?= =?utf-8?q?u6lXZbdTJ3CgbSNmzXjbkIsQ66ZJl2NJf+kr76HMaCdb6g3RlxhUQTvJjK5h8IvY6?= =?utf-8?q?N5Wa5BGxdBseUVr52gXxzfvvNgzca0LqgH3gPj7RtVr/iFxc8SV+IKb48/dkwod/v?= =?utf-8?q?JljVvA5mW77YxRzGDWpqpItx6ZvTKAny4WpCZAIO11rmdQjkWwo7yoNv0Y7NIREYg?= =?utf-8?q?nKUdV46vHkvCKilECju8ZnEfSHLocQQO1DqiGtu+iUvod835/97AAOKVmSaFkFabU?= =?utf-8?q?H9+X5zDxtB3jnZxWPqsx2+A74ACtQXmaaCWnF6vS2d7PknJqCiMWe5zgDSk70nt+1?= =?utf-8?q?Plsok9QavY6CsYLEr4qyscqJut1Zfm61tW9S09/wSR4kai6a0Fh4IIAQrIapu/5xI?= =?utf-8?q?yOk3K3Q+pma9cF2rU8bnTp+xJ0Wof+9Kb2qFUVwmevrTE5B+QyV98awORSq61wSnw?= =?utf-8?q?AN+0VXOF8v+h?= X-MS-Exchange-CrossTenant-Network-Message-Id: 434ae3cb-3d29-4c63-ebd5-08dd7addcf28 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:04.0304 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0rCIlEwUDPezvvAkC8bcyapHq1+fOHAk+9iBIzRE0fxCZQm16TwgyxPcNeJdfsG1v1Z3DYniFRHO6sZ4B+bZtg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com Extent information can be helpful to the user to coordinate memory usage with the external orchestrator and FM. Expose the details of region extents by creating the following sysfs entries. /sys/bus/cxl/devices/dax_regionX/extentX.Y /sys/bus/cxl/devices/dax_regionX/extentX.Y/offset /sys/bus/cxl/devices/dax_regionX/extentX.Y/length /sys/bus/cxl/devices/dax_regionX/extentX.Y/tag Based on an original patch by Navneet Singh. Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Tested-by: Fan Ni Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase] [iweiny: s/tag/uuid/ throughout the code] [iweiny: update sysfs docs to 2025] --- Documentation/ABI/testing/sysfs-bus-cxl | 36 ++++++++++++++++++++ drivers/cxl/core/extent.c | 58 +++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 2e26d95ac66f..6e9d60baf546 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -639,3 +639,39 @@ Description: The count is persistent across power loss and wraps back to 0 upon overflow. If this file is not present, the device does not have the necessary support for dirty tracking. + + +What: /sys/bus/cxl/devices/dax_regionX/extentX.Y/offset +Date: May, 2025 +KernelVersion: v6.16 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) [For Dynamic Capacity regions only] Users can use the + extent information to create DAX devices on specific extents. + This is done by creating and destroying DAX devices in specific + sequences and looking at the mappings created. Extent offset + within the region. + + +What: /sys/bus/cxl/devices/dax_regionX/extentX.Y/length +Date: May, 2025 +KernelVersion: v6.16 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) [For Dynamic Capacity regions only] Users can use the + extent information to create DAX devices on specific extents. + This is done by creating and destroying DAX devices in specific + sequences and looking at the mappings created. Extent length + within the region. + + +What: /sys/bus/cxl/devices/dax_regionX/extentX.Y/uuid +Date: May, 2025 +KernelVersion: v6.16 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) [For Dynamic Capacity regions only] Users can use the + extent information to create DAX devices on specific extents. + This is done by creating and destroying DAX devices in specific + sequences and looking at the mappings created. UUID of this + extent. diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c index 6df277caf974..3fb20cd7afc8 100644 --- a/drivers/cxl/core/extent.c +++ b/drivers/cxl/core/extent.c @@ -6,6 +6,63 @@ #include "core.h" +static ssize_t offset_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct region_extent *region_extent = to_region_extent(dev); + + return sysfs_emit(buf, "%#llx\n", region_extent->hpa_range.start); +} +static DEVICE_ATTR_RO(offset); + +static ssize_t length_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct region_extent *region_extent = to_region_extent(dev); + u64 length = range_len(®ion_extent->hpa_range); + + return sysfs_emit(buf, "%#llx\n", length); +} +static DEVICE_ATTR_RO(length); + +static ssize_t uuid_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct region_extent *region_extent = to_region_extent(dev); + + return sysfs_emit(buf, "%pUb\n", ®ion_extent->uuid); +} +static DEVICE_ATTR_RO(uuid); + +static struct attribute *region_extent_attrs[] = { + &dev_attr_offset.attr, + &dev_attr_length.attr, + &dev_attr_uuid.attr, + NULL +}; + +static uuid_t empty_uuid = { 0 }; + +static umode_t region_extent_visible(struct kobject *kobj, + struct attribute *a, int n) +{ + struct device *dev = kobj_to_dev(kobj); + struct region_extent *region_extent = to_region_extent(dev); + + if (a == &dev_attr_uuid.attr && + uuid_equal(®ion_extent->uuid, &empty_uuid)) + return 0; + + return a->mode; +} + +static const struct attribute_group region_extent_attribute_group = { + .attrs = region_extent_attrs, + .is_visible = region_extent_visible, +}; + +__ATTRIBUTE_GROUPS(region_extent_attribute); + static void cxled_release_extent(struct cxl_endpoint_decoder *cxled, struct cxled_extent *ed_extent) { @@ -44,6 +101,7 @@ static void region_extent_release(struct device *dev) static const struct device_type region_extent_type = { .name = "extent", .release = region_extent_release, + .groups = region_extent_attribute_groups, }; bool is_region_extent(struct device *dev) From patchwork Sun Apr 13 22:52:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049467 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4C3518EFD4 for ; Sun, 13 Apr 2025 22:52:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584742; cv=fail; b=B+H8JxjM3phy0cRjFku7RXEHBXovY+zpkwodTs3YNYDufXOr5xk7QtARJ1a4RcRzgtsPG7oIZHKUyJ0a5fTFTWKyUx+Wiqke6tfyu9XtweP10Vsf5o3v54ywvSe+4XWyNac6Yx1ZdqSFFIzPMnROeyJbN517u0IxVf6qLZl+G1M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584742; c=relaxed/simple; bh=I3WlmoctCRsRu608KNgUHS/YZw7T8Cy+O3uuocH4P6Q=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=NnqQuJ7ydOisBTGU1qKl68ZPb8JsFJMn0w7wnjplyf9DaxIiJUhxRNfZRqln1yjnFVXcDH7kb5GGOVfZA7Q5hUl+cghA99dJRgrsAa4wIXAf1kmQGzZixcjaOfoGaTPqFrUlIPCOX/1CQrw+4xjbZFMGp/m4WVgkkRXmQDDi80o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mJwfMipa; arc=fail smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mJwfMipa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584740; x=1776120740; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=I3WlmoctCRsRu608KNgUHS/YZw7T8Cy+O3uuocH4P6Q=; b=mJwfMipa3q7jw13GcS53teSkrfjrMzG1PLBEmWJiexMlcvwsPtSXIS4J eRGDZYOv9VkmWV899WikAyP5cEqH/gJUWJSPddflPDQ4J1xNuvVkekXUa yChvGwjuTzx8JXITaj+56XsJlgfETq/9Y6FLeUaQN96VHXuFOmOZULwVu eQsArnFrP/djPP9MPyU5j9OciGXPFrVYyOHw/1mUV2NKKGEOZQJE0zmPS AhwbYCPQwKxvdhyXRih/GHd0CrU5cO5ppcM0MJvp5yaIYgkxFOB314wKF 6WTdnuoQp6L4lxsb9KQBLA3UTuIaOG71BHLev2iFSDo+VD1wbsg0ScBNz A==; X-CSE-ConnectionGUID: HNaoJMSyQ0OjufisRobgdg== X-CSE-MsgGUID: O5cGHQPYTtOnarSPN7EYHA== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="71431128" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="71431128" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:19 -0700 X-CSE-ConnectionGUID: cJFQ5mnFREyj02i9NpsGVQ== X-CSE-MsgGUID: AGMRhiOMTUe88Ic8DWwUOw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="134405567" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa005.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:19 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:18 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:18 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:18 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fpxlzGU5IcG9PLvzOTvNoBqP1EJwYuxViYM5NXhZ2xvTKM+Kujfp5DOoP2Wt/vvIdSYjWSemoBM1xvkU3y5oAsTgxaikjanDp7UQkq2wjNoZauGNgK0xh8jr4royjaHBcaUPCj7aZUjlfOPFyKeTOvIFiZcGbVhpiU156VPLH58tjvi9jVST6i8uwLP4sJKKNeeI2zmBnMjjJ9ACaEt6FHf2mj4qNC+SwMdn9rfLUwLq9kJXfJSwjcRR2vUANnvX9DvM60Rz6F8ItM+iXsn9vB1SG901312qUX+a6b+D3hlULQfi2ryrqgSQzy/fEuw65Ud/xfsNX4h1jr1/TAHJ5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jyAR64KNiC3ycVI/tr64P83Q9uWSXaiB1fBj4L9rC4k=; b=rfxKIEBQeDGGtNpslYzRX8FEuC1Ukp8QvG1wsYyzRidfqn6SBImBGvc2zMdzeDg6wcXxMLrvpcZV+Nf7EYXcXweoh7EAaeGqjd7TbDczey/cdaL30020JZgdysj7dcwhbbIDoR1DxkYC2pVgDP/AqMfedhRU/BiT67kdZ3d6rRU2zcgCGuTCTlniVSZzbgkXk8ch3YcdB05I1of04xw2avXSYsON3o2m632wDh/GgmcnmtmSKCJVW/wIBo1X3l2bRL168wKcnWagHuwK2eXt3+g+J1OLmJH7Hh+M8s8TzhzzZ4T2d/m04yyfEd1sreXH10ndQuOGpO8oYLlRs1BOEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:05 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:05 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:22 -0500 Subject: [PATCH v9 14/19] dax/bus: Factor out dev dax resize logic Message-ID: <20250413-dcd-type2-upstream-v9-14-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=8758; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=I3WlmoctCRsRu608KNgUHS/YZw7T8Cy+O3uuocH4P6Q=; b=SZlCPR6IP2f3V11QT1z1FurtF22u7DLngSx2eSok0vdhz6XAvsfvGjBYrZJyn92y5KcIVMN9C nOQz0m2c+e8CivFJGOP3HPXBytKHHFfFZFUrKgMA+bdddXtBILVoFNJ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: b7297b40-7696-4093-ef03-08dd7addd014 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?T76jdU4mzbM4mL9bzt0hVu1Bl1iGXzf?= =?utf-8?q?+JC5h1Iru16SlGdOjkCW3Wq7XBWu+rRTaPbkcGExr0ZFYpzgDMIIcpkq/cZjNhPc3?= =?utf-8?q?AUpzupOR41eHuswhVayfiAyrsASLCGC+EXQCmk/pB9KrgVA5HFQEhUMFh6Ob2rjFS?= =?utf-8?q?/ACa8a/0rsyZmiRyYrAxPrS1y1xPGkKFXbY9EOw/rRfytSBzdvIfUQ09YDaE2xdPJ?= =?utf-8?q?V2hgCVvopvVupqxh3c3YC+Qe2MkWWv4SJ7psdoMPcDM2nr3Va6IAZ2gaXLBDFeAyq?= =?utf-8?q?Yv8215HsctA0nkBBJALV4Q7hu3fttRJqNV27zcw2lzKSaAsPLy50GZ03EzlcCdUd7?= =?utf-8?q?ofZGb0MISeUTGzpg9YKMEGFHASwnf6x4APzUaITt1vPKySb75Zw3rp7EGh7A171aS?= =?utf-8?q?/NUnuSqGmgevbQZFW9j3+iS0yqoCyb7im1VOWbJQX9PhmYd6Gb25AJGj41UwadGpm?= =?utf-8?q?ItpR8oFvd2dU73+nP3M+ltXF5p56mXCF/4bjFTEfxsGLDpn4JedtiyFpxXySDwzRs?= =?utf-8?q?x2E4tgiRXgPsrbSpEsbOuUzamUHCfrV4yoGp2mJ2eJY6WX3uJEZcATmEbqD93W6CB?= =?utf-8?q?cNfuY30XfadjymPksiTuEKWD9pCaVB/HkrGSD+x1bXhSm/ZyKtkKnoijE7kX7/q0u?= =?utf-8?q?bupvOr71de69uoqouXqwzTGbDx5jgfHDbaBARNe7sIeb5D6XyNSgpG0QNrBlVJOZQ?= =?utf-8?q?DBOmJ1Qt4E8heAsjoUzouryYgdROrC4k4j84fJKl3y8fNTongUJCgzwOfp4OR65mB?= =?utf-8?q?/NVqQmORKWWkzH2YvEBFxnIEkFfE3P6sObEQRL957/n72B9f9JdhunwxG1DeHCdnA?= =?utf-8?q?me64Vaxc97IFa1uhhwEuxFVyqdMTdeZzIiF3obcV9UNXP2SfCkcpgCHTSysBsgkoS?= =?utf-8?q?Vy3k/wJP+NQ1YX/9wOSiu3Aq7ku3jAKuz56M/I4gVqcfwTEuYKR5jPu6H375A/Mts?= =?utf-8?q?JtHhEgShwn0Z1+D4S6FQp4Dr0ycnOQzI6f5D6NWBAjph52vTaqjcaV3umQGnEmkRM?= =?utf-8?q?ksdSCT4lqNq11lCAdrMVVadaRFGkzwGRoCXn11jGkUL6VVVVWjLY+Y+YSzGog16mb?= =?utf-8?q?/qmE/CwaYnEJRNu6YxSdicGkKRvIQ83xujHREszYxwrre2Yz5wCqm1yANBqKgTPq1?= =?utf-8?q?oPVgS2t3zXIws/MEO+1GwWMpj766XYltcOBlyok2T8RINy7O/GCK2ZuGyG7mJjOBD?= =?utf-8?q?HIXm53h3OgZBTP6CB06oYmotURXdyV6ehELfdmLQfw1CLuRRmWrKhD99e00eONQri?= =?utf-8?q?rYy+x1UocKSBUNIzz/FmEWocNsHkw5vSCrZLleLB7nPsYSlVG+8t5E/nGSLCdUIMb?= =?utf-8?q?UjI3J8B1xS7l2KKbo4FMFUgHPffgpKkX8ltdfumrzyzYwYWXY1moiiY=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?iTVoqImL4GP0kwvHnU/7+qM+1jes?= =?utf-8?q?QNNE+ZWWjlUPYTA++N7bUwMRZZCX+fZEO9CTlQEUuZtvbAgOUTJZT2+2Tt9fylMBZ?= =?utf-8?q?edYLz1rcIdLS3shN8BLM4m4R1LdQuiNSGbYlhDFCjloxvr1JJgZfu5jlOV6YX4EO4?= =?utf-8?q?cKwFv6TRytLKzfdJKL5MQnw2yuAoZCfCea8+kTGasxXH//+G2sLLFvVkCdzwdOwCr?= =?utf-8?q?8M+xZEe3Ct5MePezrR5v+SNqI29ihzpEEldeLCGefrODJNI7P+5/v35zJY3c3NW3j?= =?utf-8?q?gLkzxYa0Ix23G4f/TZmwB4dI5XtYttFen863BmGH/x984EIwoeVhcT24M6A78Lb+p?= =?utf-8?q?I9lBYiC4Q90Uo5hGTeAbKCaOX0VjIpYosz2emICm4J65kv2FFaaG9nNiNXMPzM/Ai?= =?utf-8?q?UW6GlZrVrpCpAOdVRNlxiSHt5FARNl0NEy0CMbCVQOshTy/B/I/c/VYxgNoscSphS?= =?utf-8?q?kj4DaRrwIJixHHLWFKsYoW81BGWawIRyRChsoc9qUpgl4+QpanHK75ra5QkXkhoHq?= =?utf-8?q?UOLR3z4OcuUs8VW4aw6sE42++xdOsNIVlTEE1q7FUED/EM4HTMvBuNHRoENs1k6J1?= =?utf-8?q?VJYQR0NCbPTAGXYPMu+hHTov7KbVpkgVoSy5fAnDMrf5d2gNlvQL2a9XExQjXQfnr?= =?utf-8?q?dDMLEC5xZNHR+9d7E88PHOkEEPhZYa6oa07ztE6DzNYLUg0BsCBzyy8nsisfEwlK0?= =?utf-8?q?N59wZWKNseC4JmiFgzuqLpC+Ft6mw9kHvEiTSeqbROSr6oBJCvwncsApPpa6uJ7+n?= =?utf-8?q?V/O3iLA6r1TyBMvlXhKr0TP1STw4VSko3VG8/PF5TKDYeA+E5m6vzIeCCOZDU2aBD?= =?utf-8?q?muGRApd/tCdwyQwNmz/J2yO/buv0BFe/xRvKBKsxmo5k+Mp6tJtHtZ9cChMx43qap?= =?utf-8?q?zqeUlfnbKgi1dJgvGGweHp3ZNPwxO/bARwti+nopUYkO4BkKfBQUpmfmgMbUrXn1X?= =?utf-8?q?VL1s9uuDqHyfT70rQQyVKN9cqBR4nC1w3aUNffILzH5NsZ1EaOwD+sSTFqtqTj0bH?= =?utf-8?q?tnQPFQMjs5v/k/7CyT3aq8vstPk3FMrpS3AKb6YZ3x4ykN4ICIjcC+jBEdK7mozex?= =?utf-8?q?87Q7MHOxlZi59A7Dk8MKdVT2qRI2GQjK9rqHSYeil+AeTmFtkd1eiXP9Sao6GtzgU?= =?utf-8?q?5XgGk5dxLaacsqexTZDJ2774WXw0lpQVthmbQjj884cb4yaMXJIiKeOW+DKScFpqE?= =?utf-8?q?M6DBaVlaEgeQhJybT+1rRkjAr68PGR5P4yyD1UdNrzcN+Zz7gty71c4T8SeAGWGo8?= =?utf-8?q?6kmMaIpZXyG9BXqTYTWZTkIsH/LztrS1S2IiYBALZJ3+CwlvIPTAH6kjE2Mzt6SFZ?= =?utf-8?q?tGxd2PVvfflRBpydTraiHh0M2uGxzYYPkgadcroCPFuu1OB7Q6m/e5w5d5LH6KIcP?= =?utf-8?q?PWxrSbJ+UUPicPJlYCNjtMna/9N8Rfu/p15clwiwhv1gO2Hn0nS58vXn3sT9VOlbU?= =?utf-8?q?1yRiKzGRDKi8oPjSHViMY466juGp8I4mgZeeKqDqMoDXKKilIqWs/Tic0GqFiH7bP?= =?utf-8?q?rC9ZbPbeKT2X?= X-MS-Exchange-CrossTenant-Network-Message-Id: b7297b40-7696-4093-ef03-08dd7addd014 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:05.6382 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C0L1WF7ygDjoN7/gEPVm5cu6/8fIlUS5s8umbZDKjdGk/SLNbMjaYuHtMIviomwCEnSR59SKuTI3IhGTIHJeog== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com Dynamic Capacity regions must limit dev dax resources to those areas which have extents backing real memory. Such DAX regions are dubbed 'sparse' regions. In order to manage where memory is available four alternatives were considered: 1) Create a single region resource child on region creation which reserves the entire region. Then as extents are added punch holes in this reservation. This requires new resource manipulation to punch the holes and still requires an additional iteration over the extent areas which may already have existing dev dax resources used. 2) Maintain an ordered xarray of extents which can be queried while processing the resize logic. The issue is that existing region->res children may artificially limit the allocation size sent to alloc_dev_dax_range(). IE the resource children can't be directly used in the resize logic to find where space in the region is. This also poses a problem of managing the available size in 2 places. 3) Maintain a separate resource tree with extents. This option is the same as 2) but with the different data structure. Most ideally there should be a unified representation of the resource tree not two places to look for space. 4) Create region resource children for each extent. Manage the dax dev resize logic in the same way as before but use a region child (extent) resource as the parents to find space within each extent. Option 4 can leverage the existing resize algorithm to find space within the extents. It manages the available space in a singular resource tree which is less complicated for finding space. In preparation for this change, factor out the dev_dax_resize logic. For static regions use dax_region->res as the parent to find space for the dax ranges. Future patches will use the same algorithm with individual extent resources as the parent. Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Signed-off-by: Ira Weiny --- drivers/dax/bus.c | 130 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 80 insertions(+), 50 deletions(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index d8cb5195a227..c25942a3d125 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -844,11 +844,9 @@ static int devm_register_dax_mapping(struct dev_dax *dev_dax, int range_id) return 0; } -static int alloc_dev_dax_range(struct dev_dax *dev_dax, u64 start, - resource_size_t size) +static int alloc_dev_dax_range(struct resource *parent, struct dev_dax *dev_dax, + u64 start, resource_size_t size) { - struct dax_region *dax_region = dev_dax->region; - struct resource *res = &dax_region->res; struct device *dev = &dev_dax->dev; struct dev_dax_range *ranges; unsigned long pgoff = 0; @@ -866,14 +864,14 @@ static int alloc_dev_dax_range(struct dev_dax *dev_dax, u64 start, return 0; } - alloc = __request_region(res, start, size, dev_name(dev), 0); + alloc = __request_region(parent, start, size, dev_name(dev), 0); if (!alloc) return -ENOMEM; ranges = krealloc(dev_dax->ranges, sizeof(*ranges) * (dev_dax->nr_range + 1), GFP_KERNEL); if (!ranges) { - __release_region(res, alloc->start, resource_size(alloc)); + __release_region(parent, alloc->start, resource_size(alloc)); return -ENOMEM; } @@ -1026,50 +1024,45 @@ static bool adjust_ok(struct dev_dax *dev_dax, struct resource *res) return true; } -static ssize_t dev_dax_resize(struct dax_region *dax_region, - struct dev_dax *dev_dax, resource_size_t size) +/** + * dev_dax_resize_static - Expand the device into the unused portion of the + * region. This may involve adjusting the end of an existing resource, or + * allocating a new resource. + * + * @parent: parent resource to allocate this range in + * @dev_dax: DAX device to be expanded + * @to_alloc: amount of space to alloc; must be <= space available in @parent + * + * Return the amount of space allocated or -ERRNO on failure + */ +static ssize_t dev_dax_resize_static(struct resource *parent, + struct dev_dax *dev_dax, + resource_size_t to_alloc) { - resource_size_t avail = dax_region_avail_size(dax_region), to_alloc; - resource_size_t dev_size = dev_dax_size(dev_dax); - struct resource *region_res = &dax_region->res; - struct device *dev = &dev_dax->dev; struct resource *res, *first; - resource_size_t alloc = 0; int rc; - if (dev->driver) - return -EBUSY; - if (size == dev_size) - return 0; - if (size > dev_size && size - dev_size > avail) - return -ENOSPC; - if (size < dev_size) - return dev_dax_shrink(dev_dax, size); - - to_alloc = size - dev_size; - if (dev_WARN_ONCE(dev, !alloc_is_aligned(dev_dax, to_alloc), - "resize of %pa misaligned\n", &to_alloc)) - return -ENXIO; - - /* - * Expand the device into the unused portion of the region. This - * may involve adjusting the end of an existing resource, or - * allocating a new resource. - */ -retry: - first = region_res->child; - if (!first) - return alloc_dev_dax_range(dev_dax, dax_region->res.start, to_alloc); + first = parent->child; + if (!first) { + rc = alloc_dev_dax_range(parent, dev_dax, + parent->start, to_alloc); + if (rc) + return rc; + return to_alloc; + } - rc = -ENOSPC; for (res = first; res; res = res->sibling) { struct resource *next = res->sibling; + resource_size_t alloc; /* space at the beginning of the region */ - if (res == first && res->start > dax_region->res.start) { - alloc = min(res->start - dax_region->res.start, to_alloc); - rc = alloc_dev_dax_range(dev_dax, dax_region->res.start, alloc); - break; + if (res == first && res->start > parent->start) { + alloc = min(res->start - parent->start, to_alloc); + rc = alloc_dev_dax_range(parent, dev_dax, + parent->start, alloc); + if (rc) + return rc; + return alloc; } alloc = 0; @@ -1078,21 +1071,56 @@ static ssize_t dev_dax_resize(struct dax_region *dax_region, alloc = min(next->start - (res->end + 1), to_alloc); /* space at the end of the region */ - if (!alloc && !next && res->end < region_res->end) - alloc = min(region_res->end - res->end, to_alloc); + if (!alloc && !next && res->end < parent->end) + alloc = min(parent->end - res->end, to_alloc); if (!alloc) continue; if (adjust_ok(dev_dax, res)) { rc = adjust_dev_dax_range(dev_dax, res, resource_size(res) + alloc); - break; + if (rc) + return rc; + return alloc; } - rc = alloc_dev_dax_range(dev_dax, res->end + 1, alloc); - break; + rc = alloc_dev_dax_range(parent, dev_dax, res->end + 1, alloc); + if (rc) + return rc; + return alloc; } - if (rc) - return rc; + + /* available was already calculated and should never be an issue */ + dev_WARN_ONCE(&dev_dax->dev, 1, "space not found?"); + return 0; +} + +static ssize_t dev_dax_resize(struct dax_region *dax_region, + struct dev_dax *dev_dax, resource_size_t size) +{ + resource_size_t avail = dax_region_avail_size(dax_region); + resource_size_t dev_size = dev_dax_size(dev_dax); + struct device *dev = &dev_dax->dev; + resource_size_t to_alloc; + resource_size_t alloc; + + if (dev->driver) + return -EBUSY; + if (size == dev_size) + return 0; + if (size > dev_size && size - dev_size > avail) + return -ENOSPC; + if (size < dev_size) + return dev_dax_shrink(dev_dax, size); + + to_alloc = size - dev_size; + if (dev_WARN_ONCE(dev, !alloc_is_aligned(dev_dax, to_alloc), + "resize of %pa misaligned\n", &to_alloc)) + return -ENXIO; + +retry: + alloc = dev_dax_resize_static(&dax_region->res, dev_dax, to_alloc); + if (alloc <= 0) + return alloc; to_alloc -= alloc; if (to_alloc) goto retry; @@ -1198,7 +1226,8 @@ static ssize_t mapping_store(struct device *dev, struct device_attribute *attr, to_alloc = range_len(&r); if (alloc_is_aligned(dev_dax, to_alloc)) - rc = alloc_dev_dax_range(dev_dax, r.start, to_alloc); + rc = alloc_dev_dax_range(&dax_region->res, dev_dax, r.start, + to_alloc); up_write(&dax_dev_rwsem); up_write(&dax_region_rwsem); @@ -1466,7 +1495,8 @@ static struct dev_dax *__devm_create_dev_dax(struct dev_dax_data *data) device_initialize(dev); dev_set_name(dev, "dax%d.%d", dax_region->id, dev_dax->id); - rc = alloc_dev_dax_range(dev_dax, dax_region->res.start, data->size); + rc = alloc_dev_dax_range(&dax_region->res, dev_dax, dax_region->res.start, + data->size); if (rc) goto err_range; From patchwork Sun Apr 13 22:52:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049472 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC9F41A5B85 for ; Sun, 13 Apr 2025 22:52:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584746; cv=fail; b=Y0m9jXLLXKZJ1yb30AtZf+3+Ln+Roz2bNhs0+hQN3PddUaO+VjgsgXMOVlQgJZW3+rYZ9V1CUBXV406rNvLl/wFWDXhYcl8ObhM5ndG4tPdPFGmae78c/khWd8G/+ecxuS9AKU3DzJ4AQYzQxbyRDZ3oxm4maTraIQxAk2KuG3c= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584746; c=relaxed/simple; bh=YpHLE0FN1k1Z9JdxdTPtJN/f7uJ44Q/rHM2lVf7j/Bk=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=BtJcbrWR6IC5YMCh3kyK+yLCDmiqTHPLprqiC/0TLcGJw301N8oA4CNwuCKf3jgKkiJTlqdiw8jAyB9HKYVK1e68IPBVyyrXNn0u5PoHq/Y8UMsC262BHvLccUIydEdf4mYsBuCkyC7mzDEQZSH7EafTU7k+cErcQ3NV7GW93d0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BNqZYJmd; arc=fail smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BNqZYJmd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584742; x=1776120742; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=YpHLE0FN1k1Z9JdxdTPtJN/f7uJ44Q/rHM2lVf7j/Bk=; b=BNqZYJmdjlvw5ecBlC1WTEwYdcajNJ+QcAmODnmyWzUbY3BY0lKg95iu G4cm4ECtyOq1EHgJoUXsR0F9gRiLraUzkNA2Yy2CLlq7BYZqOJyqBe/ne os0rok+nWwvD/w0+m7Bj8Qat/a2YykBsozfad9zQh2ZV8nKJPI0D0Wc+F WjepRp7HYrW05MNf6dDPtSWDRz7CYLrSrJfosgOECtRted9uPhufUp3iS 8ibFFJhmCTCqDLXyqZJZQ9cp7VkpvY4Vto79NX182nT5By/AhnX/bL8WE L+qjclej/+ElkhUxmGZpNCvgqBdTIla1u6+DEOODSCy5zeFg62RWh1P53 Q==; X-CSE-ConnectionGUID: 9md0sIugRw2S/KUQc6V/Vw== X-CSE-MsgGUID: MiBhGGpjSuurbUuS1sYPcQ== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="71431133" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="71431133" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:20 -0700 X-CSE-ConnectionGUID: 0odosYkAQUWvdJfiPQxaXw== X-CSE-MsgGUID: 8gWCtrdFSfeXaAD5DVL8qg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="134405569" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa005.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:20 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:19 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:19 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:19 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RzEm3Y/tjECJNYGRzuB/WlY6ju5pz21KTxFYaULBhqPqwiTMprdNLsAyN5pwkYcHR1fkHA8MZLOpwb6bGaqr6EGWUlvlQHeoFs7icvCaPBEbCLtMrGySv5h1nWErLAR5Luky67LKunDIVp10P0wuo75hNXvNX77COuh/5mG7MAM9VC2CAHB6UVwBuDmC7bzvYqfluf4pH4n/CRkYFOQjstgpJ/JQmLQaNvjC6cVdu/AcfAH2GVs+0P/sqP2HfJVHumccwRFsGoIoyYiOlN9vqCm2IdB3QcHol+6IT1GVV84R6aGFrLr/y1dQeKduOyxWHmF4e/Vgf1Le/GxoU9YHjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XRcH1hlq1+QlfAeeyAqjDhwx1fs/vkBOfIBaucUZuys=; b=ERFaKyw3KZbxg9zCW2JA5haoUWJGfhBTpeJhNewvY3D6glrQAc2Y2G6IpwbVufPIgVZQ0tXrBUjkE9in3Ygg7Y4uZ6RYKc5CRIak4hc/SqpsnutMXKgJybeOEbJsdjIF4Ec6OKkmk41dwv0ac0GdQypsOX1XT0/mLjiD/MLKEfrnWfHLfv/+xtz4wwP3pB0AHZ/E2Brd/lYO13VarcbJaAj0rglEhVt0wa7DQve+9rLDsdnyN7f9xgSMF3SFF9e4g2SMHeyu+LEtKp0KGvkAWRZrMYIsWgfuK+XaMoUNuEyPjlMP2TUWxAg/Uc2HUSVFIZXqbK//L3SWJBxUd+sZfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:07 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:07 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:23 -0500 Subject: [PATCH v9 15/19] dax/region: Create resources on sparse DAX regions Message-ID: <20250413-dcd-type2-upstream-v9-15-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=29379; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=YpHLE0FN1k1Z9JdxdTPtJN/f7uJ44Q/rHM2lVf7j/Bk=; b=k/yUghpVT/FSxdfIS8LZxy+Q5S8lQQ082S2i3PUvR6yby79FqvoYhhODCbL31YpqeR6F69P/p NJN/Ic5/jt0ALFQ4H47FCt2mVOx/NmEAkUEbv9CYTD4eTn5Fhv90oMa X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: 03dc5a8a-60d2-456a-fd17-08dd7addd11d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?JR8EcoT7kYMVdlFxtwmwDWZjiZqKfiB?= =?utf-8?q?lySw1W+qptvKeOwA9aO95UM2dkKBYxyBjLGJGJpmeaiTSP5ZX7aiRk3QTdLvZOLFA?= =?utf-8?q?G0ReVec5Tt75Hc8yhzbvJcH/b8MVKmxaCQ4vrrTvq1Wqb1OhimKT2IYOHkyq3ufYt?= =?utf-8?q?7ZLxsLrXmb7oeJGfiIrSgV4avQsa+2RtL9Cni6yIDfzNsn4od5e9B6KUOj9tFv4n9?= =?utf-8?q?upiZmqjaJu8wJ0M71s14ZRhO+7ZXafaoFLXTEMd9Uw3lGtq2AvQaragfpbnfyCQDI?= =?utf-8?q?ItHktouYG+R8+80H2azTqYA+s4EHLIWmXS7fUCBd0Ai948bFy/8KW3iXOqg2Y8ziv?= =?utf-8?q?74wucVk/Pw34fmpCtGxlIXbrj5SD7LaQB5gZdMN/woDtvWE+KSyz16U1REpLhFHOB?= =?utf-8?q?/bgRsjidnvO5PoEdoEVvWlszjPnDNf3MNFFuvyZXZDI6gpUoJ8C/iPlwTOf+sbIb0?= =?utf-8?q?51+aMImmhUPn2iHAkZzxOLylI3GMdbawWd1YKCbGiJQSD8oP06maoBBLOJXvJNIVY?= =?utf-8?q?mYp/EHQxbVWDT2BalHq/lVcBCrsQdHJ4yprCOuNgys9pQYCsMo+O3ZND3aXBcUMoe?= =?utf-8?q?kZVq2phYvC7kQBjQUn8FZtXem+bdh5WdC13gD/nTAZ2WzQMwf9Z2bDS0xj/GLjcmO?= =?utf-8?q?Pq9dZwWBxAtjLXo0kWGak6BFYZH8p90pfiq+amfc0dCXMbBDZwMZjSoPECngklTet?= =?utf-8?q?h5RdEmSwm6AVmE47pQ8FpyNsmEo2S/y6Qa3NCNdzdLv+2eSApV3FxYpdYOng7VmWz?= =?utf-8?q?5T6FDZxFpg7KyLtom3TKR7VEec630E8mGWQydd2o/0wSaVZQzMBoSae8KCC6M4FLO?= =?utf-8?q?yTyv21gkjHMYBboIW0VYCXEloi1CEmYNKU9VxgQvb7z9pTEM8iEohnzUynRcHHlfb?= =?utf-8?q?3u4jQnnUjoW8eYBIDvdeGig5dQyIEmO8LrHsHU+EKjP9GBju933XCSQkzyHFXB9dQ?= =?utf-8?q?eke4h+KdB2CIPggCkp1ceNyRbUyi3VkAzqhYjlFk1FC2uOCtrBSYDPgYTjNhVl2uL?= =?utf-8?q?nfqgHZUtx1OXQ2nICrcH8jItvxuIggcxnC37tgJ7J+7hCEIfUz9/L4fTCFjs1tiwy?= =?utf-8?q?sA8aByPGsfsnFlsBxZjlU5rtvZaNe5ZAJet4BIolOO2f//4TFot1O1ZyXDzq320tH?= =?utf-8?q?ZRmVWEwb4cWeLYl9I2jqiwbkYFsUhd5tcBN+HD50g3/j/LBfbqJoal8mx+rjqyCf3?= =?utf-8?q?q9GFMu/+atEQs7Ubl/n1/Rg5193unhi2SFJ+0uqtEfUiPg71jH+AtBpSAfOw96LTK?= =?utf-8?q?BNp8jDiGBrqjwZQ0gFxKkvHILcQHw+rtLG+oa0h6t7hYmqPm19jgQAD0jkKps0bHu?= =?utf-8?q?MkpZVNunuMlXFwA2VVJjX7C30bIZCJRmDFg0k5ypCGglSNBqtwnfZWA=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?ufdWCzXFLDBlz6JQkOPzsZhP5iTj?= =?utf-8?q?3A0TQ6BtVbsckGpq/BXYT8zy6dHY/ucKPZGXhj8rvlzPekrTFee1y43kLwZXoDtij?= =?utf-8?q?V8gHYuyi6hQkchiH8s8zvhio329oLmUTeJCC/FcYsTClf6nW4cx4fTeiboWG8Ak1f?= =?utf-8?q?Ti1+qnPE5KhIh9s/ocHtdwJEXjsT6TuTBARpIY0IzGqbngVcHXJiBIjd5aHAm6pI/?= =?utf-8?q?vwThYDUoOwE04Ijk6tV1KhUqdC1pkLmKo7uPOcismeml/YsqQC9KyLw7UW1aysIm0?= =?utf-8?q?e+PkbUpizGmE4+V4ub5snSwHrQfraTlvReJRIe9DwPVeSe132phXJDBR/NNrPMEUd?= =?utf-8?q?PLaHNK9WxKgaZh6kZbPXeoxfif3sSbZkbA5qB3BO/eVXOPosf5ITrFCGHcl0b5R2O?= =?utf-8?q?o1P+9oQjKKI3mvvQYlWqflD2QV2a0fmvxTM0Pra57bzcOPnQQ18dqQlu6FucgzsZ3?= =?utf-8?q?XLfs/JaiD/N0axCPDbc4iQv2rrFxZr2f5K2QM7oZWNxSmhK0UNwacCFxLCUwcATEs?= =?utf-8?q?8MOZ/KsMHldCIBw7UurnM7U0LwcxDl38SwuwRPMlP43z32TO5zpHKsvJvPi40enOP?= =?utf-8?q?rbX+/JkJIFv6MW+R+ECIHV0EkIXcSwKlr82umceefXlpvv1kXH9SnZMkZsLv++1NQ?= =?utf-8?q?wPjm1NAk5PaB/S8WMfu8Sa2QvCumKHU8oAD0PAylX3F/i7qjeca2u7MWtjbwW8h84?= =?utf-8?q?gz1Xz6eiSaeVfHweMJKDpclgOByKwpay493B99T8z9rQS6MfdE8P+xj3IJKn8Kr9d?= =?utf-8?q?MZ3wFHT7kHezo8hVhve863+FZQxvsSFfv79NSoYuiiNyQ2TIjgjoryyRPVrr1KbRU?= =?utf-8?q?tJhX6tKo7Vfp88CuoSjIAD9RTvdkcrYK5624ecoJw9ijRD0CN7/bBIBMWOujbnKXt?= =?utf-8?q?wsXW3YJh5RGKnNEjEITcS4lBSBm+HcK2GSKU1XXjF4r7orpsK4o+urnpG4G8xlYL5?= =?utf-8?q?yspApWhc8xlI9/tmhX7GerxXJHtGcTNJ0OcOP5MYcqn/ckDLFZ1hsnjYFKmiWurDi?= =?utf-8?q?oZadG6KD4hINDSSCE9fSJklePksNOOhdgRAE6FYTd5xJR03wjdTFfaBn+4G7sLXxG?= =?utf-8?q?N/XqdyHH0XjNgBQM9piiOAik7L62tXz40UlbmHap4zz9F1IrYGO4TAYCvm+AjXzyX?= =?utf-8?q?/x8RQlua6n/wdLX4eNg4V5Vzd1BkfbZQ9OBX1OhiGBwQngNNbpXowPy1JUWtiPX+F?= =?utf-8?q?uJZmG1MiOvnOATZ2084AsKcpTad+cxNUkxOl50aPl+OWNiPB5aM5ZpWEe5qVK0wNe?= =?utf-8?q?bjnFNOqttcc27EtlPDAqefw7A2Eqe9pZuKZMeaW9V3lLsMFHKauyGZoWA853cnfQX?= =?utf-8?q?+bjNVxcESUc/zwAYLa5rxDalxQ73qCl1diaLDTYdefPc0WpiztdwBRLFOaAC8IRW5?= =?utf-8?q?OVkutaGUrhewUB7+1HPY4U6CSl+TS2s88Hjy7UsrCg9WzQEKbdIKJcXjnlMqlCPBl?= =?utf-8?q?V9mzXkP8ecg8y9/EzCT45xGhLjs7m/eIAbN37c1XqDz038PEPUmEFZVkZ5jUZjrG/?= =?utf-8?q?VGRaA7eD1sbU?= X-MS-Exchange-CrossTenant-Network-Message-Id: 03dc5a8a-60d2-456a-fd17-08dd7addd11d X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:07.5027 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xJzjVKews76ExOKtU9a0AfPekF38jO09MaY+ooT8PYsbYmqWvjWPYRvIqdMvoEeam3wLPonrY22my9e3MHeXxA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com DAX regions which map dynamic capacity partitions require that memory be allowed to come and go. Recall sparse regions were created for this purpose. Now that extents can be realized within DAX regions the DAX region driver can start tracking sub-resource information. The tight relationship between DAX region operations and extent operations require memory changes to be controlled synchronously with the user of the region. Synchronize through the dax_region_rwsem and by having the region driver drive both the region device as well as the extent sub-devices. Recall requests to remove extents can happen at any time and that a host is not obligated to release the memory until it is not being used. If an extent is not used allow a release response. When extents are eligible for release. No mappings exist but data may reside in caches not yet written to the device. Call cxl_region_invalidate_memregion() to write back data to the device prior to signaling the release complete. Speculative writes after a release may dirty the cache such that a read from a newly surfaced extent may not come from the device. Call cxl_region_invalidate_memregion() prior to bringing a new extent online to ensure the cache is marked invalid. While these invalidate calls are inefficient they are the best we can do to ensure cache consistency without back invalidate. Furthermore this should occur infrequently with sufficiently large extents that real work loads should not be impacted much. The DAX layer has no need for the details of the CXL memory extent devices. Expose extents to the DAX layer as device children of the DAX region device. A single callback from the driver aids the DAX layer to determine if the child device is an extent. The DAX layer also registers a devres function to automatically clean up when the device is removed from the region. There is a race between extents being surfaced and the dax_cxl driver being loaded. Synchronizes the driver during probe by scanning for existing extents while under the device lock. Respond to extent notifications. Manage the DAX region resource tree based on the extents lifetime. Return the status of remove notifications to lower layers such that it can manage the hardware appropriately. Based on an original patch by Navneet Singh. Reviewed-by: Jonathan Cameron Signed-off-by: Ira Weiny --- Changes: [iweiny: convert range prints to %pra] --- drivers/cxl/core/core.h | 2 + drivers/cxl/core/extent.c | 83 ++++++++++++++-- drivers/cxl/core/region.c | 2 +- drivers/cxl/cxl.h | 6 ++ drivers/dax/bus.c | 246 +++++++++++++++++++++++++++++++++++++++++----- drivers/dax/bus.h | 3 +- drivers/dax/cxl.c | 61 +++++++++++- drivers/dax/dax-private.h | 40 ++++++++ drivers/dax/hmem/hmem.c | 2 +- drivers/dax/pmem.c | 2 +- include/linux/ioport.h | 3 + 11 files changed, 411 insertions(+), 39 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 1272be497926..027dd1504d77 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -22,6 +22,8 @@ cxled_to_mds(struct cxl_endpoint_decoder *cxled) return container_of(cxlds, struct cxl_memdev_state, cxlds); } +int cxl_region_invalidate_memregion(struct cxl_region *cxlr); + #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; extern struct device_attribute dev_attr_create_ram_region; diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c index 3fb20cd7afc8..4dc0dec486f6 100644 --- a/drivers/cxl/core/extent.c +++ b/drivers/cxl/core/extent.c @@ -116,6 +116,12 @@ static void region_extent_unregister(void *ext) dev_dbg(®ion_extent->dev, "DAX region rm extent HPA %pra\n", ®ion_extent->hpa_range); + /* + * Extent is not in use or an error has occur. No mappings + * exist at this point. Write and invalidate caches to ensure + * the device has all data prior to final release. + */ + cxl_region_invalidate_memregion(region_extent->cxlr_dax->cxlr); device_unregister(®ion_extent->dev); } @@ -269,20 +275,65 @@ static void calc_hpa_range(struct cxl_endpoint_decoder *cxled, hpa_range->end = hpa_range->start + range_len(dpa_range) - 1; } +static int cxlr_notify_extent(struct cxl_region *cxlr, enum dc_event event, + struct region_extent *region_extent) +{ + struct device *dev = &cxlr->cxlr_dax->dev; + struct cxl_notify_data notify_data; + struct cxl_driver *driver; + + dev_dbg(dev, "Trying notify: type %d HPA %pra\n", event, + ®ion_extent->hpa_range); + + guard(device)(dev); + + /* + * The lack of a driver indicates a notification has failed. No user + * space coordination was possible. + */ + if (!dev->driver) + return 0; + driver = to_cxl_drv(dev->driver); + if (!driver->notify) + return 0; + + notify_data = (struct cxl_notify_data) { + .event = event, + .region_extent = region_extent, + }; + + dev_dbg(dev, "Notify: type %d HPA %pra\n", event, + ®ion_extent->hpa_range); + return driver->notify(dev, ¬ify_data); +} + +struct rm_data { + struct cxl_region *cxlr; + struct range *range; +}; + static int cxlr_rm_extent(struct device *dev, void *data) { struct region_extent *region_extent = to_region_extent(dev); - struct range *region_hpa_range = data; + struct rm_data *rm_data = data; + int rc; if (!region_extent) return 0; /* - * Any extent which 'touches' the released range is removed. + * Any extent which 'touches' the released range is attempted to be + * removed. */ - if (range_overlaps(region_hpa_range, ®ion_extent->hpa_range)) { + if (range_overlaps(rm_data->range, ®ion_extent->hpa_range)) { + struct cxl_region *cxlr = rm_data->cxlr; + dev_dbg(dev, "Remove region extent HPA %pra\n", ®ion_extent->hpa_range); + rc = cxlr_notify_extent(cxlr, DCD_RELEASE_CAPACITY, region_extent); + if (rc == -EBUSY) + return 0; + region_rm_extent(region_extent); } return 0; @@ -327,8 +378,13 @@ int cxl_rm_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent) calc_hpa_range(cxled, cxlr->cxlr_dax, &dpa_range, &hpa_range); + struct rm_data rm_data = { + .cxlr = cxlr, + .range = &hpa_range, + }; + /* Remove region extents which overlap */ - return device_for_each_child(&cxlr->cxlr_dax->dev, &hpa_range, + return device_for_each_child(&cxlr->cxlr_dax->dev, &rm_data, cxlr_rm_extent); } @@ -353,8 +409,23 @@ static int cxlr_add_extent(struct cxl_dax_region *cxlr_dax, return rc; } - /* device model handles freeing region_extent */ - return online_region_extent(region_extent); + /* Ensure caches are clean prior onlining */ + cxl_region_invalidate_memregion(cxlr_dax->cxlr); + + rc = online_region_extent(region_extent); + /* device model handled freeing region_extent */ + if (rc) + return rc; + + rc = cxlr_notify_extent(cxlr_dax->cxlr, DCD_ADD_CAPACITY, region_extent); + /* + * The region device was briefly live but DAX layer ensures it was not + * used + */ + if (rc) + region_rm_extent(region_extent); + + return rc; } /* Callers are expected to ensure cxled has been attached to a region */ diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 3106df6f3636..eeabc5a6b18a 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -223,7 +223,7 @@ static struct cxl_region_ref *cxl_rr_load(struct cxl_port *port, return xa_load(&port->regions, (unsigned long)cxlr); } -static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) +int cxl_region_invalidate_memregion(struct cxl_region *cxlr) { if (!cpu_cache_has_invalidate_memregion()) { if (IS_ENABLED(CONFIG_CXL_REGION_INVALIDATION_TEST)) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index d027432b1572..a14b33eca1d0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -867,10 +867,16 @@ bool is_cxl_region(struct device *dev); extern struct bus_type cxl_bus_type; +struct cxl_notify_data { + enum dc_event event; + struct region_extent *region_extent; +}; + struct cxl_driver { const char *name; int (*probe)(struct device *dev); void (*remove)(struct device *dev); + int (*notify)(struct device *dev, struct cxl_notify_data *notify_data); struct device_driver drv; int id; }; diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index c25942a3d125..45573d077b5a 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -183,6 +183,93 @@ static bool is_sparse(struct dax_region *dax_region) return (dax_region->res.flags & IORESOURCE_DAX_SPARSE_CAP) != 0; } +static void __dax_release_resource(struct dax_resource *dax_resource) +{ + struct dax_region *dax_region = dax_resource->region; + + lockdep_assert_held_write(&dax_region_rwsem); + dev_dbg(dax_region->dev, "Extent release resource %pr\n", + dax_resource->res); + if (dax_resource->res) + __release_region(&dax_region->res, dax_resource->res->start, + resource_size(dax_resource->res)); + dax_resource->res = NULL; +} + +static void dax_release_resource(void *res) +{ + struct dax_resource *dax_resource = res; + + guard(rwsem_write)(&dax_region_rwsem); + __dax_release_resource(dax_resource); + kfree(dax_resource); +} + +int dax_region_add_resource(struct dax_region *dax_region, + struct device *device, + resource_size_t start, resource_size_t length) +{ + struct resource *new_resource; + int rc; + + struct dax_resource *dax_resource __free(kfree) = + kzalloc(sizeof(*dax_resource), GFP_KERNEL); + if (!dax_resource) + return -ENOMEM; + + guard(rwsem_write)(&dax_region_rwsem); + + dev_dbg(dax_region->dev, "DAX region resource %pr\n", &dax_region->res); + new_resource = __request_region(&dax_region->res, start, length, "extent", 0); + if (!new_resource) { + dev_err(dax_region->dev, "Failed to add region s:%pa l:%pa\n", + &start, &length); + return -ENOSPC; + } + + dev_dbg(dax_region->dev, "add resource %pr\n", new_resource); + dax_resource->region = dax_region; + dax_resource->res = new_resource; + + /* + * open code devm_add_action_or_reset() to avoid recursive write lock + * of dax_region_rwsem in the error case. + */ + rc = devm_add_action(device, dax_release_resource, dax_resource); + if (rc) { + __dax_release_resource(dax_resource); + return rc; + } + + dev_set_drvdata(device, no_free_ptr(dax_resource)); + return 0; +} +EXPORT_SYMBOL_GPL(dax_region_add_resource); + +int dax_region_rm_resource(struct dax_region *dax_region, + struct device *dev) +{ + struct dax_resource *dax_resource; + + guard(rwsem_write)(&dax_region_rwsem); + + dax_resource = dev_get_drvdata(dev); + if (!dax_resource) + return 0; + + if (dax_resource->use_cnt) + return -EBUSY; + + /* + * release the resource under dax_region_rwsem to avoid races with + * users trying to use the extent + */ + __dax_release_resource(dax_resource); + dev_set_drvdata(dev, NULL); + return 0; +} +EXPORT_SYMBOL_GPL(dax_region_rm_resource); + bool static_dev_dax(struct dev_dax *dev_dax) { return is_static(dev_dax->region); @@ -296,19 +383,41 @@ static ssize_t region_align_show(struct device *dev, static struct device_attribute dev_attr_region_align = __ATTR(align, 0400, region_align_show, NULL); +resource_size_t +dax_avail_size(struct resource *dax_resource) +{ + resource_size_t rc; + struct resource *used_res; + + rc = resource_size(dax_resource); + for_each_child_resource(dax_resource, used_res) + rc -= resource_size(used_res); + return rc; +} +EXPORT_SYMBOL_GPL(dax_avail_size); + #define for_each_dax_region_resource(dax_region, res) \ for (res = (dax_region)->res.child; res; res = res->sibling) static unsigned long long dax_region_avail_size(struct dax_region *dax_region) { - resource_size_t size = resource_size(&dax_region->res); + resource_size_t size; struct resource *res; lockdep_assert_held(&dax_region_rwsem); - if (is_sparse(dax_region)) - return 0; + if (is_sparse(dax_region)) { + /* + * Children of a sparse region represent available space not + * used space. + */ + size = 0; + for_each_dax_region_resource(dax_region, res) + size += dax_avail_size(res); + return size; + } + size = resource_size(&dax_region->res); for_each_dax_region_resource(dax_region, res) size -= resource_size(res); return size; @@ -449,15 +558,26 @@ EXPORT_SYMBOL_GPL(kill_dev_dax); static void trim_dev_dax_range(struct dev_dax *dev_dax) { int i = dev_dax->nr_range - 1; - struct range *range = &dev_dax->ranges[i].range; + struct dev_dax_range *dev_range = &dev_dax->ranges[i]; + struct range *range = &dev_range->range; struct dax_region *dax_region = dev_dax->region; + struct resource *res = &dax_region->res; lockdep_assert_held_write(&dax_region_rwsem); dev_dbg(&dev_dax->dev, "delete range[%d]: %#llx:%#llx\n", i, (unsigned long long)range->start, (unsigned long long)range->end); - __release_region(&dax_region->res, range->start, range_len(range)); + if (dev_range->dax_resource) { + res = dev_range->dax_resource->res; + dev_dbg(&dev_dax->dev, "Trim sparse extent %pr\n", res); + } + + __release_region(res, range->start, range_len(range)); + + if (dev_range->dax_resource) + dev_range->dax_resource->use_cnt--; + if (--dev_dax->nr_range == 0) { kfree(dev_dax->ranges); dev_dax->ranges = NULL; @@ -640,7 +760,7 @@ static void dax_region_unregister(void *region) struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, - unsigned long flags) + unsigned long flags, struct dax_sparse_ops *sparse_ops) { struct dax_region *dax_region; @@ -658,12 +778,16 @@ struct dax_region *alloc_dax_region(struct device *parent, int region_id, || !IS_ALIGNED(range_len(range), align)) return NULL; + if (!sparse_ops && (flags & IORESOURCE_DAX_SPARSE_CAP)) + return NULL; + dax_region = kzalloc(sizeof(*dax_region), GFP_KERNEL); if (!dax_region) return NULL; dev_set_drvdata(parent, dax_region); kref_init(&dax_region->kref); + dax_region->sparse_ops = sparse_ops; dax_region->id = region_id; dax_region->align = align; dax_region->dev = parent; @@ -845,7 +969,8 @@ static int devm_register_dax_mapping(struct dev_dax *dev_dax, int range_id) } static int alloc_dev_dax_range(struct resource *parent, struct dev_dax *dev_dax, - u64 start, resource_size_t size) + u64 start, resource_size_t size, + struct dax_resource *dax_resource) { struct device *dev = &dev_dax->dev; struct dev_dax_range *ranges; @@ -884,6 +1009,7 @@ static int alloc_dev_dax_range(struct resource *parent, struct dev_dax *dev_dax, .start = alloc->start, .end = alloc->end, }, + .dax_resource = dax_resource, }; dev_dbg(dev, "alloc range[%d]: %pa:%pa\n", dev_dax->nr_range - 1, @@ -966,7 +1092,8 @@ static int dev_dax_shrink(struct dev_dax *dev_dax, resource_size_t size) int i; for (i = dev_dax->nr_range - 1; i >= 0; i--) { - struct range *range = &dev_dax->ranges[i].range; + struct dev_dax_range *dev_range = &dev_dax->ranges[i]; + struct range *range = &dev_range->range; struct dax_mapping *mapping = dev_dax->ranges[i].mapping; struct resource *adjust = NULL, *res; resource_size_t shrink; @@ -982,12 +1109,21 @@ static int dev_dax_shrink(struct dev_dax *dev_dax, resource_size_t size) continue; } - for_each_dax_region_resource(dax_region, res) - if (strcmp(res->name, dev_name(dev)) == 0 - && res->start == range->start) { - adjust = res; - break; - } + if (dev_range->dax_resource) { + for_each_child_resource(dev_range->dax_resource->res, res) + if (strcmp(res->name, dev_name(dev)) == 0 + && res->start == range->start) { + adjust = res; + break; + } + } else { + for_each_dax_region_resource(dax_region, res) + if (strcmp(res->name, dev_name(dev)) == 0 + && res->start == range->start) { + adjust = res; + break; + } + } if (dev_WARN_ONCE(dev, !adjust || i != dev_dax->nr_range - 1, "failed to find matching resource\n")) @@ -1025,19 +1161,21 @@ static bool adjust_ok(struct dev_dax *dev_dax, struct resource *res) } /** - * dev_dax_resize_static - Expand the device into the unused portion of the - * region. This may involve adjusting the end of an existing resource, or - * allocating a new resource. + * __dev_dax_resize - Expand the device into the unused portion of the region. + * This may involve adjusting the end of an existing resource, or allocating a + * new resource. * * @parent: parent resource to allocate this range in * @dev_dax: DAX device to be expanded * @to_alloc: amount of space to alloc; must be <= space available in @parent + * @dax_resource: if sparse; the parent resource * * Return the amount of space allocated or -ERRNO on failure */ -static ssize_t dev_dax_resize_static(struct resource *parent, - struct dev_dax *dev_dax, - resource_size_t to_alloc) +static ssize_t __dev_dax_resize(struct resource *parent, + struct dev_dax *dev_dax, + resource_size_t to_alloc, + struct dax_resource *dax_resource) { struct resource *res, *first; int rc; @@ -1045,7 +1183,8 @@ static ssize_t dev_dax_resize_static(struct resource *parent, first = parent->child; if (!first) { rc = alloc_dev_dax_range(parent, dev_dax, - parent->start, to_alloc); + parent->start, to_alloc, + dax_resource); if (rc) return rc; return to_alloc; @@ -1059,7 +1198,8 @@ static ssize_t dev_dax_resize_static(struct resource *parent, if (res == first && res->start > parent->start) { alloc = min(res->start - parent->start, to_alloc); rc = alloc_dev_dax_range(parent, dev_dax, - parent->start, alloc); + parent->start, alloc, + dax_resource); if (rc) return rc; return alloc; @@ -1083,7 +1223,8 @@ static ssize_t dev_dax_resize_static(struct resource *parent, return rc; return alloc; } - rc = alloc_dev_dax_range(parent, dev_dax, res->end + 1, alloc); + rc = alloc_dev_dax_range(parent, dev_dax, res->end + 1, alloc, + dax_resource); if (rc) return rc; return alloc; @@ -1094,6 +1235,51 @@ static ssize_t dev_dax_resize_static(struct resource *parent, return 0; } +static ssize_t dev_dax_resize_static(struct dax_region *dax_region, + struct dev_dax *dev_dax, + resource_size_t to_alloc) +{ + return __dev_dax_resize(&dax_region->res, dev_dax, to_alloc, NULL); +} + +static int find_free_extent(struct device *dev, const void *data) +{ + const struct dax_region *dax_region = data; + struct dax_resource *dax_resource; + + if (!dax_region->sparse_ops->is_extent(dev)) + return 0; + + dax_resource = dev_get_drvdata(dev); + if (!dax_resource || !dax_avail_size(dax_resource->res)) + return 0; + return 1; +} + +static ssize_t dev_dax_resize_sparse(struct dax_region *dax_region, + struct dev_dax *dev_dax, + resource_size_t to_alloc) +{ + struct dax_resource *dax_resource; + ssize_t alloc; + + struct device *extent_dev __free(put_device) = + device_find_child(dax_region->dev, dax_region, + find_free_extent); + if (!extent_dev) + return 0; + + dax_resource = dev_get_drvdata(extent_dev); + if (!dax_resource) + return 0; + + to_alloc = min(dax_avail_size(dax_resource->res), to_alloc); + alloc = __dev_dax_resize(dax_resource->res, dev_dax, to_alloc, dax_resource); + if (alloc > 0) + dax_resource->use_cnt++; + return alloc; +} + static ssize_t dev_dax_resize(struct dax_region *dax_region, struct dev_dax *dev_dax, resource_size_t size) { @@ -1118,7 +1304,10 @@ static ssize_t dev_dax_resize(struct dax_region *dax_region, return -ENXIO; retry: - alloc = dev_dax_resize_static(&dax_region->res, dev_dax, to_alloc); + if (is_sparse(dax_region)) + alloc = dev_dax_resize_sparse(dax_region, dev_dax, to_alloc); + else + alloc = dev_dax_resize_static(dax_region, dev_dax, to_alloc); if (alloc <= 0) return alloc; to_alloc -= alloc; @@ -1227,7 +1416,7 @@ static ssize_t mapping_store(struct device *dev, struct device_attribute *attr, to_alloc = range_len(&r); if (alloc_is_aligned(dev_dax, to_alloc)) rc = alloc_dev_dax_range(&dax_region->res, dev_dax, r.start, - to_alloc); + to_alloc, NULL); up_write(&dax_dev_rwsem); up_write(&dax_region_rwsem); @@ -1466,6 +1655,11 @@ static struct dev_dax *__devm_create_dev_dax(struct dev_dax_data *data) struct device *dev; int rc; + if (is_sparse(dax_region) && data->size) { + dev_err(parent, "Sparse DAX region devices must be created initially with 0 size"); + return ERR_PTR(-EINVAL); + } + dev_dax = kzalloc(sizeof(*dev_dax), GFP_KERNEL); if (!dev_dax) return ERR_PTR(-ENOMEM); @@ -1496,7 +1690,7 @@ static struct dev_dax *__devm_create_dev_dax(struct dev_dax_data *data) dev_set_name(dev, "dax%d.%d", dax_region->id, dev_dax->id); rc = alloc_dev_dax_range(&dax_region->res, dev_dax, dax_region->res.start, - data->size); + data->size, NULL); if (rc) goto err_range; diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index 783bfeef42cc..ae5029ea6047 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -9,6 +9,7 @@ struct dev_dax; struct resource; struct dax_device; struct dax_region; +struct dax_sparse_ops; /* dax bus specific ioresource flags */ #define IORESOURCE_DAX_STATIC BIT(0) @@ -17,7 +18,7 @@ struct dax_region; struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, - unsigned long flags); + unsigned long flags, struct dax_sparse_ops *sparse_ops); struct dev_dax_data { struct dax_region *dax_region; diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c index 88b051cea755..011bd1dc7691 100644 --- a/drivers/dax/cxl.c +++ b/drivers/dax/cxl.c @@ -5,6 +5,57 @@ #include "../cxl/cxl.h" #include "bus.h" +#include "dax-private.h" + +static int __cxl_dax_add_resource(struct dax_region *dax_region, + struct region_extent *region_extent) +{ + struct device *dev = ®ion_extent->dev; + resource_size_t start, length; + + start = dax_region->res.start + region_extent->hpa_range.start; + length = range_len(®ion_extent->hpa_range); + return dax_region_add_resource(dax_region, dev, start, length); +} + +static int cxl_dax_add_resource(struct device *dev, void *data) +{ + struct dax_region *dax_region = data; + struct region_extent *region_extent; + + region_extent = to_region_extent(dev); + if (!region_extent) + return 0; + + dev_dbg(dax_region->dev, "Adding resource HPA %pra\n", + ®ion_extent->hpa_range); + + return __cxl_dax_add_resource(dax_region, region_extent); +} + +static int cxl_dax_region_notify(struct device *dev, + struct cxl_notify_data *notify_data) +{ + struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + struct dax_region *dax_region = dev_get_drvdata(dev); + struct region_extent *region_extent = notify_data->region_extent; + + switch (notify_data->event) { + case DCD_ADD_CAPACITY: + return __cxl_dax_add_resource(dax_region, region_extent); + case DCD_RELEASE_CAPACITY: + return dax_region_rm_resource(dax_region, ®ion_extent->dev); + case DCD_FORCED_CAPACITY_RELEASE: + default: + dev_err(&cxlr_dax->dev, "Unknown DC event %d\n", + notify_data->event); + return -ENXIO; + } +} + +struct dax_sparse_ops sparse_ops = { + .is_extent = is_region_extent, +}; static int cxl_dax_region_probe(struct device *dev) { @@ -24,15 +75,18 @@ static int cxl_dax_region_probe(struct device *dev) flags |= IORESOURCE_DAX_SPARSE_CAP; dax_region = alloc_dax_region(dev, cxlr->id, &cxlr_dax->hpa_range, nid, - PMD_SIZE, flags); + PMD_SIZE, flags, &sparse_ops); if (!dax_region) return -ENOMEM; - if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A) + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A) { + device_for_each_child(&cxlr_dax->dev, dax_region, + cxl_dax_add_resource); /* Add empty seed dax device */ dev_size = 0; - else + } else { dev_size = range_len(&cxlr_dax->hpa_range); + } data = (struct dev_dax_data) { .dax_region = dax_region, @@ -47,6 +101,7 @@ static int cxl_dax_region_probe(struct device *dev) static struct cxl_driver cxl_dax_region_driver = { .name = "cxl_dax_region", .probe = cxl_dax_region_probe, + .notify = cxl_dax_region_notify, .id = CXL_DEVICE_DAX_REGION, .drv = { .suppress_bind_attrs = true, diff --git a/drivers/dax/dax-private.h b/drivers/dax/dax-private.h index 0867115aeef2..39fb587561f8 100644 --- a/drivers/dax/dax-private.h +++ b/drivers/dax/dax-private.h @@ -16,6 +16,14 @@ struct inode *dax_inode(struct dax_device *dax_dev); int dax_bus_init(void); void dax_bus_exit(void); +/** + * struct dax_sparse_ops - Operations for sparse regions + * @is_extent: return if the device is an extent + */ +struct dax_sparse_ops { + bool (*is_extent)(struct device *dev); +}; + /** * struct dax_region - mapping infrastructure for dax devices * @id: kernel-wide unique region for a memory range @@ -27,6 +35,7 @@ void dax_bus_exit(void); * @res: resource tree to track instance allocations * @seed: allow userspace to find the first unbound seed device * @youngest: allow userspace to find the most recently created device + * @sparse_ops: operations required for sparse regions */ struct dax_region { int id; @@ -38,6 +47,7 @@ struct dax_region { struct resource res; struct device *seed; struct device *youngest; + struct dax_sparse_ops *sparse_ops; }; /** @@ -57,11 +67,13 @@ struct dax_mapping { * @pgoff: page offset * @range: resource-span * @mapping: reference to the dax_mapping for this range + * @dax_resource: if not NULL; dax sparse resource containing this range */ struct dev_dax_range { unsigned long pgoff; struct range range; struct dax_mapping *mapping; + struct dax_resource *dax_resource; }; /** @@ -100,6 +112,34 @@ struct dev_dax { */ void run_dax(struct dax_device *dax_dev); +/** + * struct dax_resource - For sparse regions; an active resource + * @region: dax_region this resources is in + * @res: resource + * @use_cnt: count the number of uses of this resource + * + * Changes to the dax_region and the dax_resources within it are protected by + * dax_region_rwsem + * + * dax_resource's are not intended to be used outside the dax layer. + */ +struct dax_resource { + struct dax_region *region; + struct resource *res; + unsigned int use_cnt; +}; + +/* + * Similar to run_dax() dax_region_{add,rm}_resource() and dax_avail_size() are + * exported but are not intended to be generic operations outside the dax + * subsystem. They are only generic between the dax layer and the dax drivers. + */ +int dax_region_add_resource(struct dax_region *dax_region, struct device *dev, + resource_size_t start, resource_size_t length); +int dax_region_rm_resource(struct dax_region *dax_region, + struct device *dev); +resource_size_t dax_avail_size(struct resource *dax_resource); + static inline struct dev_dax *to_dev_dax(struct device *dev) { return container_of(dev, struct dev_dax, dev); diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5e7c53f18491..0eea65052874 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -28,7 +28,7 @@ static int dax_hmem_probe(struct platform_device *pdev) mri = dev->platform_data; dax_region = alloc_dax_region(dev, pdev->id, &mri->range, - mri->target_node, PMD_SIZE, flags); + mri->target_node, PMD_SIZE, flags, NULL); if (!dax_region) return -ENOMEM; diff --git a/drivers/dax/pmem.c b/drivers/dax/pmem.c index c8ebf4e281f2..f927e855f240 100644 --- a/drivers/dax/pmem.c +++ b/drivers/dax/pmem.c @@ -54,7 +54,7 @@ static struct dev_dax *__dax_pmem_probe(struct device *dev) range.start += offset; dax_region = alloc_dax_region(dev, region_id, &range, nd_region->target_node, le32_to_cpu(pfn_sb->align), - IORESOURCE_DAX_STATIC); + IORESOURCE_DAX_STATIC, NULL); if (!dax_region) return ERR_PTR(-ENOMEM); diff --git a/include/linux/ioport.h b/include/linux/ioport.h index e8b2d6aa4013..a97bb3d936a7 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -27,6 +27,9 @@ struct resource { struct resource *parent, *sibling, *child; }; +#define for_each_child_resource(parent, res) \ + for (res = (parent)->child; res; res = res->sibling) + /* * IO resources have these defined flags. * From patchwork Sun Apr 13 22:52:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049470 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A13FA1C860E for ; Sun, 13 Apr 2025 22:52:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584744; cv=fail; b=RJ2eMktrICYDlYK75IJN+IZfLbDsbI7rJ78cvZ1pGDO37/+BNNGzdCTKi6H2gYxgi4qk47x5wNDyMblkdxTRF4/7M8IHLpTST67tzHIONX8S6ZghWWyHpd9nM8EtSQHFgrBKaoWVyAK/nDIGzxr7OUwaFAmAHTjOWE8rRF1BCbM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584744; c=relaxed/simple; bh=dbpnUt0Rj32jPB+ZTtMa+10bARM2SLtI6QbpS09VtA4=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=LI9YBL0Xm3gnjr+TB2kH2BN8cQ1aIoo/k6wZ1xc07rD0Ww1F4H/wpIEvKZIPNG4hSKPDFU4hoDtcmQvRKcAjVYWo3Ar9C1BfuEfeFPYyEIjCFlvjZm/59yHaTQ1/mwyRcFocxk0VfQ1np/Pke1XN4yv3SuojJ0fbN5qN5sybmdE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=GJYc0G8V; arc=fail smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="GJYc0G8V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584742; x=1776120742; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=dbpnUt0Rj32jPB+ZTtMa+10bARM2SLtI6QbpS09VtA4=; b=GJYc0G8Vn31qDmzTRX1gjW3/oRORbCrDpA8SxIRLdcUtweUqCGJ2qW2d YbuwOtCTxINDVnRebqZldpk+XmlrW2YOXRnCwI2OMCMO+BU7H/XEeTxDI m4K1vdLwd3cGqIXXiryF/+eySdtWJgbB4sPigbMSdgFT5vNJz6gJ5V4Il ld5HTXZooyL+2PCtUWmfeU6ywkLgqip8ahDRpJSm6cC3n+KoDx9TZvYwo tDdiCpMiFb6sl6jLK2n+tni1gAkoVpmIkCNoJlyo3KBBaI/d4DACiPw9F BuWlz12YCGHEyx2iFk7hzhYysQzbNJE2gCWrPB40az1EGFnqAHUamqAoc g==; X-CSE-ConnectionGUID: d2L/evoRQnKASEo0IMTwbw== X-CSE-MsgGUID: G1ru5EZeQ3iEzIF/3y+9Cg== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="71431138" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="71431138" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:20 -0700 X-CSE-ConnectionGUID: QGOhIkznRJiD28s1okWoEA== X-CSE-MsgGUID: f8EH6ZhjSjGuw8Fi6920eA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="134405570" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa005.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:21 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:20 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:20 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:19 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LdHooYh1JpyfT8w0EiR6oyCluqFXITWnC20fw1Ol+/8kLpwP7TsFAe4zAVsiLvbGOc/oor+Xi8FWNgnVmjEDHJ+mEg0D3ziYKlohjMiwUZqB+cjZnW3qdRkfEaJZhyuXp0SyJsD/e1eJjeWHts7X8lq4vlPpUWTQNuFDA0ETns0PF4i9lmZf+ut04SbrSALvwiGSsNjsuV5htkAyr5cDVQOaImwgJyvQQUqQnbHsGBR4HntlAVGCQlmQEkiqpRNX7wCkhn1EpD6z6kjf+JGRq7cOLwTDJhwgHBweG2Bquz8IwTQzN0VPILQs/lOZuCUIU6O0HLB37Dne0c4emKHyoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wNoAbBBYNS2LHiqzwRXuIEpkmfzCc69lB2/JkZ8R22g=; b=NekB3uZiYnJh9pjEKc4rnOaiVonv5oyAT1bQSCafD++ZLOxBAiZgCHqA5/9uSsDpX57QBASXnHFHW59YB2yww435NrBD7RWtux4n58dpVNeCW6ncaON9XMfRURpTsN6/MqzWesmOOmjUrLyCX+j0heDl6owPwhX0G3ZB0orolfJc20R8TjdSVLSwaglvWyCG+3d3mho54Qh3kfDYIRZDaNPZs+SU5MjhG6hFBiuCsnQnFOU7ocD7ABAcWEaxuFVYUzn14V6mQ9ISHlaN+DRGaLfZFZ9pYN/PYmc7UIDKDaA9Vtb+Sqit7g7Vkg1e0zFhxVrUQjMnLOGuSg6j1FyETQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:09 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:09 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:24 -0500 Subject: [PATCH v9 16/19] cxl/region: Read existing extents on region creation Message-ID: <20250413-dcd-type2-upstream-v9-16-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=8457; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=dbpnUt0Rj32jPB+ZTtMa+10bARM2SLtI6QbpS09VtA4=; b=70LGskBUS72k14zEP1Njc047eMZ4tEAL0gJfSp2jak5WU2AlIXmrrUYBAsAkY9W4FefcUPLLg 2SacYrWNvLpAk1H9F7j78oUDfgSc1+GfOT/QkEwP4WDjJWW7AndDprC X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: e84c113a-2d07-40ec-c33e-08dd7addd229 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?CkrqC6JZM1YO9OZ13kmA+i+qJE3aPtY?= =?utf-8?q?B6Q8ji8VqN5/xMXd7JqJ4W94gXR31a8JRlZjqfKfbpPIr8dIkKJc3jMWzfro7Rdc9?= =?utf-8?q?LcU6Xd0N5h8VI1znqpiN6rrebqCcGz7xDh1j2+vwsP7tHNnTWH8vNH8zy3RVJGpBg?= =?utf-8?q?55ylPpHiJmGyQcIuQwz4+hqz3lro6DfBRh2su8BJXNuMufpLW2l2Gk7xVLT1Jpyod?= =?utf-8?q?HEIsH2NZBzSGzYljZvBIQp14m50/0OML6kr6WkrCpIEe78HbfxcG+E9rZFJVsTZ8F?= =?utf-8?q?AaCqHRpAoKlugTwKD9K4LXPXYXBe1g4dDr+k6hodkNbTgRKwHhhSDYzd3Qz5pBj5T?= =?utf-8?q?enddYbYg8T6gzlWYvonp1wO+jJpFv28M//oE1+UYkdaIVSDvBCG89veG6c1DrJbiM?= =?utf-8?q?xPyxENEx6zOz0cCHqetP8ywhTdnLUU7zhdNsf6i/eXw/PcayeegkDKatIh43xM+j9?= =?utf-8?q?9BakSACA/rsdfZDgeVuup9smqkZUdMOQuIu2uJqo4ZEMlgjhy6tq1WHVHz0/ZGzNy?= =?utf-8?q?GCyyD+77caIpeDxe0NWHmalEVhSl4FpawEW+uE4+NRXbiStpdybbnTxtm9k8mM1uO?= =?utf-8?q?mR+svSM/pR0hyGwaFaAJkvxd/0BZbHgVBWF0c0/iSPIp931IdA5YxwfHEs/GcGIgs?= =?utf-8?q?Arno8ibWTLsWDAioqevFmS8HqwXUsU8ofgjmNIMGxXup4Z5IJviQH6KvTkbaWxqKD?= =?utf-8?q?gggyD5D5W9in/3O3ALMpr8hXpQFcHwf6IDZtsNnc3sKvFRITIv9Tj9ZEJFBsz2Mxh?= =?utf-8?q?biopTNsFCYXgLhfL5FXem2i0u7+E/SpyJC47j8iBuSvthXHNpasQzui1ixeJFyWfp?= =?utf-8?q?o/6+1UvrJmZQk9YcvRyXINO9Um52cBmqzhTRE9O+8oSDfteCsgCIBGHtNH4QZw82b?= =?utf-8?q?OWNz+IOPWlKY4YDiV0XHybXatHgakMOqdAJ6Ov8EtyQVvrIBHfWlw9zeQ0Y+sCvoW?= =?utf-8?q?hEcbnm7EUGNTclZr9npxg3fAMNSUNdXwJd9hP41fJ0GOl1p+GNoiCMc7lqSF8PU4t?= =?utf-8?q?EhDIsJyr+kmyCfLNEfAncxsMijNI8rqef/EduN1ZEUOnyq4Lfyxpqp/5WZNT/iJtz?= =?utf-8?q?8oD+6sCrwpAt2oD8s4bFuBU+9j95zUE7E0T2sZ6nPfNDNlzW2TxHCRgU0MPgf+Qfo?= =?utf-8?q?CNjlnOB6jN1T1P7RPq0/0voHAK8Uiq8ddNiGAd7GzGJtbQAcl2/sobcmGD0CNHCmK?= =?utf-8?q?7URqUML4rd/tACr6XyUjj4WrE7P4mvwckMK5u5o8SXPypi8NGC9rJ7CcR/w6aT/xE?= =?utf-8?q?JpfhGyi8I1rCT/96w1TVFto9fmZrxN+40db2y2jomsaCfGlhkkZEKkFyPYTKC0c01?= =?utf-8?q?V5YcitTv4KDx0QjO8Vf+xO+8DmARIEfuGg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?7hIqZqJST+U92Y9n4i44fjjtYwE9?= =?utf-8?q?a25t8IBR24xdbgoh4vBn5H8TG7lm7xPMizg3a09DgbvLEtMmZjcuhyjgGoTLJcG6+?= =?utf-8?q?DhWMegVcZ7wmg3Y0beH84IIx5ezkM5Fz5vXFYmeHZcoepUZI4zmvpZxKHaWb/YVRp?= =?utf-8?q?F2BIBacB5s0iuG1qmEbG3IaYYqEt20P9XZVQSKtVizbJ/hgkDx3pI8mJ/jPY7BXqE?= =?utf-8?q?jrqxLlUqtNLR6rTXevCz6nhwq+ioGRKdu0BMm6NHTfiMHn0WbYa6Vjhq71p9lo2nZ?= =?utf-8?q?vWYFQTbZ+9bWspA7C0w2sGu2B1yXdSRlsd60IvELPYPnVqyYw05smTy0pbsstLaAR?= =?utf-8?q?60XMQInVvPo13CPGngbdlg0vQbfg3Z2Zsqt5qtU84cndDYkAlfUi73NRiYeEK450R?= =?utf-8?q?PnkjOJ2pVxdKJr3wwB3+dTfozTShl3eqAgMfhcrFcGPXfZcK+zcJm4pB0BM0Bv3Xs?= =?utf-8?q?GDPTrVkdsJRkICCFI2MPe9IZFxWK/UPROhEGKlZc7e/moafHgHR3giTJJi1JzKBEt?= =?utf-8?q?IYNBF41H73X2FCoThpI/Iof2OLTaYlS8VcjRd+UyAaf+7rPqmZWJI5cPmOrsERdby?= =?utf-8?q?rQPdAdMjm1upUmOMrqFIlHNkpSvv49A9hqv4gQUBp28gn7ZXiOIGRAVd9b/iwXbJZ?= =?utf-8?q?bi+a2YzwErOd+nl2UTzUGvAoPKeTYflbJ2UGDTRtc05IrpsXwopo4guYq4LwTAc04?= =?utf-8?q?1vsvKITOngqn2K1TvYUrHmugDfnNVFlMLFoH16fX6MknnHky4aERl5sl/MjKhUkud?= =?utf-8?q?RalSViwjBL6iEag0k8tvc9/b4pzVHxwBzNwIJ/4BifTtPeOU4MWFVRf4fxr3daH09?= =?utf-8?q?DFk55QVJ157MBcSoHibw9U0ePaOyZJBwaiX7qnuASQzc8JDnwWHZiR+Zl45lT6QF7?= =?utf-8?q?fFmbbFAe+pGcPHeeWErCaoI8dOln8clpLj61NM+7Ec9z9RKjt/n67YgNxKqEX8ITv?= =?utf-8?q?3iCjDoQKYc1rYSODrHKMwrCqWOfyrZE6y2BDIG3M0QjIyC2HP6+ZsXzAI7vX1djV2?= =?utf-8?q?buZod5HhAnhdZwNMOOzynn7oBl7NGUF2/fLljqIiOpLo9+qOamDo49phcwUHWSiUN?= =?utf-8?q?7R5xiwQbu2DxGo5Oa2aVwZ9nseVx4+axdBd8HfoqtPhdDnLjomKu7GFoGl5jnazkC?= =?utf-8?q?r/R5xdBuE2THWOlIsjO/Eqx8av5807U0L9a1eQGWz8pEojvNudjbJCvIeQy0dZVwo?= =?utf-8?q?fjRH+E6lXNqmbGnJHU77PSILmsPtK4NtlRKPmye4CN8vek+Uo53OxI3C85hw9360v?= =?utf-8?q?kEcBkLC97Zpp9z7oXxmXnYK8i0Cu50ZnsrfDp3nzTvUGCZCZtnSivIf56OESrFO1S?= =?utf-8?q?detQDPt3QbB8UiGIhXwhSFxapCk1S4srKIWTUmog9/GgFikeq1+rJbx7Il7nqoZvN?= =?utf-8?q?/4Y6HAmScDjJmUzyVzBUTaCM29Yp+xHFaorw/udMQmmTuDRzpNbxebK86KpkuSM6w?= =?utf-8?q?NzBHk6YYFsRzfUY20Y6tkm5PiWfP7BKHlUbHI52nm1gYFFt771aRGG3yzjp9l7NTb?= =?utf-8?q?u4W4dwrcC0vd?= X-MS-Exchange-CrossTenant-Network-Message-Id: e84c113a-2d07-40ec-c33e-08dd7addd229 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:09.2099 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9SwpEwJbiZKm8gFKf9HBl0LfA8NclD6Fy4yMdD2qe3q9xKLlG5XoOFZLiWfH5SW3ehEDhp2R8jquq+CHGahIhA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com Dynamic capacity device extents may be left in an accepted state on a device due to an unexpected host crash. In this case it is expected that the creation of a new region on top of a DC partition can read those extents and surface them for continued use. Once all endpoint decoders are part of a region and the region is being realized, a read of the 'devices extent list' can reveal these previously accepted extents. CXL r3.1 specifies the mailbox call Get Dynamic Capacity Extent List for this purpose. The call returns all the extents for all dynamic capacity partitions. If the fabric manager is adding extents to any DCD partition, the extent list for the recovered region may change. In this case the query must retry. Upon retry the query could encounter extents which were accepted on a previous list query. Adding such extents is ignored without error because they are entirely within a previous accepted extent. Instead warn on this case to allow for differentiating bad devices from this normal condition. Latch any errors to be bubbled up to ensure notification to the user even if individual errors are rate limited or otherwise ignored. The scan for existing extents races with the dax_cxl driver. This is synchronized through the region device lock. Extents which are found after the driver has loaded will surface through the normal notification path while extents seen prior to the driver are read during driver load. Based on an original patch by Navneet Singh. Reviewed-by: Jonathan Cameron Reviewed-by: Fan Ni Signed-off-by: Ira Weiny --- Changes: [0day: fix extent count in GetExtent input payload] [iweiny: minor clean ups] [iweiny: Adjust for partition arch] --- drivers/cxl/core/core.h | 1 + drivers/cxl/core/mbox.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/region.c | 25 +++++++++++ drivers/cxl/cxlmem.h | 21 +++++++++ 4 files changed, 156 insertions(+) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 027dd1504d77..e06a46fec217 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -22,6 +22,7 @@ cxled_to_mds(struct cxl_endpoint_decoder *cxled) return container_of(cxlds, struct cxl_memdev_state, cxlds); } +int cxl_process_extent_list(struct cxl_endpoint_decoder *cxled); int cxl_region_invalidate_memregion(struct cxl_region *cxlr); #ifdef CONFIG_CXL_REGION diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index de01c6684530..8af3a4173b99 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1737,6 +1737,115 @@ int cxl_dev_dc_identify(struct cxl_mailbox *mbox, } EXPORT_SYMBOL_NS_GPL(cxl_dev_dc_identify, "CXL"); +/* Return -EAGAIN if the extent list changes while reading */ +static int __cxl_process_extent_list(struct cxl_endpoint_decoder *cxled) +{ + u32 current_index, total_read, total_expected, initial_gen_num; + struct cxl_memdev_state *mds = cxled_to_mds(cxled); + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + struct device *dev = mds->cxlds.dev; + struct cxl_mbox_cmd mbox_cmd; + u32 max_extent_count; + int latched_rc = 0; + bool first = true; + + struct cxl_mbox_get_extent_out *extents __free(kvfree) = + kvmalloc(cxl_mbox->payload_size, GFP_KERNEL); + if (!extents) + return -ENOMEM; + + total_read = 0; + current_index = 0; + total_expected = 0; + max_extent_count = (cxl_mbox->payload_size - sizeof(*extents)) / + sizeof(struct cxl_extent); + do { + u32 nr_returned, current_total, current_gen_num; + struct cxl_mbox_get_extent_in get_extent; + int rc; + + get_extent = (struct cxl_mbox_get_extent_in) { + .extent_cnt = cpu_to_le32(max(max_extent_count, + total_expected - current_index)), + .start_extent_index = cpu_to_le32(current_index), + }; + + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_DC_EXTENT_LIST, + .payload_in = &get_extent, + .size_in = sizeof(get_extent), + .size_out = cxl_mbox->payload_size, + .payload_out = extents, + .min_out = 1, + }; + + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + /* Save initial data */ + if (first) { + total_expected = le32_to_cpu(extents->total_extent_count); + initial_gen_num = le32_to_cpu(extents->generation_num); + first = false; + } + + nr_returned = le32_to_cpu(extents->returned_extent_count); + total_read += nr_returned; + current_total = le32_to_cpu(extents->total_extent_count); + current_gen_num = le32_to_cpu(extents->generation_num); + + dev_dbg(dev, "Got extent list %d-%d of %d generation Num:%d\n", + current_index, total_read - 1, current_total, current_gen_num); + + if (current_gen_num != initial_gen_num || total_expected != current_total) { + dev_warn(dev, "Extent list change detected; gen %u != %u : cnt %u != %u\n", + current_gen_num, initial_gen_num, + total_expected, current_total); + return -EAGAIN; + } + + for (int i = 0; i < nr_returned ; i++) { + struct cxl_extent *extent = &extents->extent[i]; + + dev_dbg(dev, "Processing extent %d/%d\n", + current_index + i, total_expected); + + rc = validate_add_extent(mds, extent); + if (rc) + latched_rc = rc; + } + + current_index += nr_returned; + } while (total_expected > total_read); + + return latched_rc; +} + +#define CXL_READ_EXTENT_LIST_RETRY 10 + +/** + * cxl_process_extent_list() - Read existing extents + * @cxled: Endpoint decoder which is part of a region + * + * Issue the Get Dynamic Capacity Extent List command to the device + * and add existing extents if found. + * + * A retry of 10 is somewhat arbitrary, however, extent changes should be + * relatively rare while bringing up a region. So 10 should be plenty. + */ +int cxl_process_extent_list(struct cxl_endpoint_decoder *cxled) +{ + int retry = CXL_READ_EXTENT_LIST_RETRY; + int rc; + + do { + rc = __cxl_process_extent_list(cxled); + } while (rc == -EAGAIN && retry--); + + return rc; +} + static void add_part(struct cxl_dpa_info *info, u64 start, u64 size, enum cxl_partition_mode mode) { int i = info->nr_partitions; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index eeabc5a6b18a..a43b43972bae 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3196,6 +3196,26 @@ static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) return rc; } +static int cxlr_add_existing_extents(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + int i, latched_rc = 0; + + for (i = 0; i < p->nr_targets; i++) { + struct device *dev = &p->targets[i]->cxld.dev; + int rc; + + rc = cxl_process_extent_list(p->targets[i]); + if (rc) { + dev_err(dev, "Existing extent processing failed %d\n", + rc); + latched_rc = rc; + } + } + + return latched_rc; +} + static void cxlr_dax_unregister(void *_cxlr_dax) { struct cxl_dax_region *cxlr_dax = _cxlr_dax; @@ -3231,6 +3251,11 @@ static int devm_cxl_add_dax_region(struct cxl_region *cxlr) dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent), dev_name(dev)); + if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM_A) + if (cxlr_add_existing_extents(cxlr)) + dev_err(&cxlr->dev, "Existing extent processing failed %d\n", + rc); + return devm_add_action_or_reset(&cxlr->dev, cxlr_dax_unregister, cxlr_dax); err: diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 63a38e449454..f80f70549c0b 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -600,6 +600,27 @@ struct cxl_mbox_dc_response { } __packed extent_list[]; } __packed; +/* + * Get Dynamic Capacity Extent List; Input Payload + * CXL rev 3.1 section 8.2.9.9.9.2; Table 8-166 + */ +struct cxl_mbox_get_extent_in { + __le32 extent_cnt; + __le32 start_extent_index; +} __packed; + +/* + * Get Dynamic Capacity Extent List; Output Payload + * CXL rev 3.1 section 8.2.9.9.9.2; Table 8-167 + */ +struct cxl_mbox_get_extent_out { + __le32 returned_extent_count; + __le32 total_extent_count; + __le32 generation_num; + u8 rsvd[4]; + struct cxl_extent extent[]; +} __packed; + struct cxl_mbox_get_supported_logs { __le16 entries; u8 rsvd[6]; From patchwork Sun Apr 13 22:52:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049471 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 864CD1E3761 for ; Sun, 13 Apr 2025 22:52:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584745; cv=fail; b=nUT3sI2zIQ5w0aTqB+4zFloFsZATGtzNKdkw+0kVp+itpfX5WgBtkYDIQI+/VjrZoMIAo9PktoklP8tU4Ggx53YVHIzoXMO9qtfE2un3KM+ldJ47gA6IdHM+8mzpPajtd/lZp+WJ0TdUg/g8llVJRXH5X6Y00Fq8A09AGedPrEA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584745; c=relaxed/simple; bh=aPQ2lOD5FZgte//V4Fn7ZiP0+wnZl/4oLYzAK/mFmrY=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=KxEi5AoIWmhvWfB8KM8x/MdqJ6P2uBRdtE3Er3l9gf+S7tfgGqqbRj55o8ZZ6Tiqas5n2/mLWJW9f56Ud4IKYOSYHr1r7CuDlGsmpoXFxzOmE412GWUjp+eeDxfQ1+/t/OUT/KlZsIe4syOrugdOBOQSR0ZZeApAh814g7kHV1A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OGPcYGN1; arc=fail smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OGPcYGN1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584743; x=1776120743; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=aPQ2lOD5FZgte//V4Fn7ZiP0+wnZl/4oLYzAK/mFmrY=; b=OGPcYGN1ZfX43HBqsj0kV4l5G8fvY8MrQ3fDWrFJcPIDQqjJjj4vcA6n VkUkn+4flZaYS54fdytrKVwDrt9LgQWk2iRYN9/WnaZsFJJUjGYgrs13T Xo53VsWEI1ak38U74NHs8QikeVg0BEK0c3UpQl/wRuOJBU+HZ+62SFcaE ayv7eTeZhhr0v5BYYCTyKp+bMxjAKvRQ0kqo91no1OqDffoPRbg0NpPOb GsU//aEsfhhb1v+8MzU+9KgZlZCpNuSsU5DTMXmDjTw2zCuBXtUdJFdVt 5HeuPaQNP5yiFh8tjIWfJdXsY2EXPmxqMzx2FnDNcl9Df62eZEmNTdqbS Q==; X-CSE-ConnectionGUID: WqJPtTuaSTi4NISi8TF3bQ== X-CSE-MsgGUID: X22zmqQeSXSYwPOYtOO/Sw== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="71431143" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="71431143" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:21 -0700 X-CSE-ConnectionGUID: c9bybS/FR/+oSZj1Jn00Cg== X-CSE-MsgGUID: blq2GIj7Tvu9Lxji5nAteQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="134405572" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa005.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:21 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:20 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:20 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nwMAU3Zff3b9XMbFxV9gmsB/9QZqvSLb6+/cVEVEHXNY48TLL4FPq7Vl7kHW/MqnTJ9VIaAjAqPcN+54++FxwdudZrOuhsGNF4xiq4hUXC8N1ENJ7FtpJ8df7rbZQGeJaXWmxDYiuwqjUXdo+bqSpcG34hZHoXqZWROMDmOk1PthNb0XfLwGDok7vS3nM6K3G6TazD4drkfyfKf2GdPwhWO2oSBWD+E6fE7PwvSlGqgtk34Mbl6WyuRkaJ8uczdDay1lAqo6V0/cPzg6weFIJ+IwnEUgVSi5LWFSkCfkhYZSZO2J7QwN0N5irkuguG5HgUT7DzYMH4mmFgg03RWx2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KFSfKXdaijHdUR2EX8XUxITwaBLjdOOX4vtm8g6BPfA=; b=mI3Z47dOseu8TcHkGqt/6zxoMq4AlhCfMKtva4VMXc4VuPkplkwYeXOXwiMYY9+AleGNkz0dXIY6KJBBfDEOWBbt/PEhqrnXy2DepwhnXvt/UvotvFN764jLBVOCM/YbC8uuy4nK2o+RaPw6EGawLZQPICQmO1JHtwhMYExDqKvEADWSkXPTNxJKUc8BJ6vYxzbrL6y+Zorj9kPGuasJ2H7q3k+k4AuRiNkpgCqTt9R5yjpaKFfIHaRLW9G3OoS015NWWFzv6FBZq8kkyc2Xkr/GCa/bHaUnRlvVlJniNRZA5kfRWUDnTGYy1xD5NnKZeVxdzZ1m1r6v/ZAB0u4ZRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:10 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:10 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:25 -0500 Subject: [PATCH v9 17/19] cxl/mem: Trace Dynamic capacity Event Record Message-ID: <20250413-dcd-type2-upstream-v9-17-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=3493; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=aPQ2lOD5FZgte//V4Fn7ZiP0+wnZl/4oLYzAK/mFmrY=; b=k8Fdmu2rZNvNMh7qXVZD4E1W4tdqS8wxeoYskhYO5Th92GdtLTQFn2XPde1mfzC8hCpOtvPp2 dQ//+GOxzbPDtNEqeP1qRDNxewxFwkDEJKt5QDpqPhDWSYOs9UnQhM/ X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: a4449e40-e3a4-4dbb-a319-08dd7addd321 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?3ACfaxpy7I7DizOJLlTlriFes4A8T/Q?= =?utf-8?q?8aJ4kwb3rJ9aigGE1fmi4LDYPZ24fzh5EFBnzx3YwuLLvRYPUYPRAHkcU2RbOW1oj?= =?utf-8?q?Sq2ILah8Ex7+M3LZ7ztBJs7fNe/3X/IWp8PiSSPqlB9tGzJaHIRJLb9moI/+li7vl?= =?utf-8?q?acegg6GoCerxkNVGeuUavNYPLHsdMCxjFh9jrmQ3JVaBG7BA5T22H5+9CQXyRpqcj?= =?utf-8?q?wokk3fAp7CYrmou1cnSW0IK/01mKAqCvs+yhK9GBe3We/AoZUH917bFiP80NARhaz?= =?utf-8?q?ijwtsSjrMLyhYXD+wVTnyorvaWuP9VpH6uw4nQIGUZ9K8uPMK1q5LHvz5HsiiDMEC?= =?utf-8?q?gQgCJ37rGLYiy6FCRgg7A+u+M5EfAx0ZwufxzYLwtsB+bd3Kr5Vj1vTwnKO49duEI?= =?utf-8?q?UpAEh5thSw2DrIUrlFMDlJemPT+b3udL3cKTOPhT1Dmsy4XdH+FwS7RYpxBipetJm?= =?utf-8?q?EhbjMvyGt3pNYkCAmPniINB0tHJSMpu6a+fBnStqP1djjP7awNwIpPLF13JNUhnmr?= =?utf-8?q?mBsYAuSHoGrxEIqs1kn8Gi1MkbqtGZ242h5ozvNzpCR4uXbCA58b++WZgMyoci3go?= =?utf-8?q?L8dmUQ40wAOekyF7xdF56Al/qer5Xw1HFMLIU0YXAsewCJik5S5YgPrZ9M5dybJ2J?= =?utf-8?q?0LYzHImnX+ExuAa6PrVCH9RY3ryrxaGrVp7fMdkBREd1kgLmeFibWoiCliPZL779a?= =?utf-8?q?ov/b0iWz59Hei/BEhFASc0JOzvFD4YscQrzEvRs79mUIm+ai7KsU9KU+rKTPub2g0?= =?utf-8?q?hFjgqFaIY2cUrZkS/jtwxUUzPgR3s+4LuYxEtfFqWpQ0UPZcNxlolprhymRPhbkVk?= =?utf-8?q?tJaT5KPCPz3HFnAiBp2IIBbRZ1HVU9nDNAiZNXujTYCRDYJa5cHoRZLmdjVjCpXt0?= =?utf-8?q?AkKaFeVIDkrqHm++qt/Yf3xmFEuar47AicuXzynpZhwsKoBIsPJ8rSzomP2EoThIK?= =?utf-8?q?Jjq1MKGNyObwSaV2th+icU8xn/HHaGzuVd4yhBNdP/7iSzp4fIkV8E+JjPD06DsE0?= =?utf-8?q?1E/bLI9GbLau6Px0omv2byvmMAuClQRO87RhPMLKR5zoeiyQdtpb50/1tXy/TMBFp?= =?utf-8?q?t0sZeGJooAcZX3521ZYPIP3PYlEnT5+Dq+gjPoGRIo3pljU6p5wXieILSMDBOtDJu?= =?utf-8?q?EiRnMHlvOisJQ4u2VwToV2xqI0Q0V/kMHSu+wUCl/1o2tPvdJW3qFeKnxFNrN0m9k?= =?utf-8?q?2Lb8Go4E7UNsIgUAxZ98/KiZVVascvW4xVxv3UY3UTQJF6UqZ47h56bdnbb9zoiCl?= =?utf-8?q?U4IITk/4/IpJ3+rXgLY+tCW33XHCt8RgsOcYdExVrn9bUgmK4C9Q8RlLwiDc1djqH?= =?utf-8?q?ldjLiVlOd3wJHuh02A738kx4J2Qu9xZGtozFBFRrQtw+tPJjRp4DXhk=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?wSMNgEqZaehX7vW40zrVl8/OyjDg?= =?utf-8?q?qxfoPIFed1jhanHeSDc8csCTvtLF8tQ3x44MK8eDvInlOQz1W+LV9EJxPE8tTmbTW?= =?utf-8?q?NB8C2Sopit5WT7/GaAbXmUAwdD9sKXvRB3+PgWdiZl2ya05ynbI40Tu9tymqg5v+T?= =?utf-8?q?E4ntTuTeXYXPnlMrZoaPC3Azu1ATUGco06fmNOQcxUSgFrq+Q1A1ZzutfDjHf6KcI?= =?utf-8?q?qpm0TLIPrqaQ3asUFlYOVNgamxnU73u7tQ/LgzIZ/DVcrS5UqSayF+SFoRN/eKBm+?= =?utf-8?q?fpjfSIoL/9Ng4/Y3Al/vbiErRMYpXQqxoVuW7pWZEfPgOTP9VWOOuv7htrVcs0ivR?= =?utf-8?q?Hlrb5SoCdvU7FfNNlndEMVbBt599NnGdxwvBo9CxPHeu3Vgj/Svrgj5N/A44eODHW?= =?utf-8?q?jDeCCqn3r88trSB1WLupvje7hkIPoajMTL8p7fS8FsVMpyg2Dmeu6V4kivBJwP+Jk?= =?utf-8?q?hqrsVnmBi6HQeo0jBFZ0bgHi2yBtn+GrDCLAMFgYDvWo1P3kQ14ADyZJClBoswGA3?= =?utf-8?q?Z4/BE6pgw4d216X3NOEtewGAziH/MjoXTQBBxUs32HXMqEnJ43iCrTFHCTd3tr8bG?= =?utf-8?q?QUXYvOCsj+0YLhdWFHZBGo3nMnNpvsLIZLskhVtAANWK9EdgTQxwxWWbIm+ffugQO?= =?utf-8?q?MAd5rmd2ibEzkHZ3PC9QN9A2/wLyivpjTU80rzWLiGwXEeO0ZXEiAK/jw12Jdnp+8?= =?utf-8?q?8/5HjrQo0mm5DVYIvWiHd4dBV3qcvNdvheGc5yOkbAz9v2LpymKdsd27u3qtIy9Hd?= =?utf-8?q?9SFJ7jHO4JfBTnW9UyczYYO2UNuWgNgrNpiQ4Gg7AXoh+qlme//8RV/K0iPJgpr5T?= =?utf-8?q?AvFynVYMNb+wyABdti0vrRsJ3Ms1VQSGos5UVPCVtHGcbS6nUcuUuR43PGuqHHwHp?= =?utf-8?q?r050s/jeh4vYVDepj21nx9hjujETijze5lV3X6BWckFtO2a0XHKXvNtMRK/0qSbSM?= =?utf-8?q?Jkhfkuyun5ogBEh/OFR6OEUVN2e+NZCs4sPuh2Lax6ufD6o7vbyfTQB5XYiILOwYZ?= =?utf-8?q?oQMg7a6cGirTuot2iWtkRutc5mJyE92V9oR3bBJxfAEvdKHuo0x7ZWkKbxtDPDW5b?= =?utf-8?q?PJZ8dzfhq9RvfN9NSkNI66DA4/DfsWK67+1jeSZcpBvHrMbXU3Jwk6uzIfBeQwkq8?= =?utf-8?q?nrZjWSCKjSeq0NTcPWCjNkgwuJ5YBjNvjsseG4gVceeMoEKglOsz9uN0eKrnTe83+?= =?utf-8?q?R6uA1y8U5P4xHjD3eAcHkBVKTv9o2GnFhYYwacai5M6ejqcfgox6mWJoq1gIq8CS2?= =?utf-8?q?WAOB8NfReOGLKCno8CpWGqRSJi438jOq0KWf0SXVnxvCRL+zqTth0M45O+6YTJYax?= =?utf-8?q?ifN4/3oGaI2S5GT76ZSPbeChEXvERl9dWoROYAoRSKbkfPZ9PWTQ7mV9Cdh3xp3Vn?= =?utf-8?q?wooy5jC4JJ64WlFE3jwvl8TRjhJNAVIzfdvOyZyaTZVXLexH/Rp/Vq/aLgGUtD5ac?= =?utf-8?q?vdIuBa7daUcaynBAeEJnTdNkQwQjB5On8pOjCITEDJYxFGIKItd2ypu8Nv3yRas5g?= =?utf-8?q?Tdb0F2e5HE9E?= X-MS-Exchange-CrossTenant-Network-Message-Id: a4449e40-e3a4-4dbb-a319-08dd7addd321 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:10.7614 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qYTw3yC1BBPZV9xkSDLOy1gc62cTbg0L5P/i9KJA+gQn7KVhGaW6Au9Cud8zJQSwsqkRKl0m61LU+um7OauvJg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com CXL rev 3.1 section 8.2.9.2.1 adds the Dynamic Capacity Event Records. User space can use trace events for debugging of DC capacity changes. Add DC trace points to the trace log. Based on an original patch by Navneet Singh. Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Fan Ni Signed-off-by: Ira Weiny --- Changes: [djbw: s/region/partition/] [iweiny: s/tag/uuid/] --- drivers/cxl/core/mbox.c | 4 +++ drivers/cxl/core/trace.h | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 8af3a4173b99..891a213ce7be 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1043,6 +1043,10 @@ static void __cxl_event_trace_record(const struct cxl_memdev *cxlmd, ev_type = CXL_CPER_EVENT_DRAM; else if (uuid_equal(uuid, &CXL_EVENT_MEM_MODULE_UUID)) ev_type = CXL_CPER_EVENT_MEM_MODULE; + else if (uuid_equal(uuid, &CXL_EVENT_DC_EVENT_UUID)) { + trace_cxl_dynamic_capacity(cxlmd, type, &record->event.dcd); + return; + } cxl_event_trace_record(cxlmd, type, ev_type, uuid, &record->event); } diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index 25ebfbc1616c..384017259970 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -978,6 +978,71 @@ TRACE_EVENT(cxl_poison, ) ); +/* + * Dynamic Capacity Event Record - DER + * + * CXL rev 3.1 section 8.2.9.2.1.6 Table 8-50 + */ + +#define CXL_DC_ADD_CAPACITY 0x00 +#define CXL_DC_REL_CAPACITY 0x01 +#define CXL_DC_FORCED_REL_CAPACITY 0x02 +#define CXL_DC_REG_CONF_UPDATED 0x03 +#define show_dc_evt_type(type) __print_symbolic(type, \ + { CXL_DC_ADD_CAPACITY, "Add capacity"}, \ + { CXL_DC_REL_CAPACITY, "Release capacity"}, \ + { CXL_DC_FORCED_REL_CAPACITY, "Forced capacity release"}, \ + { CXL_DC_REG_CONF_UPDATED, "Region Configuration Updated" } \ +) + +TRACE_EVENT(cxl_dynamic_capacity, + + TP_PROTO(const struct cxl_memdev *cxlmd, enum cxl_event_log_type log, + struct cxl_event_dcd *rec), + + TP_ARGS(cxlmd, log, rec), + + TP_STRUCT__entry( + CXL_EVT_TP_entry + + /* Dynamic capacity Event */ + __field(u8, event_type) + __field(u16, hostid) + __field(u8, partition_id) + __field(u64, dpa_start) + __field(u64, length) + __array(u8, uuid, UUID_SIZE) + __field(u16, sh_extent_seq) + ), + + TP_fast_assign( + CXL_EVT_TP_fast_assign(cxlmd, log, rec->hdr); + + /* Dynamic_capacity Event */ + __entry->event_type = rec->event_type; + + /* DCD event record data */ + __entry->hostid = le16_to_cpu(rec->host_id); + __entry->partition_id = rec->partition_index; + __entry->dpa_start = le64_to_cpu(rec->extent.start_dpa); + __entry->length = le64_to_cpu(rec->extent.length); + memcpy(__entry->uuid, &rec->extent.uuid, UUID_SIZE); + __entry->sh_extent_seq = le16_to_cpu(rec->extent.shared_extn_seq); + ), + + CXL_EVT_TP_printk("event_type='%s' host_id='%d' partition_id='%d' " \ + "starting_dpa=%llx length=%llx tag=%pU " \ + "shared_extent_sequence=%d", + show_dc_evt_type(__entry->event_type), + __entry->hostid, + __entry->partition_id, + __entry->dpa_start, + __entry->length, + __entry->uuid, + __entry->sh_extent_seq + ) +); + #endif /* _CXL_EVENTS_H */ #define TRACE_INCLUDE_FILE trace From patchwork Sun Apr 13 22:52:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049476 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71D1F20F081 for ; Sun, 13 Apr 2025 22:52:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584759; cv=fail; b=b7XZt5knskk3AY0z8xS5icob4ig39P4FHQURQ426WMjT6sJmlEam/t3OeZnm25s7egxIE+do4chE+AYS//AL17rGeCHhOsqDRQ0EaWURvCWGgo9Ebk3GCVEdyaxGCqdkYXUmFoEr6V09nPOThszsvi7GpinkEV4ab//CJtAjm3g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584759; c=relaxed/simple; bh=hIzly7Huia2oFq0h25HsM1O8AIHB+uENrUA+sLPFMr8=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=m9OVFa2F6gk5/vB68+5sY25C992Ojac18u9Fa2IMTjw+xed3+Xe6Ib+x3U+N7W/Dz7ysEhHDDs0yZC/+Md3HKENkNfRhl9oxL5JyR3AjK6ygSfxLdRPBCHJ6iL/guBl9B9SqoNZT6RNfNBTzRhtt32b7H3pUm/RNGi1ForLAIQ0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZXPAOmXy; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZXPAOmXy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584758; x=1776120758; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=hIzly7Huia2oFq0h25HsM1O8AIHB+uENrUA+sLPFMr8=; b=ZXPAOmXyBkcz01K3A0ywjHp/w40fyERRFn0UaTUCu5sCnW8SYKEyFoWq yeyStHYSWYvPY1PexYWej5SRtemydUmlq8KAmjHY4A0azOIzMmQlcjO6a Zfmkvgv5YMmxqL0jXWR7SKiYGAdEYuWRUls+sc/Ns6b3akq8gNJTmZSxj I5yEiRiLwI1rlXCglBkoEU/d4gHGDYonKX1velyLXGzSh19Si4VP+YNS6 Bm+rCXgN/LwB3goA5+1v3C1RC+b+A/bz4ggb+piCkCWX9ehci0tYp5gqR L+22CSOxpT4JH+nmQNpgqlAv26aRGH9wX4BRC0xShaucH7aRmHXZyTtcy Q==; X-CSE-ConnectionGUID: mM3HhavFSjCqnBbcWIijrQ== X-CSE-MsgGUID: 9HF1OwZ5RPa6tjipgfH2/A== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280971" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280971" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:37 -0700 X-CSE-ConnectionGUID: 3NXhHjWVT9ykHcVjpKcyBw== X-CSE-MsgGUID: OF3/jx1rT7+eL1hkcNEzkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657636" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:37 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:36 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:36 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.57.44) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MK+u93b3Rji4AgMikp6ENX6Au4cRzs3ZzoVuZbP8cdyMd5kPaw7ux27GmoTe/NdQABOBRUZlYMnIesaADb9RsKfPH8+d/LQd7qx74dfANbwdE3eKF9268z32cxdQ2Yxngi7oV6V7LCaAi6R5/NfuBHXqL52aLr8nY+ep7JpaxlhC4eY3IorUiTQMcsTg5nIfEF7cVBfNPCUO1gZS9ezkrTkrtIBgiH2SsqITY76p+Djmr/i3JjMImHFm2ufIN/ThTrL5/5ef67d22Y1MiTZTpXoYX4hYqKeDavA1VSWCPx/0NscOAKInO4cpHap3dGdBQeN9DDicNNsMLozSr2b2RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zNDIWhxO6Oq0+ekZCjsNilh5/+781UpnrG4oEYvajM4=; b=oVx0pn2Eh9pCCR4Ds47ykeCaWews/GHIgqlzXnE7CxFf/I8i0X1b3wI2uXSP9w/7p4QT3Gl16KyTA2DKyjTH9Z6aV2o3z3Dv8ILeNl8HLywlxpzUaMAvB3Hp+MhIIk9y2dmdtZQ3SHQgmpgnMdHca77bEsE4uj4QZQh+BNvOUZpNXTtM6GIKuGwtHRwsThVakpmNcDLJSh+p7InUVM7C8wljlqknCUThwigpdQKyONzxP9xX/ue17VxkIKr1kMnTuHmD1Q47qEvBez4y9eRRX54mQzKrPj3m8rRYyAWqwI4Eed7l2WtjYx7uhdXliqpTqc/e1IWOIwLUvxDY2vakig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:12 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:12 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:26 -0500 Subject: [PATCH v9 18/19] tools/testing/cxl: Make event logs dynamic Message-ID: <20250413-dcd-type2-upstream-v9-18-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=15739; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=hIzly7Huia2oFq0h25HsM1O8AIHB+uENrUA+sLPFMr8=; b=qpQkHpLuBE/2dAlhrUlSTLqrdxOrBEnFofzVUxY1p4gvHMRQH8V5CQTE1T7WkoNyRwDkIGx5k 8k2ldon1HX8B8v5vWbPzhgPiHnVFeWtN5qX1ArR8P9Xe++yTahE/EA1 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: 0a124e87-aaa0-43df-a72d-08dd7addd42b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?vPLn9hPeO4+YuwHnQ5I8vDnxDN/4Sme?= =?utf-8?q?9C4t+FVRmQsx2JZ4tzBAx11aScN15jWN5SxqfmUEtw8ujijWRv3IgKxz3CSvs9jCm?= =?utf-8?q?iwr6i/ZcOYPmOixzwSa4MadTpAT4sKhsRypOqAZCF2bYg1r4O0Lv/2dHZkewcph/X?= =?utf-8?q?I5ZcmevrLh58kdLsOhTpfCTAyQznbAQNY40EL00wtqQwIxr/MqkzvaqQcAQpnGgD5?= =?utf-8?q?+dNZPLBcms+Lcv136Jlb00oJnS0JmD6huEUmuwBsiUnnDTQrbc2oNlRCSTFzBpsM4?= =?utf-8?q?HAphYKeRzZtMdbjnfNaU+ygBulwzsYo9dGNqlz/jzckIn24wSIeXtAplhbH7XQpTW?= =?utf-8?q?q/OOaQnnjZgnPE2Vb1vIIoHCixkIhUPS5egpPStpFjzkN2Qq2NXSSndWBfIus3/AQ?= =?utf-8?q?yVBf1LflzocB+D1ue2ajU86vLe089g9OPqp7eUj5wCXCX45qDtjGRoNoTFUb/EjLj?= =?utf-8?q?L7DsGPqiGQcRFMsBYY+QrMvC+8Jn5C5zEGtaBV3x+TuHow2uLxjs6/LnB6TlWDhby?= =?utf-8?q?p7LG11Ny7wHTQlNzuUB4ZyuUvsuNmzG7XxiylGbI510kQqx2xCj1XNdabcJpcilOg?= =?utf-8?q?cHxWZqXiENddBt4ot/mLFpd/IJpF6W7NI1+mX8Qed02GlfsRJ57GS+jvW9p2eVupA?= =?utf-8?q?ZicIIQKn7rtfg+pdnBB6ooSvB62ktgSE9zElElZPfLqzZhQkHOWH6V8GGHCup3TzE?= =?utf-8?q?Y1isAh9jMPWF3PXuA+4+4sugWzEMiq+LBj4rcZxZ50ANjTcCeV/Hk8XBsS3QK3ge/?= =?utf-8?q?QA0iIPzutJZ04T5GVMMa2jVSpU6G7fUnfZ+U65eWFaejeg1DVBiO9N2qgpnjrX6bB?= =?utf-8?q?N8O8zXg33OrDTcHJDD0HV4yLInN4hdre1juGyTssHpNjST7zvTd+nobcU2V1WEYXu?= =?utf-8?q?NQ4n3s0gyiakLaPgMiSQBwrbIQXeGt/WpwffYVZJYkuWCdf6OQEEaN0CKXdC2CeFM?= =?utf-8?q?P7x4vV0HPm2g695F7Gx2tSAbd/ufbgIpipvF+gSR2f6ikVsGmjJX7VJlFMMkaAeSw?= =?utf-8?q?kHriRA2rg0o5cHuTHxk0RZHOjAV0ovLxFiWgQqMzMGI1QkUwQ4tgnCpDsT21mDy5M?= =?utf-8?q?l1becUh8SWhh6d6G2bTQawvF4tTHuPIFN4vH5DDirABY40Bkst2r16YkFEj9cKWXY?= =?utf-8?q?9llj2Zor1CKyqNtDWxU6GZG34vVBgl/HLAFoAqTD9hhFVy6CrhC4IILe4dl8UnHnj?= =?utf-8?q?b01QCpGAT7L9vWaRE5I5KpVUkWkwDxcsODfWswd+JvpP3I6JkwTkhrl/ozsolLben?= =?utf-8?q?7kcEU75+F3fR/qrjA1Iy+aC5tKN07KL8NGAhQJvRel/tcjBYtemoG4RThR1ljvih6?= =?utf-8?q?MAoxFIjicPVrOcJMMoILDM4Nvx0LpaOH4mlXPwTsdRpHLwF76e4GgaU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?h7TKhOSuSQbvsvwVS110vBYOfeWs?= =?utf-8?q?2jQqc6Eak6laHo4RXusuLs/jFtO8klr0MnDx+Pz7KAWKYNZ1bmrltLJiRUlMIzg5z?= =?utf-8?q?Qb5t1hydr9DBoCMwr/THYA+nb9I80K7GrOc1fTeVXIPtGaUB1xYIv8wQ0sJJ23pH9?= =?utf-8?q?pusj4NCYMtOUermM406GDd6F7ArM12b+CPfOyJ6oulGB28NsX4Pc3NXB4CVEevIOX?= =?utf-8?q?AT/gzzD+tLrK3kz31L3SlSnDo2pPrL0bnsBdfdOs+u19DSCC4una1k7ygpH57qulG?= =?utf-8?q?Frk3GCdTh8cMgUbB/owBg1zwprpSrlGkyIvfNADpC/GBGOIpKiq+4L4SoDb0CEboR?= =?utf-8?q?HayZBIDWrWDA051ASpeAx1w0PZrqFEUnx/u7PZoc3t8vTZ4pNxLRyVJIgFpKuBxBH?= =?utf-8?q?0s803cO5Q5ubqZrOfEMe/7De3uqklD8wtqIn0oo8s3I57h7lazE6TRJi/sPRQkCQ6?= =?utf-8?q?ycPPfUw51yIdUV7vvkQx7qhgf23Qd23dugm6KXhgjpxFkwhlps1BTdk+xNzAjH1NU?= =?utf-8?q?h2zOfqJ7bMhbPu6yHwwO9Hn/7T3d+k2BnpWKCCIrMC0zfR1hCLNEjLujTZwVSyu9H?= =?utf-8?q?0buW8MCw3SETHMTqNfEs5aAr0iEQtK/dPp/RdMC9ujz/TiDpkbEblFjg0ts8eule/?= =?utf-8?q?hiTWLUtl0FL3E2Rzrs9B23qI3fust256g5BoQoYsj9CbcjvfVAcntxM9fcV6sPdRh?= =?utf-8?q?ykOFhGQeStVHsKOHHmu0hKNRSU9AWWvceEoV+ZkYqBQGDWImk9VRmc+L5+QoBX2AT?= =?utf-8?q?mjxUZlgx/uMu/dQ0MjOnLzK8/IiTwQTsVFX+ECfmB41JCt6tprcuDkkOAGsbh3wEI?= =?utf-8?q?JGNtRXrAXsWAvbb9kWq6A+G7vkjMAVX1LdLTmSWncoSmgi7WLCYB9IUjL0EV9D3s0?= =?utf-8?q?P8r/C+MDFNSoByHiTHCrcx7832jETrzxurUQcVXydwR/6tqKAbrATbwm1iQvhVub1?= =?utf-8?q?El7YCkrOypnINyeLsMQCus2Yr4RtoJwh2QveJV9RY8nUo2pw0MH8DpIXMhjZFJjId?= =?utf-8?q?zCczqrT19kDlBgqRWIVa91YJ6WsSjdsSXCqMg5Tmp8ugFIcts+NxJAKVvjclhhtLn?= =?utf-8?q?6/JPCPnnURIx3sM4kePwZ6Yne3Fw5QjImNSLYBRa3lsHwoc9eJHi3T0jVTyziuymn?= =?utf-8?q?YXS7NqpIjl72gYGTB5TZ5k1yqRcrQRvty2zwNr31J7BGRLY7ES+r4LIO0j4usqnq8?= =?utf-8?q?B/euMyOf/KKSKkLfNEOT4ySbhPL6HgFQCTBGTNRdTsZDZZjk4OjX0+WFtjxsOfU+N?= =?utf-8?q?HXIqeuZOKEAK1pQk8NsXTZtqD3jS1X/D8BVf2YHxMI3+k1O3giVkXuxMS0dIbNH35?= =?utf-8?q?JSQMqwXrMSt02sE3NZsPRzuhkYhZ7zCnkrAh/yoEFnCPa7jK/nN4455KQTgM2IlCQ?= =?utf-8?q?aICWrTH1R04gKKtBaRh14/xF558f0CcBWsomml7DHTKHRN2BxuhEbsd0r6YmWQKRq?= =?utf-8?q?s/P96s4r6GLXy/tvQdBQFfnz4Kt8DZNboVCTVVMMkH6F8EwrGkrdVqFQHYJgF30XR?= =?utf-8?q?Cl5FWVZzh848?= X-MS-Exchange-CrossTenant-Network-Message-Id: 0a124e87-aaa0-43df-a72d-08dd7addd42b X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:12.5974 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1EIAslxKlp+yIFy6Tr0DqWm1oYAjhht0I2aJq7x15uGrKvUiJA/CltdeAaDzaG+3CoRJDOvZV8yPMyzPWbs1Eg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com The event logs test was created as static arrays as an easy way to mock events. Dynamic Capacity Device (DCD) test support requires events be generated dynamically when extents are created or destroyed. The current event log test has specific checks for the number of events seen including log overflow. Modify mock event logs to be dynamically allocated. Adjust array size and mock event entry data to match the output expected by the existing event test. Use the static event data to create the dynamic events in the new logs without inventing complex event injection for the previous tests. Simplify log processing by using the event log array index as the handle. Add a lock to manage concurrency required when user space is allowed to control DCD extents Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase to 6.15-rc1] --- tools/testing/cxl/test/mem.c | 268 ++++++++++++++++++++++++++----------------- 1 file changed, 162 insertions(+), 106 deletions(-) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index f2957a3e36fe..a71a72966de1 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -142,18 +142,26 @@ static struct { #define PASS_TRY_LIMIT 3 -#define CXL_TEST_EVENT_CNT_MAX 15 +#define CXL_TEST_EVENT_CNT_MAX 16 +/* 1 extra slot to accommodate that handles can't be 0 */ +#define CXL_TEST_EVENT_ARRAY_SIZE (CXL_TEST_EVENT_CNT_MAX + 1) /* Set a number of events to return at a time for simulation. */ #define CXL_TEST_EVENT_RET_MAX 4 +/* + * @last_handle: last handle (index) to have an entry stored + * @current_handle: current handle (index) to be returned to the user on get_event + * @nr_overflow: number of events added past the log size + * @lock: protect these state variables + * @events: array of pending events to be returned. + */ struct mock_event_log { - u16 clear_idx; - u16 cur_idx; - u16 nr_events; + u16 last_handle; + u16 current_handle; u16 nr_overflow; - u16 overflow_reset; - struct cxl_event_record_raw *events[CXL_TEST_EVENT_CNT_MAX]; + rwlock_t lock; + struct cxl_event_record_raw *events[CXL_TEST_EVENT_ARRAY_SIZE]; }; struct mock_event_store { @@ -194,56 +202,65 @@ static struct mock_event_log *event_find_log(struct device *dev, int log_type) return &mdata->mes.mock_logs[log_type]; } -static struct cxl_event_record_raw *event_get_current(struct mock_event_log *log) -{ - return log->events[log->cur_idx]; -} - -static void event_reset_log(struct mock_event_log *log) -{ - log->cur_idx = 0; - log->clear_idx = 0; - log->nr_overflow = log->overflow_reset; -} - /* Handle can never be 0 use 1 based indexing for handle */ -static u16 event_get_clear_handle(struct mock_event_log *log) +static u16 event_inc_handle(u16 handle) { - return log->clear_idx + 1; + handle = (handle + 1) % CXL_TEST_EVENT_ARRAY_SIZE; + if (handle == 0) + handle = 1; + return handle; } -/* Handle can never be 0 use 1 based indexing for handle */ -static __le16 event_get_cur_event_handle(struct mock_event_log *log) -{ - u16 cur_handle = log->cur_idx + 1; - - return cpu_to_le16(cur_handle); -} - -static bool event_log_empty(struct mock_event_log *log) -{ - return log->cur_idx == log->nr_events; -} - -static void mes_add_event(struct mock_event_store *mes, +/* Add the event or free it on overflow */ +static void mes_add_event(struct cxl_mockmem_data *mdata, enum cxl_event_log_type log_type, struct cxl_event_record_raw *event) { + struct device *dev = mdata->mds->cxlds.dev; struct mock_event_log *log; if (WARN_ON(log_type >= CXL_EVENT_TYPE_MAX)) return; - log = &mes->mock_logs[log_type]; + log = &mdata->mes.mock_logs[log_type]; + + guard(write_lock)(&log->lock); - if ((log->nr_events + 1) > CXL_TEST_EVENT_CNT_MAX) { + dev_dbg(dev, "Add log %d cur %d last %d\n", + log_type, log->current_handle, log->last_handle); + + /* Check next buffer */ + if (event_inc_handle(log->last_handle) == log->current_handle) { log->nr_overflow++; - log->overflow_reset = log->nr_overflow; + dev_dbg(dev, "Overflowing log %d nr %d\n", + log_type, log->nr_overflow); + devm_kfree(dev, event); return; } - log->events[log->nr_events] = event; - log->nr_events++; + dev_dbg(dev, "Log %d; handle %u\n", log_type, log->last_handle); + event->event.generic.hdr.handle = cpu_to_le16(log->last_handle); + log->events[log->last_handle] = event; + log->last_handle = event_inc_handle(log->last_handle); +} + +static void mes_del_event(struct device *dev, + struct mock_event_log *log, + u16 handle) +{ + struct cxl_event_record_raw *record; + + lockdep_assert(lockdep_is_held(&log->lock)); + + dev_dbg(dev, "Clearing event %u; record %u\n", + handle, log->current_handle); + record = log->events[handle]; + if (!record) + dev_err(dev, "Mock event index %u empty?\n", handle); + + log->events[handle] = NULL; + log->current_handle = event_inc_handle(log->current_handle); + devm_kfree(dev, record); } /* @@ -256,7 +273,7 @@ static int mock_get_event(struct device *dev, struct cxl_mbox_cmd *cmd) { struct cxl_get_event_payload *pl; struct mock_event_log *log; - u16 nr_overflow; + u16 handle; u8 log_type; int i; @@ -277,29 +294,38 @@ static int mock_get_event(struct device *dev, struct cxl_mbox_cmd *cmd) memset(cmd->payload_out, 0, struct_size(pl, records, 0)); log = event_find_log(dev, log_type); - if (!log || event_log_empty(log)) + if (!log) return 0; pl = cmd->payload_out; - for (i = 0; i < ret_limit && !event_log_empty(log); i++) { - memcpy(&pl->records[i], event_get_current(log), - sizeof(pl->records[i])); - pl->records[i].event.generic.hdr.handle = - event_get_cur_event_handle(log); - log->cur_idx++; + guard(read_lock)(&log->lock); + + handle = log->current_handle; + dev_dbg(dev, "Get log %d handle %u last %u\n", + log_type, handle, log->last_handle); + for (i = 0; i < ret_limit && handle != log->last_handle; + i++, handle = event_inc_handle(handle)) { + struct cxl_event_record_raw *cur; + + cur = log->events[handle]; + dev_dbg(dev, "Sending event log %d handle %d idx %u\n", + log_type, le16_to_cpu(cur->event.generic.hdr.handle), + handle); + memcpy(&pl->records[i], cur, sizeof(pl->records[i])); + pl->records[i].event.generic.hdr.handle = cpu_to_le16(handle); } cmd->size_out = struct_size(pl, records, i); pl->record_count = cpu_to_le16(i); - if (!event_log_empty(log)) + if (handle != log->last_handle) pl->flags |= CXL_GET_EVENT_FLAG_MORE_RECORDS; if (log->nr_overflow) { u64 ns; pl->flags |= CXL_GET_EVENT_FLAG_OVERFLOW; - pl->overflow_err_count = cpu_to_le16(nr_overflow); + pl->overflow_err_count = cpu_to_le16(log->nr_overflow); ns = ktime_get_real_ns(); ns -= 5000000000; /* 5s ago */ pl->first_overflow_timestamp = cpu_to_le64(ns); @@ -314,8 +340,8 @@ static int mock_get_event(struct device *dev, struct cxl_mbox_cmd *cmd) static int mock_clear_event(struct device *dev, struct cxl_mbox_cmd *cmd) { struct cxl_mbox_clear_event_payload *pl = cmd->payload_in; - struct mock_event_log *log; u8 log_type = pl->event_log; + struct mock_event_log *log; u16 handle; int nr; @@ -326,23 +352,20 @@ static int mock_clear_event(struct device *dev, struct cxl_mbox_cmd *cmd) if (!log) return 0; /* No mock data in this log */ - /* - * This check is technically not invalid per the specification AFAICS. - * (The host could 'guess' handles and clear them in order). - * However, this is not good behavior for the host so test it. - */ - if (log->clear_idx + pl->nr_recs > log->cur_idx) { - dev_err(dev, - "Attempting to clear more events than returned!\n"); - return -EINVAL; - } + guard(write_lock)(&log->lock); /* Check handle order prior to clearing events */ - for (nr = 0, handle = event_get_clear_handle(log); - nr < pl->nr_recs; - nr++, handle++) { + handle = log->current_handle; + for (nr = 0; nr < pl->nr_recs && handle != log->last_handle; + nr++, handle = event_inc_handle(handle)) { + + dev_dbg(dev, "Checking clear of %d handle %u plhandle %u\n", + log_type, handle, + le16_to_cpu(pl->handles[nr])); + if (handle != le16_to_cpu(pl->handles[nr])) { - dev_err(dev, "Clearing events out of order\n"); + dev_err(dev, "Clearing events out of order %u %u\n", + handle, le16_to_cpu(pl->handles[nr])); return -EINVAL; } } @@ -351,25 +374,12 @@ static int mock_clear_event(struct device *dev, struct cxl_mbox_cmd *cmd) log->nr_overflow = 0; /* Clear events */ - log->clear_idx += pl->nr_recs; - return 0; -} - -static void cxl_mock_event_trigger(struct device *dev) -{ - struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); - struct mock_event_store *mes = &mdata->mes; - int i; + for (nr = 0; nr < pl->nr_recs; nr++) + mes_del_event(dev, log, le16_to_cpu(pl->handles[nr])); + dev_dbg(dev, "Delete log %d cur %d last %d\n", + log_type, log->current_handle, log->last_handle); - for (i = CXL_EVENT_TYPE_INFO; i < CXL_EVENT_TYPE_MAX; i++) { - struct mock_event_log *log; - - log = event_find_log(dev, i); - if (log) - event_reset_log(log); - } - - cxl_mem_get_event_records(mdata->mds, mes->ev_status); + return 0; } struct cxl_event_record_raw maint_needed = { @@ -510,8 +520,27 @@ static int mock_set_timestamp(struct cxl_dev_state *cxlds, return 0; } -static void cxl_mock_add_event_logs(struct mock_event_store *mes) +/* Create a dynamically allocated event out of a statically defined event. */ +static void add_event_from_static(struct cxl_mockmem_data *mdata, + enum cxl_event_log_type log_type, + struct cxl_event_record_raw *raw) +{ + struct device *dev = mdata->mds->cxlds.dev; + struct cxl_event_record_raw *rec; + + rec = devm_kmemdup(dev, raw, sizeof(*rec), GFP_KERNEL); + if (!rec) { + dev_err(dev, "Failed to alloc event for log\n"); + return; + } + mes_add_event(mdata, log_type, rec); +} + +static void cxl_mock_add_event_logs(struct cxl_mockmem_data *mdata) { + struct mock_event_store *mes = &mdata->mes; + struct device *dev = mdata->mds->cxlds.dev; + put_unaligned_le16(CXL_GMER_VALID_CHANNEL | CXL_GMER_VALID_RANK | CXL_GMER_VALID_COMPONENT | CXL_GMER_VALID_COMPONENT_ID_FORMAT, &gen_media.rec.media_hdr.validity_flags); @@ -524,43 +553,60 @@ static void cxl_mock_add_event_logs(struct mock_event_store *mes) put_unaligned_le16(CXL_MMER_VALID_COMPONENT | CXL_MMER_VALID_COMPONENT_ID_FORMAT, &mem_module.rec.validity_flags); - mes_add_event(mes, CXL_EVENT_TYPE_INFO, &maint_needed); - mes_add_event(mes, CXL_EVENT_TYPE_INFO, + dev_dbg(dev, "Generating fake event logs %d\n", + CXL_EVENT_TYPE_INFO); + add_event_from_static(mdata, CXL_EVENT_TYPE_INFO, &maint_needed); + add_event_from_static(mdata, CXL_EVENT_TYPE_INFO, (struct cxl_event_record_raw *)&gen_media); - mes_add_event(mes, CXL_EVENT_TYPE_INFO, + add_event_from_static(mdata, CXL_EVENT_TYPE_INFO, (struct cxl_event_record_raw *)&mem_module); mes->ev_status |= CXLDEV_EVENT_STATUS_INFO; - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &maint_needed); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, + dev_dbg(dev, "Generating fake event logs %d\n", + CXL_EVENT_TYPE_FAIL); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &maint_needed); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, + (struct cxl_event_record_raw *)&mem_module); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, (struct cxl_event_record_raw *)&dram); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, (struct cxl_event_record_raw *)&gen_media); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, (struct cxl_event_record_raw *)&mem_module); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, (struct cxl_event_record_raw *)&dram); /* Overflow this log */ - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FAIL, &hardware_replace); mes->ev_status |= CXLDEV_EVENT_STATUS_FAIL; - mes_add_event(mes, CXL_EVENT_TYPE_FATAL, &hardware_replace); - mes_add_event(mes, CXL_EVENT_TYPE_FATAL, + dev_dbg(dev, "Generating fake event logs %d\n", + CXL_EVENT_TYPE_FATAL); + add_event_from_static(mdata, CXL_EVENT_TYPE_FATAL, &hardware_replace); + add_event_from_static(mdata, CXL_EVENT_TYPE_FATAL, (struct cxl_event_record_raw *)&dram); mes->ev_status |= CXLDEV_EVENT_STATUS_FATAL; } +static void cxl_mock_event_trigger(struct device *dev) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct mock_event_store *mes = &mdata->mes; + + cxl_mock_add_event_logs(mdata); + cxl_mem_get_event_records(mdata->mds, mes->ev_status); +} + static int mock_gsl(struct cxl_mbox_cmd *cmd) { if (cmd->size_out < sizeof(mock_gsl_payload)) @@ -1685,6 +1731,14 @@ static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) mdata->test_feat.data = cpu_to_le32(0xdeadbeef); } +static void init_event_log(struct mock_event_log *log) +{ + rwlock_init(&log->lock); + /* Handle can never be 0 use 1 based indexing for handle */ + log->current_handle = 1; + log->last_handle = 1; +} + static int cxl_mock_mem_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1766,7 +1820,9 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) dev_dbg(dev, "No CXL Features discovered\n"); - cxl_mock_add_event_logs(&mdata->mes); + for (int i = 0; i < CXL_EVENT_TYPE_MAX; i++) + init_event_log(&mdata->mes.mock_logs[i]); + cxl_mock_add_event_logs(mdata); cxlmd = devm_cxl_add_memdev(&pdev->dev, cxlds); if (IS_ERR(cxlmd)) From patchwork Sun Apr 13 22:52:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 14049475 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 826271A314E for ; Sun, 13 Apr 2025 22:52:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584759; cv=fail; b=pos7dUR5FRetVbOpdLZoG3/bCFK5OcOUHscylsfaq/IIwfN8RybXImzHCd8+c+QjHAs5jYMlLYfPbPJtlpDgWyd0j5AR5ernr6rIiVGaLqkrCPV/WWQqFZUKEYZY4puObvQX8EHreQp4nQjJDS8Pr0u734DylsdmvoubskagOBE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744584759; c=relaxed/simple; bh=wHMVv6tL4QfxMTR8qMmB+pCG/N575jfvmo5YYo0K9xY=; h=From:Date:Subject:Content-Type:Message-ID:References:In-Reply-To: To:CC:MIME-Version; b=cKTK9skHLjPwubW/+n3S3h24bUK1T82jwUn8D1M/VxjxbdKGNAIWF+k3d07n1QPpcm9ppLkE0Bq9VaYJm0xhisFlCCXxtBfFrCslCmKKmRgd1zr/Rpi+lSynGGyJysjdBSGXLmQ+VzYzUCdtL0YYKyqMcBQGnvrRvsAXxTQEq60= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HPugc1yl; arc=fail smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HPugc1yl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744584757; x=1776120757; h=from:date:subject:content-transfer-encoding:message-id: references:in-reply-to:to:cc:mime-version; bh=wHMVv6tL4QfxMTR8qMmB+pCG/N575jfvmo5YYo0K9xY=; b=HPugc1ylEzxnhuOoBctrGQ7LBunfX21GtBl8cJKg4tTiTvslQayZMPgH saKzKSqU7tLbjr35kbJ+p1c/ede9rxWAfD2mukRkLhWO7NC+ZlVVGvImz Rnb8kMFK8JIEYWQGhirsbrqBCnLwtzS2qwTayQsy9Ci7joKp5nYbu+7lJ wLAdgk3CLly0hkz5vUdxC3j5cRSVZDNSF9IWAP89LVtJgj/8B0v6P2DSH HBDnd8sCtvp4ou+qildDHf/IMsiWMOA1OdIMT+RzxyxNzXcO242BvQ4ak KbKzci3XMwLufAXd9qm4WNsphsxk0PuNH5v6uBycW8uhYenjxJqCHLKOw A==; X-CSE-ConnectionGUID: L1exXZI0Qn+FIRmh9uthRg== X-CSE-MsgGUID: U14AMLJ1Q4CVDei31heLzg== X-IronPort-AV: E=McAfee;i="6700,10204,11402"; a="45280966" X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="45280966" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:36 -0700 X-CSE-ConnectionGUID: o7R247WxT/Wm5SM001gjTg== X-CSE-MsgGUID: u1yOuR2GQ1aK8M7st7UCTA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,211,1739865600"; d="scan'208";a="129657635" Received: from orsmsx902.amr.corp.intel.com ([10.22.229.24]) by fmviesa007.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2025 15:52:36 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sun, 13 Apr 2025 15:52:35 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Sun, 13 Apr 2025 15:52:35 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.57.44) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.44; Sun, 13 Apr 2025 15:52:34 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h9zHSybtzuUf6yYMpirbERwz6ZLqtBbYZvKYmQdWS49xATYQikoq7/6HouGFT/ueT8+TnZpKRa4G4EsC5qEIdEnCHNpMD0HVmDQ0t1v+NdJxxvuU1YkS3nThkbR07onndcu8TsPEBszKd/pHFmZYVeGvomOSReAv5PlmW584KGOZb+y8uuU8A7jVFy2ff4NGwdXxLtnPxV1t9ty/536lo4fHcT/tGK5uZ9eE2NUvJX5RmtjvW8z23uVpAIHpcBpsOjlxXqF0C0mPmszrxdW5qmxW4QV/TX5248ImK7ypELVBz5kYQBdGpFBmk8ac+tg5Y2OdElpxLOkRUdERdVr5YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DxjHic46EGVica6+xsSexw+RIXWnwKQtDZaXL+uUKYI=; b=tjJ9XFk9WqT9IUfo5NElho/FHD0ju/lfSOiknd0DK8nRW8XM7eEyyrHoj8uC2vZe5uSoaqcOj8O50xBXQ0Bwv9rymMk/ZwFzTejeYUo2Z9ytupi+2K0unrIkNsIGaokIQg3hAf3XRoLEcfKvaNy/AyphsuGbW3OXwj7/BYwSFAzfzbDkBbheVIMJl7WPR3gUqX2CNzuKd35yPt4PCDNFveQ3PILR4gsK2xDk6ssS8yrERc+Ox7fMl+40SkPMy2oWFu8/LYmVlUPLRAF+F6PBSBLOGoAv736NioTteEEAJWazIbLda7Y3PUlrDbsuBOsYfeAMmwp88OntVQkITHwkcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) by PH7PR11MB7003.namprd11.prod.outlook.com (2603:10b6:510:20a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8632.33; Sun, 13 Apr 2025 22:52:14 +0000 Received: from MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24]) by MW4PR11MB6739.namprd11.prod.outlook.com ([fe80::a7ad:a6e8:fced:3f24%4]) with mapi id 15.20.8606.033; Sun, 13 Apr 2025 22:52:14 +0000 From: Ira Weiny Date: Sun, 13 Apr 2025 17:52:27 -0500 Subject: [PATCH v9 19/19] tools/testing/cxl: Add DC Regions to mock mem data Message-ID: <20250413-dcd-type2-upstream-v9-19-1d4911a0b365@intel.com> References: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> In-Reply-To: <20250413-dcd-type2-upstream-v9-0-1d4911a0b365@intel.com> To: Dave Jiang , Fan Ni , "Jonathan Cameron" CC: Dan Williams , Davidlohr Bueso , Alison Schofield , "Vishal Verma" , Ira Weiny , , , X-Mailer: b4 0.15-dev-c25d1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1744584735; l=24753; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=wHMVv6tL4QfxMTR8qMmB+pCG/N575jfvmo5YYo0K9xY=; b=DOc2lTVyHjID6tyGD8eW/iq+KW9krVGXgzdgNV3Z5RxYkzav+D9Xat2LtZVixYJ8J3I00wMD4 4PfhTkqhfC+DGsjJ/sQG1gQQbMLCGSyzr0wKUm5gTRMh9QQHdAx6lmq X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= X-ClientProxiedBy: MW4PR03CA0227.namprd03.prod.outlook.com (2603:10b6:303:b9::22) To MW4PR11MB6739.namprd11.prod.outlook.com (2603:10b6:303:20b::19) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB6739:EE_|PH7PR11MB7003:EE_ X-MS-Office365-Filtering-Correlation-Id: 16ec98af-ac4b-466e-1458-08dd7addd553 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?q?ymuHDkq5NOAEXLuBTGliNl/z1c3rKWA?= =?utf-8?q?qmefcUenGrk+yyCyDMAuFQ+x3iZYL3eBRYc0pgfSwV2daeY+Lq6zQ3eKuVEiMKF1f?= =?utf-8?q?k814x0eUEiFowIWC3f+j4E2gh2zx83ANBipiXCkW3oMz9Fi2bZprgljo6xTpEgWbu?= =?utf-8?q?FujRuDasl70c5h1SqB3aw4/6E/8D/8FAqv/0J0q/+oJpr7H7RgvsOrwhjT2IiX/W4?= =?utf-8?q?x01I9WX5+TvKdq17UolYOUIVs7hyFHTcElsRSGqIiTxP9iQSNpRO89vkywV00z/tE?= =?utf-8?q?40t2qsRmEvK4sE6H3hb0rFwTo2qSKM0NWUMG3RQqD0McChqKSCy4gFG4+mnQ2YlYG?= =?utf-8?q?0KCJNWzXwK7hoB1GgGknD3OmF4HZlzS7L7NPdJZWCyF/guhnH1sb06l+ewlz+1wm3?= =?utf-8?q?Q1XPctnjhVwk6GVQd876aQfsQ5SQ7ICs7aBPdjAP5blmth/yUZPLs2r137qnCbQT+?= =?utf-8?q?aJoyA/+acdGOW7eQ/VofhWoR4IsMmMF/4jP1sX/9nEI7/hlmDp21ZQEG/68YmSpZq?= =?utf-8?q?aWZ9hBfYCmgx9CunJsaR1YvE4Uh1+hyAfEKXUyWyM64dX2ZSOfJm+WDIeFikvxBHA?= =?utf-8?q?At8/W9FLj7Yq8w7ZXACaZYFVMK3uA58b3C8lXspYEAdWcgwIG7B5So+ibmQ0QrQSx?= =?utf-8?q?8qMPZQba5hNCR+hH2x8+Jqpfh87dwQAtZhnXO1FYYWgtIAns9vWFwMhYazfKPysn2?= =?utf-8?q?MjoWjnAWQXa6L7C/JYu7X+M/UYsq/kkb2Omvp2b9PgaImKWc3Dd43v81BCAzXZYXg?= =?utf-8?q?Tmg3Qa7Tf7IvXmyyrwvWE57RXuTTeafTdiLnL+pQks+eLodnYNn94MuSkjOyyFzlG?= =?utf-8?q?lcnHOZtlYTjOTVxTB66TSm+E8or+awlT0REfYv6EO+Z0XdikL2WkiI2wU/FakptGF?= =?utf-8?q?+6ohwLlohvpdA/MkXv+MRHHHGu+EhQ4EHg2BCKqunb6BfeOXy9hWvc9ALY3psoqAd?= =?utf-8?q?98/65IJeSxZ7qNqvgC0m+/iRQM9opB6rjgF29KxuBaJZctIlIRiz2zUTyjmhC6qNX?= =?utf-8?q?OgUd9aryOuaBLdHu3lNmIneV5l6Idnus35bmMEY/WSnkiJXQJS0oBm/9m85EFXqIh?= =?utf-8?q?ClUOFSWQylm5HyBDMpxwbI/hid8GEonp8+a6dQRB5+X9jimruwt+IfMHLIV9yTpZ5?= =?utf-8?q?9b/MZMtIe7NggCpLjnftHjQsX7Hpeg3dSZ8/dAWN6bNRrHRlesHrVPC6H4/M7ApkJ?= =?utf-8?q?bJnEu1OgtG/H9JeWt35CzApI5MhUklX8j1h6cR9Cjy7t5w334gb5K+pg7gbE9SNQZ?= =?utf-8?q?Rv3cCDBQ+4RUYfEm+uThqX2brRisH0dAPlW5UHMhX3dxOS6gwM2qtZzaCdbz9dhx+?= =?utf-8?q?VK2XA9EY0b95JbU8QiVVIj3ExBOKOidOeEpem1byWKfHKP3TBPL7JKU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW4PR11MB6739.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Z7dMIgPCTcZRuIXoDekrCsTvqrPP?= =?utf-8?q?LMYHrPmVLpmRNC2k9UvpJ18vWaoE8dqvnXHRgjtX7HvAojAQKy7gHn2iwECQL1cJ+?= =?utf-8?q?1M5hbXcJ2q5mouVeHdr3ORQ13sq+kyjmmojcEKPvr3/ml6lPDCXom+QjQ+uyZDRiR?= =?utf-8?q?dczoHkZPQrfTHPxmKeGkzFcQSSFQqhnQbqxK2U3OE2kS1FqyF6qIvaiuSI3ZNWAt/?= =?utf-8?q?Gnx8odo1VIzY2dNOKu+ZPQfG75aQFatc1Y5mF4VKOeloVmeS20uu8SJ84+RyiTlls?= =?utf-8?q?Z86KrXnozX/+/lgCz6tfxNqZns2DhdDmJdk7WArB+YZM6NPTDqLQ/LiK63rNUJU1w?= =?utf-8?q?hmrh3uwGjcmQ2fc0weK50uWZOAarpKoKfJJZOONdBl8FTN3+Towi1VQUSVMutJEkL?= =?utf-8?q?2EkLHobbsUEHEcQbpe2P28rNm2rTIWHXfGn/6DbMAoOcdzaOCNmF25ZhJP/BWaeAS?= =?utf-8?q?sD5QjPofjC9LcOCbgsu5/gR+zJm+d/Cnu4G/4Ql+iqiX7H6hXblb6YxdQjCgCWXwW?= =?utf-8?q?dTU9U68Rue/rZT7mMry43UdsohG4C3vtDRjwAqYoJkkzrsJ8HQFCH3esJA6QyLuHn?= =?utf-8?q?zxHQSNqKmBetvzGdG4wfIbQb0NbxdA4fdL1nWz6ZG4eLBnM8hYANUoa66z6UsPFwZ?= =?utf-8?q?RQs+OL7NH1P8IrE9/51CvOW1SL9OpBghTi4wPKBdgXjUc4peVoF2DVvxAOzFl+Tqh?= =?utf-8?q?v1hRpDNgu6bTGjKJQ4Op15lt7onGS+MrUwbk3WdW8O8Fj6QT4aoudFZ8ihhgdv0/P?= =?utf-8?q?PnYLYRUjN0CO/CQVtpB68WvzFeuZjzzhF+cnTaxt5CuugvekxFFDB2PNUTxXTXC1B?= =?utf-8?q?lfFR21AFZNUcFy74Fjbd1d+0YdYjPNtJJMjKdYWIuMjR3Fla2Hlos2O2IG8QMTwhn?= =?utf-8?q?F4laqUz8Z7HyetKNFeL86cWT69v3/IDKL+k6hmqlcSKMRZiQWy7Prk3VinzDvC1/F?= =?utf-8?q?cy5KnFJ7wd8NfscB0SgLpy+xbsMBQLVlH0gHGSMRvOlrf88lK/Pewlqp0y0iGSJX8?= =?utf-8?q?Z46fPqghv25mEqJ0KS/ahuNw5OYFKmKdsFT5nMm56uD0UugtMsoCnGH76pCnNkht0?= =?utf-8?q?sCKTm4iT9yQL/3jpC686KCQSUEj0aa8BLtotQrQTw8wEaQ5pfPzbm7pzI8hnnoIwE?= =?utf-8?q?R3QOmI2ATa9vODKaN4EV62W5vNd7U1Ne7S/Fei0XOg8huxNAhH4aDBJvM6Syx1FK/?= =?utf-8?q?ly2I56ABNBMmn6lZhbbuZUjxn7XhPiSP7Pno3Q472UPU3bkcQLoj7w/Aaoj9IHCu8?= =?utf-8?q?9qPdC0Tig+4k1fV6jr5rMbPNWvdfL7eLVTatsxpSy2zyZ1gsGxnyRcWRaRMqBiKbN?= =?utf-8?q?QBQUqZhSTbEzDmolcS3PX5MWPNtks3gG0MnzfeQi5wZUiPALHSFqqZntKNrpuN1Zw?= =?utf-8?q?sofojco9frwRo7STFqsx1LTM3uYsK6Bw6VhI1mFhTBsL0qoQpIz30itTV9zNHfoH5?= =?utf-8?q?Dkp877RM/6Xb5p07/+ycaWtxi0ScGaefdpMOMXgFShmL51FFAy34C53JXlmkZoAn4?= =?utf-8?q?Q6wTCtEkUI1/?= X-MS-Exchange-CrossTenant-Network-Message-Id: 16ec98af-ac4b-466e-1458-08dd7addd553 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB6739.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2025 22:52:14.4720 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UQJ2ufx/jBtLpUYZpyf5DhFF3mlzBclywOH1z8Z+ZvdTxnZdYBphVW/3SBO+6yITUau93kvWIB3KrxJxq3vefQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7003 X-OriginatorOrg: intel.com cxl_test provides a good way to ensure quick smoke and regression testing. The complexity of Dynamic Capacity (DC) extent processing as well as the complexity of the new sparse DAX regions can mostly be tested through cxl_test. This includes management of sparse regions and DAX devices on those regions; the management of extent device lifetimes; and the processing of DCD events. The only missing functionality from this test is actual interrupt processing. Mock memory devices can easily mock DC information and manage fake extent data. Define mock_dc_partition information within the mock memory data. Add sysfs entries on the mock device to inject and delete extents. The inject format is ::: The delete format is : Directly call the event irq callback to simulate irqs to process the test extents. Add DC mailbox commands to the CEL and implement those commands. Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Signed-off-by: Ira Weiny --- Changes: [iweiny: rebase] [djbw: s/region/partition/] [iweiny: s/tag/uuid/] --- tools/testing/cxl/test/mem.c | 753 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 753 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index a71a72966de1..a85a04168434 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -20,6 +20,7 @@ #define FW_SLOTS 3 #define DEV_SIZE SZ_2G #define EFFECT(x) (1U << x) +#define BASE_DYNAMIC_CAP_DPA DEV_SIZE #define MOCK_INJECT_DEV_MAX 8 #define MOCK_INJECT_TEST_MAX 128 @@ -113,6 +114,22 @@ static struct cxl_cel_entry mock_cel[] = { EFFECT(SECURITY_CHANGE_IMMEDIATE) | EFFECT(BACKGROUND_OP)), }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_DC_CONFIG), + .effect = CXL_CMD_EFFECT_NONE, + }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_DC_EXTENT_LIST), + .effect = CXL_CMD_EFFECT_NONE, + }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_ADD_DC_RESPONSE), + .effect = cpu_to_le16(EFFECT(CONF_CHANGE_IMMEDIATE)), + }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_RELEASE_DC), + .effect = cpu_to_le16(EFFECT(CONF_CHANGE_IMMEDIATE)), + }, }; /* See CXL 2.0 Table 181 Get Health Info Output Payload */ @@ -173,6 +190,8 @@ struct vendor_test_feat { __le32 data; } __packed; +#define NUM_MOCK_DC_REGIONS 2 + struct cxl_mockmem_data { void *lsa; void *fw; @@ -191,6 +210,20 @@ struct cxl_mockmem_data { unsigned long sanitize_timeout; struct vendor_test_feat test_feat; u8 shutdown_state; + + struct cxl_dc_partition dc_partitions[NUM_MOCK_DC_REGIONS]; + u32 dc_ext_generation; + struct mutex ext_lock; + + /* + * Extents are in 1 of 3 states + * FM (sysfs added but not sent to the host yet) + * sent (sent to the host but not accepted) + * accepted (by the host) + */ + struct xarray dc_fm_extents; + struct xarray dc_sent_extents; + struct xarray dc_accepted_exts; }; static struct mock_event_log *event_find_log(struct device *dev, int log_type) @@ -607,6 +640,251 @@ static void cxl_mock_event_trigger(struct device *dev) cxl_mem_get_event_records(mdata->mds, mes->ev_status); } +struct cxl_extent_data { + u64 dpa_start; + u64 length; + u8 uuid[UUID_SIZE]; + bool shared; +}; + +static int __devm_add_extent(struct device *dev, struct xarray *array, + u64 start, u64 length, const char *uuid, + bool shared) +{ + struct cxl_extent_data *extent; + + extent = devm_kzalloc(dev, sizeof(*extent), GFP_KERNEL); + if (!extent) + return -ENOMEM; + + extent->dpa_start = start; + extent->length = length; + memcpy(extent->uuid, uuid, min(sizeof(extent->uuid), strlen(uuid))); + extent->shared = shared; + + if (xa_insert(array, start, extent, GFP_KERNEL)) { + devm_kfree(dev, extent); + dev_err(dev, "Failed xarry insert %#llx\n", start); + return -EINVAL; + } + + return 0; +} + +static int devm_add_fm_extent(struct device *dev, u64 start, u64 length, + const char *uuid, bool shared) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + + guard(mutex)(&mdata->ext_lock); + return __devm_add_extent(dev, &mdata->dc_fm_extents, start, length, + uuid, shared); +} + +/* It is known that ext and the new range are not equal */ +static struct cxl_extent_data * +split_ext(struct device *dev, struct xarray *array, + struct cxl_extent_data *ext, u64 start, u64 length) +{ + u64 new_start, new_length; + + if (ext->dpa_start == start) { + new_start = start + length; + new_length = (ext->dpa_start + ext->length) - new_start; + + if (__devm_add_extent(dev, array, new_start, new_length, + ext->uuid, false)) + return NULL; + + ext = xa_erase(array, ext->dpa_start); + if (__devm_add_extent(dev, array, start, length, ext->uuid, + false)) + return NULL; + + return xa_load(array, start); + } + + /* ext->dpa_start != start */ + + if (__devm_add_extent(dev, array, start, length, ext->uuid, false)) + return NULL; + + new_start = ext->dpa_start; + new_length = start - ext->dpa_start; + + ext = xa_erase(array, ext->dpa_start); + if (__devm_add_extent(dev, array, new_start, new_length, ext->uuid, + false)) + return NULL; + + return xa_load(array, start); +} + +/* + * Do not handle extents which are not inside a single extent sent to + * the host. + */ +static struct cxl_extent_data * +find_create_ext(struct device *dev, struct xarray *array, u64 start, u64 length) +{ + struct cxl_extent_data *ext; + unsigned long index; + + xa_for_each(array, index, ext) { + u64 end = start + length; + + /* start < [ext) <= start */ + if (start < ext->dpa_start || + (ext->dpa_start + ext->length) <= start) + continue; + + if (end <= ext->dpa_start || + (ext->dpa_start + ext->length) < end) { + dev_err(dev, "Invalid range %#llx-%#llx\n", start, + end); + return NULL; + } + + break; + } + + if (!ext) + return NULL; + + if (start == ext->dpa_start && length == ext->length) + return ext; + + return split_ext(dev, array, ext, start, length); +} + +static int dc_accept_extent(struct device *dev, u64 start, u64 length) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_extent_data *ext; + + dev_dbg(dev, "Host accepting extent %#llx\n", start); + mdata->dc_ext_generation++; + + lockdep_assert_held(&mdata->ext_lock); + ext = find_create_ext(dev, &mdata->dc_sent_extents, start, length); + if (!ext) { + dev_err(dev, "Extent %#llx-%#llx not found\n", + start, start + length); + return -ENOMEM; + } + ext = xa_erase(&mdata->dc_sent_extents, ext->dpa_start); + return xa_insert(&mdata->dc_accepted_exts, start, ext, GFP_KERNEL); +} + +static void release_dc_ext(void *md) +{ + struct cxl_mockmem_data *mdata = md; + + xa_destroy(&mdata->dc_fm_extents); + xa_destroy(&mdata->dc_sent_extents); + xa_destroy(&mdata->dc_accepted_exts); +} + +/* Pretend to have some previous accepted extents */ +struct pre_ext_info { + u64 offset; + u64 length; +} pre_ext_info[] = { + { + .offset = SZ_128M, + .length = SZ_64M, + }, + { + .offset = SZ_256M, + .length = SZ_64M, + }, +}; + +static int devm_add_sent_extent(struct device *dev, u64 start, u64 length, + const char *tag, bool shared) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + + lockdep_assert_held(&mdata->ext_lock); + return __devm_add_extent(dev, &mdata->dc_sent_extents, start, length, + tag, shared); +} + +static int inject_prev_extents(struct device *dev, u64 base_dpa) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + int rc; + + dev_dbg(dev, "Adding %ld pre-extents for testing\n", + ARRAY_SIZE(pre_ext_info)); + + guard(mutex)(&mdata->ext_lock); + for (int i = 0; i < ARRAY_SIZE(pre_ext_info); i++) { + u64 ext_dpa = base_dpa + pre_ext_info[i].offset; + u64 ext_len = pre_ext_info[i].length; + + dev_dbg(dev, "Adding pre-extent DPA:%#llx LEN:%#llx\n", + ext_dpa, ext_len); + + rc = devm_add_sent_extent(dev, ext_dpa, ext_len, "", false); + if (rc) { + dev_err(dev, "Failed to add pre-extent DPA:%#llx LEN:%#llx; %d\n", + ext_dpa, ext_len, rc); + return rc; + } + + rc = dc_accept_extent(dev, ext_dpa, ext_len); + if (rc) + return rc; + } + return 0; +} + +static int cxl_mock_dc_partition_setup(struct device *dev) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + u64 base_dpa = BASE_DYNAMIC_CAP_DPA; + u32 dsmad_handle = 0xFADE; + u64 decode_length = SZ_512M; + u64 block_size = SZ_512; + u64 length = SZ_512M; + int rc; + + mutex_init(&mdata->ext_lock); + xa_init(&mdata->dc_fm_extents); + xa_init(&mdata->dc_sent_extents); + xa_init(&mdata->dc_accepted_exts); + + rc = devm_add_action_or_reset(dev, release_dc_ext, mdata); + if (rc) + return rc; + + for (int i = 0; i < NUM_MOCK_DC_REGIONS; i++) { + struct cxl_dc_partition *part = &mdata->dc_partitions[i]; + + dev_dbg(dev, "Creating DC partition DC%d DPA:%#llx LEN:%#llx\n", + i, base_dpa, length); + + part->base = cpu_to_le64(base_dpa); + part->decode_length = cpu_to_le64(decode_length / + CXL_CAPACITY_MULTIPLIER); + part->length = cpu_to_le64(length); + part->block_size = cpu_to_le64(block_size); + part->dsmad_handle = cpu_to_le32(dsmad_handle); + dsmad_handle++; + + rc = inject_prev_extents(dev, base_dpa); + if (rc) { + dev_err(dev, "Failed to add pre-extents for DC%d\n", i); + return rc; + } + + base_dpa += decode_length; + } + + return 0; +} + static int mock_gsl(struct cxl_mbox_cmd *cmd) { if (cmd->size_out < sizeof(mock_gsl_payload)) @@ -1582,6 +1860,192 @@ static int mock_get_supported_features(struct cxl_mockmem_data *mdata, return 0; } +static int mock_get_dc_config(struct device *dev, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_dc_config_in *dc_config = cmd->payload_in; + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + u8 partition_requested, partition_start_idx, partition_ret_cnt; + struct cxl_mbox_get_dc_config_out *resp; + int i; + + partition_requested = min(dc_config->partition_count, NUM_MOCK_DC_REGIONS); + + if (cmd->size_out < struct_size(resp, partition, partition_requested)) + return -EINVAL; + + memset(cmd->payload_out, 0, cmd->size_out); + resp = cmd->payload_out; + + partition_start_idx = dc_config->start_partition_index; + partition_ret_cnt = 0; + for (i = 0; i < NUM_MOCK_DC_REGIONS; i++) { + if (i >= partition_start_idx) { + memcpy(&resp->partition[partition_ret_cnt], + &mdata->dc_partitions[i], + sizeof(resp->partition[partition_ret_cnt])); + partition_ret_cnt++; + } + } + resp->avail_partition_count = NUM_MOCK_DC_REGIONS; + resp->partitions_returned = i; + + dev_dbg(dev, "Returning %d dc partitions\n", partition_ret_cnt); + return 0; +} + +static int mock_get_dc_extent_list(struct device *dev, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_extent_out *resp = cmd->payload_out; + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_mbox_get_extent_in *get = cmd->payload_in; + u32 total_avail = 0, total_ret = 0; + struct cxl_extent_data *ext; + u32 ext_count, start_idx; + unsigned long i; + + ext_count = le32_to_cpu(get->extent_cnt); + start_idx = le32_to_cpu(get->start_extent_index); + + memset(resp, 0, sizeof(*resp)); + + guard(mutex)(&mdata->ext_lock); + /* + * Total available needs to be calculated and returned regardless of + * how many can actually be returned. + */ + xa_for_each(&mdata->dc_accepted_exts, i, ext) + total_avail++; + + if (start_idx > total_avail) + return -EINVAL; + + xa_for_each(&mdata->dc_accepted_exts, i, ext) { + if (total_ret >= ext_count) + break; + + if (total_ret >= start_idx) { + resp->extent[total_ret].start_dpa = + cpu_to_le64(ext->dpa_start); + resp->extent[total_ret].length = + cpu_to_le64(ext->length); + memcpy(&resp->extent[total_ret].uuid, ext->uuid, + sizeof(resp->extent[total_ret])); + total_ret++; + } + } + + resp->returned_extent_count = cpu_to_le32(total_ret); + resp->total_extent_count = cpu_to_le32(total_avail); + resp->generation_num = cpu_to_le32(mdata->dc_ext_generation); + + dev_dbg(dev, "Returning %d extents of %d total\n", + total_ret, total_avail); + + return 0; +} + +static void dc_clear_sent(struct device *dev) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_extent_data *ext; + unsigned long index; + + lockdep_assert_held(&mdata->ext_lock); + + /* Any extents not accepted must be cleared */ + xa_for_each(&mdata->dc_sent_extents, index, ext) { + dev_dbg(dev, "Host rejected extent %#llx\n", ext->dpa_start); + xa_erase(&mdata->dc_sent_extents, ext->dpa_start); + } +} + +static int mock_add_dc_response(struct device *dev, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_mbox_dc_response *req = cmd->payload_in; + u32 list_size = le32_to_cpu(req->extent_list_size); + + guard(mutex)(&mdata->ext_lock); + for (int i = 0; i < list_size; i++) { + u64 start = le64_to_cpu(req->extent_list[i].dpa_start); + u64 length = le64_to_cpu(req->extent_list[i].length); + int rc; + + rc = dc_accept_extent(dev, start, length); + if (rc) + return rc; + } + + dc_clear_sent(dev); + return 0; +} + +static void dc_delete_extent(struct device *dev, unsigned long long start, + unsigned long long length) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + unsigned long long end = start + length; + struct cxl_extent_data *ext; + unsigned long index; + + dev_dbg(dev, "Deleting extent at %#llx len:%#llx\n", start, length); + + guard(mutex)(&mdata->ext_lock); + xa_for_each(&mdata->dc_fm_extents, index, ext) { + u64 extent_end = ext->dpa_start + ext->length; + + /* + * Any extent which 'touches' the released delete range will be + * removed. + */ + if ((start <= ext->dpa_start && ext->dpa_start < end) || + (start <= extent_end && extent_end < end)) + xa_erase(&mdata->dc_fm_extents, ext->dpa_start); + } + + /* + * If the extent was accepted let it be for the host to drop + * later. + */ +} + +static int release_accepted_extent(struct device *dev, u64 start, u64 length) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_extent_data *ext; + + guard(mutex)(&mdata->ext_lock); + ext = find_create_ext(dev, &mdata->dc_accepted_exts, start, length); + if (!ext) { + dev_err(dev, "Extent %#llx not in accepted state\n", start); + return -EINVAL; + } + xa_erase(&mdata->dc_accepted_exts, ext->dpa_start); + mdata->dc_ext_generation++; + + return 0; +} + +static int mock_dc_release(struct device *dev, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_dc_response *req = cmd->payload_in; + u32 list_size = le32_to_cpu(req->extent_list_size); + + for (int i = 0; i < list_size; i++) { + u64 start = le64_to_cpu(req->extent_list[i].dpa_start); + u64 length = le64_to_cpu(req->extent_list[i].length); + + dev_dbg(dev, "Extent %#llx released by host\n", start); + release_accepted_extent(dev, start, length); + } + + return 0; +} + static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd) { @@ -1673,6 +2137,18 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: rc = mock_get_supported_features(mdata, cmd); break; + case CXL_MBOX_OP_GET_DC_CONFIG: + rc = mock_get_dc_config(dev, cmd); + break; + case CXL_MBOX_OP_GET_DC_EXTENT_LIST: + rc = mock_get_dc_extent_list(dev, cmd); + break; + case CXL_MBOX_OP_ADD_DC_RESPONSE: + rc = mock_add_dc_response(dev, cmd); + break; + case CXL_MBOX_OP_RELEASE_DC: + rc = mock_dc_release(dev, cmd); + break; case CXL_MBOX_OP_GET_FEATURE: rc = mock_get_feature(mdata, cmd); break; @@ -1755,6 +2231,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) return -ENOMEM; dev_set_drvdata(dev, mdata); + rc = cxl_mock_dc_partition_setup(dev); + if (rc) + return rc; + mdata->lsa = vmalloc(LSA_SIZE); if (!mdata->lsa) return -ENOMEM; @@ -1812,6 +2292,9 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) return rc; + if (cxl_dcd_supported(mds)) + cxl_configure_dcd(mds, &range_info); + rc = cxl_dpa_setup(cxlds, &range_info); if (rc) return rc; @@ -1936,11 +2419,281 @@ static ssize_t sanitize_timeout_store(struct device *dev, static DEVICE_ATTR_RW(sanitize_timeout); +/* Return if the proposed extent would break the test code */ +static bool new_extent_valid(struct device *dev, size_t new_start, + size_t new_len) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_extent_data *extent; + size_t new_end, i; + + if (!new_len) + return false; + + new_end = new_start + new_len; + + dev_dbg(dev, "New extent %zx-%zx\n", new_start, new_end); + + guard(mutex)(&mdata->ext_lock); + dev_dbg(dev, "Checking extents starts...\n"); + xa_for_each(&mdata->dc_fm_extents, i, extent) { + if (extent->dpa_start == new_start) + return false; + } + + dev_dbg(dev, "Checking sent extents starts...\n"); + xa_for_each(&mdata->dc_sent_extents, i, extent) { + if (extent->dpa_start == new_start) + return false; + } + + dev_dbg(dev, "Checking accepted extents starts...\n"); + xa_for_each(&mdata->dc_accepted_exts, i, extent) { + if (extent->dpa_start == new_start) + return false; + } + + return true; +} + +struct cxl_test_dcd { + uuid_t id; + struct cxl_event_dcd rec; +} __packed; + +struct cxl_test_dcd dcd_event_rec_template = { + .id = CXL_EVENT_DC_EVENT_UUID, + .rec = { + .hdr = { + .length = sizeof(struct cxl_test_dcd), + }, + }, +}; + +static int log_dc_event(struct cxl_mockmem_data *mdata, enum dc_event type, + u64 start, u64 length, const char *tag_str, bool more) +{ + struct device *dev = mdata->mds->cxlds.dev; + struct cxl_test_dcd *dcd_event; + + dev_dbg(dev, "mock device log event %d\n", type); + + dcd_event = devm_kmemdup(dev, &dcd_event_rec_template, + sizeof(*dcd_event), GFP_KERNEL); + if (!dcd_event) + return -ENOMEM; + + dcd_event->rec.flags = 0; + if (more) + dcd_event->rec.flags |= CXL_DCD_EVENT_MORE; + dcd_event->rec.event_type = type; + dcd_event->rec.extent.start_dpa = cpu_to_le64(start); + dcd_event->rec.extent.length = cpu_to_le64(length); + memcpy(dcd_event->rec.extent.uuid, tag_str, + min(sizeof(dcd_event->rec.extent.uuid), + strlen(tag_str))); + + mes_add_event(mdata, CXL_EVENT_TYPE_DCD, + (struct cxl_event_record_raw *)dcd_event); + + /* Fake the irq */ + cxl_mem_get_event_records(mdata->mds, CXLDEV_EVENT_STATUS_DCD); + + return 0; +} + +static void mark_extent_sent(struct device *dev, unsigned long long start) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + struct cxl_extent_data *ext; + + guard(mutex)(&mdata->ext_lock); + ext = xa_erase(&mdata->dc_fm_extents, start); + if (xa_insert(&mdata->dc_sent_extents, ext->dpa_start, ext, GFP_KERNEL)) + dev_err(dev, "Failed to mark extent %#llx sent\n", ext->dpa_start); +} + +/* + * Format ::: + * + * start and length must be a multiple of the configured partition block size. + * Tag can be any string up to 16 bytes. + * + * Extents must be exclusive of other extents + * + * If the more flag is specified it is expected that an additional extent will + * be specified without the more flag to complete the test transaction with the + * host. + */ +static ssize_t __dc_inject_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count, + bool shared) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + unsigned long long start, length, more; + char *len_str, *uuid_str, *more_str; + size_t buf_len = count; + int rc; + + char *start_str __free(kfree) = kstrdup(buf, GFP_KERNEL); + if (!start_str) + return -ENOMEM; + + len_str = strnchr(start_str, buf_len, ':'); + if (!len_str) { + dev_err(dev, "Extent failed to find len_str: %s\n", start_str); + return -EINVAL; + } + + *len_str = '\0'; + len_str += 1; + buf_len -= strlen(start_str); + + uuid_str = strnchr(len_str, buf_len, ':'); + if (!uuid_str) { + dev_err(dev, "Extent failed to find uuid_str: %s\n", len_str); + return -EINVAL; + } + *uuid_str = '\0'; + uuid_str += 1; + + more_str = strnchr(uuid_str, buf_len, ':'); + if (!more_str) { + dev_err(dev, "Extent failed to find more_str: %s\n", uuid_str); + return -EINVAL; + } + *more_str = '\0'; + more_str += 1; + + if (kstrtoull(start_str, 0, &start)) { + dev_err(dev, "Extent failed to parse start: %s\n", start_str); + return -EINVAL; + } + + if (kstrtoull(len_str, 0, &length)) { + dev_err(dev, "Extent failed to parse length: %s\n", len_str); + return -EINVAL; + } + + if (kstrtoull(more_str, 0, &more)) { + dev_err(dev, "Extent failed to parse more: %s\n", more_str); + return -EINVAL; + } + + if (!new_extent_valid(dev, start, length)) + return -EINVAL; + + rc = devm_add_fm_extent(dev, start, length, uuid_str, shared); + if (rc) { + dev_err(dev, "Failed to add extent DPA:%#llx LEN:%#llx; %d\n", + start, length, rc); + return rc; + } + + mark_extent_sent(dev, start); + rc = log_dc_event(mdata, DCD_ADD_CAPACITY, start, length, uuid_str, more); + if (rc) { + dev_err(dev, "Failed to add event %d\n", rc); + return rc; + } + + return count; +} + +static ssize_t dc_inject_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + return __dc_inject_extent_store(dev, attr, buf, count, false); +} +static DEVICE_ATTR_WO(dc_inject_extent); + +static ssize_t dc_inject_shared_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + return __dc_inject_extent_store(dev, attr, buf, count, true); +} +static DEVICE_ATTR_WO(dc_inject_shared_extent); + +static ssize_t __dc_del_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count, + enum dc_event type) +{ + struct cxl_mockmem_data *mdata = dev_get_drvdata(dev); + unsigned long long start, length; + char *len_str; + int rc; + + char *start_str __free(kfree) = kstrdup(buf, GFP_KERNEL); + if (!start_str) + return -ENOMEM; + + len_str = strnchr(start_str, count, ':'); + if (!len_str) { + dev_err(dev, "Failed to find len_str: %s\n", start_str); + return -EINVAL; + } + *len_str = '\0'; + len_str += 1; + + if (kstrtoull(start_str, 0, &start)) { + dev_err(dev, "Failed to parse start: %s\n", start_str); + return -EINVAL; + } + + if (kstrtoull(len_str, 0, &length)) { + dev_err(dev, "Failed to parse length: %s\n", len_str); + return -EINVAL; + } + + dc_delete_extent(dev, start, length); + + if (type == DCD_FORCED_CAPACITY_RELEASE) + dev_dbg(dev, "Forcing delete of extent %#llx len:%#llx\n", + start, length); + + rc = log_dc_event(mdata, type, start, length, "", false); + if (rc) { + dev_err(dev, "Failed to add event %d\n", rc); + return rc; + } + + return count; +} + +/* + * Format : + */ +static ssize_t dc_del_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + return __dc_del_extent_store(dev, attr, buf, count, + DCD_RELEASE_CAPACITY); +} +static DEVICE_ATTR_WO(dc_del_extent); + +static ssize_t dc_force_del_extent_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + return __dc_del_extent_store(dev, attr, buf, count, + DCD_FORCED_CAPACITY_RELEASE); +} +static DEVICE_ATTR_WO(dc_force_del_extent); + static struct attribute *cxl_mock_mem_attrs[] = { &dev_attr_security_lock.attr, &dev_attr_event_trigger.attr, &dev_attr_fw_buf_checksum.attr, &dev_attr_sanitize_timeout.attr, + &dev_attr_dc_inject_extent.attr, + &dev_attr_dc_inject_shared_extent.attr, + &dev_attr_dc_del_extent.attr, + &dev_attr_dc_force_del_extent.attr, NULL }; ATTRIBUTE_GROUPS(cxl_mock_mem);