From patchwork Mon Jan 27 15:34:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13951558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FC9CC02188 for ; Mon, 27 Jan 2025 15:34:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BB4B280169; Mon, 27 Jan 2025 10:34:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 96BE6280165; Mon, 27 Jan 2025 10:34:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83279280169; Mon, 27 Jan 2025 10:34:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 67299280165 for ; Mon, 27 Jan 2025 10:34:15 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1F6211C6FFC for ; Mon, 27 Jan 2025 15:34:15 +0000 (UTC) X-FDA: 83053628070.06.2898CD0 Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf26.hostedemail.com (Postfix) with ESMTP id 3CBDA140017 for ; Mon, 27 Jan 2025 15:34:13 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=NU7rBwtZ; spf=pass (imf26.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.48 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737992053; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=dHMo2pdCBxV+82YYOBuucA32rs979HfFZCyG5MwB/hE=; b=zPFuNkPBCvEecAHN+vl972dsf4lDbZMnDvOZD12Z8t/amYTzgt3sdLb4hEE8lYmSqKt/8h dtb/sYLVn5fqOo4IoDSCho64Us3E/kd3XksrEFfvw/qO/GkR11s77wl5/9fvOVEEOq1g20 rHe4FAp5LrYQDsTMyMjNIY5TSF13ThA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737992053; a=rsa-sha256; cv=none; b=nA4ymASBktNhrYZfh1MbNtMRwqxEsFY5+zcQjUvhKHWwO+6/gXnkDwiBhngShWfKwKvjF8 S+ZEewowIoD0UD8IHf/AGTI8hpsGpYJS5RMUEHM+zVCbEzhd98M2oSPQ5FYhXqHxCt7wpC vr5V6Jq0U2o3GagfzpC8Gbth3/fXAkw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=NU7rBwtZ; spf=pass (imf26.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.48 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6dd43b08674so40875336d6.3 for ; Mon, 27 Jan 2025 07:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1737992052; x=1738596852; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=dHMo2pdCBxV+82YYOBuucA32rs979HfFZCyG5MwB/hE=; b=NU7rBwtZnRSD6S2YVWQjJpEDeAdcz3T5AtHG4F8IvplVQJcHGd4492plDZJwhUADZI RhPTht8LjvwaCXNTtjk033u0Lhv3yOI6ecpcEQjpeCfR39D1DhGzRqfM5D1xLOe/m7CB JC0/L5pm3kdTAGwZnh5F9HEByNPz0FTabBw0sTzfR8g9TlVTw3SSYVYSsvyX0bONjljk 71gsRXDcVHvtudeNnonzWX4FWdeZ2NJMmZLCIroh30FniZLbSlzfXhL3YSD98SNf/suQ UJKY2TNCuoOx7pRPIMsm28D+q6UZ66713OEm2L2O42k7n7N9YabBJos7SLpWkjfltTWl O4xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737992052; x=1738596852; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dHMo2pdCBxV+82YYOBuucA32rs979HfFZCyG5MwB/hE=; b=Eumd257eDCk2OiEZU8Z0hypTuKxszQ+SOutba90F4vOS8TyhSwVmWkOXg/eCKsIEPS mx0CEErIsuPkfZHRc/YAIRrXj+a/j8x/dh/0hnv3jAA6Mx/zIYHdxaq0lmyZkduAQm2L 25G7vSrjkDmcgB2iWofy2dUUWoNQjJuJxf7vxNGuKhFe1WMy+kglq2Gkwt5WKC6MV46J +zl9aA6iE4Y6aD/XZ9QQWN7NKRLPlsuY9fO4QAxMAtw1K/UYtZheE2FESLLC6gnTZqFM dH/wFHARAhpHawYWA14B/bnwk+pOOvic111WdmBLaU6cDpib6G7uowXVUF9FHxInJ0mi sFbg== X-Gm-Message-State: AOJu0YwD+ONs0E12d7PESpSsmYE5/L5PJNX+G2D0QlJbRLFLn0eqf0Fc c/YTNC5fSoyhkzbVxzE/FSBFY7yZbgda/S47biHip0HKKUNNQAHW2cxfUED0JwbUKGWqPC0T9rI 5 X-Gm-Gg: ASbGncvbvGt/V0TsOas5KLMIbCZPBeh4lV9qdenc2fVUHYBz83IBr4xTSl5Ngho98Zj qP6lJXJxfcaPRXJrDg76IswK4I9RKHdHrs/MHozsgp1PlXi47j9U4IBZ5FP0j2dhdyWEIhtKF00 U2wsmU8ME0sJwRUI/TzqjZmBj9qpv6rMC13dEia82zWV4jTF4zQGW0woW25HS0YmheXvd4GGJKE gpKou7oah4lZuECNV/NWb2t4WbD/+0oxqTp27uVSMHDw1c9wAaBsNDFKwyHsjuVduHRdwFHyeAU DRuoN72Var3eEAP9D21WoeaUH433bCmQdWZLmSilzGliODtYrlkizeta5eTpYDEk/op8 X-Google-Smtp-Source: AGHT+IFS5f0oDJpQ1PqRj+UQN+nYxWLut917CGhIXe5wKYcWsl9bjwQofmwkzOyRAnQ+ap2ftR9Sig== X-Received: by 2002:ad4:5c68:0:b0:6cb:c54c:b782 with SMTP id 6a1803df08f44-6e1b222f390mr706562676d6.32.1737992051647; Mon, 27 Jan 2025 07:34:11 -0800 (PST) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6e2058c2a51sm35380476d6.109.2025.01.27.07.34.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 07:34:11 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: kernel-team@meta.com, x86@kernel.org, linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, rafael@kernel.org, lenb@kernel.org, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@huawei.com, alison.schofield@intel.com, rrichter@amd.com, rppt@kernel.org, gourry@gourry.net, bfaccini@nvidia.com, haibo1.xu@intel.com, dave.jiang@intel.com, Ira Weiny , Fan Ni Subject: [PATCH v8 0/3] memory,x86,acpi: hotplug memory alignment advisement Date: Mon, 27 Jan 2025 10:34:02 -0500 Message-ID: <20250127153405.3379117-1-gourry@gourry.net> X-Mailer: git-send-email 2.48.1 MIME-Version: 1.0 X-Stat-Signature: siu6diims1wy3swgr5e7w11gg8wy8hkh X-Rspam-User: X-Rspamd-Queue-Id: 3CBDA140017 X-Rspamd-Server: rspam03 X-HE-Tag: 1737992053-198612 X-HE-Meta: U2FsdGVkX19jdDImTPWn31XrB8Wi/+F1lHowQBMLabegrVsjbixMuq1/rOW2FsRGugzQ8Bpo9EogVyjqAmHZopJ2AulAwN/Wwf1GqHbGfL1klO7xAaP1tce+gc18x5xs7p2G36M1Jci4WAk9g8OFjXcUGFxk1RowJgjGhSuOapLAC9AxUf/9AHfpObHDoQvPz91wJfVUb7kALKWHaCOHAiF6oYkVfqPPqFSRUHy6SI5yx0Va77da6xkmqqx8Cp1JwIa9gaLC64P2WbRdMTnI0u/ME/GQ9VzjQ/e4+cYujbaZxabRclXfkmKQzL5HqhqCehKVvls8K9EWZ8zY75GQl/ioda2Qot9v4w9Ghr8rx+gabDlRQuqH89tsDIC5mox4fZ4JinVEeTW8rhCFRa8l6s1nr6y5fqrIOVlkFqUSqQEUBI4aw4HEWboitwI/Pd0qnL8ecQCUcPJTsit+9QLZBZDE+LBdYTZhS7T737o7hfw0VDozbiHb3u5dV5kC09kNa8VLWlyTrSp7aTP/juF3nmib40eRi7I30zyVUyz9qu5KdvbA9dXqv1dUPcwltVHegitjEIkQWlxIwS3bel5ikF/d+yNxeK63s9KJDKDCFZ0bBlMcpX9paf5omyYM0VVTa7SNGrVhORDP069QCMNajiPi60RE4r8/h26qzXxZtNzN+Db9vOWC7uGVIgZ5P03TCRSCFAYBO0T/xYvV34M9Nj3ZA1TB1Gkbk+D+0qZsZRUBUjwag2PCu67LEGWYeUESCfBCZs+FtV6/f2Ri32GU9122/FnMJJh2x3Ff7+75sjA4bKo5wxxlo3+GKMxG0GVLBS6v4gA3IZ7BK7BFZBa/nXzEg3d9M9tqVJXU1dnCgLaD38JZNE7EA0uyxZMUOX3EFwYqHnTNnLteWZVnRHuroOR4tKoX8W8DmW6IvWjrmBnTTsz5oLeLKP5CkeIMoNHGeLDXxpOW1gbkZcoGaX1 8Gqua8Z3 EeDaX2jhAYd6vv2Dss3kqbM/Z0KdPsAJwFXEBtSgV34K2kDPdyrYf1aJv1NkEhX1de+bw75QW7ToZBAcCT9Bj9RZIIYMbI0ojD/0ODH6XgsndSo/RYrmIHBuBaS1MNauDlTtwfPPtYx+leQJrxOqTt1G7JziotJyH5JbsCSXhBtyKFh8feIL9R8lA77+V9fuB9qkt5xLFnHw8cUNmG93TiUF3eN7eOoSEqU6lWIVT1N+8hY9H1hYuIHzQH+aXZMHncONRnlwwz0DT5YBk/LtsDF0VrNfAhRMdRUWzAXMPONhAPB99i7zowRy1kfUUJhNlPheyOeT7LLKsrta4vPnj7Rw+04kjQjWzLJHu4yTe0EYY2nG9bzQuHnmgsUVvgqTbd48qyJz8/P5gjKk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: v8: nits and tag pickups When physical address regions are not aligned to memory block size, the misaligned portion is lost (stranded capacity). Block size (min/max/selected) is architecture defined. Most architectures tend to use the minimum block size or some simplistic heurist. On x86, memory block size increases up to 2GB, and is otherwise fitted to the alignment of non-hotplug (i.e. not special purpose memory). CXL exposes its memory for management through the ACPI CEDT (CXL Early Detection Table) in a field called the CXL Fixed Memory Window. Per the CXL specification, this memory must be aligned to at least 256MB. When a CFMW aligns on a size less than the block size, this causes a loss of up to 2GB per CFMW on x86. It is not uncommon for CFMW to be allocated per-device - though this behavior is BIOS defined. This patch set provides 3 things: 1) implement advise/query functions in driverse/base/memory.c to report/query architecture agnostic hotplug block alignment advice. 2) update x86 memblock size logic to consider the hotplug advice 3) add code in acpi/numa/srat.c to report CFMW alignment advice The advisement interfaces are design to be called during arch_init code prior to allocator and smp_init. start_kernel will call these through setup_arch() (via acpi and mm/init_64.c on x86), which occurs prior to mm_core_init and smp_init - so no need for atomics. There's an attempt to signal callers to advise() that query has already occurred, but this is predicated on the notion that query actually occurs (which presently only happens on the x86 arch). This is to assist debugging future users. Otherwise, the advise() call has been marked __init to help static discovery of bad call times. Once query is called the first time, it will always return the same value. Interfaces return -EBUSY and 0 respectively on systems without hotplug. Suggested-by: Ira Weiny Suggested-by: David Hildenbrand Suggested-by: Dan Williams Acked-by: Mike Rapoport (Microsoft) Acked-by: David Hildenbrand Acked-by: Dan Williams Acked-by: Dave Hansen Signed-off-by: Gregory Price Tested-by: Fan Ni Gregory Price (3): memory: implement memory_block_advise/probe_max_size x86: probe memory block size advisement value during mm init acpi,srat: give memory block size advice based on CFMWS alignment arch/x86/mm/init_64.c | 15 ++++++++---- drivers/acpi/numa/srat.c | 12 ++++++++- drivers/base/memory.c | 53 ++++++++++++++++++++++++++++++++++++++++ include/linux/memory.h | 10 ++++++++ 4 files changed, 84 insertions(+), 6 deletions(-)