From patchwork Wed Oct 16 19:24:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13838791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4677D2A542 for ; Wed, 16 Oct 2024 19:25:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E17406B0098; Wed, 16 Oct 2024 15:25:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DC95F6B0099; Wed, 16 Oct 2024 15:25:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1C426B009A; Wed, 16 Oct 2024 15:25:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A12366B0098 for ; Wed, 16 Oct 2024 15:25:21 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 67F7280A78 for ; Wed, 16 Oct 2024 19:25:12 +0000 (UTC) X-FDA: 82680443622.07.E4C4AE0 Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com [209.85.217.43]) by imf03.hostedemail.com (Postfix) with ESMTP id EBF1E2000C for ; Wed, 16 Oct 2024 19:25:14 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=FVPy8fU5; dmarc=none; spf=pass (imf03.hostedemail.com: domain of gourry@gourry.net designates 209.85.217.43 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729106560; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HrmbhAeCdsnN6DuwhCdjgoaV0xg9bEsX7zAMN4X3gqU=; b=NTshcZ/GqMlaVDbvFBRyxdrbbVFjSGUmdS94AugcthtDFjIhMiX8OQxcdVnIhfyEHreGYy R0wouP1S+GG+zI1v/b7ANOBCLOxDiWQldOx8cVnhSMgzdL+7lQEDAwireQwchShYqKNHZ3 kauDb+1dxqbAQW5Sl+0vimk3+k/vYJY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729106560; a=rsa-sha256; cv=none; b=hCBz75zH7R8jS0cnhcBvzYeBEEO6qzwn4pEPr/JOJ9u9dGNGNpz5IbiDPZFynE1fr7NiOq d3ncsDlj3WHjRMaqmuuHDbaMsEcdozriE13RqQG3wXf3r0GrprB7gbgsv045SzqFns+ib2 TElkw4BE5B4A4ZOkJNPdAC0XZfz33DY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=FVPy8fU5; dmarc=none; spf=pass (imf03.hostedemail.com: domain of gourry@gourry.net designates 209.85.217.43 as permitted sender) smtp.mailfrom=gourry@gourry.net Received: by mail-vs1-f43.google.com with SMTP id ada2fe7eead31-4a47fd930b8so34800137.1 for ; Wed, 16 Oct 2024 12:25:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1729106718; x=1729711518; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HrmbhAeCdsnN6DuwhCdjgoaV0xg9bEsX7zAMN4X3gqU=; b=FVPy8fU5moubTTMmhsTOMYg6V/fu0cV2TWNP7kTygsJJpw0K9kvph4L8WuakPEcGsk RcxG1EHlDSmiK+N+CuIUBnZwwu/jhtnnULsxqUaOJ9mJljj3kvO1XWpWxLD9leL+FNBA AGApZuIsYStHZ4Xbldmdo1kY2VOpYEAFz9i0wVBZufovExCppbyrIcmdefaa+yxgBk5O eb1LKCLRfnv2JUOx5CcsybzqDkEOEJSNcJCGOmNBxK8yHC5sESuSL2ve75x4sNcMFy8q MI/xprXj+MIm0siQ/O2FBKXBEDwzS4qTJjtbbovxcnGOJixXpsg0/yM4RmrDM4ILf+9H 7GZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729106718; x=1729711518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HrmbhAeCdsnN6DuwhCdjgoaV0xg9bEsX7zAMN4X3gqU=; b=WxnrtWxV0htm4vVu38APik3wljcTt46HD9V/VzK+0zmRTxNyGP8nZYX4ohsIuFC1pO s3kGMGGZF8maYBgY0Tsbq/PsA0cWBsMvVV9RA0Xs/c/F6YNKQFw5M2gg7Ejg/yV0deZL W+0GWqpgp//VclS1WIUX5s2wBCA/73mxkv5lJ2XfcWSqj/0e3qlswhBaPjnqNZQwFiR+ kyoq4nv8eJ7SQl9vTECPoeYaB4zrj+dFQyGGC9+tHvAAM6Y0S6ddp1h7/Da0yY/EUxrs GwExK9gIZRLhVkuQj5RpSahsJL6NtMtumAAq5PlTdT/yff0RTxk4bJIh0ckuhmt3RC1q eacw== X-Forwarded-Encrypted: i=1; AJvYcCWWZhiP5y8yB1HMJutSzKrOMZJ69QhoFFo0Ae1shc1tQ/VE57fBfSwRNCq+XqsddwNKVZ/62PF9ig==@kvack.org X-Gm-Message-State: AOJu0YxtA2ZxmRqUFzoHmM1RWa6YMJ9VT/4bGSiLsg/m6OGd7uESTLoG w5cboFDjN8zZJV61vYEmNHjqeXbSexZiCN8xZgAS/IO+0wwHyksHUY3NB/oDYkk= X-Google-Smtp-Source: AGHT+IFClvWp9HyykvU9mzj84/mtwGebanX8vIYbiVaXquh13pEcA6fQ2yrNp4hn0Qu9cQWW9s3xDg== X-Received: by 2002:a05:6102:d92:b0:4a5:ba70:1c6e with SMTP id ada2fe7eead31-4a5ba702ba3mr3418941137.29.1729106718285; Wed, 16 Oct 2024 12:25:18 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4607b38ce69sm20271651cf.90.2024.10.16.12.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 12:25:17 -0700 (PDT) From: Gregory Price To: x86@kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, david@redhat.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, rafael@kernel.org, lenb@kernel.org, rppt@kernel.org, akpm@linux-foundation.org, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, rrichter@amd.com, ytcoode@gmail.com, haibo1.xu@intel.com, dave.jiang@intel.com Subject: [PATCH v2 3/3] acpi,srat: reduce memory block size if CFMWS has a smaller alignment Date: Wed, 16 Oct 2024 15:24:45 -0400 Message-ID: <20241016192445.3118-4-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241016192445.3118-1-gourry@gourry.net> References: <20241016192445.3118-1-gourry@gourry.net> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EBF1E2000C X-Stat-Signature: pgzmyu7ksbkdbranfi1ez7dccgzowhbc X-Rspam-User: X-HE-Tag: 1729106714-810287 X-HE-Meta: U2FsdGVkX18EIJ5hVQYbfABV9bYtiFp6v6n9JnVB7t371ZfPWtN6SD5Wctb9g+UcwMZZvciTB1XJCIIfc/CK14QvR9+8UVb9AZmd0j13hzs6oWJ6HE3opnZejp7QmYogRFgdAdhKW0YQ4SYU0YtEPoBC9J98USKkJwsncr6bfIGekXfjehJKLl1ouZeFZc2HsTW9gKopFQZex2DHbHG/+Rsf1ncPqAS/hwM6xpPykfOBYdNulaXxYC+QU7eT6GGTSwFexWQd/78MQIiSWWkdmaDvBdVx9kEvKy/HINpbYdhqdkZP34XSuI2e5agZGLFkB0/zAHtchjuG0EgO4ie7HkwDAQT4oqSQ8GBm7FUI09Wy4EVCoV9ikcCwcRPcb6snSB+D7wCYJPlZ5gbAbOsu7oT4krsZvAPrJX2zN22oTgmqYWViBw/seKdYJysWfTTTi0ZmWlDBcq+vEGNzv+Lk8FQ4KGrgZjzlpEkSeldvCTS8ugWzXfvB2UMSWbZjdBF1wKWRmllzdNs49zjiQVVTahSWu+hiV1tC1KZHjyqKWyGpW0/nww+ZOmED5QWfbzxpZBLTndggPhRk85PbdTnNqyNwSOYZZKK7C+NgvnKoczfM/wmvbAxDU/I4doH20lnbsh/Md/avxZtgmbldkBFHCOEY7gbyyKUpjRNHVbUng2XKUFQo0O8SeO2Cix4K9aD/dOBql//nmTWZ7Orv6+iP7dWPkV9FatZzESxH3D5GOZbJACnOoqcuF3RJkqFpWUnPL+fb7Uq0eKXMEY4HDjNec3aPHmrqb3JgEJdqV8x6GSyeqEQ03QxEyn/DbaTZQmi9vYIOlTssnPn/Eaoozmex3rEJT454OuWus5j9IYWQxyBA9M18G8i4oLMjkAAJORhpGulai3TyG+eL5YHOdC7ZMUIT0Ma/w/yygmyxupI3OXUIH8fqP79nvztDLf8xAKibhkrK+ffkW+w8U5nAOps dQI8KIKz rjceaI3DDQeAU1rcwJ03mt1CU55MBvDme5KFbPEh10S7t1XDruBHue4FUNzpG9MIkLNlfHXRY5/WLGIpnkScXP1ET/PrOM0B1BvgHLA+mnSaPHyuX4fW8EDaue+yKa5DoqeY0spTqI1USMxy5GA6mmkZSzrEIiIThBAlkfoRJ2RQizE3fgnV26o4DOKD1KxwnDjdeiyMHlmsTPCS07Dm42Zinap247Vmz5sstFI8P+QomNlJ/FD2yqwSTtkgmrqYevZ8AV15iaaFITK1QWDERFlefYlYUlnxWZD4PeZ1qu63LGn6KC5iZ0nOqt5T3XlZXCYMlIRyY8dv3S0A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The CXL Fixed Memory Window allows for memory aligned down to the size of 256MB. However, by default on x86, memory blocks increase in size as total System RAM capacity increases. On x86, this caps out at 2G when 64GB of System RAM is reached. When the CFMWS regions are not aligned to memory block size, this results in lost capacity on either side of the alignment. Parse all CFMWS to detect the largest common denomenator among all regions, and advise memblock to reduce the block size accordingly. Suggested-by: Dan Williams Signed-off-by: Gregory Price --- drivers/acpi/numa/srat.c | 42 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c index 44f91f2c6c5d..5fc03a99570e 100644 --- a/drivers/acpi/numa/srat.c +++ b/drivers/acpi/numa/srat.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -333,6 +334,35 @@ acpi_parse_memory_affinity(union acpi_subtable_headers *header, return 0; } +/* + * CXL allows CFMW to be aligned along 256MB boundaries, but large memory + * systems default to larger alignments (2GB on x86). Misalignments can + * cause some capacity to become unreachable. Calculate the largest supported + * alignment for all CFMW to maximize the amount of mappable capacity. + */ +static int __init acpi_align_cfmws(union acpi_subtable_headers *header, + void *arg, const unsigned long table_end) +{ + struct acpi_cedt_cfmws *cfmws = (struct acpi_cedt_cfmws *)header; + u64 start = cfmws->base_hpa; + u64 size = cfmws->window_size; + unsigned long *fin_bz = arg; + unsigned long bz; + + for (bz = SZ_64T; bz >= SZ_256M; bz >>= 1) { + if (IS_ALIGNED(start, bz) && IS_ALIGNED(size, bz)) + break; + } + + /* Only adjust downward, we never want to increase block size */ + if (bz < *fin_bz && bz >= SZ_256M) + *fin_bz = bz; + else if (bz < SZ_256M) + pr_err("CFMWS: [BIOS BUG] base/size alignment violates spec\n"); + + return 0; +} + static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, void *arg, const unsigned long table_end) { @@ -501,6 +531,7 @@ acpi_table_parse_srat(enum acpi_srat_type id, int __init acpi_numa_init(void) { int i, fake_pxm, cnt = 0; + unsigned long bz = SZ_64T; if (acpi_disabled) return -EINVAL; @@ -552,6 +583,17 @@ int __init acpi_numa_init(void) } last_real_pxm = fake_pxm; fake_pxm++; + + /* Calculate and set largest supported memory block size alignment */ + acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_align_cfmws, &bz); + if (bz >= SZ_256M) { + if (memblock_advise_size_order(ffs(bz)-1) < 0) + pr_warn("CFMWS: memblock size advise failed\n"); + else + pr_info("CFMWS: memblock advised size(%ld)\n", bz); + } + + /* Then parse and fill the numa nodes with the described memory */ acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_parse_cfmws, &fake_pxm);