From patchwork Tue Oct 8 04:43:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826537 Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A14841DF733 for ; Tue, 8 Oct 2024 14:00:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396052; cv=none; b=BGwQW9XI03/fiKGCg4ICrO+CHWdvgOhNUbsqd8c4B694UsExpmwGAbJu6vGuKpDJgZ9iQibMJJuZ6zGZgc/npvSoQUJF3K3iitBlJWD9yN0F8R6iW5pP56LkrvlsSG4kYnTH7GZRIZ3t5CSaYjQHBFRudQGWBU63Tqe6L11pzBY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396052; c=relaxed/simple; bh=f24xe0s0P5sHWBZEyg+EVqdFYYoHXDXlETSsuM55DQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o9saPZJ18VvMMR2Rfs+nlVzH9+sga00oFa+iaAVu/4hNPwq00fY7TF/P6hidXhlGV/cnfXw67CX8IR00SEhyeZWxcQjjtICiDG2zd6UQSXWB7nJBrTGSixPcmQebFXP3W12fj+yuhEbmRtekBrSMoJSrQEP8mZ0BNsYMDIwMztI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=e6ygE81/; arc=none smtp.client-ip=209.85.219.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="e6ygE81/" Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-6cbc14b3b08so4794656d6.3 for ; Tue, 08 Oct 2024 07:00:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396049; x=1729000849; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tYVH9bPzhptOKAKA155SqTUHRG9AKR7xI41sAuZER68=; b=e6ygE81/SJshv+LRjedykthxtW3UO3MvQQc00UzO3sItolCnpOGN/CSJR5UUHhjz10 Nm5WpL0LLBo6diEBIPcZpQMDfxyxadhYnnJTnwajnhBbQP/Wyojzuwon6eD9u9lKi/7s v44KfQnKGIjvZYmX0VdJPxp556wG2i4m8beb8zW33jnXZD5NA78ndbvH6EpxaEj5k4Mz EP1kaafWBz7PFLM4l4kGgBqx8oR0G3kOPrEbjXD3r15/TyujmDLQ7OQ4PPZV5A/rCC+u isoSoXleZeyU08ov//WsMKjkzPY08ZRmsG5kSOfaXb6k83/0SMmrN/8aNpAunZvyxs0p v8iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396049; x=1729000849; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tYVH9bPzhptOKAKA155SqTUHRG9AKR7xI41sAuZER68=; b=tbmIDNMYuu3ml+yov0znMnMDSb1e0xsgZRwuiVmyYS4CNJba+lZf8c+l5KodAMGlDP 8kJN+kWTLCGERFxUn2LFlcks8pyD53Nn9TLnSMHGFodNQzFUV/h7M/1FFwBzIAG/ZOA5 9zpfWE3bjRxhnUJJADlWR6pE0FCrAPF/D6XKrLcrtxT5e21PU0Plv8Eyv8BRlk3ImRgC mWxcNv5yStxxCFI72pVOyz/O3hExWwwuCchWrcva3egM8oJq/KdnBT5sN6NGXXZ8EvGe oXcbmxfad0ec/ArJjhq9fhkb4ct0YjIQXLcKpN2MSbsIDrj/wDqnYUSfQNK7HvObuaD8 CgMQ== X-Forwarded-Encrypted: i=1; AJvYcCVFlUzXfa6Dn8A6ksMI1rdl593h2Y/LqqT16FJ7knOviXlbEuvMcpqgOnw27pLBy7qRf8vTOv0TpM3P@vger.kernel.org X-Gm-Message-State: AOJu0YwtZKcRYUW+ZR7kfPZcN9NWWEQod2OtdW8b8VhkZhiMpy/xUdTs LA2LR8AHSkIq8jnrlqg352a6I9c38EMJKgQVotm3yQXbPq6sCqY7QlFBbUUQabw= X-Google-Smtp-Source: AGHT+IH1G8ITusE7F8TZWngF/LW6FSnDnTSAvzMp2nN4Kfl3aRMl8OFWOkAOoxtA/bT+IIYMoMEjDg== X-Received: by 2002:a05:6214:2b91:b0:6cb:5eea:8a3e with SMTP id 6a1803df08f44-6cb9a2ed3d2mr210771516d6.10.1728396048953; Tue, 08 Oct 2024 07:00:48 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:48 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 1/3] memory: extern memory_block_size_bytes and set_memory_block_size_order Date: Tue, 8 Oct 2024 00:43:53 -0400 Message-ID: <20241008044355.4325-2-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On CXL systems, block alignment may be as small as 256MB, which may require a resize of the block size during initialization. This is done in the ACPI driver, so the resize function need to be made available. Presently, only x86 provides the functionality to resize memory block sizes. Wire up a weak stub for set_memory_block_size_order, similar to memory_block_size_bytes, that simply returns -ENODEV. Since set_memory_block_size_order is now extern, we also need to drop the __init macro. Signed-off-by: Gregory Price --- arch/x86/mm/init_64.c | 2 +- drivers/base/memory.c | 6 ++++++ include/linux/memory.h | 4 ++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ff253648706f..6086f99449fa 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1424,7 +1424,7 @@ void mark_rodata_ro(void) /* Adjustable memory block size */ static unsigned long set_memory_block_size; -int __init set_memory_block_size_order(unsigned int order) +int set_memory_block_size_order(unsigned int order) { unsigned long size = 1UL << order; diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 67858eeb92ed..f9045642f69e 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -110,6 +110,12 @@ static void memory_block_release(struct device *dev) kfree(mem); } +int __weak set_memory_block_size_order(unsigned int order) +{ + return -ENODEV; +} +EXPORT_SYMBOL_GPL(set_memory_block_size_order); + unsigned long __weak memory_block_size_bytes(void) { return MIN_MEMORY_BLOCK_SIZE; diff --git a/include/linux/memory.h b/include/linux/memory.h index c0afee5d126e..c57706178354 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -86,8 +86,8 @@ struct memory_block { }; int arch_get_memory_phys_device(unsigned long start_pfn); -unsigned long memory_block_size_bytes(void); -int set_memory_block_size_order(unsigned int order); +extern unsigned long memory_block_size_bytes(void); +extern int set_memory_block_size_order(unsigned int order); /* These states are exposed to userspace as text strings in sysfs */ #define MEM_ONLINE (1<<0) /* exposed to userspace */ From patchwork Tue Oct 8 04:43:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826538 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E63E91DF99F for ; Tue, 8 Oct 2024 14:00:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396054; cv=none; b=pgW+HH0iL5kCLr6XLkY9x2cRkIYlQ/WKLKv14nJPrNqTXVA4jW/L90+eEsal8Ruz2n18nGhJ9XUCIxjfZ+URGGV8FYKPs4WzjxYLsq7AO07g24BQyOCOnqw2Fq7Z/8oLOr0ssj0dEbA83qQt87XNHp7qi6mTybhYQJdjjuRGqJg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396054; c=relaxed/simple; bh=D9ssUjGQ6h7QeJF1IwpdOtgRJ/SdyRy/lVBrG03bkKQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I2uJoy6tMabM1O0e3MhRxspw2dkudSDlWbANvt+mieDVTTct8i/P0Xt7vyKKKTUxEJeqxf1lxdZKMvHTgYFBCFGGbkNbsM/eSMzR+WolP9UUOexTEIpOiPoYBcIDLEjqFqrAxQa8kt034n8owhsfydWm8SNagp3pn3jF79+ySBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=StN/9/9V; arc=none smtp.client-ip=209.85.219.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="StN/9/9V" Received: by mail-qv1-f46.google.com with SMTP id 6a1803df08f44-6cbc7c77c99so249386d6.3 for ; Tue, 08 Oct 2024 07:00:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396052; x=1729000852; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iOmfBB2o+BmwtYmn6lEeWqW0gfRN00aP7+NLXlUlw00=; b=StN/9/9VHUbqA+DqeHGKDKHwt6Tsn/zsnYDAfaM8fwy3k1NlJIgr/Q9LzY0ZoRNyXq +uMPnOBFTgKREJeABH/Ls/dKUkbsRLvsp5u57mnOTSd+WmANTIwUJSU9/cwt8/TkUEm9 8UBlw8NHNliC/5UQPovW+V33uUdJ73g04pTLC85V0HkT2KBIkxOEhEDFU50ataaOHFYB B/rkDpnpdrAUUCPx5rQYvweU2utBM9qTOcXJayRmh1aH+lQD2RcDsC5/b/0apCqRf8TB elOmMdTbePZRr8x8/8cxOv6EVP5RwnMzsKreKWtQcwlL6E49h/krnxfDcw847ZB2UP/m JtOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396052; x=1729000852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iOmfBB2o+BmwtYmn6lEeWqW0gfRN00aP7+NLXlUlw00=; b=LSWJJtUeASXCZxRamYTmwvKu4sUD1PubQgZEqYH9oAnRN0AJ+lVfKY7zfNWDU1hBYZ Sq51UwSwA2YSunQwZxq7wC31eHgKkKq6ZUlZSA1m+FTmoCq5GKuuRRPEs2IBa+1zunuS LRyZ667Adx4IkP37wp4yCIR6Mb2lIlGh1D6H4OTDOv2RysL43fCVuOGF7EGaTw2lZjau K0Dj5BxXGZQysn0BZLLnFo4ISkKjEwBfBkFjiG9TIMgcMqrMd2FrdU7dwWgno5b3IgnD nxFU8hTgPx1WuGsb9IMlubBxLwjV/nkX1uASUOyClgbKk8cLU2Q33W8IwoHFMTMg5Fru QwGw== X-Forwarded-Encrypted: i=1; AJvYcCX1wfpOlkQc0TCfVjEuLFxHltEogCwr46YuUKcbA9IjGtB/bk1xwMXDrqW5o+JgNRixwYmJ2Emf7RY3@vger.kernel.org X-Gm-Message-State: AOJu0YyXoTkt1z34abo4gXc+p+/8HGmf7t+jsyxN1vf0/yZArkrnqJ85 TmIF82TkIutYwdoszf4mpWiyN7v6IF3Fktb2LU8Ihd7Aljjy8+1PCXK3axAdnuQ= X-Google-Smtp-Source: AGHT+IHe1DIW8UN+mP+xqS2Oh3mXbQusW6tw3/R+BVrHed7J8ntdmkY9QJ5q2+XDvRLblwgEqMcXNQ== X-Received: by 2002:a05:6214:3b85:b0:6cb:c745:4f44 with SMTP id 6a1803df08f44-6cbc745517dmr10453496d6.26.1728396051760; Tue, 08 Oct 2024 07:00:51 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:50 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 2/3] x86/mm: if memblock size is adjusted, update the cached value Date: Tue, 8 Oct 2024 00:43:54 -0400 Message-ID: <20241008044355.4325-3-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When parsing CFMWS, we need to know the currently registered memory block size to avoid accidentally adjusting the size upward. Querying the size causes it to be cached in a static global. Update the static global if the value is subsequently updated. Print a warning that this has occurred for debugging purposes if the memory block size is changed at an unexpected time (post-init). Signed-off-by: Gregory Price --- arch/x86/mm/init_64.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 6086f99449fa..733dfa899232 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1424,6 +1424,7 @@ void mark_rodata_ro(void) /* Adjustable memory block size */ static unsigned long set_memory_block_size; +static unsigned long memory_block_size_probed; int set_memory_block_size_order(unsigned int order) { unsigned long size = 1UL << order; @@ -1432,6 +1433,19 @@ int set_memory_block_size_order(unsigned int order) return -EINVAL; set_memory_block_size = size; + + /* + * If the block size has already been probed, we need to change it. + * This can happen during ACPI/CFMWS parsing, since it needs to + * probe the system for the existing block size to avoid increasing + * the block size if lower memory happens to be misaligned + */ + if (memory_block_size_probed) { + memory_block_size_probed = size; + pr_warn("x86/mm: Memory block size changed: %ldMB\n", + size >> 20); + } + return 0; } @@ -1471,7 +1485,6 @@ static unsigned long probe_memory_block_size(void) return bz; } -static unsigned long memory_block_size_probed; unsigned long memory_block_size_bytes(void) { if (!memory_block_size_probed) From patchwork Tue Oct 8 04:43:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826539 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFB511DFDA5 for ; Tue, 8 Oct 2024 14:00:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396056; cv=none; b=BEpGGyzGLD/GQBUsmm3XSCj93Bd9QzNptdIWvc62fFcmsKoUGKM4kuQEbZnC+o+dp3xWj9fbYLAdxrSe8KSFQ/5ZZpLWXEhjjgCzSurLnPgkljrfR4mGOnJruzTl/PR3jICL1WzSnppvIZZBRHP2PZpRqsD9h8JUEoBDbAHjceM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396056; c=relaxed/simple; bh=6Fojn6KYi9Rnmp9EMOv/t6rJ9pIb1bMhH4aerHdoOZ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fOFhHpftm4YXNo/azG7Oi6z4nWIuU5ilhhG6/+so97SXJFE0XxqgtNrWxCmxF1rZvkUx9nAb1fxeCRDTe2rRr0w93VuL/jOM2IlgUrWkZ05szTKj8l/IG54dh8j399w1vmWHPCkBiOmOWe+27ExcqswFdAyxkI4rU+bcdhngtFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Oh0u6p98; arc=none smtp.client-ip=209.85.219.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Oh0u6p98" Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6cbc28f6d10so3833386d6.2 for ; Tue, 08 Oct 2024 07:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396053; x=1729000853; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=Oh0u6p98IfMFdV7CJ8/g+chEZT7jYwu/BAhJrvfPUlT4LhtEP8Nk1lKm+3U0I5/BvG Z9pDeqRaYWPHV7bvCCpId5IBT4XPM7SnbzKO2BK+RZOK9XmhQHtz7rGEU5pqGsk++q5U OSczRhi3xWZqjA4roT86+fvUGMJRkno/6vDP96SK6xy9ASf2ZtX4qsLixgU8hnxSWY0I OOBU5DGtBfv0k5TO31H+mfMeznnFznlXi2c0rQ0LeSTgsx+lzYka8vTcrAyVCz579Tum GyNjuGbdcUJvzGtEk4HmXjMbJIuvlj+QKHxfD3nVQ+DIs9l46cB6CbD08u4imORWydvq uK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396053; x=1729000853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=qOl23toZFIy2HEeOXEV3H1dL0nT7eWpfeSJBrOEJe95CMtMPdweOfsmSRIkD81p6+W RvXZ0ya7Tzkbd+w5xgSss8fxjlU9Gpvl6qHOey6jqHY9uONf4o4S7q8Cwthe61IiipkX kg1HNrB/pEjRtx0IplRbm6SQvbwjrEO81k+iuEH60j0ZtyySiW4ai64WEnJvzdIqLeiN qp4a2aM52TYsWryt0LwR2bOicPzOfle/Ji+W1ZUnXsXf27QDtERO+gJRHHIpayi8Bv6Z M8GYH8du7GGWshgcWy201lZb/mt3hW0Ls0gFW1QYQhk7w+7P5KtBaK6oT7QEj60jj2Nb Y38w== X-Forwarded-Encrypted: i=1; AJvYcCWIshPAcdm0IS/opHspHb8fNjEFUAVUEe+tyShmxAY9CqRPBs74dHixCJ3X2w5lBTKpDA9cNZFT5ZoY@vger.kernel.org X-Gm-Message-State: AOJu0YwsAztwASxYUkO8B3YBc7A6GkRFSy2u7RgAzLXfFjiSS2cmAYLg 4dXVCqDHeEeXSj9W70bJsT39BoBRqBWVnjyA1qwYHv0lqBS02M+om+VPHsgVVE4= X-Google-Smtp-Source: AGHT+IEWQaL+bYEjt3RyVfxhNXMy9Cz/JlOtCh7gn34GRctxubICCfzpYSUr2ZJbwepGsgKWAr958Q== X-Received: by 2002:a0c:e5c4:0:b0:6cb:a705:c05e with SMTP id 6a1803df08f44-6cba705c0aamr170960556d6.6.1728396053465; Tue, 08 Oct 2024 07:00:53 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:53 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 3/3] acpi,srat: reduce memory block size if CFMWS has a smaller alignment Date: Tue, 8 Oct 2024 00:43:55 -0400 Message-ID: <20241008044355.4325-4-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The CXL Fixed Memory Window allows for memory aligned down to the size of 256MB. However, by default on x86, memory blocks increase in size as total System RAM capacity increases. On x86, this caps out at 2G when 64GB of System RAM is reached. When the CFMWS regions are not aligned to memory block size, this results in lost capacity on either side of the alignment. Parse all CFMWS to detect the largest common denomenator among all regions, and reduce the block size accordingly. This can only be done when MEMORY_HOTPLUG and SPARSEMEM configs are enabled, but the surrounding code may not necessarily require these configs, so build accordingly. Suggested-by: Dan Williams Signed-off-by: Gregory Price --- drivers/acpi/numa/srat.c | 48 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c index 44f91f2c6c5d..9367d36eba9a 100644 --- a/drivers/acpi/numa/srat.c +++ b/drivers/acpi/numa/srat.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -333,6 +334,37 @@ acpi_parse_memory_affinity(union acpi_subtable_headers *header, return 0; } +#if defined(CONFIG_MEMORY_HOTPLUG) +/* + * CXL allows CFMW to be aligned along 256MB boundaries, but large memory + * systems default to larger alignments (2GB on x86). Misalignments can + * cause some capacity to become unreachable. Calculate the largest supported + * alignment for all CFMW to maximize the amount of mappable capacity. + */ +static int __init acpi_align_cfmws(union acpi_subtable_headers *header, + void *arg, const unsigned long table_end) +{ + struct acpi_cedt_cfmws *cfmws = (struct acpi_cedt_cfmws *)header; + u64 start = cfmws->base_hpa; + u64 size = cfmws->window_size; + unsigned long *fin_bz = arg; + unsigned long bz; + + for (bz = SZ_64T; bz >= SZ_256M; bz >>= 1) { + if (IS_ALIGNED(start, bz) && IS_ALIGNED(size, bz)) + break; + } + + /* Only adjust downward, we never want to increase block size */ + if (bz < *fin_bz && bz >= SZ_256M) + *fin_bz = bz; + else if (bz < SZ_256M) + pr_err("CFMWS: [BIOS BUG] base/size alignment violates spec\n"); + + return 0; +} +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, void *arg, const unsigned long table_end) { @@ -501,6 +533,10 @@ acpi_table_parse_srat(enum acpi_srat_type id, int __init acpi_numa_init(void) { int i, fake_pxm, cnt = 0; +#if defined(CONFIG_MEMORY_HOTPLUG) + unsigned long block_sz = memory_block_size_bytes(); + unsigned long cfmw_align = block_sz; +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ if (acpi_disabled) return -EINVAL; @@ -552,6 +588,18 @@ int __init acpi_numa_init(void) } last_real_pxm = fake_pxm; fake_pxm++; + +#if defined(CONFIG_MEMORY_HOTPLUG) + /* Calculate and set largest supported memory block size alignment */ + acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_align_cfmws, + &cfmw_align); + if (cfmw_align < block_sz && cfmw_align >= SZ_256M) { + if (set_memory_block_size_order(ffs(cfmw_align)-1)) + pr_warn("CFMWS: Unable to adjust memory block size\n"); + } +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + + /* Then parse and fill the numa nodes with the described memory */ acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_parse_cfmws, &fake_pxm);