From patchwork Tue Oct 8 04:43:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826541 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48AEB1DF244 for ; Tue, 8 Oct 2024 14:00:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396051; cv=none; b=AzJzl8G8ecm9W247WmqQV8N51fGRRB1zAt4AZjBAJitKVFBEDk1MUFWNsOo/JfqLZr44aKSpB4pIYAPccak69XvOj6UwpNKIEzclRyhD6UFAJ3LA+EbIr8ou2nol0v6ls2fope++wkMGdvnD75kzO3xVFs5mO1XjXn/WtcOVaSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396051; c=relaxed/simple; bh=f24xe0s0P5sHWBZEyg+EVqdFYYoHXDXlETSsuM55DQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N8/M389se3p3RNORadYZ97kHYtOnNnxFjqMBV8EEKZphU1Rv/MKNeaUZrngXsaMOYk6OuUUgd2HAEqD7N5QxjUr+Rb5QZrNs6XdMMv6YICtZQg4/jke+4CUCt7FXcZ8iQxLLVsTd2ZQd1iJ1/s0FwEGtJWYAhRt0gHn2TDpAp0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=e6ygE81/; arc=none smtp.client-ip=209.85.219.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="e6ygE81/" Received: by mail-qv1-f47.google.com with SMTP id 6a1803df08f44-6cb54eef205so51888666d6.2 for ; Tue, 08 Oct 2024 07:00:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396049; x=1729000849; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tYVH9bPzhptOKAKA155SqTUHRG9AKR7xI41sAuZER68=; b=e6ygE81/SJshv+LRjedykthxtW3UO3MvQQc00UzO3sItolCnpOGN/CSJR5UUHhjz10 Nm5WpL0LLBo6diEBIPcZpQMDfxyxadhYnnJTnwajnhBbQP/Wyojzuwon6eD9u9lKi/7s v44KfQnKGIjvZYmX0VdJPxp556wG2i4m8beb8zW33jnXZD5NA78ndbvH6EpxaEj5k4Mz EP1kaafWBz7PFLM4l4kGgBqx8oR0G3kOPrEbjXD3r15/TyujmDLQ7OQ4PPZV5A/rCC+u isoSoXleZeyU08ov//WsMKjkzPY08ZRmsG5kSOfaXb6k83/0SMmrN/8aNpAunZvyxs0p v8iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396049; x=1729000849; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tYVH9bPzhptOKAKA155SqTUHRG9AKR7xI41sAuZER68=; b=azAezD+sYUF+ZaoCWvT3bTKFW+fQZDNJbZfPI+48K7yYNaGlCMuAjMmH5a5ltIX9RX 1GzeZ3BI2uWLw1cVFKZGsPbgY1mpKzbIkSSO5z8Xm/aryE3dJUbwvsc0GXTUqkynbCIH TzvQXwRpz5A2YlPg5wZDwB4N3QiGEx/9J6/rSy9ZFsEpxWtlMwh1RxUpktkZztABDio1 eDSasRXoAhj/zYiDz+azE2wGFRGJAydpsnGWv0MDxkgq3VsI+D3yD54EjwXnPoBGIW0L pEa07Jsyf792tOhbJg1ULPnSUKcSlrMXz/hJMhiG20VewWWd5xF6UmIeFqywTE6ASkXq nt9g== X-Gm-Message-State: AOJu0YyWBpGgabUKCnYOFgGeTkhdNKttm4FyUaNoBZ2dSAiXJR1nsoPO Le+swW1LoU2rj2wFwcokq0VvxpzPGCCkM5Hoype3y0DbYkqjp2dAcw2BB4mPUAcOa/xxrViaG7L o X-Google-Smtp-Source: AGHT+IH1G8ITusE7F8TZWngF/LW6FSnDnTSAvzMp2nN4Kfl3aRMl8OFWOkAOoxtA/bT+IIYMoMEjDg== X-Received: by 2002:a05:6214:2b91:b0:6cb:5eea:8a3e with SMTP id 6a1803df08f44-6cb9a2ed3d2mr210771516d6.10.1728396048953; Tue, 08 Oct 2024 07:00:48 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:48 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 1/3] memory: extern memory_block_size_bytes and set_memory_block_size_order Date: Tue, 8 Oct 2024 00:43:53 -0400 Message-ID: <20241008044355.4325-2-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On CXL systems, block alignment may be as small as 256MB, which may require a resize of the block size during initialization. This is done in the ACPI driver, so the resize function need to be made available. Presently, only x86 provides the functionality to resize memory block sizes. Wire up a weak stub for set_memory_block_size_order, similar to memory_block_size_bytes, that simply returns -ENODEV. Since set_memory_block_size_order is now extern, we also need to drop the __init macro. Signed-off-by: Gregory Price --- arch/x86/mm/init_64.c | 2 +- drivers/base/memory.c | 6 ++++++ include/linux/memory.h | 4 ++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ff253648706f..6086f99449fa 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1424,7 +1424,7 @@ void mark_rodata_ro(void) /* Adjustable memory block size */ static unsigned long set_memory_block_size; -int __init set_memory_block_size_order(unsigned int order) +int set_memory_block_size_order(unsigned int order) { unsigned long size = 1UL << order; diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 67858eeb92ed..f9045642f69e 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -110,6 +110,12 @@ static void memory_block_release(struct device *dev) kfree(mem); } +int __weak set_memory_block_size_order(unsigned int order) +{ + return -ENODEV; +} +EXPORT_SYMBOL_GPL(set_memory_block_size_order); + unsigned long __weak memory_block_size_bytes(void) { return MIN_MEMORY_BLOCK_SIZE; diff --git a/include/linux/memory.h b/include/linux/memory.h index c0afee5d126e..c57706178354 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -86,8 +86,8 @@ struct memory_block { }; int arch_get_memory_phys_device(unsigned long start_pfn); -unsigned long memory_block_size_bytes(void); -int set_memory_block_size_order(unsigned int order); +extern unsigned long memory_block_size_bytes(void); +extern int set_memory_block_size_order(unsigned int order); /* These states are exposed to userspace as text strings in sysfs */ #define MEM_ONLINE (1<<0) /* exposed to userspace */ From patchwork Tue Oct 8 04:43:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826542 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F2AC1DFD84 for ; Tue, 8 Oct 2024 14:00:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396054; cv=none; b=TNLqo+SPzrTIa03nLKp/hZOLFViWlsv5LaQCtyoZ1ytj4L1vJGxgLpgBlu7zLihzJiBKTT3YXJ2kta5xu/gefvmdrKqi/DI/fT4EopPVTWZh1mFXwepOiYkHQemHmX0ev6p8ecwcECkHs5F7eEEjsCqDH4mvlT3e+3oh2PNpbUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396054; c=relaxed/simple; bh=D9ssUjGQ6h7QeJF1IwpdOtgRJ/SdyRy/lVBrG03bkKQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I2uJoy6tMabM1O0e3MhRxspw2dkudSDlWbANvt+mieDVTTct8i/P0Xt7vyKKKTUxEJeqxf1lxdZKMvHTgYFBCFGGbkNbsM/eSMzR+WolP9UUOexTEIpOiPoYBcIDLEjqFqrAxQa8kt034n8owhsfydWm8SNagp3pn3jF79+ySBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=StN/9/9V; arc=none smtp.client-ip=209.85.219.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="StN/9/9V" Received: by mail-qv1-f49.google.com with SMTP id 6a1803df08f44-6cbc7c77c99so249436d6.3 for ; Tue, 08 Oct 2024 07:00:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396052; x=1729000852; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iOmfBB2o+BmwtYmn6lEeWqW0gfRN00aP7+NLXlUlw00=; b=StN/9/9VHUbqA+DqeHGKDKHwt6Tsn/zsnYDAfaM8fwy3k1NlJIgr/Q9LzY0ZoRNyXq +uMPnOBFTgKREJeABH/Ls/dKUkbsRLvsp5u57mnOTSd+WmANTIwUJSU9/cwt8/TkUEm9 8UBlw8NHNliC/5UQPovW+V33uUdJ73g04pTLC85V0HkT2KBIkxOEhEDFU50ataaOHFYB B/rkDpnpdrAUUCPx5rQYvweU2utBM9qTOcXJayRmh1aH+lQD2RcDsC5/b/0apCqRf8TB elOmMdTbePZRr8x8/8cxOv6EVP5RwnMzsKreKWtQcwlL6E49h/krnxfDcw847ZB2UP/m JtOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396052; x=1729000852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iOmfBB2o+BmwtYmn6lEeWqW0gfRN00aP7+NLXlUlw00=; b=Z/F5z0ppvWZanl+vrumTFb+G9LAzB+wf1sMegoNuUgoTKgRjy4JEF8wSx3/540S6Mv 0Fl4ssfGol1w+j2esDbtM9ps6TRoIeVWrRzt5+JPvNLNAsohaUhmykFLphGHHlpoba5q kscST7CN439c05ov38cqcOnG+nw3HrWyj8SgHXR8JbMl7hfc1E1ec0tMSDS4E8MHHmtH ekM1QlU2DzDF9D73c6NIQD8w09RuxOaHvinKCoX/O5wsYa27PAXJqla5ir4HOn6kAk5i ZDhkToIRGy6hAlIn9p7gyXpImFNR9TT+bfsr8FDECAhq+19QS/UkBBh2BY4+jlLhrsF8 rUqQ== X-Gm-Message-State: AOJu0YxOYncJewlKATpUF/sFb9Kr2+Ogz981TWJlV5bWJHf/sBIYPtAW nrYUyh3oCC1nC0sZp8Ko+IXnkNC8120BKqfVcr4d7TJ4nFpOZz8AMonlLNDnHTKl/nUq1LRxJNc V X-Google-Smtp-Source: AGHT+IHe1DIW8UN+mP+xqS2Oh3mXbQusW6tw3/R+BVrHed7J8ntdmkY9QJ5q2+XDvRLblwgEqMcXNQ== X-Received: by 2002:a05:6214:3b85:b0:6cb:c745:4f44 with SMTP id 6a1803df08f44-6cbc745517dmr10453496d6.26.1728396051760; Tue, 08 Oct 2024 07:00:51 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:50 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 2/3] x86/mm: if memblock size is adjusted, update the cached value Date: Tue, 8 Oct 2024 00:43:54 -0400 Message-ID: <20241008044355.4325-3-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When parsing CFMWS, we need to know the currently registered memory block size to avoid accidentally adjusting the size upward. Querying the size causes it to be cached in a static global. Update the static global if the value is subsequently updated. Print a warning that this has occurred for debugging purposes if the memory block size is changed at an unexpected time (post-init). Signed-off-by: Gregory Price --- arch/x86/mm/init_64.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 6086f99449fa..733dfa899232 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1424,6 +1424,7 @@ void mark_rodata_ro(void) /* Adjustable memory block size */ static unsigned long set_memory_block_size; +static unsigned long memory_block_size_probed; int set_memory_block_size_order(unsigned int order) { unsigned long size = 1UL << order; @@ -1432,6 +1433,19 @@ int set_memory_block_size_order(unsigned int order) return -EINVAL; set_memory_block_size = size; + + /* + * If the block size has already been probed, we need to change it. + * This can happen during ACPI/CFMWS parsing, since it needs to + * probe the system for the existing block size to avoid increasing + * the block size if lower memory happens to be misaligned + */ + if (memory_block_size_probed) { + memory_block_size_probed = size; + pr_warn("x86/mm: Memory block size changed: %ldMB\n", + size >> 20); + } + return 0; } @@ -1471,7 +1485,6 @@ static unsigned long probe_memory_block_size(void) return bz; } -static unsigned long memory_block_size_probed; unsigned long memory_block_size_bytes(void) { if (!memory_block_size_probed) From patchwork Tue Oct 8 04:43:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826543 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3A7F1DFDAB for ; Tue, 8 Oct 2024 14:00:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396056; cv=none; b=A8B7vXqvoZC3suj44biVbr9KPgHIFLIk5VezcrxwNKrVr+bcxilnQ+yAX1pZ9l1kVetdzcR67j4yRZuD1E9aiUVoV2W2Hxq7u2rcVE0Z/l83s6SzHUrNNxKgDgFXhSIJ1+2rw8HU1CN6JXdeQJ0bjlMlgN2gM9e7d1uz2oMB6vU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728396056; c=relaxed/simple; bh=6Fojn6KYi9Rnmp9EMOv/t6rJ9pIb1bMhH4aerHdoOZ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fOFhHpftm4YXNo/azG7Oi6z4nWIuU5ilhhG6/+so97SXJFE0XxqgtNrWxCmxF1rZvkUx9nAb1fxeCRDTe2rRr0w93VuL/jOM2IlgUrWkZ05szTKj8l/IG54dh8j399w1vmWHPCkBiOmOWe+27ExcqswFdAyxkI4rU+bcdhngtFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Oh0u6p98; arc=none smtp.client-ip=209.85.219.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Oh0u6p98" Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-6cb7f5f9fb7so59797026d6.1 for ; Tue, 08 Oct 2024 07:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396053; x=1729000853; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=Oh0u6p98IfMFdV7CJ8/g+chEZT7jYwu/BAhJrvfPUlT4LhtEP8Nk1lKm+3U0I5/BvG Z9pDeqRaYWPHV7bvCCpId5IBT4XPM7SnbzKO2BK+RZOK9XmhQHtz7rGEU5pqGsk++q5U OSczRhi3xWZqjA4roT86+fvUGMJRkno/6vDP96SK6xy9ASf2ZtX4qsLixgU8hnxSWY0I OOBU5DGtBfv0k5TO31H+mfMeznnFznlXi2c0rQ0LeSTgsx+lzYka8vTcrAyVCz579Tum GyNjuGbdcUJvzGtEk4HmXjMbJIuvlj+QKHxfD3nVQ+DIs9l46cB6CbD08u4imORWydvq uK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396053; x=1729000853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=jCyCm/ez2q14glQGxwefsfDNuffki4/kfZW5XMcG39FEsB9EDMCpL/8gZYQfm7HKUf KjXLvPeMkDd1vPIMXBmyGy+qM7OUo6WX1+Pn5g9BdXiozyNq8/RKY/K7lspTTxRcv6Ki dSrpknI17ZLuthQjpBbniYM5A7gyfCaH3ERuaV9So2rXRIx7wla6mgMA9B6+VfrHYIBC ltbdnFkPSUVyPbm7cw6igHc78WFbqScHmQZeygvVX9QZQHi94wZYVr0qCiyVKcByRAO7 ncToJ+TmzCjDIsbydrc5PsVd6VUIj8LSzG7qY4i5vhvI9OmuXRmM/YoiYZFAgbDceCBL r/gA== X-Gm-Message-State: AOJu0YxBaFsZQPCB0//cgx58v7w1C+vBr4qKnR4Q1rXYaMuEzfYSxJVz bRBEEYoOYNfiL0vd0Lvj7KAynFeGynTP13F4B5Z3cVkxTxbU2G4R9go3LpPutayD0yMY4xwM12Z V X-Google-Smtp-Source: AGHT+IEWQaL+bYEjt3RyVfxhNXMy9Cz/JlOtCh7gn34GRctxubICCfzpYSUr2ZJbwepGsgKWAr958Q== X-Received: by 2002:a0c:e5c4:0:b0:6cb:a705:c05e with SMTP id 6a1803df08f44-6cba705c0aamr170960556d6.6.1728396053465; Tue, 08 Oct 2024 07:00:53 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:53 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 3/3] acpi,srat: reduce memory block size if CFMWS has a smaller alignment Date: Tue, 8 Oct 2024 00:43:55 -0400 Message-ID: <20241008044355.4325-4-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The CXL Fixed Memory Window allows for memory aligned down to the size of 256MB. However, by default on x86, memory blocks increase in size as total System RAM capacity increases. On x86, this caps out at 2G when 64GB of System RAM is reached. When the CFMWS regions are not aligned to memory block size, this results in lost capacity on either side of the alignment. Parse all CFMWS to detect the largest common denomenator among all regions, and reduce the block size accordingly. This can only be done when MEMORY_HOTPLUG and SPARSEMEM configs are enabled, but the surrounding code may not necessarily require these configs, so build accordingly. Suggested-by: Dan Williams Signed-off-by: Gregory Price --- drivers/acpi/numa/srat.c | 48 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c index 44f91f2c6c5d..9367d36eba9a 100644 --- a/drivers/acpi/numa/srat.c +++ b/drivers/acpi/numa/srat.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -333,6 +334,37 @@ acpi_parse_memory_affinity(union acpi_subtable_headers *header, return 0; } +#if defined(CONFIG_MEMORY_HOTPLUG) +/* + * CXL allows CFMW to be aligned along 256MB boundaries, but large memory + * systems default to larger alignments (2GB on x86). Misalignments can + * cause some capacity to become unreachable. Calculate the largest supported + * alignment for all CFMW to maximize the amount of mappable capacity. + */ +static int __init acpi_align_cfmws(union acpi_subtable_headers *header, + void *arg, const unsigned long table_end) +{ + struct acpi_cedt_cfmws *cfmws = (struct acpi_cedt_cfmws *)header; + u64 start = cfmws->base_hpa; + u64 size = cfmws->window_size; + unsigned long *fin_bz = arg; + unsigned long bz; + + for (bz = SZ_64T; bz >= SZ_256M; bz >>= 1) { + if (IS_ALIGNED(start, bz) && IS_ALIGNED(size, bz)) + break; + } + + /* Only adjust downward, we never want to increase block size */ + if (bz < *fin_bz && bz >= SZ_256M) + *fin_bz = bz; + else if (bz < SZ_256M) + pr_err("CFMWS: [BIOS BUG] base/size alignment violates spec\n"); + + return 0; +} +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, void *arg, const unsigned long table_end) { @@ -501,6 +533,10 @@ acpi_table_parse_srat(enum acpi_srat_type id, int __init acpi_numa_init(void) { int i, fake_pxm, cnt = 0; +#if defined(CONFIG_MEMORY_HOTPLUG) + unsigned long block_sz = memory_block_size_bytes(); + unsigned long cfmw_align = block_sz; +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ if (acpi_disabled) return -EINVAL; @@ -552,6 +588,18 @@ int __init acpi_numa_init(void) } last_real_pxm = fake_pxm; fake_pxm++; + +#if defined(CONFIG_MEMORY_HOTPLUG) + /* Calculate and set largest supported memory block size alignment */ + acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_align_cfmws, + &cfmw_align); + if (cfmw_align < block_sz && cfmw_align >= SZ_256M) { + if (set_memory_block_size_order(ffs(cfmw_align)-1)) + pr_warn("CFMWS: Unable to adjust memory block size\n"); + } +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + + /* Then parse and fill the numa nodes with the described memory */ acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_parse_cfmws, &fake_pxm);